Page 172 - Socially Intelligent Agents Creating Relationships with Computers and Robots

Designing Sociable Machines 155

quality of the interaction improves. Furthermore, many of these social cues
will eventually be offered in the context of teaching the robot. To be able to
take advantage of this scaffolding, the robot must be able to correctly interpret
and react to these social cues. There are two cases where the robot can read
the human’s social cues.
The first is the ability to recognize praise, prohibition, soothing, and atten-
tional bids from robot-directed speech [9, 2]. This could serve as an important
teaching cue for reinforcing and shaping the robot’s behavior. Several inter-
esting interactions have been witnessed between Kismet and human subjects
when Kismet recognizes and expressively responds to their tone of voice. They
use Kismet’s facial expression and body posture to determine when Kismet
“understood” their intent. The video of these interactions suggests evidence of
affective feedback where the subject might issue an intent (say, an attentional
bid), the robot responds expressively (perking its ears, leaning forward, and
rounding its lips), and then the subject immediately responds in kind (perhaps
by saying, “Oh!” or, “Ah!”). Several subjects appeared to empathize with
the robot after issuing a prohibition—often reporting feeling guilty or bad for
scolding the robot and making it “sad.”
The second is the ability of humans to direct Kismet’s attention using natural
cues [1]. This could play an important role in socially situated learning by
giving the caregiver a way of showing Kismet what is important for the task,
and for establishing a shared reference. We have found that it is important for
the robot’s attention system to be tuned to the attention system of humans. It is
important that both human and robot find the same types of stimuli salient in
similar conditions. Kismet has a set of perceptual biases based on the human
pre-attentive visual system. In this way, both robot and humans are more likely
to find the same sorts of things interesting or attention-grabbing. As a result,
people can very naturally and quickly direct the robot’s attention by bringing
the target close and in front of the robot’s face, shaking the object of interest, or
moving it slowly across the centerline of the robot’s face. Each of these cues
increases the saliency of a stimulus by making it appear larger in the visual
field, or by supplementing the color or skin-tone cue with motion. Kismet’s
attention system coupled with gaze direction provides people with a powerful
and intuitive social cue for when they have succeeded in steering the robot’s
interest.

3. Summary

In this chapter, we have outlined a set of four core design issues that have
guided our work in building Kismet. When engaging another socially, humans
bring a complex set of well-established social machinery to the interaction.
Our aim is not a matter of re-engineering the human side of the equation to suit