Page 199 - Designing Sociable Robots

breazeal-79017 book March 18, 2002 14:11

180 Chapter 10

10.5 Evaluation of Expressive Behavior

The line drawing study did not ask the subjects what they thought the robot was expressing.
Clearly, however, this is an important question for my purposes. To explore this issue, a
separate questionnaire was devised. Given the wide variation in language that people use
to describe expressions and the small number of subjects, a forced choice paradigm was
adopted.
Seventeensubjectsfilledoutthequestionnaire.Mostofthesubjectswerechildren12years
of age (note that Kolb et al. [1992] found that the ability to recognize expressions continues
to develop, reaching adult level competence at approximately 14 years of age). There were
six girls, six boys, three adult men, and two adult women. Again, none of the adults had
seen the robot before. Some of the children reported minimal familiarity through reading
a children’s magazine article. There were seven pages in the questionnaire. Each page had
a large color image of Kismet displaying one of seven expressions (anger, disgust, fear,
happiness, sorrow, surprise, and a stern expression). The subjects could choose the best
match from ten possible labels (accepting, anger, bored, disgust, fear, joy, interest, sorrow,
stern, surprise). In a follow-up question, they could circle any other labels that they thought
could also apply. With respect to their best-choice answer, they were asked to specify on
a ten-point scale how confident they were of their answer, and how intense they found the
expression. The complied results are shown in table 10.5. The subjects’ responses were
significantly above random choice (10 percent), ranging from 47 percent to 83 percent.
Some of the misclassifications are initially confusing, but made understandable in light
of the aforementioned study. Given that Kismet’s surprise expression seems to convey
positive valence, it is not surprising that some subjects matched it to joy. The knitting of
the brow in Kismet’s stern expression is most likely responsible for the associations with
negative emotions such as anger and sorrow. Often, negatively valenced expressions were

Table 10.5
This table summarizes the results of the color-image-based evaluation. The questionnaire was forced choice where
the subject chose the emotive word that best matched the picture.
accepting anger bored disgust fear joy interest sorrow stern surprise % correct
anger 5.9 76.5 0 0 5.9 11.7 0 0 0 0 76.5
disgust 0 17.6 0 70.6 5.9 0 0 0 5.9 0 70.6
fear 5.9 5.9 0 0 47.1 17.6 5.9 0 0 17.6 47.1
joy 11.7 0 5.9 0 0 82.4 0 0 0 0 82.4
sorrow 0 5.9 0 0 11.7 0 0 83.4 0 0 83.4
stern 7.7 15.4 0 7.7 0 0 0 15.4 53.8 0 53.8
surprise 0 0 0 0 0 17.6 0 0 0 82.4 82.4
Forced-Choice Percentage (random = 10%)