Page 153 - Socially Intelligent Agents Creating Relationships with Computers and Robots

136 Socially Intelligent Agents

emotion, the word for that emotion, and a picture of the doll’s (emotional) face.
Also, an optional online animated guide can be included on the screen; it can
be configured to provide an audible prompt (discriminative stimuli) for child
interaction, or verbal reinforcement (putative response) for child affirmation.
The practitioner can configure many kinds of cues for ASQ to use in aid-
ing the child. The dolls can cue the child with one of the following three
choices: affect sound, hatband and lights, or internal vibration. The system
can be cued to audibly play one of three sequences to prompt the child to se-
lect a doll to match the emotion in the video clip: for instance, when a happy
clip plays, the system can say, "MATCH HAPPY" or "PUT WITH SAME",
or "TOUCH HAPPY." Likewise, reinforcements for incorrect doll selections
have three choices, such as "THAT’S SAD, MATCH HAPPY," etc. Seven dif-
ferent cue set-ups are configurable for one session with the timing, sequence,
and repeat rate tailored for each. Additionally, the practitioner may opt to
have the system play an entertaining reinforcement video clip, such as a Tig-
ger song. Our objective was to offer as much flexibility to the practitioner as
possible for customizing the screen interface for a particular session or specific
child. This is especially important because autistic children often have unique
idiosyncratic behaviors.
The child interface consists of one or more elements set up by the practi-
tioner as just discussed. Figure 16.1 shows the screen seen by the child, set up
here to show the video clip in the middle, the emotion icon at top, the dwarf
face at left, the label of the emotion at bottom, and the guide at right. When
selected, these images always appear in the same spot.
The child interacts with the system through a plush toy interface. Four inter-
active dwarves provide a tangible interface to the system, so that the child does
not have to use a keyboard or mouse. Images of the dwarves, representing an-
gry, happy, surprise, and sad, are pictured in figure 16.2, just as they appear on
the screen when selected for display. The dolls serve as engaging input devices
to the system: they are fun to hold and add a playful flavor to the interaction.

Figure 16.2. Pictures of the Dwarves.

The system design has two modes of interaction – an applied behavior mode
and a story-based mode. The first mode displays short clips, one at a time, from
various child program sources and the second mode displays an entire movie
with the story segmented by the emotions. When the video freezes, the interac-