Page 401 -

13.5 Data collection, analysis, and interpretation 391

machine learning techniques to automatically identify actions and reactions with
high degrees of confidence. Such classification methods may require manual identi-
fication of desired outputs, to be used as training sets for supervised learning.
A final interpretive challenge lies in the difficulty of understanding physiological
signals. Even if you have a clear difference in some measure that seems to come in
response to a specific event, interpreting that measure may prove challenging. You
may be tempted to classify a response as specific emotional state—happiness, sad-
ness, disgust, fear, or other examples—but data for many measures is inconclusive
(Cacioppo et al., 2000). Although triangulation through the use of multiple signals
can be a promising approach, there is no guarantee that any combination of responses
will be sufficient. Mixed or incomplete measures are a very real possibility: some
stimuli may lead to a response in one measure, with no change in another (Stern
et al., 2001).
Physiological data presents tantalizing possibilities for researchers. Although
the challenges of collecting and interpreting data from these sources are consider-
able, the possibility of identifying fine-grained, real-time responses to interfaces
is often hard to resist. Before committing your valuable human and financial re-
sources to such an effort, you may want to ask yourself if there is an easier way to
observe the phenomena of interest. You may legitimately decide that your study
of user frustration requires fine-grained detail about specific events, making post-
test questionnaires insufficiently detailed. Before concluding that physiological
data measures are required to identify incidences of frustration in real-time, you
should consider using simpler methods such as videotapes, observations, think-
aloud protocols, or time diaries. You may find that simpler methods get the job
done with much less headache and expense. For a more detailed discussion of the
use of eye tracking and physiological data into HCI design and evaluation, see
Bergstrom and Schall's practical book Eye Tracking in User Experience Design
(Bergstrom and Schall, 2014).

LAB-IN-A-BOX

Studies involving human data collection can be particularly challenging when
they involve either realistic locations or collection and correlation of multiple
data streams. Combining these two challenges can make matters even more
interesting, leading often to innovative techniques. Nadir Weibel and colleagues
struggled with these questions as they developed a multimodal set of data
collection and analysis techniques to examine a complex and multifaceted set
of HCIs: the use of electronic medical records (EMRs) by physicians during
outpatient medical visits.
Although the use of electronic medical records has expanded substantially
in recent years, the impact of this change on medical care is far from well
understood. Although researchers have known for quite some time that

(Continued)