Page 399 -

13.5 Data collection, analysis, and interpretation 389

Data collection challenges often lead researchers to choose to conduct physi-
ological studies in the comfort and convenience of the lab. Working in surroundings
that are well-lit, well-organized and well-stocked with all needed supplies is a good
strategy for minimizing the uncertainty associated with these data collection tech-
niques. However, lab studies have their limits. The idealized settings may not reflect
“real-world” situations where technologies might be used, leading to results that may
be somewhat artificial. This disconnect between the environment of the study and
the environment of use is described as reducing the ecological validity of the study.
For studies addressing how interfaces are used in practice, lab settings might simply
be unable to capture all of the richness of real usage environments. See the “LAB-
IN-A-BOX” sidebar for a description of a suite of tools developed to address these
challenges.

13.5.2 DATA ANALYSIS
Like other naturally occurring signals, eye-tracking data, motion detection systems,
and physiological measurements are all very noisy, containing artifacts and vari-
ability that can make interpretation difficult. EMG signals, for example, suffer from
significant amounts of distortion and random noise from other muscles (Raez et al.,
2006). Tonic activity levels measure physiological responses in the absence of spe-
cific responses. These “baseline” measurements can differ significantly from one
individual to the next and sometimes within individuals, due to factors such as head-
aches. Furthermore, the magnitude of response to a specific condition may be influ-
enced by the tonic levels of a given signal: the response to any given stimulus might
be lesser for a heart that is already beating quickly. Habituation is another concern:
the magnitude of response to a stimulus decreases after repeated presentation (Stern
et al., 2001). This can present a challenge for both experimental design and data
interpretation. Eye-tracking and motion detection systems face similar challenges
in distinguishing between intentional actions including saccades, pursuits, and fixa-
tions and seemingly random noise (microsaccades) (Duchowski, 2007). Appropriate
use of software tools accompanying eye-tracking hardware can help address these
difficulties.
Although a wide variety of methods has been proposed for extracting the
signal from the surrounding noise (Raez et al., 2006), their use might require
additional expertise: without a basis in a solid understanding, the application of
signal-processing tools to noisy data streams can become a case of “garbage-in,
garbage-out.”
Once you have extracted the signal in your physiological data from the noise,
your next challenge is to determine the granularity of the data that you will analyze.
Some experiments call for relatively coarse data: if you are interested in comparing
average responses for various testing conditions, you can just process data as it ar-
rives, without worrying about specific correspondences between physiological data
points and events in the computer interface. In cases where you want more detail, you
might find that capturing all of the data available from your sensors is overwhelm-
ing. Some form of downsampling (capturing one out of every n data points instead