Page 339 -

CHAPTER

Automated data collection 12

methods

12.1 INTRODUCTION
Data are the building blocks of research. As the recorded output of research efforts,
data are the raw materials that must be processed, analyzed, and interpreted to pro-
vide answers to research questions. Data collection is therefore a critical phase in any
research effort.
Data collection is also often one of the most challenging aspects of research.
Timing user task completion with a stopwatch, furiously writing notes describing user
interactions with software systems, coding notes from ethnographic observations, and
many other tasks are laborious, time consuming, and often—as a result—error-prone.
Fortunately, human-computer interaction (HCI) researchers can use the comput-
ers that are the subject of our research as powerful data collection tools. Software
tools can be used to collect vast amounts of user interaction data, often with little or
no direct effort on the part of the researcher administering the study. Interaction log-
ging software tracking keystrokes and mouse clicks, special-purpose instrumented
software designed to track use of specific features in tools, web site access logs, and
home-grown customized tools for tracking what users do and when can simplify data
collection, increase consistency, and decrease error.
Approaches to automated data collection can generally be placed on a spectrum
of ease of use and flexibility (Figure 12.1). Existing software tools such as web-
site access log analyzers can often be easily used or adapted for research purposes,
but capabilities might be limited. System observation and logging software may be
somewhat more powerful, but installation and configuration issues can be challeng-
ing. Custom-built or modified software can be crafted to meet the precise research
needs, but the development effort can be substantial.
All of these automated methods for computerized data collection are capable of
producing voluminous data sets. This can pose a substantial problem for researchers:
while generating data is easy, deciding which data to collect and how best to analyze
that data can be challenging. Although many projects involving automated data col-
lection (including data collected from human subjects—see Chapter 13) follow a
familiar arc of planning, data collection, cleaning, analysis, and iteration to refine
methods and techniques, details vary between projects due to technological differ-
ences in data acquisition methods and analytic differences associated with varying
research questions. Some of these issues will be discussed here and in Chapter 13,

Research Methods in Human-Computer Interaction. http://dx.doi.org/10.1016/B978-0-12-805390-4.00012-1 329
© 2017 Elsevier Inc. All rights reserved.