Page 356 -

346 CHAPTER 12 Automated data collection methods

Although potentially expensive and requiring some effort in configuration and man-
agement, tools in this class—led by TechSmith’s Morae software—have been widely
adopted by HCI researchers.

12.4 CUSTOM SOFTWARE
Modern computing applications are complex: word processors, spreadsheets, email
programs, and web browsers may have dozens, if not hundreds of toolbar buttons and
menu items. Which of these items are used and which are not? How would modi-
fications to the interface change usage patterns for various functions? These ques-
tions are of interest both to researchers, who might be interested in understanding
the efficacy of various strategies for grouping and rearranging controls for complex
operations, and for product developers interested in comparing the effectiveness of
proposed interface changes.
There are several approaches to collecting detailed data on the usage of com-
plex interfaces. User observations, interviews, video recordings, and other strat-
egies described elsewhere in this book can and have been used effectively for
these purposes. However, these approaches are all laborious and expensive, re-
quiring many hours spent observing users, asking questions, or coding events on
videotape.
In many cases, a more attractive alternative is to have the software collect data on
its own usage. A program designed for this sort of data collection would store every
important user action—menu choices, toolbar button selections, key presses, and
more, in a log file or database. Storage of these events in chronological order, includ-
ing a timestamp, would provide a complete history of which options were selected
and when. Analysis of this data might help developers understand which commands
are used frequently, rarely, or usually in close combination with other commands
(such as “cut” followed by “paste”).

12.4.1 INSTRUMENTED SOFTWARE
The practice of adding measurement and recording tools to software is known as in-
strumenting. Constructing instrumented software may require a fair amount of tech-
nical expertise, as code for handling user interactions must be substantially modified.
For many commercial or “closed source” software products, this level of access to
the source code may be available only to the vendor or the developer of the product.
However, macro and extension facilities in some products have been successfully
used to write instrumentation code for interface evaluation purposes. A third pos-
sibility involves open-source software. Researchers interested in studying the usage
of the interfaces of open-source projects might produce their own, instrumented ver-
sions of popular programs for use in research data collection. Detailed versions of
each of these approaches are described in the Instrumented Software for HCI Data
Collection sidebar.