Page 368 -

358 CHAPTER 12 Automated data collection methods

12.7 AUTOMATED INTERFACE EVALUATION

If using computers to collect data for HCI research is good, why not go further?
Perhaps we can build software that automatically tests and evaluates user interfaces,
generating data on usability issues or potential task performance times.
Automated inspection methods involve the analysis of aspects of user interfaces
including layout, content, and language, in order to determine how well they con-
form to design guidelines (Ivory and Hearst, 2001). Generally, these tools provide
reports indicating the extent to which an interface complies (or fails to comply) with
guidelines. These reports can help designers understand where users might run into
problems, and how an interface might be improved. The evaluations provided by
these tools are generally based on empirical evidence, accepted design practices, and
other accumulated experience. Automated inspection tools have been widely used
in assessing the accessibility of websites. These tools examine web pages in search
of images without explanatory text (<alt> tags), embedded scripts that might not be
interpretable by screen readers, lack of navigation support, and other problems that
may cause difficulties for users with disabilities. Dozens of web accessibility evalu-
ation tools—ranging from free websites to expensive commercial software—have
been developed. See Chapter 10 for more information.
Although these tools may provide some useful advice, the utility of any particular
tool may be limited by the validity and scope of the underlying guidelines: analyses
that are based on broad, well-supported guidelines are likely to be more appropriate
than those that examine a narrower range of concerns. The use of multiple inspection
methods to test interfaces from varying perspectives might be helpful. Ideally, these
tools should be seen as companions to—not replacements for—traditional design
reviews and user testing.
A variety of approaches to automated testing have been explored. Systems that
focus on the use of modeling or simulation to predict task performance times and
other quantitative and qualitative characteristics of interface usage are appealing,
but they may be difficult to construct and limited in utility (Ivory and Hearst, 2001).
Other efforts the use of modeling techniques that carry through the design and de-
velopment phase into support for automated testing (Humayoun et al., 2012; Wollner
et al., 2015).

12.8 CHALLENGES OF COMPUTERIZED DATA COLLECTION
Automated software for HCI data collection has many advantages. The use of
software to record user interaction events along with timestamps can ease data
collection for structured experiments, simplifying work that previously would
have been done with a stopwatch and paper records. Logs from web servers and
other activity tracking tools document interaction events in unstructured, “natu-
ral” activities with far less difficulty than earlier techniques of observing or vid-
eotaping users.