Page 40 -

CHAPTER
2
Experimental research

A variety of laboratory and nonlaboratory research methods are available for human-
computer interaction (HCI) researchers or practitioners when studying interfaces or
applications. The most frequently used include observations, field studies, surveys,
usability studies, interviews, focus groups, and controlled experiments (Shneiderman
et al., 2017). In order to study how users enter information into their mobile phones,
researchers may choose to observe mobile phone users in a natural setting, such as
individuals who are using a cell phone in a company lobby, an airport, or a park. They
may develop a survey that addresses questions that they would like to have answered
and ask mobile phone users to respond to the survey. They may interview a number
of mobile phone users to find out how they enter information into their phones. They
may also choose to recruit a number of participants and run a usability test in a lab-
based environment. Another option is to specify several conditions and run a strictly
controlled lab-based experiment.
We can continue to add more options to the researchers' list: focus groups, field
studies, and so on. Each of these options has its own strengths and weaknesses.
Unobtrusively observing users in natural settings may allow the researcher to iden-
tify the patterns that are most representative of the use of the mobile phone in natural
settings, but observation studies can be extremely time consuming. The researchers
may wait for hours only to find that none of the individuals being observed has used
the functions in which they are most interested. The survey approach may allow the
researchers to reach a large number of users, say over a hundred, in a short period of
time, but the participants may misunderstand the questions, the data collected may
not represent depth in understanding, and the participant sample can be highly biased.
Interviews allow the researchers to clarify questions and dig deeper with follow-up
questions when a participant provides interesting feedback. However, interviews cost
significantly more time and money than surveys. Usability tests provide a quick and
comparatively low-cost method of identifying key usability problems in an interface
or application, but they cannot guarantee that all critical design problems can be
identified.
Choosing which method to use is a highly context-dependent issue related to a
variety of factors including the primary purpose of the study, time constraints, fund-
ing, the participant pool, and the researchers' experience. We discuss in more detail in
Chapter 3 on how to select the best research method. This chapter examines experi-
mental research in general and focuses on the very basics of conducting experimental
studies. We discuss how to develop research hypotheses and how to test the validity

Research Methods in Human-Computer Interaction. http://dx.doi.org/10.1016/B978-0-12-805390-4.00002-9 25
© 2017 Elsevier Inc. All rights reserved.