Page 30 -

1.8 Understanding one research project in the context of related research 15

background research, more structure, is required for certain types of research methods.
Shneiderman describes this as a three-step process: observation, intervention, and con-
trolled experimentation. The understanding through the exploratory research, can be
utilized to build prototypes or hypotheses for experimental design (Shneiderman, 2016).
Another aspect of the research life cycle is determining when controlled,
in- laboratory studies should occur, versus studies “in the wild” (also known as field
studies or in-situ studies). There is a great discussion in the research community about
when each approach is situationally appropriate. For instance, some authors argue that
field studies are most appropriate for mobile device research, since mobile devices are
utilized in the field, with weather, noise, motion, and competing cognitive demands
playing an important role in usage (Kjeldskov and Skov, 2014). Controlled environ-
ments and precise measurement may simply not be realistic for the usage of certain
types of technologies, such as mobile devices. Another argument for the increased use
of field studies, is that, as researchers come to understand more about what specific
aspects of design lead to increased usability, then the next step is to understand how
those technologies fit into the complex work, leisure, and family lives of individuals
(Kjeldskov and Skov, 2014). Field studies may present interesting challenges related
to informed consent, since the period of data collection, and who participates, in a
controlled environment, may be easy to ascertain. But for example, data collection in
a public space (in the wild), such as marathon or a rock concert, may pose questions
about the inclusion of data from people who are not aware of the data collection and did
not consent to participate (Anstead et al., 2014). One can imagine multiple approaches
for which research methods to utilize and in what order (as described in previous para-
graphs). So perhaps researchers might first do exploratory research in the wild, before
moving to more controlled laboratory settings. Or perhaps researchers might first do
controlled laboratory experiments, and then move their research into the wild and do
field studies. There is not one answer that is right or wrong.
From personal experience, the authors can verify that both approaches are useful,
and the combination of controlled studies and field studies, often gives you interest-
ing findings that make you rethink your approaches. For instance, from the authors
of this textbook, there were three research studies of a web-based security prototype,
in a combination of controlled settings (university lab, workplace, home, and always
on a consistent laptop), from three different groups of users, where the average task
performance rate on a specific prototype was always over 90%. When that same
web-based security prototype was placed on the web, with a much more diverse set
of users utilizing the prototype, generally with a lower level of technical experience,
and with technical environment being another factor (older browsers, slow download
speeds, etc.), the average task performance rate was under 50%, a significant drop. No
research method is ever perfect, and trying out different research methods to investigate
similar phenomenon, helps you to more fully understand your area of study. It is impor-
tant to note that an individual's viewpoint on controlled laboratory experiments versus
field studies, may also be influenced by their individual disciplinary background, so,
for instance, those with engineering backgrounds may lean more naturally towards
laboratory experiments compared to those with an anthropology background.