Page 29 -

14 CHAPTER 1 Introduction to HCI research

communities, the goal is generally for multiple teams to examine the same research
question from multiple angles over time. Research results should be reported, with
enough detail so that other teams can attempt to replicate the findings and expand upon
them. Replication is considered an important part of validating research findings, even
though it is rare in HCI and often gets very little attention (Hornbaek et al., 2014) (and
many other fields of research have similar complaints). All of these efforts, if they
come up with the same general findings over time, give evidence for the scientific truth
of the findings. This is often known as “triangulation.” One data collection effort, yield-
ing one paper, is interesting in itself but does not prove anything. If you have 15 teams
of researchers, looking at similar research questions, over a period of 10 years, using
multiple research methods, and they all come to the same general conclusion about a
phenomenon, then there is some scientific proof for the phenomenon. The proof is even
stronger when multiple research methods have been used in data collection. If all of the
research teams replicate the exact same research methods over 10 years, then there is
the remote possibility that the methods themselves are flawed. However, the weight of
evidence is strengthened when multiple research methods are used.
Researchers often speak of a “research life cycle,” describing the specific steps in a
research project. Depending on who you ask, the steps can differ: for instance, (1) design-
ing research, (2) running data collection, and (3) reporting research (Hornbaek, 2011).
But there is another type of life cycle to consider: when you are entering a new area
or subspecialty of research, which methods are likely to be utilized first? On the other
hand, which methods require first having additional research in place? For instance,
two of the three coauthors of this book have been involved with performing research
to understand how people with Down syndrome (both children and adults) utilize tech-
nology and what their interface needs are. When we decided to do this research, there
was no existing HCI research on people with Down syndrome. There was no base of
literature to draw from. So we first started with an exploratory survey to understand
how children and young adults utilize technology. Then we did a series of observations
of adults with Down syndrome who were expert users about what their skills were,
and how they gained those skills. Then we utilized a usability testing methodology
to understand how adults with Down syndrome utilize social networking and touch
screens. Once we had a base of understanding about the research topic with those three
studies, only then did we do an experimental design (to understand the effectiveness
of different authentication methods for people with Down syndrome). It would have
been too premature to start with an experimental design method first, when so little
was known about the population of users and how they interact with technology. The
controls necessary for an experimental design, would have not yet been understood, so
there would have been lots of phenomenon that were unknown and not controlled for.
Often, when a research topic is new, it is important to start with a research method that
can utilized in a more exploratory way—such as surveys, interviews, focus groups,
and ethnography. Then, with a basis of understanding from a few exploratory stud-
ies, research studies utilizing more structured research methods—such as experimental
design, automated data collection, and time diaries, could be performed. That's not to
say that such an order must occur—but such an order often does occur, because more