Page 85 -

CHAPTER
4
Statistical analysis

In Chapter 2, we discussed why we need to run statistical analysis on data collected
through various methods. Appropriate selection of statistical analysis methods and
accurate interpretation of the test results are essential for user studies. After weeks,
months, or even years of arduous preparation and data collection, you finally have a
heavy set of data on hand and may feel the need to lie back and enjoy a hard-earned
break. Well, it is a little too early to relax and celebrate at this point. With many
studies, the data analysis stage is equally or even more labor intensive than the data
collection stage. Many critical decisions need to be made when analyzing the data,
such as the type of statistical method to be used, the confidence threshold, as well
as the interpretation of the significance test results. Incorrect selection of statistical
methods or inappropriate interpretation of the results can lead to erroneous conclu-
sions that let high-quality data go to waste.
This chapter discusses general data analysis procedures and commonly used sta-
tistical methods, including independent-samples t test, paired-samples t test, one-way
analysis of variance (ANOVA), factorial ANOVA, repeated measures ANOVA, cor-
1
relation, regression, chi-squared test, and four other nonparametric tests. The focus of
this chapter is not on the mathematical computation behind each method or how to use
statistical software to conduct each analysis. Instead, we focus on the contexts of use
and the assumptions of each method. We also discuss how to appropriately interpret the
results of each significance test. Through this chapter, we hope that you will be able to
choose appropriate statistical methods for data analysis, run the corresponding tests us-
ing statistical software, and accurately interpret the analysis results for your own stud-
ies. You will also learn how to assess the validity of the findings reported in academic
articles based on the experimental design and the statistical analysis procedure.

4.1 PREPARING DATA FOR STATISTICAL ANALYSIS

In most cases, the original data collected from lab-based experiments, usability tests,
field studies, surveys, and various other channels need to be carefully processed be-
fore any statistical analysis can be conducted. There are several reasons for the need
for preprocessing. First, the original data collected, especially if they are entered

1 Tests to be used when the assumptions of the parametric tests are not met. More details will be dis-
cussed in Sections 4.6 and 4.8.
Research Methods in Human-Computer Interaction. http://dx.doi.org/10.1016/B978-0-12-805390-4.00004-2 71
© 2017 Elsevier Inc. All rights reserved.