Page 90 -

76 CHAPTER 4 Statistical analysis

important and useful for data analysis since many attributes from various fields of
study are distributed normally, such as the heights of a population, student grades,
and various performance measures.
Testing a data set to determine whether it is normally distributed is a necessary
step when selecting the type of significance tests to conduct. Parametric tests assume
that the data set is normally distributed or approximately normally distributed. If you
find that the data collected is not normally distributed, you may need to consider
transforming the data so that they are normally distributed or you may adopt nonpa-
rametric tests for the analysis.
For detailed calculation of each of the measures, please refer to statistical text-
books, such as Hinkle et al. (2002), Newton and Rudestam (1999), Rosenthal and
Rosnow (2008), and Albert and Tullis (2013). Microsoft Excel offers built-in func-
tions that allow you to conveniently calculate or count various descriptive measures.

4.3 COMPARING MEANS
In user studies involving multiple conditions or groups, the ultimate objective of
the researcher is to find out whether there is any difference between the conditions
or groups. Suppose you are evaluating the effectiveness of two search engines; you
may adopt a between-group design, in which case you will recruit two groups of
participants and have each group use one of the two search engines to complete a
number of search tasks. If you choose a within-group design, you will recruit one
group of participants and have each participant complete a series of tasks using both
search engines. In either case, you want to compare the performance measures of the
two groups or conditions to find out whether there is any difference between the two
search engines.
Many studies involve three or more conditions that need to be compared. Due to
variances in the data, you should not directly compare the means of the multiple con-
ditions and claim that a difference exists as long as the means are different. Instead,
you have to use statistical significance tests to evaluate the variances that can be
explained by the independent variables and the variances that cannot be explained by
them. The significance test will suggest the probability of the observed difference oc-
curring by chance. If the probability that the difference occurs by chance is fairly low
(e.g., less than 5%), we can claim with high confidence that the observed difference
is due to the difference in the controlled independent variables.
Various significance tests are available to compare the means of multiple
groups. Commonly used tests include t tests and the ANOVA. A t test is a simplified
ANOVA involving only two groups or conditions. Two commonly used t tests are the
independent-samples t test and the paired-samples t test. When a study involves more
than two conditions, an ANOVA test has to be used. Various ANOVA methods are
available to fit the needs of different experimental designs. Commonly used ANOVA
tests include one-way ANOVA, factorial ANOVA, repeated measures ANOVA, and
ANOVA for split-plot design.