Page 512 -

16.3 Methodological considerations 505

in the recent proceedings of the ASSETS conference (well accepted as a high-quality
conference on this topic), many of the research studies in which users with disabili-
ties had to be physically present to take part in the research had 15 or fewer individu-
als taking part in the research.
Because of the small number of participants with disabilities who often take part,
this means that if a classic experimental design is used, there will often be no more
than one control group and one treatment group, as the number of participants does
not allow for statistical tests for multiple treatment groups, so repeated measures de-
signs tend to be used if experimental design is used at all (see Chapters 2–4 for more
information on experimental design). Because of the sample size issue, experimental
design is used less often in research involving people with disabilities, though it can
be used. One common approach is to have a group of participants with a specific
disability, and a group of participants without a disability as the control group (Sears
and Hanson, 2012). Care needs to be taken in justification and interpretation of re-
sults in these experimental designs; for instance, if demographic data does not iden-
tify significant differences between the groups (e.g., in age or education), that does
not mean that the groups can be considered equivalent (Sears and Hanson, 2012).
Experimental design methodologies involving people with disabilities can be done.
However, due to the sample size issue, research involving people with disabilities,
is often exploratory (rather than strict experimental design), a hybrid of quantitative
and qualitative research, or primarily qualitative. There are not many users involved,
but the data collection is certainly deep. For example, Mcintyre and Hanson (2014)
describe how they used a combination of in-person interviews involving researchers
and Blind individuals, before and after a navigation task within a building, while a
digital recorder, not researchers, recorded data as 10 Blind and low vision partici-
pants were performing the navigation task within the building.

16.3.2 DISTRIBUTED RESEARCH
If not enough participants with disabilities, who meet the inclusion criteria, are avail-
able in a local area, another approach is to do distributed research, where the users
do the research in their own home or office, without researchers present, and data is
collected via time diaries, surveys, keystroke logging, or another method. Remote
data collection generally allows for higher numbers of users (100 users or more)
to take part in the research, because you can draw from potential participants over
a much wider geographical area. In addition, a number of the challenges discussed
later in the chapter (such as scheduling and transportation) may not be present for
distributed research. However, there are drawbacks to remote data collection (see
Chapters 10 and 14). It lowers the control that the researchers have over the study,
and generally, the amount and richness of data collection will not be comparable to
a research study done in-person (Petrie et al., 2006). Furthermore, it may not be pos-
sible to ascertain much detail about the specific technical environment utilized by the
participant, when the data is collected remotely (Petrie et al., 2006). If software needs
to be installed remotely, in a technology environment that researchers do not have