Page 130 -

116 CHAPTER 5 Surveys

5.5.3 RANDOM SAMPLING OF USAGE, NOT USERS
Another approach to sampling is the random sampling of usage, not users (Lazar and
Preece, 2001). For instance, it may be that every 10th time a web page is loaded, the user
is asked to fill out a survey. Often, this survey appears in a pop-up window. This sam-
pling technique is also known as intercept sampling (Sue and Ritter, 2007). While this
gets an accurate picture of usage, a subset of users (those who use the web page often) is
over-represented and those who do not view the web page often are under-represented.

5.5.4 SELF-SELECTED SURVEYS
In a “self-selected” survey, there is a link on a web page every time that it is loaded
and everyone visiting the website is invited to fill out the survey. So, it is less about a
certain group of people being recruited to participate and more about inviting every-
one to participate. (Yes, it can be a bit fuzzy sometimes in nonprobabilistic surveys
as to whether responses are invited or self-selected.)
If a self-selected survey is used, then both the number of survey responses and
the demographic data on respondents become increasingly important in establishing
the validity of the survey data. One of the earliest web-based survey studies came
from the Georgia Institute of Technology. The entire population of web users was in-
vited to participate. Banner ads about the survey, inviting people to participate, were
placed on search engines, news sites, general advertising networks, mailing lists and
newsgroups, and also in the popular press. Everyone was invited to participate in the
surveys, which took place semiannually from 1994 to 1998. In the final survey, 5022
people responded. See Pitkow and Kehoe (1996) for a good summary of the studies
and http://www.cc.gatech.edu/gvu/user_surveys/ for detailed data.
There may be reasons why a population could theoretically be well-defined and
probabilistic sampling be used, but it is not logistically realistic. See the sidebar
(“Probabilistic Sampling Probably Not Feasible”) for an example of a situation
where self-selected surveys are the only feasible approach for a population that theo-
retically (but not realistically) could be sampled using probabilistic methods.

PROBABILISTIC SAMPLING PROBABLY NOT FEASIBLE
A well-known, ongoing survey in the accessibility community is the “Screen
Reader Survey” run by WebAIM at Utah State University, which has the
goal of learning more about the preferences of screen reader users. Screen
readers are software applications, such as JAWS, VoiceOver, Window-Eyes,
and NVDA, that allow people who are Blind or low vision to listen to the
content on the screen, from web pages, applications, and operating systems
(and they are not only installed on desktop and laptop computers, but also on
tablet computers and smartphones). The first Screen Reader Survey was run in
Dec. 2008 and Jan. 2009, with 1121 responses to the survey. The most recent
(the 6th) Screen Reader Survey was run in Jul. 2015, with 2515 responses.