Page 282 -

10.5 The process of user-based testing 271

interact with those interfaces, only how well these interfaces comply with some
guidelines. Automated tools can also help with determining a high-level view of
thousands of web pages, for example, within an organization, to determine how
many are meeting certainly basic usability requirements. For instance, Lazar et al.
(2017) utilized automated accessibility testing tools to examine which US federal
agencies had accessibility features present on a large portion of their web sites (not
whether a specific web page was fully compliant or not). Automated tools are good
for tasks such as that, determining the presence of features and getting a high-level
overview. A large number of tools exist for automated accessibility testing, including
standalone applications such as Deque WorldSpace, Cryptzone ComplianceSherriff,
and SSB Accessibility Management Platform, as well as free web-based tools such
as A-Checker, WAVE, and Functional Accessibility Evaluator, all of which check
interfaces for compliance with the WCAG 2.0 guidelines. A classic article about the
concepts behind automated usability testing can be found in the ACM Computing
Surveys (Ivory and Hearst, 2001).

10.5 THE PROCESS OF USER-BASED TESTING
User-based testing is what most people mean when they refer to usability testing.
Mostly, it means a group of representative users attempting a set of representative
tasks. This can take place very early in development, during development, or very late
in development. It is better to start doing user-based testing earlier rather than later,
when the results can influence the design more and when costs to make changes are
much lower. Ideally, user-based testing would take place during all stages of develop-
ment, but that is not always possible. Why do we do usability testing? As much as
designers try to build interfaces that match the needs of the users, the designers are not
users and even the users themselves sometimes cannot clearly identify their interface
needs. So interface prototypes, at various stages, need to be tested by users. Note that
users are testing interfaces, but users are not being tested. This is an important distinc-
tion. Furthermore, some authors even go so far as to say that the developers who create
an interface design should not be the ones who moderate a usability test (Rubin and
Chisnell, 2008). If you create an interface, you are likely to be supportive of that inter-
face, feel that you have time invested in it, and may not be as open to user suggestions.
From a strict experimental point of view, the interface developer shouldn’t moderate a
usability test or interact with the participants (although the developer can observe the
testing to learn what aspects of their design aren’t working well). However, since per-
fect design isn’t the goal of usability testing, there are situations where the interface
developer serves double duty and moderates the usability test.

10.5.1 FORMATIVE AND SUMMATIVE USABILITY TESTING
Usability testing that takes place early in development tends to be exploratory and
to test early design concepts. Sometimes, this is known as formative testing and