Page 292 -

10.5 The process of user-based testing 281

Table 10.4 Benefits and Drawbacks of Remote Usability Testing
Benefit Drawback
Easy access to a greater number of Difficult or impossible to pick up nonverbal
participants and interpersonal cues
Participants have more flexibility in Hard (or impossible) to provide
participating in a usability test on their own instructions when things “go wrong”
schedule, and researchers can run multiple Researchers can’t ask any probing
usability tests at the same time questions based on what occurs
Easy collection and analysis of clickstream Researchers often miss the context of
data what was happening
Works better for summative testing, when
you are collecting quantitative metrics

USER NIGHTS AT YAHOO! (WRITTEN BY GARY MOULTON)

Yahoo is unique in combining its UX Researchers and Accessibility specialists
to form an organization called User Experience Research and Accessibility
(UXRA). UXRA partners with product groups throughout the development
life cycle to gather, analyze, and present observations from user research
when teams are gathering requirements, have a fully developed idea, and are
working on a new mobile app or Web property. Yahoo is unique in combining
Accessibility specialists with traditional User Experience Researchers. They
work closely together with individuals that identify themselves as having
a disability to observe and quantify user interactions with, and validate the
compatibility of, mobile apps and Web properties with popular assistive
technologies (e.g. screen readers, alternate input devices, etc.).
Yahoo’s UXRA uses a wide variety of qualitative and quantitative research
tools and methodologies for providing research throughout the development
lifecycle. One of the unique methods is called “User Night” where up to 100
external users are paired individually with members of a particular Yahoo
product team, who are called “Yahoos.” Yahoos are briefed in advance and
provided a script and coaching to run their own study with their participant.
For up to an hour, they have conversations about the use of their product
and observe real-world use on the participants’ own devices (phones and/
or laptops). After the event, team members share key findings in large group
settings, and findings from these sessions are aggregated and fed back to the
entire team. This process enables rapid, larger-scale feedback than is possible to
be obtained in a single-day, five-user, traditional usability study. These events
create unique empathy among product team members for real users, their issues
as well as joys, in using the product that they spend each day building.
(Continued)