Page 435 -

426 CHAPTER 14 Online and ubiquitous HCI research

and colors of numerals found on building, signs, and elsewhere in images, even when
a human could read those numbers very easily. A human computation task might ask
multiple participants to interpret a large set of images, thus providing a large collec-
tion of labeled images. Resulting labels might be used to train improved machine
learning for classifying similar images, or to develop a search tool for identifying im-
ages matching specified descriptions. These tasks that require human—as opposed to
computer—cognition are often referred to as Human Intelligence Tasks. When such
tasks are explicitly organized with the goal of efficiently finding an accurate solution
for a computational problem, the resulting system might be called a human computa-
tion system (Law and Ahn, 2011). See the “CAPTCHA and reCAPTCHA” sidebars
for the story of the most familiar human computation tasks.

CAPTCHA AND reCAPTCHA

CAPTCHA—the Completely Automated Public Turing test to tell Computers
and Humans Apart—is perhaps the most familiar example of human
computation. The term CAPTCHA was developed by Luis von Ahn and
colleagues, who proposed the use of a problem that is hard for computers but
easy for humans as a web site security measure, suitable for distinguishing
between human visitors to a site and automated scripts pretending to be
humans (Ahn et al., 2003). The original task—deciphering letters in a word
distorted so as to defeat computer vision programs—has since spawned
numerous variations familiar to users of many web sites.
A closely related line of research explored related ideas, originally in the
realm of image annotation. Annotations in the form of image labels are required
to support image search, as computer vision tools may not be sufficiently
powerful to identify image content matching terms of interest. However, these
labels are not easy to come by, as they must be generated by humans who must
interpret the images and provide descriptions. Noting these problems, Luis
von Ahn and Laura Dabbish suggested a simple and intriguing solution: turn it
into a game. The ESP game presents two players with an image, asking them
both to provide a label describing the image. The players are challenged to
come up with an agreed-upon description, getting points for each agreement,
with large bonuses for surpassing a certain goal in a given time period. The
need for agreement creates the challenge that makes the game enjoyable, while
increasing the quality of the labels, as two participants are unlikely to agree
upon an inaccurate description. Additional labels can be generated for each
image through the use of “taboo” words: once a first pair of partners labels an
image, subsequent partners will be asked to find a label without using any of the
previously used words (Ahn and Dabbish, 2004). The ESP game introduced the
notion of “Games with a purpose”—tools that hide useful work under the guise
of a challenging and enjoyable game (Ahn and Dabbish, 2008). Just as Tom
Sawyer turned the work of painting a fence from a chore into a pleasure, games
with a purpose turn image labeling and other tedious tasks into a bit of fun.