Page 376 - Sensing, Intelligence, Motion : How Robots and Humans Move in an Unstructured World

EXPERIMENT DESIGN 351

(a) gender, with values “males” and “females”,
(b) specialization, with values “engineering,” “natural sciences,” and “social
sciences”, and
(c) age, with values “15–24,” “25–34”, and “above 34”.
A total of 48 subjects have been tested in Experiment One, with the following
distribution between genders, specializations, and age groups:
Gender: 23 males and 25 females
Specialization: engineering, 16; natural sciences, 20; social sciences, 12
Age groups: age 15–24, 32 subjects; age 25–34, 12 subjects; age above 34,
4 subjects

From the standpoint of statistical tests, subjects have been selected randomly,
and so test observations can be considered mutually independent.
One problem that must be addressed in the test protocol is avoiding the learning
effect: We need to prevent the subjects from using in one test the knowledge that
they acquired in a prior test. Then one’s performance is independent of the order
of tasks execution. This is important when randomly varying the order of tasks
between subjects, a standard technique in cognitive skills tests. This has been
achieved by making each subject go through 4 out of 8 tasks. For example, a
subject who went through the virtual–visible–LtoR task would not be subjected
to the virtual–visible–RtoL task. With this constraint, half of the subject pool did
tasks 1, 4, 5, 8, while the other half did tasks 2, 3, 6, and 7 from the list above.
As a result, each of the eight tasks should have 24 related observation sets,
each set including the path length, the completion time, and the actual data on the
generated path for one subject. (In reality, two observation sets for the physical
test, one for the visible–RtoL and the other for the invisible-LtoR combina-
tion, were documented incorrectly and were subsequently discarded, leaving 23
observations for Task 6 and Task 7 each.)
Another way to group the observed data is by the three factors (interface, visi-
bility, direction), each with two levels: (virtual, physical), (visible, invisible), and
(LtoR, RtoL). This grouping produces six data sets and is useful for studying sepa-
rate effects on human performance—for example, the effect on one’s performance
of moving a visible arm or moving left to right (see the next section). With two lost
observations mentioned above, the sizes of the six data sets are as follows:

Set 1: Interface data for “virtual”—96 observations
Set 2: Interface data for “physical”—94 observations
Set 3: Visibility data for “visible”—95 observations,
Set 4: Visibility data for “invisible”—95 observations,
Set 5: Direction data for “left to right”—95 observations,
Set 6: Direction data for “right to left”—95 observations.

A total of 12 subjects have been tested in Experiment Two.