Page 33 - Innovations in Intelligent Machines

Predicting Operator Capacity for Supervisory Control of Multiple UAVs 21
Results and Discussion. In order to determine whether or not the revised
fan-out prediction in (3) provided a more realistic estimate than the original
fan-out (1), the number of vehicles controlled in the experiment was held con-
stant (four) across all levels of automation. Thus if our proposed prediction
was accurate, we should be able to predict the actual number of vehicles the
operators were controlling. As previously discussed, all times were measured
through interactions with the interface which generally included mouse move-
ments, selection of objects such as vehicles and targets for more information,
commanding vehicles to change states, and the generation of communication
messages.
Neglect time was counted as the time when operators were not needed by
any single vehicle, and thus were monitoring the system and engaging in sec-
ondary tasks such as responding to communications. Because loiter paths were
part of the preplanned missions, oftentimes to provide for buffer periods, loiter
times were generally counted as neglect times. Loitering was only counted as
a wait time when a vehicle was left in a loiter pattern past a planned event
due to operator oversight. Interaction time was counted as any time an oper-
ator recognized that a vehicle required intervention and specifically worked
towards resolving that task. This was measured by mouse movements, clicks,
and message generations. The method of measuring NT and IT, while not
exactly the same as [11], was driven by experimental complexity in represent-
ing a more realistic environment. However, the same general concepts apply in
that neglect time is that time when each vehicle operated independently and
interaction time is that time one or more vehicles required operator attention.
As discussed previously, wait times were only calculated when one or more
vehicle required attention. Wait time due to interactions (e.g., the time it
took an operator to replan a new route once a UAV penetrated a threat area)
was subsumed in interaction time. Wait time due to queuing occurred when,
for example, a second UAV also required replanning to avoid an emergent
threat and the operator had to attend to the first vehicle’s problem before
immediately moving to the second. Wait time due to the loss of situation
awareness was measured when one or more vehicles required attention but was
not noticed by the operator. This was the most difficult wait time to capture
since operators had to show clear evidence that they did not recognize a UAV
required intervention. Examples of wait time due to loss of situation awareness
include the time UAVs spend flying into threat areas with no path correction,
and leaving UAVs in loiter patterns when they should be redirected.
Figures 5 and 6 demonstrate how the wait times varied both between the
two fan-out equations as well the increasing levels of automation under low
and high workload conditions respectively. Using the interaction, neglect, and
wait times calculated from the actual experiment, the solid line represents the
predictions using (1), the dashed line represents the predictions of (2), and the
dotted line shows how many UAVs the operators were actually controlling,
which was held constant at four.