Page 35 - Innovations in Intelligent Machines

Predicting Operator Capacity for Supervisory Control of Multiple UAVs 23
The second trend of note is the fact that for the low workload condition,
the revised fan-out model (3) provides a more conservative estimate of approx-
imately 20% under that of the model that does not consider wait times (1).
However, the third important trend in this graph demonstrates that for both
(1) and (3) the predictions were much higher than the actual number of UAVs
controlled. This spare capacity under the low workload condition was empir-
ically observed, in that subjective workload measures (NASA-TLX) and per-
formance scores were statistically the same when compared across all four
levels of autonomy (lowest pair wise comparison pvalue = .111 (t = 1.79,
DOF = 8), and p = .494 (t = .72, DOF = 8) respectively).
Thus for the low workload condition across all levels of automation, oper-
ators were underutilized and performing well. Thus they theoretically could
have controlled more vehicles. Using the revised FO model (3), under the
manual, passive, and active conditions, operators’ theoretical capacity could
have increased by ∼75% (up to 7 vehicles). Under the highest autonomy for
mission management, predictions estimate operators could theoretically con-
trol (as an upper limit) four times as many, ∼17 vehicles. Previous air traffic
control (ATC) studies have indicated that 16-17 aircraft are the upper limit
for en route air traffic controllers [17]. Since controllers are only providing nav-
igation assistance and not interacting with flight controls and mission sensors
(such as imagery), the agreement between ATC en route controller capacities
and low workload for UAV operators is not surprising.
High Workload Predictions. While the low workload results and predictions
suggest that operators are capable of controlling more than four vehicles in
MAUVE, the results from the high workload scenarios paint an entirely dif-
ferent picture. The high workload scenarios were approximately double the
workload over the low workload scenarios, and represent a worst case sce-
nario. Performance results indicate that those operators with the active level
of automation were not able to control their four UAVs effectively, but all
other operators were with varying degrees of success. As in the low work-
load condition, the revised fan-out model (3) is the more conservative and as
demonstrated in Figure 6, more closely predicts the actual number of four vehi-
cles assigned to each operator. Moreover, while under the low workload con-
dition, the estimates of controller capacity dropped almost uniformly across
automation levels by 20% for the original fan-out model. However, under high
workload, they dropped 36–67% for the model that includes wait times. The
largest difference between conditions occurred for the active level of automa-
tion. In addition to the lower number of predicted vehicles, the active condi-
tion produced statistically lower performance scores (e.g., t = 2.26, DOF = 8,
p=0.054 for the passive-active comparison). This was attributed to the
inability of subjects in the active condition to correctly weight uncertainty
parameters and is discussed in detail elsewhere [16].
As in the low workload results, subjects performed the best (in terms
of time management) under the highest level of automation for mission