Page 47 - Innovations in Intelligent Machines

Predicting Operator Capacity for Supervisory Control of Multiple UAVs 35

Fig. 15. Wait Times in the Queue across Levels of Automation

wait times. Because the SA optimization approach and other similar sto-
chastic approaches assume an a priori distribution (both in arrival rates and
service times), if such simulation methods are not used in conjunction with
experimentally derived data, results are highly speculative and lack external
validity.
This last point about the problem with assumptions highlights an inherent
limitation to both methods: Estimating interaction, neglect, and wait times.
As previously discussed, for the cost-based simulation, a distribution must be
selected for wait times, and presently there is little theoretical or empirical
basis for doing so. In addition, interaction and neglect times must be selected a
priori and while these could be estimated from system design parameters, they
are highly contextual and will likely dramatically change with different levels
of autonomy, decision support, mission complexity, operator training, etc.
Similarly, even experimentally derived interaction, neglect, and wait times
can be difficult to measure. Unfortunately the times and the associated costs
(degraded performance, etc.) are very difficult to capture in performance-
based simulations such as the one reported in this study. Through using soft-
ware that tracked users’ cursor movements and activation of control devices,
we were able to determine on a gross level when a subject was actively engaged
with a particular UAV, but subtle transitions are difficult to capture. The
use of psychophysiologic measurement devices may be of use in addition to
performance-based measures but the application of these methods needs sig-
nificantly more investigation.