NASA Human Research Program Investigators' Workshop (2012)
4193.pdf
HUMAN-AUTOMATION INTEGRATION: PRINCIPLE & METHOD FOR DESIGN AND EVALUATION
D. Billman1 and M. Feary2
1
San Jose State University Research Foundation @ NASA Ames Research Center, Moffett Field, CA 94035-1000,
dorrit.billman@nasa.gov,
2
NASA Ames Research Center, Moffett Field, CA 94035-1000, michael.s.feary@nasa.gov.
MOTIVATION & GOALS
Poor design of how automation supports human activity and goals is a major contributing factor to accidents and
incidents in safety critical work domains. Methods for evaluating quality of human-automation integration (HAI)
and for generating high-quality HAI designs would be very valuable. Our research aims to identify principles of
good HAI designs, assess the impact of hypothesized principles, and establish methods for using such principles in
evaluation and generation of HAI designs.
THE STRUCTURE ALIGHNMENT PRINCIPLE
The elements and organization of an operator's interaction with
automation (the integration structure) should align, or correspond,
with the elements and organization of the work domain (the work
structure). Structure of work comes from its entities, relations, and
operations; from the tasks and procedures carried out on those
entities; and from the constraints on how changes to entities can be
made. Entities might be physical, such as vehicles and lab equipment, or
conceptual, such as plans and reports. Often many aspects of domain and of
interaction structure can be represented in part-whole hierarchies. For
example, a larger Activity such as docking a Soyuz can be composed from
(temporally sequenced) Actions making up the Activity. When the
information provided in displays and the operations available through controls
is organized to preserve the structure of the entities and activities in the work
domain, then the interaction is well-aligned.
We studied the structure of planning work carried out by the Attitude
Determination and Control (ADCO) group in the International Space Station (ISS) Mission Control. Using our
work analysis, we applied the Structure Alignment Principle to prototype new planning software. For example, the
redesign provided analog representation of time in plans and allowed rescheduling Activities as meaningful units.
EMPIRICAL ASSESSMENT OF STRUCTURE ALIGHNMENT
We compared performance using the legacy versus new planning software. Tasks and materials were close analogs
of editing tasks done during ISS Increment 22 (2009-10). Participants were 17 trained, technical upper-level and
graduate students. Overall times were cut roughly in half in the New software condition (Condition, Task Type, and
Cond X Type F's >15, p<.001); error rate dropped from 27% in the Legacy to 13% in the New Condition. Further,
the figure shows, as predicted, the largest improvement
occurred for editing Activities, the task with the largest
improvement in alignment.
FUTURE WORK
We plan to extend this work by a) providing more structured
methods for representing and measuring the work structure,
the HAI structure, and the degree of alignment; b) providing
additionally controlled comparisons to ensure benefits came
from the increased alignment; and c) to measure benefit of
alignment to robust performance, e.g, over time and across
unexpected/problematic conditions.
REFERENCES
[1] Billman, D., Arsintescu, L., Feary, M., Lee, J., Smith,
A.,& Tiwary, R. (2011). Proceedings of the 29th Annual ACM Conference on Human Factors in Computing Systems
(CHI 2010); 2521-2530. [2] Billman, D., Feary, M., Schreckenghost, D., & Sherry, L. (2010). Proceedings of the
28th Annual ACM Conference on Human Factors in Computing Systems (CHI 2010); p 4597-4612.