What the eyes reveal: Investigating the detection of automation failures.

In order to detect automation failures in a timely manner, operators are required to monitor automated systems efficiently. The present study analysed eye movements to predict whether or not participants could detect an automation failure. Eye movements were recorded whilst 101 participants were monitoring an automated system where automation failures occurred at irregular intervals. A main result is that about 75.6 per cent of automation failure detections were predicted successfully by the corresponding eye movements. In cases where failures were detected successfully, relevant information is monitored more often and more intensively, in particular shortly before an automation failure happens and while it is happening. The findings are discussed in the context of the personnel selection and training of aviation operatives, as well as incident reporting as used in air traffic control (ATC).

Eye-tracking measurements and their link to a normative model of monitoring behaviour.

Increasing automation necessitates operators monitoring appropriately (OMA) and raises the question of how to identify them in future selections. A normative model was developed providing criteria for the identification of OMA. According to this model, the monitoring process comprises distinct monitoring phases (orientation, anticipation, detection and recheck) in which attention should be focused on relevant areas. The current study tests the normative model on the basis of eye tracking. The eye-tracking data revealed increased concentration on relevant areas during the orientation and anticipation phase in comparison to the other phases. For the assessment of monitoring behaviour in the context of personnel selection, this implies that the anticipation and orientation phases should be considered separately as they appear to be more important in the context of monitoring than the other phases. PRACTITIONER SUMMARY: A normative model was developed for the assessment of monitoring behaviour. Using the eye-tracking method, this model was tested with applicants for an Air Traffic Controller training programme. The results are relevant for the future selection of human operators, who will have to monitor highly automated systems.