To support the claims above, there are several key studies. The first study was published by the NBAA in 2003. This publication directly addressed the shift in training philosophies for pilots transitioning from conventional cockpits to glass cockpits. The NBAA provided guidance and suggested varying methods to better integrate CRM skills with automation management, mode awareness, and the use of standard automation configurations. The paper also identified four discrete levels of automation that require mastery by the pilots. These levels include the collective use of the flight director, autopilot, auto throttle, and the flight management system’s vertical and lateral path guidance features.
Before the NBAA study, NASA (1999) published a study that took a detailed look at two major air carriers that acquired fleets of advanced technology aircraft in the 1993-1997 timeframe. The volunteers for this study consisted of 150 line pilots that were all entering the transition training program. The study provided important insights into how the difference in age, experience, and attitude impacts the effectiveness of the ab initio glass cockpit training program.
The effects of automation integration were also studied by Endsley (1997), and his paper presented the results of three studies that evaluated the effects of automation levels on performance, situation awareness, and workload under both normal and failure conditions. The research found that automation could enhance system performance if it provided aids to the implementation of a task, but hindered the task if there had to be a joint human-automation decision loop.
Kaber and Endsley (1997) published a similar paper that described their research on the effects of adaptive automation pertaining to human performance within complex, dynamic control systems. In this study, subjects performed a control task and a secondary monitoring task with various levels of automation. The testing involved five levels of automation with varying time cycles. The results indicated that the level of automation of the secondary monitoring task has a significant effect on the primary control task. Conversely, if the primary task is more automated there is a greater amount of perceptual resources to observe secondary task events.
A year before Kaber and Endlsey’s research, Parasuraman, Mouloua, and Malloy (1996) published a paper that discussed the effects of adaptive task allocation on the monitoring of automated systems. This study examined the effect of adaptive task allocation on the ability to monitor for automation failures during multitask flight simulations. The test consisted of monitoring an automated engine status task while simultaneously performing fuel management calculations over a thirty minute time frame. The results of this test showed that there was a low probability of detecting the failure of the automation after 40 minutes of automated operation, but a higher likelihood of detection exists after intermediate periods (10 minutes) of manual control.
Pilot performance under situations involving adaptive automation was again studied in 1995, by Scallen, Hancock, and Duley. The purpose of their study was to investigate the responses of pilots to the rapid cycling of automation. The task load was simulated by a battery of tests consisting of tracking, fuel management, and system monitoring subtasks. The tracking tasks were alternated between manual and automatic control at 15, 30, or 60 second intervals. From this study, the authors concluded that excessively short cycles of automation are disruptive to operator performance that requires multitasking.
Scallen, Hancock, and Duley (1994) published another paper that described the responses of experienced pilots to configuration changes in the automation cycle. In this study, pilots were presented with a situation that involved multi-tasking. Their results confirmed other studies that a considerable workload is developed by the sheer task of switching between manual and automatic modes. They also contend that automation can impair manual monitoring performance, especially under emergency conditions. They labeled this phenomenon “automation deficit.”
Another study examined how pilots responded to both passive and active automation tasks. This study was conducted by Idaszak and Hulin (1989) and employed a process control task in which subjects were tasked with passive and active monitoring of a system. They had to respond to limits, alarms, and deviations in process parameter values. Results indicated that the active group was faster at detecting anomalies. The researchers in this study attribute this to the fact that active participation (as opposed to passive) increases the operators’ workload and serves as a source of motivation to improve performance.
Finally, the earliest results of this literature search were found to be in 1981, when Wickens and Kellel published a paper on their research that investigated the probability of failure detection, in dynamic systems, by operators. This study involved subjects that were tasked with monitoring and controlling a tracking task. Within the control groups it was found that the speed of failure detection was significantly slower in the automated condition than the manual condition. This was attributable to the belief that the subjects had a better mental representation of the system dynamics in the manual mode because of visual and kinesthetic feedback.