The Game-Changing Role of Technology

The defining characteristic of the modern age is the exponential growth of technology. Rather than displacing the roles of humans, in certain ways technology has radically increased the value and importance of human performance skillsets.

A key reason is that the monetary value of technology systems operated by personnel can be tremendously high.  A pertinent case is the B-2 Spirit stealth bomber, which had a total production cost of around 2 billion US dollars, per plane.  Not only do operators of expensive technology systems carry huge responsibilities, but they also have to develop extremely high standards of expertise, with high associated training costs.

The Training Industry Challenge

With these types of scenarios, the training industry has been daunted with the challenge of how to ensure the effectiveness of training programs for high priority personnel.  Jet pilots are a classic example.  Competency requires pushing the cognitive and physiological barriers of human performance. Alongside this, expert proficiency comes with a very high price tag, requiring thousands of hours of flight experience. Yet at the end of the day, and regardless of training investments, some pilots excel in training, while others fail. Traditionally there has been no way to properly understand or predict such outcomes.

Innovation in Human Performance Systems

To tackle this challenge, a multidisciplinary coalition of neuroscientists, experts in simulation training, and flight training specialists attempted to discover what actually goes on in the minds of jet pilots during training.  In a truly innovative setup, they took an L-29 jet plane and integrated the dashboard with a NeuroTracker system, and then hooked up pilots with eye tracking and ECG equipment.

The goal was to study the real-time impacts of flight in terms of actual neurophysical training loads – a world-first in aviation.

Determining Workload Limits

A key concept used was ‘spare cognitive capacity’, that is, the attentional resources still available when performing a task.  This is both relative to the complexity of the task, and to the capabilities of the individual.  For example, driving leaves some people enough spare cognitive capacity to talk on a cell phone, but for others, this is a dangerous distraction.

The aim was to use NeuroTracker to measure pilots’ spare cognitive capacity while actually performing 3 difficulty levels of flight maneuvers, and to replicate the tests in a simulator. This would then provide an objective assessment of the workload effects of specific flight tasks, revealing how these impact pilot performance and physiological metrics.

Flight Test Results

Flight performance was evaluated using the Cognitive Assessment Tool Set (CATS), and pilots were asked to subjectively evaluate the workloads experienced under each flight condition.

Overall, the findings showed that the more difficult the flight maneuver was, the less spare cognitive capacity was available for the NeuroTracker task. These effects were much greater for live flight versus simulated flight.

Reductions in spare cognitive capacity also correlated with lower flight technical performance.

Self-assessments revealed that pilots greatly underestimated the true cognitive workloads, as established by NeuroTracker, CATS, and physiological measures.  In effect, pilots were not aware when their workload capacities had become overloaded, lowering the effectiveness of their training.

Adaptive Training

This study shed new light on the direct relationships between mental and physiological workloads, and their combined influence on training performance.  The data could be of direct benefit for customizing training programs to fit individual needs.

For example, it could be used to restrict a weaker pilot to low difficulty live flights and medium difficulty simulated flights. Or alternatively, to set high difficulty flights for a highly proficient pilot.  This would tailor the demands of training through an on-going Goldilocks approach, ensuring every training session is optimized to each pilot’s training needs.

Training Industry Benefits

This study represents the first year of a multi-year research project, which will go on to include the dimension of expertise to investigate its influence on workload capacities. Although this research is specific to pilots, the assessment principles translate to training programs involving high-cost coupled with high-levels of expertise acquisition.

Fundamentally, this approach measures trainee workload capacities in real-time, in parallel with task performance metrics.  It can be used across both training and operational platforms, and across military and commercial domains. As such it paves the way to applications that will improve the effectiveness of training programs in the following areas:

Reduced attrition rates – assess trainee workload capabilities and filter trainees for program selection based on their training competency and completion expectations.

Customized training – adapt training to meet each trainee’s specific needs, such as modulating training tasks to specific workload strengths and weaknesses.

Accelerated learning – engineer training difficulty through a sweet spot approach, optimizing training stimulation to individual workload capacities.

Adaptive learning – real-time alteration of training content to each trainee’s skill level and cognitive state as their abilities adapt as a function of time spent in training.

Training device selection – by comparing variations in workload rates for one training system compared to another, individual trainees can be matched to devices based learning efficiency, lowering overall training costs.

Looking forward, it is easy to envisage this kind of scientific research leading to improved training outcomes across many industries.  In this case, those outcomes will likely be accelerated, due to the novel partnership between scientific laboratories and industry leaders in commercial training solutions.

About the Study

The Faubert Applied Research Centre, the University of Montreal, Rockwell Collins (avionics and simulation training company), and the University of Iowa’s Operator Performance Lab, partnered their fields of expertise to come up with an innovative way to assess the mental loads of flying. The paper was presented at the Interservice/Industry Training, Simulation and Education Conference (I/ITSEC) 2017 and was awarded best paper for the Training Category.

Perceptual-Cognitive & Physiological Assessment of Training Effectiveness

‍