Deep Learning Forecast of Cognitive Workload Using fNIRS DataNicolas Grimaldi, Yunmei Liu, Ryan McKendrick, Jaime Ruiz, and David Kaber.

Introduction: In the domain of helicopter piloting, the pilot’s performance is driven by many cognitive processes, demanding substantial cognitive resources. The pilot must maintain situation awareness and perform rapid decision-making. An objective of integrated helicopter technologies is to predict and effectively manage pilot cognitive workload to ensure safety and efficiency throughout flight. Methods: In this study, we collected data on seven participants, including three experienced pilots, using a UH-60V cockpit simulator to perform 46 distinct trials under various flight conditions. fNIRS neuroimaging was used to collect high-resolution neurophysiological data for exploring and forecasting cognitive workload using a collection of deep learning models. Model implementation: Three deep learning architectures are detailed in this work: a stacked LSTM model, a CNN-LSTM hybrid, and a transformer model. Results: An evaluation of three Seq2Seq models, each with two distinct forecasting lengths (10s and 30s), revealed LSTM-based architectures as superior performers for 10s forecasting tasks. Discussion: The LSTM-based models’ superior performance suggested potential limitations with the transformer’s self-attention mechanisms for our specific application. Surprisingly, the CNN-LSTM architecture did not surpass the stacked LSTM model’s performance during forecasting tasks. Conclusion: Future research directions include exploring diverse time-series Seq2Seq methods and forecasting cognitive workload as ordinal measures, offering insights into shifting cognitive demands.