Towards a Multimodal Model of Cognitive Workload Through Synchronous Optical Brain Imaging and Eye Tracking Measures

Download

fnhum-13-00375.pdf

Date

2019-10-23

Author

Isbilir, Erdinc
Çakır, Murat Perit
Acartürk, Cengiz
Tekerek, Ali Simsek

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

115
views

119
downloads

Recent advances in neuroimaging technologies have rendered multimodal analysis of operators' cognitive processes in complex task settings and environments increasingly more practical. In this exploratory study, we utilized optical brain imaging and mobile eye tracking technologies to investigate the behavioral and neurophysiological differences among expert and novice operators while they operated a human-machine interface in normal and adverse conditions. In congruence with related work, we observed that experts tended to have lower prefrontal oxygenation and exhibit gaze patterns that are better aligned with the optimal task sequence with shorter fixation durations as compared to novices. These trends reached statistical significance only in the adverse condition where the operators were prompted with an unexpected error message. Comparisons between hemodynamic and gaze measures before and after the error message indicated that experts' neurophysiological response to the error involved a systematic increase in bilateral dorsolateral prefrontal cortex (dlPFC) activity accompanied with an increase in fixation durations, which suggests a shift in their attentional state, possibly from routine process execution to problem detection and resolution. The novices' response was not as strong as that of experts, including a slight increase only in the left dlPFC with a decreasing trend in fixation durations, which is indicative of visual search behavior for possible cues to make sense of the unanticipated situation. A linear discriminant analysis model capitalizing on the covariance structure among hemodynamic and eye movement measures could distinguish experts from novices with 91% accuracy. Despite the small sample size, the performance of the linear discriminant analysis combining eye fixation and dorsolateral oxygenation measures before and after an unexpected event suggests that multimodal approaches may be fruitful for distinguishing novice and expert performance in similar neuroergonomic applications in the field.

Subject Keywords

Neuroergonomics, fNIRS, Eye tracking, Human machine interface, Cognitive workload

URI

https://hdl.handle.net/11511/30462

Journal

FRONTIERS IN HUMAN NEUROSCIENCE

DOI

https://doi.org/10.3389/fnhum.2019.00375

Collections

Graduate School of Informatics, Article

Suggestions

OpenMETU
Core

Evaluation of UAS Camera Operator Interfaces in a Simulated Task Environment An Optical Brain Imaging Approach Çakır, Murat Perit; Akay, Daryal; Ayaz, Hasan; İşler, Veysi (null; 2012-07-11) In this paper we focus on the effect of different interface designs on the performance and cognitive workload of sensor operators (SO) during a target detection task in a simulated environment. Functional near-infrared (fNIR) spectroscopy is used to investigate whether there is a relationship between target detection performance across three SO interfaces and brain activation data obtained from the subjects’ prefrontal cortices that are associated with relevant higher-order cognitive functions such as atten...
Representing temporal knowledge in connectionist expert systems Alpaslan, Ferda Nur (1996-09-27) This paper introduces a new temporal neural networks model which can be used in connectionist expert systems. Also, a Variation of backpropagation algorithm, called the temporal feedforward backpropagation algorithm is introduced as a method for training the neural network. The algorithm was tested using training examples extracted from a medical expert system. A series of experiments were carried out using the temporal model and the temporal backpropagation algorithm. The experiments indicated that the alg...
Exploring hand movement dynamics during the Simon task: a mouse tracking study. İkizoğlu, Hatice Buke; Çakır, Murat Perit; Department of Cognitive Sciences (2019) Understanding how the cognitive system processes information in real-time is one of the key concerns for researchers in experimental psychology and cognitive science. Several measures and software tools have been proposed to explore different aspects of cognitive processing phenomena. One of these methods, mouse-tracking, allows researchers to collect data about the dynamic unfolding of motor responses by recording the participants’ mouse movements during cognitive tasks. In the present study, we replicated...
An Approach to manage variability in object-oriented applications with feature models Bulut, Ender; Şener, Cevat; Department of Computer Engineering (2014) In this thesis, an approach to manage variability in object-oriented applications by using a feature modeling language and a simple source code generation technique has been designed and developed. This approach provides developing configurable object oriented applications in a practical way. That is, an application developed with our approach takes just a configuration file including user selections in a pre-defined domain as input and then automatically configure and manage itself with respect to these se...
A temporal neural network model for constructing connectionist expert system knowledge bases Alpaslan, Ferda Nur (Elsevier BV, 1996-04-01) This paper introduces a temporal feedforward neural network model that can be applied to a number of neural network application areas, including connectionist expert systems. The neural network model has a multi-layer structure, i.e. the number of layers is not limited. Also, the model has the flexibility of defining output nodes in any layer. This is especially important for connectionist expert system applications.

Citation Formats

E. Isbilir, M. P. Çakır, C. Acartürk, and A. S. Tekerek, “Towards a Multimodal Model of Cognitive Workload Through Synchronous Optical Brain Imaging and Eye Tracking Measures,” FRONTIERS IN HUMAN NEUROSCIENCE, pp. 0–0, 2019, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/30462.