Towards the Exploitation of Mental Workload in Virtual Reality Systems

Despite the rapid emergence of autonomous systems and the development of affective computing, little studies considered mental workload in the design of VR training scenarios. This thesis aims to contribute to the development of adaptive VR systems based on the users’ mental workload. We propose 3 research axes: induction, recognition, and exploitation of mental workload in VR, and a definition of “Affective and Cognitive Virtual Reality”. First, we study the impact of wearing a VR HMD on the users’ mental efforts. In addition, the potential influence of walking and accommodation to VR effect are analyzed. Then, we propose a methodological approach to introduce mental workload assessment in the design of VR training scenarios in complex immersive virtual environments. This methodology enables to modulate users’ mental workload levels over time. User studies in a VR flight simulator are conducted to evaluate the approach. Finally, we propose an all-in-one solution to recognize users mental workloads in real-time using integrated physiological sensors into a VR HMD. This setup is compared to a commercial-grade system with regards to the mental workload recognition accuracy. The influences of the type of measures, sensors, and signal normalization methods is also investigated.