Detecting Users' Emotional States during Passive Social Media Use
A study detecting users' emotional states during passive social media use using physiological and behavioral signals.
May 15, 2024
VR Grand Piano
[WIP] A personal project to play on a grand piano in an adaptive VR environment using a MIDI keyboard
Mar 3, 2023
Characterizing Physiological Responses to Fear, Frustration, and Insight in Virtual Reality
Physiological sensing often complements studies of human behavior in virtual reality (VR) to detect users’ affective and cognitive states. Some psychological states, such as fear and frustration, can be particularly hard to differentiate from a physiological perspective as they are close in the arousal and valence emotional space. Moreover, it is largely unclear how users’ physiological reactions are expressed in response to transient psychological states such as fear, frustration, and insight—especially since these are rich indicators for characterizing users’ responses to dynamic systems but are hard to capture in highly interactive settings. We conducted a study (N=24) to analyze participants’ pulmonary, electrodermal, cardiac, and pupillary responses to moments of fear, frustration, and insight in immersive settings. Participants interacted in five VR environments, throughout which we measured their physiological reactions and analyzed the patterns. We also measured subjective fear and frustration using questionnaires.
Oct 24, 2022
A Survey on Affective and Cognitive VR
In this paper, we introduce the term "Affective and Cognitive VR" and its main research lines. This survey also clarifies the different models of Affective and Cognitive States (ACS), presents the methods for measuring them with their respective advantages and drawbacks in VR, and showcases Affective and Cognitive VR studies done in an Immersive Virtual Environment (IVE) in a non-clinical context. We provide a comprehensive list of references with the analysis of 63 research articles and summarize future works directions.
Sep 6, 2021
Towards the Exploitation of Mental Workload in Virtual Reality Systems
My PhD thesis aims to contribute to the development of adaptive VR systems based on the users' mental workload. We investigate 3 research axes: induction, recognition, and exploitation of mental workload in VR. In addition, we propose a definition of “Affective and Cognitive Virtual Reality”.
Jan 14, 2021
Towards Real-Time Recognition of Users’ Mental Workload Using Integrated Physiological Sensors Into a VR HMD
This paper describes an ``all-in-one'' solution for the real-time recognition of users' mental workloads in VR through the customization of a commercial HMD with physiological sensors. We conducted an extensive user study (N = 75) to collect mental workload data. The classification accuracy of 4 levels of mental workload using sensors embedded in the HMD is compared to the classification performance using commercial grade systems. In addition, we discuss the role of the signal normalization procedures and the contribution of the different physiological signals on the recognition accuracy.
Nov 6, 2020