HP Omnicept Cognitive Load Database (HPO-CLD) – Developing a Multimodal Inference Engine for Detecting Real-time Mental Workload in VR
Abstract. The HP-Omnicept CL database was created as part of an effort to reliably estimate users’ mental effort (i.e., cognitive load) in real-time while they completed cognitively demanding tasks in virtual reality (VR) environments. In this technical report, we present a novel and robust machine learning method for assessing cognitive workload based on behavioral and physiological measures. Over approximately one hour, participants completed multiple tasks, requiring different levels of mental effort while we passively recorded their physiology, tracked their performance, and collected self-reports. We used these data to train machine learning models to predict task difficulty and participants’ momentary self-reported cognitive load. Using a novel labeling pipeline, we achieved an average classification accuracy of 79.08% with a mean absolute error 0.1106. These results indicate that, with a combination of behavioral and physiological indicators, we can reliably predict cognitive load in real-time, without calibration. As part of this white paper, we are releasing our test dataset (n = 100) for use by researchers and developers interested in machine learning, virtual reality, learning & memory, cognition, or psychophysiology. This dataset includes recordings from multiple sensors (including pupillometry, eye-tracking, pulse plethysmography, head and hand tracking), self-reported cognitive effort, behavioral task performance, and demographic information on the sample.
Full technical report and downloadable dataset available April 30, 2021
Siegel, E.H., Wei, J., Gomes, A., Oliviera, M., Sundaramoorthy, P., Smathers, K., Vankipuram, M., Ghosh, S., Horii, H., Bailenson, J. & Ballagas, R. “HP Omnicept Cognitive Load Database (HPO-CLD) – Developing a Multimodal Inference Engine for Detecting Real-time Mental Workload in VR” Technical Report, HP Labs, Palo Alto, CA. Available at: https://developers.hp.com/omnicept/omnicept-open-data-set-abstract