A multimodal study of the interplay between stress, executive function, and biometrics in game-based assessment

Abstract

Managing stress is a crucial soft skill that affects cognitive performance and health. Stress detection through biometrics can be used to improve and evaluate stress management. However, measuring the effects of stress on biometrics and executive functions is difficult and dependent on the individual. Despite these challenges, this paper presents a case study that collects a comprehensive multimodal dataset with two stress metrics, four biometric signals, and twenty-two executive function metrics from Game-based Assessment (GBA) trace data specifically designed for this purpose. The experiments suggest that biometrics, especially the heart rate and skin temperature, are effective predictors of stress. Additionally, noteworthy correlations were observed between heart rate and certain executive function variables. The levels of GBA that measured shifting and processing speed showed a higher heart rate than the response inhibition levels. This case study, together with the developed stress detectors, enables the detection of persons who struggle to manage stress and measure their executive function performance under stressful situations.