16 | Detection of carsickness through facial expressions analysis

Bougard C, Fenaux E, Henry E, Jelassi O, Bringoux L | Stellantis, Telecom Paris & Aix MU CNRS ISM, France

Background: Carsickness is a common condition causing discomfort during travel. Early detection could enable timely interventions to reduce symptoms. This study investigates facial expressions as indicators of carsickness, aiming to develop a reliable predictive model for detecting carsickness based on emotion recognition from facial analysis.

Materials and Methods: Twenty-four healthy participants took part as front passengers in a real driving slalom session (0.2 Hz lateral movements, C4 Picasso Citroën). Symptoms were self-reported after each 300 m slalom using a 5-point Likert scale (0 = no symptoms, 4 = moderate nausea). Sessions lasted up to 20 minutes or ended earlier if participants rated 4. Video was recorded at 30 fps using a Logitech® C920 webcam. The dataset included 2,849 samples, with 909 labeled as “carsick”, and the remainder as "non-carsick". Two classes of trials were defined: low carsickness (<2) and high carsickness (>2.7). Each frame was analyzed using a pre-trained emotion recognition network, yielding a vector of 7 emotional probabilities (anger, disgust, fear, joy, surprise, neutral). These vectors were used to train an Explainable Boosting Machine (EBM) model.

Results: The trained EBM model achieved an average AUC of 0.84, with a particularly promising True Positive Rate of 66% and False Positive Rate of 15% at the optimal threshold. Further, using 12 interactions yielded the best performance, reaching 79.4% accuracy and 0.85 AUC. This approach demonstrated a slight improvement in detecting carsick individuals and a robust performance for non-carsick individuals. Importantly, this approach requires no environment-specific adaptation, making it versatile across varied use cases.