The Empathetic Car : Exploring Emotion Inference via Driver Behaviour and Traffic Context

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

17 Scopus Citations
View graph of relations

Author(s)

  • Shu Liu
  • Kevin Koch
  • Simon Föll
  • Xiaoxi He
  • Tina Menke
  • Elgar Fleisch
  • Felix Wortmann

Detail(s)

Original languageEnglish
Article number117
Journal / PublicationProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Volume5
Issue number3
Online published14 Sept 2021
Publication statusPublished - 2021
Externally publishedYes

Abstract

An empathetic car that is capable of reading the driver's emotions has been envisioned by many car manufacturers. Emotion inference enables in-vehicle applications to improve driver comfort, well-being, and safety. Available emotion inference approaches use physiological, facial, and speech-related data to infer emotions during driving trips. However, existing solutions have two major limitations: Relying on sensors that are not built into the vehicle restricts emotion inference to those people leveraging corresponding devices (e.g., smartwatches). Relying on modalities such as facial expressions and speech raises privacy concerns. By contrast, researchers in mobile health have been able to infer affective states (e.g., emotions) based on behavioral and contextual patterns decoded in available sensor streams, e.g., obtained by smartphones. We transfer this rationale to an in-vehicle setting by analyzing the feasibility of inferring driver emotions by passively interpreting the data streams of the control area network (CAN-bus) and the traffic context (inferred from the front-view camera). Therefore, our approach does not rely on particularly privacy-sensitive data streams such as the driver facial video or driver speech, but is built based on existing CAN-bus data and traffic information, which is available in current high-end or future vehicles. To assess our approach, we conducted a four-month field study on public roads covering a variety of uncontrolled daily driving activities. Hence, our results were generated beyond the confines of a laboratory environment. Ultimately, our proposed approach can accurately recognise drivers' emotions and achieve comparable performance as the medical-grade physiological sensor-based state-of-the-art baseline method. © 2021 ACM.

Research Area(s)

  • Control area network (CAN), Driving behaviours, Emotion recognition, Intelligent vehicle, Traffic contexts

Citation Format(s)

The Empathetic Car: Exploring Emotion Inference via Driver Behaviour and Traffic Context. / Liu, Shu; Koch, Kevin; Zhou, Zimu et al.
In: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 5, No. 3, 117, 2021.

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review