XAI in affective computing: a preliminary study

Sajno E., Rossi A., De Gaspari S., Sansoni M., Brizzi G., Riva G.

Machine Learning  ECG  HRV  Emotion recognition  Explainable AI (XAI) 

Affective computing is a rapidly growing field that aims to understand human emotions through Artificial Intelligence. One of the most promising ways to achieve this goal is the use of physiological data (e.g. electrocardiogram - ECG) and Machine Learning (ML) algorithms to classify affective states. ECG correlates, such as Heart Rate Variability (HRV) and its features, are reported as viable indicators in both dimensional approaches, especially for valence, and in detecting discrete emotions. In this preliminary study, we used the ECG data from the open-source HCI Tagging Database, which includes physiological data and self-referred feedback from 30 subjects who watched videos designed to elicit different emotions. The subjects evaluated their reactions using a three-dimensional affective space defined by arousal, valence, and dominance levels and reported the emotions they felt. To classify the affective states, we trained and tested different classification algorithms on the HRV features, using as labels, each self-reported feedback (i.e., valence, arousal, dominance, and emotions). The results showed that HRV features, when combined with normalization methods and ML algorithms, were effective in recognizing emotions as experienced by individuals. In particular, the study showed that Decision Tree was the best-performing algorithm for predicting emotions based on HRV data. Additionally, an Explainable AI (XAI) model provided insights into the weight of these features in the ML discrimination phases. Overall, the study highlights the potential of HRV as a valid and unobtrusive source for detecting emotional states.

Source: Annual review of cybertherapy and telemedicine 21 (2023): 40–46.

Publisher: IOS Press, Amsterdam , Paesi Bassi