IMR Press / JIN / Volume 23 / Issue 1 / DOI: 10.31083/j.jin2301018
Open Access Original Research
Fusion of Multi-domain EEG Signatures Improves Emotion Recognition
Xiaomin Wang1,2,†Yu Pei2,3,†Zhiguo Luo2,3Shaokai Zhao2,3Liang Xie2,3,*Ye Yan1,2,3Erwei Yin2,3Shuang Liu1Dong Ming1
Show Less
1 Academy of Medical Engineering and Translational Medicine, Tianjin University, 300072 Tianjin, China
2 Tianjin Artificial Intelligence Innovation Center (TAIIC), 300450 Tianjin, China
3 Defense Innovation Institute, Academy of Military Sciences (AMS), 100071 Beijing, China
*Correspondence: xielnudt@gmail.com (Liang Xie)
These authors contributed equally.
J. Integr. Neurosci. 2024, 23(1), 18; https://doi.org/10.31083/j.jin2301018
Submitted: 7 April 2023 | Revised: 19 May 2023 | Accepted: 22 May 2023 | Published: 19 January 2024
Copyright: © 2024 The Author(s). Published by IMR Press.
This is an open access article under the CC BY 4.0 license.
Abstract

Background: Affective computing has gained increasing attention in the area of the human-computer interface where electroencephalography (EEG)-based emotion recognition occupies an important position. Nevertheless, the diversity of emotions and the complexity of EEG signals result in unexplored relationships between emotion and multichannel EEG signal frequency, as well as spatial and temporal information. Methods: Audio-video stimulus materials were used that elicited four types of emotions (sad, fearful, happy, neutral) in 32 male and female subjects (age 21–42 years) while collecting EEG signals. We developed a multidimensional analysis framework using a fusion of phase-locking value (PLV), microstates, and power spectral densities (PSDs) of EEG features to improve emotion recognition. Results: An increasing trend of PSDs was observed as emotional valence increased, and connections in the prefrontal, temporal, and occipital lobes in high-frequency bands showed more differentiation between emotions. Transition probability between microstates was likely related to emotional valence. The average cross-subject classification accuracy of features fused by Discriminant Correlation Analysis achieved 64.69%, higher than that of single mode and direct-concatenated features, with an increase of more than 7%. Conclusions: Different types of EEG features have complementary properties in emotion recognition, and combining EEG data from three types of features in a correlated way, improves the performance of emotion classification.

Keywords
electroencephalography (EEG)
emotion
classification
power spectral density (PSD)
microstate
phase-locking value (PLV)
Funding
62076250/National Natural Science Foundation of China
61901505/National Natural Science Foundation of China
61703407/National Natural Science Foundation of China
Figures
Fig. 1.
Share
Back to top