Emotion Recognition Based on Meta Bi-Modal Learning Model

Most emotion recognition models were based on single modal speech, which was prone to ambiguity and ignores the relationship between continuous and discrete emotions. To address these issues, a meta bi-modal learning (MBL) model is proposed, which realizes single modal continuous emotion, namely valence, activation and control, to assist in the recognition of bi-modal discrete emotion. The feature fusion method adopts the cross-modal attention mechanism, effectively solving the problem of aligning modal sequence data. At the same time, during the auxiliary task training process, the sharing of hard parameters in multi-task learning achieved three-dimensional information exchange of validity, activation and control. Using each speaker蒺s sentence as a small sample to adapt the learning model to different speakers and enhance their generalization ability. The experimental results show that on the script and dialogue datasets of the interactive emotional dyadic motion capture database, the emotion recognition rates using the MBL model are 71. 24% and 69. 12%, respectively, demonstrating good performance. © 2023 Beijing University of Posts and Telecommunications. All rights reserved.