Publication: Spanish MEACorpus 2023: a multimodal speech–text corpus for emotion analysis in Spanish from natural environments
Authors
Pan, Ronghao ; García Díaz, José Antonio ; Rodríguez García, Miguel Ángel ; Valencia García, Rafael
item.page.secondaryauthor
Facultades de la UMU::Facultad de Informática
item.page.director
Publisher
Elsevier
publication.page.editor
publication.page.department
DOI
https://doi.org/10.1016/j.csi.2024.103856
item.page.type
info:eu-repo/semantics/article
Description
Abstract
In human–computer interaction, emotion recognition provides a deeper understanding of the user’s emotions, enabling empathetic and effective responses based on the user’s emotional state. While deep learning models have improved emotion recognition solutions, it is still an active area of research. One important limitation is that most emotion recognition systems use only text as input, ignoring features such as voice intonation. Another limitation is the limited number of datasets available for multimodal emotion recognition. In addition, most published datasets contain emotions that are simulated by professionals and produce limited results in real-world scenarios. In other languages, such as Spanish, hardly any datasets are available. Therefore, our contributions to emotion recognition are as follows. First, we compile and annotate a new corpus for multimodal emotion recognition in Spanish (Spanish MEACorpus 2023), which contains 13.16 h of speech divided into 5129 segments labeled by considering Ekman’s six basic emotions. The dataset is extracted from YouTube videos in natural environments. Second, we explore several deep learning models for emotion recognition using text- and audio-based features. Third, we evaluate different multimodal techniques to build a multimodal recognition system that improves the results of unimodal models, achieving a Macro F1-score of 87.745%, using late fusion with concatenation strategy approach.
publication.page.subject
Citation
Computer Standards & Interfaces, 2024, Vol. 90 : 103856
item.page.embargo
Collections
Ir a Estadísticas
Este ítem está sujeto a una licencia Creative Commons. http://creativecommons.org/licenses/by/4.0/