Multimodal Emotion Recognition
52 papers with code • 3 benchmarks • 8 datasets
This is a leaderboard for multimodal emotion recognition on the IEMOCAP dataset. The modality abbreviations are A: Acoustic T: Text V: Visual
Please include the modality in the bracket after the model name.
All models must use standard five emotion categories and are evaluated in standard leave-one-session-out (LOSO). See the papers for references.
Benchmarks
These leaderboards are used to track progress in Multimodal Emotion Recognition
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
Libraries
Use these libraries to find Multimodal Emotion Recognition models and implementations
IEMOCAP
MELD
EMOTIC
CPED
DEAP
RESD
Latest papers with no code
MMA-DFER: MultiModal Adaptation of unimodal models for Dynamic Facial Expression Recognition in-the-wild
Within the field of multimodal DFER, recent methods have focused on exploiting advances of self-supervised learning (SSL) for pre-training of strong multimodal encoders.
Multimodal Emotion Recognition by Fusing Video Semantic in MOOC Learning Scenarios
The method proposed in this paper not only contributes to a deeper understanding of the impact of instructional videos on learners' emotional states but also provides a beneficial reference for future research on emotion recognition in MOOC learning scenarios.
UniMEEC: Towards Unified Multimodal Emotion Recognition and Emotion Cause
In this paper, we propose a Unified Multimodal Emotion recognition and Emotion-Cause analysis framework (UniMEEC) to explore the causality and complementarity between emotion and emotion cause.
Multi-Modal Emotion Recognition by Text, Speech and Video Using Pretrained Transformers
Due to the complex nature of human emotions and the diversity of emotion representation methods in humans, emotion recognition is a challenging field.
A Two-Stage Multimodal Emotion Recognition Model Based on Graph Contrastive Learning
To address the above issues, we propose a two-stage emotion recognition model based on graph contrastive learning (TS-GCL).
Adversarial Representation with Intra-Modal and Inter-Modal Graph Contrastive Learning for Multimodal Emotion Recognition
However, the existing feature fusion methods have usually mapped the features of different modalities into the same feature space for information fusion, which can not eliminate the heterogeneity between different modalities.
Modality-Collaborative Transformer with Hybrid Feature Reconstruction for Robust Emotion Recognition
As a vital aspect of affective computing, Multimodal Emotion Recognition has been an active research area in the multimedia community.
DER-GCN: Dialogue and Event Relation-Aware Graph Convolutional Neural Network for Multimodal Dialogue Emotion Recognition
Specifically, we construct a weighted multi-relationship graph to simultaneously capture the dependencies between speakers and event relations in a dialogue.
Deep Imbalanced Learning for Multimodal Emotion Recognition in Conversations
The main task of Multimodal Emotion Recognition in Conversations (MERC) is to identify the emotions in modalities, e. g., text, audio, image and video, which is a significant development direction for realizing machine intelligence.
Joyful: Joint Modality Fusion and Graph Contrastive Learning for Multimodal Emotion Recognition
In this paper, we propose a method for joint modality fusion and graph contrastive learning for multimodal emotion recognition (Joyful), where multimodality fusion, contrastive learning, and emotion recognition are jointly optimized.
