Multimodal Emotion Recognition

39 papers with code • 3 benchmarks • 7 datasets

This is a leaderboard for multimodal emotion recognition on the IEMOCAP dataset. The modality abbreviations are A: Acoustic T: Text V: Visual

Please include the modality in the bracket after the model name.

All models must use standard five emotion categories and are evaluated in standard leave-one-session-out (LOSO). See the papers for references.


Use these libraries to find Multimodal Emotion Recognition models and implementations

Most implemented papers

Multimodal Speech Emotion Recognition and Ambiguity Resolution

Demfier/multimodal-speech-emotion-recognition 12 Apr 2019

In this work, we adopt a feature-engineering based approach to tackle the task of speech emotion recognition.

Multimodal Speech Emotion Recognition Using Audio and Text

david-yoon/multimodal-speech-emotion 10 Oct 2018

Speech emotion recognition is a challenging task, and extensive reliance has been placed on models that use audio features in building well-performing classifiers.

Complementary Fusion of Multi-Features and Multi-Modalities in Sentiment Analysis

robertjkeck2/EmoTe 17 Apr 2019

Therefore, in this paper, based on audio and text, we consider the task of multimodal sentiment analysis and propose a novel fusion strategy including both multi-feature fusion and multi-modality fusion to improve the accuracy of audio-text sentiment analysis.

End-to-End Multimodal Emotion Recognition using Deep Neural Networks

tzirakis/Multimodal-Emotion-Recognition 27 Apr 2017

The system is then trained in an end-to-end fashion where - by also taking advantage of the correlations of the each of the streams - we manage to significantly outperform the traditional approaches based on auditory and visual handcrafted features for the prediction of spontaneous and natural emotions on the RECOLA database of the AVEC 2016 research challenge on emotion recognition.

Context-Dependent Sentiment Analysis in User-Generated Videos

senticnet/sc-lstm ACL 2017

Multimodal sentiment analysis is a developing area of research, which involves the identification of sentiments in videos.

Multi-Modal Emotion recognition on IEMOCAP Dataset using Deep Learning

Samarth-Tripathi/IEMOCAP-Emotion-Detection 16 Apr 2018

Emotion recognition has become an important field of research in Human Computer Interactions as we improve upon the techniques for modelling the various aspects of behaviour.

DialogueRNN: An Attentive RNN for Emotion Detection in Conversations

SenticNet/conv-emotion 1 Nov 2018

Emotion detection in conversations is a necessary step for a number of applications, including opinion mining over chat history, social media threads, debates, argumentation mining, understanding consumer feedback in live conversations, etc.

Emotion Recognition in Audio and Video Using Deep Neural Networks

julieeF/CS231N-Project 15 Jun 2020

Humans are able to comprehend information from multiple domains for e. g. speech, text and visual.

Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling

SenticNet/hfusion 16 Jun 2018

Multimodal sentiment analysis is a very actively growing field of research.

Investigation of Multimodal Features, Classifiers and Fusion Methods for Emotion Recognition

zeroQiaoba/EmotiW2018 13 Sep 2018

We test our method in the EmotiW 2018 challenge and we gain promising results.