Multimodal Sentiment Analysis
Multimodal sentiment analysis is the task of performing sentiment analysis with multiple data sources - e.g. a camera feed of someone's face and their recorded speech.
State-of-the-art leaderboards
| Trend | Dataset | Best Method | Paper title | Paper | Code | Compare |
|---|---|---|---|---|---|---|
|
Greatest papers with code
ICON: Interactive Conversational Memory Network for Multimodal Emotion Detection
EMNLP 2018
• SenticNet/conv-emotion
• 
Emotion recognition in conversations is crucial for building empathetic machines.
#2 best model for
Emotion Recognition in Conversation
on IEMOCAP
EMOTION RECOGNITION IN CONTEXT EMOTION RECOGNITION IN CONVERSATION MULTIMODAL EMOTION RECOGNITION MULTIMODAL SENTIMENT ANALYSIS
Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors
Humans convey their intentions through the usage of both verbal and nonverbal behaviors during face-to-face communication.
Context-Dependent Sentiment Analysis in User-Generated Videos
ACL 2017 • soujanyaporia/multimodal-sentiment-analysis
Multimodal sentiment analysis is a developing area of research, which involves the identification of sentiments in videos.
MULTIMODAL EMOTION RECOGNITION MULTIMODAL SENTIMENT ANALYSIS NAMED ENTITY RECOGNITION SARCASM DETECTION
Multimodal Speech Emotion Recognition Using Audio and Text
• david-yoon/multimodal-speech-emotion
•
Speech emotion recognition is a challenging task, and extensive reliance has been placed on models that use audio features in building well-performing classifiers.
EMOTION CLASSIFICATION MULTIMODAL EMOTION RECOGNITION MULTIMODAL SENTIMENT ANALYSIS SPEECH EMOTION RECOGNITION
Efficient Low-rank Multimodal Fusion with Modality-Specific Factors
ACL 2018
• Justin1904/Low-rank-Multimodal-Fusion
• 
Previous research in this field has exploited the expressiveness of tensors for multimodal representation.
Multimodal Sentiment Analysis with Word-Level Fusion and Reinforcement Learning
• pliang279/MFN
•
In this paper, we propose the Gated Multimodal Embedding LSTM with Temporal Attention (GME-LSTM(A)) model that is composed of 2 modules.
Multi-attention Recurrent Network for Human Communication Comprehension
• pliang279/MFN
•
AI must understand each modality and the interactions between them that shape human communication.
#3 best model for
Multimodal Sentiment Analysis
on MOSI
Multimodal Sentiment Analysis To Explore the Structure of Emotions
ICLR 2018
• anthonyhu/tumblr-emotions
•
We propose a novel approach to multimodal sentiment analysis using deep neural networks combining visual analysis and natural language processing.
Select-Additive Learning: Improving Generalization in Multimodal Sentiment Analysis
• HaohanWang/SelectAdditiveLearning
In this paper, we propose a Select-Additive Learning (SAL) procedure that improves the generalizability of trained neural networks for multimodal sentiment analysis.
Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities
• hainow/MCTN
•
Our method is based on the key insight that translation from a source to a target modality provides a method of learning joint representations using only the source modality as input.