Multimodal Sentiment Analysis
Multimodal sentiment analysis is the task of performing sentiment analysis with multiple data sources - e.g. a camera feed of someone's face and their recorded speech.
( Image credit: ICON: Interactive Conversational Memory Network for Multimodal Emotion Detection )
Leaderboards
| TREND | DATASET | BEST METHOD | PAPER TITLE | PAPER | CODE | COMPARE |
|---|---|---|---|---|---|---|
|
||||||
|
Greatest papers with code
ICON: Interactive Conversational Memory Network for Multimodal Emotion Detection
EMNLP 2018
• SenticNet/conv-emotion
• 
Emotion recognition in conversations is crucial for building empathetic machines.
#3 best model for
Emotion Recognition in Conversation
on IEMOCAP
EMOTION RECOGNITION IN CONTEXT EMOTION RECOGNITION IN CONVERSATION MULTIMODAL EMOTION RECOGNITION MULTIMODAL SENTIMENT ANALYSIS
Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors
Humans convey their intentions through the usage of both verbal and nonverbal behaviors during face-to-face communication.
Multimodal Transformer for Unaligned Multimodal Language Sequences
ACL 2019
• yaohungt/Multimodal-Transformer
•
Human language is often multimodal, which comprehends a mixture of natural language, facial gestures, and acoustic behaviors.
#2 best model for
Multimodal Sentiment Analysis
on MOSI
Context-Dependent Sentiment Analysis in User-Generated Videos
ACL 2017
• soujanyaporia/multimodal-sentiment-analysis
• 
Multimodal sentiment analysis is a developing area of research, which involves the identification of sentiments in videos.
EMOTION RECOGNITION IN CONVERSATION MULTIMODAL EMOTION RECOGNITION MULTIMODAL SENTIMENT ANALYSIS NAMED ENTITY RECOGNITION SARCASM DETECTION
Multimodal Speech Emotion Recognition Using Audio and Text
• david-yoon/multimodal-speech-emotion
•
Speech emotion recognition is a challenging task, and extensive reliance has been placed on models that use audio features in building well-performing classifiers.
EMOTION CLASSIFICATION MULTIMODAL EMOTION RECOGNITION MULTIMODAL SENTIMENT ANALYSIS SPEECH EMOTION RECOGNITION
Efficient Low-rank Multimodal Fusion with Modality-Specific Factors
ACL 2018
• Justin1904/Low-rank-Multimodal-Fusion
•
Previous research in this field has exploited the expressiveness of tensors for multimodal representation.
Multi-attention Recurrent Network for Human Communication Comprehension
• pliang279/MFN
•
AI must understand each modality and the interactions between them that shape human communication.
#6 best model for
Multimodal Sentiment Analysis
on MOSI
Multimodal Sentiment Analysis with Word-Level Fusion and Reinforcement Learning
• pliang279/MFN
•
In this paper, we propose the Gated Multimodal Embedding LSTM with Temporal Attention (GME-LSTM(A)) model that is composed of 2 modules.
Multimodal Sentiment Analysis To Explore the Structure of Emotions
ICLR 2018
• anthonyhu/tumblr-emotions
•
We propose a novel approach to multimodal sentiment analysis using deep neural networks combining visual analysis and natural language processing.
Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling
Multimodal sentiment analysis is a very actively growing field of research.
MULTIMODAL EMOTION RECOGNITION MULTIMODAL SENTIMENT ANALYSIS