Search Results for author:
Found 106 papers, 72 papers with code
CMU-MOSEAS: A Multimodal Language Dataset for Spanish, Portuguese, German and French
It covers a diverse set topics and speakers, and carries supervision of 20 labels including sentiment (and subjectivity), emotions, and attributes.
Knowledge Enhanced Reflection Generation for Counseling Dialogues
We propose a pipeline that collects domain knowledge through web mining, and show that retrieval from both domain-specific and commonsense knowledge bases improves the quality of generated responses.
Causal Augmentation for Causal Sentence Classification
Scarcity of annotated causal texts leads to poor robustness when training state-of-the-art language models for causal sentence classification.
Contrastive Chain-of-Thought Prompting
Compared to the conventional chain of thought, our approach provides both valid and invalid reasoning demonstrations, to guide the model to reason step-by-step while reducing reasoning mistakes.
Mustango: Toward Controllable Text-to-Music Generation
With recent advancements in text-to-audio and text-to-music based on latent diffusion models, the quality of generated content has been reaching new heights.
Ranked #1 on
Text-to-Music Generation
on MusicBench
Video2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer model
These distinct features are then employed as guiding input to our music generation model.
Language Guided Visual Question Answering: Elevate Your Multimodal Language Model Using Knowledge-Enriched Prompts
Visual question answering (VQA) is the task of answering questions about an image.
Adapter Pruning using Tropical Characterization
Adapters are widely popular parameter-efficient transfer learning approaches in natural language processing that insert trainable modules in between layers of a pre-trained language model.
Language Model Unalignment: Parametric Red-Teaming to Expose Hidden Harms and Biases
On open-source models such as VICUNA-7B and LLAMA-2-CHAT 7B AND 13B, it shows an attack success rate of more than 91%.
MM-BigBench: Evaluating Multimodal Models on Multimodal Content Comprehension Tasks
Consequently, our work complements research on the performance of MLLMs in multimodal comprehension tasks, achieving a more comprehensive and holistic evaluation of MLLMs.
Large Language Models for Automated Open-domain Scientific Hypotheses Discovery
Raw web corpora that are necessary for developing hypotheses in the published papers are also collected in the dataset, with the final goal of creating a system that automatically generates valid, novel, and helpful (to human researchers) hypotheses, given only a pile of raw web corpora.
Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment
In this work, we propose a new safety evaluation benchmark RED-EVAL that carries out red-teaming.
Ranked #1 on
Text Generation
on HarmfulQA
SAS Video-QA: Self-Adaptive Sampling for Efficient Video Question-Answering
Video question--answering is a fundamental task in the field of video understanding.
Flacuna: Unleashing the Problem Solving Power of Vicuna using FLAN Fine-Tuning
Interestingly, despite being introduced four years ago, T5-based LLMs, such as FLAN-T5, continue to outperform the latest decoder-based LLMs, such as LLAMA and VICUNA, on tasks that require general problem-solving skills.
INSTRUCTEVAL: Towards Holistic Evaluation of Instruction-Tuned Large Language Models
Instruction-tuned large language models have revolutionized natural language processing and have shown great potential in applications such as conversational agents.
ADAPTERMIX: Exploring the Efficacy of Mixture of Adapters for Low-Resource TTS Adaptation
There are significant challenges for speaker adaptation in text-to-speech for languages that are not widely spoken or for speakers with accents or dialects that are not well-represented in the training data.
Domain-Expanded ASTE: Rethinking Generalization in Aspect Sentiment Triplet Extraction
Aspect Sentiment Triplet Extraction (ASTE) is a subtask of Aspect-Based Sentiment Analysis (ABSA) that considers each opinion term, their expressed sentiment, and the corresponding aspect targets.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+2
Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sources
Specifically, CoK consists of three stages: reasoning preparation, dynamic knowledge adapting, and answer consolidation.
Beyond Words: A Comprehensive Survey of Sentence Representations
In this paper, we provide an overview of the different methods for sentence representation learning, including both traditional and deep learning-based techniques.
Sentence Embedder Guided Utterance Encoder (SEGUE) for Spoken Language Understanding
The pre-trained speech encoder wav2vec 2. 0 performs very well on various spoken language understanding (SLU) tasks.
Uncertainty Guided Label Denoising for Document-level Distant Relation Extraction
Therefore, how to select effective pseudo labels to denoise DS data is still a challenge in document-level distant relation extraction.
ReMask: A Robust Information-Masking Approach for Domain Counterfactual Generation
Our experiments empirically show that the counterfactual samples sourced from our masked text lead to improved domain transfer on 10 out of 12 domain sentiment classification settings, with an average of 2% accuracy improvement over the state-of-the-art for unsupervised domain adaptation (UDA).
A Review of Deep Learning Techniques for Speech Processing
The power of deep learning techniques has opened up new avenues for research and innovation in the field of speech processing, with far-reaching implications for a range of industries and applications.
Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model
The immense scale of the recent large language models (LLM) allows many interesting properties, such as, instruction- and chain-of-thought-based fine-tuning, that has significantly improved zero- and few-shot performance in many natural language processing (NLP) tasks.
Ranked #3 on
Audio Generation
on AudioCaps
LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models
The success of large language models (LLMs), like GPT-4 and ChatGPT, has led to the development of numerous cost-effective and accessible alternatives that are created by finetuning open-access LLMs with task-specific data (e. g., ChatDoctor) or instruction data (e. g., Alpaca).
Evaluating Parameter-Efficient Transfer Learning Approaches on SURE Benchmark for Speech Understanding
To mitigate this issue, parameter-efficient transfer learning algorithms, such as adapters and prefix tuning, have been proposed as a way to introduce a few trainable parameters that can be plugged into large pre-trained language models such as BERT, and HuBERT.
UDApter -- Efficient Domain Adaptation Using Adapters
We even outperform unsupervised domain adaptation methods such as DANN and DSN in sentiment classification, and we are within 0. 85% F1 for natural language inference task, by fine-tuning only a fraction of the full model parameters.
A Dataset for Hyper-Relational Extraction and a Cube-Filling Approach
Hence, we propose CubeRE, a cube-filling model inspired by table-filling approaches and explicitly considers the interaction between relation triplets and qualifiers.
Ranked #2 on
Hyper-Relational Extraction
on HyperRED
Few-shot Multimodal Sentiment Analysis based on Multimodal Probabilistic Fusion Prompts
To tackle this problem, we propose a novel method called Multimodal Probabilistic Fusion Prompts (MultiPoint) that leverages diverse cues from different modalities for multimodal sentiment detection in the few-shot scenario.
Two is Better than Many? Binary Classification as an Effective Approach to Multi-Choice Question Answering
We show the efficacy of our proposed approach in different tasks -- abductive reasoning, commonsense question answering, science question answering, and sentence completion.
SAT: Improving Semi-Supervised Text Classification with Simple Instance-Adaptive Self-Training
Self-training methods have been explored in recent years and have exhibited great performance in improving semi-supervised learning.
MM-Align: Learning Optimal Transport-based Alignment Dynamics for Fast and Accurate Inference on Missing Modality Sequences
Existing multimodal tasks mostly target at the complete input modality setting, i. e., each modality is either complete or completely missing in both training and test sets.
Multiview Contextual Commonsense Inference: A New Dataset and Task
Our results show that the proposed pre-training objectives are effective at adapting the pre-trained T5-Large model for the contextual commonsense inference task.
Ranked #1 on
Multiview Contextual Commonsense Inference
on CICERO
(using extra training data)
WikiDes: A Wikipedia-Based Dataset for Generating Short Descriptions from Paragraphs
In this paper, we introduce WikiDes, a novel dataset to generate short descriptions of Wikipedia articles for the problem of text summarization.
DoubleMix: Simple Interpolation-Based Data Augmentation for Text Classification
This paper proposes a simple yet effective interpolation-based data augmentation approach termed DoubleMix, to improve the robustness of models in text classification.
SANCL: Multimodal Review Helpfulness Prediction with Selective Attention and Natural Contrastive Learning
With the boom of e-commerce, Multimodal Review Helpfulness Prediction (MRHP), which aims to sort product reviews according to the predicted helpfulness scores has become a research hotspot.
Analyzing Modality Robustness in Multimodal Sentiment Analysis
In this work, we hope to address that by (i) Proposing simple diagnostic checks for modality robustness in a trained multimodal model.
Vector-Quantized Input-Contextualized Soft Prompts for Natural Language Understanding
VIP particularly focuses on two aspects -- contextual prompts that learns input-specific contextualization of the soft prompt tokens through a small-scale sentence encoder and quantized prompts that maps the contextualized prompts to a set of learnable codebook vectors through a Vector quantization network.
So Different Yet So Alike! Constrained Unsupervised Text Style Transfer
Automatic transfer of text between domains has become popular in recent times.
CICERO: A Dataset for Contextualized Commonsense Inference in Dialogues
This paper addresses the problem of dialogue reasoning with contextualized commonsense inference.
Ranked #1 on
Answer Generation
on CICERO
RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extraction
We introduce the task setting of Zero-Shot Relation Triplet Extraction (ZeroRTE) to encourage further research in low-resource relation extraction methods.
Ranked #1 on
Zero-shot Relation Triplet Extraction
on Wiki-ZSL
KNOT: Knowledge Distillation using Optimal Transport for Solving NLP Tasks
We propose a new approach, Knowledge Distillation using Optimal Transport (KNOT), to distill the natural language semantic knowledge from multiple teacher networks to a student network.
Emotion Recognition in Conversation
Knowledge Distillation
+4
PIP: Physical Interaction Prediction via Mental Simulation with Span Selection
However, there is a lack of intuitive physics models that are tested on long physical interaction sequences with multiple interactions among different objects.
Ranked #1 on
Semantic Object Interaction Classification
on SPACE
STaCK: Sentence Ordering with Temporal Commonsense Knowledge
Sentence order prediction is the task of finding the correct order of sentences in a randomly ordered document.
Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis
In this work, we propose a framework named MultiModal InfoMax (MMIM), which hierarchically maximizes the Mutual Information (MI) in unimodal input pairs (inter-modality) and between multimodal fusion result and unimodal input in order to maintain task-related information through multimodal fusion.
Ranked #5 on
Multimodal Sentiment Analysis
on CMU-MOSI
Improving Distantly Supervised Relation Extraction with Self-Ensemble Noise Filtering
Distantly supervised models are very popular for relation extraction since we can obtain a large amount of training data using the distant supervision method without human annotation.
Aspect Sentiment Triplet Extraction Using Reinforcement Learning
We first focus on sentiments expressed in a sentence, then identify the target aspect and opinion terms for that sentiment.
Ranked #6 on
Aspect Sentiment Triplet Extraction
on ASTE-Data-V2
Aspect Sentiment Triplet Extraction
reinforcement-learning
+1
M2H2: A Multimodal Multiparty Hindi Dataset For Humor Recognition in Conversations
We propose several strong multimodal baselines and show the importance of contextual and multimodal information for humor recognition in conversations.
Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis
Multimodal sentiment analysis aims to extract and integrate semantic information collected from multiple modalities to recognize the expressed emotions and sentiment in multimodal data.
Exemplars-guided Empathetic Response Generation Controlled by the Elements of Human Communication
We empirically show that these approaches yield significant improvements in empathetic response quality in terms of both automated and human-evaluated metrics.
More Identifiable yet Equally Performant Transformers for Text Classification
In this work, we provide deeper theoretical analysis and empirical observations on the identifiability of attention weights.
CIDER: Commonsense Inference for Dialogue Explanation and Reasoning
Commonsense inference to understand and explain human language is a fundamental research problem in natural language processing.
Deep Neural Approaches to Relation Triplets Extraction: A Comprehensive Survey
Recently, with the advances made in continuous representation of words (word embeddings) and deep neural architectures, many research works are published in the area of relation extraction and it is very difficult to keep track of so many papers.
Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering
Open-domain Question Answering (OpenQA) is an important task in Natural Language Processing (NLP), which aims to answer a question in the form of natural language based on large-scale unstructured documents.
Machine Reading Comprehension
Open-Domain Question Answering
Recognizing Emotion Cause in Conversations
We address the problem of recognizing emotion cause in conversations, define two novel sub-tasks of this problem, and provide a corresponding dialogue-level dataset, along with strong Transformer-based baselines.
Ranked #1 on
Recognizing Emotion Cause in Conversations
on RECCON
Improving Zero Shot Learning Baselines with Commonsense Knowledge
Zero shot learning -- the problem of training and testing on a completely disjoint set of classes -- relies greatly on its ability to transfer knowledge from train classes to test classes.
Persuasive Dialogue Understanding: the Baselines and Negative Results
Persuasion aims at forming one's opinion and action via a series of persuasive messages containing persuader's strategies.
MTAG: Modal-Temporal Attention Graph for Unaligned Human Multimodal Language Sequences
Human communication is multimodal in nature; it is through multiple modalities such as language, voice, and facial expressions, that opinions and emotions are expressed.
Multimodal Research in Vision and Language: A Review of Current and Emerging Trends
Deep Learning and its applications have cascaded impactful research and development with a diverse range of modalities present in the real-world data.
COSMIC: COmmonSense knowledge for eMotion Identification in Conversations
In this paper, we address the task of utterance level emotion recognition in conversations using commonsense knowledge.
Ranked #11 on
Emotion Recognition in Conversation
on DailyDialog
MIME: MIMicking Emotions for Empathetic Response Generation
Current approaches to empathetic response generation view the set of emotions expressed in the input text as a flat structure, where all the emotions are treated uniformly.
Utterance-level Dialogue Understanding: An Empirical Study
Most of these approaches account for the context for effective understanding.
Investigating Gender Bias in BERT
As a result, predictions of downstream NLP models can vary noticeably by varying gender words, such as replacing "he" to "she", or even gender-neutral words.
Dialogue Relation Extraction with Document-level Heterogeneous Graph Attention Networks
This graph is fed to a graph attention network for context propagation among relevant nodes, which effectively captures the dialogue context.
Ranked #7 on
Dialog Relation Extraction
on DialogRE
(F1c (v1) metric)
Discriminative Dictionary Design for Action Classification in Still Images and Videos
Specifically, we pose the selection of potent local descriptors as filtering based feature selection problem which ranks the local features per category based on a novel measure of distinctiveness.
MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis
In this paper, we aim to learn effective modality representations to aid the process of fusion.
Improving Aspect-Level Sentiment Analysis with Aspect Extraction
Aspect-based sentiment analysis (ABSA), a popular research area in NLP has two distinct parts -- aspect extraction (AE) and labeling the aspects with sentiment polarity (ALSA).
KinGDOM: Knowledge-Guided DOMain adaptation for sentiment analysis
Cross-domain sentiment analysis has received significant attention in recent years, prompted by the need to combat the domain gap between different applications that make use of sentiment analysis.
Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research
Sentiment analysis as a field has come a long way since it was first introduced as a task nearly 20 years ago.
Factorized Multimodal Transformer for Multimodal Sequential Learning
As machine learning leaps towards better generalization to real world, multimodal sequential learning becomes a fundamental research area.
WildMix Dataset and Spectro-Temporal Transformer Model for Monoaural Audio Source Separation
To this end, we introduce a novel trasnformer-based model called Spectro-Temporal Transformer (STT).
Conversational Transfer Learning for Emotion Recognition
We propose an approach, TL-ERC, where we pre-train a hierarchical dialogue model on multi-turn conversations (source) and then transfer its parameters to a conversational emotion classifier (target).
Ranked #19 on
Emotion Recognition in Conversation
on DailyDialog
DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation
Emotion recognition in conversation (ERC) has received much attention, lately, from researchers due to its potential widespread applications in diverse areas, such as health-care, education, and human resources.
Ranked #1 on
Emotion Recognition in Conversation
on SEMAINE
Variational Fusion for Multimodal Sentiment Analysis
Multimodal fusion is considered a key step in multimodal tasks such as sentiment analysis, emotion detection, question answering, and others.
Towards Multimodal Sarcasm Detection (An \_Obviously\_ Perfect Paper)
As a first step towards enabling the development of multimodal approaches for sarcasm detection, we propose a new sarcasm dataset, Multimodal Sarcasm Detection Dataset (MUStARD), compiled from popular TV shows.
Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper)
As a first step towards enabling the development of multimodal approaches for sarcasm detection, we propose a new sarcasm dataset, Multimodal Sarcasm Detection Dataset (MUStARD), compiled from popular TV shows.
Multi-task Learning for Multi-modal Emotion Recognition and Sentiment Analysis
In this paper, we present a deep multi-task learning framework that jointly performs sentiment and emotion analysis both.
Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances
Emotion is intrinsic to humans and consequently emotion understanding is a key part of human-like artificial intelligence (AI).
Ranked #6 on
Emotion Recognition in Conversation
on EC
Sentiment and Sarcasm Classification with Multitask Learning
We argue that knowledge in sarcasm detection can also be beneficial to sentiment classification and vice versa.
Phonetic-enriched Text Representation for Chinese Sentiment Analysis with Reinforcement Learning
Furthermore, we also fuse phonetic features with textual and visual features in order to mimic the way humans read and understand Chinese text.
DialogueRNN: An Attentive RNN for Emotion Detection in Conversations
Emotion detection in conversations is a necessary step for a number of applications, including opinion mining over chat history, social media threads, debates, argumentation mining, understanding consumer feedback in live conversations, etc.
Ranked #3 on
Emotion Recognition in Conversation
on SEMAINE
Emotion Classification
Emotion Recognition in Conversation
+2
MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations
We propose several strong multimodal baselines and show the importance of contextual and multimodal information for emotion recognition in conversations.
ICON: Interactive Conversational Memory Network for Multimodal Emotion Detection
Emotion recognition in conversations is crucial for building empathetic machines.
Ranked #42 on
Emotion Recognition in Conversation
on IEMOCAP
Emotion Recognition in Conversation
General Classification
+2
Contextual Inter-modal Attention for Multi-modal Sentiment Analysis
We evaluate our proposed approach on two multi-modal sentiment analysis benchmark datasets, viz.
Ranked #6 on
Multimodal Sentiment Analysis
on MOSI
IARM: Inter-Aspect Relation Modeling with Memory Networks in Aspect-Based Sentiment Analysis
Sentiment analysis has immense implications in e-commerce through user feedback mining.
Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph
Analyzing human multimodal language is an emerging area of research in NLP.
Ranked #9 on
Multimodal Sentiment Analysis
on CMU-MOSEI
(using extra training data)
Conversational Memory Network for Emotion Recognition in Dyadic Dialogue Videos
Emotion recognition in conversations is crucial for the development of empathetic machines.
Ranked #44 on
Emotion Recognition in Conversation
on IEMOCAP
Modeling Inter-Aspect Dependencies for Aspect-Based Sentiment Analysis
Aspect-based Sentiment Analysis is a fine-grained task of sentiment classification for multiple aspects in a sentence.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+3
Anaphora and Coreference Resolution: A Review
Entity resolution aims at resolving repeated references to an entity in a document and forms a core component of natural language processing (NLP) research.
CASCADE: Contextual Sarcasm Detection in Online Discussion Forums
The literature in automated sarcasm detection has mainly focused on lexical, syntactic and semantic-level analysis of text.
Ranked #1 on
Sarcasm Detection
on SARC (all-bal)
Multimodal Sentiment Analysis: Addressing Key Issues and Setting up the Baselines
We compile baselines, along with dataset split, for multimodal sentiment analysis.
A Deep Learning Approach for Multimodal Deception Detection
Automatic deception detection is an important task that has gained momentum in computational linguistics due to its potential applications.
Multi-attention Recurrent Network for Human Communication Comprehension
AI must understand each modality and the interactions between them that shape human communication.
Ranked #9 on
Multimodal Sentiment Analysis
on MOSI
Memory Fusion Network for Multi-view Sequential Learning
In this paper, we present a new neural architecture for multi-view sequential learning called the Memory Fusion Network (MFN) that explicitly accounts for both interactions in a neural architecture and continuously models them through time.
Basic tasks of sentiment analysis
Subjectivity detection is the task of identifying objective and subjective sentences.
Recent Trends in Deep Learning Based Natural Language Processing
Deep learning methods employ multiple processing layers to learn hierarchical representations of data and have produced state-of-the-art results in many domains.
Benchmarking Multimodal Sentiment Analysis
We propose a framework for multimodal sentiment analysis and emotion recognition using convolutional neural network-based feature extraction from text and visual modalities.
Tensor Fusion Network for Multimodal Sentiment Analysis
Multimodal sentiment analysis is an increasingly popular research area, which extends the conventional language-based definition of sentiment analysis to a multimodal setup where other relevant modalities accompany language.
Developing a concept-level knowledge base for sentiment analysis in Singlish
In this paper, we present Singlish sentiment lexicon, a concept-level knowledge base for sentiment analysis that associates multiword expressions to a set of emotion labels and a polarity value.
Context-Dependent Sentiment Analysis in User-Generated Videos
Multimodal sentiment analysis is a developing area of research, which involves the identification of sentiments in videos.
Ranked #3 on
Emotion Recognition in Conversation
on CPED
Emotion Recognition in Conversation
General Classification
+4
SenticNet 4: A Semantic Resource for Sentiment Analysis Based on Conceptual Primitives
An important difference between traditional AI systems and human intelligence is the human ability to harness commonsense knowledge gleaned from a lifetime of learning and experience to make informed decisions.
A Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural Networks
Sarcasm detection is a key task for many natural language processing tasks.
