Search Results for author:
Found 36 papers, 26 papers with code
The Jumping Reasoning Curve? Tracking the Evolution of Reasoning Performance in GPT-[n] and o-[n] Models on Multimodal Puzzles
To this end, we track the evolution of the GPT-[n] and o-[n] series models on challenging multimodal puzzles, requiring fine-grained visual perception with abstract or algorithmic reasoning.
Emma-X: An Embodied Multimodal Action Model with Grounded Chain of Thought and Look-ahead Spatial Reasoning
Traditional reinforcement learning-based robotic control methods are often task-specific and fail to generalize across diverse environments or unseen objects and instructions.
MixEval-X: Any-to-Any Evaluations from Real-World Data Mixtures
Perceiving and generating diverse modalities are crucial for AI models to effectively learn from and engage with real-world signals, necessitating reliable evaluations for their development.
Improving Text-To-Audio Models with Synthetic Captions
It is an open challenge to obtain high quality training data, especially captions, for text-to-audio models.
Ranked #5 on
Audio Generation
on AudioCaps
(CLAP_LAION metric)
Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization
These models do not explicitly focus on the presence of concepts or events and their temporal ordering in the output audio with respect to the input prompt.
PuzzleVQA: Diagnosing Multimodal Reasoning Challenges of Language Models with Abstract Visual Patterns
To diagnose the reasoning challenges in large multimodal models, we progressively guide the models with our ground truth reasoning explanations for visual perception, inductive reasoning, and deductive reasoning.
Are Language Models Puzzle Prodigies? Algorithmic Puzzles Unveil Serious Challenges in Multimodal Reasoning
We present a new dataset, AlgoPuzzleVQA designed to challenge and evaluate the capabilities of multimodal language models in solving algorithmic puzzles that necessitate both visual understanding, language understanding, and complex algorithmic reasoning.
Ranked #1 on
Multimodal Reasoning
on AlgoPuzzleVQA
Evaluating LLMs' Mathematical and Coding Competency through Ontology-guided Interventions
Recent advancements in Large Language Models (LLMs) have showcased striking results on existing logical reasoning benchmarks, with some models even surpassing human performance.
Mustango: Toward Controllable Text-to-Music Generation
Through extensive experiments, we show that the quality of the music generated by Mustango is state-of-the-art, and the controllability through music-specific text prompts greatly outperforms other models such as MusicGen and AudioLDM2.
Ranked #1 on
Text-to-Music Generation
on MusicBench
Language Guided Visual Question Answering: Elevate Your Multimodal Language Model Using Knowledge-Enriched Prompts
Visual question answering (VQA) is the task of answering questions about an image.
Flacuna: Unleashing the Problem Solving Power of Vicuna using FLAN Fine-Tuning
Interestingly, despite being introduced four years ago, T5-based LLMs, such as FLAN-T5, continue to outperform the latest decoder-based LLMs, such as LLAMA and VICUNA, on tasks that require general problem-solving skills.
ReTAG: Reasoning Aware Table to Analytic Text Generation
The task of table summarization involves generating text that both succinctly and accurately represents the table or a specific set of highlighted cells within a table.
Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model
The immense scale of the recent large language models (LLM) allows many interesting properties, such as, instruction- and chain-of-thought-based fine-tuning, that has significantly improved zero- and few-shot performance in many natural language processing (NLP) tasks.
Ranked #5 on
Audio Generation
on AudioCaps
(FAD metric)
Two is Better than Many? Binary Classification as an Effective Approach to Multi-Choice Question Answering
We show the efficacy of our proposed approach in different tasks -- abductive reasoning, commonsense question answering, science question answering, and sentence completion.
Ranked #2 on
Sentence Completion
on HellaSwag
Multiview Contextual Commonsense Inference: A New Dataset and Task
Our results show that the proposed pre-training objectives are effective at adapting the pre-trained T5-Large model for the contextual commonsense inference task.
Ranked #1 on
Multiview Contextual Commonsense Inference
on CICERO
(using extra training data)
Generating Intermediate Steps for NLI with Next-Step Supervision
The Natural Language Inference (NLI) task often requires reasoning over multiple steps to reach the conclusion.
CICERO: A Dataset for Contextualized Commonsense Inference in Dialogues
This paper addresses the problem of dialogue reasoning with contextualized commonsense inference.
Ranked #1 on
Answer Generation
on CICERO
STaCK: Sentence Ordering with Temporal Commonsense Knowledge
Sentence order prediction is the task of finding the correct order of sentences in a randomly ordered document.
Exemplars-guided Empathetic Response Generation Controlled by the Elements of Human Communication
We empirically show that these approaches yield significant improvements in empathetic response quality in terms of both automated and human-evaluated metrics.
CIDER: Commonsense Inference for Dialogue Explanation and Reasoning
Commonsense inference to understand and explain human language is a fundamental research problem in natural language processing.
Recognizing Emotion Cause in Conversations
We address the problem of recognizing emotion cause in conversations, define two novel sub-tasks of this problem, and provide a corresponding dialogue-level dataset, along with strong Transformer-based baselines.
Ranked #1 on
Recognizing Emotion Cause in Conversations
on RECCON
Improving Zero Shot Learning Baselines with Commonsense Knowledge
Zero shot learning -- the problem of training and testing on a completely disjoint set of classes -- relies greatly on its ability to transfer knowledge from train classes to test classes.
Persuasive Dialogue Understanding: the Baselines and Negative Results
Persuasion aims at forming one's opinion and action via a series of persuasive messages containing persuader's strategies.
COSMIC: COmmonSense knowledge for eMotion Identification in Conversations
In this paper, we address the task of utterance level emotion recognition in conversations using commonsense knowledge.
Ranked #12 on
Emotion Recognition in Conversation
on DailyDialog
MIME: MIMicking Emotions for Empathetic Response Generation
Current approaches to empathetic response generation view the set of emotions expressed in the input text as a flat structure, where all the emotions are treated uniformly.
Utterance-level Dialogue Understanding: An Empirical Study
Most of these approaches account for the context for effective understanding.
Visual Interest Prediction with Attentive Multi-Task Transfer Learning
Visual interest & affect prediction is a very interesting area of research in the area of computer vision.
KinGDOM: Knowledge-Guided DOMain adaptation for sentiment analysis
Cross-domain sentiment analysis has received significant attention in recent years, prompted by the need to combat the domain gap between different applications that make use of sentiment analysis.
DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation
Emotion recognition in conversation (ERC) has received much attention, lately, from researchers due to its potential widespread applications in diverse areas, such as health-care, education, and human resources.
Ranked #1 on
Emotion Recognition in Conversation
on SEMAINE
Emotion Classification
Emotion Recognition in Conversation
+1
Multi-task Learning for Multi-modal Emotion Recognition and Sentiment Analysis
In this paper, we present a deep multi-task learning framework that jointly performs sentiment and emotion analysis both.
Contextual Inter-modal Attention for Multi-modal Sentiment Analysis
We evaluate our proposed approach on two multi-modal sentiment analysis benchmark datasets, viz.
Ranked #7 on
Multimodal Sentiment Analysis
on MOSI
A Multi-task Ensemble Framework for Emotion, Sentiment and Intensity Prediction
In this paper, through multi-task ensemble framework we address three problems of emotion and sentiment analysis i. e. "emotion classification & intensity", "valence, arousal & dominance for emotion" and "valence & arousal} for sentiment".
A Multilayer Perceptron based Ensemble Technique for Fine-grained Financial Sentiment Analysis
In this paper, we propose a novel method for combining deep learning and classical feature based models using a Multi-Layer Perceptron (MLP) network for financial sentiment analysis.
IITP at SemEval-2017 Task 5: An Ensemble of Deep Learning and Feature Based Models for Financial Sentiment Analysis
In this paper we propose an ensemble based model which combines state of the art deep learning sentiment analysis algorithms like Convolution Neural Network (CNN) and Long Short Term Memory (LSTM) along with feature based models to identify optimistic or pessimistic sentiments associated with companies and stocks in financial texts.
