Search Results for author:
Found 270 papers, 146 papers with code
Counterfactual Off-Policy Training for Neural Dialogue Generation
Open-domain dialogue generation suffers from the data insufficiency problem due to the vast size of potential responses.
Counterfactual Vision-and-Language Navigation via Adversarial Path Sampler
In particular, we present a model-agnostic adversarial path sampler (APS) that learns to sample challenging paths that force the navigator to improve based on the navigation performance.
Learning to Prioritize: Precision-Driven Sentence Filtering for Long Text Summarization
Neural text summarization has shown great potential in recent years.
BlobGEN-Vid: Compositional Text-to-Video Generation with Blob Video Representations
In addition, we introduce a learnable module to interpolate text embeddings so that users can control semantics in specific frames and obtain smooth object transitions.
Aristotle: Mastering Logical Reasoning with A Logic-Complete Decompose-Search-Resolve Framework
In the context of large language models (LLMs), current advanced reasoning methods have made impressive strides in various reasoning tasks.
AntiLeak-Bench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledge
Data contamination hinders fair LLM evaluation by introducing test data into newer models' training sets.
Combating Multimodal LLM Hallucination via Bottom-Up Holistic Reasoning
Our framework systematically addresses potential issues in both visual and textual inputs by verifying and integrating perception-level information with cognition-level commonsense knowledge, ensuring more reliable outputs.
RuleArena: A Benchmark for Rule-Guided Reasoning with LLMs in Real-World Scenarios
This paper introduces RuleArena, a novel and challenging benchmark designed to evaluate the ability of large language models (LLMs) to follow complex, real-world rules in reasoning.
Embracing AI in Education: Understanding the Surge in Large Language Model Use by Secondary Students
The impressive essay writing and problem-solving capabilities of large language models (LLMs) like OpenAI's ChatGPT have opened up new avenues in education.
Disentangling Memory and Reasoning Ability in Large Language Models
Large Language Models (LLMs) have demonstrated strong performance in handling complex tasks requiring both extensive knowledge and reasoning abilities.
Scaling LLM Inference with Optimized Sample Compute Allocation
To scale up inference efficiently with a limited compute, it is crucial to find an optimal allocation for sample compute budgets: Which sampling configurations (model, temperature, language, etc.)
CBT-Bench: Evaluating Large Language Models on Assisting Cognitive Behavior Therapy
There is a significant gap between patient needs and available mental health support today.
Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Sampling
To address these limitations, we introduce Speculative Knowledge Distillation (SKD), a novel approach that leverages cooperation between student and teacher models to generate high-quality training data on-the-fly while aligning with the student's inference-time distribution.
COrAL: Order-Agnostic Language Modeling for Efficient Iterative Refinement
Our approach models multiple token dependencies within manageable context windows, enabling the model to perform iterative refinement internally during the generation process.
Detecting Training Data of Large Language Models via Expectation Maximization
In this paper, we introduce EM-MIA, a novel MIA method for LLMs that iteratively refines membership scores and prefix scores via an expectation-maximization algorithm, leveraging the duality that the estimates of these scores can be improved by each other.
Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Models
To support this investigation, we introduce ECHOQA, a benchmark spanning scientific, factual, and commonsense knowledge.
Uncovering Factor Level Preferences to Improve Human-Model Alignment
Our factor level analysis reveals a substantial discrepancy between human and LLM preferences in generation tasks, whereas LLMs show strong alignment with human preferences in evaluation tasks.
T2V-Turbo-v2: Enhancing Video Generation Model Post-Training through Data, Reward, and Conditional Guidance Design
In this paper, we focus on enhancing a diffusion-based text-to-video (T2V) model during the post-training phase by distilling a highly capable consistency model from a pretrained T2V model.
Gödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement
The rapid advancement of large language models (LLMs) has significantly enhanced the capabilities of AI-driven agents across various tasks.
Human Bias in the Face of AI: The Role of Human Judgement in AI Generated Text Evaluation
As AI advances in text generation, human trust in AI generated content remains constrained by biases that go beyond concerns of accuracy.
A Gradient Analysis Framework for Rewarding Good and Penalizing Bad Examples in Language Models
Beyond maximum likelihood estimation (MLE), the standard objective of a language model (LM) that optimizes good examples probabilities, many studies have explored ways that also penalize bad examples for enhancing the quality of output distribution, including unlikelihood training, exponential maximizing average treatment effect (ExMATE), and direct preference optimization (DPO).
Can Editing LLMs Inject Harm?
Then, we find that editing attacks can inject both types of misinformation into LLMs, and the effectiveness is particularly high for commonsense misinformation injection.
Benchmarks as Microscopes: A Call for Model Metrology
Modern language models (LMs) pose a new challenge in capability assessment.
Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data
Furthermore, while model performance improves across all tasks as LLM size increases, only factual question answering shows an increase in memorization, whereas machine translation and reasoning tasks exhibit greater generalization, producing more novel outputs.
RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answering
Question answering based on retrieval augmented generation (RAG-QA) is an important research topic in NLP and has a wide range of real-world applications.
DebUnc: Mitigating Hallucinations in Large Language Model Agent Communication with Uncertainty Estimations
To enhance Large Language Model (LLM) capabilities, multi-agent debates have been introduced, where multiple LLMs discuss solutions to a problem over several rounds of debate.
MMSci: A Dataset for Graduate-Level Multi-Discipline Multimodal Scientific Understanding
The results highlight the high difficulty of these tasks and the significant performance gap among models.
VSP: Assessing the dual challenges of perception and reasoning in spatial planning tasks for VLMs
One understudied capability in VLMs is visual spatial planning -- the ability to comprehend the spatial arrangements of objects and devise action plans to achieve desired outcomes in visual scenes.
Losing Visual Needles in Image Haystacks: Vision Language Models are Easily Distracted in Short and Long Contexts
We present LoCoVQA, a dynamic benchmark generator for evaluating long-context extractive reasoning in vision language models (VLMs).
Investigating the Transferability of Code Repair for Low-Resource Programming Languages
To explain this behavior, we perform a further analysis and find that contrary to preexisting beliefs, the correlation between reasoning ability and code correction ability is weak.
Global Human-guided Counterfactual Explanations for Molecular Properties via Reinforcement Learning
RLHEX provides a flexible framework to incorporate different human-designed principles into the counterfactual explanation generation process, aligning these explanations with domain expertise.
BPO: Staying Close to the Behavior LLM Creates Better Online LLM Alignment
Direct alignment from preferences (DAP) has emerged as a promising paradigm for aligning large language models (LLMs) to human desiderata from pre-collected, offline preference datasets.
WildVision: Evaluating Vision-Language Models in the Wild with Human Preferences
Recent breakthroughs in vision-language models (VLMs) emphasize the necessity of benchmarking human preferences in real-world multimodal interactions.
MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos
Multimodal Language Language Models (MLLMs) demonstrate the emerging abilities of "world models" -- interpreting and reasoning about complex real-world dynamics.
TC-Bench: Benchmarking Temporal Compositionality in Text-to-Video and Image-to-Video Generation
Video generation has many unique challenges beyond those of image generation.
Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense?
We present a novel task and benchmark for evaluating the ability of text-to-image(T2I) generation models to produce images that align with commonsense in real life, which we call Commonsense-T2I.
Unveiling the Impact of Coding Data Instruction Fine-Tuning on Large Language Models Reasoning
First, coding data tuning enhances the overall reasoning capabilities of LLMs across different model families and scales.
T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedback
In this work, we aim to break the quality bottleneck of a video consistency model (VCM) to achieve $\textbf{both fast and high-quality video generation}$.
FASTopic: Pretrained Transformer is a Fast, Adaptive, Stable, and Transferable Topic Model
We further propose a novel Embedding Transport Plan (ETP) method.
From Text to Pixel: Advancing Long-Context Understanding in MLLMs
The rapid progress in Multimodal Large Language Models (MLLMs) has significantly advanced their ability to process and understand complex visual and textual information.
A Survey on Large Language Models for Critical Societal Domains: Finance, Healthcare, and Law
In the fast-evolving domain of artificial intelligence, large language models (LLMs) such as GPT-3 and GPT-4 are revolutionizing the landscapes of finance, healthcare, and law: domains characterized by their reliance on professional expertise, challenging data acquisition, high-stakes, and stringent regulatory compliance.
Automatic Layout Planning for Visually-Rich Documents with Instruction-Following Models
Recent advancements in instruction-following models have made user interactions with models more user-friendly and efficient, broadening their applicability.
Ferret-v2: An Improved Baseline for Referring and Grounding with Large Language Models
While Ferret seamlessly integrates regional understanding into the Large Language Model (LLM) to facilitate its referring and grounding capability, it poses certain limitations: constrained by the pre-trained fixed visual encoder and failed to perform well on broader tasks.
Ranked #150 on
Visual Question Answering
on MM-Vet
Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2)
With advances in the quality of text-to-image (T2I) models has come interest in benchmarking their prompt faithfulness -- the semantic coherence of generated images to the prompts they were conditioned on.
Towards Safety and Helpfulness Balanced Responses via Controllable Large Language Models
As large language models (LLMs) become easily accessible nowadays, the trade-off between safety and helpfulness can significantly impact user experience.
Lost in Translation? Translation Errors and Challenges for Fair Assessment of Text-to-Image Models on Multilingual Concepts
Benchmarks of the multilingual capabilities of text-to-image (T2I) models compare generated images prompted in a test language to an expected image distribution over a concept set.
Reward Guided Latent Consistency Distillation
By distilling a latent consistency model (LCM) from a pre-trained teacher latent diffusion model (LDM), LCD facilitates the generation of high-fidelity images within merely 2 to 4 inference steps.
AKEW: Assessing Knowledge Editing in the Wild
Knowledge editing injects knowledge updates into language models to keep them correct and up-to-date.
Hire a Linguist!: Learning Endangered Languages with In-Context Linguistic Descriptions
On the contrary, we observe that 2000 endangered languages, though without a large corpus, have a grammar book or a dictionary.
A Survey on Data Selection for Language Models
A major factor in the recent success of large language models is the use of enormous and ever-growing text datasets for unsupervised pre-training.
Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinement
We discovered that such a contrary is due to LLM's bias in evaluating their own output.
Understanding Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation
To understand how pre-training with a next-token prediction objective contributes to the emergence of such reasoning capability, we propose that we can view an LM as deriving new conclusions by aggregating indirect reasoning paths seen at pre-training time.
Weak-to-Strong Jailbreaking on Large Language Models
In this paper, we propose the weak-to-strong jailbreaking attack, an efficient method to attack aligned LLMs to produce harmful text.
Position: AI/ML Influencers Have a Place in the Academic Process
As the number of accepted papers at AI and ML conferences reaches into the thousands, it has become unclear how researchers access and read research publications.
Efficient Online Data Mixing For Language Model Pre-Training
The data used to pretrain large language models has a decisive impact on a model's downstream performance, which has led to a large body of work on data selection methods that aim to automatically determine the most suitable data to use for pretraining.
Text as Images: Can Multimodal Large Language Models Follow Printed Instructions in Pixels?
Recent multimodal large language models (MLLMs) have shown promising instruction following capabilities on vision-language tasks.
LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback
Recent large language models (LLM) are leveraging human feedback to improve their generation quality.
GPT-4V(ision) as a Generalist Evaluator for Vision-Language Tasks
Automatically evaluating vision-language tasks is challenging, especially when it comes to reflecting human judgments due to limitations in accounting for fine-grained details.
A Survey on Detection of LLMs-Generated Content
The burgeoning capabilities of advanced large language models (LLMs) such as ChatGPT have led to an increase in synthetic content generation with implications across a variety of sectors, including media, cybersecurity, public discourse, and education.
MAF: Multi-Aspect Feedback for Improving Reasoning in Large Language Models
Language Models (LMs) have shown impressive performance in various natural language tasks.
Mastering Robot Manipulation with Multimodal Prompts through Pretraining and Multi-task Fine-tuning
In this work, we tackle the problem of training a robot to understand multimodal prompts, interleaving vision signals with text descriptions.
ASSERT: Automated Safety Scenario Red Teaming for Evaluating the Robustness of Large Language Models
As large language models are integrated into society, robustness toward a suite of prompts is increasingly important to maintain reliability in a high-variance environment. Robustness evaluations must comprehensively encapsulate the various settings in which a user may invoke an intelligent system.
Empowering Psychotherapy with Large Language Models: Cognitive Distortion Detection through Diagnosis of Thought Prompting
Mental illness remains one of the most critical public health issues of our time, due to the severe scarcity and accessibility limit of professionals.
Guiding Language Model Reasoning with Planning Tokens
To encourage a more structural generation of CoT steps, we propose a hierarchical generation scheme: we let the LM generate a planning token at the start of each reasoning step, intuitively serving as a high-level plan of the current step, and add their embeddings to the model parameters.
Zero-Shot Detection of Machine-Generated Codes
We then modify the previous zero-shot text detection method, DetectGPT (Mitchell et al., 2023) by utilizing a surrogate white-box model to estimate the probability of the rightmost tokens, allowing us to identify code snippets generated by language models.
Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models
This study serves as a clarion call for a collective effort to overhaul and fortify the safety of open-source LLMs against malicious attackers.
Guiding Instruction-based Image Editing via Multimodal Large Language Models
Extensive experimental results demonstrate that expressive instructions are crucial to instruction-based image editing, and our MGIE can lead to a notable improvement in automatic metrics and human evaluation while maintaining competitive inference efficiency.
Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies
Large language models (LLMs) have demonstrated remarkable performance across a wide array of NLP tasks.
VELMA: Verbalization Embodiment of LLM Agents for Vision and Language Navigation in Street View
In this work, we propose VELMA, an embodied LLM agent that uses a verbalization of the trajectory and of visual environment observations as contextual prompt for the next action.
Multilingual Conceptual Coverage in Text-to-Image Models
We propose "Conceptual Coverage Across Languages" (CoCo-CroLa), a technique for benchmarking the degree to which any generative text-to-image system provides multilingual parity to its training language in terms of tangible nouns.
Generate then Select: Open-ended Visual Question Answering Guided by World Knowledge
The open-ended Visual Question Answering (VQA) task requires AI models to jointly reason over visual and natural language inputs using world knowledge.
DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Text
However, this progress also presents a significant challenge in detecting the origin of a given text, and current research on detection methods lags behind the rapid evolution of LLMs.
LayoutGPT: Compositional Visual Planning and Generation with Large Language Models
When combined with a downstream image generation model, LayoutGPT outperforms text-to-image models/systems by 20-40% and achieves comparable performance as human users in designing visual layouts for numerical and spatial correctness.
ALGO: Synthesizing Algorithmic Programs with LLM-Generated Oracle Verifiers
To address these challenges, we propose ALGO, a framework that synthesizes Algorithmic programs with LLM-Generated Oracles to guide the generation and verify their correctness.
On the Risk of Misinformation Pollution with Large Language Models
In this paper, we comprehensively investigate the potential misuse of modern Large Language Models (LLMs) for generating credible-sounding misinformation and its subsequent impact on information-intensive applications, particularly Open-Domain Question Answering (ODQA) systems.
EDIS: Entity-Driven Image Search over Multimodal Web Content
Making image retrieval methods practical for real-world search applications requires significant progress in dataset scales, entity comprehension, and multimodal information fusion.
Let's Think Frame by Frame with VIP: A Video Infilling and Prediction Dataset for Evaluating Video Chain-of-Thought
Despite exciting recent results showing vision-language systems' capacity to reason about images using natural language, their capacity for video reasoning remains under-explored.
Text-guided 3D Human Generation from 2D Collections
To address this \texttt{T3H} task, we propose Compositional Cross-modal Human (CCH).
Ranked #1 on
Text-to-3D-Human Generation
on SHHQ
The Knowledge Alignment Problem: Bridging Human and External Knowledge for Large Language Models
Large language models often necessitate grounding on external knowledge to generate faithful and reliable answers.
INSTRUCTSCORE: Explainable Text Generation Evaluation with Finegrained Feedback
By harnessing both explicit human instruction and the implicit knowledge of GPT-4, we fine-tune a text evaluation metric based on LLaMA, producing both a score for generated text and a human readable diagnostic report.
Fact-Checking Complex Claims with Program-Guided Reasoning
Fact-checking real-world claims often requires collecting multiple pieces of evidence and applying complex multi-step reasoning.
Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning
We also introduce a self-refinement module, which utilizes the symbolic solver's error messages to revise symbolic formalizations.
Collaborative Generative AI: Integrating GPT-k for Efficient Editing in Text-to-Image Generation
We conduct a series of experiments to compare the common edits made by humans and GPT-k, evaluate the performance of GPT-k in prompting T2I, and examine factors that may influence this process.
LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation
Existing automatic evaluation on text-to-image synthesis can only provide an image-text matching score, without considering the object-level compositionality, which results in poor correlation with human judgments.
Discffusion: Discriminative Diffusion Models as Few-shot Vision and Language Learners
Diffusion models, such as Stable Diffusion, have shown incredible performance on text-to-image generation.
Data Augmentation for Diverse Voice Conversion in Noisy Environments
Voice conversion (VC) models have demonstrated impressive few-shot conversion quality on the clean, native speech populations they're trained on.
Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillings
Recent advances in large language models elicit reasoning in a chain-of-thought that allows models to decompose problems in a human-like fashion.
Multimodal Procedural Planning via Dual Text-Image Prompting
The key challenges of MPP are to ensure the informativeness, temporal coherence, and accuracy of plans across modalities.
Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved with Text
We release Multimodal C4, an augmentation of the popular text-only C4 corpus with images interleaved.
Users are the North Star for AI Transparency
We conduct a broad literature survey, identifying many clusters of similar conceptions of transparency, tying each back to our north star with analysis of how it furthers or hinders our ideal AI transparency goals.
ReDi: Efficient Learning-Free Diffusion Inference via Trajectory Retrieval
Diffusion models show promising generation capability for a variety of data.
Improving Few-Shot Generalization by Exploring and Exploiting Auxiliary Data
In this work, we focus on Few-shot Learning with Auxiliary Data (FLAD), a training paradigm that assumes access to auxiliary data during few-shot learning in hopes of improving generalization.
Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning
This study aims to examine the in-context learning phenomenon through a Bayesian lens, viewing real-world LLMs as latent variable models.
SWING: Balancing Coverage and Faithfulness for Dialogue Summarization
Missing information is a common issue of dialogue summarization where some information in the reference summaries is not covered in the generated summaries.
Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness
Neural text-to-SQL models have achieved remarkable performance in translating natural language questions into SQL queries.
CausalDialogue: Modeling Utterance-level Causality in Conversations
Despite their widespread adoption, neural conversation models have yet to exhibit natural chat capabilities with humans.
Tokenization Consistency Matters for Generative Models on Extractive NLP Tasks
We show that, with consistent tokenization, the model performs better in both in-domain and out-of-domain datasets, with a notable average of +1. 7 F2 gain when a BART model is trained on SQuAD and evaluated on 8 QA datasets.
Foveate, Attribute, and Rationalize: Towards Physically Safe and Trustworthy AI
Users' physical safety is an increasing concern as the market for intelligent systems continues to grow, where unconstrained systems may recommend users dangerous actions that can lead to serious injury.
SESCORE2: Learning Text Generation Evaluation via Synthesizing Realistic Mistakes
In this paper, we propose SESCORE2, a self-supervised approach for training a model-based metric for text generation evaluation.
Improving Cross-task Generalization of Unified Table-to-text Models with Compositional Task Configurations
There has been great progress in unifying various table-to-text tasks using a single encoder-decoder model trained via multi-task learning (Xie et al., 2022).
Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis
In this work, we improve the compositional skills of T2I models, specifically more accurate attribute binding and better image compositions.
Offline Reinforcement Learning with Closed-Form Policy Improvement Operators
Behavior constrained policy optimization has been demonstrated to be a successful paradigm for tackling Offline Reinforcement Learning.
Tell Me What Happened: Unifying Text-guided Video Completion via Multimodal Masked Video Generation
Inspired by this, we introduce a novel task, text-guided video completion (TVC), which requests the model to generate a video from partial frames guided by an instruction.
Ranked #3 on
Video Prediction
on BAIR Robot Pushing
Bridging the Training-Inference Gap for Dense Phrase Retrieval
Building dense retrievers requires a series of standard procedures, including training and validating neural models and creating indexes for efficient search.
WikiWhy: Answering and Explaining Cause-and-Effect Questions
As large language models (LLMs) grow larger and more sophisticated, assessing their "reasoning" capabilities in natural language grows more challenging.
An Exploration of Data Efficiency in Intra-Dataset Task Transfer for Dialog Understanding
Transfer learning is an exciting area of Natural Language Processing that has the potential to both improve model performance and increase data efficiency.
CPL: Counterfactual Prompt Learning for Vision and Language Models
Prompt tuning is a new few-shot transfer learning technique that only tunes the learnable prompt for pre-trained vision and language models such as CLIP.
SafeText: A Benchmark for Exploring Physical Safety in Language Models
Understanding what constitutes safe text is an important issue in natural language processing and can often prevent the deployment of models deemed harmful and unsafe.
ULN: Towards Underspecified Vision-and-Language Navigation
Vision-and-Language Navigation (VLN) is a task to guide an embodied agent moving to a target position using language instructions.
Mitigating Covertly Unsafe Text within Natural Language Systems
An increasingly prevalent problem for intelligent technologies is text safety, as uncontrolled systems may generate recommendations to their users that lead to injury or life-threatening consequences.
Language Agnostic Multilingual Information Retrieval with Contrastive Learning
Multilingual information retrieval (IR) is challenging since annotated training data is costly to obtain in many languages.
CLIP also Understands Text: Prompting CLIP for Phrase Understanding
However, the application of its text encoder solely for text understanding has been less explored.
Not All Errors are Equal: Learning Text Generation Metrics using Stratified Error Synthesis
Is it possible to build a general and automatic natural language generation (NLG) evaluation metric?
ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering
With the recent advance in large pre-trained language models, researchers have achieved record performances in NLP tasks that mostly focus on language pattern matching.
Ranked #2 on
Question Answering
on ConvFinQA
Flexible Attention-Based Multi-Policy Fusion for Efficient Deep Reinforcement Learning
In this work, we present Knowledge-Grounded RL (KGRL), an RL paradigm fusing multiple knowledge policies and aiming for human-like efficiency and flexibility.
Dynamic Latent Separation for Deep Learning
A core problem in machine learning is to learn expressive latent variables for model prediction on complex data that involves multiple sub-components in a flexible and interpretable fashion.
Visualize Before You Write: Imagination-Guided Open-Ended Text Generation
Recent advances in text-to-image synthesis make it possible to visualize machine imaginations for a given context.
Anticipating the Unseen Discrepancy for Vision and Language Navigation
In this paper, we propose an Unseen Discrepancy Anticipating Vision and Language Navigation (DAVIS) that learns to generalize to unseen environments via encouraging test-time visual consistency.
An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modeling
Masked visual modeling (MVM) has been recently proven effective for visual pre-training.
Ranked #1 on
Video Question Answering
on LSMDC-MC
Causal Balancing for Domain Generalization
While machine learning models rapidly advance the state-of-the-art on various real-world tasks, out-of-domain (OOD) generalization remains a challenging problem given the vulnerability of these models to spurious correlations.
Neuro-Symbolic Procedural Planning with Commonsense Prompting
Procedural planning aims to implement complex high-level goals by decomposition into sequential simpler low-level steps.
Towards Understanding Gender-Seniority Compound Bias in Natural Language Generation
Our results show that GPT-2 amplifies bias by considering women as junior and men as senior more often than the ground truth in both domains.
FETA: A Benchmark for Few-Sample Task Transfer in Open-Domain Dialogue
Task transfer, transferring knowledge contained in related tasks, holds the promise of reducing the quantity of labeled data required to fine-tune language models.
KETOD: Knowledge-Enriched Task-Oriented Dialogue
We also propose two new models, SimpleToDPlus and Combiner, for the proposed task.
HybriDialogue: An Information-Seeking Dialogue Dataset Grounded on Tabular and Textual Data
A pressing challenge in current dialogue systems is to successfully converse with users on topics with information distributed across different modalities.
Imagination-Augmented Natural Language Understanding
Human brains integrate linguistic and perceptual information simultaneously to understand natural language, and hold the critical ability to render imaginations.
End-to-end Dense Video Captioning as Sequence Generation
Dense video captioning aims to identify the events of interest in an input video, and generate descriptive captions for each event.
Ranked #4 on
Dense Video Captioning
on ViTT
(CIDEr metric, using extra
training data)
Towards Large-Scale Interpretable Knowledge Graph Reasoning for Dialogue Systems
Our proposed method allows a single transformer model to directly walk on a large-scale knowledge graph to generate responses.
Addressing Issues of Cross-Linguality in Open-Retrieval Question Answering Systems For Emergent Domains
Open-retrieval question answering systems are generally trained and tested on large datasets in well-established domains.
PECO: Examining Single Sentence Label Leakage in Natural Language Inference Datasets through Progressive Evaluation of Cluster Outliers
Building natural language inference (NLI) benchmarks that are both challenging for modern techniques, and free from shortcut biases is difficult.
Relational Graph Learning for Grounded Video Description Generation
Such a setting can help explain the decisions of captioning models and prevents the model from hallucinating object words in its description.
VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling
Further, unlike previous studies that found pre-training tasks on video inputs (e. g., masked frame modeling) not very effective, we design a new pre-training task, Masked Visual-token Modeling (MVM), for better video modeling.
Ranked #21 on
Zero-Shot Video Retrieval
on DiDeMo
MIC: Model-agnostic Integrated Cross-channel Recommenders
However, existing work are primarily built upon pre-defined retrieval channels, including User-CF (U2U), Item-CF (I2I), and Embedding-based Retrieval (U2I), thus access to the limited correlation between users and items which solely entail from partial information of latent interactions.
Off-policy Reinforcement Learning with Optimistic Exploration and Distribution Correction
Improving the sample efficiency of reinforcement learning algorithms requires effective exploration.
Attacking Open-domain Question Answering by Injecting Misinformation
We curate both human-written and model-generated false documents that we inject into the evidence corpus of QA models and assess the impact on the performance of these systems.
Open-Domain Question-Answering for COVID-19 and Other Emergent Domains
In this work, we present such a system for the emergent domain of COVID-19.
Self-Supervised Knowledge Assimilation for Expert-Layman Text Style Transfer
This is a particularly notable issue in the medical domain, where layman are often confused by medical text online.
A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Space
We verify some of our linguistic findings by looking at the effect of morphological segmentation on English-Inuktitut alignment, in addition to examining the effect of word order agreement on isomorphism for 66 zero-shot language pairs from a different corpus.
D-REX: Dialogue Relation Extraction with Explanations
Existing research studies on cross-sentence relation extraction in long-form multi-party conversations aim to improve relation extraction without considering the explainability of such methods.
Ranked #7 on
Dialog Relation Extraction
on DialogRE
FinQA: A Dataset of Numerical Reasoning over Financial Data
In contrast to existing tasks on general domain, the finance domain includes complex numerical reasoning and understanding of heterogeneous representations.
Ranked #4 on
Question Answering
on FinQA
Neural Stylistic Response Generation with Disentangled Latent Variables
Generating open-domain conversational responses in the desired style usually suffers from the lack of p