Search Results for author:
Found 101 papers, 59 papers with code
An Exploratory Study on Long Dialogue Summarization: What Works and What’s Next
Dialogue summarization helps readers capture salient information from long conversations in meetings, interviews, and TV series.
Knowledge-Augmented Methods for Natural Language Processing
Knowledge in natural language processing (NLP) has been a rising trend especially after the advent of large scale pre-trained models.
Modeling Entity Knowledge for Fact Verification
Fact verification is a challenging task of identifying the truthfulness of given claims based on the retrieval of relevant evidence texts.
Unleashing In-context Learning of Autoregressive Models for Few-shot Image Manipulation
We also introduce a relation regularization method to further disentangle image transformation features from irrelevant contents in exemplar images.
Self-Generated Critiques Boost Reward Modeling for Language Models
Reward modeling is crucial for aligning large language models (LLMs) with human preferences, especially in reinforcement learning from human feedback (RLHF).
Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Following
To address this gap, we introduce Multi-IF, a new benchmark designed to assess LLMs' proficiency in following multi-turn and multilingual instructions.
Law of the Weakest Link: Cross Capabilities of Large Language Models
The development and evaluation of Large Language Models (LLMs) have largely focused on individual capabilities.
Why mamba is effective? Exploit Linear Transformer-Mamba Network for Multi-Modality Image Fusion
To solve this problem, we propose a dual-branch image fusion network called Tmamba.
See What LLMs Cannot Answer: A Self-Challenge Framework for Uncovering LLM Weaknesses
We then build a benchmark, SC-G4, consisting of 1, 835 instances generated by GPT-4 using these patterns, with human-annotated gold responses.
Improving Multilingual Instruction Finetuning via Linguistically Natural and Diverse Datasets
Advancements in Large Language Models (LLMs) have significantly enhanced instruction-following capabilities.
WPO: Enhancing RLHF with Weighted Preference Optimization
This method not only addresses the distributional gap problem but also enhances the optimization process without incurring additional costs.
CoDi-2: In-Context Interleaved and Interactive Any-to-Any Generation
We present CoDi-2 a Multimodal Large Language Model (MLLM) for learning in-context interleaved multimodal representations.
CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generation
We present CoDi-2, a versatile and interactive Multimodal Large Language Model (MLLM) that can follow complex multimodal interleaved instructions, conduct in-context learning (ICL), reason, chat, edit, etc., in an any-to-any input-output modality paradigm.
The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions
Recent progress in Large Language Models (LLMs) has produced models that exhibit remarkable performance across a variety of NLP tasks.
Auto-Instruct: Automatic Instruction Generation and Ranking for Black-Box Language Models
Large language models (LLMs) can perform a wide range of tasks by following natural language instructions, without the necessity of task-specific fine-tuning.
Soft Convex Quantization: Revisiting Vector Quantization with Convex Optimization
SCQ works like a differentiable convex optimization (DCO) layer: in the forward pass, we solve for the optimal convex combination of codebook vectors that quantize the inputs.
Sparse Modular Activation for Efficient Sequence Modeling
To validate the effectiveness of SMA on sequence modeling, we design a novel neural architecture, SeqBoat, which employs SMA to sparsely activate a Gated Attention Unit (GAU) based on the state representations learned from an SSM.
Fine-Tuning Language Models with Advantage-Induced Policy Alignment
Reinforcement learning from human feedback (RLHF) has emerged as a reliable approach to aligning large language models (LLMs) to human preferences.
In-Context Demonstration Selection with Cross Entropy Difference
Our method is based on the observation that the effectiveness of in-context demonstrations negatively correlates with the perplexity of the test example by a language model that was finetuned on that demonstration.
PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents
PEARL outperforms zero-shot and chain-of-thought prompting on this dataset, and ablation experiments show that each stage of PEARL is critical to its performance.
i-Code Studio: A Configurable and Composable Framework for Integrative AI
Artificial General Intelligence (AGI) requires comprehensive understanding and generation capabilities for a variety of tasks spanning different modalities and functionalities.
LMGQS: A Large-scale Dataset for Query-focused Summarization
We hypothesize that there is a hidden query for each summary sentence in a generic summarization annotation, and we utilize a large-scale pretrained language model to recover it.
InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT
While large models such as GPT-3 demonstrate exceptional performance in zeroshot and fewshot summarization tasks, their extensive serving and fine-tuning costs hinder their utilization in various applications.
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data
The convergence of text, visual, and audio data is a key step towards human-like artificial intelligence, however the current Vision-Language-Speech landscape is dominated by encoder-only models which lack generative abilities.
Any-to-Any Generation via Composable Diffusion
We present Composable Diffusion (CoDi), a novel generative model capable of generating any combination of output modalities, such as language, image, video, or audio, from any combination of input modalities.
Ranked #8 on
Audio Generation
on AudioCaps
(FAD metric)
Small Models are Valuable Plug-ins for Large Language Models
Large language models (LLMs) such as GPT-3 and GPT-4 are powerful but their weights are often publicly unavailable and their immense sizes make the models difficult to be tuned with common hardware.
Automatic Prompt Optimization with "Gradient Descent" and Beam Search
Large Language Models (LLMs) have shown impressive performance as general purpose agents, but their abilities remain highly dependent on prompts which are hand written with onerous trial-and-error effort.
G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
In this work, we present G-Eval, a framework of using large language models with chain-of-thoughts (CoT) and a form-filling paradigm, to assess the quality of NLG outputs.
How Does In-Context Learning Help Prompt Tuning?
This motivates the use of parameter-efficient adaptation methods such as prompt tuning (PT), which adds a small number of tunable embeddings to an otherwise frozen model, and in-context learning (ICL), in which demonstrations of the task are provided to the model in natural language without any additional training.
APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning
Logical reasoning of text is an important ability that requires understanding the information present in the text, their interconnections, and then reasoning through them to infer new conclusions.
Unifying Vision, Text, and Layout for Universal Document Processing
UDOP leverages the spatial correlation between textual content and document image to model image, text, and layout modalities with one uniform representation.
Ranked #5 on
Visual Question Answering (VQA)
on InfographicVQA
(using extra training data)
Improving Commonsense in Vision-Language Models via Knowledge Graph Riddles
Through our analysis, we find one important reason is that existing large-scale VL datasets do not contain much commonsense knowledge, which motivates us to improve the commonsense of VL-models from the data perspective.
UniSumm and SummZoo: Unified Model and Diverse Benchmark for Few-Shot Summarization
The high annotation costs and diverse demands of various summarization tasks motivate the development of few-shot summarization.
Empowering Language Models with Knowledge Graph Reasoning for Question Answering
Answering open-domain questions requires world knowledge about in-context entities.
MACSum: Controllable Summarization with Mixed Attributes
We propose two simple and effective parameter-efficient approaches for the new task of mixed controllable summarization based on hard prompt tuning and soft prefix tuning.
Retrieval Augmentation for Commonsense Reasoning: A Unified Approach
However, applying such methods to commonsense reasoning tasks faces two unique challenges, i. e., the lack of a general large-scale corpus for retrieval and a corresponding effective commonsense retriever.
Global Contrastive Batch Sampling via Optimization on Sample Permutations
Contrastive Learning has recently achieved state-of-the-art performance in a wide range of tasks.
Towards a Unified Multi-Dimensional Evaluator for Text Generation
We re-frame NLG evaluation as a Boolean Question Answering (QA) task, and by guiding the model with different questions, we can use one evaluator to evaluate from multiple dimensions.
Task Compass: Scaling Multi-task Pre-training with Task Prefix
Leveraging task-aware annotated data as supervised signals to assist with self-supervised learning on large-scale unlabeled data has become a new trend in pre-training language models.
Ranked #1 on
Sentence Completion
on HellaSwag
A Unified Encoder-Decoder Framework with Entity Memory
The entity knowledge is stored in the memory as latent representations, and the memory is pre-trained on Wikipedia along with encoder-decoder parameters.
FAST: Improving Controllability for Text Generation with Feedback Aware Self-Training
Controllable text generation systems often leverage control codes to direct various properties of the output like style and length.
Generate rather than Retrieve: Large Language Models are Strong Context Generators
We call our method generate-then-read (GenRead), which first prompts a large language model to generate contextutal documents based on a given question, and then reads the generated documents to produce the final answer.
Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization
Z-Code++ creates new state of the art on 9 out of 13 text summarization tasks across 5 languages.
REVIVE: Regional Visual Representation Matters in Knowledge-Based Visual Question Answering
Specifically, we observe that in most state-of-the-art knowledge-based VQA methods: 1) visual features are extracted either from the whole image or in a sliding window manner for retrieving knowledge, and the important relationship within/among object regions is neglected; 2) visual features are not well utilized in the final answering model, which is counter-intuitive to some extent.
Ranked #11 on
Visual Question Answering (VQA)
on OK-VQA
Leveraging Locality in Abstractive Text Summarization
Our experimental results show that our model has a better performance compared with strong baselines with efficient attention modules, and our analysis provides further insights into our locality-aware modeling strategy.
Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners
The goal of this work is to build flexible video-language models that can generalize to various video-to-text tasks from few examples, such as domain-specific captioning, question answering, and future event prediction.
Automatic Rule Induction for Interpretable Semi-Supervised Learning
Semi-supervised learning has shown promise in allowing NLP models to generalize from small amounts of labeled data.
i-Code: An Integrative and Composable Multimodal Learning Framework
Human intelligence is multimodal; we integrate visual, linguistic, and acoustic signals to maintain a holistic worldview.
Impossible Triangle: What's Next for Pre-trained Language Models?
Recent development of large-scale pre-trained language models (PLM) have significantly improved the capability of models in various NLP tasks, in terms of performance after task-specific fine-tuning and zero-shot / few-shot learning.
Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data
Surprisingly, we found that REtrieving from the traINing datA (REINA) only can lead to significant gains on multiple NLG and NLU tasks.
Leveraging Visual Knowledge in Language Tasks: An Empirical Study on Intermediate Pre-training for Cross-modal Knowledge Transfer
Pre-trained language models are still far from human performance in tasks that need understanding of properties (e. g. appearance, measurable quantity) and affordances of everyday objects in the real world since the text lacks such information due to reporting bias.
Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts
A set of knowledge experts seek diverse reasoning on KG to encourage various generation outputs.
SapientML: Synthesizing Machine Learning Pipelines by Learning from Human-Written Solutions
In this work we propose an AutoML technique SapientML, that can learn from a corpus of existing datasets and their human-written pipelines, and efficiently generate a high-quality pipeline for a predictive task on a new dataset.
AdaPrompt: Adaptive Model Training for Prompt-based NLP
However, for prompt learning, there are still two salient gaps between NLP tasks and pretraining.
Unsupervised Multi-Granularity Summarization
In this paper, we propose the first unsupervised multi-granularity summarization framework, GranuSum.
CLIP-Event: Connecting Text and Images with Event Structures
Vision-language (V+L) pretraining models have achieved great success in supporting multimedia applications by understanding the alignments between images and text.
MLP Architectures for Vision-and-Language Modeling: An Empirical Study
Based on this, we ask an even bolder question: can we have an all-MLP architecture for VL modeling, where both VL fusion and the vision encoder are replaced with MLPs?
Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention
In particular, we focus on the task of Commonsense Reasoning, demonstrating that the proposed external attention mechanism can augment existing transformer models and significantly improve the model's reasoning capabilities.
Ranked #1 on
Common Sense Reasoning
on CommonsenseQA
(using extra training data)
An Empirical Study of Training End-to-End Vision-and-Language Transformers
Vision-and-language (VL) pre-training has proven to be highly effective on various VL downstream tasks.
Ranked #20 on
Cross-Modal Retrieval
on COCO 2014
(using extra training data)
SYNERGY: Building Task Bots at Scale Using Symbolic Knowledge and Machine Teaching
We propose SYNERGY, a hybrid learning framework where a task bot is developed in two steps: (i) Symbolic knowledge to neural networks: Large amounts of simulated dialog sessions are generated based on task-specific symbolic knowledge which is represented as a task schema consisting of dialog flows and task-oriented databases.
Summ^N: A Multi-Stage Summarization Framework for Long Input Dialogues and Documents
To the best of our knowledge, Summ$^N$ is the first multi-stage split-then-summarize framework for long input summarization.
Leveraging Knowledge in Multilingual Commonsense Reasoning
Then we utilize a diverse of 4 English knowledge sources to provide more comprehensive coverage of knowledge in different formats.
DYLE: Dynamic Latent Extraction for Abstractive Long-Input Summarization
Transformer-based models have achieved state-of-the-art performance on short-input summarization.
End-to-End Segmentation-based News Summarization
In this paper, we bring a new way of digesting news content by introducing the task of segmenting a news article into multiple sections and generating the corresponding summary to each section.
Dict-BERT: Enhancing Language Model Pre-training with Dictionary
In addition to training with the masked language modeling objective, we propose two novel self-supervised pre-training tasks on word and sentence-level alignment between input text sequence and rare word definitions to enhance language modeling representation with dictionary.
KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering
The recent proposed Fusion-in-Decoder (FiD), which is built on top of the pretrained generative model T5, achieves the state-of-the-art performance in the reading module.
An Exploratory Study on Long Dialogue Summarization: What Works and What's Next
Dialogue summarization helps readers capture salient information from long conversations in meetings, interviews, and TV series.
DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarization
For a dialogue, it corrupts a window of text with dialogue-inspired noise, and guides the model to reconstruct this window based on the content of the remaining conversation.
Does Knowledge Help General NLU? An Empirical Study
It is often observed in knowledge-centric tasks (e. g., common sense question and answering, relation classification) that the integration of external knowledge such as entity representation into language models can help provide useful information to boost the performance.
Want To Reduce Labeling Cost? GPT-3 Can Help
Data annotation is a time-consuming and labor-intensive process for many NLP tasks.
Sentence-Permuted Paragraph Generation
Generating paragraphs of diverse contents is important in many applications.
MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization
MediaSum, a large-scale media interview dataset consisting of 463. 6K transcripts with abstractive summaries.
RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systems
For task-oriented dialog systems to be maximally useful, it must be able to process conversations in a way that is (1) generalizable with a small number of training examples for new task domains, and (2) robust to user input in various styles, modalities or domains.
Fusing Context Into Knowledge Graph for Commonsense Question Answering
However, although a KG contains rich structural information, it lacks the context to provide a more precise understanding of the concepts.
Ranked #5 on
Common Sense Reasoning
on CommonsenseQA
(using extra training data)
Mixed-Lingual Pre-training for Cross-lingual Summarization
Cross-lingual Summarization (CLS) aims at producing a summary in the target language for an article in the source language.
SPLAT: Speech-Language Joint Pre-Training for Spoken Language Understanding
Besides conducting a self-supervised masked language modeling task on the two individual modules using unpaired speech and text, SPLAT aligns representations from the two modules in a shared latent space using a small amount of paired speech and text.
JAKET: Joint Pre-training of Knowledge Graph and Language Understanding
Knowledge graphs (KGs) contain rich information about world knowledge, entities and relations.
Injecting Entity Types into Entity-Guided Text Generation
Our model has a multi-step decoder that injects the entity types into the process of entity mention generation.
Accelerating Real-Time Question Answering via Question Generation
Although deep neural networks have achieved tremendous success for question answering (QA), they are still suffering from heavy computational and energy cost for real product deployment.
Mind The Facts: Knowledge-Boosted Coherent Abstractive Text Summarization
In this work, we propose a novel architecture that extends Transformer encoder-decoder architecture in order to improve on these shortcomings.
Filtered Inner Product Projection for Crosslingual Embedding Alignment
Due to widespread interest in machine translation and transfer learning, there are numerous algorithms for mapping multiple embeddings to a shared representation space.
Meta Dialogue Policy Learning
Dialog policy determines the next-step actions for agents and hence is central to a dialogue system.
Boosting Naturalness of Language in Task-oriented Dialogues via Adversarial Training
The natural language generation (NLG) module in a task-oriented dialogue system produces user-facing utterances conveying required information.
Data Augmentation for Spoken Language Understanding via Pretrained Language Models
The training of spoken language understanding (SLU) models often faces the problem of data scarcity.
A Hierarchical Network for Abstractive Meeting Summarization with Cross-Domain Pretraining
With the abundance of automatic meeting transcripts, meeting summarization is of great interest to both participants and other parties.
Enhancing Factual Consistency of Abstractive Summarization
Automatic abstractive summaries are found to often distort or fabricate facts in the article.
Few-shot Natural Language Generation for Task-Oriented Dialog
It is pre-trained on a large set of annotated NLG corpus to acquire the controllable generation ability, and fine-tuned with only a few domain-specific labels to adapt to new domains.
Ranked #4 on
Data-to-Text Generation
on MULTIWOZ 2.1
TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising
Text summarization aims to extract essential information from a piece of text and transform the text into a concise version.
Leveraging Lead Bias for Zero-shot Abstractive News Summarization
A typical journalistic convention in news articles is to deliver the most salient information in the beginning, also known as the lead bias.
Multi-task Learning for Natural Language Generation in Task-Oriented Dialogue
In this paper, we propose a novel multi-task learning framework, NLG-LM, for natural language generation.
SIM: A Slot-Independent Neural Model for Dialogue State Tracking
In this paper, we put forward a slot-independent neural model (SIM) to track dialogue states while keeping the model complexity invariant to the number of dialogue slots.
Make Lead Bias in Your Favor: A Simple and Effective Method for News Summarization
For example, the pretrained model without finetuning outperforms pointer-generator network on CNN/DailyMail dataset.
Embedding Imputation with Grounded Language Information
In this paper, we propose an approach for embedding imputation which uses grounded information in the form of a knowledge graph.
Out-of-Vocabulary Embedding Imputation with Grounded Language Information by Graph Convolutional Networks
In this paper, we propose an approach for embedding imputation which uses grounded information in the form of a knowledge graph.
Zero-training Sentence Embedding via Orthogonal Basis
We model the semantic meaning of a word in a sentence based on two aspects.
SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering
Conversational question answering (CQA) is a novel QA task that requires understanding of dialogue context.
Ranked #3 on
Question Answering
on CoQA
(Overall metric)
Conversational Question Answering
Machine Reading Comprehension
+1
Parameter-free Sentence Embedding via Orthogonal Basis
Inspired by the Gram-Schmidt Process in geometric theory, we build an orthogonal basis of the subspace spanned by a word and its surrounding context in a sentence.
Learning to Attend On Essential Terms: An Enhanced Retriever-Reader Model for Open-domain Question Answering
In this paper we propose a retriever-reader model that learns to attend on essential terms during the question answering process.
FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension
This paper introduces a new neural structure called FusionNet, which extends existing attention approaches from three perspectives.
Ranked #26 on
Question Answering
on SQuAD1.1 dev
