Search Results for author:
Found 58 papers, 17 papers with code
EASE: Extractive-Abstractive Summarization End-to-End using the Information Bottleneck Principle
To achieve the best of both worlds, we propose EASE, an extractive-abstractive framework that generates concise abstractive summaries that can be traced back to an extractive summary.
Attention or Convolution: Transformer Encoders in Audio Language Models for Inference Efficiency
In this paper, we show that a simple self-supervised pre-trained audio model can achieve comparable inference efficiency to more complicated pre-trained models with speech transformer encoders.
Effective Long-Context Scaling of Foundation Models
We also examine the impact of various design choices in the pretraining process, including the data mix and the training curriculum of sequence lengths -- our ablation experiments suggest that having abundant long texts in the pretrain dataset is not the key to achieving strong performance, and we empirically verify that long context continual pretraining is more efficient and similarly effective compared to pretraining from scratch with long sequences.
LLM-QAT: Data-Free Quantization Aware Training for Large Language Models
Several post-training quantization methods have been applied to large language models (LLMs), and have been shown to perform well down to 8-bits.
VideoOFA: Two-Stage Pre-Training for Video-to-Text Generation
We propose a new two-stage pre-training framework for video-to-text generation tasks such as video captioning and video question answering: A generative encoder-decoder model is first jointly pre-trained on massive image-text data to learn fundamental vision-language concepts, and then adapted to video data in an intermediate video-text pre-training stage to learn video-specific skills such as spatio-temporal reasoning.
How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrieval
We hence propose a new DA approach with diverse queries and sources of supervision to progressively train a generalizable DR. As a result, DRAGON, our dense retriever trained with diverse augmentation, is the first BERT-base-sized DR to achieve state-of-the-art effectiveness in both supervised and zero-shot evaluations and even competes with models using more complex late interaction (ColBERTv2 and SPLADE++).
STRUDEL: Structured Dialogue Summarization for Dialogue Comprehension
Abstractive dialogue summarization has long been viewed as an important standalone task in natural language processing, but no previous work has explored the possibility of whether abstractive dialogue summarization can also be used as a means to boost an NLP system's performance on other important dialogue comprehension tasks.
Improving Faithfulness of Abstractive Summarization by Controlling Confounding Effect of Irrelevant Sentences
Lack of factual correctness is an issue that still plagues state-of-the-art summarization systems despite their impressive progress on generating seemingly fluent summaries.
CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrieval
In this paper, we unify different multi-vector retrieval models from a token routing viewpoint and propose conditional token interaction via dynamic lexical routing, namely CITADEL, for efficient and effective multi-vector retrieval.
Bridging the Training-Inference Gap for Dense Phrase Retrieval
Building dense retrievers requires a series of standard procedures, including training and validating neural models and creating indexes for efficient search.
Structured Summarization: Unified Text Segmentation and Segment Labeling as a Generation Task
In this work, we propose a single encoder-decoder neural network that can handle long documents and conversations, trained simultaneously for both segmentation and segment labeling using only standard supervision.
Adapting Pretrained Text-to-Text Models for Long Text Sequences
We present an empirical study of adapting an existing pretrained text-to-text model for long-sequence inputs.
Ranked #1 on
Text Summarization
on QMSum
BiT: Robustly Binarized Multi-distilled Transformer
Modern pre-trained transformers have rapidly advanced the state-of-the-art in machine learning, but have also grown in parameters and computational complexity, making them increasingly difficult to deploy in resource-constrained environments.
CONFIT: Toward Faithful Dialogue Summarization with Linguistically-Informed Contrastive Fine-tuning
Using human evaluation and automatic faithfulness metrics, we show that our model significantly reduces all kinds of factual errors on the dialogue summarization, SAMSum corpus.
Simple Local Attentions Remain Competitive for Long-Context Tasks
Many NLP tasks require processing long contexts beyond the length limit of pretrained models.
Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?
Despite their recent popularity and well-known advantages, dense retrievers still lag behind sparse methods such as BM25 in their ability to reliably match salient phrases and rare entities in the query and to generalize to out-of-domain data.
Ranked #2 on
Passage Retrieval
on EntityQuestions
Investigating Crowdsourcing Protocols for Evaluating the Factual Consistency of Summaries
Current pre-trained models applied to summarization are prone to factual inconsistencies which either misrepresent the source text or introduce extraneous information.
Domain-matched Pre-training Tasks for Dense Retrieval
Pre-training on larger datasets with ever increasing model size is now a proven recipe for increased performance across almost all NLP tasks.
Ranked #2 on
Passage Retrieval
on Natural Questions
(using extra training data)
Syntax-augmented Multilingual BERT for Cross-lingual Transfer
In recent years, we have seen a colossal effort in pre-training multilingual text encoders using large-scale corpora in many languages to facilitate cross-lingual transfer learning.
RECONSIDER: Improved Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering
State-of-the-art Machine Reading Comprehension (MRC) models for Open-domain Question Answering (QA) are typically trained for span selection using distantly supervised positive examples and heuristically retrieved negative examples.
ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining
While online conversations can cover a vast amount of information in many different formats, abstractive text summarization has primarily focused on modeling solely news articles.
EASE: Extractive-Abstractive Summarization with Explanations
Current abstractive summarization systems outperform their extractive counterparts, but their widespread adoption is inhibited by the inherent lack of interpretability.
Learning Better Structured Representations Using Low-rank Adaptive Label Smoothing
Training with soft targets instead of hard targets has been shown to improve performance and calibration of deep neural networks.
NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
We review the EfficientQA competition from NeurIPS 2020.
FiD-Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generation
Natural language (NL) explanations of model predictions are gaining popularity as a means to understand and verify decisions made by large black-box pre-trained models, for NLP tasks such as Question Answering (QA) and Fact Verification.
Human Evaluation of Spoken vs. Visual Explanations for Open-Domain QA
While research on explaining predictions of open-domain QA systems (ODQA) to users is gaining momentum, most works have failed to evaluate the extent to which explanations improve user trust.
UniK-QA: Unified Representations of Structured and Unstructured Knowledge for Open-Domain Question Answering
We study open-domain question answering with structured, unstructured and semi-structured knowledge sources, including text, tables, lists and knowledge bases.
Ranked #1 on
Open-Domain Question Answering
on WebQuestions
(using extra training data)
Knowledge Base Question Answering
Open-Domain Question Answering
Towards Understanding the Behaviors of Optimal Deep Active Learning Algorithms
Active learning (AL) algorithms may achieve better performance with fewer data because the model guides the data selection process.
Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation
Models pretrained with self-supervised objectives on large text corpora achieve state-of-the-art performance on English text summarization tasks.
RECONSIDER: Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering
State-of-the-art Machine Reading Comprehension (MRC) models for Open-domain Question Answering (QA) are typically trained for span selection using distantly supervised positive examples and heuristically retrieved negative examples.
Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing
Task-oriented semantic parsing is a critical component of virtual assistants, which is responsible for understanding the user's intents (set reminder, play music, etc.).
Efficient One-Pass End-to-End Entity Linking for Questions
We present ELQ, a fast end-to-end entity linking model for questions, which uses a biencoder to jointly perform mention detection and linking in one pass.
Conversational Semantic Parsing
In this paper, we propose a semantic representation for such task-oriented conversational systems that can represent concepts such as co-reference and context carryover, enabling comprehensive understanding of queries in a session.
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval
We propose a simple and efficient multi-hop dense retrieval approach for answering complex open-domain questions, which achieves state-of-the-art performance on two multi-hop datasets, HotpotQA and multi-evidence FEVER.
Ranked #14 on
Question Answering
on HotpotQA
MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmark
Scaling semantic parsing models for task-oriented dialog systems to new languages is often expensive and time-consuming due to the lack of available datasets.
DocTag2Vec: An Embedding Based Multi-label Learning Approach for Document Tagging
Tagging news articles or blog posts with relevant tags from a collection of predefined ones is coined as document tagging in this work.
Online Article Ranking as a Constrained, Dynamic, Multi-Objective Optimization Problem
The content ranking problem in a social news website, is typically a function that maximizes a scalar metric of interest like dwell-time.
Rank-to-engage: New Listwise Approaches to Maximize Engagement
For many internet businesses, presenting a given list of items in an order that maximizes a certain metric of interest (e. g., click-through-rate, average engagement time etc.)
RIPML: A Restricted Isometry Property based Approach to Multilabel Learning
The multilabel learning problem with large number of labels, features, and data-points has generated a tremendous interest recently.
Domain Adaptation for Named Entity Recognition in Online Media with Word Embeddings
In this paper, we propose methods to effectively adapt models learned on one domain onto other domains using distributed word representations.
Extractive Summarization under Strict Length Constraints
In this paper we report a comparison of various techniques for single-document extractive summarization under strict length budgets, which is a common commercial use case (e. g. summarization of news articles by news aggregators).
Chinese Whispers: Cooperative Paraphrase Acquisition
We present a framework for the acquisition of sentential paraphrases based on crowdsourcing.
