Search Results for author:
Found 40 papers, 19 papers with code
The Effect of Pretraining on Extractive Summarization for Scientific Documents
Large pretrained models have seen enormous success in extractive summarization tasks.
Continual Few-Shot Learning for Text Classification
In this work, we propose a continual few-shot learning (CFL) task, in which a system is challenged with a difficult phenomenon and asked to learn to correct mistakes with only a few (10 to 15) training examples.
Interactive Query-Assisted Summarization via Deep Reinforcement Learning
Interactive summarization is a task that facilitates user-guided exploration of information within a document set.
An Overview of Uncertainty Calibration for Text Classification and the Role of Distillation
Many recalibration methods have been proposed in the literature for quantifying predictive uncertainty and calibrating model outputs, with varying degrees of complexity.
Efficient Tool Use with Chain-of-Abstraction Reasoning
LLM agents trained with our method also show more efficient tool use, with inference speed being on average ~1. 4x faster than baseline tool-augmented LLMs.
PathFinder: Guided Search over Multi-Step Reasoning Paths
Using constrained reasoning, PathFinder integrates novel quality constraints, pruning, and exploration methods to enhance the efficiency and the quality of generation.
Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading
Large language models (LLMs) have advanced in large strides due to the effectiveness of the self-attention mechanism that processes and compares all tokens at once.
Crystal: Introspective Reasoners Reinforced with Self-Feedback
Extensive work has shown that the performance and interpretability of commonsense reasoning can be improved via knowledge-augmented reasoning methods, where the knowledge that underpins the reasoning process is explicitly verbalized and utilized.
Don't throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decoding
The key idea is not to throw out the value network, a byproduct of PPO training for evaluating partial output sequences, when decoding text out of the policy network.
Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning
It is also a general-purpose model that can do both text-to-image and image-to-text generation, allowing us to introduce self-contained contrastive decoding methods that produce high-quality outputs.
Ranked #2 on
Text-to-Image Generation
on MS COCO
Shepherd: A Critic for Language Model Generation
As large language models improve, there is increasing interest in techniques that leverage these models' capabilities to refine their own outputs.
OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization
To this end, we create OPT-IML Bench: a large benchmark for Instruction Meta-Learning (IML) of 2000 NLP tasks consolidated into task categories from 8 existing benchmarks, and prepare an evaluation framework to measure three types of model generalizations: to tasks from fully held-out categories, to held-out tasks from seen categories, and to held-out instances from seen tasks.
Ranked #26 on
Natural Language Inference
on RTE
Training Trajectories of Language Models Across Scales
Why do larger language models demonstrate more desirable behaviors?
MURMUR: Modular Multi-Step Reasoning for Semi-Structured Data-to-Text Generation
We propose MURMUR, a neuro-symbolic modular approach to text generation from semi-structured data with multi-step reasoning.
Complementary Explanations for Effective In-Context Learning
Large language models (LLMs) have exhibited remarkable capabilities in learning from explanations in prompts, but there has been limited understanding of exactly how these explanations function or why they are effective.
Improving In-Context Few-Shot Learning via Self-Supervised Training
Self-supervised pretraining has made few-shot learning possible for many NLP tasks.
Efficient Large Scale Language Modeling with Mixtures of Experts
This paper presents a detailed empirical study of how autoregressive MoE language models scale in comparison with dense models in a wide range of settings: in- and out-of-domain language modeling, zero- and few-shot priming, and full-shot fine-tuning.
Few-shot Learning with Multilingual Language Models
Large-scale generative language models such as GPT-3 are competitive few-shot learners.
Proposition-Level Clustering for Multi-Document Summarization
Text clustering methods were traditionally incorporated into multi-document summarization (MDS) as a means for coping with considerable information repetition.
Multi-Document Keyphrase Extraction: Dataset, Baselines and Review
Keyphrase extraction has been extensively researched within the single-document setting, with an abundance of methods, datasets and applications.
iFacetSum: Coreference-based Interactive Faceted Summarization for Multi-Document Exploration
We introduce iFacetSum, a web application for exploring topical document sets.
Efficiently Summarizing Text and Graph Encodings of Multi-Document Clusters
We also show improvements in a transfer-only setup on the DUC-2004 dataset.
Extending Multi-Document Summarization Evaluation to the Interactive Setting
In this paper, we develop an end-to-end evaluation framework for interactive summarization, focusing on expansion-based interaction, which considers the accumulating information along a user session.
Data Augmentation for Abstractive Query-Focused Multi-Document Summarization
The progress in Query-focused Multi-Document Summarization (QMDS) has been limited by the lack of sufficient largescale high-quality training datasets.
Dual Reinforcement-Based Specification Generation for Image De-Rendering
Since these are sequence models, we must choose an ordering of the objects in the graphics programs for likelihood training.
DORB: Dynamically Optimizing Multiple Rewards with Bandits
Further, it is important to consider using a dynamic combination and curriculum of metric rewards that flexibly changes over time.
FENAS: Flexible and Expressive Neural Architecture Search
Architecture search is the automatic process of designing the model or cell structure that is optimal for the given dataset or task.
Evaluating Interactive Summarization: an Expansion-Based Framework
Allowing users to interact with multi-document summarizers is a promising direction towards improving and customizing summary results.
Summary-Source Proposition-level Alignment: Task, Datasets and Supervised Baseline
Aligning sentences in a reference summary with their counterparts in source documents was shown as a useful auxiliary summarization task, notably for generating training data for salience detection.
Multi-Source Domain Adaptation for Text Classification via DistanceNet-Bandits
Next, we develop a DistanceNet model which uses these distance measures, or a mixture of these distance measures, as an additional loss function to be minimized jointly with the task's loss function, so as to achieve better unsupervised domain adaptation.
Continual and Multi-Task Architecture Search
Architecture search is the process of automatically learning the neural model or cell structure that best suits the given task.
Crowdsourcing Lightweight Pyramids for Manual Summary Evaluation
Conducting a manual evaluation is considered an essential part of summary evaluation methodology.
AutoSeM: Automatic Task Selection and Mixing in Multi-Task Learning
To address these issues, we present AutoSeM, a two-stage MTL pipeline, where the first stage automatically selects the most useful auxiliary tasks via a Beta-Bernoulli multi-armed bandit with Thompson Sampling, and the second stage learns the training mixing ratio of these selected auxiliary tasks via a Gaussian Process based Bayesian optimization framework.
Game-Based Video-Context Dialogue
Current dialogue systems focus more on textual and speech context knowledge and are usually based on two speakers.
Dynamic Multi-Level Multi-Task Learning for Sentence Simplification
In this work, we first present a strong pointer-copy mechanism based sequence-to-sequence sentence simplification model, and then improve its entailment and paraphrasing capabilities via multi-task learning with related auxiliary tasks of entailment and paraphrase generation.
Ranked #2 on
Text Simplification
on Newsela
Soft Layer-Specific Multi-Task Summarization with Entailment and Question Generation
An accurate abstractive summary of a document should contain all its salient information and should be logically entailed by the input document.
Ranked #33 on
Text Summarization
on GigaWord
Multi-Reward Reinforced Summarization with Saliency and Entailment
Abstractive text summarization is the task of compressing and rewriting a long document into a short summary while maintaining saliency, directed logical entailment, and non-redundancy.
Ranked #41 on
Abstractive Text Summarization
on CNN / Daily Mail
Towards Improving Abstractive Summarization via Entailment Generation
Abstractive summarization, the task of rewriting and compressing a document into a short summary, has achieved considerable success with neural sequence-to-sequence models.
Reinforced Video Captioning with Entailment Rewards
Sequence-to-sequence models have shown promising improvements on the temporal task of video captioning, but they optimize word-level cross-entropy loss during training.
Multi-Task Video Captioning with Video and Entailment Generation
Video captioning, the task of describing the content of a video, has seen some promising improvements in recent years with sequence-to-sequence models, but accurately learning the temporal and logical dynamics involved in the task still remains a challenge, especially given the lack of sufficient annotated data.
