Search Results for author:
Found 26 papers, 18 papers with code
LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond
To address this, we propose a new protocol for inconsistency detection benchmark creation and implement it in a 10-domain benchmark called SummEdits.
On Learning to Summarize with Large Language Models as References
Therefore, we investigate a new learning paradigm of text summarization models that considers the LLMs as the reference or the gold-standard oracle on commonly used summarization datasets such as the CNN/DailyMail dataset.
Towards Interpretable and Efficient Automatic Reference-Based Summarization Evaluation
Interpretability and efficiency are two important considerations for the adoption of neural automatic metrics.
Socratic Pretraining: Question-Driven Pretraining for Controllable Summarization
In long document controllable summarization, where labeled data is scarce, pretrained models struggle to adapt to the task and effectively respond to user queries.
Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation
4) We evaluate existing automatic metrics using the collected human annotations across evaluation protocols and demonstrate how our benchmark leads to more statistically stable and significant results.
Prompted Opinion Summarization with GPT-3.5
Large language models have shown impressive performance across a wide variety of tasks, including text summarization.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
We propose to use sentence-compression data to train the post-editing model to take a summary with extrinsic entity errors marked with special tokens and output a compressed, well-formed summary with those errors removed.
CREATIVESUMM: Shared Task on Automatic Summarization for Creative Writing
We detail the process of curating these datasets for the task, as well as the metrics used for the evaluation of the submissions.
FOLIO: Natural Language Reasoning with First-Order Logic
We present FOLIO, a human-annotated, open-domain, and logically complex and diverse dataset for reasoning in natural language (NL), equipped with first order logic (FOL) annotations.
Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors
The propensity of abstractive summarization systems to make factual errors has been the subject of significant study, including work on models to detect factual errors and annotation of errors in current systems' outputs.
QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization
Factual consistency is an essential quality of text summarization models in practical settings.
Exploring Neural Models for Query-Focused Summarization
Query-focused summarization (QFS) aims to produce summaries that answer particular questions of interest, enabling greater user control and personalization.
Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand
We therefore propose a generalization of leaderboards, bidimensional leaderboards (Billboards), that simultaneously tracks progress in language generation models and metrics for their evaluation.
AnswerSumm: A Manually-Curated Dataset and Pipeline for Answer Summarization
One goal of answer summarization is to produce a summary that reflects the range of answer perspectives.
CaPE: Contrastive Parameter Ensembling for Reducing Hallucination in Abstractive Summarization
Then, we fine-tune a base summarization model, which is trained on all training samples, on the clean (noisy) subset to obtain an \textit{expert} (\textit{anti-expert}) model.
ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining
While online conversations can cover a vast amount of information in many different formats, abstractive text summarization has primarily focused on modeling solely news articles.
Multi-Perspective Abstractive Answer Summarization
A major obstacle for multi-perspective, abstractive answer summarization is the absence of a dataset to provide supervision for producing such summaries.
Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation
Models pretrained with self-supervised objectives on large text corpora achieve state-of-the-art performance on English text summarization tasks.
SummEval: Re-evaluating Summarization Evaluation
The scarcity of comprehensive up-to-date studies on evaluation metrics for text summarization and the lack of consensus regarding evaluation protocols continue to inhibit progress.
Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering
Training a QA model on this data gives a relative improvement over a previous unsupervised model in F1 score on the SQuAD dataset by about 14%, and 20% when the answer is a named entity, achieving state-of-the-art performance on SQuAD for unsupervised QA.
ScisummNet: A Large Annotated Corpus and Content-Impact Models for Scientific Paper Summarization with Citation Networks
Scientific article summarization is challenging: large, annotated corpora are not available, and the summary should ideally include the article's impacts on research community.
Ranked #1 on
Scientific Document Summarization
on CL-SciSumm
Creating A Neural Pedagogical Agent by Jointly Learning to Review and Assess
Our model outperforms existing approaches over several metrics in predicting user response correctness, notably out-performing other methods on new users without large question-response histories.
Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model
Automatic generation of summaries from multiple news articles is a valuable tool as the number of online publications grows rapidly.
Ranked #5 on
Multi-Document Summarization
on Multi-News
What Should I Learn First: Introducing LectureBank for NLP Education and Prerequisite Chain Learning
The dataset will be useful for educational purposes such as lecture preparation and organization as well as applications such as reading list generation.
Sarcasm Analysis using Conversation Context
To address the first issue, we investigate several types of Long Short-Term Memory (LSTM) networks that can model both the conversation context and the current turn.
TutorialBank: A Manually-Collected Corpus for Prerequisite Chains, Survey Extraction and Resource Recommendation
The field of Natural Language Processing (NLP) is growing rapidly, with new research published daily along with an abundance of tutorials, codebases and other online resources.
