Search Results for author:
Found 29 papers, 17 papers with code
Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systems
The "Summary of a Haystack" (SummHay) task then requires a system to process the Haystack and generate, given a query, a summary that identifies the relevant insights and precisely cites the source documents.
Prompt Leakage effect and defense strategies for multi-turn LLM interactions
We measure the mitigation effect of 7 black-box defense strategies, along with finetuning an open-source model to defend against leakage attempts.
MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents
We release LLM-AggreFact, code for data synthesis, and models.
Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experiment
The interactive nature of Large Language Models (LLMs) theoretically allows models to refine and improve their answers, yet systematic analysis of the multi-turn behavior of LLMs remains limited.
Salespeople vs SalesBot: Exploring the Role of Educational Value in Conversational Recommender Systems
Making big purchases requires consumers to research or consult a salesperson to gain domain expertise.
Automatic and Human-AI Interactive Text Generation
In this tutorial, we focus on text-to-text generation, a class of natural language generation (NLG) tasks, that takes a piece of text as input and then generates a revision that is improved according to some specific criteria (e. g., readability or linguistic styles), while largely retaining the original meaning and the length of the text.
Beyond the Chat: Executable and Verifiable Text-Editing with LLMs
Conversational interfaces powered by Large Language Models (LLMs) have recently become a popular way to obtain feedback during document editing.
Art or Artifice? Large Language Models and the False Promise of Creativity
Inspired by the Torrance Test of Creative Thinking (TTCT), which measures creativity as a process, we use the Consensual Assessment Technique [3] and propose the Torrance Test of Creative Writing (TTCW) to evaluate creativity as a product.
Embrace Divergence for Richer Insights: A Multi-document Summarization Benchmark and a Case Study on Summarizing Diverse Information from News Articles
In this paper, we propose a new task of summarizing diverse information encountered in multiple news articles encompassing the same event.
XGen-7B Technical Report
Most open-source LLMs, on the other hand, are limited in their ability to support longer sequence lengths, which is a key requirement for many tasks that require inference over an input context.
Next Steps for Human-Centered Generative AI: A Technical Perspective
Through iterative, cross-disciplinary discussions, we define and propose next-steps for Human-centered Generative AI (HGAI).
Did You Read the Instructions? Rethinking the Effectiveness of Task Definitions in Instruction Learning
Large language models (LLMs) have shown impressive performance in following natural language instructions to solve unseen tasks.
SWiPE: A Dataset for Document-Level Simplification of Wikipedia Pages
Text simplification research has mostly focused on sentence-level simplification, even though many desirable edits - such as adding relevant background information or reordering content - may require document-level context.
LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond
To address this, we propose a new protocol for inconsistency detection benchmark creation and implement it in a 10-domain benchmark called SummEdits.
Designing and Evaluating Interfaces that Highlight News Coverage Diversity Using Discord Questions
In a second usability study, we developed and implemented a reading exercise with 95 novice news readers to measure exposure to coverage diversity.
Discord Questions: A Computational Approach To Diversity Analysis in News Coverage
There are many potential benefits to news readers accessing diverse sources.
Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors
We compare performance of state-of-the-art factuality metrics, including recent ChatGPT-based metrics, on this stratified benchmark and show that their performance varies significantly across different types of summarization models.
Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets
Precisely assessing the progress in natural language generation (NLG) tasks is challenging, and human evaluation to establish a preference in a model's output over another is often necessary.
Quiz Design Task: Helping Teachers Create Quizzes with Automated Question Generation
Question generation (QGen) models are often evaluated with standardized NLG metrics that are based on n-gram overlap.
NewsPod: Automatic and Interactive News Podcasts
News podcasts are a popular medium to stay informed and dive deep into news topics.
SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization
In this work, we revisit the use of NLI for inconsistency detection, finding that past work suffered from a mismatch in input granularity between NLI datasets (sentence-level), and inconsistency detection (document level).
MixQG: Neural Question Generation with Mixed Answer Types
Asking good questions is an essential ability for both human and machine intelligence.
Can Transformer Models Measure Coherence In Text: Re-Thinking the Shuffle Test
The Shuffle Test is the most common task to evaluate whether NLP models can measure coherence in text.
Can Transformer Models Measure Coherence In Text? Re-Thinking the Shuffle Test
The Shuffle Test is the most common task to evaluate whether NLP models can measure coherence in text.
Keep it Simple: Unsupervised Simplification of Multi-Paragraph Text
This work presents Keep it Simple (KiS), a new approach to unsupervised text simplification which learns to balance a reward across three properties: fluency, salience and simplicity.
What's The Latest? A Question-driven News Chatbot
This work describes an automatic news chatbot that draws content from a diverse set of news articles and creates conversations with a user about the news.
News Headline Grouping as a Challenging NLU Task
Recent progress in Natural Language Understanding (NLU) has seen the latest models outperform human performance on many standard tasks.
The Summary Loop: Learning to Write Abstractive Summaries Without Examples
This work presents a new approach to unsupervised abstractive summarization based on maximizing a combination of coverage and fluency for a given length constraint.
Ranked #51 on
Abstractive Text Summarization
on CNN / Daily Mail
newsLens: building and visualizing long-ranging news stories
We propose a method to aggregate and organize a large, multi-source dataset of news articles into a collection of major stories, and automatically name and visualize these stories in a working system.
