Search Results for author:
Found 16 papers, 6 papers with code
Twenty Years of Confusion in Human Evaluation: NLG Needs Evaluation Sheets and Standardised Definitions
Human assessment remains the most trusted form of evaluation in NLG, but highly diverse approaches and a proliferation of different quality criteria used by researchers make it difficult to compare results and draw conclusions across papers, with adverse implications for meta-evaluation and reproducibility.
Rome was built in 1776: A Case Study on Factual Correctness in Knowledge-Grounded Response Generation
We demonstrate the benefit of our Conv-FEVER dataset by showing that the models trained on this data perform reasonably well to detect factually inconsistent responses with respect to the provided knowledge through evaluation on our human annotated data.
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
We introduce GEM, a living benchmark for natural language Generation (NLG), its Evaluation, and Metrics.
Ranked #1 on
Extreme Summarization
on GEM-XSum
Abstractive Text Summarization
Cross-Lingual Abstractive Summarization
+5
Local Knowledge Powered Conversational Agents
State-of-the-art conversational agents have advanced significantly in conjunction with the use of large transformer-based language models.
Learning to Plan and Realize Separately for Open-Ended Dialogue Systems
Achieving true human-like ability to conduct a conversation remains an elusive goal for open-ended dialogue systems.
Understanding the Impact of Experiment Design for Evaluating Dialogue System Output
Evaluation of output from natural language generation (NLG) systems is typically conducted via crowdsourced human judgments.
Active Defense Against Social Engineering: The Case for Human Language Technology
We describe a system that supports natural language processing (NLP) components for active defenses against social engineering attacks.
The Panacea Threat Intelligence and Active Defense Platform
We describe Panacea, a system that supports natural language processing (NLP) components for active defenses against social engineering attacks.
Adaptation of a Lexical Organization for Social Engineering Detection and Response Generation
We present a paradigm for extensible lexicon development based on Lexical Conceptual Structure to support social engineering detection and response generation.
Detecting Asks in SE attacks: Impact of Linguistic and Structural Knowledge
Social engineers attempt to manipulate users into undertaking actions such as downloading malware by clicking links or providing access to money or sensitive information.
Studying the Effects of Cognitive Biases in Evaluation of Conversational Agents
To investigate, we conducted a between-subjects study with 77 crowdsourced workers to understand the role of cognitive biases, specifically anchoring bias, when humans are asked to evaluate the output of conversational agents.
Natural Language Generation Using Reinforcement Learning with External Rewards
We propose an approach towards natural language generation using a bidirectional encoder-decoder which incorporates external rewards through reinforcement learning (RL).
Emotional Neural Language Generation Grounded in Situational Contexts
Emotional language generation is one of the keys to human-like artificial intelligence.
Towards Best Experiment Design for Evaluating Dialogue System Output
To overcome the limitations of automated metrics (e. g. BLEU, METEOR) for evaluating dialogue systems, researchers typically use human judgments to provide convergent evidence.
I Stand With You: Using Emojis to Study Solidarity in Crisis Events
We study how emojis are used to express solidarity in social media in the context of two major crisis events - a natural disaster, Hurricane Irma in 2017 and terrorist attacks that occurred on November 2015 in Paris.
A Survey of Natural Language Generation Techniques with a Focus on Dialogue Systems - Past, Present and Future Directions
We provide a comprehensive review towards building open domain dialogue systems, an important application of natural language generation.
