Search Results for author:
Found 61 papers, 24 papers with code
Evaluating the Evaluation Metrics for Style Transfer: A Case Study in Multilingual Formality Transfer
While the field of style transfer (ST) has been growing rapidly, it has been hampered by a lack of standardized practices for automatic evaluation.
JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correction
We present a new parallel corpus, JHU FLuency-Extended GUG corpus (JFLEG) for developing and evaluating grammatical error correction (GEC).
TGIF: A New Dataset and Benchmark on Animated GIF Description
The motivation for this work is to develop a testbed for image sequence description systems, where the task is to generate natural language descriptions for animated GIFs or video clips.
Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer
Style transfer is the task of automatically transforming a piece of text in one particular style into another.
Yara Parser: A Fast and Accurate Dependency Parser
At its fastest, Yara can parse about 4000 sentences per second when in greedy mode (1 beam).
GLEU Without Tuning
The GLEU metric was proposed for evaluating grammatical error corrections using n-gram overlap with a set of reference sentences, as opposed to precision/recall of specific annotated errors (Napoles et al., 2015).
Dialogue Act Classification with Context-Aware Self-Attention
Recent work in Dialogue Act classification has treated the task as a sequence labeling problem using hierarchical deep neural networks.
Ranked #3 on
Dialogue Act Classification
on Switchboard corpus
There's No Comparison: Reference-less Evaluation Metrics in Grammatical Error Correction
We show that reference-less grammaticality metrics correlate very strongly with human judgments and are competitive with the leading reference-based evaluation metrics.
This Email Could Save Your Life: Introducing the Task of Email Subject Line Generation
In this paper, we propose and study the task of email subject line generation: automatically generating an email subject line from the email body.
Ranked #2 on
Abstractive Text Summarization
on AESLC
Discourse Coherence in the Wild: A Dataset, Evaluation and Methods
To date there has been very little work on assessing discourse coherence methods on real-world data.
Ranked #2 on
Coherence Evaluation
on GCDC + RST - F1
Finding Good Conversations Online: The Yahoo News Annotated Comments Corpus
This work presents a dataset and annotation scheme for the new task of identifying {``}good{''} conversations that occur online, which we call ERICs: Engaging, Respectful, and/or Informative Conversations.
Event Extraction as Question Generation and Answering
In this paper, we propose QGA-EE, which enables a Question Generation (QG) model to generate questions that incorporate rich contextual information instead of using fixed templates.
Multimodal Categorization of Crisis Events in Social Media
Recent developments in image classification and natural language processing, coupled with the rapid growth in social media usage, have enabled fundamental advances in detecting breaking events around the world in real-time.
XFORMAL: A Benchmark for Multilingual Formality Style Transfer
We take the first step towards multilingual style transfer by creating and releasing XFORMAL, a benchmark of multiple formal reformulations of informal text in Brazilian Portuguese, French, and Italian.
Rhetoric, Logic, and Dialectic: Advancing Theory-based Argument Quality Assessment in Natural Language Processing
Though preceding work in computational argument quality (AQ) mostly focuses on assessing overall AQ, researchers agree that writers would benefit from feedback targeting individual dimensions of argumentation theory.
Creating a Domain-diverse Corpus for Theory-based Argument Quality Assessment
Computational models of argument quality (AQ) have focused primarily on assessing the overall quality or just one specific characteristic of an argument, such as its convincingness or its clarity.
Reassessing the Goals of Grammatical Error Correction: Fluency Instead of Grammaticality
The field of grammatical error correction (GEC) has grown substantially in recent years, with research directed at both evaluation metrics and improved system performance against those metrics.
CrisisLTLSum: A Benchmark for Local Crisis Event Timeline Extraction and Summarization
This paper presents CrisisLTLSum, the largest dataset of local crisis event timelines available to date.
BUMP: A Benchmark of Unfaithful Minimal Pairs for Meta-Evaluation of Faithfulness Metrics
The proliferation of automatic faithfulness metrics for summarization has produced a need for benchmarks to evaluate them.
A New Task and Dataset on Detecting Attacks on Human Rights Defenders
The ability to conduct retrospective analyses of attacks on human rights defenders over time and by location is important for humanitarian organizations to better understand historical or ongoing human rights violations and thus better manage the global impact of such events.
A Review of Human Evaluation for Style Transfer
However, in style transfer papers, we find that protocols for human evaluations are often underspecified and not standardized, which hampers the reproducibility of research in this field and progress toward better human and automatic evaluation methods.
Clustering of Social Media Messages for Humanitarian Aid Response during Crisis
Social media has quickly grown into an essential tool for people to communicate and express their needs during crisis events.
Journalistic Guidelines Aware News Image Captioning
The task of news article image captioning aims to generate descriptive and informative captions for news article images.
Explain then Rank: Scale Calibration of Neural Rankers Using Natural Language Explanations from Large Language Models
The process of scale calibration in ranking systems involves adjusting the outputs of rankers to correspond with significant qualities like click-through rates or relevance, crucial for mirroring real-world value and thereby boosting the system's effectiveness and reliability.
Detecting Sarcasm in Multimodal Social Platforms
Sarcasm is a peculiar form of sentiment expression, where the surface sentiment differs from the implied sentiment.
Humor in Collective Discourse: Unsupervised Funniness Detection in the New Yorker Cartoon Caption Contest
The New Yorker publishes a weekly captionless cartoon.
Is getting the right answer just about choosing the right words? The role of syntactically-informed features in short answer scoring
Developments in the educational landscape have spurred greater interest in the problem of automatically scoring short answer questions.
How do you correct run-on sentences it's not as easy as it seems
Run-on sentences are common grammatical mistakes but little research has tackled this problem to date.
An Empirical Analysis of Formality in Online Communication
This paper presents an empirical study of linguistic formality.
A Report on the 2017 Native Language Identification Shared Task
We believe this makes for a more interesting shared task while building on the methods and results from the previous two shared tasks.
GEC into the future: Where are we going and how do we get there?
The field of grammatical error correction (GEC) has made tremendous bounds in the last ten years, but new questions and obstacles are revealing themselves.
The Unbearable Weight of Generating Artificial Errors for Grammatical Error Correction
In recent years, sequence-to-sequence models have been very effective for end-to-end grammatical error correction (GEC).
Personalizing Grammatical Error Correction: Adaptation to Proficiency Level and L1
Grammar error correction (GEC) systems have become ubiquitous in a variety of software applications, and have started to approach human-level performance for some datasets.
Enabling Robust Grammatical Error Correction in New Domains: Data Sets, Metrics, and Analyses
Until now, grammatical error correction (GEC) has been primarily evaluated on text written by non-native English speakers, with a focus on student essays.
The ApposCorpus: A new multilingual, multi-domain dataset for factual appositive generation
News articles, image captions, product reviews and many other texts mention people and organizations whose name recognition could vary for different audiences.
GTN-ED: Event Detection Using Graph Transformer Networks
Recent works show that the graph structure of sentences, generated from dependency parsers, has potential for improving event detection.
Ol\'a, Bonjour, Salve! XFORMAL: A Benchmark for Multilingual Formality Style Transfer
We take the first step towards multilingual style transfer by creating and releasing XFORMAL, a benchmark of multiple formal reformulations of informal text in Brazilian Portuguese, French, and Italian.
A Novel Framework for Detecting Important Subevents from Crisis Events via Dynamic Semantic Graphs
Furthermore, we show that our approach significantly outperforms event detection baselines, highlighting the importance of aggregating information across tweets for our task.
An Exploration of Post-Editing Effectiveness in Text Summarization
Automatic summarization methods are efficient but can suffer from low quality.
Counterfactual Editing for Search Result Explanation
In this work, we aim to investigate the effectiveness of this perspective via proposing and evaluating counterfactual explanations for the task of SeRE.
Missing Information, Unresponsive Authors, Experimental Flaws: The Impossibility of Assessing the Reproducibility of Previous Human Evaluations in NLP
We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human evaluations in NLP more/less reproducible.
Little Giants: Exploring the Potential of Small LLMs as Evaluation Metrics in Summarization in the Eval4NLP 2023 Shared Task
This paper describes and analyzes our participation in the 2023 Eval4NLP shared task, which focuses on assessing the effectiveness of prompt-based techniques to empower Large Language Models to handle the task of quality estimation, particularly in the context of evaluating machine translations and summaries.
Dissecting users' needs for search result explanations
There is a growing demand for transparency in search engines to understand how search results are curated and to enhance users' trust.
