Search Results for author:
Found 50 papers, 31 papers with code
proScript: Partially Ordered Scripts Generation
Scripts – prototypical event sequences describing everyday activities – have been shown to help understand narratives by providing expectations, resolving ambiguity, and filling in unstated information.
J-UniMorph: Japanese Morphological Annotation through the Universal Feature Schema
We introduce a Japanese Morphology dataset, J-UniMorph, developed based on the UniMorph feature schema.
Test-time Augmentation for Factual Probing
Factual probing is a method that uses prompts to test if a language model "knows" certain world knowledge facts.
PlaSma: Making Small Language Models Better Procedural Knowledge Models for (Counterfactual) Planning
In addition, we introduce a novel task, Counterfactual Planning, that requires a revision of a plan to cope with a counterfactual situation.
Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinations
In this work, we evaluate LLM APIs (ChatGPT, GPT-3, and GPT-4) on the Japanese national medical licensing examinations from the past five years, including the current year.
Causal schema induction for knowledge discovery
In this work, we investigate how to apply schema induction models to the task of knowledge discovery for enhanced search of English-language news texts.
Analyzing the Performance of GPT-3.5 and GPT-4 in Grammatical Error Correction
GPT-3 and GPT-4 models are powerful, achieving high performance on a variety of Natural Language Processing tasks.
Empirical Investigation of Neural Symbolic Reasoning Strategies
Neural reasoning accuracy improves when generating intermediate reasoning steps.
Do Deep Neural Networks Capture Compositionality in Arithmetic Reasoning?
Compositionality is a pivotal property of symbolic reasoning.
I2D2: Inductive Knowledge Distillation with NeuroLogic and Self-Imitation
Here, we investigate an alternative that a priori seems impossible: can smaller language models (e. g., GPT-2) win over models that are orders of magnitude larger and better (e. g., GPT-3), if powered with novel commonsense distillation algorithms?
RealTime QA: What's the Answer Right Now?
We introduce REALTIME QA, a dynamic question answering (QA) platform that announces questions and evaluates systems on a regular basis (weekly in this version).
Towards Automated Document Revision: Grammatical Error Correction, Fluency Edits, and Beyond
Natural language processing technology has rapidly improved automated grammatical error correction tasks, and the community begins to explore document-level revision as one of the next challenges.
Twist Decoding: Diverse Generators Guide Each Other
Our extensive evaluations on machine translation and scientific paper summarization demonstrate that Twist decoding substantially outperforms each model decoded in isolation over various scenarios, including cases where domain-specific and general-purpose models are both available.
ELQA: A Corpus of Metalinguistic Questions and Answers about English
We present ELQA, a corpus of questions and answers in and about the English language.
A Call for Clarity in Beam Search: How It Works and When It Stops
Based on this finding, we introduce a patience factor, a simple modification to this beam decoding implementation, that generalizes the stopping criterion and provides flexibility to the depth of search.
Interscript: A dataset for interactive learning of scripts through error feedback
We present a new dataset, Interscript, containing user feedback on a deployed model that generates complex everyday tasks.
Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand
We therefore propose a generalization of leaderboards, bidimensional leaderboards (Billboards), that simultaneously tracks progress in language generation models and metrics for their evaluation.
Transparent Human Evaluation for Image Captioning
We establish THumB, a rubric-based human evaluation protocol for image captioning models.
Can Machines Learn Morality? The Delphi Experiment
As AI systems become increasingly powerful and pervasive, there are growing concerns about machines' morality or a lack thereof.
Improving Neural Model Performance through Natural Language Feedback on Their Explanations
A class of explainable NLP models for reasoning tasks support their decisions by generating free-form or structured explanations, but what happens when these supporting structures contain errors?
proScript: Partially Ordered Scripts Generation via Pre-trained Language Models
Scripts - standardized event sequences describing typical everyday activities - have been shown to help understand narratives by providing expectations, resolving ambiguity, and filling in unstated information.
GrammarTagger: A Multilingual, Minimally-Supervised Grammar Profiler for Language Education
We present GrammarTagger, an open-source grammar profiler which, given an input text, identifies grammatical features useful for language education.
A Dataset for Tracking Entities in Open Domain Procedural Text
Our solution is a new task formulation where given just a procedural text as input, the task is to generate a set of state change tuples(entity, at-tribute, before-state, after-state)for each step, where the entity, attribute, and state values must be predicted from an open vocabulary.
COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs
Next, we show that ATOMIC 2020 is better suited for training knowledge models that can generate accurate, representative knowledge for new, unseen entities and events.
WIQA: A dataset for ``What if...'' reasoning over procedural text
We introduce WIQA, the first large-scale dataset of {``}What if...{''} questions over procedural text.
The Universal Decompositional Semantics Dataset and Decomp Toolkit
We present the Universal Decompositional Semantics (UDS) dataset (v1. 0), which is bundled with the Decomp toolkit (v0. 1).
WIQA: A dataset for "What if..." reasoning over procedural text
We introduce WIQA, the first large-scale dataset of "What if..." questions over procedural text.
Uncertain Natural Language Inference
We introduce Uncertain Natural Language Inference (UNLI), a refinement of Natural Language Inference (NLI) that shifts away from categorical labels, targeting instead the direct prediction of subjective probability assessments.
WinoGrande: An Adversarial Winograd Schema Challenge at Scale
The key steps of the dataset construction consist of (1) a carefully designed crowdsourcing procedure, followed by (2) systematic bias reduction using a novel AfLite algorithm that generalizes human-detectable word associations to machine-detectable embedding associations.
Ranked #9 on
Coreference Resolution
on Winograd Schema Challenge
Efficient Online Scalar Annotation with Bounded Support
We describe a novel method for efficiently eliciting scalar annotations for dataset construction and system quality estimation by human judgments.
GEC into the future: Where are we going and how do we get there?
The field of grammatical error correction (GEC) has made tremendous bounds in the last ten years, but new questions and obstacles are revealing themselves.
Grammatical Error Correction with Neural Reinforcement Learning
We propose a neural encoder-decoder model with reinforcement learning (NRL) for grammatical error correction (GEC).
Error-repair Dependency Parsing for Ungrammatical Texts
We propose a new dependency parsing scheme which jointly parses a sentence and repairs grammatical errors by extending the non-directional transition-based formalism of Goldberg and Elhadad (2010) with three additional actions: SUBSTITUTE, DELETE, INSERT.
JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correction
We present a new parallel corpus, JHU FLuency-Extended GUG corpus (JFLEG) for developing and evaluating grammatical error correction (GEC).
There's No Comparison: Reference-less Evaluation Metrics in Grammatical Error Correction
We show that reference-less grammaticality metrics correlate very strongly with human judgments and are competitive with the leading reference-based evaluation metrics.
Robsut Wrod Reocginiton via semi-Character Recurrent Neural Network
Inspired by the findings from the Cmabrigde Uinervtisy effect, we propose a word recognition model based on a semi-character level recurrent neural network (scRNN).
GLEU Without Tuning
The GLEU metric was proposed for evaluating grammatical error corrections using n-gram overlap with a set of reference sentences, as opposed to precision/recall of specific annotated errors (Napoles et al., 2015).
Reassessing the Goals of Grammatical Error Correction: Fluency Instead of Grammaticality
The field of grammatical error correction (GEC) has grown substantially in recent years, with research directed at both evaluation metrics and improved system performance against those metrics.
