Search Results for author:
Found 17 papers, 8 papers with code
Clustering Examples in Multi-Dataset Benchmarks with Item Response Theory
In natural language processing, multi-dataset benchmarks for common tasks (e. g., SuperGLUE for natural language inference and MRQA for question answering) have risen in importance.
RAMP: Retrieval and Attribute-Marking Enhanced Prompting for Attribute-Controlled Translation
Attribute-controlled translation (ACT) is a subtask of machine translation that involves controlling stylistic or linguistic attributes (like formality and gender) of translation outputs.
(QA)$^2$: Question Answering with Questionable Assumptions
Naturally occurring information-seeking questions often contain questionable assumptions -- assumptions that are false or unverifiable.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
BBQ: A Hand-Built Bias Benchmark for Question Answering
It is well documented that NLP models learn social biases, but little work has been done on how these biases manifest in model outputs for applied tasks like question answering (QA).
Comparing Test Sets with Item Response Theory
Recent years have seen numerous NLP datasets introduced to evaluate the performance of fine-tuned models on natural language understanding tasks.
Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work?
However, we fail to observe more granular correlations between probing and target task performance, highlighting the need for further work on broad-coverage probing benchmarks.
English Intermediate-Task Training Improves Zero-Shot Cross-Lingual Transfer Too
Intermediate-task training---fine-tuning a pretrained model on an intermediate task before fine-tuning again on the target task---often improves model performance substantially on language understanding tasks in monolingual English settings.
Ranked #20 on
Zero-Shot Cross-Lingual Transfer
on XTREME
Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?
However, we fail to observe more granular correlations between probing and target task performance, highlighting the need for further work on broad-coverage probing benchmarks.
jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models
We introduce jiant, an open source toolkit for conducting multitask and transfer learning experiments on English NLU tasks.
Do Attention Heads in BERT Track Syntactic Dependencies?
We investigate the extent to which individual attention heads in pretrained transformer language models, such as BERT and RoBERTa, implicitly capture syntactic dependency relations.
Generalized Inner Loop Meta-Learning
Many (but not all) approaches self-qualifying as "meta-learning" in deep learning and reinforcement learning fit a common pattern of approximating the solution to a nested optimization problem.
Inducing Constituency Trees through Neural Machine Translation
Latent tree learning(LTL) methods learn to parse sentences using only indirect supervision from a downstream task.
Investigating BERT's Knowledge of Language: Five Analysis Methods with NPIs
We conclude that a variety of methods is necessary to reveal all relevant aspects of a model's grammatical knowledge in a given domain.
The Unbearable Weight of Generating Artificial Errors for Grammatical Error Correction
In recent years, sequence-to-sequence models have been very effective for end-to-end grammatical error correction (GEC).
Grammar Induction with Neural Language Models: An Unusual Replication
A substantial thread of recent work on latent tree learning has attempted to develop neural network models with parse-valued latent variables and train them on non-parsing tasks, in the hope of having them discover interpretable tree structure.
Training a Ranking Function for Open-Domain Question Answering
In recent years, there have been amazing advances in deep learning methods for machine reading.
