Search Results for author:
Found 22 papers, 18 papers with code
What’s in Your Head? Emergent Behaviour in Multi-Task Transformer Models
In this work, we examine the behaviour of non-target heads, that is, the output of heads when given input that belongs to a different task than the one they were trained for.
Jump to Conclusions: Short-Cutting Transformers With Linear Transformations
Moreover, in the context of language modeling, our method allows "peeking" into early layer representations of GPT-2 and BERT, showing that often LMs already predict the final output in early layers.
Crawling the Internal Knowledge-Base of Language Models
Here, we propose to address this goal by extracting a knowledge-graph of facts from a given language model.
Understanding Transformer Memorization Recall Through Idioms
In this work, we offer the first methodological framework for probing and characterizing recall of memorized sequences in transformer LMs.
Analyzing Transformers in Embedding Space
In this work, we present a theoretical analysis where all parameters of a trained Transformer are interpreted by projecting them into the embedding space, that is, the space of vocabulary items they operate on.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
Don't Blame the Annotator: Bias Already Starts in the Annotation Instructions
In this work, we hypothesize that annotators pick up on patterns in the crowdsourcing instructions, which bias them to write many similar examples that are then over-represented in the collected data.
Inferring Implicit Relations in Complex Questions with Language Models
A prominent challenge for modern language understanding systems is the ability to answer implicit reasoning questions, where the required reasoning steps for answering the question are not mentioned in the text explicitly.
LM-Debugger: An Interactive Tool for Inspection and Intervention in Transformer-Based Language Models
The opaque nature and unexplained behavior of transformer-based language models (LMs) have spurred a wide interest in interpreting their predictions.
Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space
Transformer-based language models (LMs) are at the core of modern NLP, but their internal prediction construction process is opaque and largely not understood.
SCROLLS: Standardized CompaRison Over Long Language Sequences
NLP benchmarks have largely focused on short texts, such as sentences and paragraphs, even though long texts comprise a considerable amount of natural language in the wild.
Ranked #8 on
Long-range modeling
on SCROLLS
Break, Perturb, Build: Automatic Perturbation of Reasoning Paths Through Question Decomposition
We evaluate a range of RC models on our evaluation sets, which reveals large performance gaps on generated examples compared to the original data.
What's in your Head? Emergent Behaviour in Multi-Task Transformer Models
In this work, we examine the behaviour of non-target heads, that is, the output of heads when given input that belongs to a different task than the one they were trained for.
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies
A key limitation in current datasets for multi-hop reasoning is that the required steps for answering the question are mentioned in it explicitly.
Transformer Feed-Forward Layers Are Key-Value Memories
Feed-forward layers constitute two-thirds of a transformer model's parameters, yet their role in the network remains under-explored.
Injecting Numerical Reasoning Skills into Language Models
In this work, we show that numerical reasoning is amenable to automatic data generation, and thus one can inject this skill into pre-trained LMs, by generating large amounts of data, and training in a multi-task setup.
Ranked #8 on
Question Answering
on DROP Test
Break It Down: A Question Understanding Benchmark
Understanding natural language questions entails the ability to break down a question into the requisite steps for computing its answer.
Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets
Crowdsourcing has been the prevalent paradigm for creating natural language understanding datasets in recent years.
DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusion
We author a set of rules for identifying a diverse set of discourse phenomena in raw text, and decomposing the text into two independent sentences.
Emergence of Communication in an Interactive World with Consistent Speakers
Training agents to communicate with one another given task-based supervision only has attracted considerable attention recently, due to the growing interest in developing models for human-agent interaction.
Learning to Search in Long Documents Using Document Structure
Reading comprehension models are based on recurrent neural networks that sequentially process the document tokens.
Evaluating Semantic Parsing against a Simple Web-based Question Answering Model
Semantic parsing shines at analyzing complex natural language that involves composition and computation over multiple pieces of evidence.
Ranked #1 on
Question Answering
on COMPLEXQUESTIONS
