Search Results for author:
Found 9 papers, 5 papers with code
Analyzing BERT’s Knowledge of Hypernymy via Prompting
The high performance of large pretrained language models (LLMs) such as BERT on NLP tasks has prompted questions about BERT’s linguistic capabilities, and how they differ from humans’.
A Fine-Grained Analysis of BERTScore
BERTScore, a recently proposed automatic metric for machine translation quality, uses BERT, a large pre-trained language model to evaluate candidate translations with respect to a gold translation.
ACT-Thor: A Controlled Benchmark for Embodied Action Understanding in Simulated Environments
This paves the way for a systematic way of evaluating embodied AI agents that understand grounded actions.
Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms
Most studies determine which edges belong in a LM's circuit by performing causal interventions on each edge independently, but this scales poorly with model size.
Do Pre-Trained Language Models Detect and Understand Semantic Underspecification? Ask the DUST!
In this work, we propose a novel Dataset of semantically Underspecified Sentences grouped by Type (DUST) and use it to study whether pre-trained language models (LMs) correctly identify and interpret underspecified sentences.
When Language Models Fall in Love: Animacy Processing in Transformer Language Models
However, we also show that even when presented with stories about atypically animate entities, such as a peanut in love, LMs adapt: they treat these entities as animate, though they do not adapt as well as humans.
Identifying and Adapting Transformer-Components Responsible for Gender Bias in an English Language Model
Language models (LMs) exhibit and amplify many types of undesirable biases learned from the training data, including gender bias.
ChapGTP, ILLC's Attempt at Raising a BabyLM: Improving Data Efficiency by Automatic Task Formation
We present the submission of the ILLC at the University of Amsterdam to the BabyLM challenge (Warstadt et al., 2023), in the strict-small track.
How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model
Concretely, we use mechanistic interpretability techniques to explain the (limited) mathematical abilities of GPT-2 small.
