Search Results for author:
Found 15 papers, 8 papers with code
Do Language Models Understand Anything? On the Ability of LSTMs to Understand Negative Polarity Items
In this paper, we attempt to link the inner workings of a neural language model to linguistic theory, focusing on a complex phenomenon well discussed in formal linguis- tics: (negative) polarity items.
Analysing Neural Language Models: Contextual Decomposition Reveals Default Reasoning in Number and Gender Assignment
Extensive research has recently shown that recurrent neural language models are able to process a wide range of grammatical phenomena.
diagNNose: A Library for Neural Activation Analysis
In this paper we introduce diagNNose, an open source library for analysing the activations of deep neural networks.
Language Modelling as a Multi-Task Problem
In this paper, we propose to study language modelling as a multi-task problem, bringing together three strands of research: multi-task learning, linguistics, and interpretability.
Attention vs non-attention for a Shapley-based explanation method
In this work, we consider Contextual Decomposition (CD) -- a Shapley-based input feature attribution method that has been shown to work well for recurrent NLP models -- and we test the extent to which it is useful for models that contain attention operations.
Language Models Use Monotonicity to Assess NPI Licensing
We investigate the semantic knowledge of language models (LMs), focusing on (1) whether these LMs create categories of linguistic environments based on their semantic monotonicity properties, and (2) whether these categories play a similar role in LMs as in human language understanding, using negative polarity item licensing as a case study.
Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations
We investigate the extent to which modern, neural language models are susceptible to structural priming, the phenomenon whereby the structure of a sentence makes the same structure more probable in a follow-up sentence.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
The Birth of Bias: A case study on the evolution of gender bias in an English language model
With full access to the data and to the model parameters as they change during every step while training, we can map in detail how the representation of gender develops, what patterns in the dataset drive this, and how the model's internal state relates to the bias in a downstream task (semantic textual similarity).
Feature Interactions Reveal Linguistic Structure in Language Models
We study feature interactions in the context of feature attribution methods for post-hoc interpretability.
Curriculum Learning with Adam: The Devil Is in the Wrong Details
We present a number of different case studies with different common hand-crafted and automated CL approaches to illustrate this phenomenon, and we find that none of them outperforms optimisation with only Adam with well-chosen hyperparameters.
DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformers
In recent years, many interpretability methods have been proposed to help interpret the internal states of Transformer-models, at different levels of precision and complexity.
ChapGTP, ILLC's Attempt at Raising a BabyLM: Improving Data Efficiency by Automatic Task Formation
We present the submission of the ILLC at the University of Amsterdam to the BabyLM challenge (Warstadt et al., 2023), in the strict-small track.
Transparency at the Source: Evaluating and Interpreting Language Models With Access to the True Distribution
With access to the underlying true source, our results show striking differences and outcomes in learning dynamics between different classes of words.
Attribution and Alignment: Effects of Local Context Repetition on Utterance Production and Comprehension in Dialogue
Language models are often used as the backbone of modern dialogue systems.
