Search Results for author:
Found 26 papers, 16 papers with code
AnaLog: Testing Analytical and Deductive Logic Learnability in Language Models
We investigate the extent to which pre-trained language models acquire analytical and deductive logical reasoning capabilities as a side effect of learning word prediction.
Analysing Human Strategies of Information Transmission as a Function of Discourse Context
Speakers are thought to use rational information transmission strategies for efficient communication (Genzel and Charniak, 2002; Aylett and Turk, 2004; Jaeger and Levy, 2007).
Is Information Density Uniform in Task-Oriented Dialogues?
The Uniform Information Density principle states that speakers plan their utterances to reduce fluctuations in the density of the information transmitted.
From Language Models over Tokens to Language Models over Characters
Modern language models are internally -- and mathematically -- distributions over token strings rather than \emph{character} strings, posing numerous challenges for programmers building user applications on top of them.
Towards a Similarity-adjusted Surprisal Theory
Surprisal theory posits that the cognitive effort required to comprehend a word is determined by its contextual predictability, quantified as surprisal.
Surprise! Uniform Information Density Isn't the Whole Story: Predicting Surprisal Contours in Long-form Discourse
While these fluctuations can be viewed as theoretically uninteresting noise on top of a uniform target, another explanation is that UID is not the only functional pressure regulating information content in a language.
On the Proper Treatment of Tokenization in Psycholinguistics
The paper argues that token-level language models should be (approximately) marginalized into character-level language models before they are used in psycholinguistic studies to compute the surprisal of a region of interest; then, the marginalized character-level language model can be used to compute the surprisal of an arbitrary character substring, which we term a focal area, that the experimenter may wish to use as a predictor.
Generalized Measures of Anticipation and Responsivity in Online Language Processing
We introduce a generalization of classic information-theoretic measures of predictive uncertainty in online language processing, based on the simulation of expected continuations of incremental linguistic contexts.
LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks
There is an increasing trend towards evaluating NLP models with LLMs instead of human judgments, raising questions about the validity of these evaluations, as well as their reproducibility in the case of proprietary models.
LLM Agents in Interaction: Measuring Personality Consistency and Linguistic Alignment in Interacting Populations of Large Language Models
While both agent interaction and personalisation are vibrant topics in research on large language models (LLMs), there has been limited focus on the effect of language interaction on the behaviour of persona-conditioned LLM agents.
Attribution and Alignment: Effects of Local Context Repetition on Utterance Production and Comprehension in Dialogue
Language models are often used as the backbone of modern dialogue systems.
Information Value: Measuring Utterance Predictability as Distance from Plausible Alternatives
We present information value, a measure which quantifies the predictability of an utterance relative to a set of plausible alternatives.
Speaking the Language of Your Listener: Audience-Aware Adaptation via Plug-and-Play Theory of Mind
Inspired by psycholinguistic theories, we endow our speaker with the ability to adapt its referring expressions via a simulation module that monitors the effectiveness of planned utterances from the listener's perspective.
What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variability
In Natural Language Generation (NLG) tasks, for any input, multiple communicative goals are plausible, and any goal can be put into words, or produced, in multiple ways.
Interpretable Word Sense Representations via Definition Generation: The Case of Semantic Change Analysis
We propose using automatically generated natural language definitions of contextualised word usages as interpretable word and word sense representations.
Towards Pragmatic Production Strategies for Natural Language Generation Tasks
This position paper proposes a conceptual framework for the design of Natural Language Generation (NLG) systems that follow efficient and effective production strategies in order to achieve complex communicative goals.
Construction Repetition Reduces Information Rate in Dialogue
We hypothesise that speakers use construction repetition to mitigate information rate, leading to an overall decrease in utterance information content over the course of a dialogue.
State-of-the-art generalisation research in NLP: A taxonomy and review
We present a taxonomy for characterising and understanding generalisation research in NLP.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
Do Not Fire the Linguist: Grammatical Profiles Help Language Models Detect Semantic Change
In this work, we explore whether large pre-trained contextualised language models, a common tool for lexical semantic change detection, are sensitive to such morphosyntactic changes.
Grammatical Profiling for Semantic Change Detection
Semantics, morphology and syntax are strongly interdependent.
Refer, Reuse, Reduce: Generating Subsequent References in Visual and Conversational Contexts
We propose a generation model that produces referring utterances grounded in both the visual and the conversational context.
UiO-UvA at SemEval-2020 Task 1: Contextualised Embeddings for Lexical Semantic Change Detection
We apply contextualised word embeddings to lexical semantic change detection in the SemEval-2020 Shared Task 1.
Analysing Lexical Semantic Change with Contextualised Word Representations
This paper presents the first unsupervised approach to lexical semantic change that makes use of contextualised word representations.
Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information
We show that `diagnostic classifiers', trained to predict number from the internal states of a language model, provide a detailed understanding of how, when, and where this information is represented.
Semi-supervised emotion lexicon expansion with label propagation and specialized word embeddings
There exist two main approaches to automatically extract affective orientation: lexicon-based and corpus-based.
