Search Results for author:
Found 13 papers, 7 papers with code
Different Tokenization Schemes Lead to Comparable Performance in Spanish Number Agreement
The relationship between language model tokenization and performance is an open area of research.
Detecting Hallucination and Coverage Errors in Retrieval Augmented Generation for Controversial Topics
We explore a strategy to handle controversial topics in LLM-based chatbots based on Wikipedia's Neutral Point of View (NPOV) principle: acknowledge the absence of a single true answer and surface multiple perspectives.
A Bit of a Problem: Measurement Disparities in Dataset Sizes Across Languages
We release a tool to obtain byte premiums for any two languages, enabling comparisons of dataset sizes across languages for more equitable multilingual model development and data practices.
When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages
However, concrete evidence for the effects of multilinguality on language modeling performance in individual languages remains scarce.
Structural Priming Demonstrates Abstract Grammatical Representations in Multilingual Language Models
We measure crosslingual structural priming in large language models, comparing model behavior to human experimental results from eight crosslingual experiments covering six languages, and four monolingual structural priming experiments in three non-English languages.
Crosslingual Structural Priming and the Pre-Training Dynamics of Bilingual Language Models
Do multilingual language models share abstract grammatical representations across languages, and if so, when do these develop?
Characterizing Learning Curves During Language Model Pre-Training: Learning, Forgetting, and Stability
We quantify the final surprisal, within-run variability, age of acquisition, forgettability, and cross-run variability of learning curves for individual tokens in context.
Characterizing and Measuring Linguistic Dataset Drift
In this paper, we propose three dimensions of linguistic dataset drift: vocabulary, structural, and semantic drift.
Language Model Behavior: A Comprehensive Survey
Transformer language models have received widespread public attention, yet their generated text is often surprising even to NLP researchers.
The Geometry of Multilingual Language Model Representations
The subspace means differ along language-sensitive axes that are relatively stable throughout middle layers, and these axes encode information such as token vocabularies.
Word Acquisition in Neural Language Models
We investigate how neural language models acquire individual words during training, extracting learning curves and ages of acquisition for over 600 words on the MacArthur-Bates Communicative Development Inventory (Fenson et al., 2007).
Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models
In this paper, we detail the relationship between convolutions and self-attention in natural language tasks.
Encodings of Source Syntax: Similarities in NMT Representations Across Target Languages
We train neural machine translation (NMT) models from English to six target languages, using NMT encoder representations to predict ancestor constituent labels of source language words.
