Search Results for author:
Found 17 papers, 10 papers with code
An Embarrassingly Simple Method to Mitigate Undesirable Properties of Pretrained Language Model Tokenizers
We introduce FLOTA (Few Longest Token Approximation), a simple yet effective method to improve the tokenization of pretrained language models (PLMs).
Dialect prejudice predicts AI decisions about people's character, employability, and criminality
Here, we demonstrate that language models embody covert racism in the form of dialect prejudice: we extend research showing that Americans hold raciolinguistic stereotypes about speakers of African American English and find that language models have the same prejudice, exhibiting covert stereotypes that are more negative than any human stereotypes about African Americans ever experimentally recorded, although closest to the ones from before the civil rights movement.
Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models
Motivated by this discrepancy, we challenge the prevailing constrained evaluation paradigm for values and opinions in LLMs and explore more realistic unconstrained evaluations.
Graph-enhanced Large Language Models in Asynchronous Plan Reasoning
Reasoning about asynchronous plans is challenging since it requires sequential and parallel planning to optimize time costs.
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
Language models have become a critical technology to tackling a wide range of natural language processing tasks, yet many details about how the best-performing language models were developed are not reported.
Paloma: A Benchmark for Evaluating Language Model Fit
We invite submissions to our benchmark and organize results by comparability based on compliance with guidelines such as removal of benchmark contamination from pretraining.
Counting the Bugs in ChatGPT's Wugs: A Multilingual Investigation into the Morphological Capabilities of a Large Language Model
Large language models (LLMs) have recently reached an impressive level of linguistic capability, prompting comparisons with human language skills.
Unsupervised Detection of Contextualized Embedding Bias with Application to Ideology
We propose a fully unsupervised method to detect bias in contextualized embeddings.
The Better Your Syntax, the Better Your Semantics? Probing Pretrained Language Models for the English Comparative Correlative
Construction Grammar (CxG) is a paradigm from cognitive linguistics emphasising the connection between syntax and semantics.
CaMEL: Case Marker Extraction without Labels
We introduce CaMEL (Case Marker Extraction without Labels), a novel and challenging task in computational morphology that is especially relevant for low-resource languages.
Geographic Adaptation of Pretrained Language Models
While pretrained language models (PLMs) have been shown to possess a plethora of linguistic knowledge, the existing body of research has largely neglected extralinguistic knowledge, which is generally difficult to obtain by pretraining on text alone.
Modeling Ideological Salience and Framing in Polarized Online Groups with Graph Neural Networks and Structured Sparsity
The increasing polarization of online political discourse calls for computational tools that automatically detect and monitor ideological divides in social media.
Superbizarre Is Not Superb: Derivational Morphology Improves BERT's Interpretation of Complex Words
How does the input segmentation of pretrained language models (PLMs) affect their interpretations of complex words?
Dynamic Contextualized Word Embeddings
Static word embeddings that represent words by a single vector cannot capture the variability of word meaning in different linguistic and extralinguistic contexts.
A Graph Auto-encoder Model of Derivational Morphology
The auto-encoder models MWF in English surprisingly well by combining syntactic and semantic information with associative information from the mental lexicon.
Predicting the Growth of Morphological Families from Social and Linguistic Factors
We present the first study that examines the evolution of morphological families, i. e., sets of morphologically related words such as {``}trump{''}, {``}antitrumpism{''}, and {``}detrumpify{''}, in social media.
DagoBERT: Generating Derivational Morphology with a Pretrained Language Model
Can pretrained language models (PLMs) generate derivationally complex words?
