Search Results for author:
Found 15 papers, 8 papers with code
Don’t Forget About Pronouns: Removing Gender Bias in Language Models Without Losing Factual Gender Information
The representations in large language models contain multiple types of gender information.
MYTE: Morphology-Driven Byte Encoding for Better and Fairer Multilingual Language Modeling
A major consideration in multilingual language modeling is how to best represent languages with diverse vocabularies and scripts.
Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models
Despite their popularity in non-English NLP, multilingual language models often underperform monolingual ones due to inter-language competition for model parameters.
Debiasing Algorithm through Model Adaptation
Large language models are becoming the go-to solution for the ever-growing number of tasks.
Exploring the Impact of Training Data Distribution and Subword Tokenization on Gender Bias in Machine Translation
We study the effect of tokenization on gender bias in machine translation, an aspect that has been largely overlooked in previous works.
Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languages
Multilingual language models have recently gained attention as a promising solution for representing multiple languages in a single model.
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions.
You Can Have Your Data and Balance It Too: Towards Balanced and Efficient Multilingual Models
Our method outperforms standard training methods in low-resource languages and retrains performance on high-resource languages while using the same amount of data.
Don't Forget About Pronouns: Removing Gender Bias in Language Models Without Losing Factual Gender Information
The representations in large language models contain multiple types of gender information.
A Balanced Data Approach for Evaluating Cross-Lingual Transfer: Mapping the Linguistic Blood Bank
We show that the choice of pretraining languages affects downstream cross-lingual transfer for BERT-based models.
Examining Cross-lingual Contextual Embeddings with Orthogonal Structural Probes
The evaluated information is encoded in a shared cross-lingual embedding space.
Introducing Orthogonal Constraint in Structural Probes
With the recent success of pre-trained models in NLP, a significant focus was put on interpreting their representations.
Gender Coreference and Bias Evaluation at WMT 2020
Our work presents the largest evidence for the phenomenon in more than 19 systems submitted to the WMT over four diverse target languages: Czech, German, Polish, and Russian.
Syntax Representation in Word Embeddings and Neural Networks -- A Survey
Neural networks trained on natural language processing tasks capture syntax even though it is not provided as a supervision signal.
Universal Dependencies according to BERT: both more specific and more general
This work focuses on analyzing the form and extent of syntactic abstraction captured by BERT by extracting labeled dependency trees from self-attentions.
