Search Results for author:
Found 8 papers, 3 papers with code
Towards Privacy-Aware Sign Language Translation at Scale
A major impediment to the advancement of sign language translation (SLT) is data scarcity.
Text Rendering Strategies for Pixel Language Models
Pixel-based language models process text rendered as images, which allows them to handle any script, making them a promising approach to open vocabulary language modelling.
PHD: Pixel-Based Language Modeling of Historical Documents
We then pre-train our model, PHD, on a combination of synthetic scans and real historical newspapers from the 1700-1900 period.
Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Models
Language models such as mBERT, XLM-R, and BLOOM aim to achieve multilingual generalization or compression to facilitate transfer to a large number of (potentially unseen) languages.
Language Modelling with Pixels
We pretrain the 86M parameter PIXEL model on the same English data as BERT and evaluate on syntactic and semantic tasks in typologically diverse languages, including various non-Latin scripts.
Ranked #1 on
Named Entity Recognition (NER)
on MasakhaNER
Challenges and Strategies in Cross-Cultural NLP
Various efforts in the Natural Language Processing (NLP) community have been made to accommodate linguistic diversity and serve speakers of many different languages.
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models
In this work, we provide a systematic and comprehensive empirical comparison of pretrained multilingual language models versus their monolingual counterparts with regard to their monolingual task performance.
PuzzLing Machines: A Challenge on Learning From Small Data
To expose this problem in a new light, we introduce a challenge on learning from small data, PuzzLing Machines, which consists of Rosetta Stone puzzles from Linguistic Olympiads for high school students.
