Search Results for author:
Found 21 papers, 10 papers with code
Quand être absent de mBERT n’est que le commencement : Gérer de nouvelles langues à l’aide de modèles de langues multilingues (When Being Unseen from mBERT is just the Beginning : Handling New Languages With Multilingual Language Models)
Dans ce travail, en comparant des modèles multilingues et monolingues, nous montrons que de tels modèles se comportent de multiples façons sur des langues inconnues.
MultiLexNorm: A Shared Task on Multilingual Lexical Normalization
This task is beneficial for downstream analysis, as it provides a way to harmonize (often spontaneous) linguistic variation.
Byte Latent Transformer: Patches Scale Better Than Tokens
We introduce the Byte Latent Transformer (BLT), a new byte-level LLM architecture that, for the first time, matches tokenization-based LLM performance at scale with significant improvements in inference efficiency and robustness.
Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models
We focus on mathematical reasoning and without in-language math data, facilitate cross-lingual transfer by composing language and math capabilities.
Spirit LM: Interleaved Spoken and Written Language Model
Our model is based on a 7B pretrained text language model that we extend to the speech modality by continuously training it on text and speech units.
Ranked #1 on
Language Modelling
on SALMon
(using extra training data)
Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning
It is also a general-purpose model that can do both text-to-image and image-to-text generation, allowing us to introduce self-contained contrastive decoding methods that produce high-quality outputs.
Ranked #2 on
Text-to-Image Generation
on COCO
The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants
We use this dataset to evaluate the capabilities of multilingual masked language models (MLMs) and large language models (LLMs).
The Gender-GAP Pipeline: A Gender-Aware Polyglot Pipeline for Gender Characterisation in 55 Languages
We showcase it to report gender representation in WMT training data and development data for the News task, confirming that current data is skewed towards masculine representation.
Evaluating and Modeling Attribution for Cross-Lingual Question Answering
Based on these models, we improve the attribution level of a cross-lingual question-answering system.
In What Languages are Generative Language Models the Most Formal? Analyzing Formality Distribution across Languages
Multilingual generative language models (LMs) are increasingly fluent in a large variety of languages.
Languages You Know Influence Those You Learn: Impact of Language Characteristics on Multi-Lingual Text-to-Text Transfer
For a given language, we are able to predict zero-shot performance, that increases on a logarithmic scale with the number of few-shot target language data points.
Cross-Lingual Open-Domain Question Answering with Answer Sentence Generation
Our cross-lingual generative system outperforms answer sentence selection baselines for all 5 languages and monolingual generative pipelines for three out of five languages studied.
First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT
Such transfer emerges by fine-tuning on a task of interest in one language and evaluating on a distinct language, not seen during the fine-tuning.
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models
Focusing on the latter, we show that this failure to transfer is largely related to the impact of the script used to write such languages.
Building a User-Generated Content North-African Arabizi Treebank: Tackling Hell
We introduce the first treebank for a romanized user-generated content variety of Algerian, a North-African Arabic dialect known for its frequent usage of code-switching.
Les mod\`eles de langue contextuels Camembert pour le fran\ccais : impact de la taille et de l'h\'et\'erog\'en\'eit\'e des donn\'ees d'entrainement (C AMEM BERT Contextual Language Models for French: Impact of Training Data Size and Heterogeneity )
L{'}utilisation pratique de ces mod{\`e}les {---} dans toutes les langues sauf l{'}anglais {---} {\'e}tait donc limit{\'e}e. La sortie r{\'e}cente de plusieurs mod{\`e}les monolingues fond{\'e}s sur BERT (Devlin et al., 2019), notamment pour le fran{\c{c}}ais, a d{\'e}montr{\'e} l{'}int{\'e}r{\^e}t de ces mod{\`e}les en am{\'e}liorant l{'}{\'e}tat de l{'}art pour toutes les t{\^a}ches {\'e}valu{\'e}es.
Establishing a New State-of-the-Art for French Named Entity Recognition
The French TreeBank developed at the University Paris 7 is the main source of morphosyntactic and syntactic annotations for French.
Can Multilingual Language Models Transfer to an Unseen Dialect? A Case Study on North African Arabizi
Building natural language processing systems for non standardized and low resource languages is a difficult challenge.
CamemBERT: a Tasty French Language Model
We show that the use of web crawled data is preferable to the use of Wikipedia data.
Ranked #1 on
Natural Language Inference
on XNLI French
Enhancing BERT for Lexical Normalization
In this article, focusing on User Generated Content (UGC), we study the ability of BERT to perform lexical normalisation.
ELMoLex: Connecting ELMo and Lexicon Features for Dependency Parsing
We augment the deep Biaffine (BiAF) parser (Dozat and Manning, 2016) with novel features to perform competitively: we utilize an indomain version of ELMo features (Peters et al., 2018) which provide context-dependent word representations; we utilize disambiguated, embedded, morphosyntactic features from lexicons (Sagot, 2018), which complements the existing feature set.
