Search Results for author:
Found 47 papers, 17 papers with code
BERTrade: Using Contextual Embeddings to Parse Old French
The successes of contextual word embeddings learned by training large-scale language models, while remarkable, have mostly occurred for languages where significant amounts of raw texts are available and where annotated data in downstream tasks have a relatively regular spelling.
Can Character-based Language Models Improve Downstream Task Performances In Low-Resource And Noisy Language Scenarios?
Recent impressive improvements in NLP, largely based on the success of contextual neural language models, have been mostly demonstrated on at most a couple dozen high- resource languages.
Automatic Normalisation of Early Modern French
Spelling normalisation is a useful step in the study and analysis of historical language texts, whether it is manual analysis by experts or automatic analysis using downstream natural language processing (NLP) tools.
Complex Labelling and Similarity Prediction in Legal Texts: Automatic Analysis of France’s Court of Cassation Rulings
The first step in the detection of divergences is to detect similar cases, which is currently done manually by experts.
From FreEM to D’AlemBERT: a Large Corpus and a Language Model for Early Modern French
anguage models for historical states of language are becoming increasingly important to allow the optimal digitisation and analysis of old textual sources.
Le projet FREEM : ressources, outils et enjeux pour l’étude du français d’Ancien Régime (The F RE EM project: Resources, tools and challenges for the study of Ancien Régime French)
En dépit de leur qualité certaine, les ressources et outils disponibles pour l’analyse du français d’Ancien Régime ne sont plus à même de répondre aux enjeux de la recherche en linguistique et en littérature pour cette période.
Probing Multilingual Cognate Prediction Models
Character-based neural machine translation models have become the reference models for cognate prediction, a historical linguistics task.
Quand être absent de mBERT n’est que le commencement : Gérer de nouvelles langues à l’aide de modèles de langues multilingues (When Being Unseen from mBERT is just the Beginning : Handling New Languages With Multilingual Language Models)
Dans ce travail, en comparant des modèles multilingues et monolingues, nous montrons que de tels modèles se comportent de multiples façons sur des langues inconnues.
Why do small language models underperform? Studying Language Model Saturation via the Softmax Bottleneck
In this paper, we find that such saturation can be explained by a mismatch between the hidden dimension of smaller models and the high rank of the target contextual probability distribution.
Making Sentence Embeddings Robust to User-Generated Content
NLP models have been known to perform poorly on user-generated content (UGC), mainly because it presents a lot of lexical variations and deviates from the standard texts on which most of these models were trained.
On the Scaling Laws of Geographical Representation in Language Models
Language models have long been shown to embed geographical information in their hidden representations.
Anisotropy Is Inherent to Self-Attention in Transformers
The representation degeneration problem is a phenomenon that is widely observed among self-supervised learning methods based on Transformers.
Modular Speech-to-Text Translation for Zero-Shot Cross-Modal Transfer
Recent research has shown that independently trained encoders and decoders, combined through a shared fixed-size representation, can achieve competitive performance in speech-to-text translation.
From Text to Source: Results in Detecting Large Language Model-Generated Content
The research also explores Model Attribution, encompassing source model identification, model family, and model size classification, in addition to quantization and watermarking detection.
Headless Language Models: Learning without Predicting with Contrastive Weight Tying
Self-supervised pre-training of language models usually consists in predicting probability distributions over extensive token vocabularies.
SONAR: Sentence-Level Multimodal and Language-Agnostic Representations
Our single text encoder, covering 200 languages, substantially outperforms existing sentence embeddings such as LASER3 and LabSE on the xsim and xsim++ multilingual similarity search tasks.
Is Anisotropy Inherent to Transformers?
The representation degeneration problem is a phenomenon that is widely observed among self-supervised learning methods based on Transformers.
Towards a Robust Detection of Language Model Generated Text: Is ChatGPT that Easy to Detect?
This paper proposes a methodology for developing and evaluating ChatGPT detectors for French text, with a focus on investigating their robustness on out-of-domain data and against common attack schemes.
Data-Efficient French Language Modeling with CamemBERTa
In this paper, we introduce CamemBERTa, a French DeBERTa model that builds upon the DeBERTaV3 architecture and training objective.
When your Cousin has the Right Connections: Unsupervised Bilingual Lexicon Induction for Related Data-Imbalanced Languages
Most existing approaches for unsupervised bilingual lexicon induction (BLI) depend on good quality static or contextual embeddings requiring large monolingual corpora for both languages.
Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluation
One of the major challenges of machine translation (MT) is ambiguity, which can in some cases be resolved by accompanying context such as images.
MANTa: Efficient Gradient-Based Tokenization for Robust End-to-End Language Modeling
The resulting system offers a trade-off between the expressiveness of byte-level models and the speed of models trained using subword tokenization.
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions.
SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations
We present SpeechMatrix, a large-scale multilingual corpus of speech-to-speech translations mined from real speech of European Parliament recordings.
DP-Parse: Finding Word Boundaries from Raw Speech with an Instance Lexicon
Finding word boundaries in continuous speech is challenging as there is little or no equivalent of a 'space' delimiter between words.
MaskEval: Weighted MLM-Based Evaluation for Text Summarization and Simplification
In text summarization and simplification, system outputs must be evaluated along multiple dimensions such as relevance, factual consistency, fluency, and grammaticality, and a wide range of possible outputs could be of high quality.
T-Modules: Translation Modules for Zero-Shot Cross-Modal Machine Translation
We present a new approach to perform zero-shot cross-modal transfer between speech and text for translation tasks.
UniMorph 4.0: Universal Morphology
The project comprises two major thrusts: a language-independent feature schema for rich morphological annotation and a type-level resource of annotated data in diverse languages realizing that schema.
From FreEM to D'AlemBERT: a Large Corpus and a Language Model for Early Modern French
Because these historical states are at the same time more complex to process and more scarce in the corpora available, specific efforts are necessary to train natural language processing (NLP) tools adapted to the data.
Towards a Cleaner Document-Oriented Multilingual Crawled Corpus
The need for raw large raw corpora has dramatically increased in recent years with the introduction of transfer learning and semi-supervised learning methods to Natural Language Processing.
Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP
What are the units of text that we want to model?
Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios?
Recent impressive improvements in NLP, largely based on the success of contextual neural language models, have been mostly demonstrated on at most a couple dozen high-resource languages.
Rethinking Automatic Evaluation in Sentence Simplification
In the context of Sentence Simplification, this is particularly challenging: the task requires by nature to replace complex words with simpler ones that shares the same meaning.
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
With the success of large-scale pre-training and multilingual modeling in Natural Language Processing (NLP), recent years have seen a proliferation of large, web-mined text datasets covering hundreds of languages.
First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT
Such transfer emerges by fine-tuning on a task of interest in one language and evaluating on a distinct language, not seen during the fine-tuning.
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models
Focusing on the latter, we show that this failure to transfer is largely related to the impact of the script used to write such languages.
Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering
Coupled with the availability of large scale datasets, deep learning architectures have enabled rapid progress on the Question Answering task.
A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages
They actually equal or improve the current state of the art in tagging and parsing for all five languages.
Establishing a New State-of-the-Art for French Named Entity Recognition
The French TreeBank developed at the University Paris 7 is the main source of morphosyntactic and syntactic annotations for French.
MUSS: Multilingual Unsupervised Sentence Simplification by Mining Paraphrases
Progress in sentence simplification has been hindered by a lack of labeled parallel simplification data, particularly in languages other than English.
Ranked #2 on
Text Simplification
on ASSET
ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations
Furthermore, we motivate the need for developing better methods for automatic evaluation using ASSET, since we show that current popular metrics may not be suitable when multiple simplification transformations are performed.
Neural language modeling of free word order argument structure
Neural language models trained with a predictive or masked objective have proven successful at capturing short and long distance syntactic dependencies.
CamemBERT: a Tasty French Language Model
We show that the use of web crawled data is preferable to the use of Wikipedia data.
Ranked #1 on
Dependency Parsing
on French GSD
Controllable Sentence Simplification
Text simplification aims at making a text easier to read and understand by simplifying grammar and structure while keeping the underlying information identical.
Ranked #3 on
Text Simplification
on ASSET
Reference-less Quality Estimation of Text Simplification Systems
We show that n-gram-based MT metrics such as BLEU and METEOR correlate the most with human judgment of grammaticality and meaning preservation, whereas simplicity is best evaluated by basic length-based metrics.
External Lexical Information for Multilingual Part-of-Speech Tagging
Morphosyntactic lexicons and word vector representations have both proven useful for improving the accuracy of statistical part-of-speech taggers.
