Search Results for author:
Found 32 papers, 5 papers with code
Unsupervised multiple-choice question generation for out-of-domain Q&A fine-tuning
Pre-trained models have shown very good performances on a number of question answering benchmarks especially when fine-tuned on multiple question answering datasets at once.
Effective Data Augmentation for Sentence Classification Using One VAE per Class
In recent years, data augmentation has become an important field of machine learning.
RW-KD: Sample-wise Loss Terms Re-Weighting for Knowledge Distillation
Knowledge Distillation (KD) is extensively used in Natural Language Processing to compress the pre-training and task-specific fine-tuning phases of large neural language models.
A Methodology for Building a Diachronic Dataset of Semantic Shifts and its Application to QC-FR-Diac-V1.0, a Free Reference for French
Different algorithms have been proposed to detect semantic shifts (changes in a word meaning over time) in a diachronic corpus.
Refining an Almost Clean Translation Memory Helps Machine Translation
While recent studies have been dedicated to cleaning very noisy parallel corpora to improve Machine Translation training, we focus in this work on filtering a large and mostly clean Translation Memory.
Exploiting Domain-Specific Knowledge for Judgment Prediction Is No Panacea
Legal judgment prediction (LJP) usually consists in a text classification task aimed at predicting the verdict on the basis of the fact description.
EUROPA: A Legal Multilingual Keyphrase Generation Dataset
Keyphrase generation has primarily been explored within the context of academic research articles, with a particular focus on scientific domains and the English language.
Data Augmentation is Dead, Long Live Data Augmentation
Textual data augmentation (DA) is a prolific field of study where novel techniques to create artificial data are regularly proposed, and that has demonstrated great efficiency on small data settings, at least for text classification tasks.
On the importance of Data Scale in Pretraining Arabic Language Models
Pretraining monolingual language models have been proven to be vital for performance in Arabic Natural Language Processing (NLP) tasks.
LABO: Towards Learning Optimal Label Regularization via Bi-level Optimization
Label Smoothing (LS) is another simple, versatile and efficient regularization which can be applied to various supervised classification tasks.
Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging
Moreover, we observe that this simple optimization technique is able to outperform the state-of-the-art KD methods for compact models.
Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language Understanding
There is a growing body of work in recent years to develop pre-trained language models (PLMs) for the Arabic language.
CILDA: Contrastive Data Augmentation using Intermediate Layer Knowledge Distillation
Knowledge distillation (KD) is an efficient framework for compressing large-scale pre-trained language models.
JABER and SABER: Junior and Senior Arabic BERt
Language-specific pre-trained models have proven to be more accurate than multilingual ones in a monolingual evaluation setting, Arabic is no exception.
NATURE: Natural Auxiliary Text Utterances for Realistic Spoken Language Evaluation
We apply NATURE to common slot-filling and intent detection benchmarks and demonstrate that simple perturbations from the standard evaluation set by NATURE can deteriorate model performance significantly.
Pseudo Knowledge Distillation: Towards Learning Optimal Instance-specific Label Smoothing Regularization
Knowledge Distillation (KD) is an algorithm that transfers the knowledge of a trained, typically larger, neural network into another model under training.
Knowledge Distillation with Noisy Labels for Natural Language Understanding
Knowledge Distillation (KD) is extensively used to compress and deploy large pre-trained language models on edge devices for real-world applications.
RAIL-KD: RAndom Intermediate Layer Mapping for Knowledge Distillation
To address these problems, we propose a RAndom Intermediate Layer Knowledge Distillation (RAIL-KD) approach in which, intermediate layers from the teacher model are selected randomly to be distilled into the intermediate layers of the student model.
End-to-End Self-Debiasing Framework for Robust NLU Training
Existing Natural Language Understanding (NLU) models have been shown to incorporate dataset biases leading to strong performance on in-distribution (ID) test sets but poor performance on out-of-distribution (OOD) ones.
Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognition
In this work, we examine the ability of NER models to use contextual information when predicting the type of an ambiguous entity.
SC-LSTM: Learning Task-Specific Representations in Multi-Task Learning for Sequence Labeling
Multi-task learning (MTL) has been studied recently for sequence labeling.
WiRe57 : A Fine-Grained Benchmark for Open Information Extraction
We build a reference for the task of Open Information Extraction, on five documents.
Ranked #4 on
Open Information Extraction
on WiRe57
Extracting Parallel Sentences with Bidirectional Recurrent Neural Networks to Improve Machine Translation
Parallel sentence extraction is a task addressing the data sparsity problem found in multilingual natural language processing applications.
Robust Lexical Features for Improved Neural Network Named-Entity Recognition
While some features do remain in state-of-the-art systems, lexical features have been mostly discarded, with the exception of gazetteers.
Ranked #22 on
Named Entity Recognition (NER)
on Ontonotes v5 (English)
(using extra training data)
A Deep Neural Network Approach To Parallel Sentence Extraction
Parallel sentence extraction is a task addressing the data sparsity problem found in multilingual natural language processing applications.
Translating Implicit Discourse Connectives Based on Cross-lingual Annotation and Alignment
Implicit discourse connectives and relations are distributed more widely in Chinese texts, when translating into English, such connectives are usually translated explicitly.
BUCC 2017 Shared Task: a First Attempt Toward a Deep Learning Framework for Identifying Parallel Sentences in Comparable Corpora
This paper describes our participation in BUCC 2017 shared task: identifying parallel sentences in comparable corpora.
Texto4Science: a Quebec French Database of Annotated Short Text Messages
In October 2009, was launched the Quebec French part of the international sms4science project, called texto4science.
