Search Results for author:
Found 20 papers, 4 papers with code
Universal-KD: Attention-based Output-Grounded Intermediate Layer Knowledge Distillation
Intermediate layer matching is shown as an effective approach for improving knowledge distillation (KD).
RW-KD: Sample-wise Loss Terms Re-Weighting for Knowledge Distillation
Knowledge Distillation (KD) is extensively used in Natural Language Processing to compress the pre-training and task-specific fine-tuning phases of large neural language models.
On the importance of Data Scale in Pretraining Arabic Language Models
Pretraining monolingual language models have been proven to be vital for performance in Arabic Natural Language Processing (NLP) tasks.
AraMUS: Pushing the Limits of Data and Model Scale for Arabic Natural Language Processing
Developing monolingual large Pre-trained Language Models (PLMs) is shown to be very successful in handling different tasks in Natural Language Processing (NLP).
Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language Understanding
There is a growing body of work in recent years to develop pre-trained language models (PLMs) for the Arabic language.
CILDA: Contrastive Data Augmentation using Intermediate Layer Knowledge Distillation
Knowledge distillation (KD) is an efficient framework for compressing large-scale pre-trained language models.
JABER and SABER: Junior and Senior Arabic BERt
Language-specific pre-trained models have proven to be more accurate than multilingual ones in a monolingual evaluation setting, Arabic is no exception.
NATURE: Natural Auxiliary Text Utterances for Realistic Spoken Language Evaluation
We apply NATURE to common slot-filling and intent detection benchmarks and demonstrate that simple perturbations from the standard evaluation set by NATURE can deteriorate model performance significantly.
RAIL-KD: RAndom Intermediate Layer Mapping for Knowledge Distillation
To address these problems, we propose a RAndom Intermediate Layer Knowledge Distillation (RAIL-KD) approach in which, intermediate layers from the teacher model are selected randomly to be distilled into the intermediate layers of the student model.
Knowledge Distillation with Noisy Labels for Natural Language Understanding
Knowledge Distillation (KD) is extensively used to compress and deploy large pre-trained language models on edge devices for real-world applications.
End-to-End Self-Debiasing Framework for Robust NLU Training
Existing Natural Language Understanding (NLU) models have been shown to incorporate dataset biases leading to strong performance on in-distribution (ID) test sets but poor performance on out-of-distribution (OOD) ones.
Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognition
In this work, we examine the ability of NER models to use contextual information when predicting the type of an ambiguous entity.
Towards Zero-Shot Knowledge Distillation for Natural Language Processing
Knowledge Distillation (KD) is a common knowledge transfer algorithm used for model compression across a variety of deep learning based natural language processing (NLP) solutions.
SEDAR: a Large Scale French-English Financial Domain Parallel Corpus
This paper describes the acquisition, preprocessing and characteristics of SEDAR, a large scale English-French parallel corpus for the financial domain.
Contextualized Word Representations from Distant Supervision with and for NER
We describe a special type of deep contextualized word representation that is learned from distant supervision annotations and dedicated to named entity recognition.
Robust Lexical Features for Improved Neural Network Named-Entity Recognition
While some features do remain in state-of-the-art systems, lexical features have been mostly discarded, with the exception of gazetteers.
Ranked #22 on
Named Entity Recognition (NER)
on Ontonotes v5 (English)
(using extra training data)
WiNER: A Wikipedia Annotated Corpus for Named Entity Recognition
We revisit the idea of mining Wikipedia in order to generate named-entity annotations.
WikiCoref: An English Coreference-annotated Corpus of Wikipedia Articles
This paper presents WikiCoref, an English corpus annotated for anaphoric relations, where all documents are from the English version of Wikipedia.
