Search Results for author:
Found 4 papers, 0 papers with code
Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems
We present results from a large-scale experiment on pretraining encoders with non-embedding parameter counts ranging from 700M to 9. 3B, their subsequent distillation into smaller models ranging from 17M-170M parameters, and their application to the Natural Language Understanding (NLU) component of a virtual assistant system.
Cross-Lingual Natural Language Inference
intent-classification
+5
Industry Scale Semi-Supervised Learning for Natural Language Understanding
This paper presents a production Semi-Supervised Learning (SSL) pipeline based on the student-teacher framework, which leverages millions of unlabeled examples to improve Natural Language Understanding (NLU) tasks.
Enhance Robustness of Sequence Labelling with Masked Adversarial Training
Adversarial training (AT) has shown strong regularization effects on deep learning algorithms by introducing small input perturbations to improve model robustness.
Ranked #3 on
Chunking
on CoNLL 2000
(using extra training data)
SeqVAT: Virtual Adversarial Training for Semi-Supervised Sequence Labeling
Virtual adversarial training (VAT) is a powerful technique to improve model robustness in both supervised and semi-supervised settings.
Ranked #7 on
Chunking
on CoNLL 2000
