Search Results for author:
Found 16 papers, 2 papers with code
What is Lost in Knowledge Distillation?
Deep neural networks (DNNs) have improved NLP tasks significantly, but training and maintaining such networks could be costly.
Training Mixed-Domain Translation Models via Federated Learning
Training mixed-domain translation models is a complex task that demands tailored architectures and costly data preparation techniques.
Dynamic Position Encoding for Transformers
To solve this problem, position embeddings are defined exclusively for each time step to enrich word information.
Communication-Efficient Federated Learning for Neural Machine Translation
These new components are placed in between original layers.
Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMax
We exploit a semi-supervised approach based on KD to train a model on augmented data.
From Fully Trained to Fully Random Embeddings: Improving Neural Machine Translation with Compact Word Embedding Tables
Embedding matrices are key components in neural natural language processing (NLP) models that are responsible to provide numerical representations of input tokens.\footnote{In this paper words and subwords are referred to as \textit{tokens} and the term \textit{embedding} only refers to embeddings of inputs.}
Robust Embeddings Via Distributions
Despite recent monumental advances in the field, many Natural Language Processing (NLP) models still struggle to perform adequately on noisy domains.
Revisiting Robust Neural Machine Translation: A Transformer Case Study
There is a large body of work in the NMT literature on analyzing the behavior of conventional models for the problem of noise but Transformers are relatively understudied in this context.
ALP-KD: Attention-Based Layer Projection for Knowledge Distillation
Knowledge distillation is considered as a training and compression strategy in which two neural networks, namely a teacher and a student, are coupled together during training.
Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers
With the growth of computing power neural machine translation (NMT) models also grow accordingly and become better.
Tailoring Neural Architectures for Translating from Morphologically Rich Languages
A morphologically complex word (MCW) is a hierarchical constituent with meaning-preserving subunits, so word-based models which rely on surface forms might not be powerful enough to translate such structures.
Improving Character-based Decoding Using Target-Side Morphological Information for Neural Machine Translation
Recently, neural machine translation (NMT) has emerged as a powerful alternative to conventional statistical approaches.
Investigating Backtranslation in Neural Machine Translation
A prerequisite for training corpus-based machine translation (MT) systems -- either Statistical MT (SMT) or Neural MT (NMT) -- is the availability of high-quality parallel data.
Enriching Phrase Tables for Statistical Machine Translation Using Mixed Embeddings
PBSMT engines by default provide four probability scores in phrase tables which are considered as the main set of bilingual features.
