Search Results for author:
Found 4 papers, 3 papers with code
Improving Transformer Optimization Through Better Initialization
As Transformer models are becoming larger and more expensive to train, recent research has focused on understanding and improving optimization in these models.
Improving Transformer Optimization Through Better Initialization
As Transformer models are becoming larger and more expensive to train, recent research has focused on understanding and improving optimization in these models.
DiMS: Distilling Multiple Steps of Iterative Non-Autoregressive Transformers for Machine Translation
The student is optimized to predict the output of the teacher after multiple decoding steps while the teacher follows the student via a slow-moving average.
Improving Non-Autoregressive Translation Models Without Distillation
Empirically, we show that CMLMC achieves state-of-the-art NAR performance when trained on raw data without distillation and approaches AR performance on multiple datasets.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
