Search Results for author:
Found 10 papers, 1 papers with code
Modeling Task-Aware MIMO Cardinality for Efficient Multilingual Neural Machine Translation
In this paper, we propose to efficiently increase the capacity for multilingual NMT by increasing the cardinality.
Multi-Head Highly Parallelized LSTM Decoder for Neural Machine Translation
This has to be computed n times for a sequence of length n. The linear transformations involved in the LSTM gate and state computations are the major cost factors in this.
Learning Hard Retrieval Decoder Attention for Transformers
The Transformer translation model is based on the multi-head attention mechanism, which can be parallelized easily.
Rewiring the Transformer with Depth-Wise LSTMs
Stacking non-linear layers allows deep neural networks to model complicated functions, and including residual connections in Transformer layers is beneficial for convergence and performance.
Learning Source Phrase Representations for Neural Machine Translation
Considering that modeling phrases instead of words has significantly improved the Statistical Machine Translation (SMT) approach through the use of larger translation blocks ("phrases") and its reordering ability, modeling NMT at phrase level is an intuitive proposal to help the model capture long-distance relationships.
Dynamically Adjusting Transformer Batch Size by Monitoring Gradient Direction Change
We propose to automatically and dynamically determine batch sizes by accumulating gradients of mini-batches and performing an optimization step at just the time when the direction of gradients starts to fluctuate.
Probing Word Translations in the Transformer and Trading Decoder for Encoder Layers
Due to its effectiveness and performance, the Transformer translation model has attracted wide attention, most recently in terms of probing-based approaches.
Lipschitz Constrained Parameter Initialization for Deep Transformers
In this paper, we first empirically demonstrate that a simple modification made in the official implementation, which changes the computation order of residual connection and layer normalization, can significantly ease the optimization of deep Transformers.
UdS Submission for the WMT 19 Automatic Post-Editing Task
In this paper, we describe our submission to the English-German APE shared task at WMT 2019.
Neutron: An Implementation of the Transformer Translation Model and its Variants
The Transformer translation model is easier to parallelize and provides better performance compared to recurrent seq2seq models, which makes it popular among industry and research community.
