Search Results for author:
Found 19 papers, 12 papers with code
Attention Is All You Need
The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration.
Ranked #2 on
Multimodal Machine Translation
on Multi30K
(BLUE (DE-EN) metric)
One Model To Learn Them All
We present a single model that yields good results on a number of problems spanning multiple domains.
Large Scale Multi-Domain Multi-Task Learning with MultiModel
We present a single model that yields good results on a number of problems spanning multiple domains.
Image Transformer
Image generation has been successfully cast as an autoregressive sequence generation or transformation problem.
Ranked #3 on
Density Estimation
on CIFAR-10
Fast Decoding in Sequence Models using Discrete Latent Variables
Finally, we evaluate our model end-to-end on the task of neural machine translation, where it is an order of magnitude faster at decoding than comparable autoregressive models.
Tensor2Tensor for Neural Machine Translation
Tensor2Tensor is a library for deep learning models that is well-suited for neural machine translation and includes the reference implementation of the state-of-the-art Transformer model.
The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation
Each of these new approaches consists of a fundamental architecture accompanied by a set of modeling and training techniques that are in principle applicable to other seq2seq architectures.
Ranked #26 on
Machine Translation
on WMT2014 English-French
Theory and Experiments on Vector Quantized Autoencoders
Deep neural networks with discrete latent variables offer the promise of better symbolic reasoning, and learning abstractions that are more useful to new tasks.
Weakly Supervised Grammatical Error Correction using Iterative Decoding
We describe an approach to Grammatical Error Correction (GEC) that is effective at making use of models trained on large amounts of weakly supervised bitext.
Mesh-TensorFlow: Deep Learning for Supercomputers
We use Mesh-TensorFlow to implement an efficient data-parallel, model-parallel version of the Transformer sequence-to-sequence model.
Ranked #10 on
Language Modelling
on One Billion Word
Corpora Generation for Grammatical Error Correction
We provide systematic analysis that compares the two approaches to data generation and highlights the effectiveness of ensembling.
Towards a better understanding of Vector Quantized Autoencoders
Deep neural networks with discrete latent variables offer the promise of better symbolic reasoning, and learning abstractions that are more useful to new tasks.
Stand-Alone Self-Attention in Vision Models
The natural question that arises is whether attention can be a stand-alone primitive for vision models instead of serving as just an augmentation on top of convolutions.
High Resolution Medical Image Analysis with Spatial Partitioning
It is infeasible to train CNN models directly on such high resolution images, because neural activations of a single image do not fit in the memory of a single GPU/TPU, and naive data and model parallelism approaches do not work.
Conformer: Convolution-augmented Transformer for Speech Recognition
Recently Transformer and Convolution neural network (CNN) based models have shown promising results in Automatic Speech Recognition (ASR), outperforming Recurrent neural networks (RNNs).
Ranked #12 on
Speech Recognition
on LibriSpeech test-other
(using extra training data)
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Bottleneck Transformers for Visual Recognition
Finally, we present a simple adaptation of the BoTNet design for image classification, resulting in models that achieve a strong performance of 84. 7% top-1 accuracy on the ImageNet benchmark while being up to 1. 64x faster in compute time than the popular EfficientNet models on TPU-v3 hardware.
Ranked #52 on
Instance Segmentation
on COCO minival
Scaling Local Self-Attention for Parameter Efficient Visual Backbones
Self-attention models have recently been shown to have encouraging improvements on accuracy-parameter trade-offs compared to baseline convolutional models such as ResNet-50.
Ranked #209 on
Image Classification
on ImageNet
Simple and Efficient ways to Improve REALM
Dense retrieval has been shown to be effective for retrieving relevant documents for Open Domain QA, surpassing popular sparse retrieval methods like BM25.
Decoder Denoising Pretraining for Semantic Segmentation
We propose a decoder pretraining approach based on denoising, which can be combined with supervised pretraining of the encoder.
