Search Results for author:
Found 17 papers, 12 papers with code
The Ungrounded Alignment Problem
This paper examines a simplified version of the general problem, where an unsupervised learner is presented with a sequence of images for the characters in a text corpus, and this learner is later evaluated on its ability to recognize specific (possibly rare) sequential patterns.
Transformer Layers as Painters
We further show that some classes of problems have robustness to skipping layers, running the layers in an order different from how they were trained, or running the layers in parallel.
Helpful Neighbors: Leveraging Neighbors in Geographic Feature Pronunciation
If one sees the place name Houston Mercer Dog Run in New York, how does one know how to pronounce it?
DF-Conformer: Integrated architecture of Conv-TasNet and Conformer using linear complexity self-attention for speech enhancement
To make the model computationally feasible, we extend the Conformer using linear complexity attention and stacked 1-D dilated depthwise convolution layers.
A Comparative Study on Neural Architectures and Training Methods for Japanese Speech Recognition
End-to-end (E2E) modeling is advantageous for automatic speech recognition (ASR) especially for Japanese since word-based tokenization of Japanese is not trivial, and E2E modeling is able to model character sequences directly.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
CodeTrans: Towards Cracking the Language of Silicon's Code Through Self-Supervised Deep Learning and High Performance Computing
Simultaneously, the transformer model, especially its combination with transfer learning, has been proven to be a powerful technique for natural language processing tasks.
ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing
Here, we trained two auto-regressive models (Transformer-XL, XLNet) and four auto-encoder models (BERT, Albert, Electra, T5) on data from UniRef and BFD containing up to 393 billion amino acids.
Ranked #1 on
Protein Secondary Structure Prediction
on CASP12
Dimensionality Reduction
Protein Secondary Structure Prediction
Natural Questions: a Benchmark for Question Answering Research
The public release consists of 307, 373 training examples with single annotations, 7, 830 examples with 5-way annotations for development data, and a further 7, 842 examples 5-way annotated sequestered as test data.
Ranked #7 on
Question Answering
on Natural Questions (long)
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models.
Character-Level Language Modeling with Deeper Self-Attention
LSTMs and other RNN variants have shown strong performance on character-level language modeling.
Ranked #8 on
Language Modelling
on Hutter Prize
The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation
Each of these new approaches consists of a fundamental architecture accompanied by a set of modeling and training techniques that are in principle applicable to other seq2seq architectures.
Ranked #27 on
Machine Translation
on WMT2014 English-French
Tensor2Tensor for Neural Machine Translation
Tensor2Tensor is a library for deep learning models that is well-suited for neural machine translation and includes the reference implementation of the state-of-the-art Transformer model.
Large Scale Multi-Domain Multi-Task Learning with MultiModel
We present a single model that yields good results on a number of problems spanning multiple domains.
Accurate Supervised and Semi-Supervised Machine Reading for Long Documents
We also evaluate the model in a semi-supervised setting by downsampling the WikiReading training set to create increasingly smaller amounts of supervision, while leaving the full unlabeled document corpus to train a sequence autoencoder on document windows.
One Model To Learn Them All
We present a single model that yields good results on a number of problems spanning multiple domains.
Attention Is All You Need
The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration.
Ranked #2 on
Multimodal Machine Translation
on Multi30K
(BLUE (DE-EN) metric)
WikiReading: A Novel Large-scale Language Understanding Task over Wikipedia
The task contains a rich variety of challenging classification and extraction sub-tasks, making it well-suited for end-to-end models such as deep neural networks (DNNs).
