Search Results for author:
Found 13 papers, 8 papers with code
Conditional Augmentation for Generative Modeling
We present conditional augmentation (CondAugment), a simple and powerful method of regularizing generative models.
PaLM: Scaling Language Modeling with Pathways
To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM.
Ranked #1 on
Coreference Resolution
on Winograd Schema Challenge
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
Next, we detail the training process, the design of our training corpus, and our data curation techniques, which we believe is a key ingredient to the success of the model.
Ranked #35 on
Language Modelling
on LAMBADA
Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images
We present a hierarchical VAE that, for the first time, generates samples quickly while outperforming the PixelCNN in log-likelihood on all natural image benchmarks.
Ranked #2 on
Image Generation
on FFHQ 1024 x 1024
(bits/dimension metric)
Generative Pretraining from Pixels
Inspired by progress in unsupervised representation learning for natural language, we examine whether similar models can learn useful representations for images.
Ranked #14 on
Image Classification
on STL-10
(using extra training data)
Representation Learning
Self-Supervised Image Classification
Language Models are Few-Shot Learners
By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do.
Ranked #1 on
Question Answering
on CoQA
(Overall metric)
Scaling Laws for Neural Language Models
We study empirical scaling laws for language model performance on the cross-entropy loss.
Generating Long Sequences with Sparse Transformers
Transformers are powerful sequence models, but require time and memory that grows quadratically with the sequence length.
Language Models are Unsupervised Multitask Learners
Natural language processing tasks, such as question answering, machine translation, reading comprehension, and summarization, are typically approached with supervised learning on taskspecific datasets.
Ranked #1 on
Language Modelling
on enwik8
(using extra training data)
Exploring Neural Transducers for End-to-End Speech Recognition
In this work, we perform an empirical comparison among the CTC, RNN-Transducer, and attention-based Seq2Seq models for end-to-end speech recognition.
Reducing Bias in Production Speech Models
Replacing hand-engineered pipelines with end-to-end deep learning systems has enabled strong results in applications like speech and object recognition.
Convolutional Recurrent Neural Networks for Small-Footprint Keyword Spotting
Keyword spotting (KWS) constitutes a major component of human-technology interfaces.
Active Learning for Speech Recognition: the Power of Gradients
For speech recognition, confidence scores and other likelihood-based active learning methods have been shown to be effective.
