Search Results for author:
Found 26 papers, 20 papers with code
Conditional Augmentation for Generative Modeling
We present conditional augmentation (CondAugment), a simple and powerful method of regularizing generative models.
Robust Speech Recognition via Large-Scale Weak Supervision
We study the capabilities of speech processing systems trained simply to predict large amounts of transcripts of audio on the internet.
Ranked #1 on
Speech Recognition
on Common Voice English
(using extra training data)
Text and Code Embeddings by Contrastive Pre-Training
Similarly to text embeddings, we train code embedding models on (text, code) pairs, obtaining a 20. 8% relative improvement over prior best work on code search.
Ranked #1 on
Code Search
on CodeSearchNet
Unsupervised Neural Machine Translation with Generative Language Models Only
We show how to derive state-of-the-art unsupervised neural machine translation systems from generatively pre-trained language models.
Multimodal Neurons in Artificial Neural Networks
It’s the fact that you plug visual information into the rich tapestry of memory that brings it to life."
Evaluating CLIP: Towards Characterization of Broader Capabilities and Downstream Implications
Recently, there have been breakthroughs in computer vision ("CV") models that are more generalizable with the advent of models such as CLIP and ALIGN.
Evaluating Large Language Models Trained on Code
We introduce Codex, a GPT language model fine-tuned on publicly available code from GitHub, and study its Python code-writing capabilities.
Ranked #1 on
Multi-task Language Understanding
on BBH-alg
Learning Transferable Visual Models From Natural Language Supervision
State-of-the-art computer vision systems are trained to predict a fixed set of predetermined object categories.
Ranked #1 on
Zero-Shot Learning
on COCO-MLT
(using extra training data)
Zero-Shot Text-to-Image Generation
Text-to-image generation has traditionally focused on finding better modeling assumptions for training on a fixed dataset.
Ranked #44 on
Text-to-Image Generation
on COCO
(using extra training data)
Learning to summarize with human feedback
We collect a large, high-quality dataset of human comparisons between summaries, train a model to predict the human-preferred summary, and use that model as a reward function to fine-tune a summarization policy using reinforcement learning.
Scaling Laws for Autoregressive Generative Modeling
The optimal model size also depends on the compute budget through a power-law, with exponents that are nearly universal across all data domains.
Learning to summarize from human feedback
We collect a large, high-quality dataset of human comparisons between summaries, train a model to predict the human-preferred summary, and use that model as a reward function to fine-tune a summarization policy using reinforcement learning.
Generative Pretraining from Pixels
Inspired by progress in unsupervised representation learning for natural language, we examine whether similar models can learn useful representations for images.
Ranked #16 on
Image Classification
on STL-10
(using extra training data)
Representation Learning
Self-Supervised Image Classification
Language Models are Few-Shot Learners
By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do.
Jukebox: A Generative Model for Music
We introduce Jukebox, a model that generates music with singing in the raw audio domain.
Scaling Laws for Neural Language Models
We study empirical scaling laws for language model performance on the cross-entropy loss.
Fine-Tuning Language Models from Human Preferences
Most work on reward learning has used simulated environments, but complex information about values is often expressed in natural language, and we believe reward learning for language is a key to making RL practical and safe for real-world tasks.
Release Strategies and the Social Impacts of Language Models
Large language models have a range of beneficial uses: they can assist in prose, poetry, and programming; analyze dataset biases; and more.
Generating Long Sequences with Sparse Transformers
Transformers are powerful sequence models, but require time and memory that grows quadratically with the sequence length.
Language Models are Unsupervised Multitask Learners
Natural language processing tasks, such as question answering, machine translation, reading comprehension, and summarization, are typically approached with supervised learning on taskspecific datasets.
Ranked #1 on
Language Modelling
on enwik8
(using extra training data)
Improving Language Understanding by Generative Pre-Training
We demonstrate that large gains on these tasks can be realized by generative pre-training of a language model on a diverse corpus of unlabeled text, followed by discriminative fine-tuning on each specific task.
Ranked #3 on
Natural Language Inference
on SciTail
Improving GANs Using Optimal Transport
We present Optimal Transport GAN (OT-GAN), a variant of generative adversarial nets minimizing a new metric measuring the distance between the generator distribution and the data distribution.
Proximal Policy Optimization Algorithms
We propose a new family of policy gradient methods for reinforcement learning, which alternate between sampling data through interaction with the environment, and optimizing a "surrogate" objective function using stochastic gradient ascent.
Learning to Generate Reviews and Discovering Sentiment
We explore the properties of byte-level recurrent language models.
Ranked #9 on
Subjectivity Analysis
on SUBJ
Improved Techniques for Training GANs
We present a variety of new architectural features and training procedures that we apply to the generative adversarial networks (GANs) framework.
Ranked #14 on
Conditional Image Generation
on CIFAR-10
(Inception score metric)
Conditional Image Generation
Semi-Supervised Image Classification
+1
Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
In recent years, supervised learning with convolutional networks (CNNs) has seen huge adoption in computer vision applications.
Ranked #9 on
Image Clustering
on Tiny-ImageNet
