Search Results for author: Augustus Odena

Found 23 papers, 12 papers with code

Show Your Work: Scratchpads for Intermediate Computation with Language Models

no code implementations30 Nov 2021 Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, Augustus Odena

Large pre-trained language models perform remarkably well on tasks that can be done "in one pass", such as generating realistic text or synthesizing computer programs.

Program Synthesis with Large Language Models

1 code implementation16 Aug 2021 Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, Charles Sutton

Our largest models, even without finetuning on a code dataset, can synthesize solutions to 59. 6 percent of the problems from MBPP using few-shot learning with a well-designed prompt.

Few-Shot Learning Program Synthesis

Learnability and Complexity of Quantum Samples

1 code implementation22 Oct 2020 Murphy Yuezhen Niu, Andrew M. Dai, Li Li, Augustus Odena, Zhengli Zhao, Vadim Smelyanskyi, Hartmut Neven, Sergio Boixo

Given a quantum circuit, a quantum computer can sample the output distribution exponentially faster in the number of bits than classical computers.

Benchmarking

SMYRF: Efficient Attention using Asymmetric Clustering

1 code implementation11 Oct 2020 Giannis Daras, Nikita Kitaev, Augustus Odena, Alexandros G. Dimakis

We also show that SMYRF can be used interchangeably with dense attention before and after training.

Clustering

Top-k Training of GANs: Improving GAN Performance by Throwing Away Bad Samples

2 code implementations NeurIPS 2020 Samarth Sinha, Zhengli Zhao, Anirudh Goyal, Colin Raffel, Augustus Odena

We introduce a simple (one line of code) modification to the Generative Adversarial Network (GAN) training algorithm that materially improves results with no increase in computational cost: When updating the generator parameters, we simply zero out the gradient contributions from the elements of the batch that the critic scores as `least realistic'.

Generative Adversarial Network

Learning to Represent Programs with Property Signatures

no code implementations ICLR 2020 Augustus Odena, Charles Sutton

We introduce the notion of property signatures, a representation for programs and program specifications meant for consumption by machine learning algorithms.

Vocal Bursts Type Prediction

Improved Consistency Regularization for GANs

no code implementations11 Feb 2020 Zhengli Zhao, Sameer Singh, Honglak Lee, Zizhao Zhang, Augustus Odena, Han Zhang

Recent work has increased the performance of Generative Adversarial Networks (GANs) by enforcing a consistency cost on the discriminator.

Image Generation

Small-GAN: Speeding Up GAN Training Using Core-sets

no code implementations ICML 2020 Samarth Sinha, Han Zhang, Anirudh Goyal, Yoshua Bengio, Hugo Larochelle, Augustus Odena

Recent work by Brock et al. (2018) suggests that Generative Adversarial Networks (GANs) benefit disproportionately from large mini-batch sizes.

Active Learning Anomaly Detection +1

Improving Differentially Private Models with Active Learning

no code implementations2 Oct 2019 Zhengli Zhao, Nicolas Papernot, Sameer Singh, Neoklis Polyzotis, Augustus Odena

Broad adoption of machine learning techniques has increased privacy concerns for models trained on sensitive data such as medical records.

Active Learning

Discriminator Rejection Sampling

1 code implementation ICLR 2019 Samaneh Azadi, Catherine Olsson, Trevor Darrell, Ian Goodfellow, Augustus Odena

We propose a rejection sampling scheme using the discriminator of a GAN to approximately correct errors in the GAN generator distribution.

Image Generation

Skill Rating for Generative Models

no code implementations14 Aug 2018 Catherine Olsson, Surya Bhupatiraju, Tom Brown, Augustus Odena, Ian Goodfellow

We explore a new way to evaluate generative models using insights from evaluation of competitive games between human players.

TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing

3 code implementations28 Jul 2018 Augustus Odena, Ian Goodfellow

We then discuss the application of CGF to the following goals: finding numerical errors in trained neural networks, generating disagreements between neural networks and quantized versions of those networks, and surfacing undesirable behavior in character level language models.

Self-Attention Generative Adversarial Networks

49 code implementations arXiv 2018 Han Zhang, Ian Goodfellow, Dimitris Metaxas, Augustus Odena

In this paper, we propose the Self-Attention Generative Adversarial Network (SAGAN) which allows attention-driven, long-range dependency modeling for image generation tasks.

Conditional Image Generation Generative Adversarial Network

Realistic Evaluation of Deep Semi-Supervised Learning Algorithms

7 code implementations NeurIPS 2018 Avital Oliver, Augustus Odena, Colin Raffel, Ekin D. Cubuk, Ian J. Goodfellow

However, we argue that these benchmarks fail to address many issues that these algorithms would face in real-world applications.

Is Generator Conditioning Causally Related to GAN Performance?

no code implementations ICML 2018 Augustus Odena, Jacob Buckman, Catherine Olsson, Tom B. Brown, Christopher Olah, Colin Raffel, Ian Goodfellow

Motivated by this, we study the distribution of singular values of the Jacobian of the generator in Generative Adversarial Networks (GANs).

Changing Model Behavior at Test-Time Using Reinforcement Learning

no code implementations24 Feb 2017 Augustus Odena, Dieterich Lawson, Christopher Olah

Machine learning models are often used at test-time subject to constraints and trade-offs not present at training-time.

BIG-bench Machine Learning reinforcement-learning +2

Conditional Image Synthesis With Auxiliary Classifier GANs

36 code implementations ICML 2017 Augustus Odena, Christopher Olah, Jonathon Shlens

We expand on previous work for image quality assessment to provide two new analyses for assessing the discriminability and diversity of samples from class-conditional image synthesis models.

Ranked #13 on Conditional Image Generation on CIFAR-10 (Inception score metric)

Conditional Image Generation Image Quality Assessment

Semi-Supervised Learning with Generative Adversarial Networks

7 code implementations5 Jun 2016 Augustus Odena

We extend Generative Adversarial Networks (GANs) to the semi-supervised context by forcing the discriminator network to output class labels.

Faster Asynchronous SGD

no code implementations15 Jan 2016 Augustus Odena

Approaches have been proposed to circumvent this problem that quantify staleness in terms of the number of elapsed updates.

Cannot find the paper you are looking for? You can Submit a new open access paper.