Search Results for author: Ekin D. Cubuk

Found 28 papers, 18 papers with code

On the surprising tradeoff between ImageNet accuracy and perceptual similarity

no code implementations9 Mar 2022 Manoj Kumar, Neil Houlsby, Nal Kalchbrenner, Ekin D. Cubuk

Perceptual distances between images, as measured in the space of pre-trained deep features, have outperformed prior low-level, pixel-based metrics on assessing image similarity.

No One Representation to Rule Them All: Overlapping Features of Training Methods

no code implementations ICLR 2022 Raphael Gontijo-Lopes, Yann Dauphin, Ekin D. Cubuk

Despite being able to capture a range of features of the data, high accuracy models trained with supervision tend to make similar predictions.

Contrastive Learning

Multi-Task Self-Training for Learning General Representations

no code implementations ICCV 2021 Golnaz Ghiasi, Barret Zoph, Ekin D. Cubuk, Quoc V. Le, Tsung-Yi Lin

The results suggest self-training is a promising direction to aggregate labeled and unlabeled training data for learning general feature representations.

Multi-Task Learning

Revisiting ResNets: Improved Training and Scaling Strategies

4 code implementations NeurIPS 2021 Irwan Bello, William Fedus, Xianzhi Du, Ekin D. Cubuk, Aravind Srinivas, Tsung-Yi Lin, Jonathon Shlens, Barret Zoph

Using improved training and scaling strategies, we design a family of ResNet architectures, ResNet-RS, which are 1. 7x - 2. 7x faster than EfficientNets on TPUs, while achieving similar accuracies on ImageNet.

Action Classification Document Image Classification +2

Crystal Structure Search with Random Relaxations Using Graph Networks

no code implementations5 Dec 2020 Gowoon Cheon, Lusann Yang, Kevin McCloskey, Evan J. Reed, Ekin D. Cubuk

Materials design enables technologies critical to humanity, including combating climate change with solar cells and batteries.

Data Augmentation Domain Generalization

Kohn-Sham equations as regularizer: building prior knowledge into machine-learned physics

1 code implementation17 Sep 2020 Li Li, Stephan Hoyer, Ryan Pederson, Ruoxi Sun, Ekin D. Cubuk, Patrick Riley, Kieron Burke

Including prior knowledge is important for effective machine learning models in physics, and is usually achieved by explicitly adding loss terms or constraints on model architectures.

Rethinking Pre-training and Self-training

2 code implementations NeurIPS 2020 Barret Zoph, Golnaz Ghiasi, Tsung-Yi Lin, Yin Cui, Hanxiao Liu, Ekin D. Cubuk, Quoc V. Le

For example, on the COCO object detection dataset, pre-training benefits when we use one fifth of the labeled data, and hurts accuracy when we use all labeled data.

 Ranked #1 on Semantic Segmentation on PASCAL VOC 2012 test (using extra training data)

Data Augmentation Object Detection +1

Naive-Student: Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation

1 code implementation ECCV 2020 Liang-Chieh Chen, Raphael Gontijo Lopes, Bowen Cheng, Maxwell D. Collins, Ekin D. Cubuk, Barret Zoph, Hartwig Adam, Jonathon Shlens

We view this work as a notable step towards building a simple procedure to harness unlabeled video sequences and extra images to surpass state-of-the-art performance on core computer vision tasks.

Optical Flow Estimation Panoptic Segmentation +2

ReMixMatch: Semi-Supervised Learning with Distribution Matching and Augmentation Anchoring

1 code implementation ICLR 2020 David Berthelot, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Kihyuk Sohn, Han Zhang, Colin Raffel

We improve the recently-proposed ``MixMatch semi-supervised learning algorithm by introducing two new techniques: distribution alignment and augmentation anchoring.

Affinity and Diversity: Quantifying Mechanisms of Data Augmentation

no code implementations20 Feb 2020 Raphael Gontijo-Lopes, Sylvia J. Smullin, Ekin D. Cubuk, Ethan Dyer

Though data augmentation has become a standard component of deep neural network training, the underlying mechanism behind the effectiveness of these techniques remains poorly understood.

Data Augmentation

JAX, M.D.: A Framework for Differentiable Physics

1 code implementation9 Dec 2019 Samuel S. Schoenholz, Ekin D. Cubuk

We introduce JAX MD, a software package for performing differentiable physics simulations with a focus on molecular dynamics.

Drug Discovery

AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty

12 code implementations ICLR 2020 Dan Hendrycks, Norman Mu, Ekin D. Cubuk, Barret Zoph, Justin Gilmer, Balaji Lakshminarayanan

We propose AugMix, a data processing technique that is simple to implement, adds limited computational overhead, and helps models withstand unforeseen corruptions.

Domain Generalization Image Classification

RandAugment: Practical automated data augmentation with a reduced search space

15 code implementations NeurIPS 2020 Ekin D. Cubuk, Barret Zoph, Jonathon Shlens, Quoc V. Le

Additionally, due to the separate search phase, these approaches are unable to adjust the regularization strength based on model or dataset size.

Data Augmentation Image Classification +1

JAX MD: End-to-End Differentiable, Hardware Accelerated, Molecular Dynamics in Pure Python

no code implementations25 Sep 2019 Samuel S. Schoenholz, Ekin D. Cubuk

In this work we bring the substantial advances in software that have taken place in machine learning to MD with JAX, M. D.

Drug Discovery

Learning Data Augmentation Strategies for Object Detection

6 code implementations ECCV 2020 Barret Zoph, Ekin D. Cubuk, Golnaz Ghiasi, Tsung-Yi Lin, Jonathon Shlens, Quoc V. Le

Importantly, the best policy found on COCO may be transferred unchanged to other detection datasets and models to improve predictive accuracy.

Image Augmentation Image Classification +1

Using learned optimizers to make models robust to input noise

no code implementations8 Jun 2019 Luke Metz, Niru Maheswaranathan, Jonathon Shlens, Jascha Sohl-Dickstein, Ekin D. Cubuk

State-of-the art vision models can achieve superhuman performance on image classification tasks when testing and training data come from the same distribution.

General Classification Image Classification +1

Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation

2 code implementations6 Jun 2019 Raphael Gontijo Lopes, Dong Yin, Ben Poole, Justin Gilmer, Ekin D. Cubuk

Deploying machine learning systems in the real world requires both high accuracy on clean data and robustness to naturally occurring corruptions.

Data Augmentation Object Detection

AutoAugment: Learning Augmentation Strategies From Data

1 code implementation CVPR 2019 Ekin D. Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, Quoc V. Le

In our implementation, we have designed a search space where a policy consists of many sub-policies, one of which is randomly chosen for each image in each mini-batch.

Data Augmentation

SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition

29 code implementations18 Apr 2019 Daniel S. Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D. Cubuk, Quoc V. Le

On LibriSpeech, we achieve 6. 8% WER on test-other without the use of a language model, and 5. 8% WER with shallow fusion with a language model.

Automatic Speech Recognition Data Augmentation

Accelerated search and design of stretchable graphene kirigami using machine learning

1 code implementation18 Aug 2018 Paul Z. Hanakata, Ekin D. Cubuk, David K. Campbell, Harold S. Park

Making kirigami-inspired cuts into a sheet has been shown to be an effective way of designing stretchable materials with metamorphic properties where the 2D shape can transform into complex 3D shapes.

Computational Physics Disordered Systems and Neural Networks

AutoAugment: Learning Augmentation Policies from Data

24 code implementations24 May 2018 Ekin D. Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, Quoc V. Le

In our implementation, we have designed a search space where a policy consists of many sub-policies, one of which is randomly chosen for each image in each mini-batch.

Fine-Grained Image Classification Image Augmentation

Realistic Evaluation of Deep Semi-Supervised Learning Algorithms

7 code implementations NeurIPS 2018 Avital Oliver, Augustus Odena, Colin Raffel, Ekin D. Cubuk, Ian J. Goodfellow

However, we argue that these benchmarks fail to address many issues that these algorithms would face in real-world applications.

Machine learning determination of atomic dynamics at grain boundaries

no code implementations4 Mar 2018 Tristan A. Sharp, Spencer L. Thomas, Ekin D. Cubuk, Samuel S. Schoenholz, David J. Srolovitz, Andrea J. Liu

In polycrystalline materials, grain boundaries are sites of enhanced atomic motion, but the complexity of the atomic structures within a grain boundary network makes it difficult to link the structure and atomic dynamics.

Materials Science

Cannot find the paper you are looking for? You can Submit a new open access paper.