Contrastive Learning is Just Meta-Learning

no code implementations • ICLR 2022 • Renkun Ni, Manli Shu, Hossein Souri, Micah Goldblum, Tom Goldstein

Contrastive learning has recently taken off as a paradigm for learning from unlabeled data.

Contrastive Learning Data Augmentation +1

GradInit: Learning to Initialize Neural Networks for Stable and Efficient Training

1 code implementation • NeurIPS 2021 • Chen Zhu, Renkun Ni, Zheng Xu, Kezhi Kong, W. Ronny Huang, Tom Goldstein

Innovations in neural architectures have fostered significant breakthroughs in language modeling and computer vision.

Image Classification Language Modelling +2

WrapNet: Neural Net Inference with Ultra-Low-Precision Arithmetic

no code implementations • ICLR 2021 • Renkun Ni, Hong-Min Chu, Oscar Castaneda, Ping-Yeh Chiang, Christoph Studer, Tom Goldstein

Low-precision neural networks represent both weights and activations with few bits, drastically reducing the multiplication complexity.

Arithmetic Quantization

Data Augmentation for Meta-Learning

1 code implementation • 14 Oct 2020 • Renkun Ni, Micah Goldblum, Amr Sharaf, Kezhi Kong, Tom Goldstein

Conventional image classifiers are trained by randomly sampling mini-batches of images.

Data Augmentation Meta-Learning

WrapNet: Neural Net Inference with Ultra-Low-Resolution Arithmetic

no code implementations • 26 Jul 2020 • Renkun Ni, Hong-Min Chu, Oscar Castañeda, Ping-Yeh Chiang, Christoph Studer, Tom Goldstein

Low-resolution neural networks represent both weights and activations with few bits, drastically reducing the multiplication complexity.

Arithmetic Quantization

Certified Defenses for Adversarial Patches

1 code implementation • ICLR 2020 • Ping-Yeh Chiang, Renkun Ni, Ahmed Abdelkader, Chen Zhu, Christoph Studer, Tom Goldstein

Adversarial patch attacks are among one of the most practical threat models against real-world computer vision systems.

Improving the Tightness of Convex Relaxation Bounds for Training Certifiably Robust Classifiers

no code implementations • 22 Feb 2020 • Chen Zhu, Renkun Ni, Ping-Yeh Chiang, Hengduo Li, Furong Huang, Tom Goldstein

Convex relaxations are effective for training and certifying neural networks against norm-bounded adversarial attacks, but they leave a large gap between certifiable and empirical robustness.

Unraveling Meta-Learning: Understanding Feature Representations for Few-Shot Tasks

1 code implementation • ICML 2020 • Micah Goldblum, Steven Reich, Liam Fowl, Renkun Ni, Valeriia Cherepanova, Tom Goldstein

In doing so, we introduce and verify several hypotheses for why meta-learned models perform better.

General Classification Meta-Learning

WITCHcraft: Efficient PGD attacks with random step size

no code implementations • 18 Nov 2019 • Ping-Yeh Chiang, Jonas Geiping, Micah Goldblum, Tom Goldstein, Renkun Ni, Steven Reich, Ali Shafahi

State-of-the-art adversarial attacks on neural networks use expensive iterative methods and numerous random restarts from different initial points.

Improved Training of Certifiably Robust Models

no code implementations • 25 Sep 2019 • Chen Zhu, Renkun Ni, Ping-Yeh Chiang, Hengduo Li, Furong Huang, Tom Goldstein

Convex relaxations are effective for training and certifying neural networks against norm-bounded adversarial attacks, but they leave a large gap between certifiable and empirical (PGD) robustness.

Learning Accurate Low-Bit Deep Neural Networks with Stochastic Quantization

1 code implementation • 3 Aug 2017 • Yinpeng Dong, Renkun Ni, Jianguo Li, Yurong Chen, Jun Zhu, Hang Su

This procedure can greatly compensate the quantization error and thus yield better accuracy for low-bit DNNs.

Quantization