Understanding Robustness in Teacher-Student Setting: A New Perspective

no code implementations • 25 Feb 2021 • Zhuolin Yang, Zhaoxi Chen, Tiffany Cai, Xinyun Chen, Bo Li, Yuandong Tian

Extensive experiments show that student specialization correlates strongly with model robustness in different scenarios, including student trained via standard training, adversarial training, confidence-calibrated adversarial training, and training with robust feature dataset.

BIG-bench Machine Learning Data Augmentation