Initialization Matters for Adversarial Transfer Learning

1 code implementation • 10 Dec 2023 • Andong Hua, Jindong Gu, Zhiyu Xue, Nicholas Carlini, Eric Wong, Yao Qin

Based on this, we propose Robust Linear Initialization (RoLI) for adversarial finetuning, which initializes the linear head with the weights obtained by adversarial linear probing to maximally inherit the robustness from pretraining.

Adversarial Robustness Image Classification +1