Stochastic Local Winner-Takes-All Networks Enable Profound Adversarial Robustness

5 Dec 2021  ·  Konstantinos P. Panousis, Sotirios Chatzis, Sergios Theodoridis ·

This work explores the potency of stochastic competition-based activations, namely Stochastic Local Winner-Takes-All (LWTA), against powerful (gradient-based) white-box and black-box adversarial attacks; we especially focus on Adversarial Training settings. In our work, we replace the conventional ReLU-based nonlinearities with blocks comprising locally and stochastically competing linear units. The output of each network layer now yields a sparse output, depending on the outcome of winner sampling in each block. We rely on the Variational Bayesian framework for training and inference; we incorporate conventional PGD-based adversarial training arguments to increase the overall adversarial robustness. As we experimentally show, the arising networks yield state-of-the-art robustness against powerful adversarial attacks while retaining very high classification rate in the benign case.

PDF Abstract

Datasets


Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Adversarial Robustness CIFAR-10 Stochastic-LWTA/PGD/WideResNet-34-10 Attack: AutoAttack 82.6 # 1
Accuracy 92.26 # 1
Robust Accuracy 84.3 # 1
Adversarial Defense CIFAR-10 Stochastic-LWTA/PGD/WideResNet-34-10 Accuracy 84.3 # 1
Attack: AutoAttack 82.6 # 1
Adversarial Robustness CIFAR-10 Stochastic-LWTA/PGD/WideResNet-34-5 Attack: AutoAttack 81.22 # 2
Accuracy 91.88 # 2
Robust Accuracy 83.4 # 2
Adversarial Defense CIFAR-10 Stochastic-LWTA/PGD/WideResNet-34-5 Attack: AutoAttack 81.22 # 2
Adversarial Defense CIFAR-10 Ours (Stochastic-LWTA/PGD/WideResNet-34-5) Accuracy 83.4 # 2
Adversarial Defense CIFAR-10 Ours (Stochastic-LWTA/PGD/WideResNet-34-1) Accuracy 81.87 # 3
Attack: AutoAttack 74.71 # 3

Methods


No methods listed for this paper. Add relevant methods here