Search Results for author:
Found 11 papers, 5 papers with code
Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late in Training
Specifically, we find that SAM efficiently selects flatter minima late in training.
On the Optimization and Generalization of Two-layer Transformers with Sign Gradient Descent
We also show that Adam behaves similarly to SignGD in terms of both optimization and generalization in this setting.
On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning Paradigm
The pretraining-finetuning paradigm has become the prevailing trend in modern deep learning.
Going Beyond Neural Network Feature Similarity: The Network Feature Complexity and Its Interpretation Using Category Theory
In this paper, we expand the concept of equivalent feature and provide the definition of what we call functionally equivalent features.
Defects of Convolutional Decoder Networks in Frequency Representation
In this paper, we prove the representation defects of a cascaded convolutional decoder network, considering the capacity of representing different frequency components of an input sample.
Batch Normalization Is Blind to the First and Second Derivatives of the Loss
In this paper, we prove the effects of the BN operation on the back-propagation of the first and second derivatives of the loss.
Towards a Unified Game-Theoretic View of Adversarial Perturbations and Robustness
This paper provides a unified view to explain different adversarial attacks and defense methods, i. e. the view of multi-order interactions between input variables of DNNs.
A Unified Game-Theoretic Interpretation of Adversarial Robustness
This paper provides a unified view to explain different adversarial attacks and defense methods, \emph{i. e.} the view of multi-order interactions between input variables of DNNs.
Towards a Game-Theoretic View of Baseline Values in the Shapley Value
In the computation of Shapley values, people usually set an input variable to its baseline value to represent the absence of this variable.
Can We Faithfully Represent Masked States to Compute Shapley Values on a DNN?
Masking some input variables of a deep neural network (DNN) and computing output changes on the masked input sample represent a typical way to compute attributions of input variables in the sample.
A Unified Game-Theoretic Interpretation of Adversarial Robustness
This paper provides a unified view to explain different adversarial attacks and defense methods, i. e. the view of multi-order interactions between input variables of DNNs.
