Search Results for author:
Found 6 papers, 2 papers with code
On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning Paradigm
The pretraining-finetuning paradigm has become the prevailing trend in modern deep learning.
The Importance of Prompt Tuning for Automated Neuron Explanations
Recent advances have greatly increased the capabilities of large language models (LLMs), but our understanding of the models and their safety has not progressed as fast.
Quantifying the Knowledge in a DNN to Explain Knowledge Distillation for Classification
This paper provides a new perspective to explain the success of knowledge distillation, i. e., quantifying knowledge points encoded in intermediate layers of a DNN for classification, based on the information theory.
Demystify Optimization and Generalization of Over-parameterized PAC-Bayesian Learning
In addition to being a pure generalization bound analysis tool, PAC-Bayesian bound can also be incorporated into an objective function to train a probabilistic neural network, making them a powerful and relevant framework that can numerically provide a tight generalization bound for supervised learning.
On the Equivalence between Neural Network and Support Vector Machine
Therefore, in this work, we propose to establish the equivalence between NN and SVM, and specifically, the infinitely wide NN trained by soft margin loss and the standard soft margin SVM with NTK trained by subgradient descent.
Explaining Knowledge Distillation by Quantifying the Knowledge
Whereas, in the scenario of learning from raw data, the DNN learns visual concepts sequentially.
