Search Results for author:
Found 3 papers, 1 papers with code
Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillation
From this observation, we consider an intuitive KD loss function, the mean squared error (MSE) between the logit vectors, so that the student model can directly learn the logit of the teacher model.
Understanding Knowledge Distillation
To verify this conjecture, we test an extreme logit learning model, where the KD is implemented with Mean Squared Error (MSE) between the student's logit and the teacher's logit.
Adaptive Local Bayesian Optimization Over Multiple Discrete Variables
In the machine learning algorithms, the choice of the hyperparameter is often an art more than a science, requiring labor-intensive search with expert experience.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
