Search Results for author:
Found 4 papers, 0 papers with code
A Comprehensive Review of Knowledge Distillation in Computer Vision
Deep learning techniques have been demonstrated to surpass preceding cutting-edge machine learning techniques in recent years, with computer vision being one of the most prominent examples.
Insights into the Lottery Ticket Hypothesis and Iterative Magnitude Pruning
Lottery ticket hypothesis for deep neural networks emphasizes the importance of initialization used to re-train the sparser networks obtained using the iterative magnitude pruning process.
Distilling Inductive Bias: Knowledge Distillation Beyond Model Compression
With the rapid development of computer vision, Vision Transformers (ViTs) offer the tantalizing prospect of unified information processing across visual and textual domains.
Knowledge Distillation in Vision Transformers: A Critical Review
A number of model compression methods have been proposed in the literature such as weight quantization, weight multiplexing, pruning and Knowledge Distillation (KD).
