Search Results for author:
Found 3 papers, 0 papers with code
Spirit Distillation: A Model Compression Method with Multi-domain Knowledge Transfer
To further improve the robustness of the student, we extend SD to Enhanced Spirit Distillation (ESD) in exploiting a more comprehensive knowledge by introducing the proximity domain which is similar to the target domain for feature extraction.
Spirit Distillation: Precise Real-time Semantic Segmentation of Road Scenes with Insufficient Data
Inspired by the ideas of Fine-tuning-based Transfer Learning (FTT) and feature-based knowledge distillation, we propose a new knowledge distillation method for cross-domain knowledge transference and efficient data-insufficient network training, named Spirit Distillation(SD), which allow the student network to mimic the teacher network to extract general features, so that a compact and accurate student network can be trained for real-time semantic segmentation of road scenes.
Activation Map Adaptation for Effective Knowledge Distillation
Model compression becomes a recent trend due to the requirement of deploying neural networks on embedded and mobile devices.
