no code implementations • 6 Dec 2022 • Usma Niyaz, Deepti R. Bathula
Unlike conventional techniques that share the same type of knowledge with all networks, we propose to train individual networks with different forms of information to enhance the learning process.
no code implementations • 21 Oct 2021 • Usma Niyaz, Deepti R. Bathula
Knowledge distillation (KD) is an effective model compression technique where a compact student network is taught to mimic the behavior of a complex and highly trained teacher network.