1 code implementation • ECCV 2020 • Kunran Xu, Lai Rui, Yishi Li, Lin Gu
From this perspective, we systematically analyze the distillation mechanism and demonstrate that the L2-norm of the feature in penultimate layer would be too large under the influence of label noise, and the temperature T in KD could be regarded as a correction factor for L2-norm to suppress the impact of noise.
1 code implementation • 11 Jul 2022 • Kunran Xu, Huawei Zhang, Yishi Li, Yuhao Zhang, Rui Lai, Yi Liu
Tiny machine learning (TinyML), executing AI workloads on resource and power strictly restricted systems, is an important and challenging topic.