1 code implementation • 4 May 2022 • Alina Kolesnikova, Yuri Kuratov, Vasily Konovalov, Mikhail Burtsev
We propose two simple yet effective alignment techniques to make knowledge distillation to the students with reduced vocabulary.
Knowledge Distillation