Search Results for author:
Found 2 papers, 1 papers with code
Align-to-Distill: Trainable Attention Alignment for Knowledge Distillation in Neural Machine Translation
The Attention Alignment Module in A2D performs a dense head-by-head comparison between student and teacher attention heads across layers, turning the combinatorial mapping heuristics into a learning problem.
Adaptive Attention Link-based Regularization for Vision Transformers
Although transformer networks are recently employed in various vision tasks with outperforming performance, extensive training data and a lengthy training time are required to train a model to disregard an inductive bias.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
