Search Results for author:
Found 2 papers, 2 papers with code
Establishing a stronger baseline for lightweight contrastive models
A common practice to address this problem is to introduce a pretrained contrastive teacher model and train the lightweight networks with distillation signals generated by the teacher.
Efficient Sub-structured Knowledge Distillation
Specifically, we transfer the knowledge from a teacher model to its student model by locally matching their predictions on all sub-structures, instead of the whole output space.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
