Search Results for author:
Found 3 papers, 0 papers with code
Wisdom of Committee: Distilling from Foundation Model to Specialized Application Model
Further, to accommodate the dissimilarity among the teachers in the committee, we introduce DiverseDistill, which allows the student to understand the expertise of each teacher and extract task knowledge.
LEVI: Generalizable Fine-tuning via Layer-wise Ensemble of Different Views
By combining two complementing models, LEVI effectively suppresses problematic features in both the fine-tuning data and pre-trained model and preserves useful features for new tasks.
Talking Models: Distill Pre-trained Knowledge to Downstream Models via Interactive Communication
In this paper, we extend KD with an interactive communication process to help students of downstream tasks learn effectively from pre-trained foundation models.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
