Search Results for author:
Found 6 papers, 0 papers with code
DIVA: Dataset Derivative of a Learning Task
A learning task is a function from a training set to the validation error, which can be represented by a trained deep neural network (DNN).
Representation Consolidation from Multiple Expert Teachers
Indeed, we observe experimentally that standard distillation of task-specific teachers, or using these teacher representations directly, **reduces** downstream transferability compared to a task-agnostic generalist model.
Representation Consolidation for Training Expert Students
Traditionally, distillation has been used to train a student model to emulate the input/output functionality of a teacher.
Supervised Momentum Contrastive Learning for Few-Shot Classification
In this work we investigate the complementary roles of these two sources of information by combining instance-discriminative contrastive learning and supervised learning in a single framework called Supervised Momentum Contrastive learning (SUPMOCO).
Mixed-Privacy Forgetting in Deep Networks
We show that the influence of a subset of the training samples can be removed -- or "forgotten" -- from the weights of a network trained on large-scale image classification tasks, and we provide strong computable bounds on the amount of remaining information after forgetting.
LQF: Linear Quadratic Fine-Tuning
Classifiers that are linear in their parameters, and trained by optimizing a convex loss function, have predictable behavior with respect to changes in the training data, initial conditions, and optimization.
