Search Results for author:
Found 6 papers, 2 papers with code
Distillation Scaling Laws
We provide a distillation scaling law that estimates distilled model performance based on a compute budget and its allocation between the student and teacher.
Theory, Analysis, and Best Practices for Sigmoid Self-Attention
Attention is a key part of the transformer architecture.
Poly-View Contrastive Learning
Contrastive learning typically matches pairs of related views among a number of unrelated negative views.
Optimal Regret Bounds for Collaborative Learning in Bandits
We consider regret minimization in a general collaborative multi-agent multi-armed bandit model, in which each agent faces a finite set of arms and may communicate with other agents through a central controller.
Chained Generalisation Bounds
This work discusses how to derive upper bounds for the expected generalisation error of supervised learning algorithms by means of the chaining technique.
