Search Results for author:
Found 19 papers, 3 papers with code
A Negative Result on Gradient Matching for Selective Backprop
With increasing scale in model and dataset size, the training of deep neural networks becomes a massive computational burden.
Continual Learning with Low Rank Adaptation
Recent work using pretrained transformers has shown impressive performance when fine-tuned with data from the downstream problem of interest.
Renate: A Library for Real-World Continual Learning
We discuss requirements for the use of continual learning algorithms in practice, from which we derive design principles for Renate.
PASHA: Efficient HPO and NAS with Progressive Resource Allocation
Hyperparameter optimization (HPO) and neural architecture search (NAS) are methods of choice to obtain the best-in-class machine learning models, but in practice they can be costly to run.
Continual Learning with Transformers for Image Classification
This phenomenon is known as catastrophic forgetting and it is often difficult to prevent due to practical constraints, such as the amount of data that can be stored or the limited computation sources that can be used.
Gradient-Matching Coresets for Rehearsal-Based Continual Learning
Most widely-used CL methods rely on a rehearsal memory of data points to be reused while training on new data.
Memory Efficient Continual Learning with Transformers
Moreover, applications increasingly rely on large pre-trained neural networks, such as pre-trained Transformers, since the resources or data might not be available in sufficiently large quantities to practitioners to train the model from scratch.
Gradient-matching coresets for continual learning
We devise a coreset selection method based on the idea of gradient matching: The gradients induced by the coreset should match, as closely as possible, those induced by the original training dataset.
A resource-efficient method for repeated HPO and NAS problems
In this work we consider the problem of repeated hyperparameter and neural architecture search (HNAS).
Amazon SageMaker Autopilot: a white box AutoML solution at scale
AutoML systems provide a black-box solution to machine learning problems by selecting the right way of processing features, choosing an algorithm and tuning the hyperparameters of the entire pipeline.
A Linear Bandit for Seasonal Environments
Here we present a contextual bandit algorithm which detects and adapts to abrupt changes of the reward function and leverages previous estimations whenever the environment falls back to a previously observed state.
Learning to Rank in the Position Based Model with Bandit Feedback
Personalization is a crucial aspect of many online experiences.
Linear Bandits with Stochastic Delayed Feedback
Stochastic linear bandits are a natural and well-studied model for structured exploration/exploitation problems and are widely used in applications such as online marketing and recommendation.
On Context-Dependent Clustering of Bandits
We investigate a novel cluster-of-bandit algorithm CAB for collaborative recommendation tasks that implements the underlying feedback sharing mechanism by estimating the neighborhood of users in a context-dependent manner.
Online Clustering of Bandits
We introduce a novel algorithmic approach to content recommendation based on adaptive clustering of exploration-exploitation ("bandit") strategies.
A Gang of Bandits
Multi-armed bandit problems are receiving a great deal of attention because they adequately formalize the exploration-exploitation trade-offs arising in several industrially relevant applications, such as online advertisement and, more generally, recommendation systems.
A Linear Time Active Learning Algorithm for Link Classification
We provide a theoretical analysis within this model, showing that we can achieve an optimal (to whithin a constant factor) number of mistakes on any graph $G = (V, E)$ such that $|E|$ is at least order of $|V|^{3/2}$ by querying at most order of $|V|^{3/2}$ edge labels.
A Scalable Multiclass Algorithm for Node Classification
We introduce a scalable algorithm, MUCCA, for multiclass node classification in weighted graphs.
See the Tree Through the Lines: The Shazoo Algorithm
Although it is known how to predict the nodes of an unweighted tree in a nearly optimal way, in the weighted case a fully satisfactory algorithm is not available yet.
