Search Results for author:
Found 93 papers, 29 papers with code
On the Iteration Complexity of Hypergradient Computations
We study a general class of bilevel optimization problems, in which the upper-level objective is defined via the solution of a fixed point equation.
Neural Conditional Probability for Inference
We introduce NCP (Neural Conditional Probability), a novel operator-theoretic approach for learning conditional distributions with a particular focus on inference tasks.
Operator World Models for Reinforcement Learning
Policy Mirror Descent (PMD) is a powerful and theoretically sound methodology for sequential decision-making.
From Biased to Unbiased Dynamics: An Infinitesimal Generator Approach
We contrast our approach to more common ones based on the transfer operator, showing that it can provably learn the spectral properties of the unbiased system from biased data.
Contextual Continuum Bandits: Static Versus Dynamic Regret
We study the contextual continuum bandits problem, where the learner sequentially receives a side information vector and has to choose an action in a convex set, minimizing a function associated to the context.
Learning the Infinitesimal Generator of Stochastic Diffusion Processes
Additionally, we elucidate how the distortion between the intrinsic energy-induced metric of the stochastic diffusion and the RKHS metric used for generator estimation impacts the spectral learning bounds.
Nonsmooth Implicit Differentiation: Deterministic and Stochastic Convergence Rates
In the deterministic case, we provide a linear rate for AID and an improved linear rate for ITD which closely match the ones for the smooth setting.
Morphological Symmetries in Robotics
We present a comprehensive framework for studying and leveraging morphological symmetries in robotic systems.
A randomized algorithm to solve reduced rank operator regression
We present and analyze an algorithm designed for addressing vector-valued regression problems involving possibly infinite-dimensional input and output spaces.
Consistent Long-Term Forecasting of Ergodic Dynamical Systems
We study the evolution of distributions under the action of an ergodic dynamical system, which may be stochastic in nature.
Dynamics Harmonic Analysis of Robotic Systems: Application in Data-Driven Koopman Modelling
We introduce the use of harmonic analysis to decompose the state space of symmetric robotic systems into orthogonal isotypic subspaces.
Learning invariant representations of time-homogeneous stochastic dynamical systems
We consider the general class of time-homogeneous stochastic dynamical systems, both discrete and continuous, and study the problem of learning a representation of the state that faithfully captures its dynamics.
Estimating Koopman operators with sketching to provably learn large scale dynamical systems
Our empirical and theoretical analysis shows that the proposed estimators provide a sound and efficient way to learn large scale dynamical systems.
Gradient-free optimization of highly smooth functions: improved analysis and a new algorithm
The first algorithm uses a gradient estimator based on randomization over the $\ell_2$ sphere due to Bach and Perchet (2016).
Transfer learning for atomistic simulations using GNNs and kernel mean embeddings
Interatomic potentials learned using machine learning methods have been successfully applied to atomistic simulations.
Robust Meta-Representation Learning via Global Label Inference and Classification
Empirically, MeLa outperforms existing methods across a diverse range of benchmarks, in particular under a more challenging setting where the number of training tasks is limited and labels are task-specific.
Schedule-Robust Online Continual Learning
A continual learning (CL) algorithm learns from a non-stationary data stream.
High Probability Bounds for Stochastic Subgradient Schemes with Heavy Tailed Noise
In this work we study high probability bounds for stochastic subgradient methods under heavy tailed noise.
Group Meritocratic Fairness in Linear Contextual Bandits
This is a very strong notion of fairness, since the relative rank is not directly observed by the agent and depends on the underlying reward model and on the distribution of rewards.
Meta Representation Learning with Contextual Linear Bandits
We aim to leverage this information in order to learn a new downstream bandit task, which shares the same representation.
A gradient estimator via L1-randomization for online zero-order optimization with two point feedback
We present a novel gradient estimator based on two function evaluations and randomization on the $\ell_1$-sphere.
Learning Dynamical Systems via Koopman Operator Regression in Reproducing Kernel Hilbert Spaces
We formalize a framework to learn the Koopman operator from finite data trajectories of the dynamical system.
Characterizing metastable states with the help of machine learning
Present-day atomistic simulations generate long trajectories of ever more complex systems.
Multi-task Representation Learning with Stochastic Linear Bandits
We study the problem of transfer-learning in the setting of stochastic linear bandit tasks.
Distribution Regression with Sliced Wasserstein Kernels
We study the theoretical properties of a kernel ridge regression estimator based on such representation, for which we prove universal consistency and excess risk bounds.
Bilevel Optimization with a Lower-level Contraction: Optimal Sample Complexity without Warm-start
We analyse a general class of bilevel problems, in which the upper-level problem consists in the minimization of a smooth objective function and the lower-level problem is to find the fixed point of a smooth contraction map.
A Gang of Adversarial Bandits
Users are in a social network and the learner is aided by a-priori knowledge of the strengths of the social links between all pairs of users.
Concentration inequalities under sub-Gaussian and sub-exponential conditions
We prove analogues of the popular bounded difference inequality (also called McDiarmid's inequality) for functions of independent random variables under sub-gaussian and sub-exponential conditions.
The Role of Global Labels in Few-Shot Classification and How to Infer Them
Few-shot learning is a central problem in meta-learning, where learners must quickly adapt to new tasks given limited training data.
Multitask Online Mirror Descent
We introduce and analyze MT-OMD, a multitask generalization of Online Mirror Descent (OMD) which operates by sharing updates between tasks.
Conditional Meta-Learning of Linear Representations
Standard meta-learning for representation learning aims to find a common representation to be shared across multiple tasks.
Some Hoeffding- and Bernstein-type Concentration Inequalities
We prove concentration inequalities for functions of independent random variables {under} sub-gaussian and sub-exponential conditions.
Distributed Zero-Order Optimization under Adversarial Noise
We study the problem of distributed zero-order optimization for a class of strongly convex functions.
Optimization and Control Statistics Theory Statistics Theory
Robust Unsupervised Learning via L-Statistic Minimization
Designing learning algorithms that are resistant to perturbations of the underlying data distribution is a problem of wide practical and theoretical importance.
Online Model Selection: a Rested Bandit Formulation
Unlike known model selection efforts in the recent bandit literature, our algorithm exploits the specific structure of the problem to learn the unknown parameters of the expected loss function so as to identify the best arm as quickly as possible.
Fair regression via plug-in estimator and recalibration with statistical guarantees
We study the problem of learning an optimal regression function subject to a fairness constraint.
Exploiting MMD and Sinkhorn Divergences for Fair and Transferable Representation Learning
One way to reach this goal is by modifying the data representation in order to meet certain fairness constraints.
The Advantage of Conditional Meta-Learning for Biased Regularization and Fine Tuning
However, these methods may perform poorly on heterogeneous environments of tasks, where the complexity of the tasks’ distribution cannot be captured by a single meta- parameter vector.
Estimating weighted areas under the ROC curve
Exponential bounds on the estimation error are given for the plug-in estimator of weighted areas under the ROC curve.
Convergence Properties of Stochastic Hypergradients
Bilevel optimization problems are receiving increasing attention in machine learning as they provide a natural framework for hyperparameter optimization and meta-learning.
The Advantage of Conditional Meta-Learning for Biased Regularization and Fine-Tuning
However, these methods may perform poorly on heterogeneous environments of tasks, where the complexity of the tasks' distribution cannot be captured by a single meta-parameter vector.
Generalization Properties of Optimal Transport GANs with Latent Distribution Learning
The Generative Adversarial Networks (GAN) framework is a well-established paradigm for probability matching and realistic sample generation.
Online Parameter-Free Learning of Multiple Low Variance Tasks
We propose a method to learn a common bias vector for a growing sequence of low-variance tasks.
On the Iteration Complexity of Hypergradient Computation
We study a general class of bilevel problems, consisting in the minimization of an upper-level objective which depends on the solution to a parametric fixed-point equation.
Multi-source Domain Adaptation via Weighted Joint Distributions Optimal Transport
The problem of domain adaptation on an unlabeled target dataset using knowledge from multiple labelled source datasets is becoming increasingly important.
Exploiting Higher Order Smoothness in Derivative-free Optimization and Continuous Bandits
The gradient is estimated by a randomized procedure involving two function evaluations and a smoothing kernel.
Meta-learning with Stochastic Linear Bandits
The goal is to select a learning algorithm which works well on average over a class of bandits tasks, that are sampled from a task-distribution.
Efficient Tensor Kernel methods for sparse regression
Second, we use a Nystrom-type subsampling approach, which allows for a training phase with a smaller number of data points, so to reduce the computational cost.
Distance-Based Regularisation of Deep Networks for Fine-Tuning
Our bound is highly relevant for fine-tuning, because providing a network with a good initialisation based on transfer learning means that learning can modify the weights less, and hence achieve tighter generalisation.
Online-Within-Online Meta-Learning
We study the problem of learning a series of tasks in a fully online Meta-Learning setting.
MARTHE: Scheduling the Learning Rate Via Online Hypergradients
We study the problem of fitting task-specific learning rate schedules from the perspective of hyperparameter optimization, aiming at good generalization.
Scheduling the Learning Rate Via Hypergradients: New Insights and a New Algorithm
We study the problem of fitting task-specific learning rate schedules from the perspective of hyperparameter optimization.
Learning Fair and Transferable Representations
Developing learning methods which do not discriminate subgroups in the population is a central goal of algorithmic fairness.
Leveraging Labeled and Unlabeled Data for Consistent Fair Binary Classification
We study the problem of fair binary classification using the notion of Equal Opportunity.
Sinkhorn Barycenters with Free Support via Frank-Wolfe Algorithm
We present a novel algorithm to estimate the barycenter of arbitrary probability distributions with respect to the Sinkhorn divergence.
Learning Discrete Structures for Graph Neural Networks
With this work, we propose to jointly learn the graph structure and the parameters of graph convolutional networks (GCNs) by approximately solving a bilevel program that learns a discrete probability distribution on the edges of the graph.
Ranked #3 on
Node Classification
on Cora: fixed 20 node per class
Learning-to-Learn Stochastic Gradient Descent with Biased Regularization
We study the problem of learning-to-learn: inferring a learning algorithm that works well on tasks sampled from an unknown distribution.
Leveraging Low-Rank Relations Between Surrogate Tasks in Structured Prediction
We study the interplay between surrogate methods for structured prediction and techniques from multitask learning designed to leverage relationships between surrogate outputs.
Uniform concentration and symmetrization for weak interactions
The method to derive uniform bounds with Gaussian and Rademacher complexities is extended to the case where the sample average is replaced by a nonlinear statistic.
General Fair Empirical Risk Minimization
We tackle the problem of algorithmic fairness, where the goal is to avoid the unfairly influence of sensitive information, in the general context of regression with possible continuous sensitive attributes.
Learning To Learn Around A Common Mean
We show that, in this setting, the LTL problem can be reformulated as a Least Squares (LS) problem and we exploit a novel meta- algorithm to efficiently solve it.
Bilevel learning of the Group Lasso structure
Regression with group-sparsity penalty plays a central role in high-dimensional prediction problems.
Taking Advantage of Multitask Learning for Fair Classification
In this paper we show how it is possible to get the best of both worlds: optimize model accuracy and fairness without explicitly using the sensitive feature in the functional form of the model, thereby treating different individuals equally.
Far-HO: A Bilevel Programming Package for Hyperparameter Optimization and Meta-Learning
In (Franceschi et al., 2018) we proposed a unified mathematical framework, grounded on bilevel programming, that encompasses gradient-based hyperparameter optimization and meta-learning.
Differential Properties of Sinkhorn Approximation for Learning with Wasserstein Distance
Applications of optimal transport have recently gained remarkable attention thanks to the computational advantages of entropic regularization.
Approximating Hamiltonian dynamics with the Nyström method
Simulating the time-evolution of quantum mechanical systems is BQP-hard and expected to be one of the foremost applications of quantum computers.
Incremental Learning-to-Learn with Statistical Guarantees
In learning-to-learn the goal is to infer a learning algorithm that works well on a class of tasks sampled from an unknown meta distribution.
Empirical bounds for functions with weak interactions
We provide sharp empirical estimates of expectation, variance and normal approximation for a class of statistics whose variation in any argument does not change too much when another argument is modified.
Empirical Risk Minimization under Fairness Constraints
It encourages the conditional risk of the learned classifier to be approximately constant with respect to the sensitive variable.
A Bridge Between Hyperparameter Optimization and Learning-to-learn
We consider a class of a nested optimization problems involving inner and outer objectives.
Quantum machine learning: a classical perspective
Recently, increased computational power and data availability, as well as algorithmic advances, have led machine learning techniques to impressive results in regression, classification, data-generation and reinforcement learning tasks.
Reexamining Low Rank Matrix Factorization for Trace Norm Regularization
A standard optimization strategy is based on formulating the problem as one of low rank matrix factorization which, however, leads to a non-convex problem.
Consistent Multitask Learning with Nonlinear Output Relations
However, in practice assuming the tasks to be linearly related might be restrictive, and allowing for nonlinear structures is a challenge.
Forward and Reverse Gradient-Based Hyperparameter Optimization
We study two procedures (reverse-mode and forward-mode) for computing the gradient of the validation error with respect to the hyperparameters of any iterative learning algorithm such as stochastic gradient descent.
Mistake Bounds for Binary Matrix Completion
We study the problem of completing a binary matrix in an online learning setting.
Regret Bounds for Lifelong Learning
We consider the problem of transfer learning in an online setting.
Bounds for Vector-Valued Function Estimation
Multi-task learning and one-vs-all multi-category learning are treated as examples.
Unsupervised Cross-Dataset Transfer Learning for Person Re-Identification
Most existing person re-identification (Re-ID) approaches follow a supervised learning framework, in which a large number of labelled matching pairs are required for training.
Fitting Spectral Decay with the $k$-Support Norm
The spectral $k$-support norm enjoys good estimation properties in low rank matrix learning problems, empirically outperforming the trace norm.
New Perspectives on $k$-Support and Cluster Norms
We note that the spectral box-norm is essentially equivalent to the cluster norm, a multitask learning regularizer introduced by [Jacob et al. 2009a], and which in turn can be interpreted as a perturbation of the spectral k-support norm.
Learning With Dataset Bias in Latent Subcategory Models
However, previous work has highlighted the possible danger of simply training a model from the combined datasets, due to the presence of bias.
The Benefit of Multitask Representation Learning
In particular, focusing on the important example of half-space learning, we derive the regime in which multitask representation learning is beneficial over independent task learning, as a function of the sample size, the number of tasks and the intrinsic data dimensionality.
Spectral k-Support Norm Regularization
The $k$-support norm has successfully been applied to sparse vector prediction problems.
New Perspectives on k-Support and Cluster Norms
We further extend the $k$-support norm to matrices, and we observe that it is a special case of the matrix cluster norm.
An Inequality with Applications to Structured Sparsity and Multitask Dictionary Learning
From concentration inequalities for the suprema of Gaussian or Rademacher processes an inequality is derived.
A New Convex Relaxation for Tensor Completion
We study the problem of learning a tensor from a set of linear measurements.
On Sparsity Inducing Regularization Methods for Machine Learning
During the past years there has been an explosion of interest in learning methods based on sparsity regularization.
Optimal kernel choice for large-scale two-sample tests
A means of parameter selection for the two-sample test based on the MMD is proposed.
Sparse coding for multitask and transfer learning
We investigate the use of sparse coding and dictionary learning in the context of multitask and transfer learning.
A Family of Penalty Functions for Structured Sparsity
We study the problem of learning a sparse linear regression vector under additional conditions on the structure of its sparsity pattern.
Fast Prediction on a Tree
Given an $n$-vertex weighted tree with structural diameter $S$ and a subset of $m$ vertices, we present a technique to compute a corresponding $m \times m$ Gram matrix of the pseudoinverse of the graph Laplacian in $O(n+ m^2 + m S)$ time.
Online Prediction on Large Diameter Graphs
Current on-line learning algorithms for predicting the labelling of a graph have an important limitation in the case of large diameter graphs; the number of mistakes made by such algorithms may be proportional to the square root of the number of vertices, even when tackling simple problems.
