no code implementations • ICML 2020 • Riccardo Grazzi, Saverio Salzo, Massimiliano Pontil, Luca Franceschi

We study a general class of bilevel optimization problems, in which the upper-level objective is defined via the solution of a fixed point equation.

no code implementations • 1 Jul 2024 • Vladimir R. Kostic, Karim Lounici, Gregoire Pacreau, Pietro Novelli, Giacomo Turri, Massimiliano Pontil

We introduce NCP (Neural Conditional Probability), a novel operator-theoretic approach for learning conditional distributions with a particular focus on inference tasks.

no code implementations • 28 Jun 2024 • Pietro Novelli, Marco Pratticò, Massimiliano Pontil, Carlo Ciliberto

Policy Mirror Descent (PMD) is a powerful and theoretically sound methodology for sequential decision-making.

no code implementations • 13 Jun 2024 • Timothée Devergne, Vladimir Kostic, Michele Parrinello, Massimiliano Pontil

We contrast our approach to more common ones based on the transfer operator, showing that it can provably learn the spectral properties of the unbiased system from biased data.

no code implementations • 9 Jun 2024 • Arya Akhavan, Karim Lounici, Massimiliano Pontil, Alexandre B. Tsybakov

We study the contextual continuum bandits problem, where the learner sequentially receives a side information vector and has to choose an action in a convex set, minimizing a function associated to the context.

no code implementations • 21 May 2024 • Vladimir R. Kostic, Karim Lounici, Helene Halconruy, Timothee Devergne, Massimiliano Pontil

Additionally, we elucidate how the distortion between the intrinsic energy-induced metric of the stochastic diffusion and the RKHS metric used for generator estimation impacts the spectral learning bounds.

1 code implementation • 18 Mar 2024 • Riccardo Grazzi, Massimiliano Pontil, Saverio Salzo

In the deterministic case, we provide a linear rate for AID and an improved linear rate for ITD which closely match the ones for the smooth setting.

3 code implementations • 23 Feb 2024 • Daniel Ordoñez-Apraez, Giulio Turrisi, Vladimir Kostic, Mario Martin, Antonio Agudo, Francesc Moreno-Noguer, Massimiliano Pontil, Claudio Semini, Carlos Mastalli

We present a comprehensive framework for studying and leveraging morphological symmetries in robotic systems.

1 code implementation • 28 Dec 2023 • Giacomo Turri, Vladimir Kostic, Pietro Novelli, Massimiliano Pontil

We present and analyze an algorithm designed for addressing vector-valued regression problems involving possibly infinite-dimensional input and output spaces.

no code implementations • 20 Dec 2023 • Prune Inzerilli, Vladimir Kostic, Karim Lounici, Pietro Novelli, Massimiliano Pontil

We study the evolution of distributions under the action of an ergodic dynamical system, which may be stochastic in nature.

1 code implementation • 12 Dec 2023 • Daniel Ordoñez-Apraez, Vladimir Kostic, Giulio Turrisi, Pietro Novelli, Carlos Mastalli, Claudio Semini, Massimiliano Pontil

We introduce the use of harmonic analysis to decompose the state space of symmetric robotic systems into orthogonal isotypic subspaces.

1 code implementation • 19 Jul 2023 • Vladimir R. Kostic, Pietro Novelli, Riccardo Grazzi, Karim Lounici, Massimiliano Pontil

We consider the general class of time-homogeneous stochastic dynamical systems, both discrete and continuous, and study the problem of learning a representation of the state that faithfully captures its dynamics.

1 code implementation • NeurIPS 2023 • Giacomo Meanti, Antoine Chatalic, Vladimir R. Kostic, Pietro Novelli, Massimiliano Pontil, Lorenzo Rosasco

Our empirical and theoretical analysis shows that the proposed estimators provide a sound and efficient way to learn large scale dynamical systems.

no code implementations • 3 Jun 2023 • Arya Akhavan, Evgenii Chzhen, Massimiliano Pontil, Alexandre B. Tsybakov

The first algorithm uses a gradient estimator based on randomization over the $\ell_2$ sphere due to Bach and Perchet (2016).

1 code implementation • NeurIPS 2023 • John Falk, Luigi Bonati, Pietro Novelli, Michele Parrinello, Massimiliano Pontil

Interatomic potentials learned using machine learning methods have been successfully applied to atomistic simulations.

1 code implementation • 22 Dec 2022 • Ruohan Wang, Isak Falk, Massimiliano Pontil, Carlo Ciliberto

Empirically, MeLa outperforms existing methods across a diverse range of benchmarks, in particular under a more challenging setting where the number of training tasks is limited and labels are task-specific.

no code implementations • 11 Oct 2022 • Ruohan Wang, Marco Ciccone, Giulia Luise, Andrew Yapp, Massimiliano Pontil, Carlo Ciliberto

A continual learning (CL) algorithm learns from a non-stationary data stream.

no code implementations • 17 Aug 2022 • Daniela A. Parletta, Andrea Paudice, Massimiliano Pontil, Saverio Salzo

In this work we study high probability bounds for stochastic subgradient methods under heavy tailed noise.

1 code implementation • 7 Jun 2022 • Riccardo Grazzi, Arya Akhavan, John Isak Texas Falk, Leonardo Cella, Massimiliano Pontil

This is a very strong notion of fairness, since the relative rank is not directly observed by the agent and depends on the underlying reward model and on the distribution of rewards.

no code implementations • 30 May 2022 • Leonardo Cella, Karim Lounici, Massimiliano Pontil

We aim to leverage this information in order to learn a new downstream bandit task, which shares the same representation.

no code implementations • 27 May 2022 • Arya Akhavan, Evgenii Chzhen, Massimiliano Pontil, Alexandre B. Tsybakov

We present a novel gradient estimator based on two function evaluations and randomization on the $\ell_1$-sphere.

1 code implementation • 27 May 2022 • Vladimir Kostic, Pietro Novelli, Andreas Maurer, Carlo Ciliberto, Lorenzo Rosasco, Massimiliano Pontil

We formalize a framework to learn the Koopman operator from finite data trajectories of the dynamical system.

1 code implementation • 15 Apr 2022 • Pietro Novelli, Luigi Bonati, Massimiliano Pontil, Michele Parrinello

Present-day atomistic simulations generate long trajectories of ever more complex systems.

no code implementations • 21 Feb 2022 • Leonardo Cella, Karim Lounici, Grégoire Pacreau, Massimiliano Pontil

We study the problem of transfer-learning in the setting of stochastic linear bandit tasks.

1 code implementation • 8 Feb 2022 • Dimitri Meunier, Massimiliano Pontil, Carlo Ciliberto

We study the theoretical properties of a kernel ridge regression estimator based on such representation, for which we prove universal consistency and excess risk bounds.

2 code implementations • NeurIPS 2023 • Riccardo Grazzi, Massimiliano Pontil, Saverio Salzo

We analyse a general class of bilevel problems, in which the upper-level problem consists in the minimization of a smooth objective function and the lower-level problem is to find the fixed point of a smooth contraction map.

no code implementations • NeurIPS 2021 • Mark Herbster, Stephen Pasteris, Fabio Vitale, Massimiliano Pontil

Users are in a social network and the learner is aided by a-priori knowledge of the strengths of the social links between all pairs of users.

no code implementations • NeurIPS 2021 • Andreas Maurer, Massimiliano Pontil

We prove analogues of the popular bounded difference inequality (also called McDiarmid's inequality) for functions of independent random variables under sub-gaussian and sub-exponential conditions.

no code implementations • NeurIPS 2021 • Ruohan Wang, Massimiliano Pontil, Carlo Ciliberto

Few-shot learning is a central problem in meta-learning, where learners must quickly adapt to new tasks given limited training data.

no code implementations • NeurIPS 2021 • Nicolò Cesa-Bianchi, Pierre Laforgue, Andrea Paudice, Massimiliano Pontil

We introduce and analyze MT-OMD, a multitask generalization of Online Mirror Descent (OMD) which operates by sharing updates between tasks.

no code implementations • 30 Mar 2021 • Giulia Denevi, Massimiliano Pontil, Carlo Ciliberto

Standard meta-learning for representation learning aims to find a common representation to be shared across multiple tasks.

no code implementations • 11 Feb 2021 • Andreas Maurer, Massimiliano Pontil

We prove concentration inequalities for functions of independent random variables {under} sub-gaussian and sub-exponential conditions.

no code implementations • NeurIPS 2021 • Arya Akhavan, Massimiliano Pontil, Alexandre B. Tsybakov

We study the problem of distributed zero-order optimization for a class of strongly convex functions.

Optimization and Control Statistics Theory Statistics Theory

no code implementations • 14 Dec 2020 • Andreas Maurer, Daniela A. Parletta, Andrea Paudice, Massimiliano Pontil

Designing learning algorithms that are resistant to perturbations of the underlying data distribution is a problem of wide practical and theoretical importance.

no code implementations • 7 Dec 2020 • Leonardo Cella, Claudio Gentile, Massimiliano Pontil

Unlike known model selection efforts in the recent bandit literature, our algorithm exploits the specific structure of the problem to learn the unknown parameters of the expected loss function so as to identify the best arm as quickly as possible.

no code implementations • NeurIPS 2020 • Evgenii Chzhen, Christophe Denis, Mohamed Hebiri, Luca Oneto, Massimiliano Pontil

We study the problem of learning an optimal regression function subject to a fairness constraint.

no code implementations • NeurIPS 2020 • Luca Oneto, Michele Donini, Giulia Luise, Carlo Ciliberto, Andreas Maurer, Massimiliano Pontil

One way to reach this goal is by modifying the data representation in order to meet certain fairness constraints.

1 code implementation • NeurIPS 2020 • Giulia Denevi, Massimiliano Pontil, Carlo Ciliberto

However, these methods may perform poorly on heterogeneous environments of tasks, where the complexity of the tasks’ distribution cannot be captured by a single meta- parameter vector.

no code implementations • NeurIPS 2020 • Andreas Maurer, Massimiliano Pontil

Exponential bounds on the estimation error are given for the plug-in estimator of weighted areas under the ROC curve.

no code implementations • 13 Nov 2020 • Riccardo Grazzi, Massimiliano Pontil, Saverio Salzo

Bilevel optimization problems are receiving increasing attention in machine learning as they provide a natural framework for hyperparameter optimization and meta-learning.

no code implementations • 25 Aug 2020 • Giulia Denevi, Massimiliano Pontil, Carlo Ciliberto

However, these methods may perform poorly on heterogeneous environments of tasks, where the complexity of the tasks' distribution cannot be captured by a single meta-parameter vector.

no code implementations • 29 Jul 2020 • Giulia Luise, Massimiliano Pontil, Carlo Ciliberto

The Generative Adversarial Networks (GAN) framework is a well-established paradigm for probability matching and realistic sample generation.

1 code implementation • 11 Jul 2020 • Giulia Denevi, Dimitris Stamos, Massimiliano Pontil

We propose a method to learn a common bias vector for a growing sequence of low-variance tasks.

1 code implementation • 29 Jun 2020 • Riccardo Grazzi, Luca Franceschi, Massimiliano Pontil, Saverio Salzo

We study a general class of bilevel problems, consisting in the minimization of an upper-level objective which depends on the solution to a parametric fixed-point equation.

1 code implementation • 23 Jun 2020 • Rosanna Turrisi, Rémi Flamary, Alain Rakotomamonjy, Massimiliano Pontil

The problem of domain adaptation on an unlabeled target dataset using knowledge from multiple labelled source datasets is becoming increasingly important.

no code implementations • NeurIPS 2020 • Arya Akhavan, Massimiliano Pontil, Alexandre B. Tsybakov

The gradient is estimated by a randomized procedure involving two function evaluations and a smoothing kernel.

no code implementations • NeurIPS 2020 • Evgenii Chzhen, Christophe Denis, Mohamed Hebiri, Luca Oneto, Massimiliano Pontil

It demands the distribution of the predicted output to be independent of the sensitive attribute.

no code implementations • ICML 2020 • Leonardo Cella, Alessandro Lazaric, Massimiliano Pontil

The goal is to select a learning algorithm which works well on average over a class of bandits tasks, that are sampled from a task-distribution.

no code implementations • 23 Mar 2020 • Feliks Hibraj, Marcello Pelillo, Saverio Salzo, Massimiliano Pontil

Second, we use a Nystrom-type subsampling approach, which allows for a training phase with a smaller number of data points, so to reduce the computational cost.

1 code implementation • ICLR 2021 • Henry Gouk, Timothy M. Hospedales, Massimiliano Pontil

Our bound is highly relevant for fine-tuning, because providing a network with a good initialisation based on transfer learning means that learning can modify the weights less, and hence achieve tighter generalisation.

1 code implementation • NeurIPS 2019 • Giulia Denevi, Dimitris Stamos, Carlo Ciliberto, Massimiliano Pontil

We study the problem of learning a series of tasks in a fully online Meta-Learning setting.

1 code implementation • 18 Oct 2019 • Michele Donini, Luca Franceschi, Massimiliano Pontil, Orchid Majumder, Paolo Frasconi

We study the problem of fitting task-specific learning rate schedules from the perspective of hyperparameter optimization, aiming at good generalization.

no code implementations • 25 Sep 2019 • Michele Donini, Luca Franceschi, Orchid Majumder, Massimiliano Pontil, Paolo Frasconi

We study the problem of fitting task-specific learning rate schedules from the perspective of hyperparameter optimization.

no code implementations • NeurIPS 2020 • Luca Oneto, Michele Donini, Andreas Maurer, Massimiliano Pontil

Developing learning methods which do not discriminate subgroups in the population is a central goal of algorithmic fairness.

1 code implementation • NeurIPS 2019 • Evgenii Chzhen, Christophe Denis, Mohamed Hebiri, Luca Oneto, Massimiliano Pontil

We study the problem of fair binary classification using the notion of Equal Opportunity.

1 code implementation • NeurIPS 2019 • Giulia Luise, Saverio Salzo, Massimiliano Pontil, Carlo Ciliberto

We present a novel algorithm to estimate the barycenter of arbitrary probability distributions with respect to the Sinkhorn divergence.

2 code implementations • 28 Mar 2019 • Luca Franceschi, Mathias Niepert, Massimiliano Pontil, Xiao He

With this work, we propose to jointly learn the graph structure and the parameters of graph convolutional networks (GCNs) by approximately solving a bilevel program that learns a discrete probability distribution on the edges of the graph.

Ranked #3 on Node Classification on Cora: fixed 20 node per class

1 code implementation • 25 Mar 2019 • Giulia Denevi, Carlo Ciliberto, Riccardo Grazzi, Massimiliano Pontil

We study the problem of learning-to-learn: inferring a learning algorithm that works well on tasks sampled from an unknown distribution.

no code implementations • 2 Mar 2019 • Giulia Luise, Dimitris Stamos, Massimiliano Pontil, Carlo Ciliberto

We study the interplay between surrogate methods for structured prediction and techniques from multitask learning designed to leverage relationships between surrogate outputs.

no code implementations • 5 Feb 2019 • Andreas Maurer, Massimiliano Pontil

The method to derive uniform bounds with Gaussian and Rademacher complexities is extended to the case where the sample average is replaced by a nonlinear statistic.

no code implementations • 29 Jan 2019 • Luca Oneto, Michele Donini, Massimiliano Pontil

We tackle the problem of algorithmic fairness, where the goal is to avoid the unfairly influence of sensitive information, in the general context of regression with possible continuous sensitive attributes.

no code implementations • NeurIPS 2018 • Giulia Denevi, Carlo Ciliberto, Dimitris Stamos, Massimiliano Pontil

We show that, in this setting, the LTL problem can be reformulated as a Least Squares (LS) problem and we exploit a novel meta- algorithm to efficiently solve it.

no code implementations • NeurIPS 2018 • Jordan Frecon, Saverio Salzo, Massimiliano Pontil

Regression with group-sparsity penalty plays a central role in high-dimensional prediction problems.

no code implementations • 19 Oct 2018 • Luca Oneto, Michele Donini, Amon Elders, Massimiliano Pontil

In this paper we show how it is possible to get the best of both worlds: optimize model accuracy and fairness without explicitly using the sensitive feature in the functional form of the model, thereby treating different individuals equally.

2 code implementations • 13 Jun 2018 • Luca Franceschi, Riccardo Grazzi, Massimiliano Pontil, Saverio Salzo, Paolo Frasconi

In (Franceschi et al., 2018) we proposed a unified mathematical framework, grounded on bilevel programming, that encompasses gradient-based hyperparameter optimization and meta-learning.

2 code implementations • NeurIPS 2018 • Giulia Luise, Alessandro Rudi, Massimiliano Pontil, Carlo Ciliberto

Applications of optimal transport have recently gained remarkable attention thanks to the computational advantages of entropic regularization.

no code implementations • 6 Apr 2018 • Alessandro Rudi, Leonard Wossnig, Carlo Ciliberto, Andrea Rocchetto, Massimiliano Pontil, Simone Severini

Simulating the time-evolution of quantum mechanical systems is BQP-hard and expected to be one of the foremost applications of quantum computers.

no code implementations • 21 Mar 2018 • Giulia Denevi, Carlo Ciliberto, Dimitris Stamos, Massimiliano Pontil

In learning-to-learn the goal is to infer a learning algorithm that works well on a class of tasks sampled from an unknown meta distribution.

no code implementations • 11 Mar 2018 • Andreas Maurer, Massimiliano Pontil

We provide sharp empirical estimates of expectation, variance and normal approximation for a class of statistics whose variation in any argument does not change too much when another argument is modified.

2 code implementations • NeurIPS 2018 • Michele Donini, Luca Oneto, Shai Ben-David, John Shawe-Taylor, Massimiliano Pontil

It encourages the conditional risk of the learned classifier to be approximately constant with respect to the sensitive variable.

1 code implementation • 18 Dec 2017 • Luca Franceschi, Michele Donini, Paolo Frasconi, Massimiliano Pontil

We consider a class of a nested optimization problems involving inner and outer objectives.

no code implementations • 26 Jul 2017 • Carlo Ciliberto, Mark Herbster, Alessandro Davide Ialongo, Massimiliano Pontil, Andrea Rocchetto, Simone Severini, Leonard Wossnig

Recently, increased computational power and data availability, as well as algorithmic advances, have led machine learning techniques to impressive results in regression, classification, data-generation and reinforcement learning tasks.

no code implementations • 27 Jun 2017 • Carlo Ciliberto, Dimitris Stamos, Massimiliano Pontil

A standard optimization strategy is based on formulating the problem as one of low rank matrix factorization which, however, leads to a non-convex problem.

no code implementations • NeurIPS 2017 • Carlo Ciliberto, Alessandro Rudi, Lorenzo Rosasco, Massimiliano Pontil

However, in practice assuming the tasks to be linearly related might be restrictive, and allowing for nonlinear structures is a challenge.

2 code implementations • ICML 2017 • Luca Franceschi, Michele Donini, Paolo Frasconi, Massimiliano Pontil

We study two procedures (reverse-mode and forward-mode) for computing the gradient of the validation error with respect to the hyperparameters of any iterative learning algorithm such as stochastic gradient descent.

no code implementations • NeurIPS 2016 • Mark Herbster, Stephen Pasteris, Massimiliano Pontil

We study the problem of completing a binary matrix in an online learning setting.

no code implementations • 27 Oct 2016 • Pierre Alquier, The Tien Mai, Massimiliano Pontil

We consider the problem of transfer learning in an online setting.

no code implementations • 5 Jun 2016 • Andreas Maurer, Massimiliano Pontil

Multi-task learning and one-vs-all multi-category learning are treated as examples.

no code implementations • CVPR 2016 • Peixi Peng, Tao Xiang, Yao-Wei Wang, Massimiliano Pontil, Shaogang Gong, Tiejun Huang, Yonghong Tian

Most existing person re-identification (Re-ID) approaches follow a supervised learning framework, in which a large number of labelled matching pairs are required for training.

no code implementations • 4 Jan 2016 • Andrew M. McDonald, Massimiliano Pontil, Dimitris Stamos

The spectral $k$-support norm enjoys good estimation properties in low rank matrix learning problems, empirically outperforming the trace norm.

no code implementations • 27 Dec 2015 • Andrew M. McDonald, Massimiliano Pontil, Dimitris Stamos

We note that the spectral box-norm is essentially equivalent to the cluster norm, a multitask learning regularizer introduced by [Jacob et al. 2009a], and which in turn can be interpreted as a perturbation of the spectral k-support norm.

no code implementations • CVPR 2015 • Dimitris Stamos, Samuele Martelli, Moin Nabi, Andrew McDonald, Vittorio Murino, Massimiliano Pontil

However, previous work has highlighted the possible danger of simply training a model from the combined datasets, due to the presence of bias.

no code implementations • 23 May 2015 • Andreas Maurer, Massimiliano Pontil, Bernardino Romera-Paredes

In particular, focusing on the important example of half-space learning, we derive the regime in which multitask representation learning is beneficial over independent task learning, as a function of the sample size, the number of tasks and the intrinsic data dimensionality.

no code implementations • NeurIPS 2014 • Andrew M. McDonald, Massimiliano Pontil, Dimitris Stamos

The $k$-support norm has successfully been applied to sparse vector prediction problems.

no code implementations • 6 Mar 2014 • Andrew M. McDonald, Massimiliano Pontil, Dimitris Stamos

We further extend the $k$-support norm to matrices, and we observe that it is a special case of the matrix cluster norm.

no code implementations • 8 Feb 2014 • Andreas Maurer, Massimiliano Pontil, Bernardino Romera-Paredes

From concentration inequalities for the suprema of Gaussian or Rademacher processes an inequality is derived.

no code implementations • NeurIPS 2013 • Bernardino Romera-Paredes, Massimiliano Pontil

We study the problem of learning a tensor from a set of linear measurements.

no code implementations • 25 Mar 2013 • Andreas Argyriou, Luca Baldassarre, Charles A. Micchelli, Massimiliano Pontil

During the past years there has been an explosion of interest in learning methods based on sparsity regularization.

no code implementations • NeurIPS 2012 • Arthur Gretton, Dino Sejdinovic, Heiko Strathmann, Sivaraman Balakrishnan, Massimiliano Pontil, Kenji Fukumizu, Bharath K. Sriperumbudur

A means of parameter selection for the two-sample test based on the MMD is proposed.

no code implementations • 4 Sep 2012 • Andreas Maurer, Massimiliano Pontil, Bernardino Romera-Paredes

We investigate the use of sparse coding and dictionary learning in the context of multitask and transfer learning.

no code implementations • NeurIPS 2010 • Jean Morales, Charles A. Micchelli, Massimiliano Pontil

We study the problem of learning a sparse linear regression vector under additional conditions on the structure of its sparsity pattern.

no code implementations • NeurIPS 2008 • Mark Herbster, Massimiliano Pontil, Sergio R. Galeano

Given an $n$-vertex weighted tree with structural diameter $S$ and a subset of $m$ vertices, we present a technique to compute a corresponding $m \times m$ Gram matrix of the pseudoinverse of the graph Laplacian in $O(n+ m^2 + m S)$ time.

no code implementations • NeurIPS 2008 • Mark Herbster, Guy Lever, Massimiliano Pontil

Current on-line learning algorithms for predicting the labelling of a graph have an important limitation in the case of large diameter graphs; the number of mistakes made by such algorithms may be proportional to the square root of the number of vertices, even when tackling simple problems.

Cannot find the paper you are looking for? You can
Submit a new open access paper.