Search Results for author:
Found 59 papers, 19 papers with code
Closed-form Filtering for Non-linear Systems
Sequential Bayesian Filtering aims to estimate the current state distribution of a Hidden Markov Model, given the past observations.
GloptiNets: Scalable Non-Convex Optimization with Certificates
We present a novel approach to non-convex optimization with certificates, which handles smooth functions on the hypercube or on the torus.
Non-Parametric Learning of Stochastic Differential Equations with Fast Rates of Convergence
We propose a novel non-parametric learning paradigm for the identification of drift and diffusion coefficients of non-linear stochastic differential equations, which relies upon discrete-time observations of the state.
Approximation of optimization problems with constraints through kernel Sum-Of-Squares
Assuming further that the functions appearing in the problem are smooth, focusing on pointwise equality constraints enables the use of scattering inequalities to mitigate the curse of dimensionality in sampling the constraints.
Vector-Valued Least-Squares Regression under Output Regularity Assumptions
We propose and analyse a reduced-rank method for solving least-squares regression problems with infinite dimensional output.
Active Labeling: Streaming Stochastic Gradients
The workhorse of machine learning is stochastic gradient descent.
Non-Convex Optimization with Certificates and Fast Rates Through Kernel Sums of Squares
We consider potentially non-convex optimization problems, for which optimal rates of approximation depend on the dimension of the parameter space and the smoothness of the function to be optimized.
On the Benefits of Large Learning Rates for Kernel Methods
This paper studies an intriguing phenomenon related to the good generalization performance of estimators obtained by using large learning rates within gradient descent algorithms.
Measuring dissimilarity with diffeomorphism invariance
Measures of similarity (or dissimilarity) are a key ingredient to many machine learning algorithms.
Nyström Kernel Mean Embeddings
Our main result is an upper bound on the approximation error of this procedure.
Near-optimal estimation of smooth transport maps with kernel sums-of-squares
It was recently shown that under smoothness conditions, the squared Wasserstein distance between two distributions could be efficiently computed with appealing statistical error upper bounds.
Learning PSD-valued functions using kernel sums-of-squares
Shape constraints such as positive semi-definiteness (PSD) for matrices or convexity for functions play a central role in many applications in machine learning and sciences, including metric learning, optimal transport, and economics.
Sampling from Arbitrary Functions via PSD Models
In many areas of applied statistics and machine learning, generating an arbitrary number of independent and identically distributed (i. i. d.)
Mixability made efficient: Fast online multiclass logistic regression
Mixability has been shown to be a powerful tool to obtain algorithms with optimal regret.
PSD Representations for Effective Probability Models
Finding a good way to model probability densities is key to probabilistic inference.
A Note on Optimizing Distributions using Kernel Mean Embeddings
Kernel mean embeddings are a popular tool that consists in representing probability measures by their infinite-dimensional mean embeddings in a reproducing kernel Hilbert space.
Beyond Tikhonov: Faster Learning with Self-Concordant Losses via Iterative Regularization
The theory of spectral filtering is a remarkable tool to understand the statistical properties of learning with kernels.
On the Consistency of Max-Margin Losses
The resulting loss is also a generalization of the binary support vector machine and it is consistent under milder conditions on the discrete loss.
Beyond Tikhonov: faster learning with self-concordant losses, via iterative regularization
The theory of spectral filtering is a remarkable tool to understand the statistical properties of learning with kernels.
Online nonparametric regression with Sobolev kernels
In this work we investigate the variation of the online kernelized ridge regression algorithm in the setting of $d-$dimensional adversarial nonparametric regression.
Disambiguation of weak supervision with exponential convergence rates
Machine learning approached through supervised learning requires expensive annotation of data.
Fast rates in structured prediction
Discrete supervised learning problems such as classification are often tackled by introducing a continuous surrogate problem akin to regression.
A Dimension-free Computational Upper-bound for Smooth Optimal Transport Estimation
It is well-known that plug-in statistical estimation of optimal transport suffers from the curse of dimensionality.
Statistics Theory Optimization and Control Statistics Theory 62G05
Finding Global Minima via Kernel Approximations
We consider the global minimization of smooth functions based solely on function evaluations.
Overcoming the curse of dimensionality with Laplacian regularization in semi-supervised learning
As annotations of data can be scarce in large-scale practical problems, leveraging unlabelled examples is one of the most important aspects of machine learning.
Learning Output Embeddings in Structured Prediction
A powerful and flexible approach to structured prediction consists in embedding the structured objects to be predicted into a feature space of possibly infinite dimension by means of output kernels, and then, solving a regression problem in this output space.
Non-parametric Models for Non-negative Functions
The paper is complemented by an experimental evaluation of the model showing its effectiveness in terms of formulation, algorithmic derivation and practical results on the problems of density estimation, regression with heteroscedastic errors, and multiple quantile regression.
Consistent Structured Prediction with Max-Min Margin Markov Networks
Max-margin methods for binary classification such as the support vector machine (SVM) have been extended to the structured prediction setting under the name of max-margin Markov networks ($M^3N$), or more generally structural SVMs.
Kernel methods through the roof: handling billions of points efficiently
Kernel methods provide an elegant and principled approach to nonparametric learning, but so far could hardly be used in large scale problems, since na\"ive implementations scale poorly with data size.
Interpolation and Learning with Scale Dependent Kernels
We study the learning properties of nonparametric ridge-less least squares.
Structured and Localized Image Restoration
We present a novel approach to image restoration that leverages ideas from localized structured prediction and non-linear multi-task learning.
Efficient improper learning for online logistic regression
We consider the setting of online logistic regression and consider the regret with respect to the 2-ball of radius B.
Structured Prediction with Partial Labelling through the Infimum Loss
Annotating datasets is one of the main costs in nowadays supervised learning.
A General Framework for Consistent Structured Prediction with Implicit Loss Embeddings
We propose and analyze a novel theoretical and algorithmic framework for structured prediction.
Statistical Limits of Supervised Quantum Learning
Within the framework of statistical learning theory it is possible to bound the minimum number of samples required by a learner to reach a target accuracy.
Gain with no Pain: Efficient Kernel-PCA by Nyström Sampling
In this paper, we propose and study a Nystr\"om based approach to efficient large scale kernel principal component analysis (PCA).
Globally Convergent Newton Methods for Ill-conditioned Generalized Self-concordant Losses
In this paper, we study large-scale convex optimization algorithms based on the Newton method applied to regularized generalized self-concordant losses, which include logistic regression and softmax regression.
Efficient online learning with kernels for adversarial large scale problems
For $d$-dimensional inputs, we provide a (close to) optimal regret of order $O((\log n)^{d+1})$ with per-round time complexity and space complexity $O((\log n)^{2d})$.
Beyond Least-Squares: Fast Rates for Regularized Empirical Risk Minimization through Self-Concordance
We consider learning methods based on the regularization of a convex empirical risk by a squared Hilbertian norm, a setting that includes linear predictors and non-linear predictors through positive-definite kernels.
Affine Invariant Covariance Estimation for Heavy-Tailed Distributions
Denoting $\text{cond}(\mathbf{S})$ the condition number of $\mathbf{S}$, the computational cost of the novel estimator is $O(d^2 n + d^3\log(\text{cond}(\mathbf{S})))$, which is comparable to the cost of the sample covariance estimator in the statistically interesing regime $n \ge d$.
A General Theory for Structured Prediction with Smooth Convex Surrogates
In this work we provide a theoretical framework for structured prediction that generalizes the existing theory of surrogate methods for binary and multiclass classification based on estimating conditional probabilities with smooth convex surrogates (e. g. logistic regression).
Massively scalable Sinkhorn distances via the Nyström method
The Sinkhorn "distance", a variant of the Wasserstein distance with entropic regularization, is an increasingly popular tool in machine learning and statistical inference.
On Fast Leverage Score Sampling and Optimal Learning
Leverage score sampling provides an appealing way to perform approximate computations for large matrices.
Sharp Analysis of Learning with Discrete Losses
The problem of devising learning strategies for discrete losses (e. g., multilabeling, ranking) is currently addressed with methods and theoretical analyses ad-hoc for each loss.
Learning with SGD and Random Features
Sketching and stochastic gradient methods are arguably the most common techniques to derive efficient large scale learning algorithms.
Manifold Structured Prediction
Structured prediction provides a general framework to deal with supervised problems where the outputs have semantically rich structure.
Localized Structured Prediction
Key to structured prediction is exploiting the problem structure to simplify the learning process.
Differential Properties of Sinkhorn Approximation for Learning with Wasserstein Distance
Applications of optimal transport have recently gained remarkable attention thanks to the computational advantages of entropic regularization.
Statistical Optimality of Stochastic Gradient Descent on Hard Learning Problems through Multiple Passes
We consider stochastic gradient descent (SGD) for least-squares regression with potentially several passes over the data.
Approximating Hamiltonian dynamics with the Nyström method
Simulating the time-evolution of quantum mechanical systems is BQP-hard and expected to be one of the foremost applications of quantum computers.
Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spaces
In this paper, we study regression problems over a separable Hilbert space with the square loss, covering non-parametric regression over a reproducing kernel Hilbert space.
Exponential convergence of testing error for stochastic gradient methods
We consider binary classification problems with positive definite kernels and square loss, and study the convergence rates of stochastic gradient methods.
FALKON: An Optimal Large Scale Kernel Method
In this paper, we take a substantial step in scaling up kernel methods, proposing FALKON, a novel algorithm that allows to efficiently process millions of points.
Consistent Multitask Learning with Nonlinear Output Relations
However, in practice assuming the tasks to be linearly related might be restrictive, and allowing for nonlinear structures is a challenge.
A Consistent Regularization Approach for Structured Prediction
We propose and analyze a regularization approach for structured prediction problems.
Generalization Properties of Learning with Random Features
We study the generalization properties of ridge regression with random features in the statistical learning framework.
NYTRO: When Subsampling Meets Early Stopping
Early stopping is a well known approach to reduce the time complexity for performing training and model selection of large scale learning machines.
Less is More: Nyström Computational Regularization
We study Nystr\"om type subsampling approaches to large scale kernel methods, and prove learning bounds in the statistical learning setting, where random sampling and high probability estimates are considered.
On the Sample Complexity of Subspace Learning
A large number of algorithms in machine learning, from principal component analysis (PCA), and its non-linear (kernel) extensions, to more recent spectral embedding and support estimation methods, rely on estimating a linear subspace from samples.
