Search Results for author:
Found 108 papers, 28 papers with code
Neural reproducing kernel Banach spaces and representer theorems for deep networks
Studying the function spaces defined by neural networks helps to understand the corresponding learning models and their inductive bias.
Linear quadratic control of nonlinear systems with Koopman operator learning and the Nyström method
In this paper, we study how the Koopman operator framework can be combined with kernel methods to effectively control nonlinear dynamical systems.
Key Design Choices in Source-Free Unsupervised Domain Adaptation: An In-depth Empirical Analysis
This study provides a comprehensive benchmark framework for Source-Free Unsupervised Domain Adaptation (SF-UDA) in image classification, aiming to achieve a rigorous empirical understanding of the complex relationships between multiple key design factors in SF-UDA methods.
Efficient Numerical Integration in Reproducing Kernel Hilbert Spaces via Leverage Scores Sampling
In this work we consider the problem of numerical integration, i. e., approximating integrals with respect to a target probability measure using only pointwise evaluations of the integrand.
Sim2Real Bilevel Adaptation for Object Surface Classification using Vision-Based Tactile Sensors
Our evaluation is conducted on a dataset of tactile images obtained from a set of ten 3D printed YCB objects.
Shortcuts for causal discovery of nonlinear models by score matching
The use of simulated data in the field of causal discovery is ubiquitous due to the scarcity of annotated real data.
Estimating Koopman operators with sketching to provably learn large scale dynamical systems
Our empirical and theoretical analysis shows that the proposed estimators provide a sound and efficient way to learn large scale dynamical systems.
Scalable Causal Discovery with Score Matching
This paper demonstrates how to discover the whole causal graph from the second derivative of the log-likelihood in observational non-linear additive Gaussian noise models.
Causal Discovery with Score Matching on Additive Models with Arbitrary Noise
Causal discovery methods are intrinsically constrained by the set of assumptions needed to ensure structure identifiability.
Fast kernel methods for Data Quality Monitoring as a goodness-of-fit test
We here propose a machine learning approach for monitoring particle detectors in real-time.
Key Design Choices for Double-Transfer in Source-Free Unsupervised Domain Adaptation
Fine-tuning and Domain Adaptation emerged as effective strategies for efficiently transferring deep learning models to new target tasks.
Iterative regularization in classification via hinge loss diagonal descent
In this paper, we focus on iterative regularization in the context of classification.
Regularized ERM on random subspaces
We study a natural extension of classical empirical risk minimization, where the hypothesis space is a random subspace of a given space.
Top-Tuning: a study on transfer learning for an efficient alternative to fine tuning for image classification with fast kernel methods
In this paper, focusing on image classification, we consider a simple transfer learning approach exploiting pre-trained convolutional features as input for a fast-to-train kernel method.
Efficient Unsupervised Learning for Plankton Images
We compare our method with state-of-the-art unsupervised approaches, where a set of pre-defined hand-crafted features is used for clustering of plankton images.
Approximate Bayesian Neural Operators: Uncertainty Quantification for Parametric PDEs
Neural operators are a type of deep architecture that learns to solve (i. e. learns the nonlinear solution operator of) partial differential equations (PDEs).
Learn Fast, Segment Well: Fast Object Segmentation Learning on the iCub Robot
In this work, we focus on the instance segmentation task and provide a comprehensive study of different techniques that allow adapting an object segmentation model in presence of novel objects or different domains.
Stochastic Zeroth order Descent with Structured Directions
For smooth convex functions we prove almost sure convergence of the iterates and a convergence rate on the function values of the form $O(d/l k^{-c})$ for every $c<1/2$, which is arbitrarily close to the one of Stochastic Gradient Descent (SGD) in terms of number of iterations.
AdaTask: Adaptive Multitask Online Learning
We introduce and analyze AdaTask, a multitask online learning algorithm that adapts to the unknown structure of the tasks.
Learning Dynamical Systems via Koopman Operator Regression in Reproducing Kernel Hilbert Spaces
We formalize a framework to learn the Koopman operator from finite data trajectories of the dynamical system.
Learning new physics efficiently with nonparametric methods
We present a machine learning approach for model-independent new physics searches.
Physics Informed Shallow Machine Learning for Wind Speed Prediction
We train supervised learning algorithms using the past history of wind to predict its value at a future time (horizon).
An elementary analysis of ridge regression with random design
In this note, we provide an elementary analysis of the prediction error of ridge regression with random design.
Multiclass learning with margin: exponential rates with no bias-variance trade-off
We study the behavior of error bounds for multiclass classification under suitable margin conditions.
Iterative regularization for low complexity regularizers
Our approach is based on a primal-dual algorithm of which we analyze convergence and stability properties, even in the case where the original problem is unfeasible.
Nyström Kernel Mean Embeddings
Our main result is an upper bound on the approximation error of this procedure.
Scaling Gaussian Process Optimization by Evaluating a Few Unique Candidates Multiple Times
Computing a Gaussian process (GP) posterior has a computational cost cubical in the number of historical points.
Efficient Hyperparameter Tuning for Large Scale Kernel Ridge Regression
Our analysis shows the benefit of the proposed approach, that we hence incorporate in a library for large scale kernel methods to derive adaptively tuned solutions.
Mean Nyström Embeddings for Adaptive Compressive Learning
Compressive learning is an approach to efficient large scale learning based on sketching an entire dataset to a single mean embedding (the sketch), i. e. a vector of generalized moments.
Understanding neural networks with reproducing kernel Banach spaces
Characterizing the function spaces corresponding to neural networks can provide a way to understand their properties.
ParK: Sound and Efficient Kernel Ridge Regression by Feature Space Partitions
We introduce ParK, a new large-scale solver for kernel ridge regression.
Ada-BKB: Scalable Gaussian Process Optimization on Continuous Domains by Adaptive Discretization
In this paper, we introduce Ada-BKB (Adaptive Budgeted Kernelized Bandit), a no-regret Gaussian process optimization algorithm for functions on continuous domains, that provably runs in $O(T^2 d_\text{eff}^2)$, where $d_\text{eff}$ is the effective dimension of the explored space, and which is typically much smaller than $T$.
Learning to predict target location with turbulent odor plumes
Animal behavior and neural recordings show that the brain is able to measure both the intensity of an odor and the timing of odor encounters.
From inexact optimization to learning via gradient concentration
Optimization in machine learning typically deals with the minimization of empirical objectives defined by training data.
On the Emergence of Whole-body Strategies from Humanoid Robot Push-recovery Learning
Balancing and push-recovery are essential capabilities enabling humanoid robots to solve complex locomotion tasks.
Structured Prediction for CRiSP Inverse Kinematics Learning with Misspecified Robot Models
Among these, computing the inverse kinematics of a redundant robot arm poses a significant challenge due to the non-linear structure of the robot, the hard joint constraints and the non-invertible kinematics map.
From Handheld to Unconstrained Object Detection: a Weakly-supervised On-line Learning Approach
We show that the robot can improve adaptation to novel domains, either by interacting with a human teacher (Active Learning) or with an autonomous supervision (Semi-supervised Learning).
Fast Object Segmentation Learning with Kernel-based Methods for Robotics
Our approach is validated on the YCB-Video dataset which is widely adopted in the computer vision and robotics community, demonstrating that we can achieve and even surpass performance of the state-of-the-art, with a significant reduction (${\sim}6\times$) of the training time.
Fast Region Proposal Learning for Object Detection for Robotics
This shortens training time while maintaining state-of-the-art performance.
Decentralised Learning with Random Features and Distributed Gradient Descent
Under standard source and capacity assumptions, we establish high probability bounds on the predictive performance for each agent as a function of the step size, number of iterations, inverse spectral gap of the communication matrix and number of Random Features.
For interpolating kernel machines, minimizing the norm of the ERM solution minimizes stability
We study the average $\mbox{CV}_{loo}$ stability of kernel ridge-less regression and derive corresponding risk bounds.
Kernel methods through the roof: handling billions of points efficiently
Kernel methods provide an elegant and principled approach to nonparametric learning, but so far could hardly be used in large scale problems, since na\"ive implementations scale poorly with data size.
Interpolation and Learning with Scale Dependent Kernels
We study the learning properties of nonparametric ridge-less least squares.
Regularized ERM on random subspaces
We study a natural extension of classical empirical risk minimization, where the hypothesis space is a random subspace of a given space.
Iterative regularization for convex regularizers
We study iterative regularization for linear models, when the bias is convex but not necessarily strongly convex.
Asymptotics of Ridge (less) Regression under General Source Condition
We analyze the prediction error of ridge regression in an asymptotic regime where the sample size and dimension go to infinity at a proportional rate.
Hyperbolic Manifold Regression
Geometric representation learning has recently shown great promise in several machine learning settings, ranging from relational learning to language processing and generative models.
Near-linear Time Gaussian Process Optimization with Adaptive Batching and Resparsification
Gaussian processes (GP) are one of the most successful frameworks to model uncertainty.
Constructing fast approximate eigenspaces with application to the fast graph Fourier transforms
We investigate numerically efficient approximations of eigenspaces associated to symmetric and general matrices.
A General Framework for Consistent Structured Prediction with Implicit Loss Embeddings
We propose and analyze a novel theoretical and algorithmic framework for structured prediction.
Statistical and Computational Trade-Offs in Kernel K-Means
We investigate the efficiency of k-means in terms of both statistical and computational requirements.
Fast approximation of orthogonal matrices and application to PCA
We study the problem of approximating orthogonal matrices so that their application is numerically fast and yet accurate.
Gain with no Pain: Efficient Kernel-PCA by Nyström Sampling
In this paper, we propose and study a Nystr\"om based approach to efficient large scale kernel principal component analysis (PCA).
Multi-Scale Vector Quantization with Reconstruction Trees
Our main technical contribution is an analysis of the expected distortion achieved by the proposed algorithm, when the data are assumed to be sampled from a fixed unknown distribution.
Implicit Regularization of Accelerated Methods in Hilbert Spaces
We study learning properties of accelerated gradient descent methods for linear least-squares in Hilbert spaces.
Reproducing kernel Hilbert spaces on manifolds: Sobolev and Diffusion spaces
We study reproducing kernel Hilbert spaces (RKHS) on a Riemannian manifold.
Gaussian Process Optimization with Adaptive Sketching: Scalable and No Regret
Moreover, we show that our procedure selects at most $\tilde{O}(d_{eff})$ points, where $d_{eff}$ is the effective dimension of the explored space, which is typically much smaller than both $d$ and $t$.
Theory III: Dynamics and Generalization in Deep Networks
In particular, gradient descent induces a dynamics of the normalized weights which converge for $t \to \infty$ to an equilibrium which corresponds to a minimum norm (or maximum margin) solution.
Beating SGD Saturation with Tail-Averaging and Minibatching
While stochastic gradient descent (SGD) is one of the major workhorses in machine learning, the learning properties of many practically used variants are poorly understood.
On Fast Leverage Score Sampling and Optimal Learning
Leverage score sampling provides an appealing way to perform approximate computations for large matrices.
Learning with SGD and Random Features
Sketching and stochastic gradient methods are arguably the most common techniques to derive efficient large scale learning algorithms.
Manifold Structured Prediction
Structured prediction provides a general framework to deal with supervised problems where the outputs have semantically rich structure.
Dirichlet-based Gaussian Processes for Large-scale Calibrated Classification
In this paper, we study the problem of deriving fast and accurate classification algorithms with uncertainty quantification.
Speeding-up Object Detection Training for Robotics with FALKON
We address the size and imbalance of training data by exploiting the stochastic subsampling intrinsic into the method and a novel, fast, bootstrapping approach.
Iterate averaging as regularization for stochastic gradient descent
We propose and analyze a variant of the classic Polyak-Ruppert averaging scheme, broadly used in stochastic gradient methods.
Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spaces
In this paper, we study regression problems over a separable Hilbert space with the square loss, covering non-parametric regression over a reproducing kernel Hilbert space.
Theory of Deep Learning III: explaining the non-overfitting puzzle
In this note, we show that the dynamics associated to gradient descent minimization of nonlinear networks is topologically equivalent, near the asymptotically stable minima of the empirical error, to linear gradient system in a quadratic potential with a degenerate (for square loss) or almost degenerate (for logistic or crossentropy loss) Hessian.
Optimal Rates for Learning with Nyström Stochastic Gradient Methods
In the setting of nonparametric regression, we propose and study a combination of stochastic gradient methods with Nystr\"om subsampling, allowing multiple passes over the data and mini-batches.
Are we done with object recognition? The iCub robot's perspective
We report on an extensive study of the benefits and limitations of current deep learning approaches to object recognition in robot vision scenarios, introducing a novel dataset used for our investigation.
Solving $\ell^p\!$-norm regularization with tensor kernels
In this paper, we discuss how a suitable family of tensor kernels can be used to efficiently solve nonparametric extensions of $\ell^p$ regularized learning methods.
Don't relax: early stopping for convex regularization
We consider the problem of designing efficient regularization algorithms when regularization is encoded by a (strongly) convex functional.
Generalization Properties of Doubly Stochastic Learning Algorithms
In this paper, we provide an in-depth theoretical analysis for different variants of doubly stochastic learning algorithms within the setting of nonparametric regression in a reproducing kernel Hilbert space and considering the square loss.
FALKON: An Optimal Large Scale Kernel Method
In this paper, we take a substantial step in scaling up kernel methods, proposing FALKON, a novel algorithm that allows to efficiently process millions of points.
Consistent Multitask Learning with Nonlinear Output Relations
However, in practice assuming the tasks to be linearly related might be restrictive, and allowing for nonlinear structures is a challenge.
Convergence of the Forward-Backward Algorithm: Beyond the Worst Case with the Help of Geometry
We provide a comprehensive study of the convergence of the forward-backward algorithm under suitable geometric conditions, such as conditioning or {\L}ojasiewicz properties.
Optimal Learning for Multi-pass Stochastic Gradient Methods
We analyze the learning properties of the stochastic gradient method when multiple passes over the data and mini-batches are allowed.
Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Review
The paper characterizes classes of functions for which deep learning can be exponentially better than shallow learning.
Optimal Rates for Multi-pass Stochastic Gradient Methods
As a byproduct, we derive optimal convergence results for batch gradient methods (even in the non-attainable cases).
Generalization Properties and Implicit Regularization for Multiple Passes SGM
We study the generalization properties of stochastic gradient methods for learning with convex loss functions and linearly parameterized functions.
A Consistent Regularization Approach for Structured Prediction
We propose and analyze a regularization approach for structured prediction problems.
Incremental Robot Learning of New Objects with Fixed Update Time
We consider object recognition in the context of lifelong learning, where a robotic agent learns to discriminate between a growing number of object classes as it accumulates experience about the environment.
Generalization Properties of Learning with Random Features
We study the generalization properties of ridge regression with random features in the statistical learning framework.
Incremental Semiparametric Inverse Dynamics Learning
This paper presents a novel approach for incremental semiparametric inverse dynamics learning.
NYTRO: When Subsampling Meets Early Stopping
Early stopping is a well known approach to reduce the time complexity for performing training and model selection of large scale learning machines.
Holographic Embeddings of Knowledge Graphs
Learning embeddings of entities and relations is an efficient and versatile method to perform machine learning on relational data such as knowledge graphs.
Ranked #8 on
Link Prediction
on FB15k
Enabling Depth-driven Visual Attention on the iCub Humanoid Robot: Instructions for Use and New Perspectives
The importance of depth perception in the interactions that humans have within their nearby space is a well established fact.
Deep Convolutional Networks are Hierarchical Kernel Machines
In i-theory a typical layer of a hierarchical architecture consists of HW modules pooling the dot products of the inputs to the layer with the transformations of a few templates under a group.
Less is More: Nyström Computational Regularization
We study Nystr\"om type subsampling approaches to large scale kernel methods, and prove learning bounds in the statistical learning setting, where random sampling and high probability estimates are considered.
Learning Multiple Visual Tasks while Discovering their Structure
Multi-task learning is a natural approach for computer vision applications that require the simultaneous solution of several distinct but related problems, e. g. object detection, classification, tracking of multiple agents, or denoising, to name a few.
Real-world Object Recognition with Off-the-shelf Deep Conv Nets: How Many Objects can iCub Learn?
In this paper we investigate such possibility, while taking further steps in developing a computational vision system to be embedded on a robotic platform, the iCub humanoid robot.
Convex Learning of Multiple Tasks and their Structure
In this context a fundamental question is how to incorporate the tasks structure in the learning problem. We tackle this question by studying a general computational framework that allows to encode a-priori knowledge of the tasks structure in the form of a convex penalty; in this setting a variety of previously proposed methods can be recovered as special cases, including linear and non-linear approaches.
Iterative Regularization for Learning with Convex Loss Functions
We consider the problem of supervised learning with convex loss functions and propose a new form of iterative regularization based on the subgradient method.
On Invariance and Selectivity in Representation Learning
We discuss data representation which can be learned automatically from data, are invariant to transformations, and at the same time selective, in the sense that two points have the same representation only if they are one the transformation of the other.
On the Sample Complexity of Subspace Learning
A large number of algorithms in machine learning, from principal component analysis (PCA), and its non-linear (kernel) extensions, to more recent spectral embedding and support estimation methods, rely on estimating a linear subspace from samples.
Learning An Invariant Speech Representation
Recognition of speech, and in particular the ability to generalize and learn from small sets of labelled examples like humans do, depends on an appropriate representation of the acoustic input.
Learning with incremental iterative regularization
Within a statistical learning setting, we propose and study an iterative regularization algorithm for least squares defined by an incremental gradient method.
A Deep Representation for Invariance And Music Classification
We present the main theoretical and computational aspects of a framework for unsupervised learning of invariant audio representations, empirically evaluated on music genre classification.
Unsupervised Learning of Invariant Representations in Hierarchical Architectures
It also suggests that the main computational goal of the ventral stream of visual cortex is to provide a hierarchical representation of new objects/images which is invariant to transformations, stable, and discriminative for recognition---and that this representation may be continuously learned in an unsupervised way during development and visual experience.
iCub World: Friendly Robots Help Building Good Vision Data-Sets
In this paper we present and start analyzing the iCub World data-set, an object recognition data-set, we acquired using a Human-Robot Interaction (HRI) scheme and the iCub humanoid robot platform.
On Learnability, Complexity and Stability
We consider the fundamental question of learnability of a hypotheses class in the supervised learning setting and in the general learning setting introduced by Vladimir Vapnik.
Multiclass Learning with Simplex Coding
In this paper we dicuss a novel framework for multiclass learning, defined by a suitable coding/decoding strategy, namely the simplex coding, that allows to generalize to multiple classes a relaxation approach commonly used in binary classification.
Learning Probability Measures with respect to Optimal Transport Metrics
We study the problem of estimating, in the sense of optimal transport metrics, a measure which is assumed supported on a manifold embedded in a Hilbert space.
Learning Manifolds with K-Means and K-Flats
We study the problem of estimating a manifold from random samples.
Learning Sets with Separating Kernels
We consider the problem of learning a set from random samples.
Kernels for Vector-Valued Functions: a Review
Kernel methods are among the most popular techniques in machine learning.
Spectral Regularization for Support Estimation
In this paper we consider the problem of learning from data the support of a probability distribution when the distribution {\em does not} have a density (with respect to some reference measure).
A Primal-Dual Algorithm for Group Sparse Regularization with Overlapping Groups
We deal with the problem of variable selection when variables must be selected group-wise, with possibly overlapping groups defined a priori.
On Invariance in Hierarchical Models
A goal of central importance in the study of hierarchical models for object recognition -- and indeed the visual cortex -- is that of understanding quantitatively the trade-off between invariance and selectivity, and how invariance and discrimination properties contribute towards providing an improved representation useful for learning from data.
