Search Results for author:
Found 25 papers, 10 papers with code
CHANI: Correlation-based Hawkes Aggregation of Neurons with bio-Inspiration
The present work aims at proving mathematically that a neural network inspired by biology can learn a classification task thanks to local transformations only.
Derivatives of Stochastic Gradient Descent in parametric optimization
We investigate the behavior of the derivatives of the iterates of Stochastic Gradient Descent (SGD) with respect to that parameter and show that they are driven by an inexact SGD recursion on a different objective function, perturbed by the convergence of the original SGD.
Convergence of Message Passing Graph Neural Networks with Generic Aggregation On Large Random Graphs
We study the convergence of message passing graph neural networks on random graph models to their continuous counterpart as the number of nodes tends to infinity.
Gradient scarcity with Bilevel Optimization for Graph Learning
To alleviate this issue, we study several solutions: we propose to resort to latent graph learning using a Graph-to-Graph model (G2G), graph regularization to impose a prior structure on the graph, or optimizing on a larger graph than the original one with a reduced diameter.
On the Robustness of Text Vectorizers
A fundamental issue in machine learning is the robustness of the model with respect to changes in the input.
A Lower Bound and a Near-Optimal Algorithm for Bilevel Empirical Risk Minimization
Bilevel optimization problems, which are problems where two optimization problems are nested, have more and more applications in machine learning.
The derivatives of Sinkhorn-Knopp converge
We show that the derivatives of the Sinkhorn-Knopp algorithm, or iterative proportional fitting procedure, converge towards the derivatives of the entropic regularization of the optimal transport problem with a locally uniform linear convergence rate.
Benchopt: Reproducible, efficient and collaborative optimization benchmarks
Numerical validation is at the core of machine learning research as it allows to assess the actual impact of new methods, and to confirm the agreement between theory and practice.
Automatic differentiation of nonsmooth iterative algorithms
Is there a limiting object for nonsmooth piggyback automatic differentiation (AD)?
A framework for bilevel optimization that enables stochastic and global variance reduction algorithms
However, computing the gradient of the value function involves solving a linear system, which makes it difficult to derive unbiased stochastic estimates.
Supervised learning of analysis-sparsity priors with automatic differentiation
In most situations, this dictionary is not known, and is to be recovered from pairs of ground-truth signals and measurements, by minimizing the reconstruction error.
A theory of optimal convex regularization for low-dimensional recovery
To perform recovery, we consider the minimization of a convex regularizer subject to a data fit constraint.
On the Universality of Graph Neural Networks on Large Random Graphs
In the large graph limit, GNNs are known to converge to certain "continuous" models known as c-GNNs, which directly enables a study of their approximation power on random graph models.
Implicit differentiation for fast hyperparameter selection in non-smooth convex learning
Finding the optimal hyperparameters of a model can be cast as a bilevel optimization problem, typically solved using zero-order techniques.
Model identification and local linear convergence of coordinate descent
For composite nonsmooth optimization problems, Forward-Backward algorithm achieves model identification (e. g. support identification for the Lasso) after a finite number of iterations, provided the objective function is regular enough.
Convergence and Stability of Graph Convolutional Networks on Large Random Graphs
We study properties of Graph Convolutional Networks (GCNs) by analyzing their behavior on standard models of random graphs, where nodes are represented by random latent variables and edges are drawn according to a similarity kernel.
Automated data-driven selection of the hyperparameters for Total-Variation based texture segmentation
This work extends the Stein's Unbiased GrAdient estimator of the Risk of Deledalle et al. to the case of correlated Gaussian noise, deriving a general automatic tuning of regularization parameters.
Implicit differentiation of Lasso-type models for hyperparameter optimization
Our approach scales to high-dimensional data by leveraging the sparsity of the solutions.
Sparse and Smooth: improved guarantees for Spectral Clustering in the Dynamic Stochastic Block Model
Existing results show that, in the relatively sparse case where the expected degree grows logarithmically with the number of nodes, guarantees in the static case can be extended to the dynamic case and yield improved error bounds when the DSBM is sufficiently smooth in time, that is, the communities do not change too much between two time steps.
Linear Support Vector Regression with Linear Constraints
This paper studies the addition of linear constraints to the Support Vector Regression (SVR) when the kernel is linear.
Block based refitting in $\ell_{12}$ sparse regularisation
This is done through the use of refitting block penalties that only act on the support of the estimated solution.
Dual Extrapolation for Sparse Generalized Linear Models
Generalized Linear Models (GLM) form a wide class of regression and classification models, where prediction is a function of a linear combination of the input variables.
Characterizing the maximum parameter of the total-variation denoising through the pseudo-inverse of the divergence
Though, it is of importance when tuning the regularization parameter as it allows fixing an upper-bound on the grid for which the optimal parameter is sought.
Low Complexity Regularization of Linear Inverse Problems
Inverse problems and regularization theory is a central theme in contemporary signal processing, where the goal is to reconstruct an unknown signal from partial indirect, and possibly noisy, measurements of it.
Model Consistency of Partly Smooth Regularizers
We show that a generalized "irrepresentable condition" implies stable model selection under small noise perturbations in the observations and the design matrix, when the regularization parameter is tuned proportionally to the noise level.
