Search Results for author:
Found 40 papers, 19 papers with code
Benchopt: Reproducible, efficient and collaborative optimization benchmarks
Numerical validation is at the core of machine learning research as it allows to assess the actual impact of new methods, and to confirm the agreement between theory and practice.
Celer: a Fast Solver for the Lasso with Dual Extrapolation
Here, we propose an extrapolation technique starting from a sequence of iterates in the dual that leads to the construction of improved dual points.
Dual Extrapolation for Sparse Generalized Linear Models
Generalized Linear Models (GLM) form a wide class of regression and classification models, where prediction is a function of a linear combination of the input variables.
Implicit differentiation of Lasso-type models for hyperparameter optimization
Our approach scales to high-dimensional data by leveraging the sparsity of the solutions.
Implicit differentiation for fast hyperparameter selection in non-smooth convex learning
Finding the optimal hyperparameters of a model can be cast as a bilevel optimization problem, typically solved using zero-order techniques.
Optimal mini-batch and step sizes for SAGA
Using these bounds, and since the SAGA algorithm is part of this JacSketch family, we suggest a new standard practice for setting the step sizes and mini-batch size for SAGA that are competitive with a numerical grid search.
Statistical control for spatio-temporal MEG/EEG source imaging with desparsified multi-task Lasso
To deal with this, we adapt the desparsified Lasso estimator -- an estimator tailored for high dimensional linear model that asymptotically follows a Gaussian distribution under sparsity and moderate feature correlation assumptions -- to temporal data corrupted with autocorrelated noise.
Spatially relaxed inference on high-dimensional linear models
This calls for a reformulation of the statistical inference problem, that takes into account the underlying spatial structure: if covariates are locally correlated, it is acceptable to detect them up to a given spatial uncertainty.
GAP Safe Screening Rules for Sparse-Group-Lasso
We adapt to the case of Sparse-Group Lasso recent safe screening rules that discard early in the solver irrelevant features/groups.
GAP Safe Screening Rules for Sparse-Group Lasso
For statistical learning in high dimension, sparse regularizations have proven useful to boost both computational and statistical efficiency.
Stochastic smoothing of the top-K calibrated hinge loss for deep imbalanced classification
In modern classification tasks, the number of labels is getting larger and larger, as is the size of the datasets encountered in practice.
Handling correlated and repeated measurements with the smoothed multivariate square-root Lasso
Sparsity promoting norms are frequently used in high dimensional regression.
LassoBench: A High-Dimensional Hyperparameter Optimization Benchmark Suite for Lasso
While Weighted Lasso sparse regression has appealing statistical guarantees that would entail a major real-world impact in finance, genomics, and brain imaging applications, it is typically scarcely adopted due to its complex high-dimensional space composed by thousands of hyperparameters.
Safe Grid Search with Optimal Complexity
Popular machine learning estimators involve regularization parameters that can be challenging to tune, and standard strategies rely on grid search for this task.
Gap Safe screening rules for sparsity enforcing penalties
In high dimensional regression settings, sparsity enforcing penalties have proved useful to regularize the data-fitting term.
Score-Based Change Detection for Gradient-Based Learning Machines
The widespread use of machine learning algorithms calls for automatic change detection algorithms to monitor their behavior over time.
Efficient Smoothed Concomitant Lasso Estimation for High Dimensional Regression
In high dimensional settings, sparse structures are crucial for efficiency, both in term of memory, computation and performance.
From safe screening rules to working sets for faster Lasso-type solvers
For the Lasso estimator a WS is a set of features, while for a Group Lasso it refers to a set of groups.
Generalized Concomitant Multi-Task Lasso for sparse multimodal regression
Results on multimodal neuroimaging problems with M/EEG data are also reported.
On the benefits of output sparsity for multi-label classification
The modern multi-label problems are typically large-scale in terms of number of observations, features and labels, and the amount of labels can even be comparable with the amount of observations.
Characterizing the maximum parameter of the total-variation denoising through the pseudo-inverse of the divergence
Though, it is of importance when tuning the regularization parameter as it allows fixing an upper-bound on the grid for which the optimal parameter is sought.
Gossip Dual Averaging for Decentralized Optimization of Pairwise Functions
In decentralized networks (of sensors, connected objects, etc.
Mind the duality gap: safer rules for the Lasso
In this paper, we propose new versions of the so-called $\textit{safe rules}$ for the Lasso.
GAP Safe screening rules for sparse multi-task and multi-class models
The GAP Safe rule can cope with any iterative solver and we illustrate its performance on coordinate descent for multi-task Lasso, binary and multinomial logistic regression, demonstrating significant speed ups on all tested datasets with respect to previous safe rules.
Extending Gossip Algorithms to Distributed Estimation of U-Statistics
Efficient and robust algorithms for decentralized estimation in networks are essential to many distributed systems.
Adaptive Multinomial Matrix Completion
The task of estimating a matrix given a sample of observed entries is known as the \emph{matrix completion problem}.
Learning Heteroscedastic Models by Convex Programming under Group Sparsity
Popular sparse estimation methods based on $\ell_1$-relaxation, such as the Lasso and the Dantzig selector, require the knowledge of the variance of the noise in order to properly tune the regularization parameter.
Screening Rules for Lasso with Non-Convex Sparse Regularizers
Leveraging on the convexity of the Lasso problem , screening rules help in accelerating solvers by discarding irrelevant variables, during the optimization process.
Support recovery and sup-norm convergence rates for sparse pivotal estimation
In high dimensional sparse regression, pivotal estimators are estimators for which the optimal regularization parameter is independent of the noise level.
Provably Convergent Working Set Algorithm for Non-Convex Regularized Regression
Owing to their statistical properties, non-convex sparse regularizers have attracted much interest for estimating a sparse linear model from high dimensional data.
Screening Rules and its Complexity for Active Set Identification
Screening rules were recently introduced as a technique for explicitly identifying active structures such as sparsity, in optimization problem arising in machine learning.
Model identification and local linear convergence of coordinate descent
For composite nonsmooth optimization problems, Forward-Backward algorithm achieves model identification (e. g. support identification for the Lasso) after a finite number of iterations, provided the objective function is regular enough.
Statistical control for spatio-temporal MEG/EEG source imaging with desparsified mutli-task Lasso
To deal with this, we adapt the desparsified Lasso estimator ---an estimator tailored for high dimensional linear model that asymptotically follows a Gaussian distribution under sparsity and moderate feature correlation assumptions--- to temporal data corrupted with autocorrelated noise.
Block based refitting in $\ell_{12}$ sparse regularisation
This is done through the use of refitting block penalties that only act on the support of the estimated solution.
Differentially Private Coordinate Descent for Composite Empirical Risk Minimization
In this paper, we propose Differentially Private proximal Coordinate Descent (DP-CD), a new method to solve composite DP-ERM problems.
Supervised learning of analysis-sparsity priors with automatic differentiation
In most situations, this dictionary is not known, and is to be recovered from pairs of ground-truth signals and measurements, by minimizing the reconstruction error.
High-Dimensional Private Empirical Risk Minimization by Greedy Coordinate Descent
In this paper, we study differentially private empirical risk minimization (DP-ERM).
Identify ambiguous tasks combining crowdsourced labels by weighting Areas Under the Margin
We adapt the AUM to identify ambiguous tasks in crowdsourced learning scenarios, introducing the Weighted Areas Under the Margin (WAUM).
A two-head loss function for deep Average-K classification
Average-K classification is an alternative to top-K classification in which the number of labels returned varies with the ambiguity of the input image but must average to K over all the samples.
