Search Results for author: Sylvain Sardy

Found 5 papers, 3 papers with code

A phase transition for finding needles in nonlinear haystacks with LASSO artificial neural networks

1 code implementation21 Jan 2022 Xiaoyu Ma, Sylvain Sardy, Nick Hengartner, Nikolai Bobenko, Yen Ting Lin

To fit sparse linear associations, a LASSO sparsity inducing penalty with a single hyperparameter provably allows to recover the important features (needles) with high probability in certain regimes even if the sample size is smaller than the dimension of the input vector (haystack).

What needles do sparse neural networks find in nonlinear haystacks

no code implementations7 Jun 2020 Sylvain Sardy, Nicolas W Hengartner, Nikolai Bonenko, Yen Ting Lin

Using a sparsity inducing penalty in artificial neural networks (ANNs) avoids over-fitting, especially in situations where noise is high and the training set is small in comparison to the number of features.

Model selection with lasso-zero: adding straw to the haystack to better find needles

1 code implementation14 May 2018 Pascaline Descloux, Sylvain Sardy

The high-dimensional linear model $y = X \beta^0 + \epsilon$ is considered and the focus is put on the problem of recovering the support $S^0$ of the sparse vector $\beta^0.$ We introduce Lasso-Zero, a new $\ell_1$-based estimator whose novelty resides in an "overfit, then threshold" paradigm and the use of noise dictionaries concatenated to $X$ for overfitting the response.

Model Selection

Quantile universal threshold: model selection at the detection edge for high-dimensional linear regression

no code implementations5 Dec 2014 Jairo Diaz-Rodriguez, Sylvain Sardy

To estimate a sparse linear model from data with Gaussian noise, consilience from lasso and compressed sensing literatures is that thresholding estimators like lasso and the Dantzig selector have the ability in some situations to identify with high probability part of the significant covariates asymptotically, and are numerically tractable thanks to convexity.

Model Selection regression

Cannot find the paper you are looking for? You can Submit a new open access paper.