no code implementations • ICML 2020 • Stéphane d'Ascoli, Maria Refinetti, Giulio Biroli, Florent Krzakala

We demonstrate that the latter two contributions are the crux of the double descent: they lead to the overfitting peak at the interpolation threshold and to the decay of the test error upon overparametrization.

no code implementations • 24 Oct 2024 • Yatin Dandi, Luca Pesce, Hugo Cui, Florent Krzakala, Yue M. Lu, Bruno Loureiro

This provides a sharp description of the impact of feature learning in the generalization of two-layer neural networks, beyond the random features and lazy training regimes.

no code implementations • 1 Sep 2024 • Ziao Wang, Kilian Müller, Matthew Filipovich, Julien Launay, Ruben Ohana, Gustave Pariente, Safa Mokaadi, Charles Brossollet, Fabien Moreau, Alessandro Cappelli, Iacopo Poli, Igor Carron, Laurent Daudet, Florent Krzakala, Sylvain Gigan

Modern machine learning relies nearly exclusively on dedicated electronic hardware accelerators.

1 code implementation • 7 Aug 2024 • Antoine Maillard, Emanuele Troiani, Simon Martin, Florent Krzakala, Lenka Zdeborová

We consider the problem of learning a target function corresponding to a single hidden layer neural network, with a quadratic activation function after the first layer, and random weights.

1 code implementation • 4 Jun 2024 • Luca Arnaboldi, Yatin Dandi, Florent Krzakala, Bruno Loureiro, Luca Pesce, Ludovic Stephan

We study the impact of the batch size $n_b$ on the iteration time $T$ of training two-layer neural networks with one-pass stochastic gradient descent (SGD) on multi-index target functions of isotropic covariates.

1 code implementation • 24 May 2024 • Luca Arnaboldi, Yatin Dandi, Florent Krzakala, Luca Pesce, Ludovic Stephan

Here, we investigate the training dynamics of two-layer shallow neural networks trained with gradient-based algorithms, and discuss how they learn pertinent features in multi-index models, that is target functions with low-dimensional relevant directions.

1 code implementation • 24 May 2024 • Emanuele Troiani, Yatin Dandi, Leonardo Defilippis, Lenka Zdeborová, Bruno Loureiro, Florent Krzakala

Our findings unfold in three parts: (i) we identify under which conditions a trivial subspace can be learned with a single step of a first-order algorithm for any $\alpha\!>\! 0$; (ii) if the trivial subspace is empty, we provide necessary and sufficient conditions for the existence of an easy subspace where directions that can be learned only above a certain sample complexity $\alpha\!>\!\alpha_c$, where $\alpha_{c}$ marks a computational phase transition.

no code implementations • 7 Mar 2024 • Pierre Mergny, Justin Ko, Florent Krzakala, Lenka Zdeborová

We consider the task of estimating a low-rank matrix from non-linear and noisy observations.

no code implementations • 6 Mar 2024 • Pierre Mergny, Justin Ko, Florent Krzakala

We discuss the inhomogeneous spiked Wigner model, a theoretical framework recently introduced to study structured noise in various learning scenarios, through the prism of random matrix theory, with a specific focus on its spectral properties.

no code implementations • 21 Feb 2024 • Lucas Clarté, Adrien Vandenbroucque, Guillaume Dalle, Bruno Loureiro, Florent Krzakala, Lenka Zdeborová

We investigate popular resampling methods for estimating the uncertainty of statistical models, such as subsampling, bootstrap and the jackknife, and their performance in high-dimensional supervised regression tasks.

no code implementations • 8 Feb 2024 • Kasimir Tanner, Matteo Vilucchio, Bruno Loureiro, Florent Krzakala

This work investigates adversarial training in the context of margin-based linear classifiers in the high-dimensional regime where the dimension $d$ and the number of data points $n$ diverge with a fixed ratio $\alpha = n / d$.

1 code implementation • 7 Feb 2024 • Hugo Cui, Luca Pesce, Yatin Dandi, Florent Krzakala, Yue M. Lu, Lenka Zdeborová, Bruno Loureiro

In this manuscript, we investigate the problem of how two-layer neural networks learn features from data, and improve over the kernel regime, after being trained with a single gradient descent step.

no code implementations • 6 Feb 2024 • Hugo Cui, Freya Behrens, Florent Krzakala, Lenka Zdeborová

Many empirical studies have provided evidence for the emergence of algorithmic mechanisms (abilities) in the learning of language models, that lead to qualitative improvements of the model capabilities.

1 code implementation • 5 Feb 2024 • Yatin Dandi, Emanuele Troiani, Luca Arnaboldi, Luca Pesce, Lenka Zdeborová, Florent Krzakala

In particular, multi-pass GD with finite stepsize is found to overcome the limitations of gradient flow and single-pass GD given by the information exponent (Ben Arous et al., 2021) and leap exponent (Abbe et al., 2023) of the target function.

1 code implementation • 5 Oct 2023 • Hugo Cui, Florent Krzakala, Eric Vanden-Eijnden, Lenka Zdeborová

We study the problem of training a flow-based generative model, parametrized by a two-layer autoencoder, to sample from a high-dimensional Gaussian mixture.

1 code implementation • 27 Aug 2023 • Davide Ghio, Yatin Dandi, Florent Krzakala, Lenka Zdeborová

Recent years witnessed the development of powerful generative models based on flows, diffusion or autoregressive neural networks, achieving remarkable success in generating data from examples with applications in a broad range of areas.

1 code implementation • 30 May 2023 • Matteo Vilucchio, Emanuele Troiani, Vittorio Erba, Florent Krzakala

We study robust linear regression in high-dimension, when both the dimension $d$ and the number of data points $n$ diverge with a fixed ratio $\alpha=n/d$, and study a data model that includes outliers.

2 code implementations • 29 May 2023 • Luca Arnaboldi, Florent Krzakala, Bruno Loureiro, Ludovic Stephan

These insights are grounded in the reduction of SGD dynamics to a stochastic process in lower dimensions, where escaping mediocrity equates to calculating an exit time.

1 code implementation • 29 May 2023 • Yatin Dandi, Florent Krzakala, Bruno Loureiro, Luca Pesce, Ludovic Stephan

The picture drastically improves over multiple gradient steps: we show that a batch-size of $n = \mathcal{O}(d)$ is indeed enough to learn multiple target directions satisfying a staircase property, where more and more directions can be learned over time.

2 code implementations • 5 Mar 2023 • Lucas Clarté, Bruno Loureiro, Florent Krzakala, Lenka Zdeborová

Despite their incredible performance, it is well reported that deep neural networks tend to be overoptimistic about their prediction confidence.

1 code implementation • 17 Feb 2023 • Luca Pesce, Florent Krzakala, Bruno Loureiro, Ludovic Stephan

Motivated by the recent stream of results on the Gaussian universality of the test and training errors in generalized linear estimation, we ask ourselves the question: "when is a single Gaussian enough to characterize the error?".

1 code implementation • 12 Feb 2023 • Luca Arnaboldi, Ludovic Stephan, Florent Krzakala, Bruno Loureiro

This manuscript investigates the one-pass stochastic gradient descent (SGD) dynamics of a two-layer neural network trained on Gaussian data and labels generated by a similar, though not necessarily identical, target function.

no code implementations • 1 Feb 2023 • Hugo Cui, Florent Krzakala, Lenka Zdeborová

We consider the problem of learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights.

1 code implementation • 23 Oct 2022 • Lucas Clarté, Bruno Loureiro, Florent Krzakala, Lenka Zdeborová

Uncertainty quantification is a central challenge in reliable and trustworthy machine learning.

1 code implementation • 12 Oct 2022 • Cedric Gerbelot, Emanuele Troiani, Francesca Mignacco, Florent Krzakala, Lenka Zdeborova

We prove closed-form equations for the exact high-dimensional asymptotics of a family of first order gradient-based methods, learning an estimator (e. g. M-estimator, shallow neural network, ...) from observations on Gaussian data with empirical risk minimization.

2 code implementations • 26 May 2022 • Federica Gerace, Florent Krzakala, Bruno Loureiro, Ludovic Stephan, Lenka Zdeborová

We argue that there is a large universality class of high-dimensional input data for which we obtain the same minimum training loss as for Gaussian data with corresponding data covariance.

1 code implementation • 26 May 2022 • Luca Pesce, Bruno Loureiro, Florent Krzakala, Lenka Zdeborová

A simple model to study subspace clustering is the high-dimensional $k$-Gaussian mixture model where the cluster means are sparse vectors.

1 code implementation • 7 Feb 2022 • Lucas Clarté, Bruno Loureiro, Florent Krzakala, Lenka Zdeborová

In this manuscript, we characterise uncertainty for learning from limited number of samples of high-dimensional Gaussian input data and labels generated by the probit model.

2 code implementations • 1 Feb 2022 • Rodrigo Veiga, Ludovic Stephan, Bruno Loureiro, Florent Krzakala, Lenka Zdeborová

Despite the non-convex optimization landscape, over-parametrized shallow networks are able to achieve global convergence under gradient descent.

no code implementations • 31 Jan 2022 • Bruno Loureiro, Cédric Gerbelot, Maria Refinetti, Gabriele Sicuro, Florent Krzakala

From the sampling of data to the initialisation of parameters, randomness is ubiquitous in modern Machine Learning practice.

no code implementations • 29 Jan 2022 • Hugo Cui, Bruno Loureiro, Florent Krzakala, Lenka Zdeborová

We find that our rates tightly describe the learning curves for this class of data sets, and are also observed on real data.

no code implementations • 19 Jan 2022 • Ali Bereyhi, Bruno Loureiro, Florent Krzakala, Ralf R. Müller, Hermann Schulz-Baldes

Unlike the classical linear model, nonlinear generative models have been addressed sparsely in the literature of statistical learning.

no code implementations • NeurIPS 2021 • Bruno Loureiro, Gabriele Sicuro, Cedric Gerbelot, Alessandro Pacco, Florent Krzakala, Lenka Zdeborová

Generalised linear models for multi-class classification problems are one of the fundamental building blocks of modern machine learning tasks.

2 code implementations • 7 Jun 2021 • Bruno Loureiro, Gabriele Sicuro, Cédric Gerbelot, Alessandro Pacco, Florent Krzakala, Lenka Zdeborová

Generalised linear models for multi-class classification problems are one of the fundamental building blocks of modern machine learning tasks.

no code implementations • NeurIPS 2021 • Hugo Cui, Bruno Loureiro, Florent Krzakala, Lenka Zdeborová

In this work, we unify and extend this line of work, providing characterization of all regimes and excess error decay rates that can be observed in terms of the interplay of noise and regularization.

1 code implementation • 16 May 2021 • Sebastian Goldt, Florent Krzakala, Lenka Zdeborová, Nicolas Brunel

The advent of comprehensive synaptic wiring diagrams of large neural circuits has created the field of connectomics and given rise to a number of open research questions.

1 code implementation • 23 Feb 2021 • Maria Refinetti, Sebastian Goldt, Florent Krzakala, Lenka Zdeborová

Here, we show theoretically that two-layer neural networks (2LNN) with only a few hidden neurons can beat the performance of kernel learning on a simple Gaussian mixture classification task.

1 code implementation • NeurIPS 2021 • Bruno Loureiro, Cédric Gerbelot, Hugo Cui, Sebastian Goldt, Florent Krzakala, Marc Mézard, Lenka Zdeborová

While still solvable in a closed form, this generalization is able to capture the learning curves for a broad range of realistic data sets, thus redeeming the potential of the teacher-student framework.

1 code implementation • 6 Jan 2021 • Alessandro Cappelli, Ruben Ohana, Julien Launay, Laurent Meunier, Iacopo Poli, Florent Krzakala

In the white-box setting, our defense works by obfuscating the parameters of the random projection.

no code implementations • 11 Dec 2020 • Julien Launay, Iacopo Poli, Kilian Müller, Gustave Pariente, Igor Carron, Laurent Daudet, Florent Krzakala, Sylvain Gigan

We present a photonic accelerator for Direct Feedback Alignment, able to compute random projections with trillions of parameters.

1 code implementation • 8 Dec 2020 • Antoine Maillard, Florent Krzakala, Yue M. Lu, Lenka Zdeborová

We consider the phase retrieval problem, in which the observer wishes to recover a $n$-dimensional real or complex signal $\mathbf{X}^\star$ from the (possibly noisy) observation of $|\mathbf{\Phi} \mathbf{X}^\star|$, in which $\mathbf{\Phi}$ is a matrix of size $m \times n$.

Information Theory Disordered Systems and Neural Networks Information Theory

no code implementations • 20 Sep 2020 • Antoine Baker, Indaco Biazzo, Alfredo Braunstein, Giovanni Catania, Luca Dall'Asta, Alessandro Ingrosso, Florent Krzakala, Fabio Mazza, Marc Mézard, Anna Paola Muntoni, Maria Refinetti, Stefano Sarao Mannelli, Lenka Zdeborová

We conclude that probabilistic risk estimation is capable to enhance performance of digital contact tracing and should be considered in the currently developed mobile applications.

1 code implementation • 25 Jun 2020 • Sebastian Goldt, Bruno Loureiro, Galen Reeves, Florent Krzakala, Marc Mézard, Lenka Zdeborová

Here, we go beyond this simple paradigm by studying the performance of neural networks trained on data drawn from pre-trained generative models.

1 code implementation • NeurIPS 2020 • Julien Launay, Iacopo Poli, François Boniface, Florent Krzakala

Despite being the workhorse of deep learning, the backpropagation algorithm is no panacea.

1 code implementation • NeurIPS 2020 • Jonathan Dong, Ruben Ohana, Mushegh Rafayelyan, Florent Krzakala

Reservoir Computing is a class of simple yet efficient Recurrent Neural Networks where internal weights are fixed at random and only a linear output layer is trained.

no code implementations • NeurIPS 2020 • Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová

Despite the widespread use of gradient-based algorithms for optimizing high-dimensional non-convex functions, understanding their ability of finding good minima instead of being trapped in spurious ones remains to a large extent an open problem.

no code implementations • 11 Jun 2020 • Cedric Gerbelot, Alia Abbara, Florent Krzakala

For sufficiently strongly convex problems, we show that the two-layer vector approximate message passing algorithm (2-MLVAMP) converges, where the convergence analysis is done by checking the stability of an equivalent dynamical system, which gives the result for such problems.

no code implementations • NeurIPS 2020 • Benjamin Aubin, Florent Krzakala, Yue M. Lu, Lenka Zdeborová

We consider a commonly studied supervised classification of a synthetic dataset whose labels are generated by feeding a one-layer neural network with random iid inputs.

no code implementations • NeurIPS 2020 • Francesca Mignacco, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová

We define a particular stochastic process for which SGD can be extended to a continuous-time limit that we call stochastic gradient flow.

1 code implementation • NeurIPS 2020 • Antoine Maillard, Bruno Loureiro, Florent Krzakala, Lenka Zdeborová

We consider the phase retrieval problem of reconstructing a $n$-dimensional real or complex signal $\mathbf{X}^{\star}$ from $m$ (possibly noisy) observations $Y_\mu = | \sum_{i=1}^n \Phi_{\mu i} X^{\star}_i/\sqrt{n}|$, for a large class of correlated real and complex random sensing matrices $\mathbf{\Phi}$, in a high-dimensional setting where $m, n\to\infty$ while $\alpha = m/n=\Theta(1)$.

no code implementations • 2 Jun 2020 • Julien Launay, Iacopo Poli, Kilian Müller, Igor Carron, Laurent Daudet, Florent Krzakala, Sylvain Gigan

As neural networks grow larger and more complex and data-hungry, training costs are skyrocketing.

1 code implementation • 3 Apr 2020 • Antoine Baker, Benjamin Aubin, Florent Krzakala, Lenka Zdeborová

We introduce Tree-AMP, standing for Tree Approximate Message Passing, a python package for compositional inference in high-dimensional tree-structured models.

2 code implementations • 2 Mar 2020 • Stéphane d'Ascoli, Maria Refinetti, Giulio Biroli, Florent Krzakala

We obtain a precise asymptotic expression for the bias-variance decomposition of the test error, and show that the bias displays a phase transition at the interpolation threshold, beyond which it remains constant.

no code implementations • ICML 2020 • Francesca Mignacco, Florent Krzakala, Yue M. Lu, Lenka Zdeborová

We also illustrate the interpolation peak at low regularization, and analyze the role of the respective sizes of the two clusters.

no code implementations • ICML 2020 • Federica Gerace, Bruno Loureiro, Florent Krzakala, Marc Mézard, Lenka Zdeborová

In particular, we show how to obtain analytically the so-called double descent behaviour for logistic regression with a peak at the interpolation threshold, we illustrate the superiority of orthogonal against random Gaussian projections in learning with random features, and discuss the role played by correlations in the data generated by the hidden manifold model.

no code implementations • 11 Feb 2020 • Cédric Gerbelot, Alia Abbara, Florent Krzakala

We consider the problem of learning a coefficient vector $x_{0}$ in $R^{N}$ from noisy linear observations $y=Fx_{0}+w$ in $R^{M}$ in the high dimensional limit $M, N$ to infinity with $\alpha=M/N$ fixed.

no code implementations • 5 Dec 2019 • Alia Abbara, Benjamin Aubin, Florent Krzakala, Lenka Zdeborová

Statistical learning theory provides bounds of the generalization gap, using in particular the Vapnik-Chervonenkis dimension and the Rademacher complexity.

no code implementations • 4 Dec 2019 • Benjamin Aubin, Bruno Loureiro, Antoine Baker, Florent Krzakala, Lenka Zdeborová

We consider the problem of compressed sensing and of (real-valued) phase retrieval with random measurement matrix.

1 code implementation • NeurIPS 2019 • Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota, Florent Krzakala, Lenka Zdeborová

Gradient-based algorithms are effective for many machine learning tasks, but despite ample recent effort and some progress, it often remains unclear why they work in practice in optimising high-dimensional non-convex functions and why they find good minima instead of being trapped in spurious ones. Here we present a quantitative theory explaining this behaviour in a spiked matrix-tensor model. Our framework is based on the Kac-Rice analysis of stationary points and a closed-form analysis of gradient-flow originating from statistical physics.

1 code implementation • 22 Oct 2019 • Ruben Ohana, Jonas Wacker, Jonathan Dong, Sébastien Marmin, Florent Krzakala, Maurizio Filippone, Laurent Daudet

Approximating kernel functions with random features (RFs)has been a successful application of random projections for nonparametric estimation.

1 code implementation • 25 Sep 2019 • Sebastian Goldt, Marc Mézard, Florent Krzakala, Lenka Zdeborová

We demonstrate that learning of the hidden manifold model is amenable to an analytical treatment by proving a "Gaussian Equivalence Property" (GEP), and we use the GEP to show how the dynamics of two-layer neural networks trained using one-pass stochastic gradient descent is captured by a set of integro-differential equations that track the performance of the network at all times.

no code implementations • NeurIPS Workshop Deep_Invers 2019 • Benjamin Aubin, Bruno Loureiro, Antoine Baker, Florent Krzakala, Lenka Zdeborova

We consider the problem of compressed sensing and of (real-valued) phase retrieval with random measurement matrix.

no code implementations • 18 Jul 2019 • Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota, Florent Krzakala, Lenka Zdeborová

Gradient-based algorithms are effective for many machine learning tasks, but despite ample recent effort and some progress, it often remains unclear why they work in practice in optimising high-dimensional non-convex functions and why they find good minima instead of being trapped in spurious ones.

3 code implementations • NeurIPS 2019 • Sebastian Goldt, Madhu S. Advani, Andrew M. Saxe, Florent Krzakala, Lenka Zdeborová

Deep neural networks achieve stellar generalisation even when they have enough parameters to easily fit all their training data.

1 code implementation • 11 Jun 2019 • Alia Abbara, Antoine Baker, Florent Krzakala, Lenka Zdeborová

In a noiseless linear estimation problem, one aims to reconstruct a vector x* from the knowledge of its linear projections y=Phi x*.

2 code implementations • 11 Jun 2019 • Julien Launay, Iacopo Poli, Florent Krzakala

In this work, we focus on direct feedback alignment and present a set of best practices justified by observations of the alignment angles.

2 code implementations • NeurIPS 2019 • Benjamin Aubin, Bruno Loureiro, Antoine Maillard, Florent Krzakala, Lenka Zdeborová

Here, we replace the sparsity assumption by generative modelling, and investigate the consequences on statistical and algorithmic properties.

no code implementations • 1 Feb 2019 • Stefano Sarao Mannelli, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová

In this work we analyse quantitatively the interplay between the loss landscape and performance of descent algorithms in a prototypical inference problem, the spiked matrix-tensor model.

no code implementations • 25 Jan 2019 • Sebastian Goldt, Madhu S. Advani, Andrew M. Saxe, Florent Krzakala, Lenka Zdeborová

Deep neural networks achieve stellar generalisation on a variety of problems, despite often being large enough to easily fit all their training data.

no code implementations • 21 Dec 2018 • Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová

Gradient-descent-based algorithms and their stochastic versions have widespread applications in machine learning and statistical inference.

no code implementations • 6 Dec 2018 • Jean Barbier, Mohamad Dia, Nicolas Macris, Florent Krzakala, Lenka Zdeborová

We characterize the detectability phase transitions in a large set of estimation problems, where we show that there exists a gap between what currently known polynomial algorithms (in particular spectral methods and approximate message-passing) can do and what is expected information theoretically.

1 code implementation • 30 Oct 2018 • Jonathan Dong, Florent Krzakala, Sylvain Gigan

We introduce a generalized version of phase retrieval called multiplexed phase retrieval.

no code implementations • 17 Sep 2018 • Andre Manoel, Florent Krzakala, Gaël Varoquaux, Bertrand Thirion, Lenka Zdeborová

We introduce an iterative optimization scheme for convex objectives consisting of a linear loss and a non-separable penalty, based on the expectation-consistent approximation and the vector approximate message-passing (VAMP) algorithm.

no code implementations • 3 Jul 2018 • Fabrizio Antenucci, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová

Approximate message passing algorithm enjoyed considerable attention in the last decade.

no code implementations • 25 Jun 2018 • Ahmed El Alaoui, Florent Krzakala, Michael. I. Jordan

We study the fundamental limits of detecting the presence of an additive rank-one perturbation, or spike, to a Wigner matrix.

1 code implementation • NeurIPS 2018 • Benjamin Aubin, Antoine Maillard, Jean Barbier, Florent Krzakala, Nicolas Macris, Lenka Zdeborová

Heuristic tools from statistical physics have been used in the past to locate the phase transitions and compute the optimal learning and generalization errors in the teacher-student scenario in multi-layer neural networks.

2 code implementations • NeurIPS 2018 • Marylou Gabrié, Andre Manoel, Clément Luneau, Jean Barbier, Nicolas Macris, Florent Krzakala, Lenka Zdeborová

We examine a class of deep learning models with a tractable method to compute information-theoretic quantities.

1 code implementation • 10 Aug 2017 • Jean Barbier, Florent Krzakala, Nicolas Macris, Léo Miolane, Lenka Zdeborová

Non-rigorous predictions for the optimal errors existed for special cases of GLMs, e. g. for the perceptron, in the field of statistical physics based on the so-called replica method.

no code implementations • 2 Jun 2017 • Andre Manoel, Florent Krzakala, Eric W. Tramel, Lenka Zdeborová

In statistical learning for real-world large-scale data problems, one must often resort to "streaming" algorithms which operate sequentially on small batches of data.

no code implementations • 10 Feb 2017 • Eric W. Tramel, Marylou Gabrié, Andre Manoel, Francesco Caltagirone, Florent Krzakala

Restricted Boltzmann machines (RBMs) are energy-based neural-networks which are commonly used as the building blocks for deep architectures neural architectures.

no code implementations • 24 Jan 2017 • Andre Manoel, Florent Krzakala, Marc Mézard, Lenka Zdeborová

We consider the problem of reconstructing a signal from multi-layered (possibly) non-linear measurements.

no code implementations • 10 Oct 2016 • Thibault Lesieur, Caterina De Bacco, Jess Banks, Florent Krzakala, Cris Moore, Lenka Zdeborová

We consider the problem of Gaussian mixture clustering in the high-dimensional limit where the data consists of $m$ points in $n$ dimensions, $n, m \rightarrow \infty$ and $\alpha = m/n$ stays finite.

no code implementations • 15 Sep 2016 • Jonathan Dong, Sylvain Gigan, Florent Krzakala, Gilles Wainrib

As a proof of concept, binary networks have been successfully trained to predict the chaotic Mackey-Glass time series.

no code implementations • NeurIPS 2016 • Jean Barbier, Mohamad Dia, Nicolas Macris, Florent Krzakala, Thibault Lesieur, Lenka Zdeborova

We also show that for a large set of parameters, an iterative algorithm called approximate message-passing is Bayes optimal.

no code implementations • 13 Jun 2016 • Eric W. Tramel, Andre Manoel, Francesco Caltagirone, Marylou Gabrié, Florent Krzakala

In this work, we consider compressed sensing reconstruction from $M$ measurements of $K$-sparse structured signals which do not possess a writable correlation model.

no code implementations • 20 May 2016 • Alaa Saade, Florent Krzakala, Marc Lelarge, Lenka Zdeborová

We consider the problem of clustering partially labeled data from a minimal number of randomly chosen pairwise comparisons between the items.

no code implementations • 25 Jan 2016 • Alaa Saade, Marc Lelarge, Florent Krzakala, Lenka Zdeborová

We consider the problem of grouping items into clusters based on few random pairwise comparisons between the items.

no code implementations • NeurIPS 2015 • Marylou Gabrie, Eric W. Tramel, Florent Krzakala

Restricted Boltzmann machines are undirected neural networks which have been shown tobe effective in many applications, including serving as initializations fortraining deep multi-layer neural networks.

1 code implementation • 8 Nov 2015 • Lenka Zdeborová, Florent Krzakala

Many questions of fundamental interest in todays science can be formulated as inference problems: Some partial, or noisy, observations are performed over a set of variables and the goal is to recover, or infer, the values of the variables based on the indirect information contained in the measurements.

no code implementations • 22 Oct 2015 • Alaa Saade, Francesco Caltagirone, Igor Carron, Laurent Daudet, Angélique Drémeau, Sylvain Gigan, Florent Krzakala

Random projections have proven extremely useful in many signal processing and machine learning applications.

no code implementations • 5 Oct 2015 • Boshra Rajaei, Eric W. Tramel, Sylvain Gigan, Florent Krzakala, Laurent Daudet

In this paper, the problem of compressive imaging is addressed using natural randomization by means of a multiply scattering medium.

1 code implementation • 14 Jul 2015 • Thibault Lesieur, Florent Krzakala, Lenka Zdeborová

This paper considers probabilistic estimation of a low-rank matrix from non-linear element-wise measurements of its elements.

no code implementations • NeurIPS 2015 • Alaa Saade, Florent Krzakala, Lenka Zdeborová

We propose a spectral algorithm for these two tasks called MaCBetH (for Matrix Completion with the Bethe Hessian).

no code implementations • 9 Jun 2015 • Marylou Gabrié, Eric W. Tramel, Florent Krzakala

Restricted Boltzmann machines are undirected neural networks which have been shown to be effective in many applications, including serving as initializations for training deep multi-layer neural networks.

1 code implementation • 1 Mar 2015 • Thibault Lesieur, Florent Krzakala, Lenka Zdeborova

We study optimal estimation for sparse principal component analysis when the number of non-zero elements is small but on the same order as the dimension of the data.

no code implementations • 23 Feb 2015 • Eric W. Tramel, Angélique Drémeau, Florent Krzakala

Approximate Message Passing (AMP) has been shown to be an excellent statistical approach to signal inference and compressed sensing problem.

no code implementations • 31 Jan 2015 • Alaa Saade, Florent Krzakala, Marc Lelarge, Lenka Zdeborová

We describe two spectral algorithms for this task based on the non-backtracking and the Bethe Hessian operators.

1 code implementation • 17 Jun 2014 • Andre Manoel, Florent Krzakala, Eric W. Tramel, Lenka Zdeborová

Approximate Message Passing (AMP) has been shown to be a superior method for inference problems, such as the recovery of signals from sets of noisy, lower-dimensionality measurements, both in terms of reconstruction accuracy and in computational efficiency.

3 code implementations • NeurIPS 2014 • Alaa Saade, Florent Krzakala, Lenka Zdeborová

We show that this approach combines the performances of the non-backtracking operator, thus detecting clusters all the way down to the theoretical limit in the stochastic block model, with the computational, theoretical and memory advantages of real symmetric matrices.

no code implementations • 6 Feb 2014 • Yoshiyuki Kabashima, Florent Krzakala, Marc Mézard, Ayaka Sakata, Lenka Zdeborová

We use the tools of statistical mechanics - the cavity and replica methods - to analyze the achievability and computational tractability of the inference problems in the setting of Bayes-optimal inference, which amounts to assuming that the two matrices have random independent elements generated from some known distribution, and this information is available to the inference algorithm.

no code implementations • NeurIPS 2013 • Christophe Schulke, Francesco Caltagirone, Florent Krzakala, Lenka Zdeborová

We study numerically the phase diagram of the blind calibration problem, and show that even in cases where convex relaxation is possible, our algorithm requires a smaller number of measurements and/or signals in order to perform well.

no code implementations • 24 Jun 2013 • Florent Krzakala, Cristopher Moore, Elchanan Mossel, Joe Neeman, Allan Sly, Lenka Zdeborová, Pan Zhang

Spectral algorithms are classic approaches to clustering and community detection in networks.

no code implementations • 17 Jul 2012 • Xiaoran Yan, Cosma Rohilla Shalizi, Jacob E. Jensen, Florent Krzakala, Cristopher Moore, Lenka Zdeborova, Pan Zhang, Yaojia Zhu

We present the first principled and tractable approach to model selection between standard and degree-corrected block models, based on new large-graph asymptotics for the distribution of log-likelihood ratios under the stochastic block model, finding substantial departures from classical results for sparse graphs.

1 code implementation • 18 Jun 2012 • Florent Krzakala, Marc Mézard, François Sausset, Yifan Sun, Lenka Zdeborová

We further develop the asymptotic analysis of the corresponding phase diagrams with and without measurement noise, for different distribution of signals, and discuss the best possible reconstruction performances regardless of the algorithm.

Statistical Mechanics Information Theory Information Theory

1 code implementation • 20 Sep 2011 • Florent Krzakala, Marc Mézard, François Sausset, Yifan Sun, Lenka Zdeborová

Compressed sensing is triggering a major evolution in signal acquisition.

Statistical Mechanics Information Theory Information Theory

no code implementations • 14 Sep 2011 • Aurelien Decelle, Florent Krzakala, Cristopher Moore, Lenka Zdeborová

In this paper we extend our previous work on the stochastic block model, a commonly used generative model for social and biological networks, and the problem of inferring functional groups or communities from the topology of the network.

Statistical Mechanics Disordered Systems and Neural Networks Social and Information Networks Physics and Society

no code implementations • 6 Feb 2011 • Aurelien Decelle, Florent Krzakala, Cristopher Moore, Lenka Zdeborová

We present an asymptotically exact analysis of the problem of detecting communities in sparse random networks.

Cannot find the paper you are looking for? You can
Submit a new open access paper.

Contact us on:
hello@paperswithcode.com
.
Papers With Code is a free resource with all data licensed under CC-BY-SA.