You need to log in to edit.

You can create a new account if you don't have one.

Or, discuss a change on Slack.

You can create a new account if you don't have one.

Or, discuss a change on Slack.

no code implementations • 20 Dec 2021 • Francesca Mignacco, Pierfrancesco Urbani

In the under-parametrized regime, where the final training error is positive, the SGD dynamics reaches a stationary state and we define an effective temperature from the fluctuation-dissipation theorem, computed from dynamical mean-field theory.

no code implementations • 8 Mar 2021 • Francesca Mignacco, Pierfrancesco Urbani, Lenka Zdeborová

In this paper we investigate how gradient-based algorithms such as gradient descent, (multi-pass) stochastic gradient descent, its persistent variant, and the Langevin algorithm navigate non-convex loss-landscapes and which of them is able to reach the best generalization error at limited sample complexity.

no code implementations • NeurIPS 2021 • Stefano Sarao Mannelli, Pierfrancesco Urbani

The optimization step in many machine learning problems rarely relies on vanilla gradient descent but it is common practice to use momentum-based accelerated methods.

no code implementations • 4 Jan 2021 • Pierfrancesco Urbani

In the noisy case one has a set of controllable stochastic processes and a cost function that is a functional of their trajectories.

Optimization and Control Disordered Systems and Neural Networks

no code implementations • 21 Dec 2020 • Eran Bouchbinder, Edan Lerner, Corrado Rainone, Pierfrancesco Urbani, Francesco Zamponi

We study a recently introduced and exactly solvable mean-field model for the density of vibrational states $\mathcal{D}(\omega)$ of a structurally disordered system.

Disordered Systems and Neural Networks Soft Condensed Matter Statistical Mechanics

no code implementations • 5 Oct 2020 • Silvio Franz, Antonio Sclocchi, Pierfrancesco Urbani

This algorithm allows to "surf" between isostatic marginally stable configurations and to investigate some properties of such landscape.

Disordered Systems and Neural Networks Statistical Mechanics

no code implementations • NeurIPS 2020 • Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová

Despite the widespread use of gradient-based algorithms for optimizing high-dimensional non-convex functions, understanding their ability of finding good minima instead of being trapped in spurious ones remains to a large extent an open problem.

no code implementations • NeurIPS 2020 • Francesca Mignacco, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová

We define a particular stochastic process for which SGD can be extended to a continuous-time limit that we call stochastic gradient flow.

no code implementations • 1 Feb 2019 • Stefano Sarao Mannelli, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová

In this work we analyse quantitatively the interplay between the loss landscape and performance of descent algorithms in a prototypical inference problem, the spiked matrix-tensor model.

no code implementations • 21 Dec 2018 • Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová

Gradient-descent-based algorithms and their stochastic versions have widespread applications in machine learning and statistical inference.

no code implementations • 3 Jul 2018 • Fabrizio Antenucci, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová

Approximate message passing algorithm enjoyed considerable attention in the last decade.

no code implementations • 15 May 2018 • Fabrizio Antenucci, Silvio Franz, Pierfrancesco Urbani, Lenka Zdeborová

An algorithmically hard phase was described in a range of inference problems: even if the signal can be reconstructed with a small error from an information theoretic point of view, known algorithms fail unless the noise-to-signal ratio is sufficiently small.

Cannot find the paper you are looking for? You can
Submit a new open access paper.

Contact us on:
hello@paperswithcode.com
.
Papers With Code is a free resource with all data licensed under CC-BY-SA.