Search Results for author: Pierfrancesco Urbani

Found 12 papers, 0 papers with code

The effective noise of Stochastic Gradient Descent

no code implementations20 Dec 2021 Francesca Mignacco, Pierfrancesco Urbani

In the under-parametrized regime, where the final training error is positive, the SGD dynamics reaches a stationary state and we define an effective temperature from the fluctuation-dissipation theorem, computed from dynamical mean-field theory.

Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem

no code implementations8 Mar 2021 Francesca Mignacco, Pierfrancesco Urbani, Lenka Zdeborová

In this paper we investigate how gradient-based algorithms such as gradient descent, (multi-pass) stochastic gradient descent, its persistent variant, and the Langevin algorithm navigate non-convex loss-landscapes and which of them is able to reach the best generalization error at limited sample complexity.


Analytical Study of Momentum-Based Acceleration Methods in Paradigmatic High-Dimensional Non-Convex Problems

no code implementations NeurIPS 2021 Stefano Sarao Mannelli, Pierfrancesco Urbani

The optimization step in many machine learning problems rarely relies on vanilla gradient descent but it is common practice to use momentum-based accelerated methods.

Numerical Integration

Disordered high-dimensional optimal control

no code implementations4 Jan 2021 Pierfrancesco Urbani

In the noisy case one has a set of controllable stochastic processes and a cost function that is a functional of their trajectories.

Optimization and Control Disordered Systems and Neural Networks

Low-frequency vibrational spectrum of mean-field disordered systems

no code implementations21 Dec 2020 Eran Bouchbinder, Edan Lerner, Corrado Rainone, Pierfrancesco Urbani, Francesco Zamponi

We study a recently introduced and exactly solvable mean-field model for the density of vibrational states $\mathcal{D}(\omega)$ of a structurally disordered system.

Disordered Systems and Neural Networks Soft Condensed Matter Statistical Mechanics

Surfing on minima of isostatic landscapes: avalanches and unjamming transition

no code implementations5 Oct 2020 Silvio Franz, Antonio Sclocchi, Pierfrancesco Urbani

This algorithm allows to "surf" between isostatic marginally stable configurations and to investigate some properties of such landscape.

Disordered Systems and Neural Networks Statistical Mechanics

Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval

no code implementations NeurIPS 2020 Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová

Despite the widespread use of gradient-based algorithms for optimizing high-dimensional non-convex functions, understanding their ability of finding good minima instead of being trapped in spurious ones remains to a large extent an open problem.

Passed & Spurious: Descent Algorithms and Local Minima in Spiked Matrix-Tensor Models

no code implementations1 Feb 2019 Stefano Sarao Mannelli, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová

In this work we analyse quantitatively the interplay between the loss landscape and performance of descent algorithms in a prototypical inference problem, the spiked matrix-tensor model.

Marvels and Pitfalls of the Langevin Algorithm in Noisy High-dimensional Inference

no code implementations21 Dec 2018 Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová

Gradient-descent-based algorithms and their stochastic versions have widespread applications in machine learning and statistical inference.

Approximate Survey Propagation for Statistical Inference

no code implementations3 Jul 2018 Fabrizio Antenucci, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová

Approximate message passing algorithm enjoyed considerable attention in the last decade.

Glassy nature of the hard phase in inference problems

no code implementations15 May 2018 Fabrizio Antenucci, Silvio Franz, Pierfrancesco Urbani, Lenka Zdeborová

An algorithmically hard phase was described in a range of inference problems: even if the signal can be reconstructed with a small error from an information theoretic point of view, known algorithms fail unless the noise-to-signal ratio is sufficiently small.

Cannot find the paper you are looking for? You can Submit a new open access paper.