1 code implementation • 7 Feb 2024 • Hugo Cui, Luca Pesce, Yatin Dandi, Florent Krzakala, Yue M. Lu, Lenka Zdeborová, Bruno Loureiro
To our knowledge, our results provides the first tight description of the impact of feature learning in the generalization of two-layer neural networks in the large learning rate regime $\eta=\Theta_{d}(d)$, beyond perturbative finite width corrections of the conjugate and neural tangent kernels.
no code implementations • 5 Feb 2024 • Yatin Dandi, Emanuele Troiani, Luca Arnaboldi, Luca Pesce, Lenka Zdeborová, Florent Krzakala
In particular, multi-pass GD with finite stepsize is found to overcome the limitations of gradient flow and single-pass GD given by the information exponent (Ben Arous et al., 2021) and leap exponent (Abbe et al., 2023) of the target function.
no code implementations • 9 Sep 2023 • Neha S. Wadia, Yatin Dandi, Michael I. Jordan
The rapid progress in machine learning in recent years has been based on a highly productive connection to gradient-based optimization.
1 code implementation • 27 Aug 2023 • Davide Ghio, Yatin Dandi, Florent Krzakala, Lenka Zdeborová
Recent years witnessed the development of powerful generative models based on flows, diffusion or autoregressive neural networks, achieving remarkable success in generating data from examples with applications in a broad range of areas.
1 code implementation • 29 May 2023 • Yatin Dandi, Florent Krzakala, Bruno Loureiro, Luca Pesce, Ludovic Stephan
The picture drastically improves over multiple gradient steps: we show that a batch-size of $n = \mathcal{O}(d)$ is indeed enough to learn multiple target directions satisfying a staircase property, where more and more directions can be learned over time.
no code implementations • 13 Apr 2022 • Yatin Dandi, Anastasia Koloskova, Martin Jaggi, Sebastian U. Stich
Decentralized learning provides an effective framework to train machine learning models with data distributed over arbitrary communication graphs.
no code implementations • 7 Nov 2021 • Avinandan Bose, Aniket Das, Yatin Dandi, Piyush Rai
In this work, we propose a novel generative model that learns a flexible non-parametric prior over interpolation trajectories, conditioned on a pair of source and target images.
no code implementations • 6 Nov 2021 • Yatin Dandi, Arthur Jacot
Spectral analysis is a powerful tool, decomposing any function into simpler parts.
no code implementations • NeurIPS Workshop DLDE 2021 • Avinandan Bose, Aniket Das, Yatin Dandi, Piyush Rai
A range of applications require learning image generation models whose latent space effectively captures the high-level factors of variation in the data distribution, which can be judged by its ability to interpolate between images smoothly.
no code implementations • 25 Jun 2021 • Yatin Dandi, Luis Barba, Martin Jaggi
A major obstacle to achieving global convergence in distributed and federated learning is the misalignment of gradients across clients, or mini-batches due to heterogeneity and stochasticity of the distributed data.
no code implementations • 4 Dec 2020 • Arnout Devos, Yatin Dandi
In this paper, we propose a learning algorithm that enables a model to quickly exploit commonalities among related tasks from an unseen task distribution, before quickly adapting to specific tasks from that same distribution.
no code implementations • 15 Jun 2020 • Yatin Dandi, Homanga Bharadhwaj, Abhishek Kumar, Piyush Rai
Recent approaches, such as ALI and BiGAN frameworks, develop methods of inference of latent variables in GANs by adversarially training an image generator along with an encoder to match two joint distributions of image and latent vector pairs.
no code implementations • 17 Dec 2019 • Yatin Dandi, Aniket Das, Soumye Singhal, Vinay P. Namboodiri, Piyush Rai
The proposed model allows minor variations in content across frames while maintaining the temporal dependence through latent vectors encoding the pose or motion features.