Search Results for author: Yatin Dandi

Found 13 papers, 3 papers with code

Asymptotics of feature learning in two-layer networks after one gradient-step

1 code implementation7 Feb 2024 Hugo Cui, Luca Pesce, Yatin Dandi, Florent Krzakala, Yue M. Lu, Lenka Zdeborová, Bruno Loureiro

To our knowledge, our results provides the first tight description of the impact of feature learning in the generalization of two-layer neural networks in the large learning rate regime $\eta=\Theta_{d}(d)$, beyond perturbative finite width corrections of the conjugate and neural tangent kernels.

The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents

no code implementations5 Feb 2024 Yatin Dandi, Emanuele Troiani, Luca Arnaboldi, Luca Pesce, Lenka Zdeborová, Florent Krzakala

In particular, multi-pass GD with finite stepsize is found to overcome the limitations of gradient flow and single-pass GD given by the information exponent (Ben Arous et al., 2021) and leap exponent (Abbe et al., 2023) of the target function.

A Gentle Introduction to Gradient-Based Optimization and Variational Inequalities for Machine Learning

no code implementations9 Sep 2023 Neha S. Wadia, Yatin Dandi, Michael I. Jordan

The rapid progress in machine learning in recent years has been based on a highly productive connection to gradient-based optimization.

Decision Making

Sampling with flows, diffusion and autoregressive neural networks: A spin-glass perspective

1 code implementation27 Aug 2023 Davide Ghio, Yatin Dandi, Florent Krzakala, Lenka Zdeborová

Recent years witnessed the development of powerful generative models based on flows, diffusion or autoregressive neural networks, achieving remarkable success in generating data from examples with applications in a broad range of areas.

Denoising

How Two-Layer Neural Networks Learn, One (Giant) Step at a Time

1 code implementation29 May 2023 Yatin Dandi, Florent Krzakala, Bruno Loureiro, Luca Pesce, Ludovic Stephan

The picture drastically improves over multiple gradient steps: we show that a batch-size of $n = \mathcal{O}(d)$ is indeed enough to learn multiple target directions satisfying a staircase property, where more and more directions can be learned over time.

Data-heterogeneity-aware Mixing for Decentralized Learning

no code implementations13 Apr 2022 Yatin Dandi, Anastasia Koloskova, Martin Jaggi, Sebastian U. Stich

Decentralized learning provides an effective framework to train machine learning models with data distributed over arbitrary communication graphs.

NeurInt : Learning to Interpolate through Neural ODEs

no code implementations7 Nov 2021 Avinandan Bose, Aniket Das, Yatin Dandi, Piyush Rai

In this work, we propose a novel generative model that learns a flexible non-parametric prior over interpolation trajectories, conditioned on a pair of source and target images.

Image Generation

Understanding Layer-wise Contributions in Deep Neural Networks through Spectral Analysis

no code implementations6 Nov 2021 Yatin Dandi, Arthur Jacot

Spectral analysis is a powerful tool, decomposing any function into simpler parts.

NeurInt-Learning Interpolation by Neural ODEs

no code implementations NeurIPS Workshop DLDE 2021 Avinandan Bose, Aniket Das, Yatin Dandi, Piyush Rai

A range of applications require learning image generation models whose latent space effectively captures the high-level factors of variation in the data distribution, which can be judged by its ability to interpolate between images smoothly.

Image Generation

Implicit Gradient Alignment in Distributed and Federated Learning

no code implementations25 Jun 2021 Yatin Dandi, Luis Barba, Martin Jaggi

A major obstacle to achieving global convergence in distributed and federated learning is the misalignment of gradients across clients, or mini-batches due to heterogeneity and stochasticity of the distributed data.

Federated Learning

Model-Agnostic Learning to Meta-Learn

no code implementations4 Dec 2020 Arnout Devos, Yatin Dandi

In this paper, we propose a learning algorithm that enables a model to quickly exploit commonalities among related tasks from an unseen task distribution, before quickly adapting to specific tasks from that same distribution.

Image Classification regression +2

Generalized Adversarially Learned Inference

no code implementations15 Jun 2020 Yatin Dandi, Homanga Bharadhwaj, Abhishek Kumar, Piyush Rai

Recent approaches, such as ALI and BiGAN frameworks, develop methods of inference of latent variables in GANs by adversarially training an image generator along with an encoder to match two joint distributions of image and latent vector pairs.

Jointly Trained Image and Video Generation using Residual Vectors

no code implementations17 Dec 2019 Yatin Dandi, Aniket Das, Soumye Singhal, Vinay P. Namboodiri, Piyush Rai

The proposed model allows minor variations in content across frames while maintaining the temporal dependence through latent vectors encoding the pose or motion features.

Disentanglement Image Generation +1

Cannot find the paper you are looking for? You can Submit a new open access paper.