no code implementations • 29 Sep 2022 • Alireza Mousavi-Hosseini, Sejun Park, Manuela Girotti, Ioannis Mitliagkas, Murat A. Erdogdu
We further demonstrate that, SGD-trained ReLU NNs can learn a single-index target of the form $y=f(\langle\boldsymbol{u},\boldsymbol{x}\rangle) + \epsilon$ by recovering the principal direction, with a sample complexity linear in $d$ (up to log factors), where $f$ is a monotonic function with at most polynomial growth, and $\epsilon$ is the noise.
no code implementations • 20 Oct 2021 • Manuela Girotti, Ioannis Mitliagkas, Gauthier Gidel
We theoretically analyze the Feedback Alignment (FA) algorithm, an efficient alternative to backpropagation for training neural networks.
no code implementations • 10 Dec 2020 • Charles Guille-Escuret, Baptiste Goujaud, Manuela Girotti, Ioannis Mitliagkas
Since smoothness and strong convexity are not continuous, we propose a comprehensive study of existing alternative metrics which we prove to be continuous.