Search Results for author:
Found 7 papers, 0 papers with code
Annihilation of Spurious Minima in Two-Layer ReLU Networks
We study the optimization problem associated with fitting two-layer ReLU neural networks with respect to the squared loss, where labels are generated by a target network.
Analytic Study of Families of Spurious Minima in Two-Layer ReLU Neural Networks: A Tale of Symmetry II
In particular, we derive analytic estimates for the loss at different minima, and prove that modulo $O(d^{-1/2})$-terms the Hessian spectrum concentrates near small positive constants, with the exception of $\Theta(d)$ eigenvalues which grow linearly with~$d$.
Equivariant bifurcation, quadratic equivariants, and symmetry breaking for the standard representation of $S_n$
Motivated by questions originating from the study of a class of shallow student-teacher neural networks, methods are developed for the analysis of spurious minima in classes of gradient equivariant dynamics related to neural nets.
Symmetry Breaking in Symmetric Tensor Decomposition
In this note, we consider the highly nonconvex optimization problem associated with computing the rank decomposition of symmetric tensors.
Analytic Characterization of the Hessian in Shallow ReLU Models: A Tale of Symmetry
We consider the optimization problem associated with fitting two-layers ReLU networks with respect to the squared loss, where labels are generated by a target network.
Symmetry & critical points for a model shallow neural network
We consider the optimization problem associated with fitting two-layer ReLU networks with $k$ hidden neurons, where labels are assumed to be generated by a (teacher) neural network.
On the Principle of Least Symmetry Breaking in Shallow ReLU Models
We consider the optimization problem associated with fitting two-layer ReLU networks with respect to the squared loss, where labels are assumed to be generated by a target network.
