# Algorithmic Regularization in Over-parameterized Matrix Sensing and Neural Networks with Quadratic Activations

26 Dec 2017Yuanzhi LiTengyu MaHongyang Zhang

We show that the gradient descent algorithm provides an implicit regularization effect in the learning of over-parameterized matrix factorization models and one-hidden-layer neural networks with quadratic activations. Concretely, we show that given $\tilde{O}(dr^{2})$ random linear measurements of a rank $r$ positive semidefinite matrix $X^{\star}$, we can recover $X^{\star}$ by parameterizing it by $UU^\top$ with $U\in \mathbb R^{d\times d}$ and minimizing the squared loss, even if $r \ll d$... (read more)

PDF Abstract

# Code Add Remove

No code implementations yet. Submit your code now