Spatially heterogeneous learning by a deep student machine

no code implementations • 15 Feb 2023 • Hajime Yoshino

Remarkably both the theory and simulation suggest generalization-ability of the student machines, which are only weakly correlated with the teacher in the center, does not vanish even in the deep limit $L \gg 1$ where the system becomes heavily over-parametrized.

From complex to simple : hierarchical free-energy landscape renormalized in deep neural networks

no code implementations • 22 Oct 2019 • Hajime Yoshino

We develop a statistical mechanical approach based on the replica method to study the design space of deep and wide neural networks constrained to meet a large number of training data.