no code implementations • 4 Apr 2023 • Emily Dinan, Sho Yaida, Susan Zhang
We perform an effective-theory analysis of forward-backward signal propagation in wide and deep Transformers, i. e., residual neural networks with multi-head self-attention blocks and multilayer perceptron blocks.
no code implementations • 10 Oct 2022 • Sho Yaida
In this note, we first derive a one-parameter family of hyperparameter scaling strategies that interpolates between the neural-tangent scaling and mean-field/maximal-update scaling.
no code implementations • 18 Jun 2021 • Daniel A. Roberts, Sho Yaida, Boris Hanin
This book develops an effective theory approach to understanding deep neural networks of practical relevance.
no code implementations • 30 Sep 2019 • Sho Yaida
Gaussian processes are ubiquitous in nature and engineering.
2 code implementations • ICLR 2020 • Judy Hoffman, Daniel A. Roberts, Sho Yaida
Design of reliable systems must guarantee stability against input perturbations.
2 code implementations • ICLR 2019 • Sho Yaida
The notion of the stationary equilibrium ensemble has played a central role in statistical mechanics.