Transformers Get Stable: An End-to-End Signal Propagation Theory for Language Models

1 code implementation • 14 Mar 2024 • Akhil Kedia, Mohd Abbas Zaidi, Sushil Khyalia, Jungho Jung, Harshith Goka, Haejun Lee

In spite of their huge success, transformer models remain difficult to scale in depth.

Decoder Image Classification +3