no code implementations • 3 Jun 2024 • Yunhao Ni, Yuxin Guo, Junlong Jia, Lei Huang
Layer normalization (LN) is a ubiquitous technique in deep learning but our theoretical understanding to it remains elusive.
1 code implementation • 26 May 2023 • Xi Weng, Yunhao Ni, Tengwei Song, Jie Luo, Rao Muhammad Anwer, Salman Khan, Fahad Shahbaz Khan, Lei Huang
In this work, we introduce Spectral Transformation (ST), a framework to modulate the spectrum of embedding and to seek for functions beyond whitening that can avoid dimensional collapse.