no code implementations • ICLR 2022 • Kirby Banman, Liam Peet-Pare, Nidhi Hegde, Alona Fyshe, Martha White
In this work, we show that SGDm under covariate shift with a fixed step-size can be unstable and diverge.
1 code implementation • ICLR 2021 • Yangchen Pan, Kirby Banman, Martha White
Recent work has shown that sparse representations -- where only a small percentage of units are active -- can significantly reduce interference.