no code implementations • 30 Sep 2022 • Sungyub Kim, Sihwan Park, KyungSu Kim, Eunho Yang
Explaining generalizations and preventing over-confident predictions are central goals of studies on the loss landscape of neural networks.
no code implementations • 29 Sep 2021 • Sungyub Kim, Sihwan Park, Yong-Deok Kim, Eunho Yang
To mitigate this issue, we propose simple bias decay methods including a novel adaptive one and found that this simple remedy can fill a large portion of the performance gaps that occur in large batch optimization.