1 code implementation • 26 Feb 2024 • Hiroki Furuta, Gouki Minegishi, Yusuke Iwasawa, Yutaka Matsuo
Grokking has been actively explored to reveal the mystery of delayed generalization.
1 code implementation • 30 Oct 2023 • Gouki Minegishi, Yusuke Iwasawa, Yutaka Matsuo
We aim to analyze the mechanism of grokking from the lottery ticket hypothesis, identifying the process to find the lottery tickets (good sparse subnetworks) as the key to describing the transitional phase between memorization and generalization.