1 code implementation • 15 Apr 2024 • Tamay Besiroglu, Ege Erdil, Matthew Barnett, Josh You
Hoffmann et al. (2022) propose three methods for estimating a compute-optimal scaling law.
1 code implementation • 9 Mar 2024 • Anson Ho, Tamay Besiroglu, Ege Erdil, David Owen, Robi Rahman, Zifan Carl Guo, David Atkinson, Neil Thompson, Jaime Sevilla
We investigate the rate at which algorithms for pre-training language models have improved since the advent of deep learning.
no code implementations • 20 Sep 2023 • Ege Erdil, Tamay Besiroglu
Key questions remain about the intensity of regulatory responses to AI, physical bottlenecks in production, the economic value of superhuman abilities, and the rate at which AI automation could occur.
no code implementations • 19 Apr 2023 • Ege Erdil, Jaime Sevilla
Using a random effects model, we improve on this baseline for relative mean square error made on predicting out-of-sample world record improvements as the comparison metric at a $p < 10^{-5}$ significance level.
no code implementations • 10 Dec 2022 • Ege Erdil, Tamay Besiroglu
Using Shapley values to attribute performance improvements, we find that algorithmic improvements have been roughly as important as the scaling of compute for progress computer vision.