no code implementations • 8 Nov 2023 • C. Daniel Freeman, Laura Culp, Aaron Parisi, Maxwell L Bileschi, Gamaleldin F Elsayed, Alex Rizkowsky, Isabelle Simpson, Alex Alemi, Azade Nova, Ben Adlam, Bernd Bohnet, Gaurav Mishra, Hanie Sedghi, Igor Mordatch, Izzeddin Gur, Jaehoon Lee, JD Co-Reyes, Jeffrey Pennington, Kelvin Xu, Kevin Swersky, Kshiteej Mahajan, Lechao Xiao, Rosanne Liu, Simon Kornblith, Noah Constant, Peter J. Liu, Roman Novak, Yundi Qian, Noah Fiedel, Jascha Sohl-Dickstein
We introduce and study the problem of adversarial arithmetic, which provides a simple yet challenging testbed for language model alignment.
no code implementations • 16 Oct 2023 • Yixin Liu, Avi Singh, C. Daniel Freeman, John D. Co-Reyes, Peter J. Liu
With these methods, we present a thorough empirical study on a series of PaLM 2 models and find: (1) The quality and style of the step-by-step solutions used for fine-tuning can make a significant impact on the model performance; (2) While solution re-ranking and majority voting are both effective for improving the model performance when used separately, they can also be used together for an even greater performance boost; (3) Multi-task fine-tuning that sequentially separates the solution generation and evaluation tasks can offer improved performance compared with the solution fine-tuning baseline.
no code implementations • CVPR 2023 • Erik Gärtner, Luke Metz, Mykhaylo Andriluka, C. Daniel Freeman, Cristian Sminchisescu
We propose a new approach to learned optimization where we represent the computation of an optimizer's update step using a neural network.
1 code implementation • 17 Nov 2022 • Luke Metz, James Harrison, C. Daniel Freeman, Amil Merchant, Lucas Beyer, James Bradbury, Naman Agrawal, Ben Poole, Igor Mordatch, Adam Roberts, Jascha Sohl-Dickstein
While deep learning models have replaced hand-designed features across many domains, these models are still trained with hand-designed optimizers.
1 code implementation • 22 Mar 2022 • Luke Metz, C. Daniel Freeman, James Harrison, Niru Maheswaranathan, Jascha Sohl-Dickstein
We further leverage our analysis to construct a learned optimizer that is both faster and more memory efficient than previous work.
1 code implementation • 10 Nov 2021 • Luke Metz, C. Daniel Freeman, Samuel S. Schoenholz, Tal Kachman
Differentiable programming techniques are widely used in the community and are responsible for the machine learning renaissance of the past several decades.
1 code implementation • 24 Jun 2021 • C. Daniel Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, Olivier Bachem
We present Brax, an open source library for rigid body simulation with a focus on performance and parallelism on accelerators, written in JAX.
no code implementations • 14 Jan 2021 • Luke Metz, C. Daniel Freeman, Niru Maheswaranathan, Jascha Sohl-Dickstein
We show that a population of randomly initialized learned optimizers can be used to train themselves from scratch in an online fashion, without resorting to a hand designed optimizer in any part of the process.
1 code implementation • 1 Jan 2021 • Luke Metz, Niru Maheswaranathan, Ruoxi Sun, C. Daniel Freeman, Ben Poole, Jascha Sohl-Dickstein
We present TaskSet, a dataset of tasks for use in training and evaluating optimizers.
no code implementations • 1 Jan 2021 • Luke Metz, Niru Maheswaranathan, C. Daniel Freeman, Ben Poole, Jascha Sohl-Dickstein
In this work we focus on general-purpose learned optimizers capable of training a wide variety of problems with no user-specified hyperparameters.
no code implementations • 23 Sep 2020 • Luke Metz, Niru Maheswaranathan, C. Daniel Freeman, Ben Poole, Jascha Sohl-Dickstein
In this work we focus on general-purpose learned optimizers capable of training a wide variety of problems with no user-specified hyperparameters.
no code implementations • 27 Feb 2020 • Luke Metz, Niru Maheswaranathan, Ruoxi Sun, C. Daniel Freeman, Ben Poole, Jascha Sohl-Dickstein
We present TaskSet, a dataset of tasks for use in training and evaluating optimizers.
2 code implementations • NeurIPS 2019 • C. Daniel Freeman, Luke Metz, David Ha
That useful models can arise out of the messy and slow optimization process of evolution suggests that forward-predictive modeling can arise as a side-effect of optimization under the right circumstances.
Model-based Reinforcement Learning reinforcement-learning +1
1 code implementation • 24 Oct 2018 • Luke Metz, Niru Maheswaranathan, Jeremy Nixon, C. Daniel Freeman, Jascha Sohl-Dickstein
Deep learning has shown that learned functions can dramatically outperform hand-designed functions on perceptual tasks.
1 code implementation • 4 Nov 2016 • C. Daniel Freeman, Joan Bruna
Our theoretical work quantifies and formalizes two important \emph{folklore} facts: (i) the landscape of deep linear networks has a radically different topology from that of deep half-rectified ones, and (ii) that the energy landscape in the non-linear case is fundamentally controlled by the interplay between the smoothness of the data distribution and model over-parametrization.
1 code implementation • 16 Mar 2016 • C. Daniel Freeman, C. M. Herdman, K. B. Whaley
We present an error correcting protocol that enhances the lifetime of stabilizer code based qubits which are susceptible to the creation of pairs of localized defects (due to string-like error operators) at finite temperature, such as the toric code.
Quantum Physics Statistical Mechanics
1 code implementation • 9 May 2014 • C. Daniel Freeman, C. M. Herdman, Dylan J Gorman, K. B. Whaley
In contrast to the size-independent bound predicted for the toric code in the thermodynamic limit, we identify a low-temperature regime on finite lattices below a size-dependent crossover temperature with nontrivial finite-size and temperature scaling of the relaxation time.
Statistical Mechanics Quantum Physics