Search Results for author:
Found 8 papers, 2 papers with code
Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping
Using an extended and formalized version of the Q/C map analysis of Poole et al. (2016), along with Neural Tangent Kernel theory, we identify the main pathologies present in deep networks that prevent them from training fast and generalizing to unseen data, and show how these can be avoided by carefully controlling the "shape" of the network's initialization-time kernel function.
Faster Improvement Rate Population Based Training
Our experiments show that FIRE PBT is able to outperform PBT on the ImageNet benchmark and match the performance of networks that were trained with a hand-tuned learning rate schedule.
Open-Ended Learning Leads to Generally Capable Agents
The resulting space is exceptionally diverse in terms of the challenges posed to agents, and as such, even measuring the learning progress of an agent is an open research problem.
Perception-Prediction-Reaction Agents for Deep Reinforcement Learning
We employ a temporal hierarchy, using a slow-ticking recurrent core to allow information to flow more easily over long time spans, and three fast-ticking recurrent cores with connections designed to create an information asymmetry.
A Generalized Framework for Population Based Training
Population Based Training (PBT) is a recent approach that jointly optimizes neural network weights and hyperparameters which periodically copies weights of the best performers and mutates hyperparameters during training.
Population Based Training of Neural Networks
Neural networks dominate the modern machine learning landscape, but their training and success still suffer from sensitivity to empirical choices of hyperparameters such as model architecture, loss function, and optimisation algorithm.
Tuning the Scheduling of Distributed Stochastic Gradient Descent with Bayesian Optimization
We present an optimizer which uses Bayesian optimization to tune the system parameters of distributed stochastic gradient descent (SGD).
Learning Runtime Parameters in Computer Systems with Delayed Experience Injection
This paper investigates the use of deep reinforcement learning for runtime parameters of cloud databases under latency constraints.
