Search Results for author:
Found 6 papers, 2 papers with code
FOSI: Hybrid First and Second Order Optimization
Popular machine learning approaches forgo second-order information due to the difficulty of computing curvature in high dimensions.
Optimizing Data Collection in Deep Reinforcement Learning
We show that simulator kernel fusion speedups with a simple simulator are $11. 3\times$ and increase by up to $1024\times$ as simulator complexity increases in terms of memory bandwidth requirements.
RL-Scope: Cross-Stack Profiling for Deep Reinforcement Learning Workloads
Deep reinforcement learning (RL) has made groundbreaking advancements in robotics, data center management and other applications.
It's Not What Machines Can Learn, It's What We Cannot Teach
Can deep neural networks learn to solve any task, and in particular problems of high complexity?
Taming Momentum in a Distributed Asynchronous Environment
We propose DANA: a novel technique for asynchronous distributed SGD with momentum that mitigates gradient staleness by computing the gradient on an estimated future position of the model's parameters.
DANA: Scalable Out-of-the-box Distributed ASGD Without Retuning
We propose DANA, a novel approach that scales out-of-the-box to large clusters using the same hyperparameters and learning schedule optimized for training on a single worker, while maintaining similar final accuracy without additional overhead.
