Search Results for author:
Found 5 papers, 1 papers with code
An Adaptive Deep RL Method for Non-Stationary Environments with Piecewise Stable Context
One of the key challenges in deploying RL to real-world applications is to adapt to variations of unknown environment contexts, such as changing terrains in robotic tasks and fluctuated bandwidth in congestion control.
Tutel: Adaptive Mixture-of-Experts at Scale
On efficiency, Flex accelerates SwinV2-MoE, achieving up to 1. 55x and 2. 11x speedup in training and inference over Fairseq, respectively.
CrossoverScheduler: Overlapping Multiple Distributed Training Applications in a Crossover Manner
Distributed deep learning workloads include throughput-intensive training tasks on the GPU clusters, where the Distributed Stochastic Gradient Descent (SGD) incurs significant communication delays after backward propagation, forces workers to wait for the gradient synchronization via a centralized parameter server or directly in decentralized workers.
Simulating Performance of ML Systems with Offline Profiling
We advocate that simulation based on offline profiling is a promising approach to better understand and improve the complex ML systems.
Stanza: Layer Separation for Distributed Training in Deep Learning
Thus, we propose layer separation in distributed training: the majority of the nodes just train the convolutional layers, and the rest train the fully connected layers only.
