Search Results for author:
Found 4 papers, 1 papers with code
ROAM: memory-efficient large DNN training via optimized operator ordering and memory layout
However, a memory-efficient execution plan that includes a reasonable operator execution order and tensor memory layout can significantly increase the models' memory efficiency and reduce overheads from high-level techniques.
Auto-Parallelizing Large Models with Rhino: A Systematic Approach on Production AI Platform
We present Rhino, a system for accelerating tensor programs with automatic parallelization on AI platform for real production environment.
TAP: Accelerating Large-Scale DNN Training Through Tensor Automatic Parallelisation
However, finding a suitable model parallel schedule for an arbitrary neural network is a non-trivial task due to the exploding search space.
Go Wider Instead of Deeper
To achieve better performance with fewer trainable parameters, recent methods are proposed to go shallower by parameter sharing or model compressing along with the depth.
Ranked #677 on
Image Classification
on ImageNet
