Search Results for author:
Found 3 papers, 0 papers with code
Fine-Grained Embedding Dimension Optimization During Training for Recommender Systems
Huge embedding tables in modern Deep Learning Recommender Models (DLRM) require prohibitively large memory during training and inference.
Heterogeneity-Aware Asynchronous Decentralized Training
Is it possible to get the best of both worlds - designing a distributed training method that has both high performance as All-Reduce in homogeneous environment and good heterogeneity tolerance as AD-PSGD?
Hop: Heterogeneity-Aware Decentralized Training
Based on a unique characteristic of decentralized training that we have identified, the iteration gap, we propose a queue-based synchronization mechanism that can efficiently implement backup workers and bounded staleness in the decentralized setting.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
