Search Results for author:
Found 7 papers, 2 papers with code
Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces
Benchmarking and co-design are essential for driving optimizations and innovation around ML models, ML software, and next-generation hardware.
ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale
In this paper, we extend the open-source ASTRA-sim infrastructure and endow it with the capabilities to model state-of-the-art and emerging distributed training models and platforms.
COMET: A Comprehensive Cluster Design Methodology for Distributed Deep Learning Training
To facilitate the design space exploration of such massive DL training clusters, we introduce COMET, a holistic cluster design methodology and workflow to jointly study the impact of parallelization strategies and key cluster resource provisioning on the performance of distributed DL training.
Impact of RoCE Congestion Control Policies on Distributed Training of DNNs
Our results indicate that previously proposed RoCE congestion control schemes have little impact on the end-to-end performance of training workloads, motivating the necessity of designing an optimized, yet low-overhead, congestion control scheme based on the characteristics of distributed training platforms and workloads.
Themis: A Network Bandwidth-Aware Collective Scheduling Policy for Distributed Training of DL Models
Distributed training is a solution to reduce DNN training time by splitting the task across multiple NPUs (e. g., GPU/TPU).
Exploring Multi-dimensional Hierarchical Network Topologies for Efficient Distributed Training of Trillion Parameter DL Models
High-performance distributed training platforms should leverage multi-dimensional hierarchical networks, which interconnect accelerators through different levels of the network, to dramatically reduce expensive NICs required for the scale-out network.
Restructuring, Pruning, and Adjustment of Deep Models for Parallel Distributed Inference
In this paper, we consider the parallel implementation of an already-trained deep model on multiple processing nodes (a. k. a.
