Search Results for author:
Found 4 papers, 1 papers with code
Progressive Gradient Flow for Robust N:M Sparsity Training in Transformers
In this work, we study the effectiveness of existing sparse training recipes at \textit{high-sparsity regions} and argue that these methods fail to sustain the model quality on par with low-sparsity regions.
Subgraph Stationary Hardware-Software Inference Co-Design
For the stream of queries, SUSHI yields up to 25% improvement in latency, 0. 98% increase in served accuracy.
VEGETA: Vertically-Integrated Extensions for Sparse/Dense GEMM Tile Acceleration on CPUs
Therefore, as DL workloads embrace sparsity to reduce the computations and memory size of models, it is also imperative for CPUs to add support for sparsity to avoid under-utilization of the dense matrix engine and inefficient usage of the caches and registers.
COMET: A Comprehensive Cluster Design Methodology for Distributed Deep Learning Training
To facilitate the design space exploration of such massive DL training clusters, we introduce COMET, a holistic cluster design methodology and workflow to jointly study the impact of parallelization strategies and key cluster resource provisioning on the performance of distributed DL training.
