Search Results for author:
Found 11 papers, 2 papers with code
ForestColl: Efficient Collective Communications on Heterogeneous Network Fabrics
ForestColl also outperforms other state-of-the-art schedule generation techniques with both up to 61\% more efficient generated schedules and orders of magnitude faster schedule generation speed.
Tessel: Boosting Distributed Execution of Large DNN Models via Flexible Schedule Search
This paper presents Tessel, an automated system that searches for efficient schedules for distributed DNN training and inference for diverse operator placement strategies.
Look-Up mAI GeMM: Increasing AI GeMMs Performance by Nearly 2.5x via msGeMM
AI models are increasing in size and recent advancement in the community has shown that unlike HPC applications where double precision datatype are required, lower-precision datatypes such as fp8 or int4 are sufficient to bring the same model quality both for training and inference.
SuperScaler: Supporting Flexible DNN Parallelization via a Unified Abstraction
SuperScaler is a system that facilitates the design and generation of highly flexible parallelization plans.
Error-Covariance Analysis of Monocular Pose Estimation Using Total Least Squares
First, the optimization framework is formulated for the pose estimation problem with observation vectors extracted from unit vectors from the camera center-of-projection, pointing towards the image features.
TACCL: Guiding Collective Algorithm Synthesis using Communication Sketches
TACCL uses a novel communication sketch abstraction to get crucial information from the designer to significantly reduce the search space and guide the synthesizer towards better algorithms.
Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads
Therefore, we present CoCoNeT, with a DSL to express a program with both computation and communication.
Scaling Distributed Training with Adaptive Summation
This paper introduces a novel method to combine gradients called Adasum (for adaptive sum) that converges faster than prior work.
Distributed Training of Embeddings using Graph Analytics
This paper presents a distributed training framework for a class of applications that use Skip-gram-like models to generate embeddings.
CHET: Compiler and Runtime for Homomorphic Evaluation of Tensor Programs
Just as the hardware ISA interface enabled hardware advances to proceed independent of software advances in the compiler and language runtimes, HISA decouples compiler optimizations and runtimes for supporting FHE applications from advancements in the underlying FHE schemes.
Parallel Stochastic Gradient Descent with Sound Combiners
This paper proposes SYMSGD, a parallel SGD algorithm that, to a first-order approximation, retains the sequential semantics of SGD.
