Search Results for author:
Found 12 papers, 3 papers with code
MSCCL++: Rethinking GPU Communication Abstractions for Cutting-edge AI Applications
This paper discusses an alternative communication library interface for AI applications that offers both portability and performance by reducing redundant efforts while maintaining flexibility for customization.
ForestColl: Throughput-Optimal Collective Communications on Heterogeneous Network Fabrics
ForestColl supports any network fabrics, including both switching fabrics and direct accelerator connections.
Tessel: Boosting Distributed Execution of Large DNN Models via Flexible Schedule Search
This paper presents Tessel, an automated system that searches for efficient schedules for distributed DNN training and inference for diverse operator placement strategies.
Look-Up mAI GeMM: Increasing AI GeMMs Performance by Nearly 2.5x via msGeMM
AI models are increasing in size and recent advancement in the community has shown that unlike HPC applications where double precision datatype are required, lower-precision datatypes such as fp8 or int4 are sufficient to bring the same model quality both for training and inference.
SuperScaler: Supporting Flexible DNN Parallelization via a Unified Abstraction
SuperScaler is a system that facilitates the design and generation of highly flexible parallelization plans.
Error-Covariance Analysis of Monocular Pose Estimation Using Total Least Squares
First, the optimization framework is formulated for the pose estimation problem with observation vectors extracted from unit vectors from the camera center-of-projection, pointing towards the image features.
TACCL: Guiding Collective Algorithm Synthesis using Communication Sketches
TACCL uses a novel communication sketch abstraction to get crucial information from the designer to significantly reduce the search space and guide the synthesizer towards better algorithms.
Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads
Therefore, we present CoCoNeT, with a DSL to express a program with both computation and communication.
Scaling Distributed Training with Adaptive Summation
This paper introduces a novel method to combine gradients called Adasum (for adaptive sum) that converges faster than prior work.
Distributed Training of Embeddings using Graph Analytics
This paper presents a distributed training framework for a class of applications that use Skip-gram-like models to generate embeddings.
CHET: Compiler and Runtime for Homomorphic Evaluation of Tensor Programs
Just as the hardware ISA interface enabled hardware advances to proceed independent of software advances in the compiler and language runtimes, HISA decouples compiler optimizations and runtimes for supporting FHE applications from advancements in the underlying FHE schemes.
Parallel Stochastic Gradient Descent with Sound Combiners
This paper proposes SYMSGD, a parallel SGD algorithm that, to a first-order approximation, retains the sequential semantics of SGD.
