Search Results for author:
Found 10 papers, 4 papers with code
Harnessing Deep Learning and HPC Kernels via High-Level Loop and Tensor Abstractions on CPU Architectures
During the past decade, Deep Learning (DL) algorithms, programming systems and hardware have converged with the High Performance Computing (HPC) counterparts.
FPGA-based AI Smart NICs for Scalable Distributed AI Training Systems
The overhead of these collective communication operations in a distributed AI training system can bottleneck its performance, with more pronounced effects as the number of nodes increases.
Efficient and Generic 1D Dilated Convolution Layer for Deep Learning
Finally, we demonstrate the performance of our optimized 1D convolution layer by utilizing it in the end-to-end neural network training with real genomics datasets and achieve up to 6. 86x speedup over the oneDNN library-based implementation on Cascade Lake CPUs.
DistGNN: Scalable Distributed Training for Large-Scale Graph Neural Networks
Full-batch training on Graph Neural Networks (GNN) to learn the structure of large graphs is a critical problem that needs to scale to hundreds of compute nodes to be feasible.
Tensor Processing Primitives: A Programming Abstraction for Efficiency and Portability in Deep Learning & HPC Workloads
The TPP specification is platform-agnostic, thus code expressed via TPPs is portable, whereas the TPP implementation is highly-optimized and platform-specific.
Optimizing Deep Learning Recommender Systems' Training On CPU Cluster Architectures
During the last two years, the goal of many researchers has been to squeeze the last bit of performance out of HPC system for AI tasks.
High-Performance Deep Learning via a Single Building Block
Deep learning (DL) is one of the most prominent branches of machine learning.
A Study of BFLOAT16 for Deep Learning Training
In this paper, we discuss the flow of tensors and various key operations in mixed precision training, and delve into details of operations, such as the rounding modes for converting FP32 tensors to BFLOAT16.
Anatomy Of High-Performance Deep Learning Convolutions On SIMD Architectures
Convolution layers are prevalent in many classes of deep neural networks, including Convolutional Neural Networks (CNNs) which provide state-of-the-art results for tasks like image recognition, neural machine translation and speech recognition.
Distributed, Parallel, and Cluster Computing
Mixed Precision Training of Convolutional Neural Networks using Integer Operations
The state-of-the-art (SOTA) for mixed precision training is dominated by variants of low precision floating point operations, and in particular, FP16 accumulating into FP32 Micikevicius et al. (2017).
