Search Results for author:
Found 12 papers, 4 papers with code
Harnessing Deep Learning and HPC Kernels via High-Level Loop and Tensor Abstractions on CPU Architectures
During the past decade, Deep Learning (DL) algorithms, programming systems and hardware have converged with the High Performance Computing (HPC) counterparts.
Efficient and Generic 1D Dilated Convolution Layer for Deep Learning
Finally, we demonstrate the performance of our optimized 1D convolution layer by utilizing it in the end-to-end neural network training with real genomics datasets and achieve up to 6. 86x speedup over the oneDNN library-based implementation on Cascade Lake CPUs.
DistGNN: Scalable Distributed Training for Large-Scale Graph Neural Networks
Full-batch training on Graph Neural Networks (GNN) to learn the structure of large graphs is a critical problem that needs to scale to hundreds of compute nodes to be feasible.
Tensor Processing Primitives: A Programming Abstraction for Efficiency and Portability in Deep Learning & HPC Workloads
The TPP specification is platform-agnostic, thus code expressed via TPPs is portable, whereas the TPP implementation is highly-optimized and platform-specific.
Optimizing Deep Learning Recommender Systems' Training On CPU Cluster Architectures
During the last two years, the goal of many researchers has been to squeeze the last bit of performance out of HPC system for AI tasks.
K-TanH: Efficient TanH For Deep Learning
We propose K-TanH, a novel, highly accurate, hardware efficient approximation of popular activation function TanH for Deep Learning.
High-Performance Deep Learning via a Single Building Block
Deep learning (DL) is one of the most prominent branches of machine learning.
A Study of BFLOAT16 for Deep Learning Training
In this paper, we discuss the flow of tensors and various key operations in mixed precision training, and delve into details of operations, such as the rounding modes for converting FP32 tensors to BFLOAT16.
Anatomy Of High-Performance Deep Learning Convolutions On SIMD Architectures
Convolution layers are prevalent in many classes of deep neural networks, including Convolutional Neural Networks (CNNs) which provide state-of-the-art results for tasks like image recognition, neural machine translation and speech recognition.
Distributed, Parallel, and Cluster Computing
Mixed Precision Training of Convolutional Neural Networks using Integer Operations
The state-of-the-art (SOTA) for mixed precision training is dominated by variants of low precision floating point operations, and in particular, FP16 accumulating into FP32 Micikevicius et al. (2017).
On Scale-out Deep Learning Training for Cloud and HPC
The exponential growth in use of large deep neural networks has accelerated the need for training these deep neural networks in hours or even minutes.
Distributed Deep Learning Using Synchronous Stochastic Gradient Descent
We design and implement a distributed multinode synchronous SGD algorithm, without altering hyper parameters, or compressing data, or altering algorithmic behavior.
