Search Results for author:
Found 14 papers, 8 papers with code
Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks
Large-scale distributed training of deep neural networks suffer from the generalization gap caused by the increase in the effective mini-batch size.
μ-cuDNN: Accelerating Deep Learning Frameworks with Micro-Batching
NVIDIA cuDNN is a low-level library that provides GPU kernels frequently used in deep learning.
Batched Sparse Matrix Multiplication for Accelerating Graph Convolutional Networks
We evaluated the performance of the GCNs application on TSUBAME3. 0 implementing NVIDIA Tesla P100 GPU, and our batched approach shows significant speedups of up to 1. 59x and 1. 37x in training and inference, respectively.
Distributed, Parallel, and Cluster Computing
Combined Spatial and Temporal Blocking for High-Performance Stencil Computation on FPGAs Using OpenCL
Furthermore, we estimate that the upcoming Stratix 10 devices can achieve a performance of up to 3. 5 TFLOP/s and 1. 6 TFLOP/s for 2D and 3D stencil computation, respectively.
Distributed, Parallel, and Cluster Computing Hardware Architecture
High-Performance High-Order Stencil Computation on FPGAs Using OpenCL
We show that despite the higher computation intensity and on-chip memory requirement of such stencils compared to first-order ones, our design technique with combined spatial and temporal blocking remains effective.
Distributed, Parallel, and Cluster Computing
Word Embeddings, Analogies, and Machine Learning: Beyond king - man + woman = queen
Solving word analogies became one of the most popular benchmarks for word embeddings on the assumption that linear relations between word pairs (such as \textit{king}:\textit{man} :: \textit{woman}:\textit{queen}) are indicative of the quality of the embedding.
The Case for Strong Scaling in Deep Learning: Training Large 3D CNNs with Hybrid Parallelism
We present scalable hybrid-parallel algorithms for training large-scale 3D convolutional neural networks.
Scaling Distributed Deep Learning Workloads beyond the Memory Capacity with KARMA
An alternative solution is to use out-of-core methods instead of, or in addition to, data parallelism.
A Survey on Coarse-Grained Reconfigurable Architectures from a Performance Perspective
With the end of both Dennard's scaling and Moore's law, computer users and researchers are aggressively exploring alternative forms of computing in order to continue the performance scaling that we have come to enjoy.
Hardware Architecture A.1; B.0; C.1; C.3
High-performance sparse matrix-matrix products on Intel KNL and multicore architectures
Our hash-table and heap-based algorithms are showing significant speedups from libraries in the majority of the cases while different algorithms dominate the other scenarios with different matrix size, sparsity, compression factor and operation type.
Distributed, Parallel, and Cluster Computing
Adaptive Pattern Matching with Reinforcement Learning for Dynamic Graphs
We propose an incremental graph pattern matching algorithm to deal with time-evolving graph data and also propose an adaptive optimization system based on reinforcement learning to recompute vertices in the incremental process more efficiently.
Databases
MLPerf HPC: A Holistic Benchmark Suite for Scientific Machine Learning on HPC Systems
Scientific communities are increasingly adopting machine learning and deep learning models in their applications to accelerate scientific insights.
Myths and Legends in High-Performance Computing
While some laws end, new directions are emerging, such as algorithmic scaling or novel architecture research.
