Search Results for author:
Found 35 papers, 10 papers with code
LightSeq2: Accelerated Training for Transformer-based Models on GPUs
In this paper, we present LightSeq2, a system to accelerate training for a general family of Transformer models on GPUs.
GNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUs
As the emerging trend of graph-based deep learning, Graph Neural Networks (GNNs) excel for their capability to generate high-quality node feature vectors (embeddings).
Distributed, Parallel, and Cluster Computing
TC-GNN: Bridging Sparse GNN Computation and Dense Tensor Cores on GPUs
Recently, graph neural networks (GNNs), as the backbone of graph-based machine learning, demonstrate great success in various domains (e. g., e-commerce).
MGG: Accelerating Graph Neural Networks with Fine-grained intra-kernel Communication-Computation Pipelining on Multi-GPU Platforms
For irregularly sparse and fine-grained GNN workloads, such solutions miss the opportunity to jointly schedule/optimize the computation and communication operations for high-performance delivery.
APNN-TC: Accelerating Arbitrary Precision Neural Networks on Ampere GPU Tensor Cores
Over the years, accelerating neural networks with quantization has been widely studied.
DSXplore: Optimizing Convolutional Neural Networks via Sliding-Channel Convolutions
It also brings profound impact to improve the applicability of the compute- and memory-intensive CNNs to a broad range of applications, such as mobile devices, which are generally short of computation power and memory.
Faith: An Efficient Framework for Transformer Verification on GPUs
In this paper, we propose Faith, an efficient framework for transformer verification on GPUs.
Domain-Adversarial Multi-Task Framework for Novel Therapeutic Property Prediction of Compounds
In this paper, we propose a novel domain-adversarial multi-task framework for integrating shared knowledge from multiple domains.
Comprehensive SNN Compression Using ADMM Optimization and Activity Regularization
As well known, the huge memory and compute costs of both artificial neural networks (ANNs) and spiking neural networks (SNNs) greatly hinder their deployment on edge devices with high efficiency.
On Adversarial Robustness of Point Cloud Semantic Segmentation
As such, we present a comparative study of PCSS robustness.
Dynamic Sparse Graph for Efficient Deep Learning
We propose to execute deep neural networks (DNNs) with dynamic and sparse graph (DSG) structure for compressive memory and accelerative execution during both training and inference.
Reconciling Feature-Reuse and Overfitting in DenseNet with Specialized Dropout
The insights attained here could potentially be applied as a general approach for boosting the accuracy of other CNN models with similar nonlinear connections.
Penetrating the Fog: the Path to Efficient CNN Models
In this paper, we are the first in the field to consider how to craft an effective sparse kernel design by eliminating the large design space.
PCNN: Environment Adaptive Model Without Finetuning
Convolutional Neural Networks (CNNs) have achieved tremendous success for many computer vision tasks, which shows a promising perspective of deploying CNNs on mobile platforms.
AccD: A Compiler-based Framework for Accelerating Distance-related Algorithms on CPU-FPGA Platforms
As a promising solution to boost the performance of distance-related algorithms (e. g., K-means and KNN), FPGA-based acceleration attracts lots of attention, but also comes with numerous challenges.
Distributed, Parallel, and Cluster Computing Programming Languages
Neural Network Model Extraction Attacks in Edge Devices by Hearing Architectural Hints
As neural networks continue their reach into nearly every aspect of software operations, the details of those networks become an increasingly sensitive subject.
Cryptography and Security Hardware Architecture
Proq: Projection-based Runtime Assertions for Debugging on a Quantum Computer
In this paper, we propose Proq, a runtime assertion scheme for testing and debugging quantum programs on a quantum computer.
Exploring Adversarial Attack in Spiking Neural Networks with Spike-Compatible Gradient
Recently, backpropagation through time inspired learning algorithms are widely introduced into SNNs to improve the performance, which brings the possibility to attack the models accurately given Spatio-temporal gradient maps.
Weighted-Sampling Audio Adversarial Example Attack
In this paper, we propose~\textit{weighted-sampling audio adversarial examples}, focusing on the numbers and the weights of distortion to reinforce the attack.
SGQuant: Squeezing the Last Bit on Graph Neural Networks with Specialized Quantization
With the increasing popularity of graph-based learning, Graph Neural Networks (GNNs) win lots of attention from the research and industry field because of their high accuracy.
Boosting Deep Neural Network Efficiency with Dual-Module Inference
Using Deep Neural Networks (DNNs) in machine learning tasks is promising in delivering high-quality results but challenging to meet stringent latency requirements and energy constraints because of the memory-bound and the compute-bound execution pattern of DNNs.
An Efficient Quantitative Approach for Optimizing Convolutional Neural Networks
To clear these hurdles, we propose 3D-Receptive Field (3DRF), an explainable and easy-to-compute metric, to estimate the quality of a CNN architecture and guide the search process of designs.
Scalable Adversarial Attack on Graph Neural Networks with Alternating Direction Method of Multipliers
Graph neural networks (GNNs) have achieved high performance in analyzing graph-structured data and have been widely deployed in safety-critical areas, such as finance and autonomous driving.
Uncertainty-aware Attention Graph Neural Network for Defending Adversarial Attacks
With the increasing popularity of graph-based learning, graph neural networks (GNNs) emerge as the essential tool for gaining insights from graphs.
Rubik: A Hierarchical Architecture for Efficient Graph Learning
Graph convolutional network (GCN) emerges as a promising direction to learn the inductive representation in graph data commonly used in widespread applications, such as E-commerce, social networks, and knowledge graphs.
Hardware Architecture
MPU: Towards Bandwidth-abundant SIMT Processor via Near-bank Computing
For general purpose scenarios, lightweight hardware designs for diverse data paths, architectural supports for the SIMT programming model, and end-to-end software optimizations remain challenging.
Hardware Architecture
DFSSATTEN: Dynamic Fine-grained Structured Sparse Attention Mechanism
Transformers are becoming mainstream solutions for various tasks like NLP and Computer vision.
Understanding GNN Computational Graph: A Coordinated Computation, IO, and Memory Perspective
The same data are propagated through the graph structure to perform the same neural operation multiple times in GNNs, leading to redundant computation which accounts for 92. 4% of total operators.
Transformer Acceleration with Dynamic Sparse Attention
We demonstrate that the sparse patterns are dynamic, depending on input sequences.
Dual-module Inference for Efficient Recurrent Neural Networks
Using Recurrent Neural Networks (RNNs) in sequence modeling tasks is promising in delivering high-quality results but challenging to meet stringent latency requirements because of the memory-bound execution pattern of RNNs.
Mitigating Noise-Induced Gradient Vanishing in Variational Quantum Algorithm Training
In this paper, we propose a novel training scheme to mitigate such noise-induced gradient vanishing.
Towards Efficient Ansatz Architecture for Variational Quantum Algorithms
In this paper, we propose a novel training scheme to mitigate such noise-induced gradient vanishing.
Dynamic N:M Fine-grained Structured Sparse Attention Mechanism
We evaluate the 1:2 and 2:4 sparsity under different configurations and achieve 1. 27~ 1. 89x speedups over the full-attention mechanism.
A Geometrical Approach to Evaluate the Adversarial Robustness of Deep Neural Networks
Hence, given different inputs, it requires different time for converging to an adversarial sample.
Tackling the Qubit Mapping Problem for NISQ-Era Quantum Devices
Due to little consideration in the hardware constraints, e. g., limited connections between physical qubits to enable two-qubit gates, most quantum algorithms cannot be directly executed on the Noisy Intermediate-Scale Quantum (NISQ) devices.
