Search Results for author:
Found 11 papers, 3 papers with code
Boosting Deep Neural Network Efficiency with Dual-Module Inference
Using Deep Neural Networks (DNNs) in machine learning tasks is promising in delivering high-quality results but challenging to meet stringent latency requirements and energy constraints because of the memory-bound and the compute-bound execution pattern of DNNs.
Faith: An Efficient Framework for Transformer Verification on GPUs
In this paper, we propose Faith, an efficient framework for transformer verification on GPUs.
Dynamic N:M Fine-grained Structured Sparse Attention Mechanism
We evaluate the 1:2 and 2:4 sparsity under different configurations and achieve 1. 27~ 1. 89x speedups over the full-attention mechanism.
Transformer Acceleration with Dynamic Sparse Attention
We demonstrate that the sparse patterns are dynamic, depending on input sequences.
DFSSATTEN: Dynamic Fine-grained Structured Sparse Attention Mechanism
Transformers are becoming mainstream solutions for various tasks like NLP and Computer vision.
H2Learn: High-Efficiency Learning Accelerator for High-Accuracy Spiking Neural Networks
Although spiking neural networks (SNNs) take benefits from the bio-plausible neural modeling, the low accuracy under the common local synaptic plasticity learning rules limits their application in many practical tasks.
Redefining Self-Normalization Property
Moreover, analysis on the activation's mean in the forward pass reveals that the self-normalization property gets weaker with larger fan-in of each layer, which explains the performance degradation on large benchmarks like ImageNet.
A Comprehensive and Modularized Statistical Framework for Gradient Norm Equality in Deep Neural Networks
Powered by our metric and framework, we analyze extensive initialization, normalization, and network structures.
Effective and Efficient Batch Normalization Using Few Uncorrelated Data for Statistics' Estimation
We identify that the effectiveness expects less data correlation while the efficiency expects regular execution pattern.
Hardness-Aware Deep Metric Learning
This paper presents a hardness-aware deep metric learning (HDML) framework.
Ranked #30 on
Metric Learning
on CUB-200-2011
(using extra training data)
Batch Normalization Sampling
In this paper, we propose alleviating this problem through sampling only a small fraction of data for normalization at each iteration.
