Search Results for author:
Found 9 papers, 0 papers with code
Self-Selected Attention Span for Accelerating Large Language Model Inference
We demonstrate that using this custom CUDA kernel improves the throughput of LLM inference by 28%.
Mixed-Precision Quantization with Cross-Layer Dependencies
Quantization assigning the same bit-width to all layers leads to large accuracy degradation at low precision and is wasteful at high precision settings.
Laconic Deep Learning Computing
A Laconic configuration that uses a 1K-wire weight memory interface, outperforms the 2K-wire conventional accelerator by 15. 4x and is 1. 95x more energy efficient.
DPRed: Making Typical Activation and Weight Values Matter In Deep Learning Computing
The per group precisions are selected statically for the weights and dynamically by hardware for the activations.
Bit-Tactical: Exploiting Ineffectual Computations in Convolutional Neural Networks: Which, Why, and How
We show that, during inference with Convolutional Neural Networks (CNNs), more than 2x to $8x ineffectual work can be exposed if instead of targeting those weights and activations that are zero, we target different combinations of value stream properties.
Tartan: Accelerating Fully-Connected and Convolutional Layers in Deep Learning Networks by Exploiting Numerical Precision Variability
Experiments on image classification CNNs show that on average across all networks studied, TRT outperforms a state-of-the-art bit-parallel accelerator by 1:90x without any loss in accuracy while it is 1:17x more energy efficient.
Loom: Exploiting Weight and Activation Precisions to Accelerate Convolutional Neural Networks
LM can trade-off accuracy for additional improvements in execution performance and energy efficiency and compares favorably to an accelerator that targeted only activation precisions.
Dynamic Stripes: Exploiting the Dynamic Precision Requirements of Activation Values in Neural Networks
Stripes is a Deep Neural Network (DNN) accelerator that uses bit-serial computation to offer performance that is proportional to the fixed-point precision of the activation values.
Cnvlutin2: Ineffectual-Activation-and-Weight-Free Deep Neural Network Computing
We also present a modified organization that detects the activations that are deemed as ineffectual while fetching them from memory.
