Search Results for author:
Found 21 papers, 13 papers with code
S2DNAS: Transforming Static CNN Model for Dynamic Inference via Neural Architecture Search
To this end, based on a given CNN model, we first generate a CNN architecture space in which each architecture is a multi-stage CNN generated from the given model using some predefined transformations.
A Survey on Efficient Inference for Large Language Models
This paper presents a comprehensive survey of the existing literature on efficient LLM inference.
PillarTrack: Redesigning Pillar-based Transformer Network for Single Object Tracking on Point Clouds
LiDAR-based 3D single object tracking (3D SOT) is a critical issue in robotics and autonomous driving.
LLM Inference Unveiled: Survey and Roofline Model Insights
Our survey stands out from traditional literature reviews by not only summarizing the current state of research but also by introducing a framework based on roofline model for systematic analysis of LLM inference techniques.
WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More
Large Language Models (LLMs) face significant deployment challenges due to their substantial memory requirements and the computational demands of auto-regressive text generation process.
QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuning
Diffusion models have achieved remarkable success in image generation tasks, yet their practical deployment is restrained by the high memory and time consumption.
MIM4DD: Mutual Information Maximization for Dataset Distillation
Thus, we introduce mutual information (MI) as the metric to quantify the shared information between the synthetic and the real datasets, and devise MIM4DD numerically maximizing the MI via a newly designed optimizable objective within a contrastive learning framework to update the synthetic dataset.
Post-Training Quantization for Re-parameterization via Coarse & Fine Weight Splitting
Although neural networks have made remarkable advancements in various applications, they require substantial computational and memory resources.
ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models
This paper explores a new post-hoc training-free compression paradigm for compressing Large Language Models (LLMs) to facilitate their wider adoption in various computing environments.
PB-LLM: Partially Binarized Large Language Models
This paper explores network binarization, a radical form of quantization, compressing model weights to a single bit, specifically for Large Language Models (LLMs) compression.
Latency-aware Unified Dynamic Networks for Efficient Image Recognition
Dynamic computation has emerged as a promising avenue to enhance the inference efficiency of deep networks.
Improving Post-Training Quantization on Object Detection with Task Loss-Guided Lp Metric
PTQ optimizes the quantization parameters by different metrics to minimize the perturbation of quantization.
RPTQ: Reorder-based Post-training Quantization for Large Language Models
In this paper, we identify that the challenge in quantizing activations in LLMs arises from varying ranges across channels, rather than solely the presence of outliers.
Benchmarking the Reliability of Post-training Quantization: a Particular Focus on Worst-case Performance
Post-training quantization (PTQ) is a popular method for compressing deep neural networks (DNNs) without modifying their original architecture or training procedures.
PD-Quant: Post-Training Quantization based on Prediction Difference Metric
It determines the quantization parameters by using the information of differences between network prediction before and after quantization.
Post-training Quantization on Diffusion Models
These approaches define a forward diffusion process for transforming data into noise and a backward denoising process for sampling data from noise.
Latency-aware Spatial-wise Dynamic Networks
The latency prediction model can efficiently estimate the inference latency of dynamic networks by simultaneously considering algorithms, scheduling strategies, and hardware properties.
PTQ4ViT: Post-Training Quantization Framework for Vision Transformers with Twin Uniform Quantization
We observe the distributions of activation values after softmax and GELU functions are quite different from the Gaussian distribution.
PTQ-SL: Exploring the Sub-layerwise Post-training Quantization
Network quantization is a powerful technique to compress convolutional neural networks.
ENAS4D: Efficient Multi-stage CNN Architecture Search for Dynamic Inference
The inference of a input sample can exit from early stage if the prediction of the stage is confident enough.
S2DNAS:Transforming Static CNN Model for Dynamic Inference via Neural Architecture Search
Recently, dynamic inference has emerged as a promising way to reduce the computational cost of deep convolutional neural network (CNN).
