Search Results for author:
Found 31 papers, 17 papers with code
S2DNAS: Transforming Static CNN Model for Dynamic Inference via Neural Architecture Search
To this end, based on a given CNN model, we first generate a CNN architecture space in which each architecture is a multi-stage CNN generated from the given model using some predefined transformations.
E-CAR: Efficient Continuous Autoregressive Image Generation via Multistage Modeling
Our approach not only enhances computational efficiency but also aligns naturally with image generation principles by operating in continuous token space and following a hierarchical generation process from coarse to fine details.
MVCTrack: Boosting 3D Point Cloud Tracking via Multimodal-Guided Virtual Cues
3D single object tracking is essential in autonomous driving and robotics.
LiteVAR: Compressing Visual Autoregressive Modelling with Efficient Attention and Quantization
Visual Autoregressive (VAR) has emerged as a promising approach in image generation, offering competitive potential and performance comparable to diffusion-based models.
CSKV: Training-Efficient Channel Shrinking for KV Cache in Long-Context Scenarios
Therefore, we introduce CSKV, a training-efficient Channel Shrinking technique for KV cache compression: (1) We first analyze the singular value distribution of the KV cache, revealing significant redundancy and compression potential along the channel dimension.
Learning High-Frequency Functions Made Easy with Sinusoidal Positional Encoding
Fourier features based positional encoding (PE) is commonly used in machine learning tasks that involve learning high-frequency features from low-dimensional inputs, such as 3D view synthesis and time series regression with neural tangent kernels.
DiTFastAttn: Attention Compression for Diffusion Transformer Models
Diffusion Transformers (DiT) excel at image and video generation but face computational challenges due to the quadratic complexity of self-attention operators.
PillarHist: A Quantization-aware Pillar Feature Encoder based on Height-aware Histogram
To address above issue, we first unveil the importance of different input information during PFE and identify the height dimension as a key factor in enhancing 3D detection performance.
I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models
Post-training quantization (PTQ) serves as a potent technique to accelerate the inference of large language models (LLMs).
A Closer Look at Time Steps is Worthy of Triple Speed-Up for Diffusion Model Training
Training diffusion models is always a computation-intensive task.
SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models
Large language models (LLMs) can now handle longer sequences of tokens, enabling complex tasks like book understanding and generating lengthy novels.
A Survey on Efficient Inference for Large Language Models
This paper presents a comprehensive survey of the existing literature on efficient LLM inference.
PillarTrack: Redesigning Pillar-based Transformer Network for Single Object Tracking on Point Clouds
LiDAR-based 3D single object tracking (3D SOT) is a critical issue in robotics and autonomous driving.
LLM Inference Unveiled: Survey and Roofline Model Insights
Our survey stands out from traditional literature reviews by not only summarizing the current state of research but also by introducing a framework based on roofline model for systematic analysis of LLM inference techniques.
WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More
Large Language Models (LLMs) face significant deployment challenges due to their substantial memory requirements and the computational demands of auto-regressive text generation process.
QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuning
We empirically verify that our approach modifies the activation distribution and provides meaningful temporal information, facilitating easier and more accurate quantization.
MIM4DD: Mutual Information Maximization for Dataset Distillation
Thus, we introduce mutual information (MI) as the metric to quantify the shared information between the synthetic and the real datasets, and devise MIM4DD numerically maximizing the MI via a newly designed optimizable objective within a contrastive learning framework to update the synthetic dataset.
Post-Training Quantization for Re-parameterization via Coarse & Fine Weight Splitting
Although neural networks have made remarkable advancements in various applications, they require substantial computational and memory resources.
ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models
Based on the success of the low-rank decomposition of projection matrices in the self-attention module, we further introduce ASVD to compress the KV cache.
PB-LLM: Partially Binarized Large Language Models
This paper explores network binarization, a radical form of quantization, compressing model weights to a single bit, specifically for Large Language Models (LLMs) compression.
Latency-aware Unified Dynamic Networks for Efficient Image Recognition
Dynamic computation has emerged as a promising avenue to enhance the inference efficiency of deep networks.
Improving Post-Training Quantization on Object Detection with Task Loss-Guided Lp Metric
PTQ optimizes the quantization parameters by different metrics to minimize the perturbation of quantization.
RPTQ: Reorder-based Post-training Quantization for Large Language Models
In this paper, we identify that the challenge in quantizing activations in LLMs arises from varying ranges across channels, rather than solely the presence of outliers.
Benchmarking the Reliability of Post-training Quantization: a Particular Focus on Worst-case Performance
Post-training quantization (PTQ) is a popular method for compressing deep neural networks (DNNs) without modifying their original architecture or training procedures.
PD-Quant: Post-Training Quantization based on Prediction Difference Metric
It determines the quantization parameters by using the information of differences between network prediction before and after quantization.
Post-training Quantization on Diffusion Models
These approaches define a forward diffusion process for transforming data into noise and a backward denoising process for sampling data from noise.
Latency-aware Spatial-wise Dynamic Networks
The latency prediction model can efficiently estimate the inference latency of dynamic networks by simultaneously considering algorithms, scheduling strategies, and hardware properties.
PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantization
We observe the distributions of activation values after softmax and GELU functions are quite different from the Gaussian distribution.
PTQ-SL: Exploring the Sub-layerwise Post-training Quantization
Network quantization is a powerful technique to compress convolutional neural networks.
ENAS4D: Efficient Multi-stage CNN Architecture Search for Dynamic Inference
The inference of a input sample can exit from early stage if the prediction of the stage is confident enough.
S2DNAS:Transforming Static CNN Model for Dynamic Inference via Neural Architecture Search
Recently, dynamic inference has emerged as a promising way to reduce the computational cost of deep convolutional neural network (CNN).
