Search Results for author:
Found 19 papers, 7 papers with code
Efficient Pruning of Large Language Model with Adaptive Estimation Fusion
Large language models (LLMs) have become crucial for many generative downstream tasks, leading to an inevitable trend and significant challenge to deploy them efficiently on resource-constrained devices.
EdgeQAT: Entropy and Distribution Guided Quantization-Aware Training for the Acceleration of Lightweight LLMs on the Edge
In this paper, we propose EdgeQAT, the Entropy and Distribution Guided QAT for the optimization of lightweight LLMs to achieve inference acceleration on Edge devices.
Agile-Quant: Activation-Guided Quantization for Faster Inference of LLMs on the Edge
Recent works show that 8-bit or lower weight quantization is feasible with minimal impact on end-to-end task performance, while the activation is still not quantized.
GPU Accelerated Color Correction and Frame Warping for Real-time Video Stitching
Traditional image stitching focuses on a single panorama frame without considering the spatial-temporal consistency in videos.
You Need Multiple Exiting: Dynamic Early Exiting for Accelerating Unified Vision Language Model
To handle this challenge, we propose a novel early exiting strategy for unified visual language models, which allows dynamically skip the layers in encoder and decoder simultaneously in term of input layer-wise similarities with multiple times of early exiting, namely \textbf{MuE}.
Peeling the Onion: Hierarchical Reduction of Data Redundancy for Efficient Vision Transformer Training
Vision transformers (ViTs) have recently obtained success in many applications, but their intensive computation and heavy memory usage at both training and inference time limit their generalization.
HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformers
While vision transformers (ViTs) have continuously achieved new milestones in the field of computer vision, their sophisticated network architectures with high computation and memory costs have impeded their deployment on resource-limited edge devices.
Data Level Lottery Ticket Hypothesis for Vision Transformers
That is, there exists a subset of input image patches such that a ViT can be trained from scratch by using only this subset of patches and achieve similar accuracy to the ViTs trained by using all image patches.
Layer Freezing & Data Sieving: Missing Pieces of a Generic Framework for Sparse Training
Therefore, we analyze the feasibility and potentiality of using the layer freezing technique in sparse training and find it has the potential to save considerable training costs.
SPViT: Enabling Faster Vision Transformers via Soft Token Pruning
Moreover, our framework can guarantee the identified model to meet resource specifications of mobile devices and FPGA, and even achieve the real-time execution of DeiT-T on mobile platforms.
Ranked #4 on
Efficient ViTs
on ImageNet-1K (with DeiT-S)
MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge
Systematical evaluation on accuracy, training speed, and memory footprint are conducted, where the proposed MEST framework consistently outperforms representative SOTA works.
Accelerating Framework of Transformer by Hardware Design and Model Compression Co-Optimization
Our HP can achieve higher sparsity ratio and is more flexible than other sparsity pattern.
HFSP: A Hardware-friendly Soft Pruning Framework for Vision Transformers
Recently, Vision Transformer (ViT) has continuously established new milestones in the computer vision field, while the high computation and memory cost makes its propagation in industrial production difficult.
Improving DNN Fault Tolerance using Weight Pruning and Differential Crossbar Mapping for ReRAM-based Edge AI
More importantly, our method does not require extra hardware cost compared to the traditional two-column mapping scheme.
A Compression-Compilation Framework for On-mobile Real-time BERT Applications
In this paper, we propose a compression-compilation co-design framework that can guarantee the identified model to meet both resource and real-time specifications of mobile devices.
NPAS: A Compiler-aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration
With the increasing demand to efficiently deploy DNNs on mobile edge devices, it becomes much more important to reduce unnecessary computation and increase the execution speed.
Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruning
Pre-trained large-scale language models have increasingly demonstrated high accuracy on many natural language processing (NLP) tasks.
Real-Time Execution of Large-scale Language Models on Mobile
Our framework can guarantee the identified model to meet both resource and real-time specifications of mobile devices, thus achieving real-time execution of large transformer-based models like BERT variants.
SS-Auto: A Single-Shot, Automatic Structured Weight Pruning Framework of DNNs with Ultra-High Efficiency
Structured weight pruning is a representative model compression technique of DNNs for hardware efficiency and inference accelerations.
