Search Results for author:
Found 14 papers, 4 papers with code
RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation
Although Low-Rank Adaptation (LoRA) is widely used and effective for fine-tuning, we have observed that its scaling factor can limit or even reduce performance as the rank size increases.
Quasar-ViT: Hardware-Oriented Quantization-Aware Architecture Search for Vision Transformers
This work proposes Quasar-ViT, a hardware-oriented quantization-aware architecture search framework for ViTs, to design efficient ViT models for hardware implementation while preserving the accuracy.
EdgeQAT: Entropy and Distribution Guided Quantization-Aware Training for the Acceleration of Lightweight LLMs on the Edge
In this paper, we propose EdgeQAT, the Entropy and Distribution Guided QAT for the optimization of lightweight LLMs to achieve inference acceleration on Edge devices.
Agile-Quant: Activation-Guided Quantization for Faster Inference of LLMs on the Edge
Recent works show that 8-bit or lower weight quantization is feasible with minimal impact on end-to-end task performance, while the activation is still not quantized.
SupeRBNN: Randomized Binary Neural Network Using Adiabatic Superconductor Josephson Devices
Specifically, we investigate the randomized behavior of the AQFP devices and analyze the impact of crossbar size on current attenuation, subsequently formulating the current amplitude into the values suitable for use in BNN computation.
Peeling the Onion: Hierarchical Reduction of Data Redundancy for Efficient Vision Transformer Training
Vision transformers (ViTs) have recently obtained success in many applications, but their intensive computation and heavy memory usage at both training and inference time limit their generalization.
HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformers
While vision transformers (ViTs) have continuously achieved new milestones in the field of computer vision, their sophisticated network architectures with high computation and memory costs have impeded their deployment on resource-limited edge devices.
Data Level Lottery Ticket Hypothesis for Vision Transformers
That is, there exists a subset of input image patches such that a ViT can be trained from scratch by using only this subset of patches and achieve similar accuracy to the ViTs trained by using all image patches.
Quantum Neural Network Compression
Model compression, such as pruning and quantization, has been widely applied to optimize neural networks on resource-limited classical devices.
SPViT: Enabling Faster Vision Transformers via Soft Token Pruning
Moreover, our framework can guarantee the identified model to meet resource specifications of mobile devices and FPGA, and even achieve the real-time execution of DeiT-T on mobile platforms.
Ranked #4 on
Efficient ViTs
on ImageNet-1K (with DeiT-S)
HFSP: A Hardware-friendly Soft Pruning Framework for Vision Transformers
Recently, Vision Transformer (ViT) has continuously established new milestones in the computer vision field, while the high computation and memory cost makes its propagation in industrial production difficult.
GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight Sparsity
It necessitates the sparse model inference via weight pruning, i. e., DNN weight sparsity, and it is desirable to design a new DNN weight sparsity scheme that can facilitate real-time inference on mobile devices while preserving a high sparse model accuracy.
RTMobile: Beyond Real-Time Mobile Acceleration of RNNs for Speech Recognition
Recurrent neural networks (RNNs) based automatic speech recognition has nowadays become prevalent on mobile devices such as smart phones.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
DARB: A Density-Aware Regular-Block Pruning for Deep Neural Networks
As a further optimization, we propose a density-adaptive regular-block (DARB) pruning that outperforms prior structured pruning work with high pruning ratio and decoding efficiency.
