Search Results for author:
Found 5 papers, 2 papers with code
SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks
In this paper, we introduce SLEB, a novel approach designed to streamline LLMs by eliminating redundant transformer blocks.
L4Q: Parameter Efficient Quantization-Aware Training on Large Language Models via LoRA-wise LSQ
In resource-constrained scenarios, PTQ, with its reduced training overhead, is often preferred over QAT, despite the latter's potential for higher accuracy.
Squeezing Large-Scale Diffusion Models for Mobile
The emergence of diffusion models has greatly broadened the scope of high-fidelity image synthesis, resulting in notable advancements in both practical implementation and academic research.
BitSplit-Net: Multi-bit Deep Neural Network with Bitwise Activation Function
We demonstrate that the BitSplit version of LeNet-5, VGG-9, AlexNet, and ResNet-18 can be trained to have similar classification accuracy at a lower computational cost compared to conventional multi-bit networks with low bit precision (<= 4-bit).
Neural Network-Hardware Co-design for Scalable RRAM-based BNN Accelerators
Recently, RRAM-based Binary Neural Network (BNN) hardware has been gaining interests as it requires 1-bit sense-amp only and eliminates the need for high-resolution ADC and DAC.
