Search Results for author:
Found 14 papers, 4 papers with code
Mixture of Scales: Memory-Efficient Token-Adaptive Binarization for Large Language Models
Binarization, which converts weight parameters to binary values, has emerged as an effective strategy to reduce the size of large language models (LLMs).
SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks
In this paper, we introduce SLEB, a novel approach designed to streamline LLMs by eliminating redundant transformer blocks.
L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Models
This has led to active research on quantization-aware PEFT techniques, which aim to create models with high accuracy and low memory overhead.
Squeezing Large-Scale Diffusion Models for Mobile
The emergence of diffusion models has greatly broadened the scope of high-fidelity image synthesis, resulting in notable advancements in both practical implementation and academic research.
INSTA-BNN: Binary Neural Network with INSTAnce-aware Threshold
Binary Neural Networks (BNNs) have emerged as a promising solution for reducing the memory footprint and compute costs of deep neural networks, but they suffer from quality degradation due to the lack of freedom as activations and weights are constrained to the binary values.
Improving Accuracy of Binary Neural Networks using Unbalanced Activation Distribution
We also show that adjusting the threshold values of binary activation functions results in the unbalanced distribution of the binary activation, which increases the accuracy of BNN models.
Unifying Activation- and Timing-based Learning Rules for Spiking Neural Networks
For the gradient computation across the time domain in Spiking Neural Networks (SNNs) training, two different approaches have been independently studied.
BinaryDuo: Reducing Gradient Mismatch in Binary Activation Network by Coupling Binary Activations
Binary Neural Networks (BNNs) have been garnering interest thanks to their compute cost reduction and memory savings.
Zero-shifting Technique for Deep Neural Network Training on Resistive Cross-point Arrays
By using this zero-shifting method, we show that network performance dramatically improves for imbalanced synapse devices.
Double Viterbi: Weight Encoding for High Compression Ratio and Fast On-Chip Reconstruction for Deep Neural Network
In this paper, we propose a new sparse matrix format in order to enable a highly parallel decoding process of the entire sparse matrix.
BitSplit-Net: Multi-bit Deep Neural Network with Bitwise Activation Function
We demonstrate that the BitSplit version of LeNet-5, VGG-9, AlexNet, and ResNet-18 can be trained to have similar classification accuracy at a lower computational cost compared to conventional multi-bit networks with low bit precision (<= 4-bit).
Neural Network-Hardware Co-design for Scalable RRAM-based BNN Accelerators
Recently, RRAM-based Binary Neural Network (BNN) hardware has been gaining interests as it requires 1-bit sense-amp only and eliminates the need for high-resolution ADC and DAC.
Viterbi-based Pruning for Sparse Matrix with Fixed and High Index Compression Ratio
Hence, pruning is usually restricted to inference with a batch size of one, for which an efficient parallel matrix-vector multiplication method exists.
Deep Neural Network Optimized to Resistive Memory with Nonlinear Current-Voltage Characteristics
Artificial Neural Network computation relies on intensive vector-matrix multiplications.
