Search Results for author:
Found 9 papers, 1 papers with code
SlimFit: Memory-Efficient Fine-Tuning of Transformer-based Models Using Training Dynamics
This allows SlimFit to freeze up to 95% of layers and reduce the overall on-device GPU memory usage of transformer-based models such as ViT and BERT by an average of 2. 2x, across different NLP and CV benchmarks/datasets such as GLUE, SQuAD 2. 0, CIFAR-10, CIFAR-100 and ImageNet with an average degradation of 0. 2% in accuracy.
Standard Deviation-Based Quantization for Deep Neural Networks
Quantization of deep neural networks is a promising approach that reduces the inference cost, making it feasible to run deep networks on resource-restricted devices.
Training Linear Finite-State Machines
Therefore, our FSM-based model can learn extremely long-term dependencies as it requires 1/l memory storage during training compared to LSTMs, where l is the number of time steps.
The Synthesis of XNOR Recurrent Neural Networks with Stochastic Logic
The emergence of XNOR networks seek to reduce the model size and computational cost of neural networks for their deployment on specialized hardware requiring real-time processes with limited hardware resources.
Learning to Skip Ineffectual Recurrent Computations in LSTMs
This observation suggests that a large fraction of the recurrent computations are ineffectual and can be avoided to speed up the process during the inference as they involve noncontributory multiplications/accumulations with zero-valued states.
Learning Recurrent Binary/Ternary Weights
On the software side, we evaluate the performance (in terms of accuracy) of our method using long short-term memories (LSTMs) on various sequential models including sequence classification and language modeling.
Multi-Mode Inference Engine for Convolutional Neural Networks
Their performance efficiency is limited to less than 55% on average, which leads to unnecessarily high processing latency and silicon area.
Hardware Architecture
Sparsely-Connected Neural Networks: Towards Efficient VLSI Implementation of Deep Neural Networks
The proposed architecture can save up to 90% of memory compared to the conventional implementations of fully-connected neural networks.
VLSI Implementation of Deep Neural Network Using Integral Stochastic Computing
We also synthesize the circuits in a 65 nm CMOS technology and we show that the proposed integral stochastic architecture results in up to 21% reduction in energy consumption compared to the binary radix implementation at the same misclassification rate.
