Search Results for author:
Found 18 papers, 5 papers with code
Automatic Pruning of Fine-tuning Datasets for Transformer-based Language Models
Our experiments on 5 downstream tasks and 2 language models show that, on average, fine-tuning on the winning ticket subsets results in a $0. 1 \%$ increase in the evaluation performance of the model.
Step-GRAND: A Low Latency Universal Soft-input Decoder
GRAND features both soft-input and hard-input variants that are well suited to efficient hardware implementations that can be characterized with achievable average and worst-case decoding latency.
SSS3D: Fast Neural Architecture Search For Efficient Three-Dimensional Semantic Segmentation
We present SSS3D, a fast multi-objective NAS framework designed to find computationally efficient 3D semantic scene segmentation networks.
FMAS: Fast Multi-Objective SuperNet Architecture Search for Semantic Segmentation
We present FMAS, a fast multi-objective neural architecture search framework for semantic segmentation.
BD-KD: Balancing the Divergences for Online Knowledge Distillation
We show that adaptively balancing between the reverse and forward divergences shifts the focus of the training strategy to the compact student network without limiting the teacher network's learning process.
Efficient Fine-Tuning of Compressed Language Models with Learners
We introduce Learner modules and priming, novel methods for fine-tuning that exploit the overparameterization of pre-trained language models to gain benefits in convergence speed and resource utilization.
Efficient Fine-Tuning of BERT Models on the Edge
FAR reduces fine-tuning time on the DistilBERT model and CoLA dataset by 30%, and time spent on memory operations by 47%.
Standard Deviation-Based Quantization for Deep Neural Networks
Quantization of deep neural networks is a promising approach that reduces the inference cost, making it feasible to run deep networks on resource-restricted devices.
Learning to Skip Ineffectual Recurrent Computations in LSTMs
This observation suggests that a large fraction of the recurrent computations are ineffectual and can be avoided to speed up the process during the inference as they involve noncontributory multiplications/accumulations with zero-valued states.
Learning from the Syndrome
In this paper, we introduce the syndrome loss, an alternative loss function for neural error-correcting decoders based on a relaxation of the syndrome.
Learning Recurrent Binary/Ternary Weights
On the software side, we evaluate the performance (in terms of accuracy) of our method using long short-term memories (LSTMs) on various sequential models including sequence classification and language modeling.
Multi-Mode Inference Engine for Convolutional Neural Networks
Their performance efficiency is limited to less than 55% on average, which leads to unnecessarily high processing latency and silicon area.
Hardware Architecture
Deep Learning Methods for Improved Decoding of Linear Codes
Furthermore, we demonstrate that the neural belief propagation decoder can be used to improve the performance, or alternatively reduce the computational complexity, of a close to optimal decoder of short BCH codes.
Neural Offset Min-Sum Decoding
After describing our method, we compare the performance of the two neural decoding algorithms and show that our method achieves error-correction performance within 0. 1 dB of the multiplicative approach and as much as 1 dB better than traditional belief propagation for the codes under consideration.
Neural Networks Designing Neural Networks: Multi-Objective Hyper-Parameter Optimization
The method is evaluated on the MNIST and CIFAR-10 image datasets, optimizing for both recognition accuracy and computational complexity.
Sparsely-Connected Neural Networks: Towards Efficient VLSI Implementation of Deep Neural Networks
The proposed architecture can save up to 90% of memory compared to the conventional implementations of fully-connected neural networks.
VLSI Implementation of Deep Neural Network Using Integral Stochastic Computing
We also synthesize the circuits in a 65 nm CMOS technology and we show that the proposed integral stochastic architecture results in up to 21% reduction in energy consumption compared to the binary radix implementation at the same misclassification rate.
Associative Memories Based on Multiple-Valued Sparse Clustered Networks
Associative memories are structures that store data patterns and retrieve them given partial inputs.
