Search Results for author:
Found 15 papers, 5 papers with code
Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformers
Evol-Q improves the top-1 accuracy of a fully quantized ViT-Base by $10. 30\%$, $0. 78\%$, and $0. 15\%$ for $3$-bit, $4$-bit, and $8$-bit weight quantization levels.
PerfSAGE: Generalized Inference Performance Predictor for Arbitrary Deep Learning Models on Edge Devices
The ability to accurately predict deep neural network (DNN) inference performance metrics, such as latency, power, and memory footprint, for an arbitrary DNN on a target hardware platform is essential to the design of DNN based models.
CPT-V: A Contrastive Approach to Post-Training Quantization of Vision Transformers
Borrowing the idea of contrastive loss from self-supervised learning, we find a robust way to jointly minimize a loss function using just 1, 000 calibration images.
Restructurable Activation Networks
To address this question, we propose a new paradigm called Restructurable Activation Networks (RANs) that manipulate the amount of non-linearity in models to improve their hardware-awareness and efficiency.
Super-Efficient Super Resolution for Fast Adversarial Defense at the Edge
Autonomous systems are highly vulnerable to a variety of adversarial attacks on Deep Neural Networks (DNNs).
MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers
To address this challenge, neural architecture search (NAS) promises to help design accurate ML models that meet the tight MCU memory, latency and energy constraints.
Ranked #1 on
Keyword Spotting
on Google Speech Commands V2 12
Rank and run-time aware compression of NLP Applications
We evaluate the impact of this technique on 5 NLP benchmarks across multiple tasks (Translation, Intent Detection, Language Modeling) and show that for similar accuracy values and compression factors, HMF can achieve more than 2. 32x faster inference run-time than pruning and 16. 77% better accuracy than LMF.
High Throughput Matrix-Matrix Multiplication between Asymmetric Bit-Width Operands
While existing SIMD matrix multiplication instructions for symmetric bit-width operands can support operands of mixed precision by zero- or sign-extending the narrow operand to match the size of the other operands, they cannot exploit the benefit of narrow bit-width of one of the operands.
BIG-bench Machine Learning
Vocal Bursts Intensity Prediction
The gem5 Simulator: Version 20.0+
The open-source and community-supported gem5 simulator is one of the most popular tools for computer architecture research.
Hardware Architecture
Ternary MobileNets via Per-Layer Hybrid Filter Banks
Using this proposed quantization method, we quantized a substantial portion of weight filters of MobileNets to ternary values resulting in 27. 98% savings in energy, and a 51. 07% reduction in the model size, while achieving comparable accuracy and no degradation in throughput on specialized hardware in comparison to the baseline full-precision MobileNets.
Pushing the limits of RNN Compression
This paper introduces a method to compress RNNs for resource constrained environments using Kronecker product (KP).
Run-Time Efficient RNN Compression for Inference on Edge Devices
Recurrent neural networks can be large and compute-intensive, yet many applications that benefit from RNNs run on small devices with very limited compute and storage capabilities while still having run-time constraints.
Compressing RNNs for IoT devices by 15-38x using Kronecker Products
Recurrent Neural Networks (RNN) can be difficult to deploy on resource constrained devices due to their size. As a result, there is a need for compression techniques that can significantly compress RNNs without negatively impacting task accuracy.
Ternary Hybrid Neural-Tree Networks for Highly Constrained IoT Applications
Machine learning-based applications are increasingly prevalent in IoT devices.
