Search Results for author:
Found 23 papers, 8 papers with code
HadaNorm: Diffusion Transformer Quantization through Mean-Centered Transformations
Diffusion models represent the cutting edge in image generation, but their high memory and computational demands hinder deployment on resource-constrained devices.
Efficient LLM Inference using Dynamic Input Pruning and Cache-Aware Masking
Previous work has proposed to leverage natural dynamic activation sparsity in ReLU-activated LLMs to reduce effective DRAM bandwidth per token.
Mixture of Cache-Conditional Experts for Efficient Mobile Device Inference
Mixture of Experts (MoE) LLMs have recently gained attention for their ability to enhance performance by selectively engaging specialized subnetworks or "experts" for each input.
Rapid Switching and Multi-Adapter Fusion via Sparse High Rank Adapters
In this paper, we propose Sparse High Rank Adapters (SHiRA) that directly finetune 1-2% of the base model weights while leaving others unchanged, thus, resulting in a highly sparse adapter.
Sparse High Rank Adapters
However, from a mobile deployment standpoint, we can either avoid inference overhead in the fused mode but lose the ability to switch adapters rapidly, or suffer significant (up to 30% higher) inference latency while enabling rapid switching in the unfused mode.
Characterizing Soft-Error Resiliency in Arm's Ethos-U55 Embedded Machine Learning Accelerator
As Neural Processing Units (NPU) or accelerators are increasingly deployed in a variety of applications including safety critical applications such as autonomous vehicle, and medical imaging, it is critical to understand the fault-tolerance nature of the NPUs.
GPTVQ: The Blessing of Dimensionality for LLM Quantization
In this work we show that the size versus accuracy trade-off of neural network quantization can be significantly improved by increasing the quantization dimensionality.
Efficient Edge Inference by Selective Query
Training a hybrid learner is difficult since we lack annotations of hard edge-examples.
PerfSAGE: Generalized Inference Performance Predictor for Arbitrary Deep Learning Models on Edge Devices
The ability to accurately predict deep neural network (DNN) inference performance metrics, such as latency, power, and memory footprint, for an arbitrary DNN on a target hardware platform is essential to the design of DNN based models.
Thales: Formulating and Estimating Architectural Vulnerability Factors for DNN Accelerators
As Deep Neural Networks (DNNs) are increasingly deployed in safety critical and privacy sensitive applications such as autonomous driving and biometric authentication, it is critical to understand the fault-tolerance nature of DNNs.
Restructurable Activation Networks
To address this question, we propose a new paradigm called Restructurable Activation Networks (RANs) that manipulate the amount of non-linearity in models to improve their hardware-awareness and efficiency.
UDC: Unified DNAS for Compressible TinyML Models
Deploying TinyML models on low-cost IoT hardware is very challenging, due to limited device memory capacity.
Super-Efficient Super Resolution for Fast Adversarial Defense at the Edge
Autonomous systems are highly vulnerable to a variety of adversarial attacks on Deep Neural Networks (DNNs).
Hybrid Cloud-Edge Networks for Efficient Inference
The first network is a low-capacity network that can be deployed on an edge device, whereas the second is a high-capacity network deployed in the cloud.
AutoPilot: Automating SoC Design Space Exploration for SWaP Constrained Autonomous UAVs
Balancing a computing system for a UAV requires considering both the cyber (e. g., sensor rate, compute performance) and physical (e. g., payload weight) characteristics that affect overall performance.
Mesorasi: Architecture Support for Point Cloud Analytics via Delayed-Aggregation
Point cloud analytics is poised to become a key workload on battery-powered embedded and mobile platforms in a wide range of emerging application domains, such as autonomous driving, robotics, and augmented reality, where efficiency is paramount.
CHIPKIT: An agile, reusable open-source framework for rapid test chip development
The current trend for domain-specific architectures (DSAs) has led to renewed interest in research test chips to demonstrate new specialized hardware.
Hardware Architecture
SMAUG: End-to-End Full-Stack Simulation Infrastructure for Deep Learning Workloads
In recent years, there has been tremendous advances in hardware acceleration of deep neural networks.
ASV: Accelerated Stereo Vision System
The key to ASV is to exploit unique characteristics inherent to stereo vision, and apply stereo-specific optimizations, both algorithmically and computationally.
Energy Efficient Hardware for On-Device CNN Inference via Transfer Learning
On-device CNN inference for real-time computer vision applications can result in computational demands that far exceed the energy budgets of mobile devices.
SCALE-Sim: Systolic CNN Accelerator
Systolic Arrays are one of the most popular compute substrates within Deep Learning accelerators today, as they provide extremely high efficiency for running dense matrix multiplications.
Distributed, Parallel, and Cluster Computing Hardware Architecture
Euphrates: Algorithm-SoC Co-Design for Low-Power Mobile Continuous Vision
Specifically, we propose to expose the motion data that is naturally generated by the Image Signal Processor (ISP) early in the vision pipeline to the CNN engine.
Mobile Machine Learning Hardware at ARM: A Systems-on-Chip (SoC) Perspective
Machine learning is playing an increasingly significant role in emerging mobile application domains such as AR/VR, ADAS, etc.
