Search Results for author:
Found 8 papers, 4 papers with code
Dynamic DNNs and Runtime Management for Efficient Inference on Mobile/Embedded Devices
In this thesis, we proposed a combined method, a system was developed for DNN performance trade-off management, combining the runtime trade-off opportunities in both algorithms and hardware to meet dynamically changing application performance targets and hardware constraints in real time.
Fluid Dynamic DNNs for Reliable and Adaptive Distributed Inference on Edge Devices
Distributed inference is a popular approach for efficient DNN inference at the edge.
GhostShiftAddNet: More Features from Energy-Efficient Operations
When compared to GhostNet, inference latency on the Jetson Nano is improved by 1. 3x and 2x on the GPU and CPU respectively.
Dynamic Transformer for Efficient Machine Translation on Embedded Devices
The new reduced design space results in a BLEU score increase of approximately 1% for sub-optimal models from the original design space, with a wide range for performance scaling between 0. 356s - 1. 526s for the GPU and 2. 9s - 7. 31s for the CPU.
Temporal Early Exits for Efficient Video Object Detection
In this paper, we propose temporal early exits to reduce the computational complexity of per-frame video object detection.
Optimising Resource Management for Embedded Machine Learning
Machine learning inference is increasingly being executed locally on mobile and embedded platforms, due to the clear advantages in latency, privacy and connectivity.
Incremental Training and Group Convolution Pruning for Runtime DNN Performance Scaling on Heterogeneous Embedded Platforms
Compared to the existing works, our approach can provide up to 2. 36x (energy) and 2. 73x (time) wider dynamic range with a 2. 4x smaller memory footprint at the same compression rate.
Dynamic-OFA: Runtime DNN Architecture Switching for Performance Scaling on Heterogeneous Embedded Platforms
However, the training process of such dynamic DNNs can be costly, since platform-aware models of different deployment scenarios must be retrained to become dynamic.
