Search Results for author:
Found 6 papers, 2 papers with code
SPEED: Speculative Pipelined Execution for Efficient Decoding
For Transformer decoders that employ parameter sharing, the memory operations for the tokens executing in parallel can be amortized, which allows us to accelerate generative LLM inference.
MoCA: Memory-Centric, Adaptive Execution for Multi-Tenant Deep Neural Networks
Driven by the wide adoption of deep neural networks (DNNs) across different application domains, multi-tenancy execution, where multiple DNNs are deployed simultaneously on the same hardware, has been proposed to satisfy the latency requirements of different applications while improving the overall system utilization.
Full Stack Optimization of Transformer Inference: a Survey
In this work, we survey different approaches for efficient Transformer inference, including: (i) analysis and profiling of the bottlenecks in existing Transformer architectures and their similarities and differences with previous convolutional models; (ii) implications of Transformer architecture on hardware, including the impact of non-linear operations such as Layer Normalization, Softmax, and GELU, as well as linear operations, on hardware design; (iii) approaches for optimizing a fixed Transformer architecture; (iv) challenges in finding the right mapping and scheduling of operations for Transformer models; and (v) approaches for optimizing Transformer models by adapting the architecture using neural architecture search.
ProTuner: Tuning Programs with Monte Carlo Tree Search
We explore applying the Monte Carlo Tree Search (MCTS) algorithm in a notoriously difficult task: tuning programs for high-performance deep learning and image processing.
Gemmini: Enabling Systematic Deep-Learning Architecture Evaluation via Full-Stack Integration
DNN accelerators are often developed and evaluated in isolation without considering the cross-stack, system-level effects in real-world environments.
Domain Specific Approximation for Object Detection
There is growing interest in object detection in advanced driver assistance systems and autonomous robots and vehicles.
