Search Results for author:
Found 10 papers, 2 papers with code
Tackling the Dynamicity in a Production LLM Serving System with SOTA Optimizations via Hybrid Prefill/Decode/Verify Scheduling on Efficient Meta-kernels
Meeting growing demands for low latency and cost efficiency in production-grade large language model (LLM) serving systems requires integrating advanced optimization techniques.
P/D-Serve: Serving Disaggregated Large Language Model at Scale
To overcome previous problems, this paper proposes an end-to-end system P/D-Serve, complying with the paradigm of MLOps (machine learning operations), which models end-to-end (E2E) P/D performance and enables: 1) fine-grained P/D organization, mapping the service with RoCE (RDMA over converged ethernet) as needed, to facilitate similar processing and dynamic adjustments on P/D ratios; 2) on-demand forwarding upon rejections for idle prefill, decoupling the scheduler from regular inaccurate reports and local queues, to avoid timeouts in prefill; and 3) efficient KVCache transfer via optimized D2D access.
Unlocking Data-free Low-bit Quantization with Matrix Decomposition for KV Cache Compression
In this paper, we introduce \textbf{DecoQuant}, a novel data-free low-bit quantization technique based on tensor decomposition methods, to effectively compress KV cache.
Real-time Image Enhancer via Learnable Spatial-aware 3D Lookup Tables
One is a 1D weight vector used for image-level scenario adaptation, the other is a 3D weight map aimed for pixel-wise category fusion.
Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report
As the quality of mobile cameras starts to play a crucial role in modern smartphones, more and more attention is now being paid to ISP algorithms used to improve various perceptual aspects of mobile photos.
Learning Cascaded Siamese Networks for High Performance Visual Tracking
In order to achieve high performance visual tracking in various negative scenarios, a novel cascaded Siamese network is proposed and developed based on two different deep learning networks: a matching subnetwork and a classification subnetwork.
Efficient Multi-level Correlating for Visual Tracking
Correlation filter (CF) based tracking algorithms have demonstrated favorable performance recently.
High Performance Visual Tracking with Circular and Structural Operators
Based on the proposed circular and structural operators, a set of primal confidence score maps can be obtained by circular correlating feature maps with their corresponding structural correlation filters.
A Complementary Tracking Model with Multiple Features
Discriminative Correlation Filters based tracking algorithms exploiting conventional handcrafted features have achieved impressive results both in terms of accuracy and robustness.
Large Margin Structured Convolution Operator for Thermal Infrared Object Tracking
To the best of our knowledge, we are the first to incorporate the advantages of DCF and SOSVM for TIR object tracking.
