Search Results for author:
Found 5 papers, 4 papers with code
Secure Transformer Inference
Our protocol, Secure Transformer Inference Protocol (STIP), can be applied to real-world services like ChatGPT.
PacketGame: Multi-Stream Packet Gating for Concurrent Video Inference at Scale
The resource efficiency of video analytics workloads is critical for large-scale deployments on edge nodes and cloud clusters.
MLink: Linking Black-Box Models from Multiple Domains for Collaborative Inference
The cost efficiency of model inference is critical to real-world machine learning (ML) applications, especially for delay-sensitive tasks and resource-limited devices.
InFi: End-to-End Learning to Filter Input for Resource-Efficiency in Mobile-Centric Inference
Previous efforts have tailored effective solutions for many applications, but left two essential questions unanswered: (1) theoretical filterability of an inference workload to guide the application of input filtering techniques, thereby avoiding the trial-and-error cost for resource-constrained mobile applications; (2) robust discriminability of feature embedding to allow input filtering to be widely effective for diverse inference tasks and input content.
Comprehensive and Efficient Data Labeling via Adaptive Model Scheduling
With limited computing resources and stringent delay, given a data stream and a collection of applicable resource-hungry deep-learning models, we design a novel approach to adaptively schedule a subset of these models to execute on each data item, aiming to maximize the value of the model output (e. g., the number of high-confidence labels).
