Search Results for author:
Found 6 papers, 2 papers with code
Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
However, batching multiple requests leads to an interleaving of prefill and decode iterations which makes it challenging to achieve both high throughput and low latency.
SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
SARATHI employs chunked-prefills, which splits a prefill request into equal sized chunks, and decode-maximal batching, which constructs a batch using a single prefill chunk and populates the remaining slots with decodes.
DynaQuant: Compressing Deep Learning Training Checkpoints via Dynamic Quantization
With the increase in the scale of Deep Learning (DL) training workloads in terms of compute resources and time consumption, the likelihood of encountering in-training failures rises substantially, leading to lost work and resource wastage.
Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads
At the heart of Singularity is a novel, workload-aware scheduler that can transparently preempt and elastically scale deep learning workloads to drive high utilization without impacting their correctness or performance, across a global fleet of AI accelerators (e. g., GPUs, FPGAs).
Learning Digital Circuits: A Journey Through Weight Invariant Self-Pruning Neural Networks
Recently, in the paper "Weight Agnostic Neural Networks" Gaier & Ha utilized architecture search to find networks where the topology completely encodes the knowledge.
Delog: A Privacy Preserving Log Filtering Framework for Online Compute Platforms
In many software applications, logs serve as the only interface between the application and the developer.
