ALTO: An Efficient Network Orchestrator for Compound AI Systems

no code implementations • 7 Mar 2024 • Keshav Santhanam, Deepti Raghavan, Muhammad Shahir Rahman, Thejas Venkatesh, Neha Kunjal, Pratiksha Thaker, Philip Levis, Matei Zaharia

We present ALTO, a network orchestrator for efficiently serving compound AI systems such as pipelines of language models.

Chatbot Scheduling

Model Assertions for Monitoring and Improving ML Models

1 code implementation • 3 Mar 2020 • Daniel Kang, Deepti Raghavan, Peter Bailis, Matei Zaharia

We propose methods of using model assertions at all stages of ML system deployment, including runtime monitoring, validating labels, and continuously improving ML models.

Active Learning

Restructuring Endpoint Congestion Control

1 code implementation • SIGCOMM '18 2018 • Akshay Narayan, Frank Cangialosi, Deepti Raghavan, Prateesh Goyal Srinivas Narayana, Radhika Mittal, Mohammad Alizadeh, Hari Balakrishnan

Each datapath—such as the Linux kernel TCP, UDP-based QUIC, or kernel-bypass transports like mTCP-on-DPDK—summarizes information about packet round-trip times, receptions, losses, and ECN via a well-defined interface to algorithms running in the off-datapath Congestion Control Plane (CCP).