Search Results for author:
Found 74 papers, 35 papers with code
Communication-Efficient Federated Learning with Sketching
A key insight in the design of FedSketchedSGD is that, because the Count Sketch is linear, momentum and error accumulation can both be carried out within the sketch.
LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset
Studying how people interact with large language models (LLMs) in real-world scenarios is increasingly important due to their widespread use in various applications.
Efficient Memory Management for Large Language Model Serving with PagedAttention
On top of it, we build vLLM, an LLM serving system that achieves (1) near-zero waste in KV cache memory and (2) flexible sharing of KV cache within and across requests to further reduce memory usage.
Judging LLM-as-a-judge with MT-Bench and Chatbot Arena
Evaluating large language model (LLM) based chat assistants is challenging due to their broad capabilities and the inadequacy of existing benchmarks in measuring human preferences.
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU
As a result, when running OPT-175B on a single 16GB GPU, FlexGen achieves significantly higher throughput compared to state-of-the-art offloading systems, reaching a generation throughput of 1 token/s for the first time with an effective batch size of 144.
AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving
Model parallelism is conventionally viewed as a method to scale a single large deep learning model beyond the memory limits of a single device.
Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks
Recent advances in instruction-following large language models (LLMs) have led to dramatic improvements in a range of NLP tasks.
On Optimizing the Communication of Model Parallelism
This pattern emerges when the two paradigms of model parallelism - intra-operator and inter-operator parallelism - are combined to support large models on large clusters.
CLUTR: Curriculum Learning via Unsupervised Task Representation Learning
Recently, Unsupervised Environment Design (UED) emerged as a new paradigm for zero-shot generalization by simultaneously learning a task distribution and agent policies on the generated tasks.
Scaling up Trustless DNN Inference with Zero-Knowledge Proofs
In this work, we present the first practical ImageNet-scale method to verify ML model inference non-interactively, i. e., after the inference has been done.
Context-Aware Streaming Perception in Dynamic Environments
These works evaluate accuracy offline, one image at a time.
POET: Training Neural Networks on Tiny Devices with Integrated Rematerialization and Paging
We demonstrate that it is possible to fine-tune both ResNet-18 and BERT within the memory constraints of a Cortex-M class embedded device while outperforming current edge training methods in energy efficiency.
NumS: Scalable Array Programming for the Cloud
LSHS is a local search method which optimizes operator placement by minimizing maximum memory and network load on any given node within a distributed system.
Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning
Existing model-parallel training systems either require users to manually create a parallelization plan or automatically generate one from a limited space of model parallelism configurations.
Representing Long-Range Context for Graph Neural Networks with Global Attention
Inspired by recent computer vision results that find position-invariant attention performant in learning long-range relationships, our method, which we call GraphTrans, applies a permutation-invariant Transformer module after a standard GNN module.
Balsa: Learning a Query Optimizer Without Expert Demonstrations
Query optimizers are a performance-critical component in every database system.
Composing MPC with LQR and Neural Network for Amortized Efficiency and Stable Control
The proposed scheme does not require pre-computation and can improve the amortized running time of the composed MPC with a well-trained neural network.
MESA: Offline Meta-RL for Safe Adaptation and Fault Tolerance
Safe exploration is critical for using reinforcement learning (RL) in risk-sensitive environments.
Grounded Graph Decoding Improves Compositional Generalization in Question Answering
Grounding enables the model to retain syntax information from the input in thereby significantly improving generalization over complex inputs.
Accelerating Quadratic Optimization with Reinforcement Learning
First-order methods for quadratic optimization such as OSQP are widely used for large-scale machine learning and embedded optimal control, where many related problems must be rapidly solved.
Scenic4RL: Programmatic Modeling and Generation of Reinforcement Learning Environments
To showcase the benefits, we interfaced SCENIC to an existing RTS environment Google Research Football(GRF) simulator and introduced a benchmark consisting of 32 realistic scenarios, encoded in SCENIC, to train RL agents and testing their generalization capabilities.
Learning Competitive Equilibria in Exchange Economies with Bandit Feedback
The allocations at a CE are Pareto efficient and fair.
PAC Best Arm Identification Under a Deadline
In this work, the decision-maker is given a deadline of $T$ rounds, where, on each round, it can adaptively choose which arms to pull and how many times to pull them; this distinguishes the number of decisions made (i. e., time or number of rounds) from the number of samples acquired (cost).
ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training
On all these tasks, ActNN compresses the activation to 2 bits on average, with negligible accuracy loss.
Solution: Matchmaker Paxos: A Reconfigurable Consensus Protocol
State machine replication protocols, like MultiPaxos and Raft, are at the heart of numerous distributed systems.
SoK: A Generalized Multi-Leader State Machine Replication Tutorial
MultiPaxos and Raft are the two most popular and widely deployed state machine replication protocols.
TeraPipe: Token-Level Pipeline Parallelism for Training Large-Scale Language Models
With this key idea, we design TeraPipe, a high-performance token-level pipeline parallel algorithm for synchronous model-parallel training of Transformer-based language models.
Scaling Replicated State Machines with Compartmentalization [Technical Report]
In this paper, we introduce compartmentalization, the first comprehensive technique to eliminate state machine replication bottlenecks.
Distributed, Parallel, and Cluster Computing
Ekya: Continuous Learning of Video Analytics Models on Edge Compute Servers
Compressed models that are deployed on the edge servers for inference suffer from data drift, where the live video data diverges from the training data.
Online Learning Demands in Max-min Fairness
We describe mechanisms for the allocation of a scarce resource among multiple users in a way that is efficient, fair, and strategy-proof, but when users do not know their resource requirements.
RLlib Flow: Distributed Reinforcement Learning is a Dataflow Problem
Researchers and practitioners in the field of reinforcement learning (RL) frequently leverage parallel computation, which has led to a plethora of new algorithms and systems in the last few years.
Resource Allocation in Multi-armed Bandit Exploration: Overcoming Sublinear Scaling with Adaptive Parallelism
Second, we present an algorithm for a fixed deadline setting, where we are given a time deadline and need to maximize the probability of finding the best arm.
FetchSGD: Communication-Efficient Federated Learning with Sketching
A key insight in the design of FetchSGD is that, because the Count Sketch is linear, momentum and error accumulation can both be carried out within the sketch.
Variable Skipping for Autoregressive Range Density Estimation
In this paper, we explore a technique, variable skipping, for accelerating range density estimation over deep autoregressive models.
Contrastive Code Representation Learning
Recent work learns contextual representations of source code by reconstructing tokens from their context.
Ranked #1 on
Method name prediction
on CodeSearchNet
NeuroCard: One Cardinality Estimator for All Tables
Query optimizers rely on accurate cardinality estimates to produce good execution plans.
Ansor : Generating High-Performance Tensor Programs for Deep Learning
Ansor can find high-performance programs that are outside the search space of existing state-of-the-art approaches.
ProTuner: Tuning Programs with Monte Carlo Tree Search
We explore applying the Monte Carlo Tree Search (MCTS) algorithm in a notoriously difficult task: tuning programs for high-performance deep learning and image processing.
VCG Mechanism Design with Unknown Agent Values under Stochastic Bandit Feedback
To that end, we first define three notions of regret for the welfare, the individual utilities of each agent and that of the mechanism.
AutoPhase: Juggling HLS Phase Orderings in Random Forests with Deep Reinforcement Learning
We compare the performance of AutoPhase to state-of-the-art algorithms that address the phase-ordering problem.
Hoplite: Efficient and Fault-Tolerant Collective Communication for Task-Based Distributed Systems
Task-based distributed frameworks (e. g., Ray, Dask, Hydro) have become increasingly popular for distributed applications that contain asynchronous and dynamic workloads, including asynchronous gradient descent, reinforcement learning, and model serving.
HyperSched: Dynamic Resource Reallocation for Model Development on a Deadline
Prior research in resource scheduling for machine learning training workloads has largely focused on minimizing job completion times.
Hierarchical Variational Imitation Learning of Control Programs
Autonomous agents can learn by imitating teacher demonstrations of the intended behavior.
IMPACT: Importance Weighted Asynchronous Architectures with Clipped Target Networks
To address this, we propose a new distributed reinforcement learning algorithm, IMPACT.
Gemmini: Enabling Systematic Deep-Learning Architecture Evaluation via Full-Stack Integration
DNN accelerators are often developed and evaluated in isolation without considering the cross-stack, system-level effects in real-world environments.
Blink: Fast and Generic Collectives for Distributed ML
Model parameter synchronization across GPUs introduces high overheads for data-parallel training at scale.
Checkmate: Breaking the Memory Wall with Optimal Tensor Rematerialization
We formalize the problem of trading-off DNN training time and memory requirements as the tensor rematerialization optimization problem, a generalization of prior checkpointing strategies.
NeuroVectorizer: End-to-End Vectorization with Deep Reinforcement Learning
However, these models are unable to capture the data dependency, the computation graph, or the organization of instructions.
Distributed, Parallel, and Cluster Computing Performance Programming Languages
A View on Deep Reinforcement Learning in System Optimization
We propose a set of essential metrics to guide future works in evaluating the efficacy of using deep reinforcement learning in system optimization.
Helen: Maliciously Secure Coopetitive Learning for Linear Models
Many organizations wish to collaboratively train machine learning models on their combined datasets for a common benefit (e. g., better medical research, or fraud detection).
Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules
A key challenge in leveraging data augmentation for neural network training is choosing an effective augmentation policy from a large search space of candidate operations.
Ranked #5 on
Image Classification
on SVHN
Deep Unsupervised Cardinality Estimation
To produce a truly usable estimator, we develop a Monte Carlo integration scheme on top of autoregressive models that can efficiently handle range queries with dozens of dimensions or more.
Harmonia: Near-Linear Scalability for Replicated Storage with In-Network Conflict Detection
Experimental results show that Harmonia improves the throughput of these protocols by up to 10X for a replication factor of 10, providing near-linear scalability up to the limit of our testbed.
Distributed, Parallel, and Cluster Computing
MLSys: The New Frontier of Machine Learning Systems
Machine learning (ML) techniques are enjoying rapidly increasing adoption.
Communication-efficient distributed SGD with Sketching
Large-scale distributed training of neural networks is often limited by network bandwidth, wherein the communication time overwhelms the local computation time.
Constrained Thompson Sampling for Wireless Link Optimization
We model rate selection as a stochastic multi-armed bandit (MAB) problem, where a finite set of transmission rates are modeled as independent bandit arms.
Neural Packet Classification
First, many of the existing solutions are iteratively building a decision tree by splitting nodes in the tree.
Cloud Programming Simplified: A Berkeley View on Serverless Computing
Serverless cloud computing handles virtually all the system administration operations needed to make it easier for programmers to use the cloud.
Operating Systems
The OoO VLIW JIT Compiler for GPU Inference
Current trends in Machine Learning~(ML) inference on hardware accelerated devices (e. g., GPUs, TPUs) point to alarmingly low utilization.
AutoPhase: Compiler Phase-Ordering for High Level Synthesis with Deep Reinforcement Learning
We implement a framework in the context of the LLVM compiler to optimize the ordering for HLS programs and compare the performance of deep reinforcement learning to state-of-the-art algorithms that address the phase-ordering problem.
InferLine: ML Inference Pipeline Composition Framework
The dominant cost in production machine learning workloads is not training individual models but serving predictions from increasingly complex prediction pipelines spanning multiple models, machine learning frameworks, and parallel hardware accelerators.
Distributed, Parallel, and Cluster Computing
Learning to Optimize Join Queries With Deep Reinforcement Learning
Exhaustive enumeration of all possible join orders is often avoided, and most optimizers leverage heuristics to prune the search space.
Databases
Tune: A Research Platform for Distributed Model Selection and Training
We show that this interface meets the requirements for a broad range of hyperparameter search algorithms, allows straightforward scaling of search to large clusters, and simplifies algorithm implementation.
Model-Based Value Estimation for Efficient Model-Free Reinforcement Learning
By enabling wider use of learned dynamics models within a model-free reinforcement learning algorithm, we improve value estimation, which, in turn, reduces the sample complexity of learning.
Parametrized Hierarchical Procedures for Neural Programming
Neural programs are highly accurate and structured policies that perform algorithmic tasks by controlling the behavior of a computation mechanism.
RLlib: Abstractions for Distributed Reinforcement Learning
Reinforcement learning (RL) algorithms involve the deep nesting of highly irregular computation patterns, each of which typically exhibits opportunities for distributed computation.
Ray: A Distributed Framework for Emerging AI Applications
To meet the performance requirements, Ray employs a distributed scheduler and a distributed and fault-tolerant store to manage the system's control state.
A Berkeley View of Systems Challenges for AI
With the increasing commoditization of computer vision, speech recognition and machine translation systems and the widespread deployment of learning-based back-end technologies such as digital advertising and intelligent infrastructures, AI (Artificial Intelligence) has moved from research labs to production.
Multi-Level Discovery of Deep Options
Augmenting an agent's control with useful higher-level behaviors called options can greatly reduce the sample complexity of reinforcement learning, but manually designing options is infeasible in high-dimensional and abstract state spaces.
Real-Time Machine Learning: The Missing Pieces
Machine learning applications are increasingly deployed not only to serve predictions using static models, but also as tightly-integrated components of feedback loops involving dynamic, real-time decision making.
Clipper: A Low-Latency Online Prediction Serving System
In this paper, we introduce Clipper, a general-purpose low-latency prediction serving system.
Fast and Accurate Performance Analysis of LTE Radio Access Networks
Our choice of this domain is influenced by its commonalities with several other domains that produce real-time data, our access to a large live dataset, and their real-time nature and dimensionality which makes it a natural fit for a popular analysis technique, machine learning (ML).
SparkNet: Training Deep Networks in Spark
We introduce SparkNet, a framework for training deep networks in Spark.
Highly Available Transactions: Virtues and Limitations (Extended Version)
To minimize network latency and remain online during server failures and network partitions, many modern distributed data storage systems eschew transactional functionality, which provides strong semantic guarantees for groups of multiple operations over multiple data items.
Databases
