Search Results for author:
Found 23 papers, 18 papers with code
Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
To address this issue, we introduce Chatbot Arena, an open platform for evaluating LLMs based on human preferences.
Efficiently Programming Large Language Models using SGLang
SGLang is designed for the efficient programming of LLMs and incorporates primitives for common LLM programming patterns.
Rethinking Benchmark and Contamination for Language Models with Rephrased Samples
Many have raised concerns about the trustworthiness of public benchmarks due to potential contamination in pre-training or fine-tuning datasets.
S-LoRA: Serving Thousands of Concurrent LoRA Adapters
To capitalize on these opportunities, we present S-LoRA, a system designed for the scalable serving of many LoRA adapters.
LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset
Studying how people interact with large language models (LLMs) in real-world scenarios is increasingly important due to their widespread use in various applications.
Efficient Memory Management for Large Language Model Serving with PagedAttention
On top of it, we build vLLM, an LLM serving system that achieves (1) near-zero waste in KV cache memory and (2) flexible sharing of KV cache within and across requests to further reduce memory usage.
H$_2$O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models
Based on these insights, we propose Heavy Hitter Oracle (H$_2$O), a KV cache eviction policy that dynamically retains a balance of recent and H$_2$ tokens.
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Evaluating large language model (LLM) based chat assistants is challenging due to their broad capabilities and the inadequacy of existing benchmarks in measuring human preferences.
Ranked #3 on
Long-Context Understanding
on Ada-LEval (TSort)
On Optimal Caching and Model Multiplexing for Large Model Inference
Theoretically, we provide an optimal algorithm for jointly optimizing both approaches to reduce the inference cost in both offline and online tabular settings.
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU
As a result, when running OPT-175B on a single 16GB GPU, FlexGen achieves significantly higher throughput compared to state-of-the-art offloading systems, reaching a generation throughput of 1 token/s for the first time with an effective batch size of 144.
AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving
Model parallelism is conventionally viewed as a method to scale a single large deep learning model beyond the memory limits of a single device.
On Optimizing the Communication of Model Parallelism
This pattern emerges when the two paradigms of model parallelism - intra-operator and inter-operator parallelism - are combined to support large models on large clusters.
TensorIR: An Abstraction for Automatic Tensorized Program Optimization
Finally, we build an end-to-end framework on top of our abstraction to automatically optimize deep learning models for given tensor computation primitives.
NumS: Scalable Array Programming for the Cloud
LSHS is a local search method which optimizes operator placement by minimizing maximum memory and network load on any given node within a distributed system.
GACT: Activation Compressed Training for Generic Network Architectures
Training large neural network (NN) models requires extensive memory resources, and Activation Compressed Training (ACT) is a promising approach to reduce training memory footprint.
Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning
Existing model-parallel training systems either require users to manually create a parallelization plan or automatically generate one from a limited space of model parallelism configurations.
ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training
On all these tasks, ActNN compresses the activation to 2 bits on average, with negligible accuracy loss.
Ansor: Generating High-Performance Tensor Programs for Deep Learning
Ansor can find high-performance programs that are outside the search space of existing state-of-the-art approaches.
A Hardware-Software Blueprint for Flexible Deep Learning Specialization
Specialized Deep Learning (DL) acceleration stacks, designed for a specific set of frameworks, model architectures, operators, and data types, offer the allure of high performance while sacrificing flexibility.
Learning to Optimize Tensor Programs
Efficient implementations of tensor operators, such as matrix multiplication and high dimensional convolution, are key enablers of effective deep learning systems.
TVM: An Automated End-to-End Optimizing Compiler for Deep Learning
Experimental results show that TVM delivers performance across hardware back-ends that are competitive with state-of-the-art, hand-tuned libraries for low-power CPU, mobile GPU, and server-class GPUs.
Size-to-depth: A New Perspective for Single Image Depth Estimation
To overcome these problems, we propose a new perspective for single monocular image depth estimation problem: size to depth.
MAgent: A Many-Agent Reinforcement Learning Platform for Artificial Collective Intelligence
Unlike previous research platforms on single or multi-agent reinforcement learning, MAgent focuses on supporting the tasks and the applications that require hundreds to millions of agents.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
