Search Results for author:
Found 28 papers, 6 papers with code
HeterMoE: Efficient Training of Mixture-of-Experts Models on Heterogeneous GPUs
However, existing solutions are agnostic to the performance characteristics of different MoE model components (i. e., attention and expert) and do not fully utilize each GPU's compute capability.
Lazarus: Resilient and Elastic Training of Mixture-of-Experts Models with Adaptive Expert Placement
The cost of even a single failure is significant, as all GPUs need to wait idle until the failure is resolved, potentially losing considerable training progress as training has to restart from checkpoints.
VcLLM: Video Codecs are Secretly Tensor Codecs
As the parameter size of large language models (LLMs) continues to expand, the need for a large memory footprint and high communication bandwidth have become significant bottlenecks for the training and inference of LLMs.
Conveyor: Efficient Tool-aware LLM Serving with Tool Partial Execution
In this paper, we identify a new opportunity for efficient LLM serving for requests that trigger tools: tool partial execution alongside LLM decoding.
Adaptive Skeleton Graph Decoding
Large language models (LLMs) have seen significant adoption for natural language tasks, owing their success to massive numbers of model parameters (e. g., 70B+); however, LLM inference incurs significant computation and memory costs.
Computing in the Era of Large Generative Models: From Cloud-Native to AI-Native
In this paper, we investigate the intersection of large generative AI models and cloud-native computing architectures.
Curator: Efficient Indexing for Multi-Tenant Vector Databases
Vector databases have emerged as key enablers for bridging intelligent applications with unstructured data, providing generic search and management support for embedding vectors extracted from the raw unstructured data.
Fairness in Serving Large Language Models
High-demand LLM inference services (e. g., ChatGPT and BARD) support a wide range of requests from short chat conversations to long document reading.
Punica: Multi-Tenant LoRA Serving
Our scheduler consolidates multi-tenant LoRA serving workloads in a shared GPU cluster.
Symphony: Optimized DNN Model Serving using Deferred Batch Scheduling
However, existing model serving systems cannot achieve adequate batch sizes while meeting latency objectives as these systems eagerly dispatch requests to accelerators to minimize the accelerator idle time.
Query Complexity of Active Learning for Function Family With Nearly Orthogonal Basis
Many machine learning algorithms require large numbers of labeled data to deliver state-of-the-art results.
Adaptive and Dynamic Multi-Resolution Hashing for Pairwise Summations
In this paper, we propose Adam-Hash: an adaptive and dynamic multi-resolution hashing data-structure for fast pairwise summation estimation.
A Faster $k$-means++ Algorithm
We propose a new algorithm \textsc{FastKmeans++} that only takes in $\widetilde{O}(nd + nk^2)$ time, in total.
Training Overparametrized Neural Networks in Sublinear Time
The success of deep learning comes at a tremendous computational and energy cost, and the scalability of training massively overparametrized neural networks is becoming a real barrier to the progress of artificial intelligence (AI).
Dynamic Maintenance of Kernel Density Estimation Data Structure: From Practice to Theory
In this work, we focus on the dynamic maintenance of KDE data structures with robustness to adversarial queries.
Sublinear Time Algorithm for Online Weighted Bipartite Matching
Online bipartite matching is a fundamental problem in online algorithms.
Serving and Optimizing Machine Learning Workflows on Heterogeneous Infrastructures
With the advent of ubiquitous deployment of smart devices and the Internet of Things, data sources for machine learning inference have increasingly moved to the edge of the network.
Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning
Existing model-parallel training systems either require users to manually create a parallelization plan or automatically generate one from a limited space of model parallelism configurations.
Fast Graph Neural Tangent Kernel via Kronecker Sketching
Given a kernel matrix of $n$ graphs, using sketching in solving kernel regression can reduce the running time to $o(n^3)$.
InstaHide’s Sample Complexity When Mixing Two Private Images
Inspired by InstaHide challenge [Huang, Song, Li and Arora'20], [Chen, Song and Zhuo'20] recently provides one mathematical formulation of InstaHide attack problem under Gaussian images distribution.
Sample Complexity of Deep Active Learning
In this paper, we present the first deep active learning algorithm which has a provable sample complexity.
TeraPipe: Token-Level Pipeline Parallelism for Training Large-Scale Language Models
With this key idea, we design TeraPipe, a high-performance token-level pipeline parallel algorithm for synchronous model-parallel training of Transformer-based language models.
What Can Phase Retrieval Tell Us About Private Distributed Learning?
In this work, we examine the security of InstaHide, a scheme recently proposed by \cite{hsla20} for preserving the security of private datasets in the context of distributed learning.
Graph Neural Network Acceleration via Matrix Dimension Reduction
Theoretically, we present two techniques to speed up GNTK training while preserving the generalization error: (1) We use a novel matrix decoupling method to reduce matrix dimensions during the kernel solving.
InstaHide's Sample Complexity When Mixing Two Private Images
On the current InstaHide challenge setup, where each InstaHide image is a mixture of two private images, we present a new algorithm to recover all the private images with a provable guarantee and optimal sample complexity.
On InstaHide, Phase Retrieval, and Sparse Matrix Factorization
In this work, we examine the security of InstaHide, a scheme recently proposed by [Huang, Song, Li and Arora, ICML'20] for preserving the security of private datasets in the context of distributed learning.
Ansor: Generating High-Performance Tensor Programs for Deep Learning
Ansor can find high-performance programs that are outside the search space of existing state-of-the-art approaches.
Hoplite: Efficient and Fault-Tolerant Collective Communication for Task-Based Distributed Systems
Task-based distributed frameworks (e. g., Ray, Dask, Hydro) have become increasingly popular for distributed applications that contain asynchronous and dynamic workloads, including asynchronous gradient descent, reinforcement learning, and model serving.
