Search Results for author:
Found 28 papers, 17 papers with code
Efficient Multi-Task Large Model Training via Data Heterogeneity-aware Model Management
In this paper, we investigate how to achieve high-performance training of large-scale MT MM models through data heterogeneity-aware model management optimization.
PQCache: Product Quantization-based KVCache for Long Context LLM Inference
During the prefilling phase, we apply PQ to tokens' keys for each LLM layer and head.
GraphPipe: Improving Performance and Scalability of DNN Training with Graph Pipeline Parallelism
GraphPipe partitions a DNN into a graph of stages, optimizes micro-batch schedules for these stages, and parallelizes DNN training using the discovered GPP strategies.
Optimal Kernel Orchestration for Tensor Programs with Korch
Prior approaches optimize kernel orchestration by greedily applying operator fusion, which fuses the computation of multiple operators into a single kernel, and miss a variety of optimization opportunities in kernel orchestration.
Helix: Distributed Serving of Large Language Models via Max-Flow on Heterogeneous GPUs
This paper introduces Helix, a distributed system for high-throughput, low-latency large language model (LLM) serving on heterogeneous GPU clusters.
FlexLLM: A System for Co-Serving Large Language Model Inference and Parameter-Efficient Finetuning
This is because existing systems cannot handle workloads that include a mix of inference and PEFT finetuning requests.
Generative Dense Retrieval: Memory Can Be a Burden
Memorizing-free matching mechanism from Dense Retrieval (DR) is then introduced to conduct fine-grained intra-cluster matching from clusters to relevant documents.
Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models
Experiments show that QST can reduce the total memory footprint by up to 2. 3 $\times$ and speed up the finetuning process by up to 3 $\times$ while achieving competent performance compared with the state-of-the-art.
Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems
In the rapidly evolving landscape of artificial intelligence (AI), generative large language models (LLMs) stand at the forefront, revolutionizing how we interact with our data.
Experimental Analysis of Large-scale Learnable Vector Storage Compression
Learnable embedding vector is one of the most important applications in machine learning, and is widely used in various database-related domains.
SpotServe: Serving Generative Large Language Models on Preemptible Instances
This paper aims to reduce the monetary cost for serving LLMs by leveraging preemptible GPU instances on modern clouds, which offer accesses to spare GPUs at a much cheaper price than regular instances but may be preempted by the cloud at any time.
Model-enhanced Vector Index
We empirically show that our model achieves better performance on the commonly used academic benchmarks MSMARCO Passage and Natural Questions, with comparable serving latency to dense retrieval solutions.
Improving Automatic Parallel Training via Balanced Memory Workload Optimization
Transformer models have emerged as the leading approach for achieving state-of-the-art performance across various application domains, serving as the foundation for advanced large-scale deep learning (DL) models.
Accelerating Text-to-Image Editing via Cache-Enabled Sparse Diffusion Inference
The key intuition behind our approach is to utilize the semantic mapping between the minor modifications on the input text and the affected regions on the output image.
SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
Our evaluation shows that SpecInfer outperforms existing LLM serving systems by 1. 5-2. 8x for distributed LLM inference and by 2. 6-3. 5x for offloading-based LLM inference, while preserving the same generative performance.
FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement
We first present an empirical analysis on the problems and opportunities of training MoE models, which motivates us to overcome the routing imbalance and fluctuation problems by a dynamic expert management and device placement mechanism.
Angel-PTM: A Scalable and Economical Large-scale Pre-training System in Tencent
Recent years have witnessed the unprecedented achievements of large-scale pre-trained models, especially the Transformer models.
Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism
Transformer models have achieved state-of-the-art performance on various domains of applications and gradually becomes the foundations of the advanced large deep learning (DL) models.
Distributed Graph Neural Network Training: A Survey
In recent years, many efforts have been made on distributed GNN training, and an array of training algorithms and systems have been proposed.
CALIP: Zero-Shot Enhancement of CLIP with Parameter-free Attention
Contrastive Language-Image Pre-training (CLIP) has been shown to learn visual representations with great transferability, which achieves promising accuracy for zero-shot classification.
Ranked #4 on
Training-free 3D Point Cloud Classification
on ScanObjectNN
(using extra training data)
Training-free 3D Point Cloud Classification
Transfer Learning
+1
Towards Communication-efficient Vertical Federated Learning Training via Cache-enabled Local Updates
Vertical federated learning (VFL) is an emerging paradigm that allows different parties (e. g., organizations or enterprises) to collaboratively build machine learning models with privacy protection.
ZOOMER: Boosting Retrieval on Web-scale Graphs by Regions of Interest
We introduce ZOOMER, a system deployed at Taobao, the largest e-commerce platform in China, for training and serving GNN-based recommendations over web-scale graphs.
EvoMoE: An Evolutional Mixture-of-Experts Training Framework via Dense-To-Sparse Gate
Then it diversifies the experts and continues to train the MoE with a novel Dense-to-Sparse gate (DTS-Gate).
HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework
Embedding models have been an effective learning paradigm for high-dimensional data.
PointCLIP: Point Cloud Understanding by CLIP
On top of that, we design an inter-view adapter to better extract the global feature and adaptively fuse the few-shot knowledge learned from 3D into CLIP pre-trained in 2D.
Ranked #3 on
3D Open-Vocabulary Instance Segmentation
on STPLS3D
3D Open-Vocabulary Instance Segmentation
Few-Shot Learning
+7
ROD: Reception-aware Online Distillation for Sparse Graphs
Unfortunately, many real-world networks are sparse in terms of both edges and labels, leading to sub-optimal performance of GNNs.
Memory-aware framework for fast and scalable second-order random walk over billion-edge natural graphs
In this paper, to clearly compare the efficiency of various node sampling methods, we first design a cost model and propose two new node sampling methods: one follows the acceptance-rejection paradigm to achieve a better balance between memory and time cost, and the other is optimized for fast sampling the skewed probability distributions existed in natural graphs.
DeGNN: Characterizing and Improving Graph Neural Networks with Graph Decomposition
Despite the wide application of Graph Convolutional Network (GCN), one major limitation is that it does not benefit from the increasing depth and suffers from the oversmoothing problem.
