Search Results for author:
Found 17 papers, 9 papers with code
Model-enhanced Vector Index
We empirically show that our model achieves better performance on the commonly used academic benchmarks MSMARCO Passage and Natural Questions, with comparable serving latency to dense retrieval solutions.
Improving Automatic Parallel Training via Balanced Memory Workload Optimization
Transformer models have emerged as the leading approach for achieving state-of-the-art performance across various application domains, serving as the foundation for advanced large-scale deep learning (DL) models.
FISEdit: Accelerating Text-to-image Editing via Cache-enabled Sparse Diffusion Inference
The key intuition behind our approach is to utilize the semantic mapping between the minor modifications on the input text and the affected regions on the output image.
SpecInfer: Accelerating Generative Large Language Model Serving with Speculative Inference and Token Tree Verification
A key insight behind Specinfer is to combine various collectively boost-tuned small language models to jointly predict the LLM's outputs; the predictions are organized as a token tree, whose nodes each represent a candidate token sequence.
FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement
We first present an empirical analysis on the problems and opportunities of training MoE models, which motivates us to overcome the routing imbalance and fluctuation problems by a dynamic expert management and device placement mechanism.
Angel-PTM: A Scalable and Economical Large-scale Pre-training System in Tencent
Recent years have witnessed the unprecedented achievements of large-scale pre-trained models, especially the Transformer models.
Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism
Transformer models have achieved state-of-the-art performance on various domains of applications and gradually becomes the foundations of the advanced large deep learning (DL) models.
Distributed Graph Neural Network Training: A Survey
In recent years, many efforts have been made on distributed GNN training, and an array of training algorithms and systems have been proposed.
CALIP: Zero-Shot Enhancement of CLIP with Parameter-free Attention
Contrastive Language-Image Pre-training (CLIP) has been shown to learn visual representations with great transferability, which achieves promising accuracy for zero-shot classification.
Ranked #4 on
Training-free 3D Point Cloud Classification
on ScanObjectNN
(using extra training data)
Training-free 3D Point Cloud Classification
Transfer Learning
+1
Towards Communication-efficient Vertical Federated Learning Training via Cache-enabled Local Updates
Vertical federated learning (VFL) is an emerging paradigm that allows different parties (e. g., organizations or enterprises) to collaboratively build machine learning models with privacy protection.
ZOOMER: Boosting Retrieval on Web-scale Graphs by Regions of Interest
We introduce ZOOMER, a system deployed at Taobao, the largest e-commerce platform in China, for training and serving GNN-based recommendations over web-scale graphs.
EvoMoE: An Evolutional Mixture-of-Experts Training Framework via Dense-To-Sparse Gate
Then it diversifies the experts and continues to train the MoE with a novel Dense-to-Sparse gate (DTS-Gate).
HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework
Embedding models have been an effective learning paradigm for high-dimensional data.
PointCLIP: Point Cloud Understanding by CLIP
On top of that, we design an inter-view adapter to better extract the global feature and adaptively fuse the few-shot knowledge learned from 3D into CLIP pre-trained in 2D.
Ranked #3 on
Training-free 3D Part Segmentation
on ShapeNet-Part
(using extra training data)
ROD: Reception-aware Online Distillation for Sparse Graphs
Unfortunately, many real-world networks are sparse in terms of both edges and labels, leading to sub-optimal performance of GNNs.
Memory-aware framework for fast and scalable second-order random walk over billion-edge natural graphs
In this paper, to clearly compare the efficiency of various node sampling methods, we first design a cost model and propose two new node sampling methods: one follows the acceptance-rejection paradigm to achieve a better balance between memory and time cost, and the other is optimized for fast sampling the skewed probability distributions existed in natural graphs.
DeGNN: Characterizing and Improving Graph Neural Networks with Graph Decomposition
Despite the wide application of Graph Convolutional Network (GCN), one major limitation is that it does not benefit from the increasing depth and suffers from the oversmoothing problem.
