Search Results for author:
Found 17 papers, 10 papers with code
TeleRAG: Efficient Retrieval-Augmented Generation Inference with Lookahead Retrieval
To address these challenges, we propose TeleRAG, an efficient inference system that reduces RAG latency with minimal GPU memory requirements.
FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving
We present FlashInfer: a customizable and efficient attention engine for LLM serving.
MagicPIG: LSH Sampling for Efficient LLM Generation
Large language models (LLMs) with long context windows have gained significant attention.
Improving Image De-raining Using Reference-Guided Transformers
Image de-raining is a critical task in computer vision to improve visibility and enhance the robustness of outdoor vision systems.
vMCU: Coordinated Memory Management and Kernel Optimization for DNN Inference on MCUs
The key idea is to virtualize the limited memory of MCU as a large memory pool.
Relax: Composable Abstractions for End-to-End Dynamic Machine Learning
Dynamic shape computations have become critical in modern machine learning workloads, especially in emerging large language models.
Atom: Low-bit Quantization for Efficient and Accurate LLM Serving
To maximize LLMs' serving throughput, we introduce Atom, a low-bit quantization method that achieves high throughput improvements with negligible accuracy loss.
Punica: Multi-Tenant LoRA Serving
Our scheduler consolidates multi-tenant LoRA serving workloads in a shared GPU cluster.
SparseTIR: Composable Abstractions for Sparse Compilation in Deep Learning
We propose SparseTIR, a sparse tensor compilation abstraction that offers composable formats and composable transformations for deep learning workloads.
TensorIR: An Abstraction for Automatic Tensorized Program Optimization
Finally, we build an end-to-end framework on top of our abstraction to automatically optimize deep learning models for given tensor computation primitives.
Kokoyi: Executable LaTeX for End-to-end Deep Learning
Despite substantial efforts from the deep learning system community to relieve researchers and practitioners from the burden of implementing models with ever-growing complexity, a considerable lingual gap remains between developing models in the language of mathematics and implementing them in the languages of computer.
FeatGraph: A Flexible and Efficient Backend for Graph Neural Network Systems
FeatGraph provides a flexible programming interface to express diverse GNN models by composing coarse-grained sparse templates with fine-grained user-defined functions (UDFs) on each vertex/edge.
DGL-KE: Training Knowledge Graph Embeddings at Scale
Experiments on knowledge graphs consisting of over 86M nodes and 338M edges show that DGL-KE can compute embeddings in 100 minutes on an EC2 instance with 8 GPUs and 30 minutes on an EC2 cluster with 4 machines with 48 cores/machine.
Distributed, Parallel, and Cluster Computing
Transformer on a Diet
Transformer has been widely used thanks to its ability to capture sequence information in an efficient way.
BP-Transformer: Modelling Long-Range Context via Binary Partitioning
The Transformer model is widely successful on many natural language processing tasks.
Ranked #1 on
Machine Translation
on IWSLT2015 Chinese-English
Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks
Advancing research in the emerging field of deep graph learning requires new tools to support tensor computation over graphs.
Ranked #34 on
Node Classification
on Cora
On-line Dialogue Policy Learning with Companion Teaching
On-line dialogue policy learning is the key for building evolvable conversational agent in real world scenarios.
