Search Results for author: Hailin Zhang

Found 11 papers, 9 papers with code

Retrieval-Augmented Generation for AI-Generated Content: A Survey

1 code implementation29 Feb 2024 Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, Bin Cui

The development of Artificial Intelligence Generated Content (AIGC) has been facilitated by advancements in model algorithms, the increasing scale of foundation models, and the availability of ample high-quality datasets.

Information Retrieval Large Language Model +2

Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism

2 code implementations25 Nov 2022 Xupeng Miao, Yujie Wang, Youhe Jiang, Chunan Shi, Xiaonan Nie, Hailin Zhang, Bin Cui

Transformer models have achieved state-of-the-art performance on various domains of applications and gradually becomes the foundations of the advanced large deep learning (DL) models.

Knowledge Distillation with the Reused Teacher Classifier

1 code implementation CVPR 2022 Defang Chen, Jian-Ping Mei, Hailin Zhang, Can Wang, Yan Feng, Chun Chen

Knowledge distillation aims to compress a powerful yet cumbersome teacher model into a lightweight student model without much sacrifice of performance.

Knowledge Distillation

Confidence-Aware Multi-Teacher Knowledge Distillation

1 code implementation30 Dec 2021 Hailin Zhang, Defang Chen, Can Wang

Knowledge distillation is initially introduced to utilize additional supervision from a single teacher model for the student model training.

Knowledge Distillation Transfer Learning

Model-enhanced Vector Index

1 code implementation NeurIPS 2023 Hailin Zhang, Yujing Wang, Qi Chen, Ruiheng Chang, Ting Zhang, Ziming Miao, Yingyan Hou, Yang Ding, Xupeng Miao, Haonan Wang, Bochen Pang, Yuefeng Zhan, Hao Sun, Weiwei Deng, Qi Zhang, Fan Yang, Xing Xie, Mao Yang, Bin Cui

We empirically show that our model achieves better performance on the commonly used academic benchmarks MSMARCO Passage and Natural Questions, with comparable serving latency to dense retrieval solutions.

Natural Questions Quantization +1

Adaptive Multi-Teacher Knowledge Distillation with Meta-Learning

1 code implementation11 Jun 2023 Hailin Zhang, Defang Chen, Can Wang

Multi-Teacher knowledge distillation provides students with additional supervision from multiple pre-trained teachers with diverse information sources.

Knowledge Distillation Meta-Learning

CAFE: Towards Compact, Adaptive, and Fast Embedding for Large-scale Recommendation Models

1 code implementation6 Dec 2023 Hailin Zhang, Zirui Liu, Boxuan Chen, Yikai Zhao, Tong Zhao, Tong Yang, Bin Cui

Guided by our design philosophy, we further propose a multi-level hash embedding framework to optimize the embedding tables of non-hot features.

Feature Importance Philosophy

Experimental Analysis of Large-scale Learnable Vector Storage Compression

1 code implementation27 Nov 2023 Hailin Zhang, Penghao Zhao, Xupeng Miao, Yingxia Shao, Zirui Liu, Tong Yang, Bin Cui

Learnable embedding vector is one of the most important applications in machine learning, and is widely used in various database-related domains.

Benchmarking

Coexistence under hierarchical resource exploitation: the role of R*-preemption tradeoff

no code implementations22 Aug 2019 Man Qi, Niv DeMalach, Tao Sun, Hailin Zhang

Thus, we developed an extension of resource competition theory to investigate partial and total preemption (in the latter, the preemptor is unaffected by species with lower preemption rank).

Cannot find the paper you are looking for? You can Submit a new open access paper.