SqueezeAttention: 2D Management of KV-Cache in LLM Inference via Layer-wise Optimal Budget

1 code implementation • 7 Apr 2024 • ZiHao Wang, Bin Cui, Shaoduo Gan

In this work, we found that by identifying the importance of attention layers, we could optimize the KV-cache jointly from two dimensions, i. e., sequence-wise and layer-wise.

Language Modelling Large Language Model +1

Few-shot Named Entity Recognition with Entity-level Prototypical Network Enhanced by Dispersedly Distributed Prototypes

no code implementations • COLING 2022 • Bin Ji, Shasha Li, Shaoduo Gan, Jie Yu, Jun Ma, Huijun Liu

Few-shot named entity recognition (NER) enables us to build a NER system for a new domain using very few labeled examples.

named-entity-recognition Named Entity Recognition +1

Stochastic Gradient Descent without Full Data Shuffle

1 code implementation • 12 Jun 2022 • Lijie Xu, Shuang Qiu, Binhang Yuan, Jiawei Jiang, Cedric Renggli, Shaoduo Gan, Kaan Kara, Guoliang Li, Ji Liu, Wentao Wu, Jieping Ye, Ce Zhang

In this paper, we first conduct a systematic empirical study on existing data shuffling strategies, which reveals that all existing strategies have room for improvement -- they all suffer in terms of I/O performance or convergence rate.

Computational Efficiency

BAGUA: Scaling up Distributed Learning with System Relaxations

1 code implementation • 3 Jul 2021 • Shaoduo Gan, Xiangru Lian, Rui Wang, Jianbin Chang, Chengjun Liu, Hongmei Shi, Shengzhuo Zhang, Xianghong Li, Tengxu Sun, Jiawei Jiang, Binhang Yuan, Sen yang, Ji Liu, Ce Zhang

Recent years have witnessed a growing list of systems for distributed data-parallel training.

Distributed Optimization Quantization

Towards Demystifying Serverless Machine Learning Training

1 code implementation • 17 May 2021 • Jiawei Jiang, Shaoduo Gan, Yue Liu, Fanlin Wang, Gustavo Alonso, Ana Klimovic, Ankit Singla, Wentao Wu, Ce Zhang

The appeal of serverless (FaaS) has triggered a growing interest on how to use it in data-intensive applications such as ETL, query processing, or machine learning (ML).

BIG-bench Machine Learning

1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed

2 code implementations • 4 Feb 2021 • Hanlin Tang, Shaoduo Gan, Ammar Ahmad Awan, Samyam Rajbhandari, Conglong Li, Xiangru Lian, Ji Liu, Ce Zhang, Yuxiong He

One of the most effective methods is error-compensated compression, which offers robust convergence speed even under 1-bit compression.

Scaling Unsupervised Domain Adaptation through Optimal Collaborator Selection and Lazy Discriminator Synchronization

no code implementations • 1 Jan 2021 • Akhil Mathur, Shaoduo Gan, Anton Isopoussu, Fahim Kawsar, Nadia Berthouze, Nicholas Donald Lane

Breakthroughs in unsupervised domain adaptation (uDA) have opened up the possibility of adapting models from a label-rich source domain to unlabeled target domains.

Privacy Preserving Unsupervised Domain Adaptation

APMSqueeze: A Communication Efficient Adam-Preconditioned Momentum SGD Algorithm

no code implementations • 26 Aug 2020 • Hanlin Tang, Shaoduo Gan, Samyam Rajbhandari, Xiangru Lian, Ji Liu, Yuxiong He, Ce Zhang

Adam is the important optimization algorithm to guarantee efficiency and accuracy for training many important tasks such as BERT and ImageNet.

Multi-Step Decentralized Domain Adaptation

no code implementations • 25 Sep 2019 • Akhil Mathur, Shaoduo Gan, Anton Isopoussu, Fahim Kawsar, Nadia Berthouze, Nicholas D. Lane

Despite the recent breakthroughs in unsupervised domain adaptation (uDA), no prior work has studied the challenges of applying these methods in practical machine learning scenarios.

Privacy Preserving Unsupervised Domain Adaptation

Communication Compression for Decentralized Training

no code implementations • NeurIPS 2018 • Hanlin Tang, Shaoduo Gan, Ce Zhang, Tong Zhang, Ji Liu

In this paper, We explore a natural question: {\em can the combination of both techniques lead to a system that is robust to both bandwidth and latency?}