Federated Knowledge Transfer Fine-tuning Large Server Model with Resource-Constrained IoT Clients

no code implementations • 7 Jul 2024 • Shaoyuan Chen, Linlin You, Rui Liu, Shuo Yu, Ahmed M. Abdelmoniem

Compared to the solutions based on centralized data centers, updating large models in the Internet of Things (IoT) faces challenges in coordinating knowledge from distributed clients by using their private and heterogeneous data.

Federated Learning Knowledge Distillation +2

Efficient Heterogeneous Large Language Model Decoding with Model-Attention Disaggregation

no code implementations • 3 May 2024 • Shaoyuan Chen, Wencong Xiao, Yutong Lin, Mingxing Zhang, Yingdi Shan, Jinlei Jiang, Kang Chen, Yongwei Wu

To enhance the efficiency of LLM decoding, we introduce model-attention disaggregation.

Language Modeling Language Modelling +2

Joint Transceiver Design Based on Dictionary Learning Algorithm for SCMA

no code implementations • 30 Oct 2020 • Shanshan Zhang, Wen Chen, Shaoyuan Chen

With the explosively increasing demands on the network capacity, throughput and number of connected wireless devices, massive connectivity is an urgent problem for the next generation wireless communications.

Dictionary Learning Scheduling