Search Results for author:
Found 4 papers, 1 papers with code
Scaling Federated Learning for Fine-tuning of Large Language Models
We perform an extensive sweep over the number of clients, ranging up to 32, to evaluate the impact of distributed compute on task performance in the federated averaging setting.
Federated learning using mixture of experts
In this paper, we propose a federated learning framework using a mixture of experts to balance the specialist nature of a locally trained model with the generalist knowledge of a global model in a federated learning setting.
Specialized federated learning using a mixture of experts
In federated learning, clients share a global model that has been trained on decentralized local client data.
Adaptive Blending Units: Trainable Activation Functions for Deep Neural Networks
Since ABUs learn the shape, as well as the overall scaling of the activation function, we also analyze the effects of adaptive scaling in common activation functions.
