Search Results for author: Mohammed Muqeeth

Found 4 papers, 3 papers with code

Learning to Route Among Specialized Experts for Zero-Shot Generalization

1 code implementation8 Feb 2024 Mohammed Muqeeth, Haokun Liu, Yufan Liu, Colin Raffel

Unlike past methods that learn to route among specialized models, PHATGOOSE explores the possibility that zero-shot generalization will be improved if different experts can be adaptively chosen for each token and at each layer in the model.

Zero-shot Generalization

Soft Merging of Experts with Adaptive Routing

no code implementations6 Jun 2023 Mohammed Muqeeth, Haokun Liu, Colin Raffel

To address this issue, we introduce Soft Merging of Experts with Adaptive Routing (SMEAR), which avoids discrete routing by using a single "merged" expert constructed via a weighted average of all of the experts' parameters.

Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning

2 code implementations11 May 2022 Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, Colin Raffel

ICL incurs substantial computational, memory, and storage costs because it involves processing all of the training examples every time a prediction is made.

Few-Shot Text Classification In-Context Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.