PEMT: Multi-Task Correlation Guided Mixture-of-Experts Enables Parameter-Efficient Transfer Learning

no code implementations • 23 Feb 2024 • Zhisheng Lin, Han Fu, Chenghao Liu, Zhuo Li, Jianling Sun

However, current approaches typically either train adapters on individual tasks or distill shared knowledge from source tasks, failing to fully exploit task-specific knowledge and the correlation between source and target tasks.

Transfer Learning