no code implementations • 25 Apr 2022 • Siyuan Pan, Xiaoshuang Li, Tingyao Li, Liang Hou, KaiBin Qiu, Xiaobing Tu
To address this challenge, we propose a novel method that first linearly over-parameterizes the compact layers in pruned networks to enlarge the number of fine-tuning parameters and then re-parameterizes them to the original layers after fine-tuning.