Fine-Tuning

Edit

General • 10 methods

Fine-Tuning methods in deep learning take existing trained networks and 'fine-tune' them to a new task so that information contained in the weights can be repurposed. Below you can find a continuously updating list of fine-tuning methods.

Methods

Add a Method

Method	Year	Papers
Discriminative Fine-Tuning Universal Language Model Fine-tuning for Text Classification	2018	1876
Virtual Data Augmentation Virtual Data Augmentation: A Robust and General Framework for Fine-tuning Pre-trained Models	2021	4
Leverage Learning Token-Efficient Leverage Learning in Large Language Models	2024	3
ERNIE-GEN ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation	2020	1
Child-Tuning Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning	2021	1
Spectral DeTuning Recovering the Pre-Fine-Tuning Weights of Generative Models	2024	1
MixLoRA MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts	2024	1
DSiRe Dataset Size Recovery from LoRA Weights	2024	1
SORSA SORSA: Singular Values and Orthonormal Regularized Singular Vectors Adaptation of Large Language Models	2024	1

Fine-Tuning Edit

Methods Add a Method

Fine-Tuning

Edit

Methods

Add a Method