Search Results for author:
Found 2 papers, 2 papers with code
Exploring Design Choices for Building Language-Specific LLMs
(2) Efficiency can easily improved with simple vocabulary extension and continued fine-tuning in most LLMs we study, and (3) The optimal adaptation method is highly language-dependent, and the simplest approach works well across various experimental settings.
SVFT: Parameter-Efficient Fine-Tuning with Singular Vectors
Extensive experiments on language and vision benchmarks show that SVFT recovers up to 96% of full fine-tuning performance while training only 0. 006 to 0. 25% of parameters, outperforming existing methods that only recover up to 85% performance using 0. 03 to 0. 8% of the trainable parameter budget.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
