LoRA Training in the NTK Regime has No Spurious Local Minima

1 code implementation • 19 Feb 2024 • Uijeong Jang, Jason D. Lee, Ernest K. Ryu

Low-rank adaptation (LoRA) has become the standard approach for parameter-efficient fine-tuning of large language models (LLM), but our theoretical understanding of LoRA has been limited.