Search Results for author:
Found 3 papers, 1 papers with code
Synthetic Text Generation for Training Large Language Models via Gradient Matching
In this work, we propose the first theoretically rigorous approach for generating synthetic human-readable text that guarantees the convergence and performance of LLMs during fine-tuning on a target task.
Addax: Utilizing Zeroth-Order Gradients to Improve Memory Efficiency and Performance of SGD for Fine-Tuning Language Models
In our experiments on the larger OPT-30B model, on average, Addax outperforms MeZO in terms of accuracy/F1 score by >16 and runs 30x faster on a single H100 GPU.
Optimal Differentially Private Model Training with Public Data
We show that the optimal error rates can be attained (up to log factors) by either discarding private data and training a public model, or treating public data like it is private and using an optimal DP algorithm.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
