Search Results for author:
Found 2 papers, 1 papers with code
Optimizing Pre-Training Data Mixtures with Mixtures of Data Expert Models
We propose a method to optimize language model pre-training data mixtures through efficient approximation of the cross-entropy loss corresponding to each candidate mixture via a Mixture of Data Experts (MDE).
The Vizier Gaussian Process Bandit Algorithm
Google Vizier has performed millions of optimizations and accelerated numerous research and production systems at Google, demonstrating the success of Bayesian optimization as a large-scale service.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
