Search Results for author:
Found 5 papers, 3 papers with code
Sparse Upcycling: Inference Inefficient Finetuning
Small, highly trained, open-source large language models are widely used due to their inference efficiency, but further improving their quality remains a challenge.
Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws
We modify the Chinchilla scaling laws to calculate the optimal LLM parameter count and pre-training data size to train and deploy a model of a given quality and inference demand.
MosaicBERT: A Bidirectional Encoder Optimized for Fast Pretraining
Here, we introduce MosaicBERT, a BERT-style encoder architecture and training recipe that is empirically optimized for fast pretraining.
Autonomous Reinforcement Learning: Formalism and Benchmarking
In this paper, we aim to address this discrepancy by laying out a framework for Autonomous Reinforcement Learning (ARL): reinforcement learning where the agent not only learns through its own experience, but also contends with lack of human supervision to reset between trials.
Bayesian Meta-Learning Through Variational Gaussian Processes
This problem setting can be extended to the Bayesian context, wherein rather than predicting a single label for each query data point, a model predicts a distribution of labels capturing its uncertainty.
