Search Results for author:
Found 7 papers, 5 papers with code
Mixtera: A Data Plane for Foundation Model Training
We build and present Mixtera, a data plane for foundation model training that enables users to declaratively express which data samples should be used in which proportion and in which order during training.
RedPajama: an Open Dataset for Training Large Language Models
In addition, we release RedPajama-V2, a massive web-only dataset consisting of raw, unfiltered text data together with quality signals and metadata.
Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code
Despite these efforts, such models encounter challenges such as limited multilingual capabilities, risks of catastrophic forgetting during continual pretraining, and the high costs of training models from scratch, alongside the need to align with AI safety standards and regulatory frameworks.
DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMs
Fine-tuning large language models (LLMs) greatly improves model quality for downstream tasks.
DMLR: Data-centric Machine Learning Research -- Past, Present and Future
Drawing from discussions at the inaugural DMLR workshop at ICML 2023 and meetings prior, in this report we outline the relevance of community engagement and infrastructure development for the creation of next-generation public datasets that will advance machine learning science.
DataPerf: Benchmarks for Data-Centric AI Development
Machine learning research has long focused on models rather than datasets, and prominent datasets are used for common ML tasks without regard to the breadth, difficulty, and faithfulness of the underlying problems.
SHiFT: An Efficient, Flexible Search Engine for Transfer Learning
Transfer learning can be seen as a data- and compute-efficient alternative to training models from scratch.
