Search Results for author:
Found 8 papers, 5 papers with code
OLMo: Accelerating the Science of Language Models
Given the importance of these details in scientifically studying these models, including their biases and potential risks, we believe it is essential for the research community to have access to powerful, truly open LMs.
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
Language models have become a critical technology to tackling a wide range of natural language processing tasks, yet many details about how the best-performing language models were developed are not reported.
Paloma: A Benchmark for Evaluating Language Model Fit
We invite submissions to our benchmark and organize results by comparability based on compliance with guidelines such as removal of benchmark contamination from pretraining.
Catwalk: A Unified Language Model Evaluation Framework for Many Datasets
The success of large language models has shifted the evaluation paradigms in natural language processing (NLP).
What's In My Big Data?
We open-source WIMBD's code and artifacts to provide a standard set of evaluations for new text-based corpora and to encourage more analyses and transparency around them.
HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation
By converting instructions into modules, HINT models can effectively disregard the length of instructions and few-shot example inputs in terms of compute usage.
Continued Pretraining for Better Zero- and Few-Shot Promptability
We demonstrate that a simple recipe, continued pretraining that incorporates a trainable prompt during multi-task learning, leads to improved promptability in both zero- and few-shot settings compared to existing methods, up to 31% relative.
On Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization
Combining the visual modality with pretrained language models has been surprisingly effective for simple descriptive tasks such as image captioning.
