Search Results for author:
Found 8 papers, 6 papers with code
OpenThoughts: Data Recipes for Reasoning Models
Yet, there are still many open questions about the best training recipes for reasoning since state-of-the-art models often rely on proprietary datasets with little to no public information available.
Should VLMs be Pre-trained with Image Data?
We find that pre-training with a mixture of image and text data allows models to perform better on vision-language tasks while maintaining strong performance on text-only evaluations.
DataComp-LM: In search of the next generation of training sets for language models
We introduce DataComp for Language Models (DCLM), a testbed for controlled dataset experiments with the goal of improving language models.
SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages
Southeast Asia (SEA) is a region rich in linguistic diversity and cultural variety, with over 1, 300 indigenous languages and a population of 671 million people.
Linearizing Large Language Models
Linear transformers have emerged as a subquadratic-time alternative to softmax attention and have garnered significant interest due to their fixed-size recurrent state that lowers inference cost.
Language models scale reliably with over-training and on downstream tasks
Second, we relate the perplexity of a language model to its downstream task performance by proposing a power law.
A Critical Evaluation of AI Feedback for Aligning Large Language Models
RLAIF first performs supervised fine-tuning (SFT) using demonstrations from a teacher model and then further fine-tunes the model with reinforcement learning (RL), using feedback from a critic model.
Asking More Informative Questions for Grounded Retrieval
When a model is trying to gather information in an interactive setting, it benefits from asking informative questions.
