Search Results for author:
Found 4 papers, 2 papers with code
ScalingFilter: Assessing Data Quality through Inverse Utilization of Scaling Laws
Extensive experiments reveal that semantic diversity is a reliable indicator of dataset diversity, and ScalingFilter achieves an optimal balance between downstream performance and semantic diversity.
Aligning Vision Models with Human Aesthetics in Retrieval: Benchmarks and Algorithms
In this paper, we target the realm of visual aesthetics and aim to align vision models with human aesthetic standards in a retrieval system.
Transformer as Linear Expansion of Learngene
Under the situation where we need to produce models of varying depths adapting for different resource constraints, TLEG achieves comparable results while reducing around 19x parameters stored to initialize these models and around 5x pre-training costs, in contrast to the pre-training and fine-tuning approach.
FP8-LM: Training FP8 Large Language Models
In this paper, we explore FP8 low-bit data formats for efficient training of large language models (LLMs).
