Search Results for author:
Found 7 papers, 5 papers with code
CCI3.0-HQ: a large-scale Chinese dataset of high quality designed for pre-training large language models
We present CCI3. 0-HQ (https://huggingface. co/datasets/BAAI/CCI3-HQ), a high-quality 500GB subset of the Chinese Corpora Internet 3. 0 (CCI3. 0)(https://huggingface. co/datasets/BAAI/CCI3-Data), developed using a novel two-stage hybrid filtering pipeline that significantly enhances data quality.
Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Data
Despite the availability of several open-source multimodal datasets, limitations in the scale and quality of open-source instruction data hinder the performance of VLMs trained on these datasets, leading to a significant gap compared to models trained on closed-source data.
ReTok: Replacing Tokenizer to Enhance Representation Efficiency in Large Language Model
In this work, we propose a method to improve model representation and processing efficiency by replacing the tokenizers of LLMs.
Emu3: Next-Token Prediction is All You Need
While next-token prediction is considered a promising path towards artificial general intelligence, it has struggled to excel in multimodal tasks, which are still dominated by diffusion models (e. g., Stable Diffusion) and compositional approaches (e. g., CLIP combined with LLMs).
Ranked #132 on
Visual Question Answering
on MM-Vet
Aquila2 Technical Report
This paper introduces the Aquila2 series, which comprises a wide range of bilingual models with parameter sizes of 7, 34, and 70 billion.
AquilaMoE: Efficient Training for MoE Models with Scale-Up and Scale-Out Strategies
In this paper, we present AquilaMoE, a cutting-edge bilingual 8*16B Mixture of Experts (MoE) language model that has 8 experts with 16 billion parameters each and is developed using an innovative training methodology called EfficientScale.
Contrastive Cross-domain Recommendation in Matching
Cross-domain recommendation (CDR) aims to provide better recommendation results in the target domain with the help of the source domain, which is widely used and explored in real-world systems.
