Search Results for author:
Found 14 papers, 9 papers with code
BMInf: An Efficient Toolkit for Big Model Inference and Tuning
In recent years, large-scale pre-trained language models (PLMs) containing billions of parameters have achieved promising results on various NLP tasks.
MiniCPM4: Ultra-Efficient LLMs on End Devices
Specifically, in terms of model architecture, we propose InfLLM v2, a trainable sparse attention mechanism that accelerates both prefilling and decoding phases for long-context processing.
Ultra-FineWeb: Efficient Data Filtering and Verification for High-Quality LLM Training Data
Data quality has become a key factor in enhancing model performance with the rapid development of large language models (LLMs).
Densing Law of LLMs
This paper introduces the concept of ``\textit{capacity density}'' as a new metric to evaluate the quality of the LLMs across different scales and describes the trend of LLMs in terms of both effectiveness and efficiency.
MiniCPM-V: A GPT-4V Level MLLM on Your Phone
The recent surge of Multimodal Large Language Models (MLLMs) has fundamentally reshaped the landscape of AI research and industry, shedding light on a promising path toward the next AI milestone.
Ranked #7 on
Multiple-choice
on Neptune-Full
MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
For data scaling, we introduce a Warmup-Stable-Decay (WSD) learning rate scheduler (LRS), conducive to continuous training and domain adaptation.
D-Bot: Database Diagnosis System using Large Language Models
Database administrators (DBAs) play an important role in managing, maintaining and optimizing database systems.
Predicting Emergent Abilities with Infinite Resolution Evaluation
With PassUntil, we conduct a quantitative investigation into the scaling law of task performance.
QASnowball: An Iterative Bootstrapping Framework for High-Quality Question-Answering Data Generation
Recent years have witnessed the success of question answering (QA), especially its potential to be a foundation paradigm for tackling diverse NLP tasks.
CUGE: A Chinese Language Understanding and Generation Evaluation Benchmark
We argue that for general-purpose language intelligence evaluation, the benchmark itself needs to be comprehensive and systematic.
CPM-2: Large-scale Cost-effective Pre-trained Language Models
We present a suite of cost-effective techniques for the use of PLMs to deal with the efficiency issues of pre-training, fine-tuning, and inference.
CPM: A Large-scale Generative Chinese Pre-trained Language Model
However, applying GPT-3 to address Chinese NLP tasks is still challenging, as the training corpus of GPT-3 is primarily English, and the parameters are not publicly available.
OpenAttack: An Open-source Textual Adversarial Attack Toolkit
Textual adversarial attacking has received wide and increasing attention in recent years.
Adversarial Language Games for Advanced Natural Language Intelligence
In this work, we propose a challenging adversarial language game called Adversarial Taboo as an example, in which an attacker and a defender compete around a target word.
