AutoChunk: Automated Activation Chunk for Memory-Efficient Long Sequence Inference

no code implementations • 19 Jan 2024 • Xuanlei Zhao, Shenggan Cheng, Guangyang Lu, Jiarui Fang, Haotian Zhou, Bin Jia, Ziming Liu, Yang You

The experiments demonstrate that AutoChunk can reduce over 80\% of activation memory while maintaining speed loss within 10%, extend max sequence length by 3. 2x to 11. 7x, and outperform state-of-the-art methods by a large margin.

Code Generation

Let's reward step by step: Step-Level reward model as the Navigators for Reasoning

no code implementations • 16 Oct 2023 • Qianli Ma, Haotian Zhou, Tingkai Liu, Jianbo Yuan, PengFei Liu, Yang You, Hongxia Yang

Recent years have seen considerable advancements in multi-step reasoning with Large Language Models (LLMs).

Code Generation GSM8K +2

DavIR: Data Selection via Implicit Reward for Large Language Models

no code implementations • 16 Oct 2023 • Haotian Zhou, Tingkai Liu, Qianli Ma, Yufeng Zhang, Jianbo Yuan, PengFei Liu, Yang You, Hongxia Yang

We introduce DavIR, a model-based data selection method for post-training Large Language Models.

Causal Language Modeling GSM8K +3