Search Results for author:
Found 4 papers, 3 papers with code
When Precision Meets Position: BFloat16 Breaks Down RoPE in Long-Context Training
To address this, we develop AnchorAttention, a plug-and-play attention method that alleviates numerical issues caused by BFloat16, improves long-context capabilities, and speeds up training.
CCSBench: Evaluating Compositional Controllability in LLMs for Scientific Document Summarization
To broaden the dissemination of scientific knowledge to diverse audiences, scientific document summarization must simultaneously control multiple attributes such as length and empirical focus.
Beyond Memorization: The Challenge of Random Memory Access in Language Models
Through carefully-designed synthetic tasks, covering the scenarios of full recitation, selective recitation and grounded question answering, we reveal that LMs manage to sequentially access their memory while encountering challenges in randomly accessing memorized content.
Translating Natural Language to Planning Goals with Large-Language Models
Recent large language models (LLMs) have demonstrated remarkable performance on a variety of natural language processing (NLP) tasks, leading to intense excitement about their applicability across various domains.
