Writing in the Margins: Better Inference Pattern for Long Context Retrieval

1 code implementation • 27 Aug 2024 • Melisa Russak, Umar Jamil, Christopher Bryant, Kiran Kamble, Axel Magnuson, Mateusz Russak, Waseem AlShikh

In this paper, we introduce Writing in the Margins (WiM), a new inference pattern for Large Language Models designed to optimize the handling of long input sequences in retrieval-oriented tasks.

RAG Retrieval