Retriever-Augmented Generation, or RAG, is a type of language generation model that combines pre-trained parametric and non-parametric memory for language generation. Specifically, the parametric memory is a pre-trained seq2seq model and the non-parametric memory is a dense vector index of Wikipedia, accessed with a pre-trained neural retriever. For query $x$, Maximum Inner Product Search (MIPS) is used to find the top-K documents $z_{i}$. For final prediction $y$, we treat $z$ as a latent variable and marginalize over seq2seq predictions given different documents.

Source: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks


Paper Code Results Date Stars


Task Papers Share
Retrieval 179 31.91%
Question Answering 69 12.30%
Language Modelling 40 7.13%
Large Language Model 29 5.17%
Information Retrieval 27 4.81%
Text Generation 17 3.03%
Open-Domain Question Answering 13 2.32%
Benchmarking 11 1.96%
Sentence 8 1.43%