Superposed Decoding: Multiple Generations from a Single Autoregressive Inference Pass

1 code implementation • 28 May 2024 • Ethan Shen, Alan Fan, Sarah M. Pratt, Jae Sung Park, Matthew Wallingford, Sham M. Kakade, Ari Holtzman, Ranjay Krishna, Ali Farhadi, Aditya Kusupati

We achieve this by feeding a superposition of the most recent token embeddings from the $k$ drafts as input to the next decoding step of the language model.

Code Completion Language Modelling