Focused Transformer: Contrastive Training for Context Scaling

1 code implementation • NeurIPS 2023 • Szymon Tworkowski, Konrad Staniszewski, Mikołaj Pacek, Yuhuai Wu, Henryk Michalewski, Piotr Miłoś

This novel approach enhances the structure of the (key, value) space, enabling an extension of the context length.

Contrastive Learning

Planning and Learning Using Adaptive Entropy Tree Search

1 code implementation • 12 Feb 2021 • Piotr Kozakowski, Mikołaj Pacek, Piotr Miłoś

We present Adaptive Entropy Tree Search (ANTS) - a novel algorithm combining planning and learning in the maximum entropy paradigm.