Search Results for author: Alex Vitvitskyi

Found 10 papers, 4 papers with code

What makes a good feedforward computational graph?

no code implementations10 Feb 2025 Alex Vitvitskyi, João G. M. Araújo, Marc Lackenby, Petar Veličković

As implied by the plethora of literature on graph rewiring, the choice of computational graph employed by a neural network can make a significant impact on its downstream performance.

Amplifying human performance in combinatorial competitive programming

no code implementations29 Nov 2024 Petar Veličković, Alex Vitvitskyi, Larisa Markeeva, Borja Ibarz, Lars Buesing, Matej Balog, Alexander Novikov

Recent years have seen a significant surge in complex AI systems for competitive programming, capable of performing at admirable levels against human competitors.

Round and Round We Go! What makes Rotary Positional Encodings useful?

no code implementations8 Oct 2024 Federico Barbero, Alex Vitvitskyi, Christos Perivolaropoulos, Razvan Pascanu, Petar Veličković

Positional Encodings (PEs) are a critical component of Transformer-based Large Language Models (LLMs), providing the attention mechanism with important sequence-position information.

The CLRS-Text Algorithmic Reasoning Language Benchmark

2 code implementations6 Jun 2024 Larisa Markeeva, Sean McLeish, Borja Ibarz, Wilfried Bounsi, Olga Kozlova, Alex Vitvitskyi, Charles Blundell, Tom Goldstein, Avi Schwarzschild, Petar Veličković

Three years ago, a similar issue was identified and rectified in the field of neural algorithmic reasoning, with the advent of the CLRS benchmark.

Transformers need glasses! Information over-squashing in language tasks

no code implementations6 Jun 2024 Federico Barbero, Andrea Banino, Steven Kapturowski, Dharshan Kumaran, João G. M. Araújo, Alex Vitvitskyi, Razvan Pascanu, Petar Veličković

We rely on a theoretical signal propagation analysis -- specifically, we analyse the representations of the last token in the final layer of the Transformer, as this is the representation used for next-token prediction.

Decoder

Beyond Fine-Tuning: Transferring Behavior in Reinforcement Learning

no code implementations24 Feb 2021 Víctor Campos, Pablo Sprechmann, Steven Hansen, Andre Barreto, Steven Kapturowski, Alex Vitvitskyi, Adrià Puigdomènech Badia, Charles Blundell

We introduce Behavior Transfer (BT), a technique that leverages pre-trained policies for exploration and that is complementary to transferring neural network weights.

reinforcement-learning Reinforcement Learning +2

Never Give Up: Learning Directed Exploration Strategies

6 code implementations ICLR 2020 Adrià Puigdomènech Badia, Pablo Sprechmann, Alex Vitvitskyi, Daniel Guo, Bilal Piot, Steven Kapturowski, Olivier Tieleman, Martín Arjovsky, Alexander Pritzel, Andew Bolt, Charles Blundell

Our method doubles the performance of the base agent in all hard exploration in the Atari-57 suite while maintaining a very high score across the remaining games, obtaining a median human normalised score of 1344. 0%.

Atari Games Reinforcement Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.