no code implementations • 14 Dec 2023 • Rhys Gould, Euan Ong, George Ogden, Arthur Conmy
In this work we present successor heads: attention heads that increment tokens with a natural ordering, such as numbers, months, and days.
1 code implementation • 1 Sep 2023 • Luke Bailey, Euan Ong, Stuart Russell, Scott Emmons
In this work, we focus on the image input to a vision-language model (VLM).
no code implementations • 16 Dec 2022 • Euan Ong, Petar Veličković
And with this, we construct an aggregator of $O(\log V)$ depth, yielding exponential improvements for both parallelism and dependency length while achieving performance competitive with recurrent aggregators.