Search Results for author: Euan Ong

Found 3 papers, 1 papers with code

Successor Heads: Recurring, Interpretable Attention Heads In The Wild

no code implementations14 Dec 2023 Rhys Gould, Euan Ong, George Ogden, Arthur Conmy

In this work we present successor heads: attention heads that increment tokens with a natural ordering, such as numbers, months, and days.

Language Modelling

Learnable Commutative Monoids for Graph Neural Networks

no code implementations16 Dec 2022 Euan Ong, Petar Veličković

And with this, we construct an aggregator of $O(\log V)$ depth, yielding exponential improvements for both parallelism and dependency length while achieving performance competitive with recurrent aggregators.

Cannot find the paper you are looking for? You can Submit a new open access paper.