Search Results for author: Peter Cholak

Found 2 papers, 1 papers with code

Tighter Bounds on the Expressivity of Transformer Encoders

no code implementations25 Jan 2023 David Chiang, Peter Cholak, Anand Pillay

Characterizing neural networks in terms of better-understood formal systems has the potential to yield new insights into the power and limitations of these networks.

Overcoming a Theoretical Limitation of Self-Attention

1 code implementation ACL 2022 David Chiang, Peter Cholak

We examine this limitation using two languages: PARITY, the language of bit strings with an odd number of 1s, and FIRST, the language of bit strings starting with a 1.

LEMMA Machine Translation +1

Cannot find the paper you are looking for? You can Submit a new open access paper.