Search Results for author: Dana Angluin

Found 8 papers, 1 papers with code

Concise One-Layer Transformers Can Do Function Evaluation (Sometimes)

no code implementations28 Mar 2025 Lena Strobl, Dana Angluin, Robert Frank

While transformers have proven enormously successful in a range of tasks, their fundamental properties as models of computation are not well understood.

Simulating Hard Attention Using Soft Attention

no code implementations13 Dec 2024 Andy Yang, Lena Strobl, David Chiang, Dana Angluin

Second, we demonstrate how temperature scaling allows softmax transformers to simulate a large subclass of average-hard attention transformers, those that have what we call the uniform-tieless property.

Hard Attention

Transformers as Transducers

no code implementations2 Apr 2024 Lena Strobl, Dana Angluin, David Chiang, Jonathan Rawski, Ashish Sabharwal

B-RASP[pos] enables calculations on positions (such as copying the first half of a string) and contains all first-order regular functions.

Hard Attention POS

What Formal Languages Can Transformers Express? A Survey

no code implementations1 Nov 2023 Lena Strobl, William Merrill, Gail Weiss, David Chiang, Dana Angluin

As transformers have gained prominence in natural language processing, some researchers have investigated theoretically what problems they can and cannot solve, by treating problems as formal languages.

Survey

Masked Hard-Attention Transformers Recognize Exactly the Star-Free Languages

no code implementations21 Oct 2023 Andy Yang, David Chiang, Dana Angluin

The expressive power of transformers over inputs of unbounded size can be studied through their ability to recognize classes of formal languages.

Hard Attention Position

Formal Language Recognition by Hard Attention Transformers: Perspectives from Circuit Complexity

no code implementations13 Apr 2022 Yiding Hao, Dana Angluin, Robert Frank

This paper analyzes three formal models of Transformer encoders that differ in the form of their self-attention mechanism: unique hard attention (UHAT); generalized unique hard attention (GUHAT), which generalizes UHAT; and averaging hard attention (AHAT).

Hard Attention

Regular omega-Languages with an Informative Right Congruence

no code implementations10 Sep 2018 Dana Angluin, Dana Fisman

The right congruence of a regular omega-language is not informative enough; many regular omega-languages have a trivial right congruence, and in general it is not always possible to define an omega-automaton recognizing a given language that is isomorphic to the rightcon automaton.

Cannot find the paper you are looking for? You can Submit a new open access paper.