Search Results for author: Matteo Saponati

Found 1 papers, 1 papers with code

The underlying structures of self-attention: symmetry, directionality, and emergent dynamics in Transformer training

1 code implementation15 Feb 2025 Matteo Saponati, Pascal Sager, Pau Vilimelis Aceituno, Thilo Stadelmann, Benjamin Grewe

Self-attention is essential to Transformer architectures, yet how information is embedded in the self-attention matrices and how different objective functions impact this process remains unclear.

Cannot find the paper you are looking for? You can Submit a new open access paper.