Search Results for author: Matvey Mikhalchuk

Found 2 papers, 1 papers with code

The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Models

no code implementations10 Nov 2023 Anton Razzhigaev, Matvey Mikhalchuk, Elizaveta Goncharova, Ivan Oseledets, Denis Dimitrov, Andrey Kuznetsov

In this study, we present an investigation into the anisotropy dynamics and intrinsic dimension of embeddings in transformer architectures, focusing on the dichotomy between encoders and decoders.

Cannot find the paper you are looking for? You can Submit a new open access paper.