1 code implementation • EACL 2021 • Tatyana Ruzsics, Olga Sozinova, Ximena Gutierrez-Vasques, Tanja Samardzic
We apply our methodology to analyze the model{'}s decisions on three typologically-different languages and find that a) our pattern extraction method applied to cross-attention weights uncovers variation in form of inflection morphemes, b) pattern extraction from self-attention shows triggers for such variation, c) both types of patterns are closely aligned with grammar inflection classes and class assignment criteria, for all three languages.
1 code implementation • EACL 2021 • Ximena Gutierrez-Vasques, Christian Bentz, Olga Sozinova, Tanja Samardzic
The distributions of orthographic word types are very different across languages due to typological characteristics, different writing traditions and potentially other factors.