Search Results for author:
Found 1 papers, 0 papers with code
FNetAR: Mixing Tokens with Autoregressive Fourier Transforms
In this note we examine the autoregressive generalization of the FNet algorithm, in which self-attention layers from the standard Transformer architecture are substituted with a trivial sparse-uniformsampling procedure based on Fourier transforms.
Ranked #61 on
Language Modelling
on WikiText-103
Cannot find the paper you are looking for? You can
Submit a new open access paper.
