Search Results for author:
Found 2 papers, 0 papers with code
DARTFormer: Finding The Best Type Of Attention
Given the wide and ever growing range of different efficient Transformer attention mechanisms, it is important to identify which attention is most effective when given a task.
Wide Attention Is The Way Forward For Transformers?
We demonstrate that wide single layer Transformer models can compete with or outperform deeper ones in a variety of Natural Language Processing (NLP) tasks when both are trained from scratch.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
