Search Results for author:
Found 3 papers, 0 papers with code
Revisiting Structured Dropout
Large neural networks are often overparameterised and prone to overfitting, Dropout is a widely used regularization technique to combat overfitting and improve model generalization.
DARTFormer: Finding The Best Type Of Attention
Given the wide and ever growing range of different efficient Transformer attention mechanisms, it is important to identify which attention is most effective when given a task.
Wide Attention Is The Way Forward For Transformers?
We demonstrate that wide single layer Transformer models can compete with or outperform deeper ones in a variety of Natural Language Processing (NLP) tasks when both are trained from scratch.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
