Search Results for author:
Found 8 papers, 7 papers with code
Sparsified SGD with Memory
Huge scale machine learning problems are nowadays tackled by distributed optimization algorithms, i. e. algorithms that leverage the compute power of many devices for training.
Extrapolating paths with graph neural networks
We consider the problem of path inference: given a path prefix, i. e., a partially observed sequence of nodes in a graph, we want to predict which nodes are in the missing suffix.
On the Relationship between Self-Attention and Convolutional Layers
This work provides evidence that attention layers can perform convolution and, indeed, they often learn to do so in practice.
Ranked #151 on
Image Classification
on CIFAR-10
Robust Cross-lingual Embeddings from Parallel Sentences
Recent advances in cross-lingual word embeddings have primarily relied on mapping-based methods, which project pretrained word embeddings from different languages into a shared space through a linear transformation.
Cross-Lingual Document Classification
Cross-Lingual Word Embeddings
+7
Multi-Head Attention: Collaborate Instead of Concatenate
We also show that it is possible to re-parametrize a pre-trained multi-head attention layer into our collaborative attention layer.
Group Equivariant Stand-Alone Self-Attention For Vision
We provide a general self-attention formulation to impose group equivariance to arbitrary symmetry groups.
Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth
Attention-based architectures have become ubiquitous in machine learning, yet our understanding of the reasons for their effectiveness remains limited.
Differentiable Patch Selection for Image Recognition
Neural Networks require large amounts of memory and compute to process high resolution images, even when only a small part of the image is actually informative for the task at hand.
