Search Results for author:
Found 13 papers, 9 papers with code
ST-MoE: Designing Stable and Transferable Sparse Expert Models
But advancing the state-of-the-art across a broad set of natural language tasks has been hindered by training instabilities and uncertain quality during fine-tuning.
Ranked #1 on
Common Sense Reasoning
on ARC (Easy)
Revisiting 3D ResNets for Video Recognition
A recent work from Bello shows that training and scaling strategies may be more significant than model architectures for visual recognition.
Ranked #38 on
Action Classification
on Kinetics-600
Revisiting ResNets: Improved Training and Scaling Strategies
Using improved training and scaling strategies, we design a family of ResNet architectures, ResNet-RS, which are 1. 7x - 2. 7x faster than EfficientNets on TPUs, while achieving similar accuracies on ImageNet.
Ranked #1 on
Document Image Classification
on AIP
LambdaNetworks: Modeling Long-Range Interactions Without Attention
We present lambda layers -- an alternative framework to self-attention -- for capturing long-range interactions between an input and structured contextual information (e. g. a pixel surrounded by other pixels).
Ranked #278 on
Image Classification
on ImageNet
Global Self-Attention Networks
Based on the proposed GSA module, we introduce new standalone global attention-based deep networks that use GSA modules instead of convolutions to model pixel interactions.
Global Self-Attention Networks for Image Recognition
Based on the proposed GSA module, we introduce new standalone global attention-based deep networks that use GSA modules instead of convolutions to model pixel interactions.
Stand-Alone Self-Attention in Vision Models
The natural question that arises is whether attention can be a stand-alone primitive for vision models instead of serving as just an augmentation on top of convolutions.
Attention Augmented Convolutional Networks
Convolutional networks have been the paradigm of choice in many computer vision applications.
Ranked #114 on
Image Classification
on CIFAR-100
(using extra training data)
Seq2Slate: Re-ranking and Slate Optimization with RNNs
Ranking is a central task in machine learning and information retrieval.
Neural Optimizer Search with Reinforcement Learning
We present an approach to automate the process of discovering optimization methods, with a focus on deep learning architectures.
Neural Optimizer Search using Reinforcement Learning
We present an approach to automate the process of discovering optimization methods, with a focus on deep learning architectures.
Neural Combinatorial Optimization with Reinforcement Learning
Despite the computational expense, without much engineering and heuristic designing, Neural Combinatorial Optimization achieves close to optimal results on 2D Euclidean graphs with up to 100 nodes.
