Search Results for author:
Found 18 papers, 6 papers with code
Retouchdown: Releasing Touchdown on StreetLearn as a Public Resource for Language Grounding Tasks in Street View
These have been added to the StreetLearn dataset and can be obtained via the same process as used previously for StreetLearn.
Mechanic: A Learning Rate Tuner
We introduce a technique for tuning the learning rate scale factor of any base optimization algorithm and schedule automatically, which we call \textsc{mechanic}.
Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion
Our primary technique is a reduction from non-smooth non-convex optimization to online learning, after which our results follow from standard regret bounds in online learning.
Simplifying and Understanding State Space Models with Diagonal Linear RNNs
Sequence models based on linear state spaces (SSMs) have recently emerged as a promising choice of architecture for modeling long range dependencies across various modalities.
Differentially Private Image Classification from Features
We find that linear regression is much more effective than logistic regression from both privacy and computational aspects, especially at stricter epsilon values ($\epsilon < 1$).
Ranked #32 on
Image Classification
on ImageNet
Convexifying Transformers: Improving optimization and understanding of transformer networks
To this end, we study the training problem of attention/transformer networks and introduce a novel convex analytic approach to improve the understanding and optimization of these networks.
Long Range Language Modeling via Gated State Spaces
State space models have shown to be effective at modeling long range dependencies, specially on sequence classification tasks.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
Large Scale Transfer Learning for Differentially Private Image Classification
Moreover, by systematically comparing private and non-private models across a range of large batch sizes, we find that similar to non-private setting, choice of optimizer can further improve performance substantially with DP.
Transformer Memory as a Differentiable Search Index
In this paper, we demonstrate that information retrieval can be accomplished with a single Transformer, in which all information about the corpus is encoded in the parameters of the model.
ALX: Large Scale Matrix Factorization on TPUs
We present ALX, an open-source library for distributed matrix factorization using Alternating Least Squares, written in JAX.
High-probability Bounds for Non-Convex Stochastic Optimization with Heavy Tails
We consider non-convex stochastic optimization using first-order algorithms for which the gradient estimates may have heavy tails.
Extreme Memorization via Scale of Initialization
In the case of the homogeneous ReLU activation, we show that this behavior can be attributed to the loss function.
Momentum Improves Normalized SGD
We provide an improved analysis of normalized SGD showing that adding momentum provably removes the need for large batch sizes on non-convex objectives.
Retouchdown: Adding Touchdown to StreetLearn as a Shareable Resource for Language Grounding Tasks in Street View
These have been added to the StreetLearn dataset and can be obtained via the same process as used previously for StreetLearn.
Ranked #7 on
Vision and Language Navigation
on Touchdown Dataset
VALAN: Vision and Language Agent Navigation
VALAN is a lightweight and scalable software framework for deep reinforcement learning based on the SEED RL architecture.
Transferable Representation Learning in Vision-and-Language Navigation
Vision-and-Language Navigation (VLN) tasks such as Room-to-Room (R2R) require machine agents to interpret natural language instructions and learn to act in visually realistic environments to achieve navigation goals.
Ranked #115 on
Vision and Language Navigation
on VLN Challenge
Multi-modal Discriminative Model for Vision-and-Language Navigation
Vision-and-Language Navigation (VLN) is a natural language grounding task where agents have to interpret natural language instructions in the context of visual scenes in a dynamic environment to achieve prescribed navigation goals.
