Search Results for author:
Found 25 papers, 8 papers with code
AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents
We experimentally show that such "in-the-wild" data collected by AutoRT is significantly more diverse, and that AutoRT's use of LLMs allows for instruction following data collection robots that can align to human preferences.
SARA-RT: Scaling up Robotics Transformers with Self-Adaptive Robust Attention
We present Self-Adaptive Robust Attention for Robotics Transformers (SARA-RT): a new paradigm for addressing the emerging challenge of scaling up Robotics Transformers (RT) for on-robot deployment.
RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketches
Generalization remains one of the most important desiderata for robust robot learning systems.
Protecting Sensitive Data through Federated Co-Training
Federated learning allows us to collaboratively train a model without pooling the data by iteratively aggregating the parameters of local models.
Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functions
In this work, we present a scalable reinforcement learning method for training multi-task policies from large offline datasets that can leverage both human demonstrations and autonomously collected data.
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
Our goal is to enable a single end-to-end trained model to both learn to map robot observations to actions and enjoy the benefits of large-scale pretraining on language and vision-language data from the web.
Large Language Models as General Pattern Machines
We observe that pre-trained large language models (LLMs) are capable of autoregressively completing complex token sequences -- from arbitrary ones procedurally generated by probabilistic context-free grammars (PCFG), to more rich spatial patterns found in the Abstraction and Reasoning Corpus (ARC), a general AI benchmark, prompted in the style of ASCII art.
RT-1: Robotics Transformer for Real-World Control at Scale
By transferring knowledge from large, diverse, task-agnostic datasets, modern machine learning models can solve specific downstream tasks either zero-shot or with small task-specific datasets to a high level of performance.
Token Turing Machines
The model's memory module ensures that a new observation will only be processed with the contents of the memory (and not the entire history), meaning that it can efficiently process long sequences with a bounded computational cost at each step.
Ranked #1 on
Action Detection
on Charades
Open-vocabulary Queryable Scene Representations for Real World Planning
Large language models (LLMs) have unlocked new capabilities of task planning from human instructions.
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
We show how low-level skills can be combined with large language models so that the language model provides high-level knowledge about the procedures for performing complex and temporally-extended instructions, while value functions associated with these skills provide the grounding necessary to connect this knowledge to a particular physical environment.
RL-CycleGAN: Reinforcement Learning Aware Simulation-To-Real
However, this sort of translation is typically task-agnostic, in that the translated images may not preserve all features that are relevant to the task.
Federated Learning for Emoji Prediction in a Mobile Keyboard
We show that a word-level recurrent neural network can predict emoji from text typed on a mobile keyboard.
Off-Policy Evaluation via Off-Policy Classification
However, for high-dimensional observations, such as images, models of the environment can be difficult to fit and value-based methods can make IS hard to use or even ill-conditioned, especially when dealing with continuous action spaces.
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models.
Streaming End-to-end Speech Recognition For Mobile Devices
End-to-end (E2E) models, which directly predict output character sequences given input speech, are good candidates for on-device speech recognition.
Federated Learning for Mobile Keyboard Prediction
We train a recurrent neural network language model using a distributed, on-device learning framework called federated learning for the purpose of next-word prediction in a virtual keyboard for smartphones.
Large-Scale Visual Speech Recognition
To achieve this, we constructed the largest existing visual speech recognition dataset, consisting of pairs of text and video clips of faces speaking (3, 886 hours of video).
Ranked #11 on
Lipreading
on LRS3-TED
(using extra training data)
Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducer
The model consists of an `encoder', which is initialized from a connectionist temporal classification-based (CTC) acoustic model, and a `decoder' which is partially initialized from a recurrent neural network language model trained on text data alone.
State-of-the-art Speech Recognition With Sequence-to-Sequence Models
Attention-based encoder-decoder architectures such as Listen, Attend, and Spell (LAS), subsume the acoustic, pronunciation and language model components of a traditional automatic speech recognition (ASR) system into a single neural network.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Multi-Dialect Speech Recognition With A Single Sequence-To-Sequence Model
Sequence-to-sequence models provide a simple and elegant solution for building speech recognition systems by folding separate components of a typical system, namely acoustic (AM), pronunciation (PM) and language (LM) models into a single neural network.
Multilingual Speech Recognition With A Single End-To-End Model
Training a conventional automatic speech recognition (ASR) system to support multiple languages is challenging because the sub-word unit, lexicon and word inventories are typically language specific.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Streaming Small-Footprint Keyword Spotting using Sequence-to-Sequence Models
We develop streaming keyword spotting systems using a recurrent neural network transducer (RNN-T) model: an all-neural, end-to-end trained, sequence-to-sequence model which jointly learns acoustic and language model components.
Personalized Speech recognition on mobile devices
We describe a large vocabulary speech recognition system that is accurate, has low latency, and yet has a small enough memory and computational footprint to run faster than real-time on a Nexus 5 Android smartphone.
Fast and Accurate Recurrent Neural Network Acoustic Models for Speech Recognition
We have recently shown that deep Long Short-Term Memory (LSTM) recurrent neural networks (RNNs) outperform feed forward deep neural networks (DNNs) as acoustic models for speech recognition.
