Search Results for author:
Found 21 papers, 10 papers with code
Guide Your Agent with Adaptive Multimodal Rewards
Developing an agent capable of adapting to unseen environments remains a difficult challenge in imitation learning.
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
Our goal is to enable a single end-to-end trained model to both learn to map robot observations to actions and enjoy the benefits of large-scale pretraining on language and vision-language data from the web.
Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulation
Towards an answer to this question, we study imitation learning policies in simulation and on a real robot language-conditioned manipulation task to quantify the difficulty of generalization to different (sets of) factors.
Barkour: Benchmarking Animal-level Agility with Quadruped Robots
In the second approach, we distill the specialist skills into a Transformer-based generalist locomotion policy, named Locomotion-Transformer, that can handle various terrains and adjust the robot's gait based on the perceived environment and robot states.
Towards Better Few-Shot and Finetuning Performance with Forgetful Causal Language Models
Large language models (LLM) trained using the next-token-prediction objective, such as GPT3 and PaLM, have revolutionized natural language processing in recent years by showing impressive zero-shot and few-shot capabilities across a wide range of tasks.
Instruction-Following Agents with Multimodal Transformer
Our \ours method consists of a multimodal transformer that encodes visual observations and language instructions, and a transformer-based policy that predicts actions based on encoded representations.
Spectral Decomposition Representation for Reinforcement Learning
Representation learning often plays a critical role in reinforcement learning by managing the curse of dimensionality.
Multi-Game Decision Transformers
Specifically, we show that a single transformer-based model - with a single set of weights - trained purely offline can play a suite of up to 46 Atari games simultaneously at close-to-human performance.
Multimodal Masked Autoencoders Learn Transferable Representations
We provide an empirical study of M3AE trained on a large-scale image-text dataset, and find that M3AE is able to learn generalizable representations that transfer well to downstream tasks.
f-IRL: Inverse Reinforcement Learning via State Marginal Matching
Our method outperforms adversarial imitation learning methods in terms of sample efficiency and the required number of expert trajectories on IRL benchmarks.
Weakly-Supervised Reinforcement Learning for Controllable Behavior
Reinforcement learning (RL) is a powerful framework for learning to take actions to solve tasks.
Efficient Exploration via State Marginal Matching
The SMM objective can be viewed as a two-player, zero-sum game between a state density model and a parametric policy, an idea that we use to build an algorithm for optimizing the SMM objective.
Cross-Task Knowledge Transfer for Visually-Grounded Navigation
Recent efforts on training visual navigation agents conditioned on language using deep reinforcement learning have been successful in learning policies for two different tasks: learning to follow navigational instructions and embodied question answering.
Embodied Multimodal Multitask Learning
In this paper, we propose a multitask model capable of jointly learning these multimodal tasks, and transferring knowledge of words and their grounding in visual objects across the tasks.
On the Complexity of Exploration in Goal-Driven Navigation
Next, we propose to measure the complexity of each environment by constructing dependency graphs between the goals and analytically computing \emph{hitting times} of a random walk in the graph.
Machine Learning in a data-limited regime: Augmenting experiments with synthetic data uncovers order in crumpled sheets
Machine learning has gained widespread attention as a powerful tool to identify structure in complex, high-dimensional data.
Soft Condensed Matter
Gated Path Planning Networks
Value Iteration Networks (VINs) are effective differentiable path planning modules that can be used by agents to perform navigation while still maintaining end-to-end differentiability of the entire architecture.
Combining LSTM and Latent Topic Modeling for Mortality Prediction
There is a great need for technologies that can predict the mortality of patients in intensive care units with both high accuracy and accountability.
Dual Motion GAN for Future-Flow Embedded Video Prediction
To make both synthesized future frames and flows indistinguishable from reality, a dual adversarial training method is proposed to ensure that the future-flow prediction is able to help infer realistic future-frames, while the future-frame prediction in turn leads to realistic optical flows.
Deep Variation-structured Reinforcement Learning for Visual Relationship and Attribute Detection
To capture such global interdependency, we propose a deep Variation-structured Reinforcement Learning (VRL) framework to sequentially discover object relationships and attributes in the whole image.
Robotic Search & Rescue via Online Multi-task Reinforcement Learning
Reinforcement learning (RL) is a general and well-known method that a robot can use to learn an optimal control policy to solve a particular task.
