Search Results for author:
Found 380 papers, 198 papers with code
Inferring Reward Functions from Demonstrators with Unknown Biases
Our goal is to infer reward functions from demonstrations.
Responsive Safety in Reinforcement Learning
This intuition leads to our introduction of PID control for the Lagrange multiplier in constrained RL, which we cast as a dynamical system.
Hierarchically Decoupled Morphological Transfer
Learning long-range behaviors on complex high-dimensional agents is a fundamental problem in robot learning.
CURL: Contrastive Unsupervised Representation Learning for Reinforcement Learning
CURL extracts high level features from raw pixels using a contrastive learning objective and performs off-policy control on top of the extracted features.
Planning to Explore via Latent Disagreement
To solve complex tasks, intelligent agents first need to explore their environments.
SkillBlender: Towards Versatile Humanoid Whole-Body Loco-Manipulation via Skill Blending
To that end, we introduce SkillBlender, a novel hierarchical reinforcement learning framework for versatile humanoid loco-manipulation.
Object-centric 3D Motion Field for Robot Learning from Human Videos
However, how to extract action knowledge (or action representations) from videos for policy learning remains a key challenge.
Rodrigues Network for Learning Robot Actions
Understanding and predicting articulated actions is important in robot learning.
Feel the Force: Contact-Driven Learning from Humans
Using a tactile glove to measure contact forces and a vision-based model to estimate hand pose, we train a closed-loop policy that continuously predicts the forces needed for manipulation.
Bigger, Regularized, Categorical: High-Capacity Value Functions are Efficient Multi-Task Learners
Recent advances in language modeling and vision stem from training large models on diverse, multi-task data.
Diffusion Guidance Is a Controllable Policy Improvement Operator
The resulting framework, CFGRL, is trained with the simplicity of supervised learning, yet can further improve on the policies in the data.
FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Control
Reinforcement learning (RL) has driven significant progress in robotics, but its complexity and long training times remain major bottlenecks.
EgoZero: Robot Learning from Smart Glasses
Despite recent progress in general purpose robotics, robot policies still lag far behind basic human capabilities in the real world.
DexGarmentLab: Dexterous Garment Manipulation Environment with Generalizable Policy
To improve generalization across diverse garment shapes and deformations, we propose a Hierarchical gArment-manipuLation pOlicy (HALO).
Visual Imitation Enables Contextual Humanoid Control
How can we teach humanoids to climb staircases and sit on chairs using the surrounding environment context?
Geometric Retargeting: A Principled, Ultrafast Neural Hand Retargeting Algorithm
We introduce Geometric Retargeting (GeoRT), an ultrafast, and principled neural hand retargeting algorithm for teleoperation, developed as part of our recent Dexterity Gen (DexGen) system.
OTTER: A Vision-Language-Action Model with Text-Aware Visual Feature Extraction
Vision-Language-Action (VLA) models aim to predict robotic actions based on visual observations and language instructions.
Video2Policy: Scaling up Manipulation Tasks in Simulation through Internet Videos
Finally, we show that the generated simulation data can be scaled up for training a general policy, and it can be transferred back to the real robot in a Real2Sim2Real way.
Value-Based Deep RL Scales Predictably
Third, this scaling behavior is enabled by first estimating predictable relationships between hyperparameters, which is used to manage effects of overfitting and plasticity loss unique to RL.
DexterityGen: Foundation Controller for Unprecedented Dexterity
In the real world, we use human teleoperation as a prompt to the controller to produce highly dexterous behavior.
Beyond Sight: Finetuning Generalist Robot Policies with Heterogeneous Sensors via Language Grounding
However, state-of-the-art generalist robot policies are typically trained on large datasets to predict robot actions solely from visual and proprioceptive observations.
MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximization
In this work, we introduce a framework, MaxInfoRL, for balancing intrinsic and extrinsic exploration.
Efficient Long Video Tokenization via Coordinate-based Patch Reconstruction
Our experiments show that CoordTok can drastically reduce the number of tokens for encoding long video clips.
Coarse-to-fine Q-Network with Action Sequence for Data-Efficient Robot Learning
Motivated by this, we introduce Coarse-to-fine Q-Network with Action Sequence (CQN-AS), a novel value-based RL algorithm that learns a critic network that outputs Q-values over a sequence of actions, i. e., explicitly training the value function to learn the consequence of executing action sequences.
Prioritized Generative Replay
In this work, we instead propose a prioritized, parametric version of an agent's memory, using generative models to capture online experience.
Cliqueformer: Model-Based Optimization with Structured Transformers
Large neural networks excel at prediction tasks, but their application to design problems, such as protein engineering or materials discovery, requires solving offline model-based optimization (MBO) problems.
One Step Diffusion via Shortcut Models
We introduce shortcut models, a family of generative models that use a single network and training phase to produce high-quality samples in a single or multiple sampling steps.
ElasticTok: Adaptive Tokenization for Image and Video
Efficient video tokenization remains a key bottleneck in learning general purpose vision models that are capable of processing long video sequences.
Hand-Object Interaction Pretraining from Videos
We present an approach to learn general robot manipulation priors from 3D hand-object interaction trajectories.
Body Transformer: Leveraging Robot Embodiment for Policy Learning
In recent years, the transformer architecture has become the de facto standard for machine learning algorithms applied to natural language processing and computer vision.
Semi-Supervised One-Shot Imitation Learning
We utilize this embedding space and the clustering it supports to self-generate pairings between trajectories in the large unpaired dataset.
Offline Imitation Learning Through Graph Search and Retrieval
As a result, a robot has to learn skills from suboptimal demonstrations and unstructured interactions, which remains a key challenge.
Chip Placement with Diffusion Models
Macro placement is a vital step in digital circuit design that defines the physical location of large collections of components, known as macros, on a 2D chip.
Visual Representation Learning with Stochastic Frame Prediction
To tackle this challenge, in this paper, we revisit the idea of stochastic video generation that learns to capture uncertainty in frame prediction and explore its effectiveness for representation learning.
From LLMs to Actions: Latent Codes as Bridges in Hierarchical Robot Control
This enables LLMs to flexibly communicate goals in the task plan without being entirely constrained by language limitations.
Ranked #9 on
Robot Manipulation
on CALVIN
HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation
Humanoid robots hold great promise in assisting humans in diverse environments and tasks, due to their flexibility and adaptability leveraging human-like morphology.
Closing the Visual Sim-to-Real Gap with Object-Composable NeRFs
Deep learning methods for perception are the cornerstone of many robotic systems.
MOKA: Open-World Robotic Manipulation through Mark-Based Visual Prompting
Central to our approach is a compact point-based representation of affordance, which bridges the VLM's predictions on observed images and the robot's actions in the physical world.
Twisting Lids Off with Two Hands
Manipulating objects with two multi-fingered hands has been a long-standing challenge in robotics, due to the contact-rich nature of many manipulation tasks and the complexity inherent in coordinating a high-dimensional bimanual system.
Video as the New Language for Real-World Decision Making
Moreover, we demonstrate how, like language models, video generation can serve as planners, agents, compute engines, and environment simulators through techniques such as in-context learning, planning and reinforcement learning.
Unsupervised Zero-Shot Reinforcement Learning via Functional Reward Encodings
Can we pre-train a generalist agent from a large amount of unlabeled offline trajectories such that it can be immediately adapted to any new downstream tasks in a zero-shot manner?
A StrongREJECT for Empty Jailbreaks
To create a benchmark, researchers must choose a dataset of forbidden prompts to which a victim model will respond, along with an evaluation method that scores the harmfulness of the victim model's responses.
World Model on Million-Length Video And Language With Blockwise RingAttention
To address these challenges, we curate a large dataset of diverse videos and books, utilize the Blockwise RingAttention technique to scalably train on long sequences, and gradually increase context size from 4K to 1M tokens.
Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control
Going beyond focusing on a single locomotion skill, we develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing.
Functional Graphical Models: Structure Enables Offline Data-Driven Optimization
This kind of data-driven optimization (DDO) presents a range of challenges beyond those in standard prediction problems, since we need models that successfully predict the performance of new designs that are better than the best designs seen in the training set.
Any-point Trajectory Modeling for Policy Learning
Learning from demonstration is a powerful method for teaching robots new skills, and having more demonstration data often improves policy learning.
Learning a Diffusion Model Policy from Rewards via Q-Score Matching
However, previous works fail to exploit the score-based structure of diffusion models, and instead utilize a simple behavior cloning term to train the actor, limiting their ability in the actor-critic setting.
Motion-Conditioned Image Animation for Video Editing
We introduce MoCA, a Motion-Conditioned Image Animation approach for video editing.
AlberDICE: Addressing Out-Of-Distribution Joint Actions in Offline Multi-Agent RL via Alternating Stationary Distribution Correction Estimation
To this end, we introduce AlberDICE, an offline MARL algorithm that alternatively performs centralized training of individual agents based on stationary distribution optimization.
DreamSmooth: Improving Model-based Reinforcement Learning via Reward Smoothing
Model-based reinforcement learning (MBRL) has gained much attention for its ability to learn complex behaviors in a sample-efficient way: planning actions by generating imaginary trajectories with predicted rewards.
Model-based Reinforcement Learning
reinforcement-learning
+1
Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game
Our benchmark results show that many models are vulnerable to the attack strategies in the Tensor Trust dataset.
The Power of the Senses: Generalizable Manipulation from Vision and Touch through Masked Multimodal Learning
For tasks requiring object manipulation, we seamlessly and effectively exploit the complementarity of our senses of vision and touch.
Managing extreme AI risks amid rapid progress
Artificial Intelligence (AI) is progressing rapidly, and companies are shifting their focus to developing generalist AI systems that can autonomously act and pursue goals.
Scalable Diffusion for Materials Generation
Lastly, we show that conditional generation with UniMat can scale to previously established crystal datasets with up to millions of crystals structures, outperforming random structure search (the current leading method for structure discovery) in discovering new stable materials.
Video Language Planning
We are interested in enabling visual planning for complex long-horizon tasks in the space of generated videos and language, leveraging recent advances in large generative models pretrained on Internet-scale data.
Interactive Task Planning with Language Models
To tackle this, we propose a simple framework that achieves interactive task planning with language models by incorporating both high-level planning and low-level skill execution through function calling, leveraging pretrained vision models to ground the scene in language.
Exploration with Principles for Diverse AI Supervision
Training large transformers using next-token prediction has given rise to groundbreaking advancements in AI.
Reinforcement Learning (RL)
Unsupervised Reinforcement Learning
Learning Interactive Real-World Simulators
Applications of a real-world simulator range from controllable content creation in games and movies, to training embodied agents purely in simulation that can be directly deployed in the real world.
Reinforcement Learning with Foundation Priors: Let the Embodied Agent Efficiently Learn on Its Own
We propose Reinforcement Learning with Foundation Priors (RLFP) to utilize guidance and feedback from policy, value, and success-reward foundation models.
Ring Attention with Blockwise Transformers for Near-Infinite Context
Transformers have emerged as the architecture of choice for many state-of-the-art AI models, showcasing exceptional performance across a wide range of AI applications.
Speed Co-Augmentation for Unsupervised Audio-Visual Pre-training
This work aims to improve unsupervised audio-visual pre-training.
Language Reward Modulation for Pretraining Reinforcement Learning
Using learned reward functions (LRFs) as a means to solve sparse-reward reinforcement learning (RL) tasks has yielded some steady progress in task-complexity through the years.
Convolutional Occupancy Models for Dense Packing of Complex, Novel Objects
Dense packing in pick-and-place systems is an important feature in many warehouse and logistics applications.
Learning to Model the World with Language
While current agents can learn to execute simple language instructions, we aim to build agents that leverage diverse language -- language like "this button turns on the TV" or "I put the bowls away" -- that conveys general knowledge, describes the state of the world, provides interactive feedback, and more.
SpawnNet: Learning Generalizable Visuomotor Skills from Pre-trained Networks
In this work, we present a focused study of the generalization capabilities of the pre-trained visual representations at the categorical level.
Improving Long-Horizon Imitation Through Instruction Prediction
Complex, long-horizon planning and its combinatorial nature pose steep challenges for learning-based agents.
ALP: Action-Aware Embodied Learning for Perception
In addition, we show that by training on actively collected data more relevant to the environment and task, our method generalizes more robustly to downstream tasks compared to models pre-trained on fixed datasets such as ImageNet.
Probabilistic Adaptation of Text-to-Video Models
Large text-to-video models trained on internet-scale data have demonstrated exceptional capabilities in generating high-fidelity videos from arbitrary textual descriptions.
Train Offline, Test Online: A Real Robot Learning Benchmark
Three challenges limit the progress of robot learning research: robots are expensive (few labs can participate), everyone uses different robots (findings do not generalize across labs), and we lack internet-scale robotics data.
Accelerating Reinforcement Learning with Value-Conditional State Entropy Exploration
A promising technique for exploration is to maximize the entropy of visited state distribution, i. e., state entropy, by encouraging uniform coverage of visited state space.
Blockwise Parallel Transformer for Large Context Models
Transformers have emerged as the cornerstone of state-of-the-art natural language processing models, showcasing exceptional performance across a wide range of AI applications.
Emergent Agentic Transformer from Chain of Hindsight Experience
Our method consists of relabelling target return of each trajectory to the maximum total reward among in sequence of trajectories and training an autoregressive model to predict actions conditioning on past states, actions, rewards, target returns, and task completion tokens, the resulting model, Agentic Transformer (AT), can learn to improve upon itself both at training and test time.
DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models
We focus on diffusion models, defining the fine-tuning task as an RL problem, and updating the pre-trained text-to-image diffusion models using policy gradient to maximize the feedback-trained reward.
The False Promise of Imitating Proprietary LLMs
This approach looks to cheaply imitate the proprietary model's capabilities using a weaker open-source model.
Self-Supervised Instance Segmentation by Grasping
Leveraging this insight, we learn a grasp segmentation model to segment the grasped object from before and after grasp images.
Masked Trajectory Models for Prediction, Representation, and Control
We introduce Masked Trajectory Models (MTM) as a generic abstraction for sequential decision making.
Distributional Instance Segmentation: Modeling Uncertainty and High Confidence Predictions with Latent-MaskRCNN
Existing state-of-the-art methods are often unable to capture meaningful uncertainty in challenging or ambiguous scenes, and as such can cause critical errors in high-performance applications.
RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learning
Replicating human-like dexterity in robot hands represents one of the largest open problems in robotics.
Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?
Contrary to inferences from prior work, we find that scaling dataset size and diversity does not improve performance universally (but does so on average).
Foundation Models for Decision Making: Problems, Methods, and Opportunities
In response to these developments, new paradigms are emerging for training foundation models to interact with other agents and perform long-term reasoning.
Preference Transformer: Modeling Human Preferences using Transformers for RL
In this paper, we present Preference Transformer, a neural architecture that models human preferences using transformers.
Aligning Text-to-Image Models using Human Feedback
Our results demonstrate the potential for learning from human feedback to significantly improve text-to-image models.
Robust and Versatile Bipedal Jumping Control through Reinforcement Learning
This work aims to push the limits of agility for bipedal robots by enabling a torque-controlled bipedal robot to perform robust and versatile dynamic jumps in the real world.
Guiding Pretraining in Reinforcement Learning with Large Language Models
Reinforcement learning algorithms typically struggle in the absence of a dense, well-shaped reward function.
Controllability-Aware Unsupervised Skill Discovery
One of the key capabilities of intelligent agents is the ability to discover useful skills without external supervision.
The Wisdom of Hindsight Makes Language Models Better Instruction Followers
In this paper, we consider an alternative approach: converting feedback to instruction by relabeling the original one and training the model for better alignment in a supervised manner.
Chain of Hindsight Aligns Language Models with Feedback
Applying our method to large language models, we observed that Chain of Hindsight significantly surpasses previous methods in aligning language models with human preferences.
Multi-View Masked World Models for Visual Robotic Manipulation
In this paper, we investigate how to learn good representations with multi-view data and utilize them for visual robotic manipulation.
Language Quantized AutoEncoders: Towards Unsupervised Text-Image Alignment
Recent progress in scaling up large language models has shown impressive capabilities in performing few-shot learning across a wide range of text-based tasks.
Masked Autoencoding for Scalable and Generalizable Decision Making
We are interested in learning scalable agents for reinforcement learning that can learn from large-scale, diverse sequential data similar to current large vision and language models.
Multi-Environment Pretraining Enables Transfer to Action Limited Datasets
Using massive datasets to train large-scale models has emerged as a dominant approach for broad generalization in natural language and vision applications.
VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models
We show that a text-conditioned diffusion model trained on pixel representations of images can be used to generate SVG-exportable vector graphics.
StereoPose: Category-Level 6D Transparent Object Pose Estimation from Stereo Images via Back-View NOCS
To further improve the performance of the stereo framework, StereoPose is equipped with a parallax attention module for stereo feature fusion and an epipolar loss for improving the stereo-view consistency of network predictions.
Sim-to-Real via Sim-to-Seg: End-to-end Off-road Autonomous Driving Without Real Data
In this paper, we address this challenge by presenting Sim2Seg, a re-imagining of RCAN that crosses the visual reality gap for off-road autonomous driving, without using any real-world data.
Instruction-Following Agents with Multimodal Transformer
Our \ours method consists of a multimodal transformer that encodes visual observations and language instructions, and a transformer-based policy that predicts actions based on encoded representations.
Towards Better Few-Shot and Finetuning Performance with Forgetful Causal Language Models
Large language models (LLM) trained using the next-token-prediction objective, such as GPT3 and PaLM, have revolutionized natural language processing in recent years by showing impressive zero-shot and few-shot capabilities across a wide range of tasks.
Dichotomy of Control: Separating What You Can Control from What You Cannot
While return-conditioning is at the heart of popular algorithms such as decision transformer (DT), these methods tend to perform poorly in highly stochastic environments, where an occasional high return can arise from randomness in the environment rather than the actions themselves.
Spending Thinking Time Wisely: Accelerating MCTS with Virtual Expansions
This paper proposes the Virtual MCTS (V-MCTS), a variant of MCTS that spends more search time on harder states and less search time on simpler states adaptively.
CLUTR: Curriculum Learning via Unsupervised Task Representation Learning
Recently, Unsupervised Environment Design (UED) emerged as a new paradigm for zero-shot generalization by simultaneously learning a task distribution and agent policies on the generated tasks.
Skill-Based Reinforcement Learning with Intrinsic Reward Matching
While unsupervised skill discovery has shown promise in autonomously acquiring behavioral primitives, there is still a large methodological disconnect between task-agnostic skill pretraining and downstream, task-aware finetuning.
Autoregressive Uncertainty Modeling for 3D Bounding Box Prediction
3D bounding boxes are a widespread intermediate representation in many computer vision applications.
Real-World Robot Learning with Masked Visual Pre-training
Finally, we train a 307M parameter vision transformer on a massive collection of 4. 5M images from the Internet and egocentric videos, and demonstrate clearly the benefits of scaling visual pre-training for robot learning.
Temporally Consistent Transformers for Video Generation
To generate accurate videos, algorithms have to understand the spatial and temporal dependencies in the world.
Reducing Variance in Temporal-Difference Value Estimation via Ensemble of Deep Networks
On a set of 26 benchmark Atari environments, MeanQ outperforms all tested baselines, including the best available baseline, SUNRISE, at 100K interaction steps in 16/26 environments, and by 68% on average.
Multi-Objective Policy Gradients with Topological Constraints
Multi-objective optimization models that encode ordered sequential constraints provide a solution to model various challenging problems including encoding preferences, modeling a curriculum, and enforcing measures of safety.
HARP: Autoregressive Latent Video Prediction with High-Fidelity Image Generator
Video prediction is an important yet challenging problem; burdened with the tasks of generating future frames and learning environment dynamics.
AdaCat: Adaptive Categorical Discretization for Autoregressive Models
Autoregressive generative models can estimate complex continuous data distributions, like trajectory rollouts in an RL environment, image intensities, and audio.
Fleet-DAgger: Interactive Robot Fleet Learning with Scalable Human Supervision
With continual learning, interventions from the remote pool of humans can also be used to improve the robot fleet control policy over time.
Masked World Models for Visual Control
Yet the current approaches typically train a single model end-to-end for learning both visual representations and dynamics, making it difficult to accurately model the interaction between robots and small objects.
Model-based Reinforcement Learning
Reinforcement Learning (RL)
+1
DayDreamer: World Models for Physical Robot Learning
Learning a world model to predict the outcomes of potential actions enables planning in imagination, reducing the amount of trial and error needed in the real environment.
Patch-based Object-centric Transformers for Efficient Video Generation
In this work, we present Patch-based Object-centric Video Transformer (POVT), a novel region-based video generation architecture that leverages object-centric information to efficiently model temporal dynamics in videos.
Deep Hierarchical Planning from Pixels
Despite operating in latent space, the decisions are interpretable because the world model can decode goals into images for visualization.
On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learning
From these findings, we advocate for evaluating future meta-RL methods on more challenging tasks and including multi-task pretraining with fine-tuning as a simple, yet strong baseline.
Multimodal Masked Autoencoders Learn Transferable Representations
We provide an empirical study of M3AE trained on a large-scale image-text dataset, and find that M3AE is able to learn generalizable representations that transfer well to downstream tasks.
Reward Uncertainty for Exploration in Preference-based Reinforcement Learning
Our intuition is that disagreement in learned reward model reflects uncertainty in tailored human feedback and could be useful for exploration.
Chain of Thought Imitation with Procedure Cloning
Imitation learning aims to extract high-performance policies from logged demonstrations of expert behavior.
An Empirical Investigation of Representation Learning for Imitation
We propose a modular framework for constructing representation learning algorithms, then use our framework to evaluate the utility of representation learning for imitation across several environment suites.
Coarse-to-fine Q-attention with Tree Expansion
Coarse-to-fine Q-attention enables sample-efficient robot manipulation by discretizing the translation space in a coarse-to-fine manner, where the resolution gradually increases at each layer in the hierarchy.
Sim-to-Real 6D Object Pose Estimation via Iterative Self-training for Robotic Bin Picking
To continuously improve the quality of pseudo labels, we iterate the above steps by taking the trained student model as a new teacher and re-label real data using the refined teacher model.
Imitating, Fast and Slow: Robust learning from demonstrations via decision-time planning
The goal of imitation learning is to mimic expert behavior from demonstrations, without access to an explicit reward signal.
Coarse-to-Fine Q-attention with Learned Path Ranking
We propose Learned Path Ranking (LPR), a method that accepts an end-effector goal pose, and learns to rank a set of goal-reaching paths generated from an array of path generating methods, including: path planning, Bezier curve sampling, and a learned policy.
Pretraining Graph Neural Networks for few-shot Analog Circuit Modeling and Design
We show that pretraining GNNs on prediction of output node voltages can encourage learning representations that can be adapted to new unseen topologies or prediction of new circuit level properties with up to 10x more sample efficiency compared to a randomly initialized model.
Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions
We also demonstrate that an effective style reward can be learned from a few seconds of motion capture data gathered from a German Shepherd and leads to energy-efficient locomotion strategies with natural gait transitions.
Reinforcement Learning with Action-Free Pre-Training from Videos
Our framework consists of two phases: we pre-train an action-free latent video prediction model, and then utilize the pre-trained representations for efficiently learning action-conditional world models on unseen environments.
Teachable Reinforcement Learning via Advice Distillation
Training automated agents to complete complex tasks in interactive environments is challenging: reinforcement learning requires careful hand-engineering of reward functions, imitation learning requires specialized infrastructure and access to a human expert, and learning from intermediate forms of supervision (like binary preferences) is time-consuming and extracts little information from each human intervention.
SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learning
In order to leverage unlabeled samples for reward learning, we infer pseudo-labels of the unlabeled samples based on the confidence of the preference predictor.
It Takes Four to Tango: Multiagent Selfplay for Automatic Curriculum Generation
Training such agents efficiently requires automatic generation of a goal curriculum.
Bingham Policy Parameterization for 3D Rotations in Reinforcement Learning
We propose a new policy parameterization for representing 3D rotations during reinforcement learning.
CIC: Contrastive Intrinsic Control for Unsupervised Skill Discovery
We introduce Contrastive Intrinsic Control (CIC), an algorithm for unsupervised skill discovery that maximizes the mutual information between state-transitions and latent skill vectors.
Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learning
In this work, we propose Exploratory data for Offline RL (ExORL), a data-centric approach to offline RL.
Explaining Reinforcement Learning Policies through Counterfactual Trajectories
In order for humans to confidently decide where to employ RL agents for real-world tasks, a human developer must validate that the agent will perform well at test-time.
Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents
However, the plans produced naively by LLMs often cannot map precisely to admissible actions.
Target Entropy Annealing for Discrete Soft Actor-Critic
Soft Actor-Critic (SAC) is considered the state-of-the-art algorithm in continuous action space settings.
Zero-Shot Text-Guided Object Generation with Dream Fields
Our method, Dream Fields, can generate the geometry and color of a wide range of objects without 3D supervision.
Hindsight Task Relabelling: Experience Replay for Sparse Reward Meta-RL
Meta-reinforcement learning (meta-RL) has proven to be a successful framework for leveraging experience from prior tasks to rapidly learn new related tasks, however, current meta-RL approaches struggle to learn in sparse reward environments.
Count-Based Temperature Scheduling for Maximum Entropy Reinforcement Learning
Maximum Entropy Reinforcement Learning (MaxEnt RL) algorithms such as Soft Q-Learning (SQL) and Soft Actor-Critic trade off reward and policy entropy, which has the potential to improve training stability and robustness.
Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning
We show that a single generalist policy can perform in-hand manipulation of over 100 geometrically-diverse real-world objects and generalize to new objects with unseen shape or size.
B-Pref: Benchmarking Preference-Based Reinforcement Learning
However, it is difficult to quantify the progress in preference-based RL due to the lack of a commonly adopted benchmark.
Mastering Atari Games with Limited Data
Recently, there has been significant progress in sample efficient image-based RL algorithms; however, consistent human-level performance on the Atari game benchmark remains an elusive goal.
Ranked #2 on
Atari Games 100k
on Atari 100k
URLB: Unsupervised Reinforcement Learning Benchmark
Deep Reinforcement Learning (RL) has emerged as a powerful paradigm to solve a range of complex yet specific control tasks.
Temporal-Difference Value Estimation via Uncertainty-Guided Soft Updates
Under the belief that $\beta$ is closely related to the (state dependent) model uncertainty, Entropy Regularized Q-Learning (EQL) further introduces a principled scheduling of $\beta$ by maintaining a collection of the model parameters that characterizes model uncertainty.
Towards More Generalizable One-shot Visual Imitation Learning
We then study the multi-task setting, where multi-task training is followed by (i) one-shot imitation on variations within the training tasks, (ii) one-shot imitation on new tasks, and (iii) fine-tuning on new tasks.
It Takes Four to Tango: Multiagent Self Play for Automatic Curriculum Generation
We are interested in training general-purpose reinforcement learning agents that can solve a wide variety of goals.
Improving Long-Horizon Imitation Through Language Prediction
Complex, long-horizon planning and its combinatorial nature pose steep challenges for learning-based agents.
Pretraining for Language Conditioned Imitation with Transformers
We study reinforcement learning (RL) agents which can utilize language inputs.
Semi-supervised Offline Reinforcement Learning with Pre-trained Decision Transformers
In this paper, we investigate how we can leverage large reward-free (i. e. task-agnostic) offline datasets of prior interactions to pre-train agents that can then be fine-tuned using a small reward-annotated dataset.
Autoregressive Latent Video Prediction with High-Fidelity Image Generator
Video prediction is an important yet challenging problem; burdened with the tasks of generating future frames and learning environment dynamics.
APS: Active Pretraining with Successor Features
We introduce a new unsupervised pretraining objective for reinforcement learning.
Skill Preferences: Learning to Extract and Execute Robotic Skills from Human Feedback
A promising approach to solving challenging long-horizon tasks has been to extract behavior priors (skills) by fitting generative models to large offline datasets of demonstrations.
Playful Interactions for Representation Learning
In this work, we propose to use playful interactions in a self-supervised manner to learn visual representations for downstream tasks.
Hierarchical Few-Shot Imitation with Skill Transition Models
To this end, we present Few-shot Imitation with Skill Transition Models (FIST), an algorithm that extracts skills from offline data and utilizes them to generalize to unseen tasks given a few downstream demonstrations.
The MineRL BASALT Competition on Learning from Human Feedback
Rather than training AI systems using a predefined reward function or using a labeled dataset with a predefined set of categories, we instead train the AI system using a learning signal derived from some form of human feedback, which can evolve over time as the understanding of the task changes, or as the capabilities of the AI system improve.
Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemble
Recent advance in deep offline reinforcement learning (RL) has made it possible to train strong robotic agents from offline datasets.
Scenic4RL: Programmatic Modeling and Generation of Reinforcement Learning Environments
To showcase the benefits, we interfaced SCENIC to an existing RTS environment Google Research Football(GRF) simulator and introduced a benchmark consisting of 32 realistic scenarios, encoded in SCENIC, to train RL agents and testing their generalization capabilities.
Behavioral Priors and Dynamics Models: Improving Performance and Domain Transfer in Offline RL
When combined together, they substantially improve the performance and generalization of offline RL policies.
Unsupervised Learning of Visual 3D Keypoints for Control
Prior works show that structured latent space such as visual keypoints often outperforms unstructured representations for robotic control.
Data-Efficient Exploration with Self Play for Atari
To alleviate the reliance on reward engineering it is important to develop RL algorithms capable of efficiently acquiring skills with no rewards extrinsic to the agent.
PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training
We also show that our method is able to utilize real-time human feedback to effectively prevent reward exploitation and learn new behaviors that are difficult to specify with standard reward functions.
JUMBO: Scalable Multi-task Bayesian Optimization using Offline Data
Such a decomposition can dynamically control the reliability of information derived from the online and offline data and the use of pretrained neural networks permits scalability to large offline datasets.
Decision Transformer: Reinforcement Learning via Sequence Modeling
In particular, we present Decision Transformer, an architecture that casts the problem of RL as conditional sequence modeling.
Ranked #3 on
Offline RL
on D4RL
VideoGPT: Video Generation using VQ-VAE and Transformers
We present VideoGPT: a conceptually simple architecture for scaling likelihood based generative modeling to natural videos.
Auto-Tuned Sim-to-Real Transfer
Policies trained in simulation often fail when transferred to the real world due to the `reality gap' where the simulator is unable to accurately capture the dynamics and visual properties of the real world.
Learning What To Do by Simulating the Past
Since reward functions are hard to specify, recent work has focused on learning policies from human feedback.
GEM: Group Enhanced Model for Learning Dynamical Control Systems
In this work, we take advantage of these structures to build effective dynamical models that are amenable to sample-based learning.
AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control
Our system produces high-quality motions that are comparable to those achieved by state-of-the-art tracking-based techniques, while also being able to easily accommodate large datasets of unstructured motion clips.
Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis
We present DietNeRF, a 3D neural scene representation estimated from a few images.
Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots
Developing robust walking controllers for bipedal robots is a challenging endeavor.
Mutual Information State Intrinsic Control
Reinforcement learning has been shown to be highly successful at many challenging tasks.
Pretrained Transformers as Universal Computation Engines
We investigate the capability of a transformer pretrained on natural language to generalize to other modalities with minimal finetuning -- in particular, without finetuning of the self-attention and feedforward layers of the residual blocks.
Behavior From the Void: Unsupervised Active Pre-Training
We introduce a new unsupervised pre-training method for reinforcement learning called APT, which stands for Active Pre-Training.
Improving Computational Efficiency in Visual Reinforcement Learning via Stored Embeddings
Recent advances in off-policy deep reinforcement learning (RL) have led to impressive success in complex tasks from visual observations.
Ranked #33 on
Atari Games
on Atari 2600 Amidar
Task-Agnostic Morphology Evolution
Deep reinforcement learning primarily focuses on learning behavior, usually overlooking the fact that an agent's function is largely determined by form.
State Entropy Maximization with Random Encoders for Efficient Exploration
Recent exploration methods have proven to be a recipe for improving sample-efficiency in deep reinforcement learning (RL).
MSA Transformer
Unsupervised protein language models trained across millions of diverse sequences learn structure and function of proteins.
Bottleneck Transformers for Visual Recognition
Finally, we present a simple adaptation of the BoTNet design for image classification, resulting in models that achieve a strong performance of 84. 7% top-1 accuracy on the ImageNet benchmark while being up to 1. 64x faster in compute time than the popular EfficientNet models on TPU-v3 hardware.
Ranked #54 on
Instance Segmentation
on COCO minival
Reinforcement Learning with Latent Flow
Temporal information is essential to learning effective policies with Reinforcement Learning (RL).
Compute- and Memory-Efficient Reinforcement Learning with Latent Experience Replay
In this paper, we present Latent Vector Experience Replay (LeVER), a simple modification of existing off-policy RL methods, to address these computational and memory requirements without sacrificing the performance of RL agents.
Benefits of Assistance over Reward Learning
By merging reward learning and control, assistive agents can reason about the impact of control actions on reward learning, leading to several advantages over agents based on reward learning.
Weighted Bellman Backups for Improved Signal-to-Noise in Q-Updates
Furthermore, since our weighted Bellman backups rely on maintaining an ensemble, we investigate how weighted Bellman backups interact with other benefits previously derived from ensembles: (a) Bootstrap; (b) UCB Exploration.
Discrete Predictive Representation for Long-horizon Planning
Discrete representations have been key in enabling robots to plan at more abstract levels and solve temporally-extended tasks more efficiently for decades.
Addressing Distribution Shift in Online Reinforcement Learning with Offline Datasets
As it turns out, fine-tuning offline RL agents is a non-trivial challenge, due to distribution shift – the agent encounters out-of-distribution samples during online interaction, which may cause bootstrapping error in Q-learning and instability during fine-tuning.
VideoGen: Generative Modeling of Videos using VQ-VAE and Transformers
We present VideoGen: a conceptually simple architecture for scaling likelihood based generative modeling to natural videos.
Unsupervised Active Pre-Training for Reinforcement Learning
On DMControl suite, APT beats all baselines in terms of asymptotic performance and data efficiency and dramatically improves performance on tasks that are extremely difficult for training from scratch.
Robust Imitation via Decision-Time Planning
The goal of imitation learning is to mimic expert behavior from demonstrations, without access to an explicit reward signal.
R-LAtte: Attention Module for Visual Control via Reinforcement Learning
Attention mechanisms are generic inductive biases that have played a critical role in improving the state-of-the-art in supervised learning, unsupervised pre-training and generative modeling for multiple domains including vision, language and speech.
Learning Visual Robotic Control Efficiently with Contrastive Pre-training and Data Augmentation
We present Contrastive Pre-training and Data Augmentation for Efficient Robotic Learning (CoDER), a method that utilizes data augmentation and unsupervised learning to achieve sample-efficient training of real-robot arm policies from sparse rewards.
Reset-Free Lifelong Learning with Skill-Space Planning
We propose Lifelong Skill Planning (LiSP), an algorithmic framework for non-episodic lifelong RL based on planning in an abstract space of higher-order skills.
Parallel Training of Deep Networks with Local Updates
Deep learning models trained on large data sets have been widely successful in both vision and language domains.
Trajectory-wise Multiple Choice Learning for Dynamics Generalization in Reinforcement Learning
Model-based reinforcement learning (RL) has shown great potential in various control tasks in terms of both sample-efficiency and final performance.
LaND: Learning to Navigate from Disengagements
However, we believe that these disengagements not only show where the system fails, which is useful for troubleshooting, but also provide a direct learning signal by which the robot can learn to navigate.
Decoupling Representation Learning from Reinforcement Learning
In an effort to overcome limitations of reward-driven feature learning in deep reinforcement learning (RL) from images, we propose decoupling representation learning from policy learning.
Visual Imitation Made Easy
We use commercially available reacher-grabber assistive tools both as a data collection device and as the robot's end-effector.
Robust Reinforcement Learning using Adversarial Populations
Reinforcement Learning (RL) is an effective tool for controller design but can struggle with issues of robustness, failing catastrophically when the underlying system dynamics are perturbed.
Out-of-Distribution Generalization
reinforcement-learning
+2
Dynamics Generalization via Information Bottleneck in Deep Reinforcement Learning
Despite the significant progress of deep reinforcement learning (RL) in solving sequential decision making problems, RL agents often overfit to training environments and struggle to adapt to new, unseen environments.
Hybrid Discriminative-Generative Training via Contrastive Learning
In this paper we show that through the perspective of hybrid discriminative-generative training of energy-based models we can make a direct connection between contrastive learning and supervised learning.
Efficient Empowerment Estimation for Unsupervised Stabilization
We demonstrate our solution for sample-based unsupervised stabilization on different dynamical control systems and show the advantages of our method by comparing it to the existing VLB approaches.
Variable Skipping for Autoregressive Range Density Estimation
In this paper, we explore a technique, variable skipping, for accelerating range density estimation over deep autoregressive models.
SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learning
Off-policy deep reinforcement learning (RL) has been successful in a range of challenging domains.
