Search Results for author:
Found 60 papers, 27 papers with code
Automated Discovery of Functional Actual Causes in Complex Environments
This paper introduces functional actual cause (FAC), a framework that uses context-specific independencies in the environment to restrict the set of actual causes.
Learning Action-based Representations Using Invariance
Robust reinforcement learning agents using high-dimensional observations must be able to identify relevant state features amidst many exogeneous distractors.
Multistep Inverse Is Not All You Need
In this work, we consider the Ex-BMDP model, first proposed by Efroni et al. (2022), which formalizes control problems where observations can be factorized into an action-dependent latent state which evolves deterministically, and action-independent time-correlated noise.
Diffusion World Model
We introduce Diffusion World Model (DWM), a conditional diffusion model capable of predicting multistep future states and rewards concurrently.
Zero-Shot Reinforcement Learning via Function Encoders
Although reinforcement learning (RL) can solve many challenging sequential decision making problems, achieving zero-shot transfer across related tasks remains a challenge.
An Investigation of Time Reversal Symmetry in Reinforcement Learning
We observe that utilizing the structure of time reversal in an MDP allows every environment transition experienced by an agent to be transformed into a feasible reverse-time transition, effectively doubling the number of experiences in the environment.
Accelerating Exploration with Unlabeled Prior Data
Learning to solve tasks from a sparse reward signal is a major challenge for standard reinforcement learning (RL) algorithms.
SMORE: Score Models for Offline Goal-Conditioned Reinforcement Learning
Offline Goal-Conditioned Reinforcement Learning (GCRL) is tasked with learning to achieve multiple goals in an environment purely from offline datasets using sparse reward functions.
Towards Robust Offline Reinforcement Learning under Diverse Data Corruption
Offline reinforcement learning (RL) presents a promising approach for learning reinforced policies from offline datasets without the need for costly or unsafe interactions with the environment.
$f$-Policy Gradients: A General Framework for Goal Conditioned RL using $f$-Divergences
We further introduce an entropy-regularized policy optimization objective, that we call $state$-MaxEnt RL (or $s$-MaxEnt RL) as a special case of our objective.
Motif: Intrinsic Motivation from Artificial Intelligence Feedback
Exploring rich environments and evaluating one's actions without prior knowledge is immensely challenging.
Confidence Contours: Uncertainty-Aware Annotation for Medical Semantic Segmentation
However, as these annotations cannot represent an individual annotator's uncertainty, models trained on them produce uncertainty maps that are difficult to interpret.
Structure in Reinforcement Learning: A Survey and Open Problems
We amalgamate these diverse methodologies under a unified framework, shedding light on the role of structure in the learning problem, and classify these methods into distinct patterns of incorporating structure.
Generalization Across Observation Shifts in Reinforcement Learning
We focus on bisimulation metrics, which provide a powerful means for abstracting task relevant components of the observation and learning a succinct representation space for training the agent using reinforcement learning.
LIV: Language-Image Representations and Rewards for Robotic Control
We present Language-Image Value learning (LIV), a unified objective for vision-language representation and reward learning from action-free videos with text annotations.
A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problem
Training multiple agents to coordinate is an essential problem with applications in robotics, game theory, economics, and social sciences.
When should we prefer Decision Transformers for Offline Reinforcement Learning?
Three popular algorithms for offline RL are Conservative Q-Learning (CQL), Behavior Cloning (BC), and Decision Transformer (DT), from the class of Q-Learning, Imitation Learning, and Sequence Modeling respectively.
Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learning
In goal-reaching reinforcement learning (RL), the optimal value function has a particular geometry, called quasimetric structure.
BC-IRL: Learning Generalizable Reward Functions from Demonstrations
How well do reward functions learned with inverse reinforcement learning (IRL) generalize?
Neural Constraint Satisfaction: Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement
Object rearrangement is a challenge for embodied agents because solving these tasks requires generalizing across a combinatorially large set of configurations of entities and their locations.
Confidence-aware 3D Gaze Estimation and Evaluation Metric
To address the unreliable and overconfident issues, we introduce a confidence-aware model that predicts uncertainties together with gaze angle estimations.
Dual RL: Unification and New Methods for Reinforcement and Imitation Learning
For offline RL, our analysis frames a recent offline RL method XQL in the dual framework, and we further propose a new method f-DVL that provides alternative choices to the Gumbel regression loss that fixes the known training instability issue of XQL.
Contrastive Distillation Is a Sample-Efficient Self-Supervised Loss Policy for Transfer Learning
Traditional approaches to RL have focused on learning decision policies directly from episodic decisions, while slowly and implicitly learning the semantics of compositional representations needed for generalization.
Language Control Diffusion: Efficiently Scaling through Space, Time, and Tasks
Training generalist agents is difficult across several axes, requiring us to deal with high-dimensional inputs (space), long horizons (time), and generalization to novel tasks.
Latent State Marginalization as a Low-cost Approach for Improving Exploration
While the maximum entropy (MaxEnt) reinforcement learning (RL) framework -- often touted for its exploration and robustness capabilities -- is usually motivated from a probabilistic perspective, the use of deep probabilistic models has not gained much traction in practice due to their inherent complexity.
VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training
Given the inherent cost and scarcity of in-domain, task-specific robot data, learning from large, diverse, offline human videos has emerged as a promising path towards acquiring a generally useful visual representation for control; however, how these human videos can be used for general-purpose reward learning remains an open question.
Building Human Values into Recommender Systems: An Interdisciplinary Synthesis
We collect a set of values that seem most relevant to recommender systems operating across different domains, then examine them from the perspectives of current industry practice, measurement, product design, and policy approaches.
Denoised MDPs: Learning World Models Better Than the World Itself
The ability to separate signal from noise, and reason with clean abstractions, is critical to intelligence.
Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learning
We learn this representation using a metric form of this abstraction, and show its ability to generalize to new goals in simulation manipulation tasks.
Robust Policy Learning over Multiple Uncertainty Sets
Reinforcement learning (RL) agents need to be robust to variations in safety-critical environments.
Online Decision Transformer
Recent work has shown that offline reinforcement learning (RL) can be formulated as a sequence modeling problem (Chen et al., 2021; Janner et al., 2021) and solved via approaches similar to large-scale language modeling.
A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
This survey is an overview of this nascent field.
Learning Representations for Pixel-based Control: What Matters and Why?
A wide range of methods have been proposed to enable efficient learning, leading to sample complexities similar to those in the full state setting.
Block Contextual MDPs for Continual Learning
In this work, we propose to examine this continual reinforcement learning setting through the block contextual MDP (BC-MDP) framework, which enables us to relax the assumption of stationarity.
Learning Minimal Representations with Model Invariance
The common representation acts as a implicit invariance objective to avoid the different spurious correlations captured by individual predictors.
Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observability
Generalization is a central challenge for the deployment of reinforcement learning (RL) systems in the real world.
Provably Efficient Representation Selection in Low-rank Markov Decision Processes: From Online to Offline RL
For the offline counterpart, ReLEX-LCB, we show that the algorithm can find the optimal policy if the representation class can cover the state-action space and achieves gap-dependent sample complexity.
MBRL-Lib: A Modular Library for Model-based Reinforcement Learning
MBRL-Lib is designed as a platform for both researchers, to easily develop, debug and compare new algorithms, and non-expert user, to lower the entry-bar of deploying state-of-the-art algorithms.
Model-based Reinforcement Learning
reinforcement-learning
+1
Resolving Causal Confusion in Reinforcement Learning via Robust Exploration
To address this, we present a robust exploration strategy which enables causal hypothesis-testing by interaction with the environment.
Model-Invariant State Abstractions for Model-Based Reinforcement Learning
Unlike previous forms of state abstractions, a model-invariance state abstraction leverages causal sparsity over state variables.
Multi-Task Reinforcement Learning with Context-based Representations
We posit that an efficient approach to knowledge transfer is through the use of multiple context-dependent, composable representations shared across a family of tasks.
Invariant Representations for Reinforcement Learning without Reconstruction
We study how representation learning can accelerate reinforcement learning from rich observations, such as images, without relying either on domain knowledge or pixel-reconstruction.
Automating Document Classification with Distant Supervision to Increase the Efficiency of Systematic Reviews
Conclusions: In sum, the automated procedure of document classification presented here could improve both the precision and efficiency of systematic reviews, as well as facilitating live reviews, where reviews are updated regularly.
Intervention Design for Effective Sim2Real Transfer
The goal of this work is to address the recent success of domain randomization and data augmentation for the sim2real setting.
Learning Robust State Abstractions for Hidden-Parameter Block MDPs
Further, we provide transfer and generalization bounds based on task and state similarity, along with sample complexity bounds that depend on the aggregate number of samples across tasks, rather than the number of tasks, a significant improvement over prior work that use the same environment assumptions.
Learning Invariant Representations for Reinforcement Learning without Reconstruction
We study how representation learning can accelerate reinforcement learning from rich observations, such as images, without relying either on domain knowledge or pixel-reconstruction.
Plan2Vec: Unsupervised Representation Learning by Latent Plans
In this paper we introduce plan2vec, an unsupervised representation learning approach that is inspired by reinforcement learning.
Invariant Causal Prediction for Block MDPs
Generalization across environments is critical to the successful application of reinforcement learning algorithms to real-world challenges.
Stable Policy Optimization via Off-Policy Divergence Regularization
Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO) are among the most successful policy gradient approaches in deep reinforcement learning (RL).
Out-of-Distribution Generalization via Risk Extrapolation (REx)
Distributional shift is one of the major obstacles when transferring machine learning prediction systems from the lab to the real world.
Improving Sample Efficiency in Model-Free Reinforcement Learning from Images
A promising approach is to learn a latent representation together with the control policy.
Learning Causal State Representations of Partially Observable Environments
In this paper, we propose an algorithm to approximate causal states, which are the coarsest partition of the joint history of actions and observations in partially-observable Markov decision processes (POMDP).
Natural Environment Benchmarks for Reinforcement Learning
While current benchmark reinforcement learning (RL) tasks have been useful to drive progress in the field, they are in many ways poor substitutes for learning with real-world data.
A Dissection of Overfitting and Generalization in Continuous Reinforcement Learning
The risks and perils of overfitting in machine learning are well known.
Decoupling Dynamics and Reward for Transfer Learning
Current reinforcement learning (RL) methods can successfully learn single tasks but often generalize poorly to modest perturbations in task domain or training procedure.
Composable Planning with Attributes
The tasks that an agent will need to solve often are not known during training.
Mapping the world population one building at a time
Obtaining high accuracy in estimation of population distribution in rural areas remains a very challenging task due to the simultaneous requirements of sufficient sensitivity and resolution to detect very sparse populations through remote sensing as well as reliable performance at a global scale.
Building Detection from Satellite Images on a Global Scale
Our work is some of the first to create population density maps from building detection on a large scale.
Feedback Neural Network for Weakly Supervised Geo-Semantic Segmentation
Learning from weakly-supervised data is one of the main challenges in machine learning and computer vision, especially for tasks such as image semantic segmentation where labeling is extremely expensive and subjective.
Guess Who Rated This Movie: Identifying Users Through Subspace Clustering
It is often the case that, within an online recommender system, multiple users share a common account.
