Search Results for author:
Found 61 papers, 25 papers with code
Entity-Centric Reinforcement Learning for Object Manipulation from Pixels
Manipulating objects is a hallmark of human intelligence, and an important task in domains such as robotics.
MAMBA: an Effective World Model Approach for Meta-Reinforcement Learning
Meta-reinforcement learning (meta-RL) is a promising framework for tackling challenging domains requiring efficient exploration.
Fine-Tuning Generative Models as an Inference Method for Robotic Tasks
Adaptable models could greatly benefit robotic agents operating in the real world, allowing them to deal with novel and varying conditions.
TGRL: An Algorithm for Teacher Guided Reinforcement Learning
To combine the benefits of these different forms of learning, it is common to train a policy to maximize a combination of reinforcement and teacher-student learning objectives.
Explore to Generalize in Zero-Shot RL
Our insight is that learning a policy that effectively $\textit{explores}$ the domain is harder to memorize than a policy that maximizes reward for a specific task, and therefore we expect such learned behavior to generalize well; we indeed demonstrate this empirically on several domains that are difficult for invariance-based approaches.
ContraBAR: Contrastive Bayes-Adaptive Deep RL
In meta reinforcement learning (meta RL), an agent seeks a Bayes-optimal policy -- the optimal policy when facing an unknown task that is sampled from some known task distribution.
Goal-Conditioned Supervised Learning with Sub-Goal Prediction
Based on this idea, we propose Trajectory Iterative Learner (TraIL), an extension of GCSL that further exploits the information in a trajectory, and uses it for learning to predict both actions and sub-goals.
A Deep Learning Perspective on Network Routing
Routing is, arguably, the most fundamental task in computer networking, and the most extensively studied one.
Online Tool Selection with Learned Grasp Prediction Models
However, we find that an approximate solution based on sparse tree search yields near optimal performance at a fraction of the time.
Towards Deployable RL - What's Broken with RL Research and a Potential Fix
Reinforcement learning (RL) has demonstrated great potential, but is currently full of overhyping and pipe dreams.
Learning Control by Iterative Inversion
The key challenge is a $\textit{distribution shift}$ between the desired outputs and the outputs of an initial random guess, and we prove that iterative inversion can steer the learning correctly, under rather strict conditions on the function.
Meta Reinforcement Learning with Finite Training Tasks -- a Density Estimation Approach
We show that our approach leads to bounds that depend on the dimension of the task distribution.
Unsupervised Image Representation Learning with Deep Latent Particles
We propose a new representation of visual data that disentangles object position from appearance.
Ranked #1 on
Unsupervised Facial Landmark Detection
on MAFL
Validate on Sim, Detect on Real -- Model Selection for Domain Randomization
We show that our score - VSDR - can significantly improve the accuracy of policy ranking without requiring additional real world data.
Regularization Guarantees Generalization in Bayesian Reinforcement Learning through Algorithmic Stability
In the Bayesian reinforcement learning (RL) setting, a prior distribution over the unknown problem parameters -- the rewards and transitions -- is assumed, and a policy that optimizes the (posterior) expected return is sought.
Offline Meta Reinforcement Learning -- Identifiability Challenges and Effective Data Collection Strategies
Consider the following instance of the Offline Meta Reinforcement Learning (OMRL) problem: given the complete training logs of $N$ conventional RL agents, trained on $N$ different tasks, design a meta-agent that can quickly maximize reward in a new, unseen task from the same task distribution.
Efficient Self-Supervised Data Collection for Offline Robot Learning
In this work, we propose that data collection policies should actively explore the environment to collect diverse data.
Unsupervised Feature Learning for Manipulation with Contrastive Domain Randomization
In this work, however, we show that a naive application of DR to unsupervised learning based on contrastive estimation does not promote invariance, as the loss function maximizes mutual information between the features and both the relevant and irrelevant visual properties.
Soft-IntroVAE: Analyzing and Improving the Introspective Variational Autoencoder
However, the original IntroVAE loss function relied on a particular hinge-loss formulation that is very hard to stabilize in practice, and its theoretical convergence analysis ignored important terms in the loss.
Online Safety Assurance for Deep Reinforcement Learning
We illustrate the usefulness of online safety assurance in the context of the proposed deep reinforcement learning (RL) approach to video streaming.
Offline Meta Learning of Exploration
Consider the following instance of the Offline Meta Reinforcement Learning (OMRL) problem: given the complete training logs of $N$ conventional RL agents, trained on $N$ different tasks, design a meta-agent that can quickly maximize reward in a new, unseen task from the same task distribution.
Efficient MDP Analysis for Selfish-Mining in Blockchains
We call this Probabilistic Termination Optimization (PTO), and the technique applies to any MDP whose utility is a ratio function.
Cryptography and Security
Hallucinative Topological Memory for Zero-Shot Visual Planning
In visual planning (VP), an agent learns to plan goal-directed behavior from observations of a dynamical system obtained offline, e. g., images obtained from self-supervised robot interaction.
Sub-Goal Trees -- a Framework for Goal-Based Reinforcement Learning
Reinforcement learning (RL), building on Bellman's optimality equation, naturally optimizes for a single goal, yet can be made multi-goal by augmenting the state with the goal.
Deep Residual Flow for Out of Distribution Detection
Specifically, we demonstrate the effectiveness of our method in ResNet and DenseNet architectures trained on various image datasets.
Deep Variational Semi-Supervised Novelty Detection
In this work, we propose two variational methods for training VAEs for SSAD.
Bayesian Relational Memory for Semantic Visual Navigation
We introduce a new memory architecture, Bayesian Relational Memory (BRM), to improve the generalization ability for semantic visual navigation agents in unseen environments, where an agent is given a semantic target to navigate towards.
Sub-Goal Trees -- a Framework for Goal-Directed Trajectory Prediction and Optimization
In such problems, the way we choose to represent a trajectory underlies algorithms for trajectory prediction and optimization.
Harnessing Reinforcement Learning for Neural Motion Planning
We then propose a modification of the popular DDPG RL algorithm that is tailored to motion planning domains, by exploiting the known model in the problem and the set of solved plans in the data.
Learning Robotic Manipulation through Visual Planning and Acting
We further demonstrate our approach on learning to imagine and execute in 3 environments, the final of which is deformable rope manipulation on a PR2 robot.
Domain Randomization for Active Pose Estimation
In this work, we investigate how to improve the accuracy of domain randomization based pose estimation.
Multi-Agent Reinforcement Learning with Multi-Step Generative Models
We consider model-based reinforcement learning (MBRL) in 2-agent, high-fidelity continuous control problems -- an important domain for robots interacting with other agents in the same workspace.
Learning and Planning with a Semantic Model
Building deep reinforcement learning agents that can generalize and adapt to unseen environments remains a fundamental challenge for AI.
Safe Policy Learning from Observations
The primary advantages of our approach, termed Rerouted Behavior Improvement (RBI), over other safe learning methods are its stability in the presence of value estimation errors and the elimination of a policy search process.
Distributional Multivariate Policy Evaluation and Exploration with the Bellman GAN
In this formulation, DiRL can be seen as learning a deep generative model of the value distribution, driven by the discrepancy between the distribution of the current value, and the distribution of the sum of current reward and next value.
Learning Plannable Representations with Causal InfoGAN
Finally, to generate a visual plan, we project the current and goal observations onto their respective states in the planning model, plan a trajectory, and then use the generative model to transform the trajectory to a sequence of observations.
Constrained Policy Improvement for Safe and Efficient Reinforcement Learning
To minimize the improvement penalty, the RBI idea is to attenuate rapid policy changes of low probability actions which were less frequently sampled.
Learning Robotic Assembly from CAD
We propose to leverage this prior knowledge by guiding RL along a geometric motion plan, calculated using the CAD data.
Model-Ensemble Trust-Region Policy Optimization
In this paper, we analyze the behavior of vanilla model-based reinforcement learning methods when deep neural networks are used to learn both the model and the policy, and show that the learned policy tends to exploit regions where insufficient data is available for the model to be learned, causing instability in training.
Imitation Learning from Visual Data with Multiple Intentions
In this paper we present an LfD approach for learning multiple modes of behavior from visual data.
Safer Classification by Synthesis
We posit that a generative approach is the natural remedy for this problem, and propose a method for classification using generative models.
Situationally Aware Options
We learn reusable options in different scenarios in a RoboCup soccer domain (i. e., winning/losing).
Learning Generalized Reactive Policies using Deep Neural Networks
We show that a deep neural network can be used to learn and represent a \emph{generalized reactive policy} (GRP) that maps a problem instance and a state to an action, and that the learned GRPs efficiently solve large classes of challenging problem instances.
A Machine Learning Approach to Routing
Can ideas and techniques from machine learning be leveraged to automatically generate "good" routing configurations?
Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
We explore deep reinforcement learning methods for multi-agent domains.
Ranked #1 on
SMAC+
on Def_Infantry_sequential
Constrained Policy Optimization
For many applications of reinforcement learning it can be more convenient to specify both a reward function and constraints, rather than trying to design behavior through the reward function.
Shallow Updates for Deep Reinforcement Learning
In this work we propose a hybrid approach -- the Least Squares Deep Q-Network (LS-DQN), which combines rich feature representations learned by a DRL algorithm with the stability of a linear least squares method.
Situational Awareness by Risk-Conscious Skills
In addition, the learned risk aware skills are able to mitigate reward-based model misspecification.
Learning from the Hindsight Plan -- Episodic MPC Improvement
To bring the next real-world execution closer to the hindsight plan, our approach learns to re-shape the original cost function with the goal of satisfying the following property: short horizon planning (as realistic during real executions) with respect to the shaped cost should result in mimicking the hindsight plan.
Bayesian Reinforcement Learning: A Survey
The objective of the paper is to provide a comprehensive survey on Bayesian RL algorithms and their theoretical and empirical properties.
Value Iteration Networks
We introduce the value iteration network (VIN): a fully differentiable neural network with a `planning module' embedded within.
Generalized Emphatic Temporal Difference Learning: Bias-Variance Analysis
We consider the off-policy evaluation problem in Markov decision processes with function approximation.
Emphatic TD Bellman Operator is a Contraction
Recently, \citet{SuttonMW15} introduced the emphatic temporal differences (ETD) algorithm for off-policy evaluation in Markov decision processes.
Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approach
Our first contribution is to show that a CVaR objective, besides capturing risk sensitivity, has an alternative interpretation as expected cost under worst-case modeling errors, for a given error budget.
Policy Gradient for Coherent Risk Measures
For static risk measures, our approach is in the spirit of policy gradient algorithms and combines a standard sampling approach with convex programming.
Implicit Temporal Differences
In reinforcement learning, the TD($\lambda$) algorithm is a fundamental policy evaluation method with an efficient online implementation that is suitable for large-scale problems.
Optimizing the CVaR via Sampling
Conditional Value at Risk (CVaR) is a prominent risk measure that is being used extensively in various domains.
Variance Adjusted Actor Critic Algorithms
We present an actor-critic framework for MDPs where the objective is the variance-adjusted expected return.
Scaling Up Robust MDPs by Reinforcement Learning
We consider large-scale Markov decision processes (MDPs) with parameter uncertainty, under the robust MDP paradigm.
Policy Gradients with Variance Related Risk Criteria
In this paper we devise a framework for local policy gradient style algorithms for reinforcement learning for variance related criteria.
