Search Results for author:
Found 38 papers, 10 papers with code
Fast Rates for Maximum Entropy Exploration
The second type of entropy we study is the trajectory entropy.
The Edge of Orthogonality: A Simple View of What Makes BYOL Tick
With simple linear algebra, we show that when using a linear predictor, the optimal predictor is close to an orthogonal projection, and propose a general framework based on orthonormalization that enables to interpret and give intuition on why BYOL works.
An Analysis of Quantile Temporal-Difference Learning
We analyse quantile temporal-difference learning (QTD), a distributional reinforcement learning algorithm that has proven to be a key component in several successful large-scale applications of reinforcement learning.
Distributional Reinforcement Learning
reinforcement-learning
+1
Understanding Self-Predictive Learning for Reinforcement Learning
We identify that a faster paced optimization of the predictor and semi-gradient updates on the representation, are crucial to preventing the representation collapse.
The Nature of Temporal Difference Errors in Multi-step Distributional Reinforcement Learning
We study the multi-step off-policy learning approach to distributional RL.
Distributional Reinforcement Learning
reinforcement-learning
+1
BYOL-Explore: Exploration by Bootstrapped Prediction
We present BYOL-Explore, a conceptually simple yet general approach for curiosity-driven exploration in visually-complex environments.
KL-Entropy-Regularized RL with a Generative Model is Minimax Optimal
In this work, we consider and analyze the sample complexity of model-free reinforcement learning with a generative model.
From Dirichlet to Rubin: Optimistic Exploration in RL without Bonuses
We propose the Bayes-UCBVI algorithm for reinforcement learning in tabular, stage-dependent, episodic Markov decision process: a natural extension of the Bayes-UCB algorithm by Kaufmann et al. (2012) for multi-armed bandits.
Marginalized Operators for Off-policy Reinforcement Learning
We show that the estimates for marginalized operators can be computed in a scalable way, which also generalizes prior results on marginalized importance sampling as special cases.
Biased Gradient Estimate with Drastic Variance Reduction for Meta Reinforcement Learning
Despite the empirical success of meta reinforcement learning (meta-RL), there are still a number poorly-understood discrepancies between theory and practice.
Unifying Gradient Estimators for Meta-Reinforcement Learning via Off-Policy Evaluation
Model-agnostic meta-reinforcement learning requires estimating the Hessian matrix of value functions.
Taylor Expansion of Discount Factors
In practical reinforcement learning (RL), the discount factor used for estimating value functions often differs from that used for defining the evaluation objective.
Revisiting Peng's Q($λ$) for Modern Reinforcement Learning
These results indicate that Peng's Q($\lambda$), which was thought to be unsafe, is a theoretically-sound and practically effective algorithm.
Unlocking Pixels for Reinforcement Learning via Implicit Attention
There has recently been significant interest in training reinforcement learning (RL) agents in vision-based environments.
ES-ENAS: Efficient Evolutionary Optimization for Large Hybrid Search Spaces
In this paper, we approach the problem of optimizing blackbox functions over large hybrid search spaces consisting of both combinatorial and continuous parameters.
Monte-Carlo Tree Search as Regularized Policy Optimization
The combination of Monte-Carlo tree search (MCTS) with deep reinforcement learning has led to significant advances in artificial intelligence.
Hindsight Expectation Maximization for Goal-conditioned Reinforcement Learning
We propose a graphical model framework for goal-conditioned RL, with an EM algorithm that operates on the lower bound of the RL objective.
Online Hyper-parameter Tuning in Off-policy Learning via Evolutionary Strategies
Off-policy learning algorithms have been known to be sensitive to the choice of hyper-parameters.
Self-Imitation Learning via Generalized Lower Bound Q-learning
Self-imitation learning motivated by lower-bound Q-learning is a novel and effective approach for off-policy learning.
Taylor Expansion Policy Optimization
In this work, we investigate the application of Taylor expansions in reinforcement learning.
Discrete Action On-Policy Learning with Action-Value Critic
Reinforcement learning (RL) in discrete action space is ubiquitous in real-world applications, but its complexity grows exponentially with the action-space dimension, making it challenging to apply existing on-policy gradient based deep RL algorithms efficiently.
Reinforcement Learning with Chromatic Networks
We present a neural architecture search algorithm to construct compact reinforcement learning (RL) policies, by combining ENAS and ES in a highly scalable and intuitive way.
Behavior-Guided Reinforcement Learning
We introduce a new approach for comparing reinforcement learning policies, using Wasserstein distances (WDs) in a newly defined latent behavioral space.
ES-MAML: Simple Hessian-Free Meta Learning
We introduce ES-MAML, a new framework for solving the model agnostic meta learning (MAML) problem based on Evolution Strategies (ES).
Reinforcement Learning with Chromatic Networks for Compact Architecture Search
We present a neural architecture search algorithm to construct compact reinforcement learning (RL) policies, by combining ENAS and ES in a highly scalable and intuitive way.
Learning to Score Behaviors for Guided Policy Optimization
We introduce a new approach for comparing reinforcement learning policies, using Wasserstein distances (WDs) in a newly defined latent behavioral space.
Reinforcement Learning for Integer Programming: Learning to Cut
In particular, we investigate a specific methodology for solving IPs, known as the Cutting Plane Method.
Variance Reduction for Evolution Strategies via Structured Control Variates
Evolution Strategies (ES) are a powerful class of blackbox optimization techniques that recently became a competitive alternative to state-of-the-art policy gradient (PG) algorithms for reinforcement learning (RL).
Structured Monte Carlo Sampling for Nonisotropic Distributions via Determinantal Point Processes
We propose a new class of structured methods for Monte Carlo (MC) sampling, called DPPMC, designed for high-dimensional nonisotropic distributions where samples are correlated to reduce the variance of the estimator via determinantal point processes.
Augment-Reinforce-Merge Policy Gradient for Binary Stochastic Policy
Due to the high variance of policy gradients, on-policy optimization algorithms are plagued with low sample efficiency.
Orthogonal Estimation of Wasserstein Distances
Wasserstein distances are increasingly used in a wide variety of applications in machine learning.
Provably Robust Blackbox Optimization for Reinforcement Learning
Interest in derivative-free optimization (DFO) and "evolutionary strategies" (ES) has recently surged in the Reinforcement Learning (RL) community, with growing evidence that they can match state of the art methods for policy optimization problems in Robotics.
From Complexity to Simplicity: Adaptive ES-Active Subspaces for Blackbox Optimization
ASEBO adapts to the geometry of the function and learns optimal sets of sensing directions, which are used to probe it, on-the-fly.
Discretizing Continuous Action Space for On-Policy Optimization
In this work, we show that discretizing action space for continuous control is a simple yet powerful technique for on-policy optimization.
Boosting Trust Region Policy Optimization by Normalizing Flows Policy
We propose to improve trust region policy search with normalizing flows policy.
Implicit Policy for Reinforcement Learning
We introduce Implicit Policy, a general class of expressive policies that can flexibly represent complex action distributions in reinforcement learning, with efficient algorithms to compute entropy regularized policy gradients.
Exploration by Distributional Reinforcement Learning
We propose a framework based on distributional reinforcement learning and recent attempts to combine Bayesian parameter updates with deep reinforcement learning.
Distributional Reinforcement Learning
Efficient Exploration
+2
Variational Deep Q Network
We propose a framework that directly tackles the probability distribution of the value function parameters in Deep Q Network (DQN), with powerful variational inference subroutines to approximate the posterior of the parameters.
