Search Results for author:
Found 50 papers, 19 papers with code
Abstract Reward Processes: Leveraging State Abstraction for Consistent Off-Policy Evaluation
Evaluating policies using off-policy data is crucial for applying reinforcement learning to real-world problems such as healthcare and autonomous driving.
Position: Benchmarking is Limited in Reinforcement Learning Research
Novel reinforcement learning algorithms, or improvements on existing ones, are commonly justified by evaluating their performance on benchmark environments and are compared to an ever-changing set of standard algorithms.
ICU-Sepsis: A Benchmark MDP Built from Real Medical Data
We present ICU-Sepsis, an environment that can be used in benchmarks for evaluating reinforcement learning (RL) algorithms.
From Past to Future: Rethinking Eligibility Traces
In this paper, we introduce a fresh perspective on the challenges of credit assignment and policy evaluation.
Learning Fair Representations with High-Confidence Guarantees
Representation learning is increasingly employed to generate representations that are predictive across multiple downstream tasks.
Coagent Networks: Generalized and Scaled
However, the coagent framework is not just an alternative to BDL; the two approaches can be blended: BDL can be combined with coagent learning rules to create architectures with the advantages of both approaches.
Optimization using Parallel Gradient Evaluations on Multiple Parameters
We propose a first-order method for convex optimization, where instead of being restricted to the gradient from a single parameter, gradients from multiple parameters can be used during each step of gradient descent.
Off-Policy Evaluation for Action-Dependent Non-Stationary Environments
Methods for sequential decision-making are often built upon a foundational assumption that the underlying decision process is stationary.
Low Variance Off-policy Evaluation with State-based Importance Sampling
In many domains, the exploration process of reinforcement learning will be too costly as it requires trying out suboptimal policies, resulting in a need for off-policy evaluation, in which a target policy is evaluated based on data collected from a known behaviour policy.
Enforcing Delayed-Impact Fairness Guarantees
Recent research has shown that seemingly fair machine learning models, when used to inform decisions that have an impact on peoples' lives or well-being (e. g., applications involving education, employment, and lending), can inadvertently increase social inequality in the long term.
Adaptive Rollout Length for Model-Based RL Using Model-Free Deep RL
Model-based reinforcement learning promises to learn an optimal policy from fewer interactions with the environment compared to model-free reinforcement learning by learning an intermediate model of the environment in order to predict future interactions.
Edge-Compatible Reinforcement Learning for Recommendations
In this work, we build on asynchronous coagent policy gradient algorithms \citep{kostas2020asynchronous} to propose a principled solution to this problem.
Structural Credit Assignment in Neural Networks using Reinforcement Learning
In this work, we revisit this approach and investigate if we can leverage other reinforcement learning approaches to improve learning.
SOPE: Spectrum of Off-Policy Estimators
In this paper, we present a new perspective on this bias-variance trade-off and show the existence of a spectrum of estimators whose endpoints are SIS and IS.
Fairness Guarantees under Demographic Shift
Recent studies have demonstrated that using machine learning for social applications can lead to injustice in the form of racist, sexist, and otherwise unfair and discriminatory outcomes.
Multi-Objective SPIBB: Seldonian Offline Policy Improvement with Safety Constraints in Finite MDPs
We study the problem of Safe Policy Improvement (SPI) under constraints in the offline Reinforcement Learning (RL) setting.
Universal Off-Policy Evaluation
When faced with sequential decision-making problems, it is often useful to be able to predict what would happen if decisions were made using a new policy.
High-Confidence Off-Policy (or Counterfactual) Variance Estimation
Many sequential decision-making systems leverage data collected using prior policies to propose a new policy.
Security Analysis of Safe and Seldonian Reinforcement Learning Algorithms
We analyze the extent to which existing methods rely on accurate training data for a specific class of reinforcement learning (RL) algorithms, known as Safe and Seldonian RL.
Towards Safe Policy Improvement for Non-Stationary MDPs
Many real-world sequential decision-making problems involve critical systems with financial risks and human-life risks.
Reinforcement Learning for Strategic Recommendations
Strategic recommendations (SR) refer to the problem where an intelligent agent observes the sequential behaviors and activities of users and decides when and how to interact with them to optimize some long-term objectives, both for the user and the business.
Evaluating the Performance of Reinforcement Learning Algorithms
Performance evaluations are critical for quantifying algorithmic advances in reinforcement learning.
Optimizing for the Future in Non-Stationary MDPs
Most reinforcement learning methods are based upon the key assumption that the transition dynamics and reward functions are fixed, that is, the underlying Markov decision process is stationary.
Learning Reusable Options for Multi-Task Reinforcement Learning
Reinforcement learning (RL) has become an increasingly active area of research in recent years.
Offline Contextual Bandits with High Probability Fairness Guarantees
We present RobinHood, an offline contextual bandit algorithm designed to satisfy a broad family of fairness constraints.
Reinforcement learning with a network of spiking agents
Neuroscientific theory suggests that dopaminergic neurons broadcast global reward prediction errors to large areas of the brain influencing the synaptic plasticity of the neurons in those regions.
Is the Policy Gradient a Gradient?
The policy gradient theorem describes the gradient of the expected discounted return with respect to an agent's policy parameters.
Classical Policy Gradient: Preserving Bellman's Principle of Optimality
We propose a new objective function for finite-horizon episodic Markov decision processes that better captures Bellman's principle of optimality, and provide an expression for the gradient of the objective.
Reinforcement Learning When All Actions are Not Always Available
The Markov decision process (MDP) formulation used to model many real-world sequential decision making problems does not efficiently capture the setting where the set of available decisions (actions) at each time step is stochastic.
Lifelong Learning with a Changing Action Set
have been well-studied in the lifelong learning literature, the setting where the action set changes remains unaddressed.
A New Confidence Interval for the Mean of a Bounded Random Variable
We present a new method for constructing a confidence interval for the mean of a bounded random variable from samples of the random variable.
Asynchronous Coagent Networks
Coagent policy gradient algorithms (CPGAs) are reinforcement learning algorithms for training a class of stochastic neural networks called coagent networks.
Hierarchical Reinforcement Learning
reinforcement-learning
+2
A Meta-MDP Approach to Exploration for Lifelong Reinforcement Learning
In this paper we consider the problem of how a reinforcement learning agent that is tasked with solving a sequence of reinforcement learning problems (a sequence of Markov decision processes) can use knowledge acquired early in its lifetime to improve its ability to solve new problems.
Privacy Preserving Off-Policy Evaluation
Many reinforcement learning applications involve the use of data that is sensitive, such as medical records of patients or financial information.
Learning Action Representations for Reinforcement Learning
Most model-free reinforcement learning methods leverage state representations (embeddings) for generalization, but either ignore structure in the space of actions or assume the structure is provided a priori.
Natural Option Critic
In this paper we show how the option-critic architecture can be extended to estimate the natural gradient of the expected discounted return.
A Compression-Inspired Framework for Macro Discovery
In this paper we consider the problem of how a reinforcement learning agent tasked with solving a set of related Markov decision processes can use knowledge acquired early in its lifetime to improve its ability to more rapidly solve novel, but related, tasks.
On Ensuring that Intelligent Machines Are Well-Behaved
We propose a new framework for designing machine learning algorithms that simplifies the problem of specifying and regulating undesirable behaviors.
Policy Gradient Methods for Reinforcement Learning with Function Approximation and Action-Dependent Baselines
We show how an action-dependent baseline can be used by the policy gradient theorem using function approximation, originally presented with action-independent baselines by (Sutton et al. 2000).
Data-Efficient Policy Evaluation Through Behavior Policy Search
The standard unbiased technique for evaluating a policy is to deploy the policy and observe its performance.
Decoupling Learning Rules from Representations
When creating an artificial intelligence system, we must make two decisions: what representation should be used (i. e., what parameterized function should be used) and what learning rule should be used to search through the resulting set of representable functions.
Using Options and Covariance Testing for Long Horizon Off-Policy Policy Evaluation
In addition, we can take advantage of special cases that arise due to options-based policies to further improve the performance of importance sampling.
Importance Sampling with Unequal Support
Importance sampling is often used in machine learning when training and testing data come from different distributions.
Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learning
In this paper we present a new way of predicting the performance of a reinforcement learning policy given historical data that may have been generated by a different policy.
A Notation for Markov Decision Processes
This paper specifies a notation for Markov decision processes.
Increasing the Action Gap: New Operators for Reinforcement Learning
Extending the idea of a locally consistent operator, we then derive sufficient conditions for an operator to preserve optimality, leading to a family of operators which includes our consistent Bellman operator.
Ranked #1 on
Atari Games
on Atari 2600 Elevator Action
Policy Evaluation Using the Ω-Return
The benefit of the Ω-return is that it accounts for the correlation of different length returns.
Projected Natural Actor-Critic
Natural actor-critics are a popular class of policy search algorithms for finding locally optimal policies for Markov decision processes.
TD_gamma: Re-evaluating Complex Backups in Temporal Difference Learning
We show that the lambda-return target used in the TD(lambda) family of algorithms is the maximum likelihood estimator for a specific model of how the variance of an n-step return estimate increases with n. We introduce the gamma-return estimator, an alternative target based on a more accurate model of variance, which defines the TD_gamma family of complex-backup temporal difference learning algorithms.
Policy Gradient Coagent Networks
We present a novel class of actor-critic algorithms for actors consisting of sets of interacting modules.
