Search Results for author:
Found 41 papers, 6 papers with code
Time After Time: Deep-Q Effect Estimation for Interventions on When and What to do
Problems in fields such as healthcare, robotics, and finance requires reasoning about the value both of what decision or action to take and when to take it.
Aligned Multi Objective Optimization
To date, the multi-objective optimization literature has mainly focused on conflicting objectives, studying the Pareto front, or requiring users to balance tradeoffs.
Exploiting Structure in Offline Multi-Agent RL: The Benefits of Low Interaction Rank
We study the problem of learning an approximate equilibrium in the offline multi-agent reinforcement learning (MARL) setting.
RL in Latent MDPs is Tractable: Online Guarantees via Off-Policy Evaluation
Our result builds off a new perspective on the role of off-policy evaluation guarantees and coverage coefficients in LMDPs, a perspective, that has been overlooked in the context of exploration in partially observed environments.
Generalizing Multi-Step Inverse Models for Representation Learning to Finite-Memory POMDPs
In this work, we consider the problem of discovering the agent-centric state in the more challenging high-dimensional non-Markovian setting, when the state can be decoded from a sequence of past observations.
The Bias of Harmful Label Associations in Vision-Language Models
Despite the remarkable performance of foundation vision-language models, the shared representation space for text and vision can also encode harmful label associations detrimental to fairness.
Pearl: A Production-ready Reinforcement Learning Agent
Reinforcement learning (RL) is a versatile framework for optimizing long-term goals.
PcLast: Discovering Plannable Continuous Latent States
Goal-conditioned planning benefits from learned low-dimensional representations of rich observations.
Prospective Side Information for Latent MDPs
In such an environment, the latent information remains fixed throughout each episode, since the identity of the user does not change during an interaction.
Agent-Controller Representations: Principled Offline RL with Rich Exogenous Information
We find that contemporary representation learning techniques can fail on datasets where the noise is a complex and time dependent process, which is prevalent in practical applications.
Tractable Optimality in Episodic Latent MABs
Then, through a method-of-moments approach, we design a procedure that provably learns a near-optimal policy with $O(\texttt{poly}(A) + \texttt{poly}(M, H)^{\min(M, H)})$ interactions.
Reward-Mixing MDPs with a Few Latent Contexts are Learnable
We consider episodic reinforcement learning in reward-mixing Markov decision processes (RMMDPs): at the beginning of every episode nature randomly picks a latent reward model among $M$ candidates and an agent interacts with the MDP throughout the episode for $H$ time steps.
Guaranteed Discovery of Control-Endogenous Latent States with Multi-Step Inverse Models
In many sequential decision-making tasks, the agent is not able to model the full complexity of the world, which consists of multitudes of relevant and irrelevant information.
Sample-Efficient Reinforcement Learning in the Presence of Exogenous Information
In real-world reinforcement learning applications the learner's observation space is ubiquitously high-dimensional with both relevant and irrelevant information about the task at hand.
Provable Reinforcement Learning with a Short-Term Memory
Real-world sequential decision making problems commonly involve partial observability, which requires the agent to maintain a memory of history in order to infer the latent states, plan and make good decisions.
Coordinated Attacks against Contextual Bandits: Fundamental Limits and Defense Mechanisms
This parallelization gain is fundamentally altered by the presence of adversarial users: unless there are super-polynomial number of users, we show a lower bound of $\tilde{\Omega}(\min(S, A) \cdot \alpha^2 / \epsilon^2)$ {\it per-user} interactions to learn an $\epsilon$-optimal policy for the good users.
Provable RL with Exogenous Distractors via Multistep Inverse Dynamics
We initiate the formal study of latent state discovery in the presence of such exogenous noise sources by proposing a new model, the Exogenous Block MDP (EX-BMDP), for rich observation RL.
Sparsity in Partially Controllable Linear Systems
However, in practice, we often encounter systems in which a large set of state variables evolve exogenously and independently of the control inputs; such systems are only partially controllable.
Query-Reward Tradeoffs in Multi-Armed Bandits
We consider a stochastic multi-armed bandit setting where reward must be actively queried for it to be observed.
Reinforcement Learning in Reward-Mixing MDPs
We study the problem of learning a near optimal policy for two reward-mixing MDPs.
Provably Filtering Exogenous Distractors using Multistep Inverse Dynamics
We initiate the formal study of latent state discovery in the presence of such exogenous noise sources by proposing a new model, the Exogenous Block MDP (EX-BMDP), for rich observation RL.
Minimax Regret for Stochastic Shortest Path
In this work we show that the minimax regret for this setting is $\widetilde O(\sqrt{ (B_\star^2 + B_\star) |S| |A| K})$ where $B_\star$ is a bound on the expected cost of the optimal policy from any state, $S$ is the state space, and $A$ is the action space.
RL for Latent MDPs: Regret Guarantees and a Lower Bound
In this work, we consider the regret minimization problem for reinforcement learning in latent Markov Decision Processes (LMDP).
Confidence-Budget Matching for Sequential Budgeted Learning
We analyze the performance of CBM based algorithms in different settings and show that they perform well in the presence of adversity in the contexts, initial states, and budgets.
Reinforcement Learning with Trajectory Feedback
The standard feedback model of reinforcement learning requires revealing the reward of every visited state-action pair.
Bandits with Partially Observable Confounded Data
We construct a linear bandit algorithm that takes advantage of the projected information, and prove regret bounds.
Mirror Descent Policy Optimization
Overall, MDPO is derived from the MD principles, offers a unified approach to viewing a number of popular RL algorithms, and performs better than or on-par with TRPO, PPO, and SAC in a number of continuous control tasks.
Exploration-Exploitation in Constrained MDPs
In many sequential decision-making problems, the goal is to optimize a utility function while satisfying a set of constraints on different utilities.
Optimistic Policy Optimization with Bandit Feedback
To the best of our knowledge, the two results are the first sub-linear regret bounds obtained for policy optimization algorithms with unknown transitions and bandit feedback.
Multi-step Greedy Reinforcement Learning Algorithms
We derive model-free RL algorithms based on $\kappa$-PI and $\kappa$-VI in which the surrogate problem can be solved by any discrete or continuous action RL method, such as DQN and TRPO.
Multi-step Greedy Policies in Model-Free Deep Reinforcement Learning
In this work, we explore the benefits of multi-step greedy policies in model-free RL when employed in the framework of multi-step Dynamic Programming (DP): multi-step Policy and Value Iteration.
Online Planning with Lookahead Policies
This is the first work that proves improved sample complexity as a result of {\em increasing} the lookahead horizon in online planning.
Adaptive Trust Region Policy Optimization: Global Convergence and Faster Rates for Regularized MDPs
Trust region policy optimization (TRPO) is a popular and empirically successful policy search algorithm in Reinforcement Learning (RL) in which a surrogate problem, that restricts consecutive policies to be 'close' to one another, is iteratively solved.
Tight Regret Bounds for Model-Based Reinforcement Learning with Greedy Policies
In this paper, we focus on model-based RL in the finite-state finite-horizon MDP setting and establish that exploring with \emph{greedy policies} -- act by \emph{1-step planning} -- can achieve tight minimax performance in terms of regret, $\tilde{\mathcal{O}}(\sqrt{HSAT})$.
Model-based Reinforcement Learning
reinforcement-learning
+2
Action Robust Reinforcement Learning and Applications in Continuous Control
In this work we formalize two new criteria of robustness to action uncertainty.
Exploration Conscious Reinforcement Learning Revisited
We continue and analyze properties of exploration-conscious optimal policies and characterize two general approaches to solve such criteria.
Multiple-Step Greedy Policies in Approximate and Online Reinforcement Learning
Multiple-step lookahead policies have demonstrated high empirical competence in Reinforcement Learning, via the use of Monte Carlo Tree Search or Model Predictive Control.
How to Combine Tree-Search Methods in Reinforcement Learning
Finite-horizon lookahead policies are abundantly used in Reinforcement Learning and demonstrate impressive empirical success.
Beyond the One-Step Greedy Approach in Reinforcement Learning
The famous Policy Iteration algorithm alternates between policy improvement and policy evaluation.
Multiple-Step Greedy Policies in Online and Approximate Reinforcement Learning
Multiple-step lookahead policies have demonstrated high empirical competence in Reinforcement Learning, via the use of Monte Carlo Tree Search or Model Predictive Control.
Beyond the One Step Greedy Approach in Reinforcement Learning
The famous Policy Iteration algorithm alternates between policy improvement and policy evaluation.
