Search Results for author:
Found 50 papers, 7 papers with code
Simple Ingredients for Offline Reinforcement Learning
Offline reinforcement learning algorithms have proven effective on datasets highly connected to the target downstream task.
Layered State Discovery for Incremental Autonomous Exploration
We leverage these results to design Layered Autonomous Exploration (LAE), a novel algorithm for AX that attains a sample complexity of $\tilde{\mathcal{O}}(LS^{\rightarrow}_{L(1+\epsilon)}\Gamma_{L(1+\epsilon)} A \ln^{12}(S^{\rightarrow}_{L(1+\epsilon)})/\epsilon^2)$, where $S^{\rightarrow}_{L(1+\epsilon)}$ is the number of states that are incrementally $L(1+\epsilon)$-controllable, $A$ is the number of actions, and $\Gamma_{L(1+\epsilon)}$ is the branching factor of the transitions over such states.
On the Complexity of Representation Learning in Contextual Linear Bandits
In contextual linear bandits, the reward function is assumed to be a linear combination of an unknown reward vector and a given embedding of context-arm pairs.
Improved Adaptive Algorithm for Scalable Active Learning with Weak Labeler
We design a novel algorithmic template, Weak Labeler Active Cover (WL-AC), that is able to robustly leverage the lower quality weak labelers to reduce the query complexity while retaining the desired level of accuracy.
Scalable Representation Learning in Linear Contextual Bandits with Constant Regret Guarantees
We study the problem of representation learning in stochastic contextual linear bandits.
Contextual bandits with concave rewards, and an application to fair ranking
We consider Contextual Bandits with Concave Rewards (CBCR), a multi-objective bandit problem where the desired trade-off between the rewards is defined by a known concave objective function, and the reward vector depends on an observed stochastic context.
Reaching Goals is Hard: Settling the Sample Complexity of the Stochastic Shortest Path
We also initiate the study of learning $\epsilon$-optimal policies without access to a generative model (i. e., the so-called best-policy identification problem), and show that sample-efficient learning is impossible in general.
Top $K$ Ranking for Multi-Armed Bandit with Noisy Evaluations
We consider a multi-armed bandit setting where, at the beginning of each round, the learner receives noisy independent, and possibly biased, \emph{evaluations} of the true reward of each arm and it selects $K$ arms with the objective of accumulating as much reward as possible over $T$ rounds.
Privacy Amplification via Shuffling for Linear Contextual Bandits
Contextual bandit algorithms are widely used in domains where it is desirable to provide a personalized service by leveraging contextual information, that may contain sensitive information that needs to be protected.
Differentially Private Exploration in Reinforcement Learning with Linear Representation
We first consider the setting of linear-mixture MDPs (Ayoub et al., 2020) (a. k. a.\ model-based setting) and provide an unified framework for analyzing joint and local differential private (DP) exploration.
Adaptive Multi-Goal Exploration
We introduce a generic strategy for provably efficient multi-goal exploration.
Reinforcement Learning in Linear MDPs: Constant Regret and Representation Selection
We study the role of the representation of state-action value functions in regret minimization in finite-horizon Markov Decision Processes (MDPs) with linear structure.
A Fully Problem-Dependent Regret Lower Bound for Finite-Horizon MDPs
We derive a novel asymptotic problem-dependent lower-bound for regret minimization in finite-horizon tabular Markov Decision Processes (MDPs).
A Reduction-Based Framework for Conservative Bandits and Reinforcement Learning
We also obtain a new upper bound for conservative low-rank MDP.
Stochastic Shortest Path: Minimax, Parameter-Free and Towards Horizon-Free Regret
We study the problem of learning in the stochastic shortest path (SSP) setting, where an agent seeks to minimize the expected cost accumulated before reaching a goal state.
Leveraging Good Representations in Linear Contextual Bandits
We show that the regret is indeed never worse than the regret obtained by running LinUCB on the best representation (up to a $\ln M$ factor).
Encrypted Linear Contextual Bandit
A critical aspect of bandit methods is that they require to observe the contexts --i. e., individual or group-level data-- and rewards in order to solve the sequential problem.
Meta-Reinforcement Learning With Informed Policy Regularization
Meta-reinforcement learning aims at finding a policy able to generalize to new environments.
Improved Sample Complexity for Incremental Autonomous Exploration in MDPs
We investigate the exploration of an unknown environment when no reward function is provided.
An Asymptotically Optimal Primal-Dual Incremental Algorithm for Contextual Linear Bandits
Finally, we remove forced exploration and build on confidence intervals of the optimization problem to encourage a minimum level of exploration that is better adapted to the problem structure.
Local Differential Privacy for Regret Minimization in Reinforcement Learning
Motivated by this, we study privacy in the context of finite-horizon Markov Decision Processes (MDPs) by requiring information to be obfuscated on the user side.
A Provably Efficient Sample Collection Strategy for Reinforcement Learning
One of the challenges in online reinforcement learning (RL) is that the agent needs to trade off the exploration of the environment and the exploitation of the samples to optimize its behavior.
Improved Analysis of UCRL2 with Empirical Bernstein Inequality
We consider the problem of exploration-exploitation in communicating Markov Decision Processes.
A Kernel-Based Approach to Non-Stationary Reinforcement Learning in Metric Spaces
In this work, we propose KeRNS: an algorithm for episodic reinforcement learning in non-stationary Markov Decision Processes (MDPs) whose state-action set is endowed with a metric.
Learning Adaptive Exploration Strategies in Dynamic Environments Through Informed Policy Regularization
We test the performance of our algorithm in a variety of environments where tasks may vary within each episode.
Kernel-Based Reinforcement Learning: A Finite-Time Analysis
We consider the exploration-exploitation dilemma in finite-horizon reinforcement learning problems whose state-action space is endowed with a metric.
Active Model Estimation in Markov Decision Processes
Using a number of simple domains with heterogeneous noise in their transitions, we show that our heuristic-based algorithm outperforms both our original algorithm and the maximum entropy algorithm in the small sample regime, while achieving similar asymptotic performance as that of the original algorithm.
Exploration-Exploitation in Constrained MDPs
In many sequential decision-making problems, the goal is to optimize a utility function while satisfying a set of constraints on different utilities.
Adversarial Attacks on Linear Contextual Bandits
In many of these domains, malicious agents may have incentives to attack the bandit algorithm to induce it to perform a desired behavior.
Improved Algorithms for Conservative Exploration in Bandits
In this case, it is desirable to deploy online learning algorithms (e. g., a multi-armed bandit algorithm) that interact with the system to learn a better/optimal policy under the constraint that during the learning process the performance is almost never worse than the performance of the baseline itself.
Conservative Exploration in Reinforcement Learning
While learning in an unknown Markov Decision Process (MDP), an agent should trade off exploration to discover new information about the MDP, and exploitation of the current knowledge to maximize the reward.
Concentration Inequalities for Multinoulli Random Variables
We investigate concentration inequalities for Dirichlet and Multinomial random variables.
Exploiting Language Instructions for Interpretable and Compositional Reinforcement Learning
In this work, we present an alternative approach to making an agent compositional through the use of a diagnostic classifier.
No-Regret Exploration in Goal-Oriented Reinforcement Learning
Many popular reinforcement learning problems (e. g., navigation in a maze, some Atari games, mountain car) are instances of the episodic setting under its stochastic shortest path (SSP) formulation, where an agent has to achieve a goal state while minimizing the cumulative cost.
Exploration Bonus for Regret Minimization in Discrete and Continuous Average Reward MDPs
The exploration bonus is an effective approach to manage the exploration-exploitation trade-off in Markov Decision Processes (MDPs).
Regret Bounds for Learning State Representations in Reinforcement Learning
We consider the problem of online reinforcement learning when several state representations (mapping histories to a discrete state space) are available to the learning agent.
Frequentist Regret Bounds for Randomized Least-Squares Value Iteration
We consider the exploration-exploitation dilemma in finite-horizon reinforcement learning (RL).
Smoothing Policies and Safe Policy Gradients
Policy Gradient (PG) algorithms are among the best candidates for the much-anticipated applications of reinforcement learning to real-world control tasks, such as robotics.
Exploration Bonus for Regret Minimization in Undiscounted Discrete and Continuous Markov Decision Processes
We introduce and analyse two algorithms for exploration-exploitation in discrete and continuous Markov Decision Processes (MDPs) based on exploration bonuses.
Near Optimal Exploration-Exploitation in Non-Communicating Markov Decision Processes
While designing the state space of an MDP, it is common to include states that are transient or not reachable by any policy (e. g., in mountain car, the product space of speed and position contains configurations that are not physically reachable).
Stochastic Variance-Reduced Policy Gradient
In this paper, we propose a novel reinforcement- learning algorithm consisting in a stochastic variance-reduced version of policy gradient for solving Markov Decision Processes (MDPs).
Importance Weighted Transfer of Samples in Reinforcement Learning
In the proposed approach, all the samples are transferred and used by a batch RL algorithm to solve the target task, but their contribution to the learning process is proportional to their importance weight.
Efficient Bias-Span-Constrained Exploration-Exploitation in Reinforcement Learning
We introduce SCAL, an algorithm designed to perform efficient exploration-exploitation in any unknown weakly-communicating Markov decision process (MDP) for which an upper bound $c$ on the span of the optimal bias function is known.
Cost-Sensitive Approach to Batch Size Adaptation for Gradient Descent
In this paper, we propose a novel approach to automatically determine the batch size in stochastic gradient descent methods.
Regret Minimization in MDPs with Options without Prior Knowledge
The option framework integrates temporal abstraction into the reinforcement learning model through the introduction of macro-actions (i. e., options).
Adaptive Batch Size for Safe Policy Gradients
Policy gradient methods are among the best Reinforcement Learning (RL) techniques to solve complex control problems.
Compatible Reward Inverse Reinforcement Learning
Within this subspace, using a second-order criterion, we search for the reward function that penalizes the most a deviation from the expert's policy.
Boosted Fitted Q-Iteration
This paper is about the study of B-FQI, an Approximated Value Iteration (AVI) algorithm that exploits a boosting procedure to estimate the action-value function in reinforcement learning problems.
Multi-objective Reinforcement Learning with Continuous Pareto Frontier Approximation Supplementary Material
The paper is about learning a continuous approximation of the Pareto frontier in Multi-Objective Markov Decision Problems (MOMDPs).
Multi-Objective Reinforcement Learning
reinforcement-learning
+1
Adaptive Step-Size for Policy Gradient Methods
In the last decade, policy gradient methods have significantly grown in popularity in the reinforcement--learning field.
