Search Results for author:
Found 39 papers, 12 papers with code
Offline Inverse RL: New Solution Concepts and Provably Efficient Algorithms
In this paper, we introduce a novel notion of feasible reward set capturing the opportunities and limitations of the offline setting and we analyze the complexity of its estimation.
Performance Improvement Bounds for Lipschitz Configurable Markov Decision Processes
Configurable Markov Decision Processes (Conf-MDPs) have recently been introduced as an extension of the traditional Markov Decision Processes (MDPs) to model the real-world scenarios in which there is the possibility to intervene in the environment in order to configure some of its parameters.
Information Capacity Regret Bounds for Bandits with Mediator Feedback
For a selection of policy set families, we prove nearly-matching lower bounds, scaling similarly with the capacity.
No-Regret Reinforcement Learning in Smooth MDPs
Obtaining no-regret guarantees for reinforcement learning (RL) in the case of problems with continuous state and/or action spaces is still one of the major open challenges in the field.
Inverse Reinforcement Learning with Sub-optimal Experts
For this reason, in this work, we extend the IRL formulation to problems where, in addition to demonstrations from the optimal agent, we can observe the behavior of multiple sub-optimal experts.
Parameterized Projected Bellman Operator
We formulate an optimization problem to learn PBO for generic sequential decision-making problems, and we theoretically analyze its properties in two representative classes of RL problems.
Causal Feature Selection via Transfer Entropy
We introduce a new causal feature selection approach that relies on the forward and backward feature selection procedures and leverages transfer entropy to estimate the causal flow of information from the features to the target in time series.
$(ε, u)$-Adaptive Regret Minimization in Heavy-Tailed Bandits
In this setting, we study the regret minimization problem when $\epsilon$ and $u$ are unknown to the learner and it has to adapt.
Pure Exploration under Mediators' Feedback
In this setting, the agent's goal lies in sequentially choosing which mediator to query to identify with high probability the optimal arm while minimizing the identification time, i. e., the sample complexity.
Nonlinear Feature Aggregation: Two Algorithms driven by Theory
A limitation of methods based on correlation is the assumption of linearity in the relationship between features and target.
An Option-Dependent Analysis of Regret Minimization Algorithms in Finite-Horizon Semi-Markov Decision Processes
Then, focusing on a sub-setting of HRL approaches, the options framework, we highlight how the average duration of the available options affects the planning horizon and, consequently, the regret itself.
Hierarchical Reinforcement Learning
reinforcement-learning
+1
Truncating Trajectories in Monte Carlo Reinforcement Learning
In Reinforcement Learning (RL), an agent acts in an unknown environment to maximize the expected cumulative discounted sum of an external reward signal, i. e., the expected return.
Towards Theoretical Understanding of Inverse Reinforcement Learning
We start by formally introducing the problem of estimating the feasible reward set, the corresponding PAC requirement, and discussing the properties of particular classes of rewards.
A Tale of Sampling and Estimation in Discounted Reinforcement Learning
In this paper, we present a minimax lower bound on the discounted mean estimation problem that explicitly connects the estimation error with the mixing properties of the Markov process and the discount factor.
Interpretable Linear Dimensionality Reduction based on Bias-Variance Analysis
Instead, dimensionality reduction techniques are designed to limit the number of features in a dataset by projecting them into a lower-dimensional space, possibly considering all the original features.
Information-Theoretic Regret Bounds for Bandits with Fixed Expert Advice
We investigate the problem of bandits with expert advice when the experts are fixed and known distributions over the actions.
Wasserstein Actor-Critic: Directed Exploration via Optimism for Continuous-Actions Control
Uncertainty quantification has been extensively used as a means to achieve efficient directed exploration in Reinforcement Learning (RL).
Best Arm Identification for Stochastic Rising Bandits
Then, we prove that, with a sufficiently large budget, they provide guarantees on the probability of properly identifying the optimal option at the end of the learning process.
Autoregressive Bandits
Autoregressive processes naturally arise in a large variety of real-world scenarios, including stock markets, sales forecasting, weather prediction, advertising, and pricing.
Tight Performance Guarantees of Imitator Policies with Continuous Actions
In this paper, we study BC with the goal of providing theoretical guarantees on the performance of the imitator policy in the case of continuous actions.
Stochastic Rising Bandits
This paper is in the field of stochastic Multi-Armed Bandits (MABs), i. e., those sequential selection techniques able to learn online using only the feedback given by the chosen option (a. k. a.
Simultaneously Updating All Persistence Values in Reinforcement Learning
In reinforcement learning, the performance of learning agents is highly sensitive to the choice of time discretization.
Dynamical Linear Bandits
Then, the hidden state evolves according to linear dynamics, affected by the performed action too.
Optimizing Empty Container Repositioning and Fleet Deployment via Configurable Semi-POMDPs
With the continuous growth of the global economy and markets, resource imbalance has risen to be one of the central issues in real logistic scenarios.
Storehouse: a Reinforcement Learning Environment for Optimizing Warehouse Management
Warehouse Management Systems have been evolving and improving thanks to new Data Intelligence techniques.
ARLO: A Framework for Automated Reinforcement Learning
In this work, we propose a general and flexible framework, namely ARLO: Automated Reinforcement Learning Optimizer, to construct automated pipelines for AutoRL.
Lifelong Hyper-Policy Optimization with Multiple Importance Sampling Regularization
This hyper-policy is trained to maximize the estimated future performance, efficiently reusing past data by means of importance sampling, at the cost of introducing a controlled bias.
Learning in Non-Cooperative Configurable Markov Decision Processes
This presupposes that the two actors have the same reward functions.
Subgaussian and Differentiable Importance Sampling for Off-Policy Evaluation and Learning
Importance Sampling (IS) is a widely used building block for a large variety of off-policy estimation and learning algorithms.
Exploiting Minimum-Variance Policy Evaluation for Policy Optimization
In this setting, Importance Sampling (IS) is typically employed as a what-if analysis tool, with the goal of estimating the performance of a target policy, given samples collected with a different behavioral policy.
Policy Optimization as Online Learning with Mediator Feedback
In this paper, we introduce the notion of mediator feedback that frames PO as an online learning problem over the policy space.
Control Frequency Adaptation via Action Persistence in Batch Reinforcement Learning
The choice of the control frequency of a system has a relevant impact on the ability of reinforcement learning algorithms to learn a highly performing policy.
Propagating Uncertainty in Reinforcement Learning via Wasserstein Barycenters
How does the uncertainty of the value function propagate when performing temporal difference learning?
Policy Space Identification in Configurable Environments
We study the problem of identifying the policy space of a learning agent, having access to a set of demonstrations generated by its optimal policy.
Gradient-Aware Model-based Policy Search
In this paper, we introduce a novel model-based policy search approach that exploits the knowledge of the current agent policy to learn an approximate transition model, focusing on the portions of the environment that are most relevant for policy improvement.
Feature Selection via Mutual Information: New Theoretical Insights
Mutual information has been successfully adopted in filter feature-selection methods to assess both the relevancy of a subset of features in predicting the target variable and the redundancy with respect to other variables.
Policy Optimization via Importance Sampling
Policy optimization is an effective reinforcement learning approach to solve continuous control tasks.
Configurable Markov Decision Processes
After having introduced our approach and derived some theoretical results, we present the experimental evaluation in two explicative problems to show the benefits of the environment configurability on the performance of the learned policy.
Compatible Reward Inverse Reinforcement Learning
Within this subspace, using a second-order criterion, we search for the reward function that penalizes the most a deviation from the expert's policy.
