Search Results for author:
Found 77 papers, 19 papers with code
Causal Feature Selection via Transfer Entropy
We introduce a new causal feature selection approach that relies on the forward and backward feature selection procedures and leverages transfer entropy to estimate the causal flow of information from the features to the target in time series.
Exploiting Causal Graph Priors with Posterior Sampling for Reinforcement Learning
Posterior sampling allows the exploitation of prior knowledge of the environment's transition dynamics to improve the sample efficiency of reinforcement learning.
Pure Exploration under Mediators' Feedback
In this setting, the agent's goal lies in sequentially choosing which mediator to query to identify with high probability the optimal arm while minimizing the identification time, i. e., the sample complexity.
Nonlinear Feature Aggregation: Two Algorithms driven by Theory
A limitation of methods based on correlation is the assumption of linearity in the relationship between features and target.
Stepsize Learning for Policy Gradient Methods in Contextual Markov Decision Processes
Policy-based algorithms are among the most widely adopted techniques in model-free RL, thanks to their strong theoretical groundings and good properties in continuous action spaces.
An Option-Dependent Analysis of Regret Minimization Algorithms in Finite-Horizon Semi-Markov Decision Processes
Then, focusing on a sub-setting of HRL approaches, the options framework, we highlight how the average duration of the available options affects the planning horizon and, consequently, the regret itself.
Hierarchical Reinforcement Learning
reinforcement-learning
+1
Truncating Trajectories in Monte Carlo Reinforcement Learning
In Reinforcement Learning (RL), an agent acts in an unknown environment to maximize the expected cumulative discounted sum of an external reward signal, i. e., the expected return.
Towards Theoretical Understanding of Inverse Reinforcement Learning
We start by formally introducing the problem of estimating the feasible reward set, the corresponding PAC requirement, and discussing the properties of particular classes of rewards.
A Tale of Sampling and Estimation in Discounted Reinforcement Learning
In this paper, we present a minimax lower bound on the discounted mean estimation problem that explicitly connects the estimation error with the mixing properties of the Markov process and the discount factor.
Interpretable Linear Dimensionality Reduction based on Bias-Variance Analysis
Instead, dimensionality reduction techniques are designed to limit the number of features in a dataset by projecting them into a lower-dimensional space, possibly considering all the original features.
Information-Theoretic Regret Bounds for Bandits with Fixed Expert Advice
We investigate the problem of bandits with expert advice when the experts are fixed and known distributions over the actions.
Wasserstein Actor-Critic: Directed Exploration via Optimism for Continuous-Actions Control
Uncertainty quantification has been extensively used as a means to achieve efficient directed exploration in Reinforcement Learning (RL).
Best Arm Identification for Stochastic Rising Bandits
Then, we prove that, with a sufficiently large budget, they provide guarantees on the probability of properly identifying the optimal option at the end of the learning process.
Autoregressive Bandits
Autoregressive processes naturally arise in a large variety of real-world scenarios, including e. g., stock markets, sell forecasting, weather prediction, advertising, and pricing.
Tight Performance Guarantees of Imitator Policies with Continuous Actions
In this paper, we study BC with the goal of providing theoretical guarantees on the performance of the imitator policy in the case of continuous actions.
Stochastic Rising Bandits
This paper is in the field of stochastic Multi-Armed Bandits (MABs), i. e., those sequential selection techniques able to learn online using only the feedback given by the chosen option (a. k. a.
Simultaneously Updating All Persistence Values in Reinforcement Learning
In reinforcement learning, the performance of learning agents is highly sensitive to the choice of time discretization.
Dynamic Pricing with Volume Discounts in Online Settings
We ran a real-world 4-month-long A/B testing experiment in collaboration with an Italian e-commerce company, in which our algorithm PVD-B-corresponding to A configuration-has been compared with human pricing specialists-corresponding to B configuration.
Dynamical Linear Bandits
Then, the hidden state evolves according to linear dynamics, affected by the performed action too.
Optimizing Empty Container Repositioning and Fleet Deployment via Configurable Semi-POMDPs
With the continuous growth of the global economy and markets, resource imbalance has risen to be one of the central issues in real logistic scenarios.
Storehouse: a Reinforcement Learning Environment for Optimizing Warehouse Management
Warehouse Management Systems have been evolving and improving thanks to new Data Intelligence techniques.
Analysis, Characterization, Prediction and Attribution of Extreme Atmospheric Events with Machine Learning: a Review
The accurate prediction, characterization, and attribution of atmospheric EEs is therefore a key research field, in which many groups are currently working by applying different methodologies and computational tools.
Multi-Armed Bandit Problem with Temporally-Partitioned Rewards: When Partial Feedback Counts
We provide two algorithms to address TP-MAB problems, namely, TP-UCB-FR and TP-UCB-EW, which exploit the partial information disclosed by the reward collected over time.
ARLO: A Framework for Automated Reinforcement Learning
In this work, we propose a general and flexible framework, namely ARLO: Automated Reinforcement Learning Optimizer, to construct automated pipelines for AutoRL.
Delayed Reinforcement Learning by Imitation
When the agent's observations or interactions are delayed, classic reinforcement learning tools usually fail.
Reward-Free Policy Space Compression for Reinforcement Learning
In this paper, we seek for a reward-free compression of the policy space into a finite set of representative policies, such that, given any policy $\pi$, the minimum R\'enyi divergence between the state-action distributions of the representative policies and the state-action distribution of $\pi$ is bounded.
Provably Efficient Causal Model-Based Reinforcement Learning for Systematic Generalization
In this setting, the agent can take a finite amount of reward-free interactions from a subset of these environments.
The Importance of Non-Markovianity in Maximum State Entropy Exploration
In the maximum state entropy exploration framework, an agent interacts with a reward-free environment to learn a policy that maximizes the entropy of the expected state visitations it is inducing.
Challenging Common Assumptions in Convex Reinforcement Learning
In particular, we show that erroneously optimizing the infinite trials objective in place of the actual finite trials one, as it is usually done, can lead to a significant approximation error.
Unsupervised Reinforcement Learning in Multiple Environments
Along this line, we address the problem of unsupervised reinforcement learning in a class of multiple environments, in which the policy is pre-trained with interactions from the whole class, and then fine-tuned for several tasks in any environment of the class.
Lifelong Hyper-Policy Optimization with Multiple Importance Sampling Regularization
This hyper-policy is trained to maximize the estimated future performance, efficiently reusing past data by means of importance sampling, at the cost of introducing a controlled bias.
Learning in Non-Cooperative Configurable Markov Decision Processes
This presupposes that the two actors have the same reward functions.
Subgaussian and Differentiable Importance Sampling for Off-Policy Evaluation and Learning
Importance Sampling (IS) is a widely used building block for a large variety of off-policy estimation and learning algorithms.
Reinforcement Learning in Linear MDPs: Constant Regret and Representation Selection
We study the role of the representation of state-action value functions in regret minimization in finite-horizon Markov Decision Processes (MDPs) with linear structure.
Exploiting Minimum-Variance Policy Evaluation for Policy Optimization
In this setting, Importance Sampling (IS) is typically employed as a what-if analysis tool, with the goal of estimating the performance of a target policy, given samples collected with a different behavioral policy.
Goal-Directed Planning via Hindsight Experience Replay
It has been extended from complex continuous domains through function approximators to bias the search of the planning tree in AlphaZero.
Learning to Explore Multiple Environments without Rewards
Along this line, we address the problem of learning to explore a class of multiple reward-free environments with a unique general strategy, which aims to provide a universal initialization to subsequent reinforcement learning problems specified over the same class.
Meta Learning the Step Size in Policy Gradient Methods
Policy-based algorithms are among the most widely adopted techniques in model-free RL, thanks to their strong theoretical groundings and good properties in continuous action spaces.
Meta-Reinforcement Learning by Tracking Task Non-stationarity
At test time, TRIO tracks the evolution of the latent parameters online, hence reducing the uncertainty over future tasks and obtaining fast adaptation through the meta-learned policy.
Leveraging Good Representations in Linear Contextual Bandits
We show that the regret is indeed never worse than the regret obtained by running LinUCB on the best representation (up to a $\ln M$ factor).
A Practical Guide to Multi-Objective Reinforcement Learning and Planning
Real-world decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives.
Learning to Explore a Class of Multiple Reward-Free Environments
Along this line, we address the problem of learning to explore a class of multiple reward-free environments with a unique general strategy, which aims to provide a universal initialization to subsequent reinforcement learning problems specified over the same class.
Policy Optimization as Online Learning with Mediator Feedback
In this paper, we introduce the notion of mediator feedback that frames PO as an online learning problem over the policy space.
Option Hedging with Risk Averse Reinforcement Learning
In this paper we show how risk-averse reinforcement learning can be used to hedge options.
An Asymptotically Optimal Primal-Dual Incremental Algorithm for Contextual Linear Bandits
Finally, we remove forced exploration and build on confidence intervals of the optimization problem to encourage a minimum level of exploration that is better adapted to the problem structure.
Newton Optimization on Helmholtz Decomposition for Continuous Games
In this paper, we propose NOHD (Newton Optimization on Helmholtz Decomposition), a Newton-like algorithm for multi-agent learning problems based on the decomposition of the dynamics of the system in its irrotational (Potential) and solenoidal (Hamiltonian) component.
Inverse Reinforcement Learning from a Gradient-based Learner
Inverse Reinforcement Learning addresses the problem of inferring an expert's reward function from demonstrations.
Task-Agnostic Exploration via Policy Gradient of a Non-Parametric State Entropy Estimate
In a reward-free environment, what is a suitable intrinsic objective for an agent to pursue so that it can learn an optimal task-agnostic exploration policy?
Sequential Transfer in Reinforcement Learning with a Generative Model
We are interested in how to design reinforcement learning agents that provably reduce the sample complexity for learning new tasks by transferring knowledge from previously-solved ones.
A Policy Gradient Method for Task-Agnostic Exploration
In a reward-free environment, what is a suitable intrinsic objective for an agent to pursue so that it can learn an optimal task-agnostic exploration policy?
Time-Variant Variational Transfer for Value Functions
In most of the transfer learning approaches to reinforcement learning (RL) the distribution over the tasks is assumed to be stationary.
A Novel Confidence-Based Algorithm for Structured Bandits
We study finite-armed stochastic bandits where the rewards of each arm might be correlated to those of other arms.
Sharing Knowledge in Multi-Task Deep Reinforcement Learning
We study the benefit of sharing representations among tasks to enable the effective use of deep neural networks in Multi-Task Reinforcement Learning.
Online Joint Bid/Daily Budget Optimization of Internet Advertising Campaigns
We experimentally evaluate our algorithms with synthetic settings generated from real data from Yahoo!, and we present the results of the adoption of our algorithms in a real-world application with a daily average spent of 1, 000 Euros for more than one year.
Control Frequency Adaptation via Action Persistence in Batch Reinforcement Learning
The choice of the control frequency of a system has a relevant impact on the ability of reinforcement learning algorithms to learn a highly performing policy.
MushroomRL: Simplifying Reinforcement Learning Research
MushroomRL is an open-source Python library developed to simplify the process of implementing and running Reinforcement Learning (RL) experiments.
Risk-Averse Trust Region Optimization for Reward-Volatility Reduction
In real-world decision-making problems, for instance in the fields of finance, robotics or autonomous driving, keeping uncertainty under control is as important as maximizing expected returns.
Propagating Uncertainty in Reinforcement Learning via Wasserstein Barycenters
How does the uncertainty of the value function propagate when performing temporal difference learning?
Gradient-Aware Model-based Policy Search
In this paper, we introduce a novel model-based policy search approach that exploits the knowledge of the current agent policy to learn an approximate transition model, focusing on the portions of the environment that are most relevant for policy improvement.
Policy Space Identification in Configurable Environments
We study the problem of identifying the policy space of a learning agent, having access to a set of demonstrations generated by its optimal policy.
Feature Selection via Mutual Information: New Theoretical Insights
Mutual information has been successfully adopted in filter feature-selection methods to assess both the relevancy of a subset of features in predicting the target variable and the redundancy with respect to other variables.
An Intrinsically-Motivated Approach for Learning Highly Exploring and Fast Mixing Policies
What is a good exploration strategy for an agent that interacts with an environment in the absence of external rewards?
Smoothing Policies and Safe Policy Gradients
Policy Gradient (PG) algorithms are among the best candidates for the much-anticipated applications of reinforcement learning to real-world control tasks, such as robotics.
Transfer of Value Functions via Variational Methods
We consider the problem of transferring value functions in reinforcement learning.
Policy Optimization via Importance Sampling
Policy optimization is an effective reinforcement learning approach to solve continuous control tasks.
Configurable Markov Decision Processes
After having introduced our approach and derived some theoretical results, we present the experimental evaluation in two explicative problems to show the benefits of the environment configurability on the performance of the learned policy.
Stochastic Variance-Reduced Policy Gradient
In this paper, we propose a novel reinforcement- learning algorithm consisting in a stochastic variance-reduced version of policy gradient for solving Markov Decision Processes (MDPs).
Importance Weighted Transfer of Samples in Reinforcement Learning
In the proposed approach, all the samples are transferred and used by a batch RL algorithm to solve the target task, but their contribution to the learning process is proportional to their importance weight.
Cost-Sensitive Approach to Batch Size Adaptation for Gradient Descent
In this paper, we propose a novel approach to automatically determine the batch size in stochastic gradient descent methods.
Compatible Reward Inverse Reinforcement Learning
Within this subspace, using a second-order criterion, we search for the reward function that penalizes the most a deviation from the expert's policy.
Adaptive Batch Size for Safe Policy Gradients
Policy gradient methods are among the best Reinforcement Learning (RL) techniques to solve complex control problems.
Boosted Fitted Q-Iteration
This paper is about the study of B-FQI, an Approximated Value Iteration (AVI) algorithm that exploits a boosting procedure to estimate the action-value function in reinforcement learning problems.
Unimodal Thompson Sampling for Graph-Structured Arms
We study, to the best of our knowledge, the first Bayesian algorithm for unimodal Multi-Armed Bandit (MAB) problems with graph structure.
Sparse Multi-Task Reinforcement Learning
This is equivalent to assuming that the weight vectors of the task value functions are \textit{jointly sparse}, i. e., the set of their non-zero components is small and it is shared across tasks.
Multi-objective Reinforcement Learning with Continuous Pareto Frontier Approximation Supplementary Material
The paper is about learning a continuous approximation of the Pareto frontier in Multi-Objective Markov Decision Problems (MOMDPs).
Multi-Objective Reinforcement Learning
reinforcement-learning
Adaptive Step-Size for Policy Gradient Methods
In the last decade, policy gradient methods have significantly grown in popularity in the reinforcement--learning field.
Transfer from Multiple MDPs
Transfer reinforcement learning (RL) methods leverage on the experience collected on a set of source tasks to speed-up RL algorithms.
