Search Results for author:
Found 43 papers, 5 papers with code
Restarted Bayesian Online Change-point Detector achieves Optimal Detection Delay
In this paper, we consider the problem of sequential change-point detection where both the change-points and the distributions before and after the change are assumed to be unknown.
CRIMED: Lower and Upper Bounds on Regret for Bandits with Unbounded Stochastic Corruption
In this setting, accommodating potentially unbounded corruptions, we establish a problem-dependent lower bound on regret for a given family of arm distributions.
Monte-Carlo tree search with uncertainty propagation via optimal transport
We introduce a novel backup operator that computes value nodes as the Wasserstein barycenter of their action-value children nodes; thus, propagating the uncertainty of the estimate across the tree to the root node.
AdaStop: adaptive statistical testing for sound comparisons of Deep RL agents
When comparing several RL algorithms, a major question is how many executions must be made and how can we ensure that the results of such a comparison are theoretically sound.
Bilinear Exponential Family of MDPs: Frequentist Regret Bound with Tractable Exploration and Planning
Our regret bound is order-optimal with respect to $H$ and $K$.
Risk-aware linear bandits with convex loss
In decision-making problems such as the multi-armed bandit, an agent learns sequentially by optimizing a certain feedback.
Collaborative Algorithms for Online Personalized Mean Estimation
We study the case where some of the distributions have the same mean, and the agents are allowed to actively query information from other agents.
gym-DSSAT: a crop model turned into a Reinforcement Learning environment
gym-DSSAT is a gym interface to the Decision Support System for Agrotechnology Transfer (DSSAT), a high fidelity crop simulator.
Bandits Corrupted by Nature: Lower Bounds on Regret and Robust Optimistic Algorithm
We study the corrupted bandit problem, i. e. a stochastic multi-armed bandit problem with $k$ unknown reward distributions, which are heavy-tailed and corrupted by a history-independent adversary or Nature.
Bregman Deviations of Generic Exponential Families
For the practitioner, we instantiate this novel bound to several classical families, e. g., Gaussian, Bernoulli, Exponential, Weibull, Pareto, Poisson and Chi-square yielding explicit forms of the confidence sets and the Bregman information gain.
Indexed Minimum Empirical Divergence for Unimodal Bandits
We consider a multi-armed bandit problem specified by a set of one-dimensional family exponential distributions endowed with a unimodal structure.
From Optimality to Robustness: Adaptive Re-Sampling Strategies in Stochastic Bandits
The stochastic multi-arm bandit problem has been extensively studied under standard assumptions on the arm's distribution (e. g bounded with known support, exponential family, etc).
Stochastic bandits with groups of similar arms.
For this structured problem of practical relevance, we first derive the asymptotic regret lower bound and corresponding constrained optimization problem.
From Optimality to Robustness: Dirichlet Sampling Strategies in Stochastic Bandits
The stochastic multi-arm bandit problem has been extensively studied under standard assumptions on the arm's distribution (e. g bounded with known support, exponential family, etc).
Sub-sampling for Efficient Non-Parametric Bandit Exploration
In this paper we propose the first multi-armed bandit algorithm based on re-sampling that achieves asymptotically optimal regret simultaneously for different families of arms (namely Bernoulli, Gaussian and Poisson distributions).
Learning Value Functions in Deep Policy Gradients using Residual Variance
We prove the theoretical consistency of the new gradient estimator and observe dramatic empirical improvement across a variety of continuous control tasks and algorithms.
Improved Exploration in Factored Average-Reward MDPs
We consider a regret minimization task under the average-reward criterion in an unknown Factored Markov Decision Process (FMDP).
Robust-Adaptive Interval Predictive Control for Linear Uncertain Systems
We consider the problem of stabilization of a linear system, under state and control constraints, and subject to bounded disturbances and unknown parameters in the state matrix.
Optimal Strategies for Graph-Structured Bandits
[0, 1]^{\mathcal{A}\times\mathcal{B}}$ and by a given weight matrix $\omega\!=\!
Tightening Exploration in Upper Confidence Reinforcement Learning
In pursuit of practical efficiency, we present UCRL3, following the lines of UCRL2, but with two key modifications: First, it uses state-of-the-art time-uniform concentration inequalities to compute confidence sets on the reward and (component-wise) transition distributions for each state-action pair.
Robust-Adaptive Control of Linear Systems: beyond Quadratic Costs
We consider the problem of robust and adaptive model predictive control (MPC) of a linear system, with unknown parameters that are learned along the way (adaptive), in a critical setting where failures must be prevented (robust).
Regret Bounds for Learning State Representations in Reinforcement Learning
We consider the problem of online reinforcement learning when several state representations (mapping histories to a discrete state space) are available to the learning agent.
Model-Based Reinforcement Learning Exploiting State-Action Equivalence
In the case of an unknown equivalence structure, we show through numerical experiments that C-UCRL combined with ApproxEquivalence outperforms UCRL2 in ergodic MDPs.
Model-based Reinforcement Learning
reinforcement-learning
+1
Distribution-dependent and Time-uniform Bounds for Piecewise i.i.d Bandits
The second strategy \ImpCPD makes use of the knowledge of $T$ to achieve the order optimal regret bound of $\min\big\lbrace O(\sum\limits_{i=1}^{K} \sum\limits_{g=1}^{G}\frac{\log(T/H_{1, g})}{\Delta^{opt}_{i, g}}), O(\sqrt{GT})\big\rbrace$, (where $H_{1, g}$ is the problem complexity) thereby closing an important gap with respect to the lower bound in a specific challenging setting.
Learning Multiple Markov Chains via Adaptive Allocation
We study the problem of learning the transition matrices of a set of Markov chains from a single stream of observations on each chain.
Practical Open-Loop Optimistic Planning
We consider the problem of online planning in a Markov Decision Process when given only access to a generative model, restricted to open-loop policies - i. e. sequences of actions - and under budget constraint.
Budgeted Reinforcement Learning in Continuous State Space
A Budgeted Markov Decision Process (BMDP) is an extension of a Markov Decision Process to critical applications requiring safety constraints.
Approximate Robust Control of Uncertain Dynamical Systems
This work studies the design of safe control policies for large-scale non-linear systems operating in uncertain environments.
Systems and Control Robotics
Efficient Change-Point Detection for Tackling Piecewise-Stationary Bandits
We introduce GLR-klUCB, a novel algorithm for the piecewise iid non-stationary bandit problem with bounded rewards.
Variance-Aware Regret Bounds for Undiscounted Reinforcement Learning in MDPs
The problem of reinforcement learning in an unknown and discrete Markov Decision Process (MDP) under the average-reward criterion is considered, when the learner interacts with the system in a single stream of observations, starting from an initial state without any reset.
Efficient tracking of a growing number of experts
By contrast, designing strategies that both achieve a near-optimal regret and maintain a reasonable number of weights is highly non-trivial.
Streaming kernel regression with provably adaptive mean, variance, and regularization
The variance of the noise is not assumed to be known.
Spectral Learning from a Single Trajectory under Finite-State Policies
We present spectral methods of moments for learning sequential models from a single trajectory, in stark contrast with the classical literature that assumes the availability of multiple i. i. d.
Boundary Crossing Probabilities for General Exponential Families
We consider parametric exponential families of dimension $K$ on the real line.
Random Shuffling and Resets for the Non-stationary Stochastic Bandit Problem
For the best-arm identification task, we introduce a version of Successive Elimination based on random shuffling of the $K$ arms.
Low-rank Bandits with Latent Mixtures
This induces a low-rank structure on the matrix of expected rewards r a, b from recommending item a to user b.
How hard is my MDP?" The distribution-norm to the rescue"
In Reinforcement Learning (RL), state-of-the-art algorithms require a large number of samples per state-action pair to estimate the transition kernel $p$.
Selecting Near-Optimal Approximate State Representations in Reinforcement Learning
We consider a reinforcement learning setting introduced in (Maillard et al., NIPS 2011) where the learner does not have explicit access to the states of the underlying Markov decision process (MDP).
Online allocation and homogeneous partitioning for piecewise constant mean-approximation
We here consider an extension of this problem to the case when the arms are the cells of a finite partition P of a continuous sampling space X \subset \Real^d.
Hierarchical Optimistic Region Selection driven by Curiosity
This paper aims to take a step forwards making the term ``intrinsic motivation'' from reinforcement learning theoretically well founded, focusing on curiosity-driven learning.
Selecting the State-Representation in Reinforcement Learning
Without knowing neither which of the models is the correct one, nor what are the probabilistic characteristics of the resulting MDP, it is required to obtain as much reward as the optimal policy for the correct model (or for the best of the correct models, if there are several).
Sparse Recovery with Brownian Sensing
We consider the problem of recovering the parameter alpha in R^K of a sparse function f, i. e. the number of non-zero entries of alpha is small compared to the number K of features, given noisy evaluations of f at a set of well-chosen sampling points.
