Search Results for author:
Found 44 papers, 8 papers with code
Optimal Multi-Fidelity Best-Arm Identification
In bandit best-arm identification, an algorithm is tasked with finding the arm with highest mean reward with a specified accuracy as fast as possible.
Power Mean Estimation in Stochastic Monte-Carlo Tree_Search
Monte-Carlo Tree Search (MCTS) is a widely-used strategy for online planning that combines Monte-Carlo sampling with forward tree search.
Finding good policies in average-reward Markov Decision Processes without prior knowledge
In this case, it is known that there exists an MDP with $D \simeq H$ for which the sample complexity to output an $\varepsilon$-optimal policy is $\Omega(SAD/\varepsilon^2)$ where $S$ and $A$ are the sizes of the state and action spaces.
Bandit Pareto Set Identification: the Fixed Budget Setting
We study a multi-objective pure exploration problem in a multi-armed bandit model.
Towards Instance-Optimality in Online PAC Reinforcement Learning
In this paper, we propose the first instance-dependent lower bound on the sample complexity required for the PAC identification of a near-optimal policy in any tabular episodic MDP.
Active Coverage for PAC Reinforcement Learning
In particular, we obtain a simple algorithm for PAC reward-free exploration with an instance-dependent sample complexity that, in certain MDPs which are "easy to explore", is lower than the minimax one.
Dealing with Unknown Variances in Best-Arm Identification
The problem of identifying the best arm among a collection of items having Gaussian rewards distribution is well understood when the variances are known.
Optimistic PAC Reinforcement Learning: the Instance-Dependent View
Optimistic algorithms have been extensively studied for regret minimization in episodic tabular MDPs, both from a minimax and an instance-dependent view.
Top Two Algorithms Revisited
Top Two algorithms arose as an adaptation of Thompson sampling to best arm identification in multi-armed bandit models (Russo, 2016), for parametric families of arms.
Near-Optimal Collaborative Learning in Bandits
In this paper, we provide new lower bounds on the sample complexity of pure exploration and on the regret.
Efficient Algorithms for Extreme Bandits
In this paper, we contribute to the Extreme Bandit problem, a variant of Multi-Armed Bandits in which the learner seeks to collect the largest possible reward.
Near Instance-Optimal PAC Reinforcement Learning for Deterministic MDPs
In probably approximately correct (PAC) reinforcement learning (RL), an agent is required to identify an $\epsilon$-optimal policy with probability $1-\delta$.
Top-m identification for linear bandits
Motivated by an application to drug repurposing, we propose the first algorithms to tackle the identification of the m $\ge$ 1 arms with largest means in a linear bandit model, in the fixed-confidence setting.
Optimal Thompson Sampling strategies for support-aware CVaR bandits
In this paper we study a multi-arm bandit problem in which the quality of each arm is measured by the Conditional Value at Risk (CVaR) at some level alpha of the reward distribution.
Sub-sampling for Efficient Non-Parametric Bandit Exploration
In this paper we propose the first multi-armed bandit algorithm based on re-sampling that achieves asymptotically optimal regret simultaneously for different families of arms (namely Bernoulli, Gaussian and Poisson distributions).
Episodic Reinforcement Learning in Finite MDPs: Minimax Lower Bounds Revisited
In this paper, we propose new problem-independent lower bounds on the sample complexity and regret in episodic MDPs, with a particular focus on the non-stationary case in which the transition kernel is allowed to change in each stage of the episode.
Fast active learning for pure exploration in reinforcement learning
Realistic environments often provide agents with very limited feedback.
A Kernel-Based Approach to Non-Stationary Reinforcement Learning in Metric Spaces
In this work, we propose KeRNS: an algorithm for episodic reinforcement learning in non-stationary Markov Decision Processes (MDPs) whose state-action set is endowed with a metric.
Adaptive Reward-Free Exploration
Reward-free exploration is a reinforcement learning setting studied by Jin et al. (2020), who address it by running several algorithms with regret guarantees in parallel.
Planning in Markov Decision Processes with Gap-Dependent Sample Complexity
We propose MDP-GapE, a new trajectory-based Monte-Carlo Tree Search algorithm for planning in a Markov Decision Process in which transitions have a finite support.
Kernel-Based Reinforcement Learning: A Finite-Time Analysis
We consider the exploration-exploitation dilemma in finite-horizon reinforcement learning problems whose state-action space is endowed with a metric.
Solving Bernoulli Rank-One Bandits with Unimodal Thompson Sampling
Stochastic Rank-One Bandits (Katarya et al, (2017a, b)) are a simple framework for regret minimization problems over rank-one matrices of arms.
Fixed-Confidence Guarantees for Bayesian Best-Arm Identification
We investigate and provide new insights on the sampling rule called Top-Two Thompson Sampling (TTTS).
On Multi-Armed Bandit Designs for Dose-Finding Clinical Trials
We study the problem of finding the optimal dosage in early stage clinical trials through the multi-armed bandit lens.
Efficient Change-Point Detection for Tackling Piecewise-Stationary Bandits
We introduce GLR-klUCB, a novel algorithm for the piecewise iid non-stationary bandit problem with bounded rewards.
A Practical Algorithm for Multiplayer Bandits when Arm Means Vary Among Players
We study a multiplayer stochastic multi-armed bandit problem in which players cannot communicate, and if two or more players pull the same arm, a collision occurs and the involved players receive zero reward.
Mixture Martingales Revisited with Applications to Sequential Tests and Confidence Intervals
This paper presents new deviation inequalities that are valid uniformly in time under adaptive sampling in a multi-armed bandit model.
Sequential Test for the Lowest Mean: From Thompson to Murphy Sampling
We develop refined non-asymptotic lower bounds, which show that optimality mandates very different sampling behavior for a low vs high true minimum.
What Doubling Tricks Can and Can't Do for Multi-Armed Bandits
In a broad setting, we prove that a geometric doubling trick can be used to conserve (minimax) bounds in $R\_T = O(\sqrt{T})$ but cannot conserve (distribution-dependent) bounds in $R\_T = O(\log T)$.
Pure Exploration in Infinitely-Armed Bandit Models with Fixed-Confidence
We consider the problem of near-optimal arm identification in the fixed confidence setting of the infinitely armed bandit problem when nothing is known about the arm reservoir distribution.
Multi-Player Bandits Revisited
Multi-player Multi-Armed Bandits (MAB) have been extensively studied in the literature, motivated by applications to Cognitive Radio systems.
Corrupt Bandits for Preserving Local Privacy
In this framework, motivated by privacy preservation in online recommender systems, the goal is to maximize the sum of the (unobserved) rewards, based on the observation of transformation of these rewards through a stochastic corruption process with known parameters.
Monte-Carlo Tree Search by Best Arm Identification
Recent advances in bandit tools and techniques for sequential learning are steadily enabling new applications and are promising the resolution of a range of challenging related problems.
Learning the distribution with largest mean: two bandit frameworks
Over the past few years, the multi-armed bandit model has become increasingly popular in the machine learning community, partly because of applications including online content optimization.
Asymptotically Optimal Algorithms for Budgeted Multiple Play Bandits
We study a generalization of the multi-armed bandit problem with multiple plays where there is a cost associated with pulling each arm and the agent has a budget at each time that dictates how much she can expect to spend.
On Explore-Then-Commit Strategies
We study the problem of minimising regret in two-armed bandit problems with Gaussian rewards.
Optimal Best Arm Identification with Fixed Confidence
We give a complete characterization of the complexity of best-arm identification in one-parameter bandit problems.
Maximin Action Identification: A New Bandit Framework for Games
We study an original problem of pure exploration in a strategic bandit model motivated by Monte Carlo Tree Search.
On Bayesian index policies for sequential resource allocation
This paper is about index policies for minimizing (frequentist) regret in a stochastic multi-armed bandit model, inspired by a Bayesian view on the problem.
A Spectral Algorithm with Additive Clustering for the Recovery of Overlapping Communities in Networks
This paper presents a novel spectral algorithm with additive clustering designed to identify overlapping communities in networks.
On the Complexity of Best Arm Identification in Multi-Armed Bandit Models
The stochastic multi-armed bandit model is a simple abstraction that has proven useful in many different contexts in statistics and machine learning.
On the Complexity of A/B Testing
A/B testing refers to the task of determining the best option among two alternatives that yield random outcomes.
Thompson Sampling for 1-Dimensional Exponential Family Bandits
Thompson Sampling has been demonstrated in many complex bandit models, however the theoretical guarantees available for the parametric multi-armed bandit are still limited to the Bernoulli case.
Thompson Sampling: An Asymptotically Optimal Finite Time Analysis
The question of the optimality of Thompson Sampling for solving the stochastic multi-armed bandit problem had been open since 1933.
