Search Results for author:
Found 30 papers, 3 papers with code
Non-stationary Bandits and Meta-Learning with a Small Set of Optimal Arms
We study a sequential decision problem where the learner faces a sequence of $K$-armed bandit tasks.
A New Look at Dynamic Regret for Non-Stationary Stochastic Bandits
We propose a method that achieves, in $K$-armed bandit problems, a near-optimal $\widetilde O(\sqrt{K N(S+1)})$ dynamic regret, where $N$ is the time horizon of the problem and $S$ is the number of times the identity of the optimal arm changes, without prior knowledge of $S$.
Efficient Local Planning with Linear Function Approximation
Under the assumption that the Q-functions of all policies are linear in known features of the state-action pairs, we show that our algorithms have polynomial query and computational costs in the dimension of the features, the effective planning horizon, and the targeted sub-optimality, while these costs are independent of the size of the state space.
Feature and Parameter Selection in Stochastic Linear Bandits
In the second setting, the reward parameter of the LB problem is arbitrarily selected from $M$ models represented as (possibly) overlapping balls in $\mathbb R^d$.
Improved Regret Bound and Experience Replay in Regularized Policy Iteration
We first show that the regret analysis of the Politex algorithm (a version of regularized policy iteration) can be sharpened from $O(T^{3/4})$ to $O(\sqrt{T})$ under nearly identical assumptions, and instantiate the bound with linear function approximation.
Optimization Issues in KL-Constrained Approximate Policy Iteration
We compare the use of KL divergence as a constraint vs. as a regularizer, and point out several optimization issues with the widely-used constrained approach.
On Query-efficient Planning in MDPs under Linear Realizability of the Optimal State-value Function
We consider local planning in fixed-horizon MDPs with a generative model under the assumption that the optimal value function lies close to the span of a feature map.
The Elliptical Potential Lemma Revisited
This note proposes a new proof and new perspectives on the so-called Elliptical Potential Lemma.
Regret Balancing for Bandit and RL Model Selection
Given a set of base learning algorithms, an effective model selection strategy adapts to the best learning algorithm in an online fashion.
Sample Efficient Graph-Based Optimization with Noisy Observations
We study sample complexity of optimizing "hill-climbing friendly" functions defined on a graph under noisy observations.
Model Selection in Contextual Stochastic Bandit Problems
Our methods rely on a novel and generic smoothing transformation for bandit algorithms that permits us to obtain optimal $O(\sqrt{T})$ model selection guarantees for stochastic contextual bandit problems as long as the optimal base algorithm satisfies a high probability regret guarantee.
Adaptive Approximate Policy Iteration
This is an improvement over the best existing bound of $\tilde{O}(T^{3/4})$ for the average-reward case with function approximation.
Exploration-Enhanced POLITEX
POLITEX has sublinear regret guarantees in uniformly-mixing MDPs when the value estimation error can be controlled, which can be satisfied if all policies sufficiently explore the environment.
Thompson Sampling with Approximate Inference
We study the effects of approximate inference on the performance of Thompson sampling in the $k$-armed bandit problems.
Bootstrapping Upper Confidence Bound
Upper Confidence Bound (UCB) method is arguably the most celebrated one used in online decision making with partial information feedback.
Large-Scale Markov Decision Problems via the Linear Programming Dual
Moreover, we propose an efficient algorithm, scaling with the size of the subspace but not the state space, that is able to find a policy with low excess loss relative to the best policy in this class.
New Insights into Bootstrapping for Bandits
We investigate the use of bootstrapping in the bandit setting.
Sharp convergence rates for Langevin dynamics in the nonconvex setting
We study the problem of sampling from a distribution $p^*(x) \propto \exp\left(-U(x)\right)$, where the function $U$ is $L$-smooth everywhere and $m$-strongly convex outside a ball of radius $R$, but potentially nonconvex inside this ball.
Offline Evaluation of Ranking Policies with Click Models
We analyze our estimators and prove that they are more efficient than the estimators that do not use the structure of the click model, under the assumption that the click model holds.
Model-Free Linear Quadratic Control via Reduction to Expert Prediction
Model-free approaches for reinforcement learning (RL) and continuous control find policies based only on past states and rewards, without fitting a model of the system dynamics.
Optimizing over a Restricted Policy Class in Markov Decision Processes
However, under a condition that is akin to the occupancy measures of the base policies having large overlap, we show that there exists an efficient algorithm that finds a policy that is almost as good as the best convex combination of the base policies.
A Continuation Method for Discrete Optimization and its Application to Nearest Neighbor Classification
We show the effectiveness of the proposed technique in the problem of nearest neighbor classification.
Stochastic Low-Rank Bandits
Many problems in computer vision and recommender systems involve low-rank matrices.
Posterior Sampling for Large Scale Reinforcement Learning
Our algorithm termed deterministic schedule PSRL (DS-PSRL) is efficient in terms of time, sample, and space complexity.
Conservative Contextual Linear Bandits
We prove an upper-bound on the regret of CLUCB and show that it can be decomposed into two terms: 1) an upper-bound for the regret of the standard linear UCB algorithm that grows with the time horizon and 2) a constant (does not grow with the time horizon) term that accounts for the loss of being conservative in order to satisfy the safety constraint.
Hit-and-Run for Sampling and Planning in Non-Convex Spaces
We propose the Hit-and-Run algorithm for planning and sampling problems in non-convex spaces.
Online learning in MDPs with side information
We study online learning of finite Markov decision process (MDP) problems when a side information vector is available.
Bayesian Optimal Control of Smoothly Parameterized Systems: The Lazy Posterior Sampling Algorithm
We study Bayesian optimal control of a general class of smoothly parameterized Markov decision problems.
Linear Programming for Large-Scale Markov Decision Problems
We consider the problem of controlling a Markov decision process (MDP) with a large state space, so as to minimize average cost.
Improved Algorithms for Linear Stochastic Bandits
We improve the theoretical analysis and empirical performance of algorithms for the stochastic multi-armed bandit problem and the linear stochastic multi-armed bandit problem.
