Search Results for author:
Found 26 papers, 2 papers with code
Beating Adversarial Low-Rank MDPs with Unknown Transition and Bandit Feedback
First, we improve the $poly(d, A, H)T^{5/6}$ regret bound of Zhao et al. (2024) to $poly(d, A, H)T^{2/3}$ for the full-information unknown transition setting, where d is the rank of the transitions, A is the number of actions, H is the horizon length, and T is the number of episodes.
Incentive-compatible Bandits: Importance Weighting No More
In this work we propose the first incentive-compatible algorithms that enjoy $O(\sqrt{KT})$ regret bounds.
Optimal cross-learning for contextual bandits with unknown context distributions
In this setting, we resolve an open problem of Balseiro et al. by providing an efficient algorithm with a nearly tight (up to logarithmic factors) regret bound of $\widetilde{O}(\sqrt{TK})$, independent of the number of contexts.
Towards Optimal Regret in Adversarial Linear MDPs with Bandit Feedback
The first algorithm, although computationally inefficient, ensures a regret of $\widetilde{\mathcal{O}}\left(\sqrt{K}\right)$, where $K$ is the number of episodes.
A Best-of-both-worlds Algorithm for Bandits with Delayed Feedback with Robustness to Excessive Delays
We propose a new best-of-both-worlds algorithm for bandits with variably delayed feedback.
A Blackbox Approach to Best of Both Worlds in Bandits and Beyond
Best-of-both-worlds algorithms for online learning which achieve near-optimal regret in both the adversarial and the stochastic regimes have received growing attention recently.
Best of Both Worlds Policy Optimization
Then we show that under known transitions, we can further obtain a first-order regret bound in the adversarial regime by leveraging the log-barrier regularizer.
Refined Regret for Adversarial MDPs with Linear Function Approximation
This analysis allows the loss estimators to be arbitrarily negative and might be of independent interest.
A Unified Algorithm for Stochastic Path Problems
Our regret bound matches the best known results for the well-studied special case of stochastic shortest path (SSP) with all non-positive rewards.
A Provably Efficient Model-Free Posterior Sampling Method for Episodic Reinforcement Learning
Thompson Sampling is one of the most effective methods for contextual bandits and has been generalized to posterior sampling for certain MDP settings.
A Best-of-Both-Worlds Algorithm for Bandits with Delayed Feedback
We present a modified tuning of the algorithm of Zimmert and Seldin [2020] for adversarial multiarmed bandits with delayed feedback, which in addition to the minimax optimal adversarial regret guarantee shown by Zimmert and Seldin simultaneously achieves a near-optimal regret guarantee in the stochastic setting with fixed delays.
Stochastic Online Learning with Feedback Graphs: Finite-Time and Asymptotic Optimality
We revisit the problem of stochastic online learning with feedback graphs, with the goal of devising algorithms that are optimal, up to constants, both asymptotically and in finite time.
Pushing the Efficiency-Regret Pareto Frontier for Online Learning of Portfolios and Quantum States
This algorithm, called SCHRODINGER'S BISONS, is the first efficient algorithm with polylogarithmic regret for this more general problem.
The Pareto Frontier of model selection for general Contextual Bandits
Recent progress in model selection raises the question of the fundamental limits of these techniques.
A Model Selection Approach for Corruption Robust Reinforcement Learning
We develop a model selection approach to tackle reinforcement learning with adversarial corruption in both transition and reward.
Efficient Methods for Online Multiclass Logistic Regression
Previous work (Foster et al., 2018) has highlighted the importance of improper predictors for achieving "fast rates" in the online multiclass logistic regression problem without suffering exponentially from secondary problem parameters, such as the norm of the predictors in the comparison class.
Adapting to Misspecification in Contextual Bandits
Given access to an online oracle for square loss regression, our algorithm attains optimal regret and -- in particular -- optimal dependence on the misspecification level, with no prior knowledge.
Beyond Value-Function Gaps: Improved Instance-Dependent Regret Bounds for Episodic Reinforcement Learning
Our results show that optimistic algorithms can not achieve the information-theoretic lower bounds even in deterministic MDPs unless there is a unique optimal policy.
Model Selection in Contextual Stochastic Bandit Problems
Our methods rely on a novel and generic smoothing transformation for bandit algorithms that permits us to obtain optimal $O(\sqrt{T})$ model selection guarantees for stochastic contextual bandit problems as long as the optimal base algorithm satisfies a high probability regret guarantee.
Online Learning for Active Cache Synchronization
Existing multi-armed bandit (MAB) models make two implicit assumptions: an arm generates a payoff only when it is played, and the agent observes every payoff that is generated.
An Optimal Algorithm for Adversarial Bandits with Arbitrary Delays
The result requires no advance knowledge of the delays and resolves an open problem of Thune et al. (2019).
Connections Between Mirror Descent, Thompson Sampling and the Information Ratio
The information-theoretic analysis by Russo and Van Roy (2014) in combination with minimax duality has proved a powerful tool for the analysis of online learning algorithms in full and partial information settings.
Beating Stochastic and Adversarial Semi-bandits Optimally and Simultaneously
We develop the first general semi-bandit algorithm that simultaneously achieves $\mathcal{O}(\log T)$ regret for stochastic environments and $\mathcal{O}(\sqrt{T})$ regret for adversarial environments without knowledge of the regime or the number of rounds $T$.
Tsallis-INF: An Optimal Algorithm for Stochastic and Adversarial Bandits
More generally, we define an adversarial regime with a self-bounding constraint, which includes stochastic regime, stochastically constrained adversarial regime (Wei and Luo), and stochastic regime with adversarial corruptions (Lykouris et al.) as special cases, and show that the algorithm achieves logarithmic regret guarantee in this regime and all of its special cases simultaneously with the adversarial regret guarantee.}
Factored Bandits
We introduce the factored bandits model, which is a framework for learning with limited (bandit) feedback, where actions can be decomposed into a Cartesian product of atomic actions.
Distributed Optimization of Multi-Class SVMs
Training of one-vs.-rest SVMs can be parallelized over the number of classes in a straight forward way.
