Search Results for author:
Found 67 papers, 7 papers with code
Influence Diagram Bandits
We experiment with three structured bandit problems: cascading bandits, online learning to rank in the position-based model, and rank-1 bandits.
Only Pay for What Is Uncertain: Variance-Adaptive Thompson Sampling
The bound for unknown reward variances captures the effect of the prior on learning reward variances and is the first of its kind.
Multiplier Bootstrap-based Exploration
Despite the great interest in the bandit problem, designing efficient algorithms for complex models remains challenging, as there is typically no analytical way to quantify uncertainty.
Selective Uncertainty Propagation in Offline RL
We study the finite-horizon offline reinforcement learning (RL) problem.
Thompson Sampling with Diffusion Generative Prior
In this work, we initiate the idea of using denoising diffusion models to learn priors for online decision making problems.
Multi-Task Off-Policy Learning from Bandit Feedback
We prove per-task bounds on the suboptimality of the learned policies, which show a clear improvement over not using the hierarchical model.
Bayesian Fixed-Budget Best-Arm Identification
We also provide the first lower bound on the probability of misidentification in a $2$-armed Bayesian bandit and show that our upper bound (almost) matches the lower bound.
Robust Contextual Linear Bandits
Our experiments show that RoLinTS is comparably statistically efficient to the classic methods when the misspecification is low, more robust when the misspecification is high, and significantly more computationally efficient than its naive implementation.
From Ranked Lists to Carousels: A Carousel Click Model
Our analytical results show that the user can examine more items in the carousel click model than in a single ranked list, due to the structured way of browsing.
Uplifting Bandits
We introduce a multi-armed bandit model where the reward is a sum of multiple random variables, and each action only alters the distributions of some of them.
Pessimistic Off-Policy Optimization for Learning to Rank
Off-policy learning is a framework for optimizing policies without deploying them, using data collected by another policy.
Mixed-Effect Thompson Sampling
The regret bound has two terms, one for learning the action parameters and the other for learning the shared effect parameters.
Safe Exploration for Efficient Policy Evaluation and Comparison
High-quality data plays a central role in ensuring the accuracy of policy evaluation.
Meta-Learning for Simple Regret Minimization
This is while we show that the meta simple regret of the frequentist algorithm is $\tilde{O}(\sqrt{m} n + m/ \sqrt{n})$, and thus, worse.
Deep Hierarchy in Bandits
We use this exact posterior to analyze the Bayes regret of HierTS in Gaussian bandits.
IMO$^3$: Interactive Multi-Objective Off-Policy Optimization
This problem has been studied extensively in the setting of known objective functions.
Hierarchical Bayesian Bandits
We provide a unified view of all these problems, as learning to act in a hierarchical Bayesian bandit.
Safe Data Collection for Offline and Online Policy Learning
Specifically, our goal is to develop a logging policy that efficiently explores different actions to elicit information while achieving competitive reward with a baseline production policy.
Optimal Probing with Statistical Guarantees for Network Monitoring at Scale
We propose a framework for estimating network metrics, such as latency and packet loss, with guarantees on estimation errors for a fixed monitoring budget.
No Regrets for Learning the Prior in Bandits
We propose ${\tt AdaTS}$, a Thompson sampling algorithm that adapts sequentially to bandit tasks that it interacts with.
Random Effect Bandits
It is well known that side information, such as the prior distribution of arm means in Thompson sampling, can improve the statistical efficiency of the bandit algorithm.
Thompson Sampling with a Mixture Prior
We study Thompson sampling (TS) in online decision making, where the uncertain environment is sampled from a mixture distribution.
Fixed-Budget Best-Arm Identification in Structured Bandits
We analyze our algorithm in linear and generalized linear models (GLMs) and propose a practical implementation based on a G-optimal design.
CORe: Capitalizing On Rewards in Bandit Exploration
We propose a bandit algorithm that explores purely by randomizing its past observations.
Non-Stationary Latent Bandits
The key idea is to frame this problem as a latent bandit, where the prototypical models of user behavior are learned offline and the latent state of the user is inferred online from its interactions with the models.
Differentiable Meta-Learning of Bandit Policies
Exploration policies in Bayesian bandits maximize the average reward over problem instances drawn from some distribution P. In this work, we learn such policies for an unknown distribution P using samples from P. Our approach is a form of meta-learning and exploits properties of P without making strong assumptions about its form.
Influence Diagram Bandits: Variational Thompson Sampling for Structured Bandit Problems
We propose a novel framework for structured bandits, which we call an influence diagram bandit.
Latent Bandits Revisited
A latent bandit problem is one in which the learning agent knows the arm reward distributions conditioned on an unknown discrete latent state.
Non-Stationary Off-Policy Optimization
This approach is practical and analyzable, and we provide guarantees on both the quality of off-policy optimization and the regret during online deployment.
Meta-Learning Bandit Policies by Gradient Ascent
Most bandit policies are designed to either minimize regret in any problem instance, making very few assumptions about the underlying environment, or in a Bayesian sense, assuming a prior distribution over environment parameters.
Sample Efficient Graph-Based Optimization with Noisy Observations
We study sample complexity of optimizing "hill-climbing friendly" functions defined on a graph under noisy observations.
Differentiable Bandit Exploration
In this work, we learn such policies for an unknown distribution $\mathcal{P}$ using samples from $\mathcal{P}$.
Old Dog Learns New Tricks: Randomized UCB for Bandit Problems
We propose $\tt RandUCB$, a bandit strategy that builds on theoretically derived confidence intervals similar to upper confidence bound (UCB) algorithms, but akin to Thompson sampling (TS), it uses randomization to trade off exploration and exploitation.
Randomized Exploration in Generalized Linear Bandits
GLM-TSL samples a generalized linear model (GLM) from the Laplace approximation to the posterior distribution.
Waterfall Bandits: Learning to Sell Ads Online
We design an online learning algorithm for solving this problem, which interleaves learning and optimization, and prove that this algorithm has sublinear regret.
Empirical Bayes Regret Minimization
In this work, we pioneer the idea of algorithm design by minimizing the empirical Bayes regret, the average regret over problem instances sampled from a known distribution.
Perturbed-History Exploration in Stochastic Linear Bandits
Our algorithm, perturbed-history exploration in a linear bandit (LinPHE), estimates a linear model from its perturbed history and pulls the arm with the highest value under that model.
Perturbed-History Exploration in Stochastic Multi-Armed Bandits
Finally, we empirically evaluate PHE and show that it is competitive with state-of-the-art baselines.
Garbage In, Reward Out: Bootstrapping Exploration in Multi-Armed Bandits
Specifically, it pulls the arm with the highest mean reward in a non-parametric bootstrap sample of its history with pseudo rewards.
Online Diverse Learning to Rank from Partial-Click Feedback
We assume that the user examines the list of recommended items until the user is attracted by an item, which is clicked, and does not examine the rest of the items.
BubbleRank: Safe Online Learning to Re-Rank via Implicit Click Feedback
In this paper, we study the problem of safe online learning to re-rank, where user feedback is used to improve the quality of displayed lists.
TopRank: A practical algorithm for online stochastic ranking
Online learning to rank is a sequential decision-making problem where in each round the learning agent chooses a list of items and receives feedback in the form of clicks from the user.
Conservative Exploration using Interleaving
In many practical problems, a learning agent may want to learn the best action in hindsight without ever taking a bad action, which is significantly worse than the default production action.
New Insights into Bootstrapping for Bandits
We investigate the use of bootstrapping in the bandit setting.
Offline Evaluation of Ranking Policies with Click Models
We analyze our estimators and prove that they are more efficient than the estimators that do not use the structure of the click model, under the assumption that the click model holds.
Nearly Optimal Adaptive Procedure with Change Detection for Piecewise-Stationary Bandit
Multi-armed bandit (MAB) is a class of online learning problems where a learning agent aims to maximize its expected cumulative reward while repeatedly selecting to pull arms with unknown reward distributions.
Stochastic Low-Rank Bandits
Many problems in computer vision and recommender systems involve low-rank matrices.
SpectralLeader: Online Spectral Learning for Single Topic Models
We study the problem of learning a latent variable model from a stream of data.
Bernoulli Rank-$1$ Bandits for Click Feedback
The probability that a user will click a search result depends both on its relevance and its position on the results page.
Online Learning to Rank in Stochastic Click Models
In this work, we propose BatchRank, the first online learning to rank algorithm for a broad class of click models.
Model-Independent Online Learning for Influence Maximization
We consider influence maximization (IM) in social networks, which is the problem of maximizing the number of users that become aware of a product by selecting a set of "seed" users to expose the product to.
Does Weather Matter? Causal Analysis of TV Logs
To the best of our knowledge, this is the first large-scale causal study of the impact of weather on TV watching patterns.
Stochastic Rank-1 Bandits
The main challenge of the problem is that the individual values of the row and column are unobserved.
Online Influence Maximization under Independent Cascade Model with Semi-Bandit Feedback
Specifically, we aim to learn the set of "best influencers" in a social network online while repeatedly interacting with it.
Cascading Bandits for Large-Scale Recommendation Problems
In this work, we study cascading bandits, an online learning variant of the cascade model where the goal is to recommend $K$ most attractive items from a large set of $L$ candidate items.
Graphical Model Sketch
Graphical models are a popular approach to modeling structured data but they are unsuitable for high-cardinality variables.
DCM Bandits: Learning to Rank with Multiple Clicks
This work presents the first practical and regret-optimal online algorithm for learning to rank with multiple clicks in a cascade-like click model.
Efficient Thompson Sampling for Online Matrix-Factorization Recommendation
Matrix factorization (MF) collaborative filtering is an effective and widely used method in recommendation systems.
Combinatorial Cascading Bandits
The agent observes the index of the first chosen item whose weight is zero.
Cascading Bandits: Learning to Rank in the Cascade Model
We also prove gap-dependent upper bounds on the regret of these algorithms and derive a lower bound on the regret in cascading bandits.
DUM: Diversity-Weighted Utility Maximization for Recommendations
The need for diversification of recommendation lists manifests in a number of recommender systems use cases.
Tight Regret Bounds for Stochastic Combinatorial Semi-Bandits
A stochastic combinatorial semi-bandit is an online learning problem where at each step a learning agent chooses a subset of ground items subject to constraints, and then observes stochastic weights of these items and receives their sum as a payoff.
Efficient Learning in Large-Scale Combinatorial Semi-Bandits
A stochastic combinatorial semi-bandit is an online learning problem where at each step a learning agent chooses a subset of ground items subject to combinatorial constraints, and then observes stochastic weights of these items and receives their sum as a payoff.
Learning to Act Greedily: Polymatroid Semi-Bandits
Many important optimization problems, such as the minimum spanning tree and minimum-cost flow, can be solved optimally by a greedy method.
Matroid Bandits: Fast Combinatorial Optimization with Learning
The objective in these problems is to learn how to maximize a modular function on a matroid.
Adaptive Submodular Maximization in Bandit Setting
Maximization of submodular functions has wide applications in machine learning and artificial intelligence.
