Search Results for author:
Found 75 papers, 8 papers with code
Online Influence Maximization under Independent Cascade Model with Semi-Bandit Feedback
Specifically, we aim to learn the set of "best influencers" in a social network online while repeatedly interacting with it.
Cascading Bandits for Large-Scale Recommendation Problems
In this work, we study cascading bandits, an online learning variant of the cascade model where the goal is to recommend $K$ most attractive items from a large set of $L$ candidate items.
Old Dog Learns New Tricks: Randomized UCB for Bandit Problems
We propose $\tt RandUCB$, a bandit strategy that builds on theoretically derived confidence intervals similar to upper confidence bound (UCB) algorithms, but akin to Thompson sampling (TS), it uses randomization to trade off exploration and exploitation.
Pre-trained Recommender Systems: A Causal Debiasing Perspective
In particular, we propose to develop a generic recommender that captures universal interaction patterns by training on generic user-item interaction data extracted from different domains, which can then be fast adapted to improve few-shot learning performance in unseen new domains (with limited data).
Mixed-Effect Thompson Sampling
The regret bound has two terms, one for learning the action parameters and the other for learning the shared effect parameters.
BubbleRank: Safe Online Learning to Re-Rank via Implicit Click Feedback
In this paper, we study the problem of safe online learning to re-rank, where user feedback is used to improve the quality of displayed lists.
Offline Evaluation of Ranking Policies with Click Models
We analyze our estimators and prove that they are more efficient than the estimators that do not use the structure of the click model, under the assumption that the click model holds.
TopRank: A practical algorithm for online stochastic ranking
Online learning to rank is a sequential decision-making problem where in each round the learning agent chooses a list of items and receives feedback in the form of clicks from the user.
Conservative Exploration using Interleaving
In many practical problems, a learning agent may want to learn the best action in hindsight without ever taking a bad action, which is significantly worse than the default production action.
Model-Independent Online Learning for Influence Maximization
We consider influence maximization (IM) in social networks, which is the problem of maximizing the number of users that become aware of a product by selecting a set of "seed" users to expose the product to.
New Insights into Bootstrapping for Bandits
We investigate the use of bootstrapping in the bandit setting.
SpectralLeader: Online Spectral Learning for Single Topic Models
We study the problem of learning a latent variable model from a stream of data.
Nearly Optimal Adaptive Procedure with Change Detection for Piecewise-Stationary Bandit
Multi-armed bandit (MAB) is a class of online learning problems where a learning agent aims to maximize its expected cumulative reward while repeatedly selecting to pull arms with unknown reward distributions.
Stochastic Low-Rank Bandits
Many problems in computer vision and recommender systems involve low-rank matrices.
Online Learning to Rank in Stochastic Click Models
In this work, we propose BatchRank, the first online learning to rank algorithm for a broad class of click models.
Does Weather Matter? Causal Analysis of TV Logs
To the best of our knowledge, this is the first large-scale causal study of the impact of weather on TV watching patterns.
Bernoulli Rank-$1$ Bandits for Click Feedback
The probability that a user will click a search result depends both on its relevance and its position on the results page.
Stochastic Rank-1 Bandits
The main challenge of the problem is that the individual values of the row and column are unobserved.
Efficient Learning in Large-Scale Combinatorial Semi-Bandits
A stochastic combinatorial semi-bandit is an online learning problem where at each step a learning agent chooses a subset of ground items subject to combinatorial constraints, and then observes stochastic weights of these items and receives their sum as a payoff.
Graphical Model Sketch
Graphical models are a popular approach to modeling structured data but they are unsuitable for high-cardinality variables.
DCM Bandits: Learning to Rank with Multiple Clicks
This work presents the first practical and regret-optimal online algorithm for learning to rank with multiple clicks in a cascade-like click model.
Combinatorial Cascading Bandits
The agent observes the index of the first chosen item whose weight is zero.
Cascading Bandits: Learning to Rank in the Cascade Model
We also prove gap-dependent upper bounds on the regret of these algorithms and derive a lower bound on the regret in cascading bandits.
Tight Regret Bounds for Stochastic Combinatorial Semi-Bandits
A stochastic combinatorial semi-bandit is an online learning problem where at each step a learning agent chooses a subset of ground items subject to constraints, and then observes stochastic weights of these items and receives their sum as a payoff.
Learning to Act Greedily: Polymatroid Semi-Bandits
Many important optimization problems, such as the minimum spanning tree and minimum-cost flow, can be solved optimally by a greedy method.
DUM: Diversity-Weighted Utility Maximization for Recommendations
The need for diversification of recommendation lists manifests in a number of recommender systems use cases.
Matroid Bandits: Fast Combinatorial Optimization with Learning
The objective in these problems is to learn how to maximize a modular function on a matroid.
Online Diverse Learning to Rank from Partial-Click Feedback
We assume that the user examines the list of recommended items until the user is attracted by an item, which is clicked, and does not examine the rest of the items.
Garbage In, Reward Out: Bootstrapping Exploration in Multi-Armed Bandits
Specifically, it pulls the arm with the highest mean reward in a non-parametric bootstrap sample of its history with pseudo rewards.
Efficient Thompson Sampling for Online Matrix-Factorization Recommendation
Matrix factorization (MF) collaborative filtering is an effective and widely used method in recommendation systems.
Adaptive Submodular Maximization in Bandit Setting
Maximization of submodular functions has wide applications in machine learning and artificial intelligence.
Perturbed-History Exploration in Stochastic Multi-Armed Bandits
Finally, we empirically evaluate PHE and show that it is competitive with state-of-the-art baselines.
Perturbed-History Exploration in Stochastic Linear Bandits
We evaluate our algorithms empirically and show that they are practical.
Empirical Bayes Regret Minimization
In this work, we pioneer the idea of algorithm design by minimizing the empirical Bayes regret, the average regret over problem instances sampled from a known distribution.
Waterfall Bandits: Learning to Sell Ads Online
We design an online learning algorithm for solving this problem, which interleaves learning and optimization, and prove that this algorithm has sublinear regret.
Randomized Exploration in Generalized Linear Bandits
The first, GLM-TSL, samples a generalized linear model (GLM) from the Laplace approximation to the posterior distribution.
Differentiable Bandit Exploration
In this work, we learn such policies for an unknown distribution $\mathcal{P}$ using samples from $\mathcal{P}$.
Sample Efficient Graph-Based Optimization with Noisy Observations
We study sample complexity of optimizing "hill-climbing friendly" functions defined on a graph under noisy observations.
Meta-Learning Bandit Policies by Gradient Ascent
Most bandit policies are designed to either minimize regret in any problem instance, making very few assumptions about the underlying environment, or in a Bayesian sense, assuming a prior distribution over environment parameters.
Non-Stationary Off-Policy Optimization
This approach is practical and analyzable, and we provide guarantees on both the quality of off-policy optimization and the regret during online deployment.
Latent Bandits Revisited
A latent bandit problem is one in which the learning agent knows the arm reward distributions conditioned on an unknown discrete latent state.
Influence Diagram Bandits: Variational Thompson Sampling for Structured Bandit Problems
We propose a novel framework for structured bandits, which we call an influence diagram bandit.
Influence Diagram Bandits
We experiment with three structured bandit problems: cascading bandits, online learning to rank in the position-based model, and rank-1 bandits.
Differentiable Meta-Learning of Bandit Policies
Exploration policies in Bayesian bandits maximize the average reward over problem instances drawn from some distribution P. In this work, we learn such policies for an unknown distribution P using samples from P. Our approach is a form of meta-learning and exploits properties of P without making strong assumptions about its form.
Non-Stationary Latent Bandits
The key idea is to frame this problem as a latent bandit, where the prototypical models of user behavior are learned offline and the latent state of the user is inferred online from its interactions with the models.
CORe: Capitalizing On Rewards in Bandit Exploration
We propose a bandit algorithm that explores purely by randomizing its past observations.
Fixed-Budget Best-Arm Identification in Structured Bandits
We analyze our algorithm in linear and generalized linear models (GLMs), and propose a practical implementation based on a G-optimal design.
Thompson Sampling with a Mixture Prior
We study Thompson sampling (TS) in online decision making, where the uncertain environment is sampled from a mixture distribution.
Random Effect Bandits
It is well known that side information, such as the prior distribution of arm means in Thompson sampling, can improve the statistical efficiency of the bandit algorithm.
No Regrets for Learning the Prior in Bandits
We propose ${\tt AdaTS}$, a Thompson sampling algorithm that adapts sequentially to bandit tasks that it interacts with.
Optimal Probing with Statistical Guarantees for Network Monitoring at Scale
We propose a framework for estimating network metrics, such as latency and packet loss, with guarantees on estimation errors for a fixed monitoring budget.
Safe Data Collection for Offline and Online Policy Learning
Specifically, our goal is to develop a logging policy that efficiently explores different actions to elicit information while achieving competitive reward with a baseline production policy.
Hierarchical Bayesian Bandits
We provide a unified view of all these problems, as learning to act in a hierarchical Bayesian bandit.
IMO$^3$: Interactive Multi-Objective Off-Policy Optimization
This problem has been studied extensively in the setting of known objective functions.
Deep Hierarchy in Bandits
We use this exact posterior to analyze the Bayes regret of HierTS in Gaussian bandits.
Meta-Learning for Simple Regret Minimization
The Bayesian algorithm has access to a prior distribution over the meta-parameters and its meta simple regret over $m$ bandit tasks with horizon $n$ is mere $\tilde{O}(m / \sqrt{n})$.
Safe Exploration for Efficient Policy Evaluation and Comparison
High-quality data plays a central role in ensuring the accuracy of policy evaluation.
Pessimistic Off-Policy Optimization for Learning to Rank
Off-policy learning is a framework for optimizing policies without deploying them, using data collected by another policy.
Uplifting Bandits
We introduce a multi-armed bandit model where the reward is a sum of multiple random variables, and each action only alters the distributions of some of them.
From Ranked Lists to Carousels: A Carousel Click Model
Our analytical results show that the user can examine more items in the carousel click model than in a single ranked list, due to the structured way of browsing.
Robust Contextual Linear Bandits
Our experiments show that RoLinTS is comparably statistically efficient to the classic methods when the misspecification is low, more robust when the misspecification is high, and significantly more computationally efficient than its naive implementation.
Bayesian Fixed-Budget Best-Arm Identification
We also provide a lower bound on the probability of misidentification in a $2$-armed Bayesian bandit and show that our upper bound (almost) matches it for any budget.
Multi-Task Off-Policy Learning from Bandit Feedback
We prove per-task bounds on the suboptimality of the learned policies, which show a clear improvement over not using the hierarchical model.
Thompson Sampling with Diffusion Generative Prior
In this work, we initiate the idea of using denoising diffusion models to learn priors for online decision making problems.
Selective Uncertainty Propagation in Offline RL
We consider the finite-horizon offline reinforcement learning (RL) setting, and are motivated by the challenge of learning the policy at any step h in dynamic programming (DP) algorithms.
Multiplier Bootstrap-based Exploration
Despite the great interest in the bandit problem, designing efficient algorithms for complex models remains challenging, as there is typically no analytical way to quantify uncertainty.
Only Pay for What Is Uncertain: Variance-Adaptive Thompson Sampling
We lay foundations for the Bayesian setting, which incorporates prior knowledge.
Fixed-Budget Best-Arm Identification with Heterogeneous Reward Variances
Our algorithms rely on non-uniform budget allocations among the arms where the arms with higher reward variances are pulled more often than those with lower variances.
Finite-Time Logarithmic Bayes Regret Upper Bounds
In a multi-armed bandit, we obtain $O(c_\Delta \log n)$ and $O(c_h \log^2 n)$ upper bounds for an upper confidence bound algorithm, where $c_h$ and $c_\Delta$ are constants depending on the prior distribution and the gaps of bandit instances sampled from it, respectively.
Efficient and Interpretable Bandit Algorithms
We propose CODE, a bandit algorithm based on a Constrained Optimal DEsign, that is interpretable and maximally reduces the uncertainty.
Pessimistic Off-Policy Multi-Objective Optimization
The pessimistic estimator can be optimized by policy gradients and performs well in all of our experiments.
Logic-Scaffolding: Personalized Aspect-Instructed Recommendation Explanation Generation using LLMs
The unique capabilities of Large Language Models (LLMs), such as the natural language text generation ability, position them as strong candidates for providing explanation for recommendations.
MADA: Meta-Adaptive Optimizers through hyper-gradient Descent
Since Adam was introduced, several novel adaptive optimizers for deep learning have been proposed.
Experimental Design for Active Transductive Inference in Large Language Models
We design the LLM prompt by adaptively choosing few-shot examples for a given inference query.
