Search Results for author:
Found 33 papers, 1 papers with code
On the Optimal Regret of Locally Private Linear Contextual Bandit
Contextual bandit with linear reward functions is among one of the most extensively studied models in bandit and online learning research.
Online Local False Discovery Rate Control: A Resource Allocation Approach
We start with general arrival distributions and show that a simple policy achieves a $O(\sqrt{T})$ regret.
Privacy Preserving Adaptive Experiment Design
In this paper, we investigate the tradeoff between loss of social welfare and statistical power in contextual bandit experiment.
Utility Fairness in Contextual Dynamic Pricing with Demand Learning
This paper introduces a novel contextual bandit algorithm for personalized pricing under utility fairness constraints in scenarios with uncertain demand, achieving an optimal regret upper bound.
Regret Distribution in Stochastic Bandits: Optimal Trade-off between Expectation and Tail Risk
A novel policy is proposed to characterize the optimal regret tail probability for any regret threshold.
A Simple and Optimal Policy Design with Safety against Heavy-tailed Risk for Stochastic Bandits
Starting from the two-armed bandit setting with time horizon $T$, we propose a simple policy and prove that the policy (i) enjoys the worst-case optimality for the expected regret at order $O(\sqrt{T\ln T})$ and (ii) has the worst-case tail probability of incurring a linear regret decay at an exponential rate $\exp(-\Omega(\sqrt{T}))$, a rate that we prove to be best achievable for all worst-case optimal policies.
Optimal Learning Rates for Regularized Least-Squares with a Fourier Capacity Condition
We derive minimax adaptive rates for a new, broad class of Tikhonov-regularized learning problems in Hilbert scales under general source conditions.
Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximation
This led Chen and Jiang (2019) to conjecture that concentrability (the most standard notion of coverage) and realizability (the weakest representation condition) alone are not sufficient for sample-efficient offline RL.
Dynamic Pricing and Demand Learning on a Large Network of Products: A PAC-Bayesian Approach
The seller does not know the parameters of the products' linear demand model, and can dynamically adjust product prices to learn the demand model based on sales observations.
Offline Planning and Online Learning under Recovering Rewards
Motivated by emerging applications such as live-streaming e-commerce, promotions and recommendations, we introduce and solve a general class of non-stationary multi-armed bandit problems that have the following two features: (i) the decision maker can pull and collect rewards from up to $K\,(\ge 1)$ out of $N$ different arms in each time period; (ii) the expected reward of an arm immediately drops after it is pulled, and then non-parametrically recovers as the arm's idle time increases.
Sobolev Norm Learning Rates for Conditional Mean Embeddings
We develop novel learning rates for conditional mean embeddings by applying the theory of interpolation for reproducing kernel Hilbert spaces (RKHS).
Model-Free Non-Stationary RL: Near-Optimal Regret and Applications in Multi-Agent RL and Inventory Control
We consider model-free reinforcement learning (RL) in non-stationary Markov decision processes.
Instance-Dependent Complexity of Contextual Bandits and Reinforcement Learning: A Disagreement-Based Perspective
In the classical multi-armed bandit problem, instance-dependent algorithms attain improved performance on "easy" problems with a gap between the best and second-best arm.
Near-Optimal Regret Bounds for Model-Free RL in Non-Stationary Episodic MDPs
We consider model-free reinforcement learning (RL) in non-stationary Markov decision processes (MDPs).
Privacy-Preserving Dynamic Personalized Pricing with Demand Learning
In this paper, we consider a dynamic pricing problem over $T$ time periods with an \emph{unknown} demand function of posted price and personalized information.
Provably More Efficient Q-Learning in the One-Sided-Feedback/Full-Feedback Settings
Motivated by the episodic version of the classical inventory control problem, we propose a new Q-learning-based algorithm, Elimination-Based Half-Q-Learning (HQL), that enjoys improved efficiency over existing algorithms for a wide variety of problems in the one-sided-feedback setting.
Reinforcement Learning for Non-Stationary Markov Decision Processes: The Blessing of (More) Optimism
We consider un-discounted reinforcement learning (RL) in Markov decision processes (MDPs) under drifting non-stationarity, i. e., both the reward and state transition distributions are allowed to evolve over time, as long as their respective total variations, quantified by suitable metrics, do not exceed certain variation budgets.
Online Learning and Optimization for Revenue Management Problems with Add-on Discounts
We show that our learning algorithm can converge to the optimal algorithm that has access to the true demand functions, and we prove that the convergence rate is tight up to a certain logarithmic term.
Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits under Realizability
We consider the general (stochastic) contextual bandit problem under the realizability assumption, i. e., the expected reward, as a function of contexts and actions, belongs to a general function class $\mathcal{F}$.
Blind Network Revenue Management and Bandits with Knapsacks under Limited Switches
Our work reveals a surprising result: the optimal regret rate is completely characterized by a piecewise-constant function of the switching budget, which further depends on the number of resource constraints -- to the best of our knowledge, this is the first time the number of resources constraints is shown to play a fundamental role in determining the statistical complexity of online learning problems.
Online Pricing with Offline Data: Phase Transition and Inverse Square Law
We study a single-product dynamic pricing problem over a selling horizon of $T$ periods.
Non-Stationary Reinforcement Learning: The Blessing of (More) Optimism
Notably, the interplay between endogeneity and exogeneity presents a unique challenge, absent in existing (stationary and non-stationary) stochastic online learning settings, when we apply the conventional Optimism in Face of Uncertainty principle to design algorithms with provably low dynamic regret for RL in drifting MDPs.
Phase Transitions in Bandits with Switching Constraints
We consider the classical stochastic multi-armed bandit problem with a constraint that limits the total cost incurred by switching between actions to be no larger than a given switching budget.
Shrinking the Upper Confidence Bound: A Dynamic Product Selection Problem for Urban Warehouses
In this paper, we study algorithms for dynamically identifying a large number of products (i. e., SKUs) with top customer purchase probabilities on the fly, from an ocean of potential products to offer on retailers' ultra-fast delivery platforms.
Hedging the Drift: Learning to Optimize under Non-Stationarity
Boosted by the novel bandit-over-bandit framework that adapts to the latent changes, we can further enjoy the (nearly) optimal dynamic regret bounds in a (surprisingly) parameter-free manner.
Meta Dynamic Pricing: Transfer Learning Across Experiments
We study the problem of learning shared structure \emph{across} a sequence of dynamic pricing experiments for related products.
The Lingering of Gradients: Theory and Applications
Classically, the time complexity of a first-order method is estimated by its number of gradient computations.
The Lingering of Gradients: How to Reuse Gradients Over Time
Classically, the time complexity of a first-order method is estimated by its number of gradient computations.
Inventory Balancing with Online Learning
We overcome both the challenges of model uncertainty and customer heterogeneity by judiciously synthesizing two algorithmic frameworks from the literature: inventory balancing, which "reserves" a portion of each resource for high-reward customer types which could later arrive, and online learning, which shows how to "explore" the resource consumption distributions of each customer type under different actions.
Learning to Optimize under Non-Stationarity
We introduce algorithms that achieve state-of-the-art \emph{dynamic regret} bounds for non-stationary linear stochastic bandit setting.
A Practically Competitive and Provably Consistent Algorithm for Uplift Modeling
The problem of constructing such models from randomized experiments data is known as Uplift Modeling in the literature.
Uplift Modeling with Multiple Treatments and General Response Types
It is impossible to know whether the chosen treatment is optimal for an individual subject because response under alternative treatments is unobserved.
Assortment Optimization under Unknown MultiNomial Logit Choice Models
We first propose an efficient online policy which incurs a regret $\tilde{O}(T^{2/3})$, where $T$ is the number of customers in the sales horizon.
