Search Results for author:
Found 51 papers, 8 papers with code
Distributionally Robust Policy Evaluation and Learning in Offline Contextual Bandits
We first present a policy evaluation procedure in the ambiguous environment and also give a heuristic algorithm to solve the distributionally robust policy learning problems efficiently.
Gradient-free Online Learning in Continuous Games with Delayed Rewards
Motivated by applications to online advertising and recommender systems, we consider a game-theoretic model with delayed rewards and asynchronous, payoff-based feedback.
On the Last-Iterate Convergence of Shuffling Gradient Methods
Shuffling gradient methods, which are also known as stochastic gradient descent (SGD) without replacement, are widely implemented in practice, particularly including three popular algorithms: Random Reshuffle (RR), Shuffle Once (SO), and Incremental Gradient (IG).
Stochastic contextual bandits with graph feedback: from independence number to MAS number
Interestingly, $\beta_M(G)$ interpolates between $\alpha(G)$ (the independence number of the graph) and $\mathsf{m}(G)$ (the maximum acyclic subgraph (MAS) number of the graph) as the number of contexts $M$ varies.
Revisiting the Last-Iterate Convergence of Stochastic Gradient Methods
For Lipschitz convex functions, different works have established the optimal $O(\log(1/\delta)\log T/\sqrt{T})$ or $O(\sqrt{\log(1/\delta)/T})$ high-probability convergence rates for the final iterate, where $T$ is the time horizon and $\delta$ is the failure probability.
On the Foundation of Distributionally Robust Reinforcement Learning
This is accomplished through a comprehensive modeling framework centered around distributionally robust Markov decision processes (DRMDPs).
Adaptive, Doubly Optimal No-Regret Learning in Strongly Monotone and Exp-Concave Games with Gradient Feedback
Online gradient descent (OGD) is well known to be doubly optimal under strong convexity or monotonicity assumptions: (1) in the single-agent setting, it achieves an optimal regret of $\Theta(\log T)$ for strongly convex cost functions; and (2) in the multi-agent setting of strongly monotone games, with each agent employing OGD, we obtain last-iterate convergence of the joint action to a unique Nash equilibrium at an optimal rate of $\Theta(\frac{1}{T})$.
Sample Complexity of Variance-reduced Distributionally Robust Q-learning
Further, the variance-reduced distributionally robust Q-learning combines the synchronous Q-learning with variance-reduction techniques to enhance its performance.
Stochastic Nonsmooth Convex Optimization with Heavy-Tailed Noises: High-Probability Bound, In-Expectation Rate and Initial Distance Adaptation
Recently, several studies consider the stochastic optimization problem but in a heavy-tailed noise regime, i. e., the difference between the stochastic gradient and the true gradient is assumed to have a finite $p$-th moment (say being upper bounded by $\sigma^{p}$ for some $\sigma\geq0$) where $p\in(1, 2]$, which not only generalizes the traditional finite variance assumption ($p=2$) but also has been observed in practice for several different tasks.
A Finite Sample Complexity Bound for Distributionally Robust Q-learning
We consider a reinforcement learning setting in which the deployment environment is different from the training environment.
Breaking the Lower Bound with (Little) Structure: Acceleration in Non-Convex Stochastic Optimization with Heavy-Tailed Noise
For this class of problems, we propose the first variance-reduced accelerated algorithm and establish that it guarantees a high-probability convergence rate of $O(\log(T/\delta)T^{\frac{1-p}{2p-1}})$ under a mild condition, which is faster than $\Omega(T^{\frac{1-p}{3p-2}})$.
Near-Optimal Non-Convex Stochastic Optimization under Generalized Smoothness
Two recent works established the $O(\epsilon^{-3})$ sample complexity to obtain an $O(\epsilon)$-stationary point.
Single-Trajectory Distributionally Robust Reinforcement Learning
As a framework for sequential decision-making, Reinforcement Learning (RL) has been regarded as an essential component leading to Artificial General Intelligence (AGI).
Leveraging the Hints: Adaptive Bidding in Repeated First-Price Auctions
In the past four years, a particularly important development in the digital advertising industry is the shift from second-price auctions to first-price auctions for online display ads.
Distributionally Robust Offline Reinforcement Learning with Linear Function Approximation
Among the reasons hindering reinforcement learning (RL) applications to real-world problems, two factors are critical: limited data and the mismatch between the testing environment (real environment in which the policy is deployed) and the training environment (e. g., a simulator).
Optimal Diagonal Preconditioning
Moreover, our implementation of customized solvers, combined with a random row/column sampling step, can find near-optimal diagonal preconditioners for matrices up to size 200, 000 in reasonable time, demonstrating their practical appeal.
Learning to Order for Inventory Systems with Lost Sales and Uncertain Supplies
We aim to minimize the $T$-period cost, a problem that is known to be computationally intractable even under known distributions of demand and supply.
Doubly Robust Distributionally Robust Off-Policy Evaluation and Learning
Thanks to a localization technique, LDR$^2$OPE only requires fitting a small number of regressions, just like DR methods for standard OPE.
Doubly Optimal No-Regret Online Learning in Strongly Monotone Games with Bandit Feedback
Leveraging self-concordant barrier functions, we first construct a new bandit learning algorithm and show that it achieves the single-agent optimal regret of $\tilde{\Theta}(n\sqrt{T})$ under smooth and strongly concave reward functions ($n \geq 1$ is the problem dimension).
Computational Benefits of Intermediate Rewards for Goal-Reaching Policy Learning
In both OWSP and OWMP settings, we demonstrate that adding {\em intermediate rewards} to subgoals is more computationally efficient than only rewarding the agent once it completes the goal of reaching a terminal state.
Distributed stochastic optimization with large delays
One of the most widely used methods for solving large-scale stochastic optimization problems is distributed asynchronous stochastic gradient descent (DASGD), a family of algorithms that result from parallelizing stochastic gradient descent on distributed computing architectures (possibly) asychronously.
Policy Learning with Adaptively Collected Data
Learning optimal policies from historical data enables personalization in a wide variety of applications including healthcare, digital recommendations, and online education.
No Weighted-Regret Learning in Adversarial Bandits with Delays
Using these bounds, we show that FKM and EXP3 have no weighted-regret even for $d_{t}=O\left(t\log t\right)$.
Online Multi-Armed Bandits with Adaptive Inference
During online decision making in Multi-Armed Bandits (MAB), one needs to conduct inference on the true mean reward of each arm based on data collected so far at each step.
Simple Agent, Complex Environment: Efficient Reinforcement Learning with Agent States
The time it takes to approach asymptotic performance is polynomial in the complexity of the agent's state representation and the time required to evaluate the best policy that the agent can represent.
Optimistic Dual Extrapolation for Coherent Non-monotone Variational Inequalities
The optimization problems associated with training generative adversarial neural networks can be largely reduced to certain {\em non-monotone} variational inequality problems (VIPs), whereas existing convergence results are mostly based on monotone or strongly monotone assumptions.
Dynamic Batch Learning in High-Dimensional Sparse Linear Contextual Bandits
We study the problem of dynamic batch learning in high-dimensional sparse linear contextual bandits, where a decision maker, under a given maximum-number-of-batch constraint and only able to observe rewards at the end of each batch, can dynamically decide how many individuals to include in the next batch (at the end of the current batch) and what personalized action-selection scheme to adopt within each batch.
A Unified Linear Speedup Analysis of Federated Averaging and Nesterov FedAvg
For strongly convex and convex problems, we also characterize the corresponding convergence rates for the Nesterov accelerated FedAvg algorithm, which are the first linear speedup guarantees for momentum variants of FedAvg in convex settings.
Learning to Bid Optimally and Efficiently in Adversarial First-price Auctions
In this paper, we take an online learning angle and address the fundamental problem of learning to bid in repeated first-price auctions, where both the bidder's private valuations and other bidders' bids can be arbitrary.
Distributionally Robust Batch Contextual Bandits
Leveraging this evaluation scheme, we further propose a novel learning algorithm that is able to learn a policy that is robust to adversarial perturbations and unknown covariate shifts with a performance guarantee based on the theory of uniform convergence.
Understanding l4-based Dictionary Learning: Interpretation, Stability, and Robustness
Recently, the $\ell^4$-norm maximization has been proposed to solve the sparse dictionary learning (SDL) problem.
DSAC: Distributional Soft Actor Critic for Risk-Sensitive Reinforcement Learning
In this paper, we present a new reinforcement learning (RL) algorithm called Distributional Soft Actor Critic (DSAC), which exploits the distributional information of accumulated rewards to achieve better performance.
Sequential Batch Learning in Finite-Action Linear Contextual Bandits
We study the sequential batch learning problem in linear contextual bandits with finite action sets, where the decision maker is constrained to split incoming individuals into (at most) a fixed number of batches and can only observe outcomes for the individuals within a batch at the batch's end.
Optimal No-regret Learning in Repeated First-price Auctions
In this paper, we develop the first learning algorithm that achieves a near-optimal $\widetilde{O}(\sqrt{T})$ regret bound, by exploiting two structural properties of first-price auctions, i. e. the specific feedback structure and payoff function.
Interpretable Personalization via Policy Learning with Linear Decision Boundaries
Our findings suggest that our proposed policy learning framework using tools from causal inference and Bayesian optimization provides a promising practical approach to interpretable personalization across a wide range of applications.
Delay-Adaptive Learning in Generalized Linear Contextual Bandits
In this paper, we consider online learning in generalized linear contextual bandits where rewards are not immediately observed.
Finite-Time Last-Iterate Convergence for Multi-Agent Learning in Games
In this paper, we consider multi-agent learning via online gradient descent in a class of games called $\lambda$-cocoercive games, a fairly broad class of games that admits many Nash equilibria and that properly includes unconstrained strongly monotone games.
Provably Efficient Reinforcement Learning with Aggregated States
We establish that an optimistic variant of Q-learning applied to a fixed-horizon episodic Markov decision process with an aggregated state representation incurs regret $\tilde{\mathcal{O}}(\sqrt{H^5 M K} + \epsilon HK)$, where $H$ is the horizon, $M$ is the number of aggregate states, $K$ is the number of episodes, and $\epsilon$ is the largest difference between any pair of optimal state-action values associated with a common aggregate state.
Online EXP3 Learning in Adversarial Bandits with Delayed Feedback
An adversary chooses the cost of each arm in a bounded interval, and a sequence of feedback delays \left\{ d_{t}\right\} that are unknown to the player.
Learning in Generalized Linear Contextual Bandits with Stochastic Delays
In this paper, we consider online learning in generalized linear contextual bandits where rewards are not immediately observed.
Balanced Linear Contextual Bandits
Contextual bandit algorithms are sensitive to the estimation method of the outcome model as well as the exploration method used, particularly in the presence of rich heterogeneity or complex outcome models, which can lead to difficult estimation problems along the path of learning.
Learning in Games with Lossy Feedback
We consider a game-theoretical multi-agent learning problem where the feedback information can be lost during the learning process and rewards are given by a broad class of games known as variationally stable games.
Offline Multi-Action Policy Learning: Generalization and Optimization
In many settings, a decision-maker wishes to learn a rule, or policy, that maps from observable characteristics of an individual to an action.
Distributed Asynchronous Optimization with Unbounded Delays: How Slow Can You Go?
One of the most widely used optimization methods for large-scale machine learning problems is distributed asynchronous stochastic gradient descent (DASGD).
MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels
Recent deep networks are capable of memorizing the entire data even when the labels are completely random.
Ranked #16 on
Image Classification
on WebVision-1000
Countering Feedback Delays in Multi-Agent Learning
We consider a model of game-theoretic learning based on online mirror descent (OMD) with asynchronous and delayed feedback information.
Stochastic Mirror Descent in Variationally Coherent Optimization Problems
In this paper, we examine a class of non-convex stochastic optimization problems which we call variationally coherent, and which properly includes pseudo-/quasiconvex and star-convex optimization problems.
Estimation Considerations in Contextual Bandits
We develop parametric and non-parametric contextual bandits that integrate balancing methods from the causal inference literature in their estimation to make it less prone to problems of estimation bias.
On the convergence of mirror descent beyond stochastic convex programming
In this paper, we examine the convergence of mirror descent in a class of stochastic optimization problems that are not necessarily convex (or even quasi-convex), and which we call variationally coherent.
Learning in games with continuous action sets and unknown payoff functions
This paper examines the convergence of no-regret learning in games with continuous action sets.
Simultaneous Rectification and Alignment via Robust Recovery of Low-rank Tensors
In this context, the state-of-the-art algorithms RASL'' and "TILT'' can be viewed as two special cases of our work, and yet each only performs part of the function of our method."
