Search Results for author:
Found 27 papers, 2 papers with code
Prelimit Coupling and Steady-State Convergence of Constant-stepsize Nonsmooth Contractive SA
Motivated by Q-learning, we study nonsmooth contractive stochastic approximation (SA) with constant stepsize.
Unichain and Aperiodicity are Sufficient for Asymptotic Optimality of Average-Reward Restless Bandits
We consider the infinite-horizon, average-reward restless bandit problem in discrete time.
Constant Stepsize Q-learning: Distributional Convergence, Bias and Extrapolation
By connecting the constant stepsize Q-learning to a time-homogeneous Markov chain, we show the distributional convergence of the iterates in Wasserstein distance and establish its exponential convergence rate.
Effectiveness of Constant Stepsize in Markovian LSA and Statistical Inference
Our procedure leverages the fast mixing property of constant-stepsize LSA for better covariance estimation and employs Richardson-Romberg (RR) extrapolation to reduce the bias induced by constant stepsize and Markovian data.
Optimal Attack and Defense for Reinforcement Learning
Although the defense problem is NP-hard, we show that optimal Markovian defenses can be computed (learned) in polynomial time (sample complexity) in many scenarios.
Minimally Modifying a Markov Game to Achieve Any Nash Equilibrium and Value
We study the game modification problem, where a benevolent game designer or a malevolent adversary modifies the reward function of a zero-sum Markov game so that a target deterministic or stochastic policy profile becomes the unique Markov perfect Nash equilibrium and has a value within a target range, in a way that minimizes the modification cost.
Reinforcement Learning for SBM Graphon Games with Re-Sampling
In more realistic scenarios where the block model is unknown, we propose a re-sampling scheme from a graphon integrated with the finite N-player MP-MFG model.
VISER: A Tractable Solution Concept for Games with Information Asymmetry
Many real-world games suffer from information asymmetry: one player is only aware of their own payoffs while the other player has the full game information.
Stochastic Methods in Variational Inequalities: Ergodicity, Bias and Refinements
Our work endeavors to elucidate and quantify the probabilistic structures intrinsic to these algorithms.
Sharper Model-free Reinforcement Learning for Average-reward Markov Decision Processes
In the online setting, we propose model-free RL algorithms based on reference-advantage decomposition.
On Faking a Nash Equilibrium
We characterize offline data poisoning attacks on Multi-Agent Reinforcement Learning (MARL), where an attacker may change a data set in an attempt to install a (potentially fictitious) unique Markov-perfect Nash equilibrium.
Learning to Stabilize Online Reinforcement Learning in Unbounded State Spaces
This latter objective is called stability and is especially important when the state space is unbounded, such that the states can be arbitrarily far from each other and the agent can drift far away from the desired states.
Restless Bandits with Average Reward: Breaking the Uniform Global Attractor Assumption
In both settings, our work is the first asymptotic optimality result that does not require UGAP.
SPEED: Experimental Design for Policy Evaluation in Linear Heteroscedastic Bandits
In this paper, we study the problem of optimal data collection for policy evaluation in linear bandits.
Bias and Extrapolation in Markovian Linear Stochastic Approximation with Constant Stepsizes
We consider Linear Stochastic Approximation (LSA) with a constant stepsize and Markovian data.
Reward Poisoning Attacks on Offline Multi-Agent Reinforcement Learning
In offline multi-agent reinforcement learning (MARL), agents estimate policies from a given dataset.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
RL-QN: A Reinforcement Learning Framework for Optimal Control of Queueing Systems
In this work, we consider using model-based reinforcement learning (RL) to learn the optimal control policy for queueing networks so that the average job delay (or equivalently the average queue backlog) is minimized.
Model-based Reinforcement Learning
reinforcement-learning
+1
Provable Fictitious Play for General Mean-Field Games
We propose a reinforcement learning algorithm for stationary mean-field games, where the goal is to learn a pair of mean-field state and stationary policy that constitutes the Nash equilibrium.
Dynamic Regret of Policy Optimization in Non-stationary Environments
We consider reinforcement learning (RL) in episodic MDPs with adversarial full-information reward feedback and unknown fixed transition kernels.
Risk-Sensitive Reinforcement Learning: Near-Optimal Risk-Sample Tradeoff in Regret
We study risk-sensitive reinforcement learning in episodic Markov decision processes with unknown transition kernels, where the goal is to optimize the total reward under the risk measure of exponential utility.
POLY-HOOT: Monte-Carlo Planning in Continuous Space MDPs with Non-Asymptotic Analysis
Monte-Carlo planning, as exemplified by Monte-Carlo Tree Search (MCTS), has demonstrated remarkable performance in applications with finite spaces.
Stable Reinforcement Learning with Unbounded State Space
As a proof of concept, we propose an RL policy using Sparse-Sampling-based Monte Carlo Oracle and argue that it satisfies the stability property as long as the system dynamics under the optimal policy respects a Lyapunov function.
On Reinforcement Learning for Turn-based Zero-sum Markov Games
For a concrete instance of EIS where random policy is used for "exploration", Monte-Carlo Tree Search is used for "policy improvement" and Nearest Neighbors is used for "supervised learning", we establish that this method finds an $\varepsilon$-approximate value function of Nash equilibrium in $\widetilde{O}(\varepsilon^{-(d+4)})$ steps when the underlying state-space of the game is continuous and $d$-dimensional.
Learning Zero-Sum Simultaneous-Move Markov Games Using Function Approximation and Correlated Equilibrium
In the offline setting, we control both players and aim to find the Nash Equilibrium by minimizing the duality gap.
Understanding & Generalizing AlphaGo Zero
AlphaGo Zero (AGZ) introduced a new {\em tabula rasa} reinforcement learning algorithm that has achieved superhuman performance in the games of Go, Chess, and Shogi with no prior knowledge other than the rules of the game.
Non-Asymptotic Analysis of Monte Carlo Tree Search
In effect, we establish that to learn an $\varepsilon$ approximation of the value function with respect to $\ell_\infty$ norm, MCTS combined with nearest neighbor requires a sample size scaling as $\widetilde{O}\big(\varepsilon^{-(d+4)}\big)$, where $d$ is the dimension of the state space.
Q-learning with Nearest Neighbors
In particular, for MDPs with a $d$-dimensional state space and the discounted factor $\gamma \in (0, 1)$, given an arbitrary sample path with "covering time" $ L $, we establish that the algorithm is guaranteed to output an $\varepsilon$-accurate estimate of the optimal Q-function using $\tilde{O}\big(L/(\varepsilon^3(1-\gamma)^7)\big)$ samples.
