Search Results for author:
Found 35 papers, 4 papers with code
Large-Scale Non-convex Stochastic Constrained Distributionally Robust Optimization
We prove that our algorithm finds an $\epsilon$-stationary point with a computational complexity of $\mathcal O(\epsilon^{-3k_*-5})$, where $k_*$ is the parameter of the Cressie-Read divergence.
Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance
Specifically, to solve the challenges due to dependence among adaptive update, unbounded gradient estimate and Lipschitz constant, we demonstrate that the first-order term in the descent lemma converges and its denominator is upper bounded by a function of gradient norm.
Sample Complexity Characterization for Linear Contextual MDPs
Our result for the second model is the first-known result for such a type of function approximation models.
Quickest Change Detection in Autoregressive Models
The data-driven setting where the disturbance signal parameters are unknown is further investigated, and an online and computationally efficient gradient ascent CuSum algorithm is designed.
Robust Multi-Agent Reinforcement Learning with State Uncertainty
Then we propose a robust multi-agent Q-learning (RMAQ) algorithm to find such an equilibrium, with convergence guarantees.
Achieving the Minimax Optimal Sample Complexity of Offline Reinforcement Learning: A DRO-Based Approach
We show that an improved sample complexity of $\mathcal{O}(SC^{\pi^*}\epsilon^{-2}(1-\gamma)^{-3})$ can be obtained, which matches with the minimax lower bound for offline reinforcement learning, and thus is minimax optimal.
Model-Free Robust Average-Reward Reinforcement Learning
Robust Markov decision processes (MDPs) address the challenge of model uncertainty by optimizing the worst-case performance over an uncertainty set of MDPs.
Robust Average-Reward Markov Decision Processes
We derive the robust Bellman equation for robust average-reward MDPs, prove that the optimal policy can be derived from its solution, and further design a robust relative value iteration algorithm that provably finds its solution, or equivalently, the optimal robust policy.
What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?
Various methods for Multi-Agent Reinforcement Learning (MARL) have been developed with the assumption that agents' policies are based on accurate state information.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Data-Driven Quickest Change Detection in (Hidden) Markov Models
The WADD is at most of the order of the logarithm of the ARL.
A Robust and Constrained Multi-Agent Reinforcement Learning Electric Vehicle Rebalancing Method in AMoD Systems
In this work, we design a robust and constrained multi-agent reinforcement learning (MARL) framework with state transition kernel uncertainty for EV AMoD systems.
Robust Constrained Reinforcement Learning
We then investigate a concrete example of $\delta$-contamination uncertainty set, design an online and model-free algorithm and theoretically characterize its sample complexity.
Finite-Time Error Bounds for Greedy-GQ
Our techniques in this paper provide a general approach for finite-sample analysis of non-convex two timescale value-based reinforcement learning algorithms.
Quickest Anomaly Detection in Sensor Networks With Unlabeled Samples
The goal of the fusion center is to detect the anomaly with minimal detection delay subject to false alarm constraints.
Provably Efficient Offline Reinforcement Learning with Trajectory-Wise Reward
The remarkable success of reinforcement learning (RL) heavily relies on observing the reward of every visited state-action pair.
Policy Gradient Method For Robust Reinforcement Learning
We further develop a smoothed robust policy gradient method and show that to achieve an $\epsilon$-global optimum, the complexity is $\mathcal O(\epsilon^{-3})$.
Kernel Robust Hypothesis Testing
For the Bayesian setting where the goal is to minimize the worst-case error probability, an optimal test is firstly obtained when the alphabet is finite.
Quickest Change Detection in Anonymous Heterogeneous Sensor Networks
The problem of quickest change detection (QCD) in anonymous heterogeneous sensor networks is studied.
Faster Algorithm and Sharper Analysis for Constrained Markov Decision Process
Despite the challenge of the nonconcave objective subject to nonconcave constraints, the proposed approach is shown to converge to the global optimum with a complexity of $\tilde{\mathcal O}(1/\epsilon)$ in terms of the optimality gap and the constraint violation, which improves the complexity of the existing primal-dual approach by a factor of $\mathcal O(1/\epsilon)$ \citep{ding2020natural, paternain2019constrained}.
Online Robust Reinforcement Learning with Model Uncertainty
In this paper, we focus on model-free robust RL, where the uncertainty set is defined to be centering at a misspecified MDP that generates a single sample trajectory sequentially and is assumed to be unknown.
Sample and Communication-Efficient Decentralized Actor-Critic Algorithms with Finite-Time Analysis
Actor-critic (AC) algorithms have been widely adopted in decentralized multi-agent systems to learn the optimal joint control policy.
Non-Asymptotic Analysis for Two Time-scale TDC with General Smooth Function Approximation
Temporal-difference learning with gradient correction (TDC) is a two time-scale algorithm for policy evaluation in reinforcement learning.
Greedy-GQ with Variance Reduction: Finite-time Analysis and Improved Complexity
Greedy-GQ is a value-based reinforcement learning (RL) algorithm for optimal control.
Learning Graph Neural Networks with Approximate Gradient Descent
Two types of GNNs are investigated, depending on whether labels are attached to nodes or graphs.
Variance-Reduced Off-Policy TDC Learning: Non-Asymptotic Convergence Analysis
In the Markovian setting, our algorithm achieves the state-of-the-art sample complexity $O(\epsilon^{-1} \log {\epsilon}^{-1})$ that is near-optimal.
Finite-sample Analysis of Greedy-GQ with Linear Function Approximation under Markovian Noise
Greedy-GQ is an off-policy two timescale algorithm for optimal control in reinforcement learning.
A CNN-Based Blind Denoising Method for Endoscopic Images
The number of iterations is reduced about 36% by using transfer learning in our DIP process.
Two Time-scale Off-Policy TD Learning: Non-asymptotic Analysis over Markovian Samples
Gradient-based temporal difference (GTD) algorithms are widely used in off-policy learning scenarios.
Finite-Sample Analysis for SARSA with Linear Function Approximation
For this fitted SARSA algorithm, we also provide its finite-sample analysis.
Information-Theoretic Understanding of Population Risk Improvement with Model Compression
We show that model compression can improve the population risk of a pre-trained model, by studying the tradeoff between the decrease in the generalization error and the increase in the empirical risk with model compression.
Tightening Mutual Information Based Bounds on Generalization Error
The bound is derived under more general conditions on the loss function than in existing studies; nevertheless, it provides a tighter characterization of the generalization error.
Linear-Complexity Exponentially-Consistent Tests for Universal Outlying Sequence Detection
A sequence is considered as outlying if the observations therein are generated by a distribution different from those generating the observations in the majority of the sequences.
Nonparametric Detection of Geometric Structures over Networks
Sufficient conditions on minimum and maximum sizes of candidate anomalous intervals are characterized in order to guarantee the proposed test to be consistent.
Nonparametric Detection of Anomalous Data Streams
samples drawn from a distribution p, whereas each anomalous sequence contains m i. i. d.
A Kernel-Based Nonparametric Test for Anomaly Detection over Line Networks
If anomalous interval does not exist, then all nodes receive samples generated by p. It is assumed that the distributions p and q are arbitrary, and are unknown.
