Search Results for author:
Found 41 papers, 4 papers with code
Independently-Normalized SGD for Generalized-Smooth Nonconvex Optimization
Recent studies have shown that many nonconvex machine learning problems meet a so-called generalized-smooth condition that extends beyond traditional smooth nonconvex optimization.
Model-Free Robust Reinforcement Learning with Sample Complexity Analysis
We develop algorithms for uncertainty sets defined by total variation, Chi-square divergence, and KL divergence, and provide finite sample analyses under all three cases.
Non-Asymptotic Analysis for Single-Loop (Natural) Actor-Critic with Compatible Function Approximation
Specifically, existing studies show that AC converges to an $\epsilon+\varepsilon_{\text{critic}}$ neighborhood of stationary points with the best known sample complexity of $\mathcal{O}(\epsilon^{-2})$ (up to a log factor), and NAC converges to an $\epsilon+\varepsilon_{\text{critic}}+\sqrt{\varepsilon_{\text{actor}}}$ neighborhood of the global optimum with the best known sample complexity of $\mathcal{O}(\epsilon^{-3})$, where $\varepsilon_{\text{critic}}$ is the approximation error of the critic and $\varepsilon_{\text{actor}}$ is the approximation error induced by the insufficient expressive power of the parameterized policy class.
MGDA Converges under Generalized Smoothness, Provably
We provide a comprehensive convergence analysis of these algorithms and show that they converge to an $\epsilon$-accurate Pareto stationary point with a guaranteed $\epsilon$-level average CA distance (i. e., the gap between the updating direction and the CA direction) over all iterations, where totally $\mathcal{O}(\epsilon^{-2})$ and $\mathcal{O}(\epsilon^{-4})$ samples are needed for deterministic and stochastic settings, respectively.
Finite-Time Analysis for Conflict-Avoidant Multi-Task Reinforcement Learning
However, these methods often suffer from the issue of \textit{gradient conflict} such that the tasks with larger gradients dominate the update direction, resulting in a performance degeneration on other tasks.
Constrained Reinforcement Learning Under Model Mismatch
Existing studies on constrained reinforcement learning (RL) may obtain a well-performing policy in the training environment.
Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance
Specifically, to solve the challenges due to dependence among adaptive update, unbounded gradient estimate and Lipschitz constant, we demonstrate that the first-order term in the descent lemma converges and its denominator is upper bounded by a function of gradient norm.
Large-Scale Non-convex Stochastic Constrained Distributionally Robust Optimization
We prove that our algorithm finds an $\epsilon$-stationary point with a computational complexity of $\mathcal O(\epsilon^{-3k_*-5})$, where $k_*$ is the parameter of the Cressie-Read divergence.
Sample Complexity Characterization for Linear Contextual MDPs
Our result for the second model is the first-known result for such a type of function approximation models.
Quickest Change Detection in Autoregressive Models
The data-driven setting where the disturbance signal parameters are unknown is further investigated, and an online and computationally efficient gradient ascent CuSum algorithm is designed.
Robust Multi-Agent Reinforcement Learning with State Uncertainty
Then we propose a robust multi-agent Q-learning (RMAQ) algorithm to find such an equilibrium, with convergence guarantees.
Achieving the Asymptotically Optimal Sample Complexity of Offline Reinforcement Learning: A DRO-Based Approach
We show that an improved sample complexity of $\mathcal{O}(SC^{\pi^*}\epsilon^{-2}(1-\gamma)^{-3})$ can be obtained, which asymptotically matches with the minimax lower bound for offline reinforcement learning, and thus is asymptotically minimax optimal.
Model-Free Robust Average-Reward Reinforcement Learning
Robust Markov decision processes (MDPs) address the challenge of model uncertainty by optimizing the worst-case performance over an uncertainty set of MDPs.
Robust Average-Reward Markov Decision Processes
We derive the robust Bellman equation for robust average-reward MDPs, prove that the optimal policy can be derived from its solution, and further design a robust relative value iteration algorithm that provably finds its solution, or equivalently, the optimal robust policy.
What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?
Various methods for Multi-Agent Reinforcement Learning (MARL) have been developed with the assumption that agents' policies are based on accurate state information.
Deep Reinforcement Learning
Multi-agent Reinforcement Learning
+2
Data-Driven Quickest Change Detection in (Hidden) Markov Models
The WADD is at most of the order of the logarithm of the ARL.
A Robust and Constrained Multi-Agent Reinforcement Learning Electric Vehicle Rebalancing Method in AMoD Systems
In this work, we design a robust and constrained multi-agent reinforcement learning (MARL) framework with state transition kernel uncertainty for EV AMoD systems.
Robust Constrained Reinforcement Learning
We then investigate a concrete example of $\delta$-contamination uncertainty set, design an online and model-free algorithm and theoretically characterize its sample complexity.
Finite-Time Error Bounds for Greedy-GQ
Our finite-time error bounds match with one of the stochastic gradient descent algorithms for general smooth non-convex optimization problems, despite its additonal challenge in the two time-scale updates.
Quickest Anomaly Detection in Sensor Networks With Unlabeled Samples
The goal of the fusion center is to detect the anomaly with minimal detection delay subject to false alarm constraints.
Provably Efficient Offline Reinforcement Learning with Trajectory-Wise Reward
The remarkable success of reinforcement learning (RL) heavily relies on observing the reward of every visited state-action pair.
Policy Gradient Method For Robust Reinforcement Learning
We further develop a smoothed robust policy gradient method and show that to achieve an $\epsilon$-global optimum, the complexity is $\mathcal O(\epsilon^{-3})$.
Kernel Robust Hypothesis Testing
For the Bayesian setting where the goal is to minimize the worst-case error probability, an optimal test is firstly obtained when the alphabet is finite.
Quickest Change Detection in Anonymous Heterogeneous Sensor Networks
The problem of quickest change detection (QCD) in anonymous heterogeneous sensor networks is studied.
Faster Algorithm and Sharper Analysis for Constrained Markov Decision Process
Despite the challenge of the nonconcave objective subject to nonconcave constraints, the proposed approach is shown to converge to the global optimum with a complexity of $\tilde{\mathcal O}(1/\epsilon)$ in terms of the optimality gap and the constraint violation, which improves the complexity of the existing primal-dual approach by a factor of $\mathcal O(1/\epsilon)$ \citep{ding2020natural, paternain2019constrained}.
Online Robust Reinforcement Learning with Model Uncertainty
In this paper, we focus on model-free robust RL, where the uncertainty set is defined to be centering at a misspecified MDP that generates a single sample trajectory sequentially and is assumed to be unknown.
Sample and Communication-Efficient Decentralized Actor-Critic Algorithms with Finite-Time Analysis
Actor-critic (AC) algorithms have been widely adopted in decentralized multi-agent systems to learn the optimal joint control policy.
Non-Asymptotic Analysis for Two Time-scale TDC with General Smooth Function Approximation
Temporal-difference learning with gradient correction (TDC) is a two time-scale algorithm for policy evaluation in reinforcement learning.
Greedy-GQ with Variance Reduction: Finite-time Analysis and Improved Complexity
Greedy-GQ is a value-based reinforcement learning (RL) algorithm for optimal control.
Learning Graph Neural Networks with Approximate Gradient Descent
Two types of GNNs are investigated, depending on whether labels are attached to nodes or graphs.
Variance-Reduced Off-Policy TDC Learning: Non-Asymptotic Convergence Analysis
In the Markovian setting, our algorithm achieves the state-of-the-art sample complexity $O(\epsilon^{-1} \log {\epsilon}^{-1})$ that is near-optimal.
Finite-sample Analysis of Greedy-GQ with Linear Function Approximation under Markovian Noise
Greedy-GQ is an off-policy two timescale algorithm for optimal control in reinforcement learning.
A CNN-Based Blind Denoising Method for Endoscopic Images
The number of iterations is reduced about 36% by using transfer learning in our DIP process.
Two Time-scale Off-Policy TD Learning: Non-asymptotic Analysis over Markovian Samples
Gradient-based temporal difference (GTD) algorithms are widely used in off-policy learning scenarios.
Finite-Sample Analysis for SARSA with Linear Function Approximation
For this fitted SARSA algorithm, we also provide its finite-sample analysis.
Information-Theoretic Understanding of Population Risk Improvement with Model Compression
We show that model compression can improve the population risk of a pre-trained model, by studying the tradeoff between the decrease in the generalization error and the increase in the empirical risk with model compression.
Tightening Mutual Information Based Bounds on Generalization Error
The bound is derived under more general conditions on the loss function than in existing studies; nevertheless, it provides a tighter characterization of the generalization error.
Linear-Complexity Exponentially-Consistent Tests for Universal Outlying Sequence Detection
A sequence is considered as outlying if the observations therein are generated by a distribution different from those generating the observations in the majority of the sequences.
Nonparametric Detection of Geometric Structures over Networks
Sufficient conditions on minimum and maximum sizes of candidate anomalous intervals are characterized in order to guarantee the proposed test to be consistent.
Nonparametric Detection of Anomalous Data Streams
samples drawn from a distribution p, whereas each anomalous sequence contains m i. i. d.
A Kernel-Based Nonparametric Test for Anomaly Detection over Line Networks
If anomalous interval does not exist, then all nodes receive samples generated by p. It is assumed that the distributions p and q are arbitrary, and are unknown.
