Search Results for author:
Found 14 papers, 0 papers with code
Convex Q Learning in a Stochastic Environment: Extended Version
The main contributions firstly concern properties of the relaxation, described as a deterministic convex program: we identify conditions for a bounded solution, and a significant relationship between the solution to the new convex program, and the solution to standard Q-learning.
The Curse of Memory in Stochastic Approximation: Extended Version
The remaining results are established for linear SA recursions: (ii) the bivariate parameter-disturbance process is geometrically ergodic in a topological sense; (iii) the representation for bias has a simpler form in this case, and cannot be expected to be zero if there is multiplicative noise; (iv) the asymptotic covariance of the averaged parameters is within $O(\alpha)$ of optimal.
Stability of Q-Learning Through Design and Optimism
The algorithm is a general approach to stochastic approximation which in particular applies to Q-learning with "oblivious" training even with non-linear function approximation.
High-Impedance Non-Linear Fault Detection via Eigenvalue Analysis with low PMU Sampling Rates
This technique holds several advantages over contemporary techniques: It utilizes technology that is already deployed in the field, it offers a significant degree of generality, and so far it has displayed a very high-level of sensitivity without sacrificing accuracy.
High Impedance Fault Detection Through Quasi-Static State Estimation: A Parameter Error Modeling Approach
This paper presents a model for detecting high-impedance faults (HIFs) using parameter error modeling and a two-step per-phase weighted least squares state estimation (SE) process.
Uncertainty Error Modeling for Non-Linear State Estimation With Unsynchronized SCADA and $μ$PMU Measurements
Distribution systems of the future smart grid require enhancements to the reliability of distribution system state estimation (DSSE) in the face of low measurement redundancy, unsynchronized measurements, and dynamic load profiles.
Sufficient Exploration for Convex Q-learning
The main contributions follow: (i) The dual of convex Q-learning is not precisely Manne's LP or a version of logistic Q-learning, but has similar structure that reveals the need for regularization to avoid over-fitting.
Model-Free Characterizations of the Hamilton-Jacobi-Bellman Equation and Convex Q-Learning in Continuous Time
Convex Q-learning is a recent approach to reinforcement learning, motivated by the possibility of a firmer theory for convergence, and the possibility of making use of greater a priori knowledge regarding policy or value function structure.
The ODE Method for Asymptotic Statistics in Stochastic Approximation and Reinforcement Learning
In addition to standard Lipschitz assumptions and conditions on the vanishing step-size sequence, it is assumed that the associated \textit{mean flow} $ \tfrac{d}{dt} \vartheta_t = \bar{f}(\vartheta_t)$, is globally asymptotically stable with stationary point denoted $\theta^*$, where $\bar{f}(\theta)=\text{ E}[f(\theta,\Phi)]$ with $\Phi$ having the stationary distribution of the chain.
Accelerating Optimization and Reinforcement Learning with Quasi-Stochastic Approximation
(ii) With gain $a_t = g/(1+t)$ the results are not as sharp: the rate of convergence $1/t$ holds only if $I + g A^*$ is Hurwitz.
Explicit Mean-Square Error Bounds for Monte-Carlo and Linear Stochastic Approximation
This is motivation for the focus on mean square error bounds for parameter estimates.
Optimal Matrix Momentum Stochastic Approximation and Applications to Q-learning
There are two well known SA techniques that are known to have optimal asymptotic variance: the Ruppert-Polyak averaging technique, and stochastic Newton-Raphson (SNR).
Zap Q-Learning
The Zap Q-learning algorithm introduced in this paper is an improvement of Watkins' original algorithm and recent competitors in several respects.
Approximate dynamic programming using fluid and diffusion approximations with applications to power management
Neuro-dynamic programming is a class of powerful techniques for approximating the solution to dynamic programming equations.
