Search Results for author:
Found 20 papers, 1 papers with code
Markov Chain Variance Estimation: A Stochastic Approximation Approach
We consider the problem of estimating the asymptotic variance of a function defined on a Markov chain, an important step for statistical inference of the stationary mean.
Optimization of utility-based shortfall risk: A non-asymptotic viewpoint
In the context of UBSR estimation, we derive a non-asymptotic bound on the mean-squared error of the classical sample average approximation (SAA) of UBSR.
A Cubic-regularized Policy Newton Algorithm for Reinforcement Learning
We consider the problem of control in the setting of reinforcement learning (RL), where model information is not available.
Finite time analysis of temporal difference learning with linear function approximation: Tail averaging and regularisation
We study the finite-time behaviour of the popular temporal difference (TD) learning algorithm when combined with tail-averaging.
A Gradient Smoothed Functional Algorithm with Truncated Cauchy Random Perturbations for Stochastic Optimization
In this paper, we present a stochastic gradient algorithm for minimizing a smooth objective function that is an expectation over noisy cost samples, and only the latter are observed for any given parameter.
A Survey of Risk-Aware Multi-Armed Bandits
In several applications such as clinical trials and financial portfolio optimization, the expected value (or the average reward) does not satisfactorily capture the merits of a drug or a portfolio.
Estimation of Spectral Risk Measures
We consider the problem of estimating a spectral risk measure (SRM) from i. i. d.
A Wasserstein distance approach for concentration of empirical risk estimates
Previous concentration bounds are available only for specific risk measures such as CVaR and CPT-value.
Concentration bounds for CVaR estimation: The cases of light-tailed and heavy-tailed distributions
We derive concentration bounds for CVaR estimates, considering separately the cases of light-tailed and heavy-tailed distributions.
Risk-Sensitive Reinforcement Learning via Policy Gradient Search
In this book, we consider risk-sensitive RL in two settings: one where the goal is to find a policy that optimizes the usual expected value objective while ensuring that a risk constraint is satisfied, and the other where the risk measure is the objective.
Concentration bounds for empirical conditional value-at-risk: The unbounded case
In several real-world applications involving decision making under uncertainty, the traditional expected value objective may not be suitable, as it may be necessary to control losses in the case of a rare but extreme event.
Bandit algorithms to emulate human decision making using probabilistic distortions
For the $K$-armed bandit setting, we derive an upper bound on the expected regret for our proposed algorithm, and then we prove a matching lower bound to establish the order-optimality of our algorithm.
(Bandit) Convex Optimization with Biased Noisy Gradient Oracles
Algorithms for bandit convex optimization and online learning often rely on constructing noisy gradient estimates, which are then used in appropriately adjusted first-order algorithms, replacing actual gradients.
A constrained optimization perspective on actor critic algorithms and application to network routing
We propose a novel actor-critic algorithm with guaranteed convergence to an optimal policy for a discounted reward Markov decision process.
Cumulative Prospect Theory Meets Reinforcement Learning: Prediction and Control
Cumulative prospect theory (CPT) is known to model human decisions well, with substantial empirical evidence supporting this claim.
Adaptive system optimization using random directions stochastic approximation
We prove the unbiasedness of both gradient and Hessian estimates and asymptotic (strong) convergence for both first-order and second-order schemes.
Variance-Constrained Actor-Critic Algorithms for Discounted and Average Reward MDPs
For each formulation, we first define a measure of variability for a policy, which in turn gives us a set of risk-sensitive criteria to optimize.
Two Timescale Convergent Q-learning for Sleep--Scheduling in Wireless Sensor Networks
For each criterion, we propose a convergent on-policy Q-learning algorithm that operates on two timescales, while employing function approximation to handle the curse of dimensionality associated with the underlying POMDP.
Actor-Critic Algorithms for Risk-Sensitive MDPs
For each formulation, we first define a measure of variability for a policy, which in turn gives us a set of risk-sensitive criteria to optimize.
Fast gradient descent for drifting least squares regression, with application to bandits
In the case when strong convexity in the regression problem is guaranteed, we provide bounds on the error both in expectation and high probability (the latter is often needed to provide theoretical guarantees for higher level algorithms), despite the drifting least squares solution.
