Search Results for author:
Found 17 papers, 4 papers with code
SCAFFLSA: Quantifying and Eliminating Heterogeneity Bias in Federated Linear Stochastic Approximation and Temporal Difference Learning
In this paper, we perform a non-asymptotic analysis of the federated linear stochastic approximation (FedLSA) algorithm.
Demonstration-Regularized RL
In particular, we study the demonstration-regularized reinforcement learning that leverages the expert demonstrations by KL-regularization for a policy learned by behavior cloning.
Finite-Sample Analysis of the Temporal Difference Learning
In this paper we consider the problem of obtaining sharp bounds for the performance of temporal difference (TD) methods with linear functional approximation for policy evaluation in discounted Markov Decision Processes.
Generative Flow Networks as Entropy-Regularized RL
We demonstrate how the task of learning a generative flow network can be efficiently redefined as an entropy-regularized RL problem with a specific reward and regularizer structure.
First Order Methods with Markovian Noise: from Acceleration to Variational Inequalities
We present a unified approach for the theoretical analysis of first-order gradient methods for stochastic optimization and variational inequalities.
Sharp Deviations Bounds for Dirichlet Weighted Sums with Application to analysis of Bayesian algorithms
These bounds are based on a novel integral representation of the density of a weighted Dirichlet sum.
Theoretical guarantees for neural control variates in MCMC
In this paper, we propose a variance reduction approach for Markov chains based on additive control variates and the minimization of an appropriate estimate for the asymptotic variance.
Fast Rates for Maximum Entropy Exploration
Finally, we apply developed regularization techniques to reduce sample complexity of visitation entropy maximization to $\widetilde{\mathcal{O}}(H^2SA/\varepsilon^2)$, yielding a statistical separation between maximum entropy exploration and reward-free exploration.
Rosenthal-type inequalities for linear statistics of Markov chains
In this paper, we establish novel deviation bounds for additive functionals of geometrically ergodic Markov chains similar to Rosenthal and Bernstein inequalities for sums of independent random variables.
Optimistic Posterior Sampling for Reinforcement Learning with Few Samples and Tight Guarantees
We consider reinforcement learning in an environment modeled by an episodic, finite, stage-dependent Markov decision process of horizon $H$ with $S$ states, and $A$ actions.
Finite-time High-probability Bounds for Polyak-Ruppert Averaged Iterates of Linear Stochastic Approximation
Our finite-time instance-dependent bounds for the averaged LSA iterates are sharp in the sense that the leading term we obtain coincides with the local asymptotic minimax limit.
From Dirichlet to Rubin: Optimistic Exploration in RL without Bonuses
We propose the Bayes-UCBVI algorithm for reinforcement learning in tabular, stage-dependent, episodic Markov decision process: a natural extension of the Bayes-UCB algorithm by Kaufmann et al. (2012) for multi-armed bandits.
Local-Global MCMC kernels: the best of both worlds
Recent works leveraging learning to enhance sampling have shown promising results, in particular by designing effective non-local moves and global proposals.
Tight High Probability Bounds for Linear Stochastic Approximation with Fixed Stepsize
This family of methods arises in many machine learning tasks and is used to obtain approximate solutions of a linear system $\bar{A}\theta = \bar{b}$ for which $\bar{A}$ and $\bar{b}$ can only be accessed through random estimates $\{({\bf A}_n, {\bf b}_n): n \in \mathbb{N}^*\}$.
Rates of convergence for density estimation with generative adversarial networks
In this work we undertake a thorough study of the non-asymptotic properties of the vanilla generative adversarial networks (GANs).
On the Stability of Random Matrix Product with Markovian Noise: Application to Linear Stochastic Approximation and TD Learning
This paper studies the exponential stability of random matrix products driven by a general (possibly unbounded) state space Markov chain.
Finite Time Analysis of Linear Two-timescale Stochastic Approximation with Markovian Noise
Our bounds show that there is no discrepancy in the convergence rate between Markovian and martingale noise, only the constants are affected by the mixing time of the Markov chain.
