Search Results for author:
Found 11 papers, 1 papers with code
Scalable Online Exploration via Coverability
We propose exploration objectives -- policy optimization objectives that enable downstream maximization of any reward function -- as a conceptual framework to systematize the study of exploration.
Mitigating Covariate Shift in Misspecified Regression with Applications to Reinforcement Learning
A pervasive phenomenon in machine learning applications is distribution shift, where training and deployment conditions for a machine learning model differ.
Harnessing Density Ratios for Online Reinforcement Learning
The theories of offline and online reinforcement learning, despite having evolved in parallel, have begun to show signs of the possibility for a unification, with algorithms and analysis techniques for one setting often having natural counterparts in the other.
The Optimal Approximation Factors in Misspecified Off-Policy Value Function Estimation
Theoretical guarantees in reinforcement learning (RL) are known to suffer multiplicative blow-up factors with respect to the misspecification error of function approximation.
A Few Expert Queries Suffices for Sample-Efficient RL with Resets and Linear Value Approximation
Towards establishing the minimal amount of expert queries needed, we show that, in the same setting, any learner whose exploration budget is polynomially-bounded (in terms of $d, H,$ and $|\mathcal{A}|$) will require at least $\tilde\Omega(\sqrt{d})$ oracle calls to recover a policy competing with the expert's value function.
On Query-efficient Planning in MDPs under Linear Realizability of the Optimal State-value Function
We consider local planning in fixed-horizon MDPs with a generative model under the assumption that the optimal value function lies close to the span of a feature map.
A Variant of the Wang-Foster-Kakade Lower Bound for the Discounted Setting
Recently, Wang et al. (2020) showed a highly intriguing hardness result for batch reinforcement learning (RL) with linearly realizable value function and good feature coverage in the finite-horizon case.
Exponential Lower Bounds for Planning in MDPs With Linearly-Realizable Optimal Action-Value Functions
We consider the problem of local planning in fixed-horizon and discounted Markov Decision Processes (MDPs) with linear function approximation and a generative model under the assumption that the optimal action-value function lies in the span of a feature map that is available to the planner.
Constrained Markov Decision Processes via Backward Value Functions
In standard RL, the agent is incentivized to explore any behavior as long as it maximizes rewards, but in the real world, undesired behavior can damage either the system or the agent in a way that breaks the learning process itself.
A Distributional Analysis of Sampling-Based Reinforcement Learning Algorithms
We present a distributional approach to theoretical analyses of reinforcement learning algorithms for constant step-sizes.
Learning Graph Weighted Models on Pictures
Graph Weighted Models (GWMs) have recently been proposed as a natural generalization of weighted automata over strings and trees to arbitrary families of labeled graphs (and hypergraphs).
