Search Results for author:
Found 14 papers, 2 papers with code
Taming "data-hungry" reinforcement learning? Stability in continuous state-action spaces
We introduce a novel framework for analyzing reinforcement learning (RL) in continuous state-action spaces, and use it to prove fast rates of convergence in both off-line and on-line settings.
Policy evaluation from a single path: Multi-step methods, mixing and mis-specification
For a given TD method applied to a well-specified model, its statistical error under trajectory data is similar to that of i. i. d.
Near-optimal Offline Reinforcement Learning with Linear Representation: Leveraging Variance Information with Pessimism
However, a precise understanding of the statistical limits with function representations, remains elusive, even when such a representation is linear.
Adaptive and Robust Multi-Task Learning
We study the multi-task learning problem that aims to simultaneously analyze multiple datasets collected from different sources and learn one model for each of them.
Optimal policy evaluation using kernel-based temporal difference methods
Whereas existing worst-case theory predicts cubic scaling ($H^3$) in the effective horizon, our theory reveals that there is in fact a much wider range of scalings, depending on the kernel, the stationary distribution, and the variance of the Bellman residual error.
PU-Flow: a Point Cloud Upsampling Network with Normalizing Flows
Point cloud upsampling aims to generate dense point clouds from given sparse ones, which is a challenging task due to the irregular and unordered nature of point sets.
Learning Good State and Action Representations via Tensor Decomposition
The transition kernel of a continuous-state-action Markov decision process (MDP) admits a natural tensor structure.
Risk Bounds and Rademacher Complexity in Batch Reinforcement Learning
Concretely, we view the Bellman error as a surrogate loss for the optimality gap, and prove the followings: (1) In double sampling regime, the excess risk of Empirical Risk Minimizer (ERM) is bounded by the Rademacher complexity of the function class.
Bootstrapping Fitted Q-Evaluation for Off-Policy Inference
Bootstrapping provides a flexible and effective approach for assessing the quality of batch reinforcement learning, yet its theoretical property is less understood.
Sparse Feature Selection Makes Batch Reinforcement Learning More Sample Efficient
To evaluate a new target policy, we analyze a Lasso fitted Q-evaluation method and establish a finite-sample error bound that has no polynomial dependence on the ambient dimension.
Minimax-Optimal Off-Policy Evaluation with Linear Function Approximation
We prove that this method is information-theoretically optimal and has nearly minimal estimation error.
Learning low-dimensional state embeddings and metastable clusters from time series data
This paper studies how to find compact state embeddings from high-dimensional Markov state trajectories, where the transition kernel has a small intrinsic rank.
State Aggregation Learning from Markov Transition Data
Our proposed method is a simple two-step algorithm: The first step is spectral decomposition of empirical transition matrix, and the second step conducts a linear transformation of singular vectors to find their approximate convex hull.
Adaptive Low-Nonnegative-Rank Approximation for State Aggregation of Markov Chains
The efficiency and statistical properties of our approach are illustrated on synthetic data.
