Search Results for author:
Found 62 papers, 25 papers with code
Distributional Offline Policy Evaluation with Predictive Error Guarantees
In our theoretical results, we show that for both finite and infinite horizon discounted settings, FLE can learn distributions that are close to the ground truth under total variation distance and Wasserstein distance, respectively.
Multi-task Representation Learning for Pure Exploration in Linear Bandits
In these two problems, all tasks share a common low-dimensional linear representation, and our goal is to leverage this feature to accelerate the best arm (policy) identification process for all tasks.
Near-Minimax-Optimal Risk-Sensitive Reinforcement Learning with CVaR
In this paper, we study risk-sensitive Reinforcement Learning (RL), focusing on the objective of Conditional Value at Risk (CVaR) with risk tolerance $\tau$.
Refined Value-Based Offline RL under Realizability and Partial Coverage
In offline reinforcement learning (RL) we have no opportunity to explore so we must make assumptions that the data is sufficient to guide picking a good policy, taking the form of assuming some coverage, realizability, Bellman completeness, and/or hard margin (gap).
Hybrid RL: Using Both Offline and Online Data Can Make RL Efficient
We consider a hybrid reinforcement learning setting (Hybrid RL), in which an agent has access to an offline dataset and the ability to collect experience via real-world online interaction.
Sample-efficient Safe Learning for Online Nonlinear Control with Control Barrier Functions
In particular, the framework 1) extends control barrier functions (CBFs) in a stochastic setting to achieve provable high-probability safety under uncertainty during model learning and 2) integrates an optimism-based exploration strategy to efficiently guide the safe exploration process with learned dynamics for \emph{near optimal} control performance.
Future-Dependent Value-Based Off-Policy Evaluation in POMDPs
Finally, we extend our methods to learning of dynamics and establish the connection between our approach and the well-known spectral learning methods in POMDPs.
Learning Bellman Complete Representations for Offline Policy Evaluation
We study representation learning for Offline Reinforcement Learning (RL), focusing on the important task of Offline Policy Evaluation (OPE).
PAC Reinforcement Learning for Predictive State Representations
We show that given a realizable model class, the sample complexity of learning the near optimal policy only scales polynomially with respect to the statistical complexity of the model class, without any explicit polynomial dependence on the size of the state and observation spaces.
Provably Efficient Reinforcement Learning in Partially Observable Dynamical Systems
We study Reinforcement Learning for partially observable dynamical systems using function approximation.
Computationally Efficient PAC RL in POMDPs with Latent Determinism and Conditional Embeddings
We show our algorithm's computational and statistical complexities scale polynomially with respect to the horizon and the intrinsic dimension of the feature on the observation space.
Minimum Noticeable Difference based Adversarial Privacy Preserving Image Generation
To achieve this, an adversarial loss is firstly proposed to make the deep learning models attacked by the adversarial images successfully.
Provable Benefits of Representational Transfer in Reinforcement Learning
We study the problem of representational transfer in RL, where an agent first pretrains in a number of source tasks to discover a shared representation, which is subsequently used to learn a good policy in a \emph{target task}.
Online No-regret Model-Based Meta RL for Personalized Navigation
Our theoretical analysis shows that our method is a no-regret algorithm and we provide the convergence rate in the agnostic setting.
Learning to Detect Mobile Objects from LiDAR Scans Without Labels
Current 3D object detectors for autonomous driving are almost entirely trained on human-annotated data.
Hindsight is 20/20: Leveraging Past Traversals to Aid 3D Perception
Self-driving cars must detect vehicles, pedestrians, and other traffic participants accurately to operate safely.
Efficient Reinforcement Learning in Block MDPs: A Model-free Representation Learning Approach
We present BRIEE (Block-structured Representation learning with Interleaved Explore Exploit), an algorithm for efficient reinforcement learning in Markov Decision Processes with block-structured dynamics (i. e., Block MDPs), where rich observations are generated from a set of unknown latent states.
On the Effectiveness of Iterative Learning Control
However, there is little prior theoretical work that explains the effectiveness of ILC even in the presence of large modeling errors, where optimal control methods using the misspecified model (MM) often perform poorly.
Representation Learning for Online and Offline RL in Low-rank MDPs
This work studies the question of Representation Learning in RL: how can we learn a compact low-dimensional representation such that on top of the representation we can perform RL procedures such as exploration and exploitation, in a sample efficient manner.
Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design
Specifically, we learn a conditional policy that, in an episode, first applies a sequence of transform actions to modify an agent's skeletal structure and joint attributes, and then applies control actions under the new design.
PC-MLP: Model-based Reinforcement Learning with Policy Cover Guided Exploration
Model-based Reinforcement Learning (RL) is a popular learning paradigm due to its potential sample efficiency compared to model-free RL.
Model-based Reinforcement Learning
reinforcement-learning
+1
Pessimistic Model-based Offline Reinforcement Learning under Partial Coverage
Under the assumption that the ground truth model belongs to our function class (i. e., realizability in the function class), CPPO has a PAC guarantee with offline data only providing partial coverage, i. e., it can learn a policy that competes against any policy that is covered by the offline data.
Corruption-Robust Offline Reinforcement Learning
Surprisingly, in this case, the knowledge of $\epsilon$ is necessary, as we show that being adaptive to unknown $\epsilon$ is impossible. This again contrasts with recent results on corruption-robust online RL and implies that robust offline RL is a strictly harder problem.
Mitigating Covariate Shift in Imitation Learning via Offline Data Without Great Coverage
Instead, the learner is presented with a static offline dataset of state-action-next state transition triples from a potentially less proficient behavior policy.
Mitigating Covariate Shift in Imitation Learning via Offline Data With Partial Coverage
Instead, the learner is presented with a static offline dataset of state-action-next state triples from a potentially less proficient behavior policy.
Bilinear Classes: A Structural Framework for Provable Generalization in RL
The framework incorporates nearly all existing models in which a polynomial sample complexity is achievable, and, notably, also includes new models, such as the Linear $Q^*/V^*$ model in which both the optimal $Q$-function and the optimal $V$-function are linear in some known feature space.
Optimism is All You Need: Model-Based Imitation Learning From Observation Alone
This paper studies Imitation Learning from Observations alone (ILFO) where the learner is presented with expert demonstrations that only consist of states encountered by an expert (without access to actions taken by the expert).
Fairness of Exposure in Stochastic Bandits
Contextual bandit algorithms have become widely used for recommendation in online systems (e. g. marketplaces, music streaming, news), where they now wield substantial influence on which items get exposed to the users.
MobILE: Model-Based Imitation Learning From Observation Alone
This paper studies Imitation Learning from Observations alone (ILFO) where the learner is presented with expert demonstrations that consist only of states visited by an expert (without access to actions taken by the expert).
Robust Policy Gradient against Strong Data Corruption
Our first result shows that no algorithm can find a better than $O(\epsilon)$-optimal policy under our attack model.
Finite Sample Analysis of Minimax Offline Reinforcement Learning: Completeness, Fast Rates and First-Order Efficiency
We offer a theoretical characterization of off-policy evaluation (OPE) in reinforcement learning using function approximation for marginal importance weights and $q$-functions when these are estimated using recent minimax methods.
Adaptive Federated Learning and Digital Twin for Industrial Internet of Things
Industrial Internet of Things (IoT) enables distributed intelligent services varying with the dynamic and realtime industrial devices to achieve Industry 4. 0 benefits.
Learning the Linear Quadratic Regulator from Nonlinear Observations
We introduce a new algorithm, RichID, which learns a near-optimal policy for the RichLQR with sample complexity scaling only with the dimension of the latent state space and the capacity of the decoder function class.
PC-PG: Policy Cover Directed Exploration for Provable Policy Gradient Learning
Direct policy gradient methods for reinforcement learning are a successful approach for a variety of reasons: they are model free, they directly optimize the performance metric of interest, and they allow for richly parameterized policies.
Information Theoretic Regret Bounds for Online Nonlinear Control
This work studies the problem of sequential control in an unknown, nonlinear dynamical system, where we model the underlying system dynamics as an unknown function in a known Reproducing Kernel Hilbert Space.
FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPs
In order to deal with the curse of dimensionality in reinforcement learning (RL), it is common practice to make parametric assumptions where values or policies are functions of some low dimensional feature space.
Provably Efficient Model-based Policy Adaptation
The high sample complexity of reinforcement learning challenges its use in practice.
Constrained episodic reinforcement learning in concave-convex and knapsack settings
We propose an algorithm for tabular episodic reinforcement learning with constraints.
Arbitrary Style Transfer via Multi-Adaptation Network
Arbitrary style transfer is a significant topic with research value and application prospect.
Disagreement-Regularized Imitation Learning
We present a simple and effective algorithm designed to address the covariate shift problem in imitation learning.
Exploration in Action Space
Parameter space exploration methods with black-box optimization have recently been shown to outperform state-of-the-art approaches in continuous control reinforcement learning domains.
Corruption-robust exploration in episodic reinforcement learning
We initiate the study of multi-stage episodic reinforcement learning under adversarial corruptions in both the rewards and the transition probabilities of the underlying system extending recent results for the special case of stochastic bandits.
Policy Poisoning in Batch Reinforcement Learning and Control
We study a security threat to batch reinforcement learning and control where the attacker aims to poison the learned policy.
Optimal Sketching for Kronecker Product Regression and Low Rank Approximation
For input $\mathcal{A}$ as above, we give $O(\sum_{i=1}^q \text{nnz}(A_i))$ time algorithms, which is much faster than computing $\mathcal{A}$.
Imitation Learning as $f$-Divergence Minimization
We show that the state-of-the-art methods such as GAIL and behavior cloning, due to their choice of loss function, often incorrectly interpolate between such modes.
Provably Efficient Imitation Learning from Observation Alone
We design a new model-free algorithm for ILFO, Forward Adversarial Imitation Learning (FAIL), which learns a sequence of time-dependent policies by minimizing an Integral Probability Metric between the observation distributions of the expert policy and the learner.
Efficient Model-free Reinforcement Learning in Metric Spaces
Model-free Reinforcement Learning (RL) algorithms such as Q-learning [Watkins, Dayan 92] have been widely used in practice and can achieve human level performance in applications such as video games [Mnih et al. 15].
Outcome-Driven Clustering of Acute Coronary Syndrome Patients using Multi-Task Neural Network with Attention
The results demonstrate cluster analysis using outcome-driven multi-task neural network as promising for patient classification and subtyping.
Contrasting Exploration in Parameter and Action Space: A Zeroth-Order Optimization Perspective
Black-box optimizers that explore in parameter space have often been shown to outperform more sophisticated action space exploration methods developed specifically for the reinforcement learning problem.
Model-based RL in Contextual Decision Processes: PAC bounds and Exponential Improvements over Model-free Approaches
We study the sample complexity of model-based reinforcement learning (henceforth RL) in general contextual decision processes that require strategic exploration to find a near-optimal policy.
Contextual Memory Trees
We design and study a Contextual Memory Tree (CMT), a learning memory controller that inserts new memories into an experience store of unbounded size.
Truncated Horizon Policy Search: Combining Reinforcement Learning & Imitation Learning
In this paper, we propose to combine imitation and reinforcement learning via the idea of reward shaping using an oracle.
Dual Policy Iteration
Recently, a novel class of Approximate Policy Iteration (API) algorithms have demonstrated impressive practical performance (e. g., ExIt from [2], AlphaGo-Zero from [27]).
Recurrent Predictive State Policy Networks
Predictive state policy networks consist of a recursive filter, which keeps track of a belief about the state of the environment, and a reactive policy that directly maps beliefs to actions, to maximize the cumulative reward.
Sketching for Kronecker Product Regression and P-splines
That is, TensorSketch only provides input sparsity time for Kronecker product regression with respect to the $2$-norm.
Predictive-State Decoders: Encoding the Future into Recurrent Networks
We seek to combine the advantages of RNNs and PSRs by augmenting existing state-of-the-art recurrent neural networks with Predictive-State Decoders (PSDs), which add supervision to the network's internal state representation to target predicting future observations.
Safety-Aware Algorithms for Adversarial Contextual Bandit
To address this problem, we first study online convex programming in the full information setting where in each round the learner receives an adversarial convex loss and a convex constraint.
Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Prediction
We demonstrate that AggreVaTeD --- a policy gradient extension of the Imitation Learning (IL) approach of (Ross & Bagnell, 2014) --- can leverage such an oracle to achieve faster and better solutions with less training data than a less-informed Reinforcement Learning (RL) technique.
Gradient Boosting on Stochastic Data Streams
To generalize from batch to online, we first introduce the definition of online weak learning edge with which for strongly convex and smooth loss functions, we present an algorithm, Streaming Gradient Boosting (SGB) with exponential shrinkage guarantees in the number of weak learners.
Risk-Aware Algorithms for Adversarial Contextual Bandits
To address this problem, we first study the full information setting where in each round the learner receives an adversarial convex loss and a convex constraint.
No-Regret Replanning under Uncertainty
This paper explores the problem of path planning under uncertainty.
Learning to Filter with Predictive State Inference Machines
Latent state space models are a fundamental and widely used tool for modeling dynamical systems.
