Search Results for author:
Found 21 papers, 3 papers with code
Offline Reinforcement Learning: Role of State Aggregation and Trajectory Data
We revisit the problem of offline reinforcement learning with value function realizability but without Bellman completeness.
Harnessing Density Ratios for Online Reinforcement Learning
The theories of offline and online reinforcement learning, despite having evolved in parallel, have begun to show signs of the possibility for a unification, with algorithms and analysis techniques for one setting often having natural counterparts in the other.
Offline Data Enhanced On-Policy Policy Gradient with Provable Guarantees
In this work, we propose a new hybrid RL algorithm that combines an on-policy actor-critic method with offline data.
Contextual Bandits and Imitation Learning via Preference-Based Active Queries
We consider the problem of contextual bandits and imitation learning, where the learner lacks direct knowledge of the executed action's reward.
Ticketed Learning-Unlearning Schemes
Subsequently, given any subset of examples that wish to be unlearnt, the goal is to learn, without the knowledge of the original training dataset, a good predictor that is identical to the predictor that would have been produced when learning from scratch on the surviving examples.
Hidden Poison: Machine Unlearning Enables Camouflaged Poisoning Attacks
We introduce camouflaged data poisoning attacks, a new attack vector that arises in the context of machine unlearning and other settings when model retraining may be induced.
From Gradient Flow on Population Loss to Learning with Stochastic Gradient Descent
When these potentials further satisfy certain self-bounding properties, we show that they can be used to provide a convergence guarantee for Gradient Descent (GD) and SGD (even when the paths of GF and GD/SGD are quite far apart).
Hybrid RL: Using Both Offline and Online Data Can Make RL Efficient
We consider a hybrid reinforcement learning setting (Hybrid RL), in which an agent has access to an offline dataset and the ability to collect experience via real-world online interaction.
On the Complexity of Adversarial Decision Making
A central problem in online learning and decision making -- from bandits to reinforcement learning -- is to understand what modeling assumptions lead to sample-efficient learning guarantees.
Computationally Efficient PAC RL in POMDPs with Latent Determinism and Conditional Embeddings
We show our algorithm's computational and statistical complexities scale polynomially with respect to the horizon and the intrinsic dimension of the feature on the observation space.
Provably Efficient Reinforcement Learning in Partially Observable Dynamical Systems
We study Reinforcement Learning for partially observable dynamical systems using function approximation.
Guarantees for Epsilon-Greedy Reinforcement Learning with Function Approximation
This paper presents a theoretical analysis of such policies and provides the first regret and sample-complexity bounds for reinforcement learning with myopic exploration.
SGD: The Role of Implicit Regularization, Batch-size and Multiple-epochs
We show that there is an SCO problem such that GD with any step size and number of iterations can only learn at a suboptimal rate: at least $\widetilde{\Omega}(1/n^{5/12})$.
Agnostic Reinforcement Learning with Low-Rank MDPs and Rich Observations
In this work, we consider the more realistic setting of agnostic RL with rich observation spaces and a fixed class of policies $\Pi$ that may not contain any near-optimal policy.
Neural Active Learning with Performance Guarantees
We investigate the problem of active learning in the streaming setting in non-parametric regimes, where the labels are stochastically generated from a class of functions on which we make no assumptions whatsoever.
Remember What You Want to Forget: Algorithms for Machine Unlearning
We study the problem of unlearning datapoints from a learnt model.
Second-Order Information in Non-Convex Stochastic Optimization: Power and Limitations
We design an algorithm which finds an $\epsilon$-approximate stationary point (with $\|\nabla F(x)\|\le \epsilon$) using $O(\epsilon^{-3})$ stochastic gradient and Hessian-vector products, matching guarantees that were previously available only under a stronger assumption of access to multiple queries with the same random seed.
Reinforcement Learning with Feedback Graphs
We study episodic reinforcement learning in Markov decision processes when the agent receives additional feedback per step in the form of several transition observations.
The Complexity of Making the Gradient Small in Stochastic Convex Optimization
Notably, we show that in the global oracle/statistical learning model, only logarithmic dependence on smoothness is required to find a near-stationary point, whereas polynomial dependence on smoothness is necessary in the local stochastic oracle model.
Uniform Convergence of Gradients for Non-Convex Learning and Optimization
We investigate 1) the rate at which refined properties of the empirical risk---in particular, gradients---converge to their population counterparts in standard non-convex learning tasks, and 2) the consequences of this convergence for optimization.
A Brief Study of In-Domain Transfer and Learning from Fewer Samples using A Few Simple Priors
Domain knowledge can often be encoded in the structure of a network, such as convolutional layers for vision, which has been shown to increase generalization and decrease sample complexity, or the number of samples required for successful learning.
