Search Results for author:
Found 49 papers, 18 papers with code
Hybrid Reinforcement Learning from Offline Observation Alone
Under the admissibility assumptions -- that the offline data could actually be produced by the policy class we consider -- we propose the first algorithm in the trace model setting that provably matches the performance of algorithms that leverage a reset model.
The Importance of Online Data: Understanding Preference Fine-tuning via Coverage
The two most common families of techniques -- online reinforcement learning (RL) such as Proximal Policy Optimization (PPO) and offline contrastive methods such as Direct Preference Optimization (DPO) -- were positioned as equivalent in prior work due to the fact that both have to start from the same offline preference dataset.
REBEL: Reinforcement Learning via Regressing Relative Rewards
While originally developed for continuous control problems, Proximal Policy Optimization (PPO) has emerged as the work-horse of a variety of reinforcement learning (RL) applications, including the fine-tuning of generative models.
Hybrid Inverse Reinforcement Learning
In this work, we propose using hybrid RL -- training on a mixture of online and expert data -- to curtail unnecessary exploration.
The Virtues of Pessimism in Inverse Reinforcement Learning
Inverse Reinforcement Learning (IRL) is a powerful framework for learning complex behaviors from expert demonstrations.
Inverse Reinforcement Learning without Reinforcement Learning
In this work, we demonstrate for the first time a more informed imitation learning reduction where we utilize the state distribution of the expert to alleviate the global exploration component of the RL subroutine, providing an exponential speedup in theory.
The Virtues of Laziness in Model-based RL: A Unified Objective and Algorithms
We propose a novel approach to addressing two fundamental challenges in Model-based Reinforcement Learning (MBRL): the computational expense of repeatedly finding a good policy in the learned model, and the objective mismatch between model fitting and policy computation.
Hybrid RL: Using Both Offline and Online Data Can Make RL Efficient
We consider a hybrid reinforcement learning setting (Hybrid RL), in which an agent has access to an offline dataset and the ability to collect experience via real-world online interaction.
Game-Theoretic Algorithms for Conditional Moment Matching
A variety of problems in econometrics and machine learning, including instrumental variable regression and Bellman residual minimization, can be formulated as satisfying a set of conditional moment restrictions (CMR).
Sequence Model Imitation Learning with Unobserved Contexts
We consider imitation learning problems where the learner's ability to mimic the expert increases throughout the course of an episode as more information is revealed.
Minimax Optimal Online Imitation Learning via Replay Estimation
In the tabular setting or with linear function approximation, our meta theorem shows that the performance gap incurred by our approach achieves the optimal $\widetilde{O} \left( \min({H^{3/2}} / {N}, {H} / {\sqrt{N}} \right)$ dependency, under significantly weaker assumptions compared to prior work.
Causal Imitation Learning under Temporally Correlated Noise
We develop algorithms for imitation learning from policy data that was corrupted by temporally correlated noise in expert actions.
On the Effectiveness of Iterative Learning Control
However, there is little prior theoretical work that explains the effectiveness of ILC even in the presence of large modeling errors, where optimal control methods using the misspecified model (MM) often perform poorly.
A Critique of Strictly Batch Imitation Learning
Recent work by Jarrett et al. attempts to frame the problem of offline imitation learning (IL) as one of learning a joint energy-based model, with the hope of out-performing standard behavioral cloning.
Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gap
We provide a unifying view of a large family of previous imitation learning algorithms through the lens of moment matching.
Feedback in Imitation Learning: The Three Regimes of Covariate Shift
The learner often comes to rely on features that are strongly predictive of decisions, but are subject to strong covariate shift.
CMAX++ : Leveraging Experience in Planning and Execution using Inaccurate Models
In this paper we propose CMAX++, an approach that leverages real-world experience to improve the quality of resulting plans over successive repetitions of a robotic task.
Exploration in Action Space
Parameter space exploration methods with black-box optimization have recently been shown to outperform state-of-the-art approaches in continuous control reinforcement learning domains.
TRON: A Fast Solver for Trajectory Optimization with Non-Smooth Cost Functions
TRON achieves this by exploiting the structure of the objective to adaptively smooth the cost function, resulting in a sequence of objectives that can be efficiently optimized.
Robotics Systems and Control Systems and Control
Planning and Execution using Inaccurate Models with Provable Guarantees
In this paper, we propose CMAX an approach for interleaving planning and execution.
Provably Efficient Imitation Learning from Observation Alone
We design a new model-free algorithm for ILFO, Forward Adversarial Imitation Learning (FAIL), which learns a sequence of time-dependent policies by minimizing an Integral Probability Metric between the observation distributions of the expert policy and the learner.
Contrasting Exploration in Parameter and Action Space: A Zeroth-Order Optimization Perspective
Black-box optimizers that explore in parameter space have often been shown to outperform more sophisticated action space exploration methods developed specifically for the reinforcement learning problem.
An Algorithmic Perspective on Imitation Learning
This process of learning from demonstrations, and the study of algorithms to do so, is called imitation learning.
Truncated Horizon Policy Search: Combining Reinforcement Learning & Imitation Learning
In this paper, we propose to combine imitation and reinforcement learning via the idea of reward shaping using an oracle.
Dual Policy Iteration
Recently, a novel class of Approximate Policy Iteration (API) algorithms have demonstrated impressive practical performance (e. g., ExIt from [2], AlphaGo-Zero from [27]).
Anytime Neural Network: a Versatile Trade-off Between Computation and Accuracy
We present an approach for anytime predictions in deep neural networks (DNNs).
Log-DenseNet: How to Sparsify a DenseNet
Skip connections are increasingly utilized by deep neural networks to improve accuracy and cost-efficiency.
Predictive-State Decoders: Encoding the Future into Recurrent Networks
We seek to combine the advantages of RNNs and PSRs by augmenting existing state-of-the-art recurrent neural networks with Predictive-State Decoders (PSDs), which add supervision to the network's internal state representation to target predicting future observations.
Ignoring Distractors in the Absence of Labels: Optimal Linear Projection to Remove False Positives During Anomaly Detection
We demonstrate that this method is able to remove uninformative parts of the feature space for the anomaly detection setting.
Learning Anytime Predictions in Neural Networks via Adaptive Loss Balancing
Experimentally, the adaptive weights induce more competitive anytime predictions on multiple recognition data-sets and models than non-adaptive approaches including weighing all losses equally.
Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Prediction
We demonstrate that AggreVaTeD --- a policy gradient extension of the Imitation Learning (IL) approach of (Ross & Bagnell, 2014) --- can leverage such an oracle to achieve faster and better solutions with less training data than a less-informed Reinforcement Learning (RL) technique.
Gradient Boosting on Stochastic Data Streams
To generalize from batch to online, we first introduce the definition of online weak learning edge with which for strongly convex and smooth loss functions, we present an algorithm, Streaming Gradient Boosting (SGB) with exponential shrinkage guarantees in the number of weak learners.
A Discriminative Framework for Anomaly Detection in Large Videos
We address an anomaly detection setting in which training sequences are unavailable and anomalies are scored independently of temporal ordering.
Learning Transferable Policies for Monocular Reactive MAV Control
The ability to transfer knowledge gained in previous tasks into new contexts is one of the most important mechanisms of human learning.
Introspective Perception: Learning to Predict Failures in Vision Systems
As robots aspire for long-term autonomous operations in complex dynamic environments, the ability to reliably take mission-critical decisions in ambiguous situations becomes critical.
Learning to Filter with Predictive State Inference Machines
Latent state space models are a fundamental and widely used tool for modeling dynamical systems.
Predicting Multiple Structured Visual Interpretations
We present a simple approach for producing a small number of structured visual outputs which have high recall, for a variety of tasks including monocular pose estimation and semantic scene segmentation.
Solving Games with Functional Regret Estimation
We propose a novel online learning method for minimizing regret in large extensive-form games.
Vision and Learning for Deliberative Monocular Cluttered Flight
Cameras provide a rich source of information while being passive, cheap and lightweight for small and medium Unmanned Aerial Vehicles (UAVs).
A Unified View of Large-scale Zero-sum Equilibrium Computation
The task of computing approximate Nash equilibria in large zero-sum extensive-form games has received a tremendous amount of attention due mainly to the Annual Computer Poker Competition.
Visual Chunking: A List Prediction Framework for Region-Based Object Detection
We present an efficient algorithm with provable performance for building a high-quality list of detections from any candidate set of region-based proposals.
Efficient Feature Group Sequencing for Anytime Linear Prediction
We theoretically guarantee that our algorithms achieve near-optimal linear predictions at each budget when a feature group is chosen.
Reinforcement and Imitation Learning via Interactive No-Regret Learning
Recent work has demonstrated that problems-- particularly imitation learning and structured prediction-- where a learner's predictions influence the input-distribution it is tested on can be naturally addressed by an interactive approach and analyzed using no-regret online learning.
Near Optimal Bayesian Active Learning for Decision Making
Instead of minimizing uncertainty per se, we consider a set of overlapping decision regions of these hypotheses.
SpeedMachines: Anytime Structured Prediction
Structured prediction plays a central role in machine learning applications from computational biology to computer vision.
Knapsack Constrained Contextual Submodular List Prediction with Application to Multi-document Summarization
We study the problem of predicting a set or list of options under knapsack constraint.
Computational Rationalization: The Inverse Equilibrium Problem
Modeling the purposeful behavior of imperfect agents from a small number of observations is a challenging task.
Learning Policies for Contextual Submodular Prediction
Many prediction domains, such as ad placement, recommendation, trajectory prediction, and document summarization, require predicting a set or list of options.
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Sequential prediction problems such as imitation learning, where future observations depend on previous predictions (actions), violate the common i. i. d.
