Search Results for author: Byron Boots

Found 69 papers, 18 papers with code

Leveraging Experience in Lazy Search

no code implementations10 Oct 2021 Mohak Bhardwaj, Sanjiban Choudhury, Byron Boots, Siddhartha Srinivasa

If new search problems are sufficiently similar to problems solved during training, the learned policy will choose a good edge evaluation ordering and solve the motion planning problem quickly.

Imitation Learning Motion Planning

Safe Reinforcement Learning Using Advantage-Based Intervention

1 code implementation16 Jun 2021 Nolan Wagener, Byron Boots, Ching-An Cheng

We propose a new algorithm, SAILR, that uses an intervention mechanism based on advantage functions to keep the agent safe throughout training and optimizes the agent's policy using off-the-shelf RL algorithms designed for unconstrained MDPs.

Safe Reinforcement Learning

Imitation Learning via Simultaneous Optimization of Policies and Auxiliary Trajectories

no code implementations7 May 2021 Mandy Xie, Anqi Li, Karl Van Wyk, Frank Dellaert, Byron Boots, Nathan Ratliff

Many IL methods, such as Dataset Aggregation (DAgger), combat challenges like distributional shift by interacting with oracular experts.

Imitation Learning

Fast and Efficient Locomotion via Learned Gait Transitions

no code implementations9 Apr 2021 Yuxiang Yang, Tingnan Zhang, Erwin Coumans, Jie Tan, Byron Boots

We focus on the problem of developing energy efficient controllers for quadrupedal robots.

The Value of Planning for Infinite-Horizon Model Predictive Control

2 code implementations7 Apr 2021 Nathan Hatch, Byron Boots

We show that that this value function can be used by MPC directly, resulting in more efficient and resilient behavior at runtime.

RMP2: A Structured Composable Policy Class for Robot Learning

no code implementations10 Mar 2021 Anqi Li, Ching-An Cheng, M. Asif Rana, Man Xie, Karl Van Wyk, Nathan Ratliff, Byron Boots

Using RMPflow as a structured policy class in learning has several benefits, such as sufficient expressiveness, the flexibility to inject different levels of prior knowledge as well as the ability to transfer policies between robots.

Combining pretrained CNN feature extractors to enhance clustering of complex natural images

no code implementations7 Jan 2021 Joris Guerin, Stephane Thiery, Eric Nyiri, Olivier Gibaru, Byron Boots

First, extensive experiments are conducted and show that, for a given dataset, the choice of the CNN architecture for feature extraction has a huge impact on the final clustering.

Image Clustering Unsupervised Image Classification

Towards Coordinated Robot Motions: End-to-End Learning of Motion Policies on Transform Trees

no code implementations24 Dec 2020 M. Asif Rana, Anqi Li, Dieter Fox, Sonia Chernova, Byron Boots, Nathan Ratliff

The policy structure provides the user an interface to 1) specifying the spaces that are directly relevant to the completion of the tasks, and 2) designing policies for certain tasks that do not need to be learned.

Blending MPC & Value Function Approximation for Efficient Reinforcement Learning

no code implementations ICLR 2021 Mohak Bhardwaj, Sanjiban Choudhury, Byron Boots

We further propose an algorithm that changes $\lambda$ over time to reduce the dependence on MPC as our estimates of the value function improve, and test the efficacy our approach on challenging high-dimensional manipulation tasks with biased models in simulation.

Stein Variational Model Predictive Control

no code implementations15 Nov 2020 Alexander Lambert, Adam Fishman, Dieter Fox, Byron Boots, Fabio Ramos

By casting MPC as a Bayesian inference problem, we employ variational methods for posterior computation, naturally encoding the complexity and multi-modality of the decision making problem.

Bayesian Inference Decision Making +1

Quantum Tensor Networks, Stochastic Processes, and Weighted Automata

no code implementations20 Oct 2020 Siddarth Srinivasan, Sandesh Adhikary, Jacob Miller, Guillaume Rabusseau, Byron Boots

We address this gap by showing how stationary or uniform versions of popular quantum tensor network models have equivalent representations in the stochastic processes and weighted automata literature, in the limit of infinitely long sequences.

Tensor Networks

Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion

no code implementations21 Sep 2020 Xingye Da, Zhaoming Xie, David Hoeller, Byron Boots, Animashree Anandkumar, Yuke Zhu, Buck Babich, Animesh Garg

We present a hierarchical framework that combines model-based control and reinforcement learning (RL) to synthesize robust controllers for a quadruped (the Unitree Laikago).

Explaining Fast Improvement in Online Imitation Learning

no code implementations6 Jul 2020 Xinyan Yan, Byron Boots, Ching-An Cheng

Here policies are optimized by performing online learning on a sequence of loss functions that encourage the learner to mimic expert actions, and if the online learning has no regret, the agent can provably learn an expert-like policy.

Decision Making Imitation Learning +1

Euclideanizing Flows: Diffeomorphic Reduction for Learning Stable Dynamical Systems

2 code implementations27 May 2020 Muhammad Asif Rana, Anqi Li, Dieter Fox, Byron Boots, Fabio Ramos, Nathan Ratliff

The complex motions are encoded as rollouts of a stable dynamical system, which, under a change of coordinates defined by a diffeomorphism, is equivalent to a simple, hand-specified dynamical system.

Density Estimation

Pairwise Similarity Knowledge Transfer for Weakly Supervised Object Localization

1 code implementation ECCV 2020 Amir Rahimi, Amirreza Shaban, Thalaiyasingam Ajanthan, Richard Hartley, Byron Boots

Weakly Supervised Object Localization (WSOL) methods only require image level labels as opposed to expensive bounding box annotations required by fully supervised algorithms.

Transfer Learning Weakly-Supervised Object Localization

Intra Order-preserving Functions for Calibration of Multi-Class Neural Networks

1 code implementation NeurIPS 2020 Amir Rahimi, Amirreza Shaban, Ching-An Cheng, Richard Hartley, Byron Boots

A common approach is to learn a post-hoc calibration function that transforms the output of the original network into calibrated confidence scores while maintaining the network's accuracy.

Information Theoretic Model Predictive Q-Learning

no code implementations31 Dec 2019 Mohak Bhardwaj, Ankur Handa, Dieter Fox, Byron Boots

Model-free Reinforcement Learning (RL) works well when experience can be collected cheaply and model-based RL is effective when system dynamics can be modeled accurately.

Decision Making Q-Learning

Continuous Online Learning and New Insights to Online Imitation Learning

no code implementations3 Dec 2019 Jonathan Lee, Ching-An Cheng, Ken Goldberg, Byron Boots

We prove that there is a fundamental equivalence between achieving sublinear dynamic regret in COL and solving certain EPs, and we present a reduction from dynamic regret to both static regret and convergence rate of the associated EP.

Imitation Learning

Expressiveness and Learning of Hidden Quantum Markov Models

no code implementations2 Dec 2019 Sandesh Adhikary, Siddarth Srinivasan, Geoff Gordon, Byron Boots

Extending classical probabilistic reasoning using the quantum mechanical view of probability has been of recent interest, particularly in the development of hidden quantum Markov models (HQMMs) to model stochastic processes.

A Reduction from Reinforcement Learning to No-Regret Online Learning

no code implementations14 Nov 2019 Ching-An Cheng, Remi Tachet des Combes, Byron Boots, Geoff Gordon

We present a reduction from reinforcement learning (RL) to no-regret online learning based on the saddle-point formulation of RL, by which "any" online algorithm with sublinear regret can generate policies with provable performance guarantees.

IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Data

no code implementations13 Nov 2019 Ajay Mandlekar, Fabio Ramos, Byron Boots, Silvio Savarese, Li Fei-Fei, Animesh Garg, Dieter Fox

For simple short-horizon manipulation tasks with modest variation in task instances, offline learning from a small set of demonstrations can produce controllers that successfully solve the task.

Riemannian Motion Policy Fusion through Learnable Lyapunov Function Reshaping

no code implementations7 Oct 2019 Mustafa Mukadam, Ching-An Cheng, Dieter Fox, Byron Boots, Nathan Ratliff

RMPfusion supplements RMPflow with weight functions that can hierarchically reshape the Lyapunov functions of the subtask RMPs according to the current configuration of the robot and environment.

Imitation Learning

Trajectory-wise Control Variates for Variance Reduction in Policy Gradient Methods

no code implementations8 Aug 2019 Ching-An Cheng, Xinyan Yan, Byron Boots

This can be attributed, at least in part, to the high variance in estimating the gradient of the task objective with Monte Carlo methods.

Policy Gradient Methods

Leveraging Experience in Lazy Search

no code implementations16 Jul 2019 Mohak Bhardwaj, Sanjiban Choudhury, Byron Boots, Siddhartha Srinivasa

If new search problems are sufficiently similar to problems solved during training, the learned policy will choose a good edge evaluation ordering and solve the motion planning problem quickly.

Imitation Learning Motion Planning

Provably Efficient Imitation Learning from Observation Alone

1 code implementation27 May 2019 Wen Sun, Anirudh Vemula, Byron Boots, J. Andrew Bagnell

We design a new model-free algorithm for ILFO, Forward Adversarial Imitation Learning (FAIL), which learns a sequence of time-dependent policies by minimizing an Integral Probability Metric between the observation distributions of the expert policy and the learner.

Imitation Learning OpenAI Gym

Composing Task-Agnostic Policies with Deep Reinforcement Learning

no code implementations ICLR 2020 Ahmed H. Qureshi, Jacob J. Johnson, Yuzhe Qin, Taylor Henderson, Byron Boots, Michael C. Yip

The composition of elementary behaviors to solve challenging transfer learning problems is one of the key elements in building intelligent machines.

Decision Making Motion Planning +1

Learning Quantum Graphical Models using Constrained Gradient Descent on the Stiefel Manifold

no code implementations9 Mar 2019 Sandesh Adhikary, Siddarth Srinivasan, Byron Boots

Quantum graphical models (QGMs) extend the classical framework for reasoning about uncertainty by incorporating the quantum mechanical view of probability.

An Online Learning Approach to Model Predictive Control

no code implementations24 Feb 2019 Nolan Wagener, Ching-An Cheng, Jacob Sacks, Byron Boots

In this paper, we show that there exists a close connection between MPC and online learning, an abstract theoretical framework for analyzing online decision making in the optimization literature.

Decision Making

Online Learning with Continuous Variations: Dynamic Regret and Reductions

no code implementations19 Feb 2019 Ching-An Cheng, Jonathan Lee, Ken Goldberg, Byron Boots

Furthermore, we show for COL a reduction from dynamic regret to both static regret and convergence in the associated EP, allowing us to analyze the dynamic regret of many existing algorithms.

Multi-Objective Policy Generation for Multi-Robot Systems Using Riemannian Motion Policies

1 code implementation14 Feb 2019 Anqi Li, Mustafa Mukadam, Magnus Egerstedt, Byron Boots

We propose a collection of RMPs for simple multi-robot tasks that can be used for building controllers for more complicated tasks.

Robotics

RMPflow: A Computational Graph for Automatic Motion Policy Generation

1 code implementation16 Nov 2018 Ching-An Cheng, Mustafa Mukadam, Jan Issac, Stan Birchfield, Dieter Fox, Byron Boots, Nathan Ratliff

We develop a novel policy synthesis algorithm, RMPflow, based on geometrically consistent transformations of Riemannian Motion Policies (RMPs).

Robotics Systems and Control

Differentiable MPC for End-to-end Planning and Control

2 code implementations NeurIPS 2018 Brandon Amos, Ivan Dario Jimenez Rodriguez, Jacob Sacks, Byron Boots, J. Zico Kolter

We present foundations for using Model Predictive Control (MPC) as a differentiable policy class for reinforcement learning in continuous state and action spaces.

Imitation Learning

Truncated Back-propagation for Bilevel Optimization

2 code implementations25 Oct 2018 Amirreza Shaban, Ching-An Cheng, Nathan Hatch, Byron Boots

Bilevel optimization has been recently revisited for designing and analyzing algorithms in hyperparameter tuning and meta learning tasks.

bilevel optimization Meta-Learning

Predictor-Corrector Policy Optimization

1 code implementation15 Oct 2018 Ching-An Cheng, Xinyan Yan, Nathan Ratliff, Byron Boots

We present a predictor-corrector framework, called PicCoLO, that can transform a first-order model-free reinforcement or imitation learning algorithm into a new hybrid method that leverages predictive models to accelerate policy learning.

Imitation Learning

Orthogonally Decoupled Variational Gaussian Processes

1 code implementation NeurIPS 2018 Hugh Salimbeni, Ching-An Cheng, Byron Boots, Marc Deisenroth

It adopts an orthogonal basis in the mean function to model the residues that cannot be learned by the standard coupled approach.

Gaussian Processes Variational Inference

Learning to Align Images using Weak Geometric Supervision

no code implementations4 Aug 2018 Jing Dong, Byron Boots, Frank Dellaert, Ranveer Chandra, Sudipta N. Sinha

Such descriptors are often derived using supervised learning on existing datasets with ground truth correspondences.

Semantically Meaningful View Selection

1 code implementation26 Jul 2018 Joris Guérin, Olivier Gibaru, Eric Nyiri, Stéphane Thiery, Byron Boots

Although deep learning has facilitated progress in image understanding, a robot's performance in problems like object recognition often depends on the angle from which the object is observed.

Object Recognition

Improving Image Clustering With Multiple Pretrained CNN Feature Extractors

1 code implementation20 Jul 2018 Joris Guérin, Byron Boots

For many image clustering problems, replacing raw image data with features extracted by a pretrained convolutional neural network (CNN), leads to better clustering performance.

Image Clustering

Accelerating Imitation Learning with Predictive Models

no code implementations12 Jun 2018 Ching-An Cheng, Xinyan Yan, Evangelos A. Theodorou, Byron Boots

When the model oracle is learned online, these algorithms can provably accelerate the best known convergence rate up to an order.

Imitation Learning

Truncated Horizon Policy Search: Combining Reinforcement Learning & Imitation Learning

no code implementations ICLR 2018 Wen Sun, J. Andrew Bagnell, Byron Boots

In this paper, we propose to combine imitation and reinforcement learning via the idea of reward shaping using an oracle.

Imitation Learning

Dual Policy Iteration

no code implementations NeurIPS 2018 Wen Sun, Geoffrey J. Gordon, Byron Boots, J. Andrew Bagnell

Recently, a novel class of Approximate Policy Iteration (API) algorithms have demonstrated impressive practical performance (e. g., ExIt from [2], AlphaGo-Zero from [27]).

Continuous Control

Fast Policy Learning through Imitation and Reinforcement

no code implementations26 May 2018 Ching-An Cheng, Xinyan Yan, Nolan Wagener, Byron Boots

We show that if the switching time is properly randomized, LOKI can learn to outperform a suboptimal expert and converge faster than running policy gradient from scratch.

Imitation Learning

Convergence of Value Aggregation for Imitation Learning

no code implementations22 Jan 2018 Ching-An Cheng, Byron Boots

Value aggregation is a general framework for solving imitation learning problems.

Imitation Learning

Initialization matters: Orthogonal Predictive State Recurrent Neural Networks

no code implementations ICLR 2018 Krzysztof Choromanski, Carlton Downey, Byron Boots

In this paper, we extend the theory of ORFs to Kernel Ridge Regression and show that ORFs can be used to obtain Orthogonal PSRNNs (OPSRNNs), which are smaller and faster than PSRNNs.

Time Series

Variational Inference for Gaussian Process Models with Linear Complexity

no code implementations NeurIPS 2017 Ching-An Cheng, Byron Boots

Furthermore, it yields a variational inference problem that can be solved by stochastic gradient ascent with time and space complexity that is only linear in the number of mean function parameters, regardless of the choice of kernels, likelihoods, and inducing points.

Variational Inference

Deep Forward and Inverse Perceptual Models for Tracking and Prediction

no code implementations31 Oct 2017 Alexander Lambert, Amirreza Shaban, Amit Raj, Zhen Liu, Byron Boots

We consider the problems of learning forward models that map state to high-dimensional images and inverse models that map high-dimensional images to state in robotics.

Image Generation

Learning Hidden Quantum Markov Models

no code implementations24 Oct 2017 Siddarth Srinivasan, Geoff Gordon, Byron Boots

We extend previous work on HQMMs with three contributions: (1) we show how classical hidden Markov models (HMMs) can be simulated on a quantum circuit, (2) we reformulate HQMMs by relaxing the constraints for modeling HMMs on quantum circuits, and (3) we present a learning algorithm to estimate the parameters of an HQMM from data.

Manifold Regularization for Kernelized LSTD

no code implementations15 Oct 2017 Xinyan Yan, Krzysztof Choromanski, Byron Boots, Vikas Sindhwani

Policy evaluation or value function or Q-function approximation is a key procedure in reinforcement learning (RL).

Policy Gradient Methods

Predictive-State Decoders: Encoding the Future into Recurrent Networks

no code implementations NeurIPS 2017 Arun Venkatraman, Nicholas Rhinehart, Wen Sun, Lerrel Pinto, Martial Hebert, Byron Boots, Kris M. Kitani, J. Andrew Bagnell

We seek to combine the advantages of RNNs and PSRs by augmenting existing state-of-the-art recurrent neural networks with Predictive-State Decoders (PSDs), which add supervision to the network's internal state representation to target predicting future observations.

Imitation Learning

Imitation Learning for Agile Autonomous Driving

no code implementations21 Sep 2017 Yunpeng Pan, Ching-An Cheng, Kamil Saigol, Keuntaek Lee, Xinyan Yan, Evangelos Theodorou, Byron Boots

We present an end-to-end imitation learning system for agile, off-road autonomous driving using only low-cost on-board sensors.

Robotics

Continuous-Time Gaussian Process Motion Planning via Probabilistic Inference

1 code implementation24 Jul 2017 Mustafa Mukadam, Jing Dong, Xinyan Yan, Frank Dellaert, Byron Boots

We benchmark our algorithms against several sampling-based and trajectory optimization-based motion planning algorithms on planning problems in multiple environments.

Robotics

Predictive State Recurrent Neural Networks

no code implementations NeurIPS 2017 Carlton Downey, Ahmed Hefny, Boyue Li, Byron Boots, Geoffrey Gordon

We present a new model, Predictive State Recurrent Neural Networks (PSRNNs), for filtering and prediction in dynamical systems.

Tensor Decomposition

Sparse Gaussian Processes for Continuous-Time Trajectory Estimation on Matrix Lie Groups

2 code implementations17 May 2017 Jing Dong, Byron Boots, Frank Dellaert

Continuous-time trajectory representations are a powerful tool that can be used to address several issues in many practical simultaneous localization and mapping (SLAM) scenarios, like continuously collected measurements distorted by robot motion, or during with asynchronous sensor measurements.

Robotics

Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Prediction

no code implementations ICML 2017 Wen Sun, Arun Venkatraman, Geoffrey J. Gordon, Byron Boots, J. Andrew Bagnell

We demonstrate that AggreVaTeD --- a policy gradient extension of the Imitation Learning (IL) approach of (Ross & Bagnell, 2014) --- can leverage such an oracle to achieve faster and better solutions with less training data than a less-informed Reinforcement Learning (RL) technique.

Decision Making Dependency Parsing +1

Incremental Variational Sparse Gaussian Process Regression

no code implementations NeurIPS 2016 Ching-An Cheng, Byron Boots

Recent work on scaling up Gaussian process regression (GPR) to large datasets has primarily focused on sparse GPR, which leverages a small set of basis functions to approximate the full Gaussian process during inference.

GPR Incremental Learning

4D Crop Monitoring: Spatio-Temporal Reconstruction for Agriculture

no code implementations8 Oct 2016 Jing Dong, John Gary Burnham, Byron Boots, Glen C. Rains, Frank Dellaert

Autonomous crop monitoring at high spatial and temporal resolution is a critical problem in precision agriculture.

Structure from Motion

Adaptive Probabilistic Trajectory Optimization via Efficient Approximate Inference

no code implementations22 Aug 2016 Yunpeng Pan, Xinyan Yan, Evangelos Theodorou, Byron Boots

Robotic systems must be able to quickly and robustly make decisions when operating in uncertain and dynamic environments.

Learning from Conditional Distributions via Dual Embeddings

no code implementations15 Jul 2016 Bo Dai, Niao He, Yunpeng Pan, Byron Boots, Le Song

In such problems, each sample $x$ itself is associated with a conditional distribution $p(z|x)$ represented by samples $\{z_i\}_{i=1}^M$, and the goal is to learn a function $f$ that links these conditional distributions to target values $y$.

Learning to Filter with Predictive State Inference Machines

no code implementations30 Dec 2015 Wen Sun, Arun Venkatraman, Byron Boots, J. Andrew Bagnell

Latent state space models are a fundamental and widely used tool for modeling dynamical systems.

Hilbert Space Embeddings of Predictive State Representations

no code implementations26 Sep 2013 Byron Boots, Geoffrey Gordon, Arthur Gretton

The essence is to represent the state as a nonparametric conditional embedding operator in a Reproducing Kernel Hilbert Space (RKHS) and leverage recent work in kernel methods to estimate, predict, and update the representation.

Predictive State Temporal Difference Learning

no code implementations NeurIPS 2010 Byron Boots, Geoffrey J. Gordon

We propose a new approach to value function approximation which combines linear temporal difference reinforcement learning with subspace identification.

Reduced-Rank Hidden Markov Models

no code implementations6 Oct 2009 Sajid M. Siddiqi, Byron Boots, Geoffrey J. Gordon

We introduce the Reduced-Rank Hidden Markov Model (RR-HMM), a generalization of HMMs that can model smooth state evolution as in Linear Dynamical Systems (LDSs) as well as non-log-concave predictive distributions as in continuous-observation HMMs.

Cannot find the paper you are looking for? You can Submit a new open access paper.