Search Results for author:
Found 87 papers, 24 papers with code
Dynamic Non-Prehensile Object Transport via Model-Predictive Reinforcement Learning
We investigate the problem of teaching a robot manipulator to perform dynamic non-prehensile object transport, also known as the `robot waiter' task, from a limited set of real-world demonstrations.
Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL
To the best of our knowledge, this is the first evidence that simulation transfer yields a provable gain in reinforcement learning in settings where direct sim2real transfer fails.
DATT: Deep Adaptive Trajectory Tracking for Quadrotor Control
Precise arbitrary trajectory tracking for quadrotors is challenging due to unknown nonlinear dynamics, trajectory infeasibility, and actuation limits.
Deep Model Predictive Optimization
A major challenge in robotics is to design robust policies which enable complex and agile behaviors in the real world.
LiDAR-UDA: Self-ensembling Through Time for Unsupervised LiDAR Domain Adaptation
Existing self-training methods use a model trained on labeled source data to generate pseudo labels for target data and refine the predictions via fine-tuning the network on the pseudo labels.
Stackelberg Games for Learning Emergent Behaviors During Competitive Autocurricula
Autocurricular training is an important sub-area of multi-agent reinforcement learning~(MARL) that allows multiple agents to learn emergent skills in an unsupervised co-evolving scheme.
Continuous Versatile Jumping Using Learned Action Residuals
Jumping is essential for legged robots to traverse through difficult terrains.
MAHALO: Unifying Offline Reinforcement Learning and Imitation Learning from Observations
We study a new paradigm for sequential decision making, called offline policy learning from observations (PLfO).
Adversarial Model for Offline Reinforcement Learning
We propose a novel model-based offline Reinforcement Learning (RL) framework, called Adversarial Model for Offline Reinforcement Learning (ARMOR), which can robustly learn policies to improve upon an arbitrary reference policy regardless of data coverage.
Learning Sampling Distributions for Model Predictive Control
This requires us to rely on a number of heuristics for generating samples and updating the distribution and may lead to sub-optimal performance.
Learning to Optimize in Model Predictive Control
We show that we can contend with this noise by learning how to update the control distribution more effectively and make better use of the few samples that we have.
Motion Policy Networks
Collision-free motion generation in unknown environments is a core building block for robot manipulation.
Neural Contact Fields: Tracking Extrinsic Contact with Tactile Sensing
In experiments, we find that Neural Contact Fields are able to localize multiple contact patches without making any assumptions about the geometry of the contact, and capture contact/no-contact transitions for known categories of objects with unseen shapes in unseen environment configurations.
Learning Semantics-Aware Locomotion Skills from Human Demonstration
Using only 40 minutes of human demonstration data, our framework learns to adjust the speed and gait of the robot based on perceived terrain semantics, and enables the robot to walk over 6km without failure at close-to-optimal speed.
CAFA: Class-Aware Feature Alignment for Test-Time Adaptation
That is, a model does not have a chance to learn test data in a class-discriminative manner, which was feasible in other adaptation tasks (\textit{e. g.,} unsupervised domain adaptation) via supervised losses on the source data.
Learning Implicit Priors for Motion Optimization
In this paper, we focus on the problem of integrating Energy-based Models (EBM) as guiding priors for motion optimization.
Motivating Physical Activity via Competitive Human-Robot Interaction
This project aims to motivate research in competitive human-robot interaction by creating a robot competitor that can challenge human users in certain scenarios such as physical exercise and games.
Multi-agent Reinforcement Learning
Reinforcement Learning (RL)
Nonprehensile Riemannian Motion Predictive Control
Nonprehensile manipulation involves long horizon underactuated object interactions and physical contact with different objects that can inherently introduce a high degree of uncertainty.
Leveraging Experience in Lazy Search
If new search problems are sufficiently similar to problems solved during training, the learned policy will choose a good edge evaluation ordering and solve the motion planning problem quickly.
Safe Reinforcement Learning Using Advantage-Based Intervention
We propose a new algorithm, SAILR, that uses an intervention mechanism based on advantage functions to keep the agent safe throughout training and optimizes the agent's policy using off-the-shelf RL algorithms designed for unconstrained MDPs.
Imitation Learning via Simultaneous Optimization of Policies and Auxiliary Trajectories
Many IL methods, such as Dataset Aggregation (DAgger), combat challenges like distributional shift by interacting with oracular experts.
Fast and Efficient Locomotion via Learned Gait Transitions
We focus on the problem of developing energy efficient controllers for quadrupedal robots.
The Value of Planning for Infinite-Horizon Model Predictive Control
We show that that this value function can be used by MPC directly, resulting in more efficient and resilient behavior at runtime.
Few-shot Weakly-Supervised Object Detection via Directional Statistics
When the novel objects are localized, we utilize them to learn a linear appearance model to detect novel classes in new images.
RMP2: A Structured Composable Policy Class for Robot Learning
Using RMPflow as a structured policy class in learning has several benefits, such as sufficient expressiveness, the flexibility to inject different levels of prior knowledge as well as the ability to transfer policies between robots.
Combining pretrained CNN feature extractors to enhance clustering of complex natural images
First, extensive experiments are conducted and show that, for a given dataset, the choice of the CNN architecture for feature extraction has a huge impact on the final clustering.
Towards Coordinated Robot Motions: End-to-End Learning of Motion Policies on Transform Trees
The policy structure provides the user an interface to 1) specifying the spaces that are directly relevant to the completion of the tasks, and 2) designing policies for certain tasks that do not need to be learned.
Blending MPC & Value Function Approximation for Efficient Reinforcement Learning
We further propose an algorithm that changes $\lambda$ over time to reduce the dependence on MPC as our estimates of the value function improve, and test the efficacy our approach on challenging high-dimensional manipulation tasks with biased models in simulation.
Stein Variational Model Predictive Control
By casting MPC as a Bayesian inference problem, we employ variational methods for posterior computation, naturally encoding the complexity and multi-modality of the decision making problem.
Grasping with Chopsticks: Combating Covariate Shift in Model-free Imitation Learning for Fine Manipulation
Billions of people use chopsticks, a simple yet versatile tool, for fine manipulation of everyday objects.
Quantum Tensor Networks, Stochastic Processes, and Weighted Automata
We address this gap by showing how stationary or uniform versions of popular quantum tensor network models have equivalent representations in the stochastic processes and weighted automata literature, in the limit of infinitely long sequences.
Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion
We present a hierarchical framework that combines model-based control and reinforcement learning (RL) to synthesize robust controllers for a quadruped (the Unitree Laikago).
Explaining Fast Improvement in Online Imitation Learning
Here policies are optimized by performing online learning on a sequence of loss functions that encourage the learner to mimic expert actions, and if the online learning has no regret, the agent can provably learn an expert-like policy.
Euclideanizing Flows: Diffeomorphic Reduction for Learning Stable Dynamical Systems
The complex motions are encoded as rollouts of a stable dynamical system, which, under a change of coordinates defined by a diffeomorphism, is equivalent to a simple, hand-specified dynamical system.
Pairwise Similarity Knowledge Transfer for Weakly Supervised Object Localization
Weakly Supervised Object Localization (WSOL) methods only require image level labels as opposed to expensive bounding box annotations required by fully supervised algorithms.
Intra Order-preserving Functions for Calibration of Multi-Class Neural Networks
A common approach is to learn a post-hoc calibration function that transforms the output of the original network into calibrated confidence scores while maintaining the network's accuracy.
Information Theoretic Model Predictive Q-Learning
Model-free Reinforcement Learning (RL) works well when experience can be collected cheaply and model-based RL is effective when system dynamics can be modeled accurately.
Continuous Online Learning and New Insights to Online Imitation Learning
We prove that there is a fundamental equivalence between achieving sublinear dynamic regret in COL and solving certain EPs, and we present a reduction from dynamic regret to both static regret and convergence rate of the associated EP.
Expressiveness and Learning of Hidden Quantum Markov Models
Extending classical probabilistic reasoning using the quantum mechanical view of probability has been of recent interest, particularly in the development of hidden quantum Markov models (HQMMs) to model stochastic processes.
A Reduction from Reinforcement Learning to No-Regret Online Learning
We present a reduction from reinforcement learning (RL) to no-regret online learning based on the saddle-point formulation of RL, by which "any" online algorithm with sublinear regret can generate policies with provable performance guarantees.
IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Data
For simple short-horizon manipulation tasks with modest variation in task instances, offline learning from a small set of demonstrations can produce controllers that successfully solve the task.
Riemannian Motion Policy Fusion through Learnable Lyapunov Function Reshaping
RMPfusion supplements RMPflow with weight functions that can hierarchically reshape the Lyapunov functions of the subtask RMPs according to the current configuration of the robot and environment.
Trajectory-wise Control Variates for Variance Reduction in Policy Gradient Methods
This can be attributed, at least in part, to the high variance in estimating the gradient of the task objective with Monte Carlo methods.
Leveraging Experience in Lazy Search
If new search problems are sufficiently similar to problems solved during training, the learned policy will choose a good edge evaluation ordering and solve the motion planning problem quickly.
Provably Efficient Imitation Learning from Observation Alone
We design a new model-free algorithm for ILFO, Forward Adversarial Imitation Learning (FAIL), which learns a sequence of time-dependent policies by minimizing an Integral Probability Metric between the observation distributions of the expert policy and the learner.
Composing Task-Agnostic Policies with Deep Reinforcement Learning
The composition of elementary behaviors to solve challenging transfer learning problems is one of the key elements in building intelligent machines.
Learning to Find Common Objects Across Few Image Collections
We model the selection as an energy minimization problem with unary and pairwise potential functions.
Learning Quantum Graphical Models using Constrained Gradient Descent on the Stiefel Manifold
Quantum graphical models (QGMs) extend the classical framework for reasoning about uncertainty by incorporating the quantum mechanical view of probability.
An Online Learning Approach to Model Predictive Control
In this paper, we show that there exists a close connection between MPC and online learning, an abstract theoretical framework for analyzing online decision making in the optimization literature.
Online Learning with Continuous Variations: Dynamic Regret and Reductions
Furthermore, we show for COL a reduction from dynamic regret to both static regret and convergence in the associated EP, allowing us to analyze the dynamic regret of many existing algorithms.
Multi-Objective Policy Generation for Multi-Robot Systems Using Riemannian Motion Policies
We propose a collection of RMPs for simple multi-robot tasks that can be used for building controllers for more complicated tasks.
Robotics
RMPflow: A Computational Graph for Automatic Motion Policy Generation
We develop a novel policy synthesis algorithm, RMPflow, based on geometrically consistent transformations of Riemannian Motion Policies (RMPs).
Robotics Systems and Control
Differentiable MPC for End-to-end Planning and Control
We present foundations for using Model Predictive Control (MPC) as a differentiable policy class for reinforcement learning in continuous state and action spaces.
Learning and Inference in Hilbert Space with Quantum Graphical Models
Unlike classical graphical models, QGMs represent uncertainty with density matrices in complex Hilbert spaces.
Truncated Back-propagation for Bilevel Optimization
Bilevel optimization has been recently revisited for designing and analyzing algorithms in hyperparameter tuning and meta learning tasks.
Predictor-Corrector Policy Optimization
We present a predictor-corrector framework, called PicCoLO, that can transform a first-order model-free reinforcement or imitation learning algorithm into a new hybrid method that leverages predictive models to accelerate policy learning.
Orthogonally Decoupled Variational Gaussian Processes
It adopts an orthogonal basis in the mean function to model the residues that cannot be learned by the standard coupled approach.
Adversarial Imitation via Variational Inverse Reinforcement Learning
We consider a problem of learning the reward and policy from expert examples under unknown dynamics.
Learning to Align Images using Weak Geometric Supervision
Such descriptors are often derived using supervised learning on existing datasets with ground truth correspondences.
Semantically Meaningful View Selection
Although deep learning has facilitated progress in image understanding, a robot's performance in problems like object recognition often depends on the angle from which the object is observed.
Improving Image Clustering With Multiple Pretrained CNN Feature Extractors
For many image clustering problems, replacing raw image data with features extracted by a pretrained convolutional neural network (CNN), leads to better clustering performance.
Accelerating Imitation Learning with Predictive Models
When the model oracle is learned online, these algorithms can provably accelerate the best known convergence rate up to an order.
Truncated Horizon Policy Search: Combining Reinforcement Learning & Imitation Learning
In this paper, we propose to combine imitation and reinforcement learning via the idea of reward shaping using an oracle.
Dual Policy Iteration
Recently, a novel class of Approximate Policy Iteration (API) algorithms have demonstrated impressive practical performance (e. g., ExIt from [2], AlphaGo-Zero from [27]).
Fast Policy Learning through Imitation and Reinforcement
We show that if the switching time is properly randomized, LOKI can learn to outperform a suboptimal expert and converge faster than running policy gradient from scratch.
Convergence of Value Aggregation for Imitation Learning
Value aggregation is a general framework for solving imitation learning problems.
Initialization matters: Orthogonal Predictive State Recurrent Neural Networks
In this paper, we extend the theory of ORFs to Kernel Ridge Regression and show that ORFs can be used to obtain Orthogonal PSRNNs (OPSRNNs), which are smaller and faster than PSRNNs.
Variational Inference for Gaussian Process Models with Linear Complexity
Furthermore, it yields a variational inference problem that can be solved by stochastic gradient ascent with time and space complexity that is only linear in the number of mean function parameters, regardless of the choice of kernels, likelihoods, and inducing points.
Deep Forward and Inverse Perceptual Models for Tracking and Prediction
We consider the problems of learning forward models that map state to high-dimensional images and inverse models that map high-dimensional images to state in robotics.
Learning Hidden Quantum Markov Models
We extend previous work on HQMMs with three contributions: (1) we show how classical hidden Markov models (HMMs) can be simulated on a quantum circuit, (2) we reformulate HQMMs by relaxing the constraints for modeling HMMs on quantum circuits, and (3) we present a learning algorithm to estimate the parameters of an HQMM from data.
Manifold Regularization for Kernelized LSTD
Policy evaluation or value function or Q-function approximation is a key procedure in reinforcement learning (RL).
Predictive-State Decoders: Encoding the Future into Recurrent Networks
We seek to combine the advantages of RNNs and PSRs by augmenting existing state-of-the-art recurrent neural networks with Predictive-State Decoders (PSDs), which add supervision to the network's internal state representation to target predicting future observations.
Imitation Learning for Agile Autonomous Driving
We present an end-to-end imitation learning system for agile, off-road autonomous driving using only low-cost on-board sensors.
Robotics
One-Shot Learning for Semantic Segmentation
Low-shot learning methods for image classification support learning from sparse data.
Prediction under Uncertainty in Sparse Spectrum Gaussian Processes with Applications to Filtering and Control
Sparse Spectrum Gaussian Processes (SSGPs) are a powerful tool for scaling Gaussian processes (GPs) to large datasets.
Continuous-Time Gaussian Process Motion Planning via Probabilistic Inference
We benchmark our algorithms against several sampling-based and trajectory optimization-based motion planning algorithms on planning problems in multiple environments.
Robotics
Predictive State Recurrent Neural Networks
We present a new model, Predictive State Recurrent Neural Networks (PSRNNs), for filtering and prediction in dynamical systems.
Sparse Gaussian Processes for Continuous-Time Trajectory Estimation on Matrix Lie Groups
Continuous-time trajectory representations are a powerful tool that can be used to address several issues in many practical simultaneous localization and mapping (SLAM) scenarios, like continuously collected measurements distorted by robot motion, or during with asynchronous sensor measurements.
Robotics
Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Prediction
We demonstrate that AggreVaTeD --- a policy gradient extension of the Imitation Learning (IL) approach of (Ross & Bagnell, 2014) --- can leverage such an oracle to achieve faster and better solutions with less training data than a less-informed Reinforcement Learning (RL) technique.
Incremental Variational Sparse Gaussian Process Regression
Recent work on scaling up Gaussian process regression (GPR) to large datasets has primarily focused on sparse GPR, which leverages a small set of basis functions to approximate the full Gaussian process during inference.
4D Crop Monitoring: Spatio-Temporal Reconstruction for Agriculture
Autonomous crop monitoring at high spatial and temporal resolution is a critical problem in precision agriculture.
Adaptive Probabilistic Trajectory Optimization via Efficient Approximate Inference
Robotic systems must be able to quickly and robustly make decisions when operating in uncertain and dynamic environments.
Learning from Conditional Distributions via Dual Embeddings
In such problems, each sample $x$ itself is associated with a conditional distribution $p(z|x)$ represented by samples $\{z_i\}_{i=1}^M$, and the goal is to learn a function $f$ that links these conditional distributions to target values $y$.
Learning to Filter with Predictive State Inference Machines
Latent state space models are a fundamental and widely used tool for modeling dynamical systems.
Hilbert Space Embeddings of Predictive State Representations
The essence is to represent the state as a nonparametric conditional embedding operator in a Reproducing Kernel Hilbert Space (RKHS) and leverage recent work in kernel methods to estimate, predict, and update the representation.
Predictive State Temporal Difference Learning
We propose a new approach to value function approximation which combines linear temporal difference reinforcement learning with subspace identification.
Reduced-Rank Hidden Markov Models
We introduce the Reduced-Rank Hidden Markov Model (RR-HMM), a generalization of HMMs that can model smooth state evolution as in Linear Dynamical Systems (LDSs) as well as non-log-concave predictive distributions as in continuous-observation HMMs.
