Search Results for author:
Found 85 papers, 24 papers with code
Reduced-Rank Hidden Markov Models
We introduce the Reduced-Rank Hidden Markov Model (RR-HMM), a generalization of HMMs that can model smooth state evolution as in Linear Dynamical Systems (LDSs) as well as non-log-concave predictive distributions as in continuous-observation HMMs.
Predictive State Temporal Difference Learning
We propose a new approach to value function approximation which combines linear temporal difference reinforcement learning with subspace identification.
Hilbert Space Embeddings of Predictive State Representations
The essence is to represent the state as a nonparametric conditional embedding operator in a Reproducing Kernel Hilbert Space (RKHS) and leverage recent work in kernel methods to estimate, predict, and update the representation.
Learning to Filter with Predictive State Inference Machines
Latent state space models are a fundamental and widely used tool for modeling dynamical systems.
Learning from Conditional Distributions via Dual Embeddings
In such problems, each sample $x$ itself is associated with a conditional distribution $p(z|x)$ represented by samples $\{z_i\}_{i=1}^M$, and the goal is to learn a function $f$ that links these conditional distributions to target values $y$.
Adaptive Probabilistic Trajectory Optimization via Efficient Approximate Inference
Robotic systems must be able to quickly and robustly make decisions when operating in uncertain and dynamic environments.
4D Crop Monitoring: Spatio-Temporal Reconstruction for Agriculture
Autonomous crop monitoring at high spatial and temporal resolution is a critical problem in precision agriculture.
Incremental Variational Sparse Gaussian Process Regression
Recent work on scaling up Gaussian process regression (GPR) to large datasets has primarily focused on sparse GPR, which leverages a small set of basis functions to approximate the full Gaussian process during inference.
Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Prediction
We demonstrate that AggreVaTeD --- a policy gradient extension of the Imitation Learning (IL) approach of (Ross & Bagnell, 2014) --- can leverage such an oracle to achieve faster and better solutions with less training data than a less-informed Reinforcement Learning (RL) technique.
Sparse Gaussian Processes for Continuous-Time Trajectory Estimation on Matrix Lie Groups
Continuous-time trajectory representations are a powerful tool that can be used to address several issues in many practical simultaneous localization and mapping (SLAM) scenarios, like continuously collected measurements distorted by robot motion, or during with asynchronous sensor measurements.
Robotics
Predictive State Recurrent Neural Networks
We present a new model, Predictive State Recurrent Neural Networks (PSRNNs), for filtering and prediction in dynamical systems.
Continuous-Time Gaussian Process Motion Planning via Probabilistic Inference
We benchmark our algorithms against several sampling-based and trajectory optimization-based motion planning algorithms on planning problems in multiple environments.
Robotics
Prediction under Uncertainty in Sparse Spectrum Gaussian Processes with Applications to Filtering and Control
Sparse Spectrum Gaussian Processes (SSGPs) are a powerful tool for scaling Gaussian processes (GPs) to large datasets.
One-Shot Learning for Semantic Segmentation
Low-shot learning methods for image classification support learning from sparse data.
Imitation Learning for Agile Autonomous Driving
We present an end-to-end imitation learning system for agile, off-road autonomous driving using only low-cost on-board sensors.
Robotics
Predictive-State Decoders: Encoding the Future into Recurrent Networks
We seek to combine the advantages of RNNs and PSRs by augmenting existing state-of-the-art recurrent neural networks with Predictive-State Decoders (PSDs), which add supervision to the network's internal state representation to target predicting future observations.
Manifold Regularization for Kernelized LSTD
Policy evaluation or value function or Q-function approximation is a key procedure in reinforcement learning (RL).
Learning Hidden Quantum Markov Models
We extend previous work on HQMMs with three contributions: (1) we show how classical hidden Markov models (HMMs) can be simulated on a quantum circuit, (2) we reformulate HQMMs by relaxing the constraints for modeling HMMs on quantum circuits, and (3) we present a learning algorithm to estimate the parameters of an HQMM from data.
Deep Forward and Inverse Perceptual Models for Tracking and Prediction
We consider the problems of learning forward models that map state to high-dimensional images and inverse models that map high-dimensional images to state in robotics.
Variational Inference for Gaussian Process Models with Linear Complexity
Furthermore, it yields a variational inference problem that can be solved by stochastic gradient ascent with time and space complexity that is only linear in the number of mean function parameters, regardless of the choice of kernels, likelihoods, and inducing points.
Initialization matters: Orthogonal Predictive State Recurrent Neural Networks
In this paper, we extend the theory of ORFs to Kernel Ridge Regression and show that ORFs can be used to obtain Orthogonal PSRNNs (OPSRNNs), which are smaller and faster than PSRNNs.
Convergence of Value Aggregation for Imitation Learning
Value aggregation is a general framework for solving imitation learning problems.
Fast Policy Learning through Imitation and Reinforcement
We show that if the switching time is properly randomized, LOKI can learn to outperform a suboptimal expert and converge faster than running policy gradient from scratch.
Dual Policy Iteration
Recently, a novel class of Approximate Policy Iteration (API) algorithms have demonstrated impressive practical performance (e. g., ExIt from [2], AlphaGo-Zero from [27]).
Truncated Horizon Policy Search: Combining Reinforcement Learning & Imitation Learning
In this paper, we propose to combine imitation and reinforcement learning via the idea of reward shaping using an oracle.
Accelerating Imitation Learning with Predictive Models
When the model oracle is learned online, these algorithms can provably accelerate the best known convergence rate up to an order.
Improving Image Clustering With Multiple Pretrained CNN Feature Extractors
For many image clustering problems, replacing raw image data with features extracted by a pretrained convolutional neural network (CNN), leads to better clustering performance.
Semantically Meaningful View Selection
Although deep learning has facilitated progress in image understanding, a robot's performance in problems like object recognition often depends on the angle from which the object is observed.
Learning to Align Images using Weak Geometric Supervision
Such descriptors are often derived using supervised learning on existing datasets with ground truth correspondences.
Adversarial Imitation via Variational Inverse Reinforcement Learning
We consider a problem of learning the reward and policy from expert examples under unknown dynamics.
Orthogonally Decoupled Variational Gaussian Processes
It adopts an orthogonal basis in the mean function to model the residues that cannot be learned by the standard coupled approach.
Predictor-Corrector Policy Optimization
We present a predictor-corrector framework, called PicCoLO, that can transform a first-order model-free reinforcement or imitation learning algorithm into a new hybrid method that leverages predictive models to accelerate policy learning.
Truncated Back-propagation for Bilevel Optimization
Bilevel optimization has been recently revisited for designing and analyzing algorithms in hyperparameter tuning and meta learning tasks.
Learning and Inference in Hilbert Space with Quantum Graphical Models
Unlike classical graphical models, QGMs represent uncertainty with density matrices in complex Hilbert spaces.
Differentiable MPC for End-to-end Planning and Control
We present foundations for using Model Predictive Control (MPC) as a differentiable policy class for reinforcement learning in continuous state and action spaces.
RMPflow: A Computational Graph for Automatic Motion Policy Generation
We develop a novel policy synthesis algorithm, RMPflow, based on geometrically consistent transformations of Riemannian Motion Policies (RMPs).
Robotics Systems and Control
Multi-Objective Policy Generation for Multi-Robot Systems Using Riemannian Motion Policies
We propose a collection of RMPs for simple multi-robot tasks that can be used for building controllers for more complicated tasks.
Robotics
Online Learning with Continuous Variations: Dynamic Regret and Reductions
Furthermore, we show for COL a reduction from dynamic regret to both static regret and convergence in the associated EP, allowing us to analyze the dynamic regret of many existing algorithms.
An Online Learning Approach to Model Predictive Control
In this paper, we show that there exists a close connection between MPC and online learning, an abstract theoretical framework for analyzing online decision making in the optimization literature.
Learning Quantum Graphical Models using Constrained Gradient Descent on the Stiefel Manifold
Quantum graphical models (QGMs) extend the classical framework for reasoning about uncertainty by incorporating the quantum mechanical view of probability.
Learning to Find Common Objects Across Few Image Collections
We model the selection as an energy minimization problem with unary and pairwise potential functions.
Composing Task-Agnostic Policies with Deep Reinforcement Learning
The composition of elementary behaviors to solve challenging transfer learning problems is one of the key elements in building intelligent machines.
Provably Efficient Imitation Learning from Observation Alone
We design a new model-free algorithm for ILFO, Forward Adversarial Imitation Learning (FAIL), which learns a sequence of time-dependent policies by minimizing an Integral Probability Metric between the observation distributions of the expert policy and the learner.
Leveraging Experience in Lazy Search
If new search problems are sufficiently similar to problems solved during training, the learned policy will choose a good edge evaluation ordering and solve the motion planning problem quickly.
Trajectory-wise Control Variates for Variance Reduction in Policy Gradient Methods
This can be attributed, at least in part, to the high variance in estimating the gradient of the task objective with Monte Carlo methods.
Riemannian Motion Policy Fusion through Learnable Lyapunov Function Reshaping
RMPfusion supplements RMPflow with weight functions that can hierarchically reshape the Lyapunov functions of the subtask RMPs according to the current configuration of the robot and environment.
IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Data
For simple short-horizon manipulation tasks with modest variation in task instances, offline learning from a small set of demonstrations can produce controllers that successfully solve the task.
A Reduction from Reinforcement Learning to No-Regret Online Learning
We present a reduction from reinforcement learning (RL) to no-regret online learning based on the saddle-point formulation of RL, by which "any" online algorithm with sublinear regret can generate policies with provable performance guarantees.
Expressiveness and Learning of Hidden Quantum Markov Models
Extending classical probabilistic reasoning using the quantum mechanical view of probability has been of recent interest, particularly in the development of hidden quantum Markov models (HQMMs) to model stochastic processes.
Continuous Online Learning and New Insights to Online Imitation Learning
We prove that there is a fundamental equivalence between achieving sublinear dynamic regret in COL and solving certain EPs, and we present a reduction from dynamic regret to both static regret and convergence rate of the associated EP.
Information Theoretic Model Predictive Q-Learning
Model-free Reinforcement Learning (RL) works well when experience can be collected cheaply and model-based RL is effective when system dynamics can be modeled accurately.
Intra Order-preserving Functions for Calibration of Multi-Class Neural Networks
A common approach is to learn a post-hoc calibration function that transforms the output of the original network into calibrated confidence scores while maintaining the network's accuracy.
Pairwise Similarity Knowledge Transfer for Weakly Supervised Object Localization
Weakly Supervised Object Localization (WSOL) methods only require image level labels as opposed to expensive bounding box annotations required by fully supervised algorithms.
Euclideanizing Flows: Diffeomorphic Reduction for Learning Stable Dynamical Systems
The complex motions are encoded as rollouts of a stable dynamical system, which, under a change of coordinates defined by a diffeomorphism, is equivalent to a simple, hand-specified dynamical system.
Explaining Fast Improvement in Online Imitation Learning
Here policies are optimized by performing online learning on a sequence of loss functions that encourage the learner to mimic expert actions, and if the online learning has no regret, the agent can provably learn an expert-like policy.
Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion
We present a hierarchical framework that combines model-based control and reinforcement learning (RL) to synthesize robust controllers for a quadruped (the Unitree Laikago).
Quantum Tensor Networks, Stochastic Processes, and Weighted Automata
We address this gap by showing how stationary or uniform versions of popular quantum tensor network models have equivalent representations in the stochastic processes and weighted automata literature, in the limit of infinitely long sequences.
Grasping with Chopsticks: Combating Covariate Shift in Model-free Imitation Learning for Fine Manipulation
Billions of people use chopsticks, a simple yet versatile tool, for fine manipulation of everyday objects.
Stein Variational Model Predictive Control
By casting MPC as a Bayesian inference problem, we employ variational methods for posterior computation, naturally encoding the complexity and multi-modality of the decision making problem.
Blending MPC & Value Function Approximation for Efficient Reinforcement Learning
We further propose an algorithm that changes $\lambda$ over time to reduce the dependence on MPC as our estimates of the value function improve, and test the efficacy our approach on challenging high-dimensional manipulation tasks with biased models in simulation.
Towards Coordinated Robot Motions: End-to-End Learning of Motion Policies on Transform Trees
The policy structure provides the user an interface to 1) specifying the spaces that are directly relevant to the completion of the tasks, and 2) designing policies for certain tasks that do not need to be learned.
Combining pretrained CNN feature extractors to enhance clustering of complex natural images
First, extensive experiments are conducted and show that, for a given dataset, the choice of the CNN architecture for feature extraction has a huge impact on the final clustering.
RMP2: A Structured Composable Policy Class for Robot Learning
Using RMPflow as a structured policy class in learning has several benefits, such as sufficient expressiveness, the flexibility to inject different levels of prior knowledge as well as the ability to transfer policies between robots.
Few-shot Weakly-Supervised Object Detection via Directional Statistics
When the novel objects are localized, we utilize them to learn a linear appearance model to detect novel classes in new images.
The Value of Planning for Infinite-Horizon Model Predictive Control
We show that that this value function can be used by MPC directly, resulting in more efficient and resilient behavior at runtime.
Fast and Efficient Locomotion via Learned Gait Transitions
We focus on the problem of developing energy efficient controllers for quadrupedal robots.
Imitation Learning via Simultaneous Optimization of Policies and Auxiliary Trajectories
Many IL methods, such as Dataset Aggregation (DAgger), combat challenges like distributional shift by interacting with oracular experts.
Safe Reinforcement Learning Using Advantage-Based Intervention
We propose a new algorithm, SAILR, that uses an intervention mechanism based on advantage functions to keep the agent safe throughout training and optimizes the agent's policy using off-the-shelf RL algorithms designed for unconstrained MDPs.
Leveraging Experience in Lazy Search
If new search problems are sufficiently similar to problems solved during training, the learned policy will choose a good edge evaluation ordering and solve the motion planning problem quickly.
Nonprehensile Riemannian Motion Predictive Control
Nonprehensile manipulation involves long horizon underactuated object interactions and physical contact with different objects that can inherently introduce a high degree of uncertainty.
Motivating Physical Activity via Competitive Human-Robot Interaction
This project aims to motivate research in competitive human-robot interaction by creating a robot competitor that can challenge human users in certain scenarios such as physical exercise and games.
Multi-agent Reinforcement Learning
Reinforcement Learning (RL)
Learning Implicit Priors for Motion Optimization
In this paper, we focus on the problem of integrating Energy-based Models (EBM) as guiding priors for motion optimization.
CAFA: Class-Aware Feature Alignment for Test-Time Adaptation
That is, a model does not have a chance to learn test data in a class-discriminative manner, which was feasible in other adaptation tasks (\textit{e. g.,} unsupervised domain adaptation) via supervised losses on the source data.
Learning Semantics-Aware Locomotion Skills from Human Demonstration
Using only 40 minutes of human demonstration data, our framework learns to adjust the speed and gait of the robot based on perceived terrain semantics, and enables the robot to walk over 6km without failure at close-to-optimal speed.
Neural Contact Fields: Tracking Extrinsic Contact with Tactile Sensing
In experiments, we find that Neural Contact Fields are able to localize multiple contact patches without making any assumptions about the geometry of the contact, and capture contact/no-contact transitions for known categories of objects with unseen shapes in unseen environment configurations.
Motion Policy Networks
Collision-free motion generation in unknown environments is a core building block for robot manipulation.
Learning to Optimize in Model Predictive Control
We show that we can contend with this noise by learning how to update the control distribution more effectively and make better use of the few samples that we have.
Learning Sampling Distributions for Model Predictive Control
This requires us to rely on a number of heuristics for generating samples and updating the distribution and may lead to sub-optimal performance.
Adversarial Model for Offline Reinforcement Learning
We propose a novel model-based offline Reinforcement Learning (RL) framework, called Adversarial Model for Offline Reinforcement Learning (ARMOR), which can robustly learn policies to improve upon an arbitrary reference policy regardless of data coverage.
MAHALO: Unifying Offline Reinforcement Learning and Imitation Learning from Observations
We study a new paradigm for sequential decision making, called offline policy learning from observations (PLfO).
Continuous Versatile Jumping Using Learned Action Residuals
Jumping is essential for legged robots to traverse through difficult terrains.
Stackelberg Games for Learning Emergent Behaviors During Competitive Autocurricula
Autocurricular training is an important sub-area of multi-agent reinforcement learning~(MARL) that allows multiple agents to learn emergent skills in an unsupervised co-evolving scheme.
LiDAR-UDA: Self-ensembling Through Time for Unsupervised LiDAR Domain Adaptation
Existing self-training methods use a model trained on labeled source data to generate pseudo labels for target data and refine the predictions via fine-tuning the network on the pseudo labels.
Deep Model Predictive Optimization
A major challenge in robotics is to design robust policies which enable complex and agile behaviors in the real world.
DATT: Deep Adaptive Trajectory Tracking for Quadrotor Control
Precise arbitrary trajectory tracking for quadrotors is challenging due to unknown nonlinear dynamics, trajectory infeasibility, and actuation limits.
