Search Results for author:
Found 39 papers, 11 papers with code
Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences
In this paper, we introduce Direct Nash Optimization (DNO), a provable and scalable algorithm that marries the simplicity and stability of contrastive learning with theoretical generality from optimizing general preferences.
PRISE: Learning Temporal Action Abstractions as a Sequence Compression Problem
To do so, we bring a subtle but critical component of LLM training pipelines -- input tokenization via byte pair encoding (BPE) -- to the seemingly distant task of learning skills of variable time span in continuous control domains.
LLF-Bench: Benchmark for Interactive Learning from Language Feedback
We introduce a new benchmark, LLF-Bench (Learning from Language Feedback Benchmark; pronounced as "elf-bench"), to evaluate the ability of AI agents to interactively learn from natural language feedback and instructions.
Interactive Robot Learning from Verbal Correction
A key feature of OLAF is its ability to update the robot's visuomotor neural policy based on the verbal feedback to avoid repeating mistakes in the future.
Goal Representations for Instruction Following: A Semi-Supervised Language Interface to Control
Our method achieves robust performance in the real world by learning an embedding from the labeled data that aligns language not to the goal image, but rather to the desired change between the start and goal images that the instruction corresponds to.
Improving Offline RL by Blending Heuristics
We propose Heuristic Blending (HUBL), a simple performance-improving technique for a broad class of offline RL algorithms based on value bootstrapping.
MAHALO: Unifying Offline Reinforcement Learning and Imitation Learning from Observations
We study a new paradigm for sequential decision making, called offline policy learning from observations (PLfO).
PLEX: Making the Most of the Available Data for Robotic Manipulation Pretraining
A rich representation is key to general robotic manipulation, but existing approaches to representation learning require large amounts of multimodal demonstrations.
Adversarial Model for Offline Reinforcement Learning
We propose a novel model-based offline Reinforcement Learning (RL) framework, called Adversarial Model for Offline Reinforcement Learning (ARMOR), which can robustly learn policies to improve upon an arbitrary reference policy regardless of data coverage.
Provable Reset-free Reinforcement Learning by No-Regret Reduction
Reinforcement learning (RL) so far has limited real-world applications.
ARMOR: A Model-based Framework for Improving Arbitrary Baseline Policies with Offline Data
We propose a new model-based offline RL framework, called Adversarial Models for Offline Reinforcement Learning (ARMOR), which can robustly learn policies to improve upon an arbitrary baseline policy regardless of data coverage.
MoCapAct: A Multi-Task Dataset for Simulated Humanoid Control
We demonstrate the utility of MoCapAct by using it to train a single hierarchical policy capable of tracking the entire MoCap dataset within dm_control and show the learned low-level component can be re-used to efficiently learn downstream high-level tasks.
Hindsight Learning for MDPs with Exogenous Inputs
Many resource management problems require sequential decision-making under uncertainty, where the only uncertainty affecting the decision outcomes are exogenous variables outside the control of the decision-maker.
Provably Efficient Lifelong Reinforcement Learning with Linear Function Approximation
We study lifelong reinforcement learning (RL) in a regret minimization setting of linear contextual Markov decision process (MDP), where the agent needs to learn a multi-task policy while solving a streaming sequence of tasks.
Adversarially Trained Actor Critic for Offline Reinforcement Learning
We propose Adversarially Trained Actor Critic (ATAC), a new model-free algorithm for offline reinforcement learning (RL) under insufficient data coverage, based on the concept of relative pessimism.
Safe Reinforcement Learning Using Advantage-Based Intervention
We propose a new algorithm, SAILR, that uses an intervention mechanism based on advantage functions to keep the agent safe throughout training and optimizes the agent's policy using off-the-shelf RL algorithms designed for unconstrained MDPs.
Bellman-consistent Pessimism for Offline Reinforcement Learning
The use of pessimism, when reasoning about datasets lacking exhaustive exploration has recently gained prominence in offline reinforcement learning.
Heuristic-Guided Reinforcement Learning
On the theoretical side, we characterize properties of a good heuristic and its impact on RL acceleration.
Cautiously Optimistic Policy Optimization and Exploration with Linear Function Approximation
Policy optimization methods are popular reinforcement learning algorithms, because their incremental and on-policy nature makes them more stable than the value-based counterparts.
RMP2: A Structured Composable Policy Class for Robot Learning
Using RMPflow as a structured policy class in learning has several benefits, such as sufficient expressiveness, the flexibility to inject different levels of prior knowledge as well as the ability to transfer policies between robots.
Explaining Fast Improvement in Online Imitation Learning
Here policies are optimized by performing online learning on a sequence of loss functions that encourage the learner to mimic expert actions, and if the online learning has no regret, the agent can provably learn an expert-like policy.
Policy Improvement via Imitation of Multiple Oracles
In this paper, we propose the state-wise maximum of the oracle policies' values as a natural baseline to resolve conflicting advice from multiple oracles.
Intra Order-preserving Functions for Calibration of Multi-Class Neural Networks
A common approach is to learn a post-hoc calibration function that transforms the output of the original network into calibrated confidence scores while maintaining the network's accuracy.
Continuous Online Learning and New Insights to Online Imitation Learning
We prove that there is a fundamental equivalence between achieving sublinear dynamic regret in COL and solving certain EPs, and we present a reduction from dynamic regret to both static regret and convergence rate of the associated EP.
A Reduction from Reinforcement Learning to No-Regret Online Learning
We present a reduction from reinforcement learning (RL) to no-regret online learning based on the saddle-point formulation of RL, by which "any" online algorithm with sublinear regret can generate policies with provable performance guarantees.
Riemannian Motion Policy Fusion through Learnable Lyapunov Function Reshaping
RMPfusion supplements RMPflow with weight functions that can hierarchically reshape the Lyapunov functions of the subtask RMPs according to the current configuration of the robot and environment.
Trajectory-wise Control Variates for Variance Reduction in Policy Gradient Methods
This can be attributed, at least in part, to the high variance in estimating the gradient of the task objective with Monte Carlo methods.
An Online Learning Approach to Model Predictive Control
In this paper, we show that there exists a close connection between MPC and online learning, an abstract theoretical framework for analyzing online decision making in the optimization literature.
Online Learning with Continuous Variations: Dynamic Regret and Reductions
Furthermore, we show for COL a reduction from dynamic regret to both static regret and convergence in the associated EP, allowing us to analyze the dynamic regret of many existing algorithms.
RMPflow: A Computational Graph for Automatic Motion Policy Generation
We develop a novel policy synthesis algorithm, RMPflow, based on geometrically consistent transformations of Riemannian Motion Policies (RMPs).
Robotics Systems and Control
Truncated Back-propagation for Bilevel Optimization
Bilevel optimization has been recently revisited for designing and analyzing algorithms in hyperparameter tuning and meta learning tasks.
Predictor-Corrector Policy Optimization
We present a predictor-corrector framework, called PicCoLO, that can transform a first-order model-free reinforcement or imitation learning algorithm into a new hybrid method that leverages predictive models to accelerate policy learning.
Orthogonally Decoupled Variational Gaussian Processes
It adopts an orthogonal basis in the mean function to model the residues that cannot be learned by the standard coupled approach.
Accelerating Imitation Learning with Predictive Models
When the model oracle is learned online, these algorithms can provably accelerate the best known convergence rate up to an order.
Fast Policy Learning through Imitation and Reinforcement
We show that if the switching time is properly randomized, LOKI can learn to outperform a suboptimal expert and converge faster than running policy gradient from scratch.
Convergence of Value Aggregation for Imitation Learning
Value aggregation is a general framework for solving imitation learning problems.
Variational Inference for Gaussian Process Models with Linear Complexity
Furthermore, it yields a variational inference problem that can be solved by stochastic gradient ascent with time and space complexity that is only linear in the number of mean function parameters, regardless of the choice of kernels, likelihoods, and inducing points.
Imitation Learning for Agile Autonomous Driving
We present an end-to-end imitation learning system for agile, off-road autonomous driving using only low-cost on-board sensors.
Robotics
Incremental Variational Sparse Gaussian Process Regression
Recent work on scaling up Gaussian process regression (GPR) to large datasets has primarily focused on sparse GPR, which leverages a small set of basis functions to approximate the full Gaussian process during inference.
