Search Results for author:
Found 47 papers, 19 papers with code
All Roads Lead to Likelihood: The Value of Reinforcement Learning in Fine-Tuning
From a first-principles perspective, it may seem odd that the strongest results in foundation model fine-tuning (FT) are achieved via a relatively complex, two-stage training procedure.
Multi-Turn Code Generation Through Single-Step Rewards
We provide analysis of the design choices of the reward models and policy, and show the efficacy of $\mu$Code at utilizing the execution feedback.
Process Reward Models for LLM Agents: Practical Framework and Directions
We introduce Agent Process Reward Models (AgentPRM), a simple and scalable framework for training LLM agents to continually improve through interactions.
Imitation Learning from a Single Temporally Misaligned Video
Existing approaches treat imitation as a distribution-matching problem, aligning individual frames between the agent and the demonstration.
Robotouille: An Asynchronous Planning Benchmark for LLM Agents
Effective asynchronous planning, or the ability to efficiently reason and plan over states and actions that must happen in parallel or sequentially, is essential for agents that must account for time delays, reason over diverse long-horizon tasks, and collaborate with other agents.
Motion Tracks: A Unified Representation for Human-Robot Transfer in Few-Shot Imitation Learning
We instantiate an IL policy called Motion Track Policy (MT-pi) which receives image observations and outputs motion tracks as actions.
Aligning LLMs with Domain Invariant Reward Models
Aligning large language models (LLMs) to human preferences is challenging in domains where preference data is unavailable.
Query-Efficient Planning with Language Models
The first uses LLMs as a heuristic within a search-based planner to select promising nodes to expand and propose promising actions.
Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matching
In inverse reinforcement learning (IRL), an agent seeks to replicate expert demonstrations through interactions with the environment.
Better than Your Teacher: LLM Agents that learn from Privileged AI Feedback
Our key insight is to equip the expert teachers with a privileged state -- information that is available during training but hidden at test time.
One-Shot Imitation under Mismatched Execution
Human demonstrations as prompts are a powerful way to program robots to do long-horizon manipulation tasks.
Learning to Move Like Professional Counter-Strike Players
We curate a team movement dataset comprising 123 hours of professional game play traces, and use this dataset to train a transformer-based movement model that generates human-like team movement for all players in a "Retakes" round of the game.
Hybrid Inverse Reinforcement Learning
In this work, we propose using hybrid RL -- training on a mixture of online and expert data -- to curtail unnecessary exploration.
The Virtues of Pessimism in Inverse Reinforcement Learning
Inverse Reinforcement Learning (IRL) is a powerful framework for learning complex behaviors from expert demonstrations.
Accelerating Inverse Reinforcement Learning with Expert Bootstrapping
Existing inverse reinforcement learning methods (e. g. MaxEntIRL, $f$-IRL) search over candidate reward functions and solve a reinforcement learning problem in the inner loop.
InteRACT: Transformer Models for Human Intent Prediction Conditioned on Robot Actions
In collaborative human-robot manipulation, a robot must predict human intents and adapt its actions accordingly to smoothly execute tasks.
UNcommonsense Reasoning: Abductive Reasoning about Uncommon Situations
To instead investigate the ability to model unusual, unexpected, and unlikely situations, we explore the task of uncommonsense abductive reasoning.
ManiCast: Collaborative Manipulation with Cost-Aware Human Forecasting
Seamless human-robot manipulation in close proximity relies on accurate forecasts of human motion.
Learning Shared Safety Constraints from Multi-task Demonstrations
Regardless of the particular task we want them to perform in an environment, there are often shared safety constraints we want our agents to respect.
A Game-Theoretic Framework for Joint Forecasting and Planning
On the other hand, planning for worst-case motions leads to overtly conservative behavior and a "frozen robot".
Inverse Reinforcement Learning without Reinforcement Learning
In this work, we demonstrate for the first time a more informed imitation learning reduction where we utilize the state distribution of the expert to alleviate the global exploration component of the RL subroutine, providing an exponential speedup in theory.
The Virtues of Laziness in Model-based RL: A Unified Objective and Algorithms
We propose a novel approach to addressing two fundamental challenges in Model-based Reinforcement Learning (MBRL): the computational expense of repeatedly finding a good policy in the learned model, and the objective mismatch between model fitting and policy computation.
Game-Theoretic Algorithms for Conditional Moment Matching
A variety of problems in econometrics and machine learning, including instrumental variable regression and Bellman residual minimization, can be formulated as satisfying a set of conditional moment restrictions (CMR).
Sequence Model Imitation Learning with Unobserved Contexts
We consider imitation learning problems where the learner's ability to mimic the expert increases throughout the course of an episode as more information is revealed.
Minimax Optimal Online Imitation Learning via Replay Estimation
In the tabular setting or with linear function approximation, our meta theorem shows that the performance gap incurred by our approach achieves the optimal $\widetilde{O} \left( \min({H^{3/2}} / {N}, {H} / {\sqrt{N}} \right)$ dependency, under significantly weaker assumptions compared to prior work.
Causal Imitation Learning under Temporally Correlated Noise
We develop algorithms for imitation learning from policy data that was corrupted by temporally correlated noise in expert actions.
Leveraging Experience in Lazy Search
If new search problems are sufficiently similar to problems solved during training, the learned policy will choose a good edge evaluation ordering and solve the motion planning problem quickly.
A Critique of Strictly Batch Imitation Learning
Recent work by Jarrett et al. attempts to frame the problem of offline imitation learning (IL) as one of learning a joint energy-based model, with the hope of out-performing standard behavioral cloning.
What Would the Expert $do(\cdot)$?: Causal Imitation Learning
Both approaches are able to find policies that match the result of a query to an unconfounded expert.
Learning Online from Corrective Feedback: A Meta-Algorithm for Robotics
A key challenge in Imitation Learning (IL) is that optimal state actions demonstrations are difficult for the teacher to provide.
Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gap
We provide a unifying view of a large family of previous imitation learning algorithms through the lens of moment matching.
Feedback in Imitation Learning: The Three Regimes of Covariate Shift
The learner often comes to rely on features that are strongly predictive of decisions, but are subject to strong covariate shift.
Blending MPC & Value Function Approximation for Efficient Reinforcement Learning
We further propose an algorithm that changes $\lambda$ over time to reduce the dependence on MPC as our estimates of the value function improve, and test the efficacy our approach on challenging high-dimensional manipulation tasks with biased models in simulation.
Bayesian Residual Policy Optimization: Scalable Bayesian Reinforcement Learning with Clairvoyant Experts
We first obtain an ensemble of experts, one for each latent MDP, and fuse their advice to compute a baseline policy.
Autonomous Aerial Cinematography In Unstructured Environments With Learned Artistic Decision-Making
In this work, we address the problem in its entirety and propose a complete system for real-time aerial cinematography that for the first time combines: (1) vision-based target estimation; (2) 3D signed-distance mapping for occlusion estimation; (3) efficient trajectory optimization for long time-horizon camera motion; and (4) learning-based artistic shot selection.
Leveraging Experience in Lazy Search
If new search problems are sufficiently similar to problems solved during training, the learned policy will choose a good edge evaluation ordering and solve the motion planning problem quickly.
Imitation Learning as $f$-Divergence Minimization
We show that the state-of-the-art methods such as GAIL and behavior cloning, due to their choice of loss function, often incorrectly interpolate between such modes.
Towards a Robust Aerial Cinematography Platform: Localizing and Tracking Moving Targets in Unstructured Environments
In this work, we overcome such limitations and propose a complete system for aerial cinematography that combines: (1) a vision-based algorithm for target localization; (2) a real-time incremental 3D signed-distance map algorithm for occlusion and safety computation; and (3) a real-time camera motion planner that optimizes smoothness, collisions, occlusions and artistic guidelines.
Bayes-CPACE: PAC Optimal Exploration in Continuous Space Bayes-Adaptive Markov Decision Processes
We present the first PAC optimal algorithm for Bayes-Adaptive Markov Decision Processes (BAMDPs) in continuous state and action spaces, to the best of our knowledge.
Bayesian Policy Optimization for Model Uncertainty
Addressing uncertainty is critical for autonomous systems to robustly adapt to the real world.
Autonomous drone cinematographer: Using artistic principles to create smooth, safe, occlusion-free trajectories for aerial filming
Autonomous aerial cinematography has the potential to enable automatic capture of aesthetically pleasing videos without requiring human intervention, empowering individuals with the capability of high-end film studios.
Hindsight is Only 50/50: Unsuitability of MDP based Approximate POMDP Solvers for Multi-resolution Information Gathering
The contribution of the paper helps identify the properties of a POMDP problem for which the use of MDP based POMDP solvers is inappropriate, enabling better design choices.
Bayesian Active Edge Evaluation on Expensive Graphs
We are interested in planning algorithms that actively infer the underlying structure of the valid configuration space during planning in order to find solutions with minimal effort.
Anytime Motion Planning on Large Dense Roadmaps with Expensive Edge Evaluations
We propose an algorithmic framework for efficient anytime motion planning on large dense geometric roadmaps, in domains where collision checks and therefore edge evaluations are computationally expensive.
Robotics
Learning Heuristic Search via Imitation
In this paper, we do so by training a heuristic policy that maps the partial information from the search to decide which node of the search tree to expand.
Near-Optimal Edge Evaluation in Explicit Generalized Binomial Graphs
By leveraging this property, we are able to significantly reduce computational complexity from exponential to linear in the number of edges.
Robotics
Learning to Gather Information via Imitation
The budgeted information gathering problem - where a robot with a fixed fuel budget is required to maximize the amount of information gathered from the world - appears in practice across a wide range of applications in autonomous exploration and inspection with mobile robots.
