Search Results for author:
Found 110 papers, 34 papers with code
Behavioral Cloning from Observation
In this work, we propose a two-phase, autonomous imitation learning technique called behavioral cloning from observation (BCO), that aims to provide improved performance with respect to both of these aspects.
Deep Reinforcement Learning in Parameterized Action Space
Recent work has shown that deep neural networks are capable of approximating both value functions and policies in reinforcement learning domains featuring continuous state and action spaces.
Deep Recurrent Q-Learning for Partially Observable MDPs
Deep Reinforcement Learning has yielded proficient controllers for complex tasks.
LLM+P: Empowering Large Language Models with Optimal Planning Proficiency
LLM+P takes in a natural language description of a planning problem, then returns a correct (or optimal) plan for solving that problem in natural language.
Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science
Through the success of deep learning in various domains, artificial neural networks are currently among the most used artificial intelligence methods.
Conflict-Averse Gradient Descent for Multi-task Learning
The goal of multi-task learning is to enable more efficient learning than single task learning by sharing model structures for a diverse set of tasks.
COOPERNAUT: End-to-End Driving with Cooperative Perception for Networked Vehicles
To evaluate our model, we develop AutoCastSim, a network-augmented driving simulation framework with example accident-prone scenarios.
FAMO: Fast Adaptive Multitask Optimization
One of the grand enduring goals of AI is to create generalist agents that can learn multiple different tasks from diverse data via multitask learning (MTL).
Reinforcement Learning for Optimization of COVID-19 Mitigation policies
The year 2020 has seen the COVID-19 virus lead to one of the worst global pandemics in history.
Metric Residual Networks for Sample Efficient Goal-Conditioned Reinforcement Learning
Furthermore, we introduce the metric residual network (MRN) that deliberately decomposes the action-value function Q(s, a, g) into the negated summation of a metric plus a residual asymmetric component.
Generative Adversarial Imitation from Observation
Imitation from observation (IfO) is the problem of learning directly from state-only demonstrations without having access to the demonstrator's actions.
Firefly Neural Architecture Descent: a General Approach for Growing Neural Networks
We propose firefly neural architecture descent, a general framework for progressively and dynamically growing neural networks to jointly optimize the networks' parameters and architectures.
Benchmarking Reinforcement Learning Techniques for Autonomous Navigation
Deep reinforcement learning (RL) has brought many successes for autonomous robot navigation.
Causal Dynamics Learning for Task-Independent State Abstraction
Learning dynamics models accurately is an important goal for Model-Based Reinforcement Learning (MBRL), but most MBRL methods learn a dense dynamics model which is vulnerable to spurious correlations and therefore generalizes poorly to unseen states.
Improving Grounded Natural Language Understanding through Human-Robot Dialog
Natural language understanding for robotics can require substantial domain- and platform-specific engineering.
Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition
Specifically, we 1) adopt the attention mechanism for both the coach and the players; 2) propose a variational objective to regularize learning; and 3) design an adaptive communication method to let the coach decide when to communicate with the players.
Multi-agent Reinforcement Learning
reinforcement-learning
+3
Dynamic Sparse Training for Deep Reinforcement Learning
In this paper, we introduce for the first time a dynamic sparse training approach for deep reinforcement learning to accelerate the training process.
Continual Learning and Private Unlearning
As intelligent agents become autonomous over longer periods of time, they may eventually become lifelong counterparts to specific people.
Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spaces
While recent advances in deep reinforcement learning have allowed autonomous learning agents to succeed at a variety of complex tasks, existing algorithms generally require a lot of training data.
The EMPATHIC Framework for Task Learning from Implicit Human Feedback
We train a deep neural network on this data and demonstrate its ability to (1) infer relative reward ranking of events in the training task from prerecorded human facial reactions; (2) improve the policy of an agent in the training task using live human facial reactions; and (3) transfer to a novel domain in which it evaluates robot manipulation trajectories.
Human-Computer Interaction Robotics
Importance Sampling Policy Evaluation with an Estimated Behavior Policy
We find that this estimator often lowers the mean squared error of off-policy evaluation compared to importance sampling with the true behavior policy or using a behavior policy that is estimated from a separate data set.
Deterministic Implementations for Reproducibility in Deep Reinforcement Learning
One by one, we then allow individual sources of nondeterminism to affect our otherwise deterministic implementation, and measure the impact of each source on the variance in performance.
Scalable Multiagent Driving Policies For Reducing Traffic Congestion
Next, we propose a modular transfer reinforcement learning approach, and use it to scale up a multiagent driving policy to outperform human-like traffic and existing approaches in a simulated realistic scenario, which is an order of magnitude larger than past scenarios (hundreds instead of tens of vehicles).
DM$^2$: Decentralized Multi-Agent Reinforcement Learning for Distribution Matching
The theoretical analysis shows that under certain conditions, each agent minimizing its individual distribution mismatch allows the convergence to the joint policy that generated the target distribution.
Multi-agent Reinforcement Learning
reinforcement-learning
+2
Lucid Dreaming for Experience Replay: Refreshing Past States with the Current Policy
In this work, we introduce Lucid Dreaming for Experience Replay (LiDER), a conceptually new framework that allows replay experiences to be refreshed by leveraging the agent's current policy.
t-DGR: A Trajectory-Based Deep Generative Replay Method for Continual Learning in Decision Making
Deep generative replay has emerged as a promising approach for continual learning in decision-making tasks.
Adversarial Intrinsic Motivation for Reinforcement Learning
In this paper, we investigate whether one such objective, the Wasserstein-1 distance between a policy's state visitation distribution and a target distribution, can be utilized effectively for reinforcement learning (RL) tasks.
Learning a Robust Multiagent Driving Policy for Traffic Congestion Reduction
This article establishes for the first time that a multiagent driving policy can be trained in such a way that it generalizes to different traffic flows, AV penetration, and road geometries, including on multi-lane roads.
Multistep Inverse Is Not All You Need
In this work, we consider the Ex-BMDP model, first proposed by Efroni et al. (2022), which formalizes control problems where observations can be factorized into an action-dependent latent state which evolves deterministically, and action-independent time-correlated noise.
A Scavenger Hunt for Service Robots
In this problem, the goal is to find a set of objects as quickly as possible, given probability distributions of where they may be found.
Data-Efficient Policy Evaluation Through Behavior Policy Search
The standard unbiased technique for evaluating a policy is to deploy the policy and observe its performance.
Learning Curriculum Policies for Reinforcement Learning
Curriculum learning in reinforcement learning is a training methodology that seeks to speed up learning of a difficult target task, by first training on a series of simpler tasks and transferring the knowledge acquired to the target task.
Learning Optimal Advantage from Preferences and Mistaking it for Reward
Most recent work assumes that human preferences are generated based only upon the reward accrued within those segments, or their partial return.
Task Planning in Robotics: an Empirical Comparison of PDDL-based and ASP-based Systems
PDDL is designed for task planning, and PDDL-based planners are widely used for a variety of planning problems.
Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problems
Much research in artificial intelligence is concerned with the development of autonomous agents that can interact effectively with other agents.
Traffic Optimization For a Mixture of Self-interested and Compliant Agents
This paper focuses on two commonly used path assignment policies for agents traversing a congested network: self-interested routing, and system-optimum routing.
Bootstrapping with Models: Confidence Intervals for Off-Policy Evaluation
In this context, we propose two bootstrapping off-policy evaluation methods which use learned MDP transition models in order to estimate lower confidence bounds on policy performance with limited data in both continuous and discrete state spaces.
Online Contrastive Divergence with Generative Replay: Experience Replay without Storing Data
Conceived in the early 1990s, Experience Replay (ER) has been shown to be a successful mechanism to allow online learning algorithms to reuse past experiences.
Representative Selection in Non Metric Datasets
This paper considers the problem of representative selection: choosing a subset of data points from a dataset that best represents its overall set of elements.
DJ-MC: A Reinforcement-Learning Agent for Music Playlist Recommendation
In this work we present DJ-MC, a novel reinforcement-learning framework for music recommendation that does not recommend songs individually but rather song sequences, or playlists, based on a model of preferences for both songs and song transitions.
A Century Long Commitment to Assessing Artificial Intelligence and its Impact on Society
In September 2016, Stanford's "One Hundred Year Study on Artificial Intelligence" project (AI100) issued the first report of its planned long-term periodic assessment of artificial intelligence (AI) and its impact on society.
Learning a Policy for Opportunistic Active Learning
Active learning identifies data points to label that are expected to be the most useful in improving a supervised model.
Robot Representation and Reasoning with Knowledge from Reinforcement Learning
In this work, we integrate logical-probabilistic KRR with model-based RL, enabling agents to simultaneously reason with declarative knowledge and learn from interaction experiences.
Integrating Task-Motion Planning with Reinforcement Learning for Robust Decision Making in Mobile Robots
In the outer loop, the plan is executed, and the robot learns from the execution experience via model-free RL, to further improve its task-motion plans.
HR-TD: A Regularized TD Method to Avoid Over-Generalization
Temporal Difference learning with function approximation has been widely used recently and has led to several successful results.
TD Learning with Constrained Gradients
In this work we propose a constraint on the TD update that minimizes change to the target values.
Escape Room: A Configurable Testbed for Hierarchical Reinforcement Learning
We show that the ERD presents a suite of challenges with scalable difficulty to provide a smooth learning gradient from Taxi to the Arcade Learning Environment.
Imitation Learning from Video by Leveraging Proprioception
Classically, imitation learning algorithms have been developed for idealized situations, e. g., the demonstrations are often required to be collected in the exact same environment and usually include the demonstrator's actions.
Recent Advances in Imitation Learning from Observation
Imitation learning is the process by which one agent tries to learn how to perform a certain task using information generated by another, often more-expert agent performing that same task.
Sample-efficient Adversarial Imitation Learning from Observation
We test our algorithm and conduct experiments using an imitation task on a physical robot arm and its simulated version in Gazebo and will show the improvement in learning rate and efficiency.
RIDM: Reinforced Inverse Dynamics Modeling for Learning from a Single Observed Demonstration
Augmenting reinforcement learning with imitation learning is often hailed as a method by which to improve upon learning from scratch.
Solving Service Robot Tasks: UT Austin Villa@Home 2019 Team Report
RoboCup@Home is an international robotics competition based on domestic tasks requiring autonomous capabilities pertaining to a large variety of AI technologies.
Leveraging Human Guidance for Deep Reinforcement Learning Tasks
Reinforcement learning agents can learn to solve sequential decision tasks by interacting with the environment.
Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey
Reinforcement learning (RL) is a popular paradigm for addressing sequential decision tasks in which the agent has only limited environmental feedback.
APPLD: Adaptive Planner Parameter Learning from Demonstration
Existing autonomous robot navigation systems allow robots to move from one point to another in a collision-free manner.
iCORPP: Interleaved Commonsense Reasoning and Probabilistic Planning on Robots
Robot sequential decision-making in the real world is a challenge because it requires the robots to simultaneously reason about the current world state and dynamics, while planning actions to accomplish complex tasks.
Learning and Reasoning for Robot Dialog and Navigation Tasks
More interestingly, the robot was able to learn from navigation tasks to improve its dialog strategies.
Deep R-Learning for Continual Area Sweeping
Coverage path planning is a well-studied problem in robotics in which a robot must plan a path that passes through every point in a given area repeatedly, usually with a uniform frequency.
Artificial Musical Intelligence: A Survey
Computers have been used to analyze and create music since they were first introduced in the 1950s and 1960s.
Temporal-Logic-Based Reward Shaping for Continuing Reinforcement Learning Tasks
Reward shaping is a common approach for incorporating domain knowledge into reinforcement learning in order to speed up convergence to an optimal policy.
An Imitation from Observation Approach to Transfer Learning with Dynamics Mismatch
We examine the problem of transferring a policy learned in a source environment to a target environment with different dynamics, particularly in the case where it is critical to reduce the amount of interaction with the target environment during learning.
Reducing Sampling Error in Batch Temporal Difference Learning
In this batch setting, we show that TD(0) may converge to an inaccurate value function because the update following an action is weighted according to the number of times that action occurred in the batch -- not the true probability of the action under the given policy.
A Coach-Player Framework for Dynamic Team Composition
The performance of our method is comparable or even better than the setting where all players have a full view of the environment, but no coach.
Machine versus Human Attention in Deep Reinforcement Learning Tasks
1) How similar are the visual representations learned by RL agents and humans when performing the same task?
Expected Value of Communication for Planning in Ad Hoc Teamwork
We then present a novel planning algorithm for ad hoc teamwork, determining which query to ask and planning accordingly.
DEALIO: Data-Efficient Adversarial Learning for Imitation from Observation
In imitation learning from observation IfO, a learning agent seeks to imitate a demonstrating agent using only observations of the demonstrated behavior without access to the control signals generated by the demonstrator.
Sequential Online Chore Division for Autonomous Vehicle Convoy Formation
Chore division is a class of fair division problems in which some undesirable "resource" must be shared among a set of participants, with each participant wanting to get as little as possible.
Skeletal Feature Compensation for Imitation Learning with Embodiment Mismatch
Learning from demonstrations in the wild (e. g. YouTube videos) is a tantalizing goal in imitation learning.
Reward (Mis)design for Autonomous Driving
This article considers the problem of diagnosing certain common errors in reward design.
RAIL: A modular framework for Reinforcement-learning-based Adversarial Imitation Learning
While Adversarial Imitation Learning (AIL) algorithms have recently led to state-of-the-art results on various imitation learning benchmarks, it is unclear as to what impact various design decisions have on performance.
VOILA: Visual-Observation-Only Imitation Learning for Autonomous Navigation
Is imitation learning for vision based autonomous navigation even possible in such scenarios?
Conflict Avoidance in Social Navigation -- a Survey
This survey aims to bridge this gap by introducing such a common language, using it to survey existing work, and highlighting open problems.
Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Tasks
A longstanding goal of artificial intelligence is to create artificial agents capable of learning to perform tasks that require sequential decision making.
Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation
From the task structures of multi-task demonstrations, we identify skills based on the recurring patterns and train goal-conditioned sensorimotor policies with hierarchical imitation learning.
Generalizing Curricula for Reinforcement Learning
However, there is structure that can be exploited between tasks and agents, such that knowledge gained developing a curriculum for one task can be reused to speed up creating a curriculum for a new task.
Real-world challenges for multi-agent reinforcement learning in grid-interactive buildings
Building upon prior research that highlighted the need for standardizing environments for building control research, and inspired by recently introduced challenges for real life reinforcement learning control, here we propose a non-exhaustive set of nine real world challenges for reinforcement learning control in grid-interactive buildings.
Model Predictive Control
Multi-agent Reinforcement Learning
+2
Adversarial Imitation Learning from Video using a State Observer
The imitation learning research community has recently made significant progress towards the goal of enabling artificial agents to imitate behaviors from video demonstrations alone.
Learning a Shield from Catastrophic Action Effects: Never Repeat the Same Mistake
When humans make catastrophic mistakes, they are expected to learn never to repeat them, such as a toddler who touches a hot stove and immediately learns never to do so again.
A Survey of Ad Hoc Teamwork Research
Ad hoc teamwork is the research problem of designing agents that can collaborate with new teammates without prior coordination.
Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation
Social navigation is the capability of an autonomous agent, such as a robot, to navigate in a 'socially compliant' manner in the presence of other intelligent agents such as humans.
VI-IKD: High-Speed Accurate Off-Road Navigation using Learned Visual-Inertial Inverse Kinodynamics
In this paper, we hypothesize that to enable accurate high-speed off-road navigation using a learned IKD model, in addition to inertial information from the past, one must also anticipate the kinodynamic interactions of the vehicle with the terrain in the future.
Effective Mutation Rate Adaptation through Group Elite Selection
GESMR co-evolves a population of solutions and a population of MRs, such that each MR is assigned to a group of solutions.
Models of human preference for learning reward functions
We empirically show that our proposed regret preference model outperforms the partial return preference model with finite training data in otherwise the same setting.
Value Function Decomposition for Iterative Design of Reinforcement Learning Agents
These value estimates provide insight into an agent's learning and decision-making process and enable new training methods to mitigate common problems.
BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach
Bilevel optimization (BO) is useful for solving a variety of important machine learning problems including but not limited to hyperparameter optimization, meta-learning, continual learning, and reinforcement learning.
Task Phasing: Automated Curriculum Learning from Demonstrations
Applying reinforcement learning (RL) to sparse reward domains is notoriously challenging due to insufficient guiding signals.
D-Shape: Demonstration-Shaped Reinforcement Learning via Goal Conditioning
While combining imitation learning (IL) and reinforcement learning (RL) is a promising way to address poor sample efficiency in autonomous behavior acquisition, methods that do so typically assume that the requisite behavior demonstrations are provided by an expert that behaves optimally with respect to a task reward.
Event Tables for Efficient Experience Replay
Experience replay (ER) is a crucial component of many deep reinforcement learning (RL) systems.
ABC: Adversarial Behavioral Cloning for Offline Mode-Seeking Imitation Learning
Given a dataset of expert agent interactions with an environment of interest, a viable method to extract an effective agent policy is to estimate the maximum likelihood policy indicated by this data.
Artificial Intelligence and Life in 2030: The One Hundred Year Study on Artificial Intelligence
In September 2016, Stanford's "One Hundred Year Study on Artificial Intelligence" project (AI100) issued the first report of its planned long-term periodic assessment of artificial intelligence (AI) and its impact on society.
Safe Evaluation For Offline Learning: Are We Ready To Deploy?
In our setting, we assume a source of data, which we split into a train-set, to learn an offline policy, and a test-set, to estimate a lower-bound on the offline policy using off-policy evaluation with bootstrapping.
A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems
Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to "real world" events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed.
Causal Policy Gradient for Whole-Body Mobile Manipulation
Current approaches often segregate tasks into navigation without manipulation and stationary manipulation without locomotion by manually matching parts of the action space to MoMa sub-objectives (e. g. learning base actions for locomotion objectives and learning arm actions for manipulation).
Composing Efficient, Robust Tests for Policy Selection
Empirical results demonstrate that RPOSST finds a small set of test cases that identify high quality policies in a toy one-shot game, poker datasets, and a high-fidelity racing simulator.
Principles and Guidelines for Evaluating Social Robot Navigation Algorithms
A major challenge to deploying robots widely is navigation in human-populated environments, commonly referred to as social robot navigation.
Minimum Coverage Sets for Training Robust Ad Hoc Teamwork Agents
In this work, we first propose that maximizing an AHT agent's robustness requires it to emulate policies in the minimum coverage set (MCS), the set of best-response policies to any partner policies in the environment.
Utilizing Mood-Inducing Background Music in Human-Robot Interaction
This research fills this gap by reporting the results of an experiment in which human participants were required to complete a task in the presence of an autonomous agent while listening to background music.
Wait, That Feels Familiar: Learning to Extrapolate Human Preferences for Preference Aligned Path Planning
In this work, we posit that operator preferences for visually novel terrains, which the robot should adhere to, can often be extrapolated from established terrain references within the inertial, proprioceptive, and tactile domain.
STERLING: Self-Supervised Terrain Representation Learning from Unconstrained Robot Experience
Terrain awareness, i. e., the ability to identify and distinguish different types of terrain, is a critical ability that robots must have to succeed at autonomous off-road navigation.
$f$-Policy Gradients: A General Framework for Goal Conditioned RL using $f$-Divergences
We further introduce an entropy-regularized policy optimization objective, that we call $state$-MaxEnt RL (or $s$-MaxEnt RL) as a special case of our objective.
Dobby: A Conversational Service Robot Driven by GPT-4
This work introduces a robotics platform which embeds a conversational AI agent in an embodied system for natural language understanding and intelligent decision-making for service tasks; integrating task planning and human-like conversation.
Learning Generalizable Manipulation Policies with Object-Centric 3D Representations
We introduce GROOT, an imitation learning method for learning robust policies with object-centric and 3D priors.
ICRA Roboethics Challenge 2023: Intelligent Disobedience in an Elderly Care Home
With the projected surge in the elderly population, service robots offer a promising avenue to enhance their well-being in elderly care homes.
Latent Skill Discovery for Chain-of-Thought Reasoning
Simultaneously, RSD learns a reasoning policy to determine the required reasoning skill for a given question.
Building Minimal and Reusable Causal State Abstractions for Reinforcement Learning
Two desiderata of reinforcement learning (RL) algorithms are the ability to learn from relatively little experience and the ability to learn policies that generalize to a range of problem specifications.
Sample Efficient Myopic Exploration Through Multitask Reinforcement Learning with Diverse Tasks
Multitask Reinforcement Learning (MTRL) approaches have gained increasing attention for its wide applications in many important Reinforcement Learning (RL) tasks.
Dexterous Legged Locomotion in Confined 3D Spaces with Reinforcement Learning
To address the inefficiency of tracking distant navigation goals, we introduce a hierarchical locomotion controller that combines a classical planner tasked with planning waypoints to reach a faraway global goal location, and an RL-based policy trained to follow these waypoints by generating low-level motion commands.
TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulation
This problem is more severe in mobile manipulation, where collecting demonstrations is harder than in stationary manipulation due to the lack of available and easy-to-use teleoperation interfaces.
Dyna-LfLH: Learning Agile Navigation in Dynamic Environments from Learned Hallucination
In our new Dynamic Learning from Learned Hallucination (Dyna-LfLH), we design and learn a novel latent distribution and sample dynamic obstacles from it, so the generated training data can be used to learn a motion planner to navigate in dynamic environments.
N-Agent Ad Hoc Teamwork
POAM is a policy gradient, multi-agent reinforcement learning approach to the NAHT problem, that enables adaptation to diverse teammate behaviors by learning representations of teammate behaviors.
