Search Results for author:
Found 67 papers, 11 papers with code
A Distributional Treatment of Real2Sim2Real for Vision-Driven Deformable Linear Object Manipulation
We present an integrated (or end-to-end) framework for the Real2Sim2Real problem of manipulating deformable linear objects (DLOs) based on visual perception.
Learning Visually Grounded Domain Ontologies via Embodied Conversation and Explanation
In this paper, we offer a learning framework in which the agent's knowledge gaps are overcome through corrective feedback from a teacher whenever the agent explains its (incorrect) predictions.
Learning a Neural Association Network for Self-supervised Multi-Object Tracking
Motivated by the fact that in real-world scenarios object motion can be usually represented by a Markov process, we present a novel expectation maximization (EM) algorithm that trains a neural network to associate detections for tracking, without requiring prior knowledge of their temporal correspondences.
Generating Driving Simulations via Conversation
Cyber-physical systems like autonomous vehicles are tested in simulation before deployment, using domain-specific programs for scenario specification.
Learning from Demonstration with Implicit Nonlinear Dynamics Models
We validate the efficacy of our neural network layer on the task of reproducing human handwriting motions using the LASA Human Handwriting Dataset.
SECURE: Semantics-aware Embodied Conversation under Unawareness for Lifelong Robot Learning
This paper addresses a challenging interactive task learning scenario we call rearrangement under unawareness: to manipulate a rigid-body environment in a context where the agent is unaware of a concept that is key to solving the instructed task.
OPPH: A Vision-Based Operator for Measuring Body Movements for Personal Healthcare
To address these challenges and improve the accuracy of human body motion estimation for healthcare purposes, we propose the OPPH operator designed to enhance current vision-based motion estimation methods.
Unobtrusive Monitoring of Physical Weakness: A Simulated Approach
We simulate weakness in healthy subjects by having them perform physical exercise and observing the behavioral changes in their daily activities before and after workouts.
Click to Grasp: Zero-Shot Precise Manipulation via Visual Diffusion Descriptors
Precise manipulation that is generalizable across scenes and objects remains a persistent challenge in robotics.
Adaptive Splitting of Reusable Temporal Monitors for Rare Traffic Violations
Two common issues arise when using existing techniques to produce this estimation: If violations occur rarely, simple Monte-Carlo sampling techniques can fail to produce efficient estimates; if simulation horizons are too long, importance sampling techniques (which learn proposal distributions from past simulations) can fail to converge.
Comparison of Pedestrian Prediction Models from Trajectory and Appearance Data for Autonomous Driving
Typical predictors use the trajectory history to predict future motion, however in cases of motion initiation, motion in the trajectory may only be clearly visible after a delay, which can result in the pedestrian has entered the road area before an accurate prediction can be made.
Interactive Acquisition of Fine-grained Visual Concepts by Exploiting Semantics of Generic Characterizations in Discourse
Interactive Task Learning (ITL) concerns learning about unforeseen domain concepts via natural interactions with human users.
Testing Rare Downstream Safety Violations via Upstream Adaptive Sampling of Perception Error Models
Testing black-box perceptual-control systems in simulation faces two difficulties.
Beyond RMSE: Do machine-learned models of road user interaction produce human-like behavior?
Even though the models' RMSE value differed, all the models captured the kinematic-dependent merging behavior but struggled at varying degrees to capture the more nuanced courtesy lane change and highway lane change behavior.
On Specifying for Trustworthiness
As autonomous systems (AS) increasingly become part of our daily lives, ensuring their trustworthiness is crucial.
Risk-Driven Design of Perception Systems
We formulate a risk function to quantify the effect of a given perceptual error on overall safety, and show how we can use it to design safer perception systems by including a risk-dependent term in the loss function and generating training data in risk-sensitive regions.
Learning physics-informed simulation models for soft robotic manipulation: A case study with dielectric elastomer actuators
We simulate the task of using DEA to pull a coin along a surface with frictional contact, using FEM, and evaluate the physics-informed model for simulation, control, and inference.
Robust Learning from Observation with Model Misspecification
Imitation learning (IL) is a popular paradigm for training policies in robotic systems when specifying the reward function is difficult.
Vision Checklist: Towards Testable Error Analysis of Image Models to Help System Designers Interrogate Model Capabilities
Robustness evaluations like our checklist will be crucial in future safety evaluations of visual perception modules, and be useful for a wide range of stakeholders including designers, deployers, and regulators involved in the certification of these systems.
Applications and Techniques for Fast Machine Learning in Science
In this community review report, we discuss applications and techniques for fast machine learning (ML) in science -- the concept of integrating power ML methods into the real-time experimental data processing loop to accelerate scientific discovery.
Active Altruism Learning and Information Sufficiency for Autonomous Driving
Safe interaction between vehicles requires the ability to choose actions that reveal the preferences of the other vehicles.
Beyond Discriminant Patterns: On the Robustness of Decision Rule Ensembles
Specifically, we propose to leverage causal knowledge by regarding the distributional shifts in subpopulations and deployment environments as the results of interventions on the underlying system.
Attainment Regions in Feature-Parameter Space for High-Level Debugging in Autonomous Robots
When the robot successfully executes the task, we use the attainment regions to gain insights into the limits of the controller, and its robustness.
Learning Time-Invariant Reward Functions through Model-Based Inverse Reinforcement Learning
Inverse reinforcement learning is a paradigm motivated by the goal of learning general reward functions from demonstrated behaviours.
Building Affordance Relations for Robotic Agents - A Review
Affordances describe the possibilities for an agent to perform actions with an object.
Learning data association without data association: An EM approach to neural assignment prediction
Data association is a fundamental component of effective multi-object tracking.
PILOT: Efficient Planning by Imitation Learning and Optimisation for Safe Autonomous Driving
In this paper, we present PILOT -- a planning framework that comprises an imitation neural network followed by an efficient optimiser that actively rectifies the network's plan, guaranteeing fulfilment of safety and comfort requirements.
Counterfactual Explanation and Causal Inference in Service of Robustness in Robot Control
', motivated by applications in robot control.
Residual Learning from Demonstration: Adapting DMPs for Contact-rich Manipulation
Dynamic Movement Primitives (DMP) are a popular way of extracting such policies through behaviour cloning (BC) but can struggle in the context of insertion.
Action sequencing using visual permutations
Humans can easily reason about the sequence of high level actions needed to complete tasks, but it is particularly difficult to instil this ability in robots trained from relatively few examples.
Semi-supervised Learning From Demonstration Through Program Synthesis: An Inspection Robot Case Study
The system induces a controller program by learning from immersive demonstrations using sequential importance sampling.
Learning from Demonstration with Weakly Supervised Disentanglement
We show that such alignment is best achieved through the use of labels from the end user, in an appropriately restricted vocabulary, in contrast to the conventional approach of the designer picking a prior over the latent variables.
Affordances in Robotic Tasks -- A Survey
Affordances are key attributes of what must be perceived by an autonomous robotic agent in order to effectively interact with novel objects.
Robotics
Integrating Planning and Interpretable Goal Recognition for Autonomous Driving
The ability to predict the intentions and driving trajectories of other vehicles is a key problem for autonomous driving.
Robotics
Learning rewards for robotic ultrasound scanning using probabilistic temporal ranking
This work considers the inverse problem, where the goal of the task is unknown, and a reward function needs to be inferred from exploratory example demonstrations provided by a demonstrator, for use in a downstream informative path-planning policy.
Elaborating on Learned Demonstrations with Temporal Logic Specifications
We also implement our system on a PR-2 robot to show how a demonstrator can start with an initial (sub-optimal) demonstration, then interactively improve task success by including additional specifications enforced with our differentiable LTL loss.
Lower Dimensional Kernels for Video Discriminators
This work presents an analysis of the discriminators used in Generative Adversarial Networks (GANs) for Video.
Learning Structured Representations of Spatial and Interactive Dynamics for Trajectory Prediction in Crowded Scenes
Context plays a significant role in the generation of motion for dynamic agents in interactive environments.
Disentangled Relational Representations for Explaining and Learning from Demonstration
Learning from demonstration is an effective method for human users to instruct desired robot behaviour.
E-HBA: Using Action Policies for Expert Advice and Agent Typification
Past research has studied two approaches to utilise predefined policy sets in repeated interactions: as experts, to dictate our own actions, and as types, to characterise the behaviour of other agents.
Comparative Evaluation of Multiagent Learning Algorithms in a Diverse Set of Ad Hoc Team Problems
In this work, we empirically evaluate five MAL algorithms, representing major approaches to multiagent learning but originally developed with the homogeneous setting in mind, to understand their behaviour in a set of ad hoc team problems.
Composing Diverse Policies for Temporally Extended Tasks
Robot control policies for temporally extended and sequenced tasks are often characterized by discontinuous switches between different local dynamics.
On Convergence and Optimality of Best-Response Learning with Policy Types in Multiagent Systems
In this paper, we provide theoretical guidance on two central design parameters of this method: Firstly, it is important that the user choose a posterior which can learn the true distribution of latent types, as otherwise suboptimal actions may be chosen.
An Empirical Study on the Practical Impact of Prior Beliefs over Policy Types
To address this problem, researchers have studied learning algorithms which compute posterior beliefs over a hypothesised set of policies, based on the observed actions of the other agents.
Exploiting Causality for Selective Belief Filtering in Dynamic Bayesian Networks (Extended Abstract)
Belief filtering in DBNs is the task of inferring the belief state (i. e. the probability distribution over process states) based on incomplete and uncertain observations.
Hybrid system identification using switching density networks
This paper introduces switching density networks, which rely on a categorical reparametrisation for hybrid system identification.
DynoPlan: Combining Motion Planning and Deep Neural Network based Controllers for Safe HRL
Many realistic robotics tasks are best solved compositionally, through control architectures that sequentially invoke primitives and achieve error correction through the use of loops and conditionals taking the system back to alternative earlier states.
Robotics
Learning Grasp Affordance Reasoning through Semantic Relations
We use Markov Logic Networks to build a knowledge base graph representation to obtain a probability distribution of grasp affordances for an object.
Robotics
Iterative Model-Based Reinforcement Learning Using Simulations in the Differentiable Neural Computer
The agent improves its policy in simulations generated by the DNC model and rolls out the policy to the live environment, collecting experiences in new portions or tasks of the environment for further learning.
Model-based Reinforcement Learning
reinforcement-learning
+2
An Empirical Evaluation of Adversarial Robustness under Transfer Learning
Specifically, we study the effects of using robust optimisation in the source and target networks.
Learning Programmatically Structured Representations with Perceptor Gradients
We present the perceptor gradients algorithm -- a novel approach to learning symbolic representations based on the idea of decomposing an agent's policy into i) a perceptor network extracting symbols from raw observation data and ii) a task encoding program which maps the input symbols to output actions.
Using Causal Analysis to Learn Specifications from Task Demonstrations
In this work we show that it is possible to learn a generative model for distinct user behavioral types, extracted from human demonstrations, by enforcing clustering of preferred task solutions within the latent space.
From explanation to synthesis: Compositional program induction for learning from demonstration
This work introduces an approach to learning hybrid systems from demonstrations, with an emphasis on extracting models that are explicitly verifiable and easily interpreted by robot operators.
Learning Best Response Strategies for Agents in Ad Exchanges
We address this problem using the Harsanyi-Bellman Ad Hoc Coordination (HBA) algorithm, which conceptualises this interaction in terms of a Stochastic Bayesian Game and arrives at optimal actions by best responding with respect to probabilistic beliefs maintained over a candidate set of opponent behaviour profiles.
Interpretable Latent Spaces for Learning from Demonstration
Effective human-robot interaction, such as in robot learning from human demonstration, requires the learning agent to be able to ground abstract concepts (such as those contained within instructions) in a corresponding high-dimensional sensory input stream from the world.
The ORCA Hub: Explainable Offshore Robotics through Intelligent Interfaces
To enable this to happen, the remote operator will need a high level of situation awareness and key to this is the transparency of what the autonomous systems are doing and why.
Reasoning about Unforeseen Possibilities During Policy Learning
Methods for learning optimal policies in autonomous agents often assume that the way the domain is conceptualised---its possible states and actions and their causal structure---is known in advance and does not change during learning.
Using Program Induction to Interpret Transition System Dynamics
Explaining and reasoning about processes which underlie observed black-box phenomena enables the discovery of causal mechanisms, derivation of suitable abstract representations and the formulation of more robust predictions.
Grounding Symbols in Multi-Modal Instructions
As robots begin to cohabit with humans in semi-structured environments, the need arises to understand instructions involving rich variability---for instance, learning to ground symbols in the physical world.
Explaining Transition Systems through Program Induction
Explaining and reasoning about processes which underlie observed black-box phenomena enables the discovery of causal mechanisms, derivation of suitable abstract representations and the formulation of more robust predictions.
Estimating Activity at Multiple Scales using Spatial Abstractions
Autonomous robots operating in dynamic environments must maintain beliefs over a hypothesis space that is rich enough to represent the activities of interest at different scales.
Belief and Truth in Hypothesised Behaviours
The idea is to hypothesise a set of types, each specifying a possible behaviour for the other agents, and to plan our own actions with respect to those types which we believe are most likely, given the observed actions of the agents.
A Game-Theoretic Model and Best-Response Learning Method for Ad Hoc Coordination in Multiagent Systems
Based on this model, we derive a solution, called Harsanyi-Bellman Ad Hoc Coordination (HBA), which utilises the concept of Bayesian Nash equilibrium in a planning procedure to find optimal actions in the sense of Bellman optimal control.
Bayesian Policy Reuse
We formalise the problem of policy reuse, and present an algorithm for efficiently responding to a novel task instance by reusing a policy from the library of existing policies, where the choice is based on observed 'signals' which correlate to policy performance.
Exploiting Causality for Selective Belief Filtering in Dynamic Bayesian Networks
Furthermore, we demonstrate how passivity occurs naturally in a complex system such as a multi-robot warehouse, and how PSBF can exploit this to accelerate the filtering task.
Clustering Markov Decision Processes For Continual Transfer
The source subset forms an `$\epsilon$-net' over the original set of MDPs, in the sense that for each previous MDP $M_p$, there is a source $M^s$ whose optimal policy has $<\epsilon$ regret in $M_p$.
