Search Results for author:
Found 50 papers, 26 papers with code
CoMic: Co-Training and Mimicry for Reusable Skills
Finally we show that it is possible to interleave the motion capture tracking with training on complementary tasks, enriching the resulting skill space, and enabling the reuse of skills not well covered by the motion capture data such as getting up from the ground or catching a ball.
A distributional view on multi objective policy optimization
Many real-world problems require trading off multiple competing objectives.
Gemma 2: Improving Open Language Models at a Practical Size
In this work, we introduce Gemma 2, a new addition to the Gemma family of lightweight, state-of-the-art open models, ranging in scale from 2 billion to 27 billion parameters.
RecurrentGemma: Moving Past Transformers for Efficient Open Language Models
We introduce RecurrentGemma, a family of open language models which uses Google's novel Griffin architecture.
RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitation
For robots to be useful outside labs and specialized factories we need a way to teach them new useful behaviors quickly.
RoboCat: A Self-Improving Generalist Agent for Robotic Manipulation
With RoboCat, we demonstrate the ability to generalise to new tasks and robots, both zero-shot as well as through adaptation using only 100-1000 examples for the target task.
Barkour: Benchmarking Animal-level Agility with Quadruped Robots
In the second approach, we distill the specialist skills into a Transformer-based generalist locomotion policy, named Locomotion-Transformer, that can handle various terrains and adjust the robot's gait based on the perceived environment and robot states.
Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning
We investigate whether Deep Reinforcement Learning (Deep RL) is able to synthesize sophisticated and safe movement skills for a low-cost, miniature humanoid robot that can be composed into complex behavioral strategies in dynamic environments.
Probing Transfer in Deep Reinforcement Learning without Task Engineering
By formally organising these modifications into several factors of variation, we are able to show that Analyses of Variance (ANOVA) are a potent tool for studying the effects of human-relevant domain changes on the learning and transfer performance of a deep reinforcement learning agent.
Hindering Adversarial Attacks with Implicit Neural Representations
We introduce the Lossy Implicit Network Activation Coding (LINAC) defence, an input transformation which successfully hinders several common adversarial attacks on CIFAR-$10$ classifiers for perturbations up to $\epsilon = 8/255$ in $L_\infty$ norm and $\epsilon = 0. 5$ in $L_2$ norm.
MO2: Model-Based Offline Options
The ability to discover useful behaviours from past experience and transfer them to new tasks is considered a core component of natural embodied intelligence.
The CLRS Algorithmic Reasoning Benchmark
Learning representations of algorithms is an emerging area of machine learning, seeking to bridge concepts from neural networks with classical algorithms.
A Generalist Agent
Inspired by progress in large-scale language modeling, we apply a similar approach towards building a single generalist agent beyond the realm of text outputs.
Ranked #1 on
Skill Generalization
on RGB-Stacking
Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviors
We investigate the use of prior knowledge of human and animal movement to learn reusable locomotion skills for real legged robots.
Learning Transferable Motor Skills with Hierarchical Latent Mixture Policies
We demonstrate in manipulation domains that the method can effectively cluster offline data into distinct, executable behaviours, while retaining the flexibility of a continuous latent variable model.
Few-Shot Keypoint Detection as Task Adaptation via Latent Embeddings
Dense object tracking, the ability to localize specific object points with pixel-level accuracy, is an important computer vision task with numerous downstream applications in robotics.
Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapes
We study the problem of robotic stacking with objects of complex geometry.
Ranked #2 on
Skill Generalization
on RGB-Stacking
Reasoning-Modulated Representations
Neural networks leverage robust internal representations in order to generalise.
Tactile Sim-to-Real Policy Transfer via Real-to-Sim Image Translation
Simulation has recently become key for deep reinforcement learning to safely and efficiently acquire general and complex control policies from visual and proprioceptive inputs.
On Multi-objective Policy Optimization as a Tool for Reinforcement Learning: Case Studies in Offline RL and Finetuning
Many advances that have improved the robustness and efficiency of deep reinforcement learning (RL) algorithms can, in one way or another, be understood as introducing additional objectives or constraints in the policy optimization step.
Skillful Precipitation Nowcasting using Deep Generative Models of Radar
To address these challenges, we present a Deep Generative Model for the probabilistic nowcasting of precipitation from radar.
Learning rich touch representations through cross-modal self-supervision
The sense of touch is fundamental in several manipulation tasks, but rarely used in robot manipulation.
Self-Supervised Learning
Robotics
Explicit Pareto Front Optimization for Constrained Reinforcement Learning
We propose a framework that uses a multi-objective RL algorithm to find a Pareto front of policies that trades off between the reward and constraint(s), and simultaneously searches along this front for constraint-satisfying policies.
S3K: Self-Supervised Semantic Keypoints for Robotic Manipulation via Multi-View Consistency
In this work we advocate semantic 3D keypoints as a visual representation, and present a semi-supervised training objective that can allow instance or category-level keypoints to be trained to 1-5 millimeter-accuracy with minimal supervision.
Deep Reinforcement Learning for Tactile Robotics: Learning to Type on a Braille Keyboard
Artificial touch would seem well-suited for Reinforcement Learning (RL), since both paradigms rely on interaction with an environment.
A Distributional View on Multi-Objective Policy Optimization
Many real-world problems require trading off multiple competing objectives.
Disentangled Cumulants Help Successor Representations Transfer to New Tasks
This is in contrast to the state-of-the-art reinforcement learning agents, which typically start learning each new task from scratch and struggle with knowledge transfer.
Attention-Privileged Reinforcement Learning
Image-based Reinforcement Learning is known to suffer from poor sample efficiency and generalisation to unseen visuals such as distractors (task-independent aspects of the observation space).
Continual Unsupervised Representation Learning
Continual learning aims to improve the ability of modern learning systems to deal with non-stationary distributions, typically by attempting to learn a series of tasks sequentially.
Neural Execution of Graph Algorithms
Graph Neural Networks (GNNs) are a powerful representational tool for solving problems on graph-structured inputs.
Stabilizing Transformers for Reinforcement Learning
Harnessing the transformer's ability to process long time horizons of information could provide a similar performance boost in partially observable reinforcement learning (RL) domains, but the large-scale transformers used in NLP have yet to be successfully applied to the RL setting.
Attention Privileged Reinforcement Learning for Domain Transfer
Applying reinforcement learning (RL) to physical systems presents notable challenges, given requirements regarding sample efficiency, safety, and physical constraints compared to simulated environments.
Meta-Learning with Warped Gradient Descent
On the other hand, approaches that try to control a gradient-based update rule typically resort to computing gradients through the learning process to obtain their meta-gradients, leading to methods that can not scale beyond few-shot task adaptation.
The StreetLearn Environment and Dataset
These datasets cannot be used for decision-making and reinforcement learning, however, and in general the perspective of navigation as an interactive learning task, where the actions and behaviours of a learning agent are learned simultaneously with the perception and planning, is relatively unsupported.
Learning To Follow Directions in Street View
Navigating and understanding the real world remains a key challenge in machine learning and inspires a great variety of research in areas such as language grounding, planning, navigation and computer vision.
Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks
Using domain adaptation methods to cross this "reality gap" requires a large amount of unlabelled real-world data, whilst domain randomization alone can waste modeling power.
Success at any cost: value constrained model-free continuous control
We demonstrate the efficiency of our approach using a number of continuous control benchmark tasks as well as a realistic, energy-optimized quadruped locomotion task.
Meta-Learning with Latent Embedding Optimization
We show that it is possible to bypass these limitations by learning a data-dependent latent generative representation of model parameters, and performing gradient-based meta-learning in this low-dimensional latent space.
Graph networks as learnable physics engines for inference and control
Understanding and interacting with everyday physical scenes requires rich knowledge about the structure of the world, represented either implicitly in a value or policy function, or explicitly in a transition model.
Ranked #9 on
Weather Forecasting
on SD
Progress & Compress: A scalable framework for continual learning
This is achieved by training a network with two components: A knowledge base, capable of solving previously encountered problems, which is connected to an active column that is employed to efficiently learn the current task.
The Limits and Potentials of Deep Learning for Robotics
In this paper we discuss a number of robotics-specific learning, reasoning, and embodiment challenges for deep learning.
Robotics
Learning to Navigate in Cities Without a Map
We present an interactive navigation environment that uses Google StreetView for its photographic content and worldwide coverage, and demonstrate that our learning method allows agents to learn to navigate multiple cities and to traverse to target destinations that may be kilometres away.
Reinforcement and Imitation Learning for Diverse Visuomotor Skills
We propose a model-free deep reinforcement learning method that leverages a small amount of demonstration data to assist a reinforcement learning agent.
One-Shot Reinforcement Learning for Robot Navigation with Interactive Replay
Recently, model-free reinforcement learning algorithms have been shown to solve challenging problems by learning from extensive interaction with the environment.
Distral: Robust Multitask Reinforcement Learning
Moreover, the proposed learning process is more robust and more stable---attributes that are critical in deep reinforcement learning.
Overcoming catastrophic forgetting in neural networks
The ability to learn tasks in a sequential fashion is crucial to the development of artificial intelligence.
Ranked #1 on
class-incremental learning
on cifar100
Learning to Navigate in Complex Environments
Learning to navigate in complex environments with dynamic elements is an important milestone in developing AI agents.
Sim-to-Real Robot Learning from Pixels with Progressive Nets
The progressive net approach is a general framework that enables reuse of everything from low-level visual features to high-level policies for transfer to new tasks, enabling a compositional, yet simple, approach to building complex skills.
Progressive Neural Networks
Learning to solve complex sequences of tasks--while both leveraging transfer and avoiding catastrophic forgetting--remains a key obstacle to achieving human-level intelligence.
Policy Distillation
Policies for complex visual tasks have been successfully learned with deep reinforcement learning, using an approach called deep Q-networks (DQN), but relatively large (task-specific) networks and extensive training are needed to achieve good performance.
