Search Results for author:
Found 62 papers, 29 papers with code
COMO: Compact Mapping and Odometry
We present COMO, a real-time monocular mapping and odometry system that encodes dense geometry via a compact set of 3D anchor points.
U-ARE-ME: Uncertainty-Aware Rotation Estimation in Manhattan Environments
Given a sequence of images, we can use the per-frame rotation estimates and their uncertainty to perform multi-frame optimisation, achieving robustness and temporal consistency.
Rethinking Inductive Biases for Surface Normal Estimation
Despite the growing demand for accurate surface normal estimation models, existing methods use general-purpose dense prediction models, adopting the same inductive biases as other tasks.
EscherNet: A Generative Model for Scalable View Synthesis
We introduce EscherNet, a multi-view conditioned diffusion model for view synthesis.
Fit-NGP: Fitting Object Models to Neural Graphics Primitives
Accurate 3D object pose estimation is key to enabling many robotic applications that involve challenging object interactions.
Gaussian Splatting SLAM
We present the first application of 3D Gaussian Splatting in monocular SLAM, the most fundamental but the hardest setup for Visual SLAM.
SuperPrimitive: Scene Reconstruction at a Primitive Level
We address this issue with a new image representation which we call a SuperPrimitive.
Learning a Depth Covariance Function
We propose learning a depth covariance function with applications to geometric vision tasks.
vMAP: Vectorised Object Mapping for Neural Field SLAM
We present vMAP, an object-level dense SLAM system using neural field representations.
Real-time Mapping of Physical Scene Properties with an Autonomous Robot Experimenter
Neural fields can be trained from scratch to represent the shape and appearance of 3D scenes efficiently.
Feature-Realistic Neural Fusion for Real-Time, Open Set Scene Understanding
General scene understanding for robotics requires flexible semantic representation, so that novel objects and structures which may not have been known at training time can be identified, segmented and grouped.
Learning to Complete Object Shapes for Object-level Mapping in Dynamic Scenes
In this paper, we propose a novel object-level mapping system that can simultaneously segment, track, and reconstruct objects in dynamic scenes.
Distributing Collaborative Multi-Robot Planning with Gaussian Belief Propagation
Precise coordinated planning over a forward time window enables safe and highly efficient motion when many robots must work together in tight spaces, but this would normally require centralised control of all devices which is difficult to scale.
Simultaneous Localisation and Mapping with Quadric Surfaces
Human-made environments contain a lot of structure, and we seek to take advantage of this by enabling the use of quadric surfaces as features in SLAM systems.
ReorientBot: Learning Object Reorientation for Specific-Posed Placement
Robots need the capability of placing objects in arbitrary, specific poses to rearrange the world and achieve various valuable tasks.
SafePicking: Learning Safe Object Extraction via Object-Level Mapping
We evaluate our methods using the YCB objects in both simulation and the real world, achieving safe object extraction from piles.
Auto-Lambda: Disentangling Dynamic Task Relationships
Unlike previous methods where task relationships are assumed to be fixed, Auto-Lambda is a gradient-based meta learning framework which explores continuous, dynamic task relationships via task-specific weightings, and can optimise any choice of combination of tasks through the formulation of a meta-loss; where the validation loss automatically influences task weightings throughout training.
Ranked #3 on
Robot Manipulation
on RLBench
(Succ. Rate (10 tasks, 100 demos/task) metric)
A Robot Web for Distributed Many-Device Localisation
We show that a distributed network of robots or other devices which make measurements of each other can collaborate to globally localise via efficient ad-hoc peer to peer communication.
ILabel: Interactive Neural Scene Labelling
ILabel's underlying model is a multilayer perceptron (MLP) trained from scratch in real-time to learn a joint neural scene representation.
Incremental Abstraction in Distributed Probabilistic SLAM Graphs
Scene graphs represent the key components of a scene in a compact and semantically rich way, but are difficult to build during incremental SLAM operation because of the challenges of robustly identifying abstract scene elements and optimising continually changing, complex graphs.
CodeMapping: Real-Time Dense Mapping for Sparse SLAM using Compact Scene Representations
In this paper we propose a dense mapping framework to complement sparse visual SLAM systems which takes as input the camera poses, keyframes and sparse points produced by the SLAM system and predicts a dense depth image for every keyframe.
A visual introduction to Gaussian Belief Propagation
In this article, we present a visual introduction to Gaussian Belief Propagation (GBP), an approximate probabilistic inference algorithm that operates by passing messages between the nodes of arbitrarily structured factor graphs.
Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisation
We present a coarse-to-fine discretisation method that enables the use of discrete reinforcement learning approaches in place of unstable and data-inefficient actor-critic methods in continuous robotics domains.
Ranked #7 on
Robot Manipulation
on RLBench
Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulation
Despite the success of reinforcement learning methods, they have yet to have their breakthrough moment when applied to a broad range of robotic manipulation tasks.
Ranked #1 on
Robot Task Planning
on RLBench
Bootstrapping Semantic Segmentation with Regional Contrast
We present ReCo, a contrastive learning framework designed at a regional level to assist learning in semantic segmentation.
SIMstack: A Generative Shape and Instance Model for Unordered Object Stacks
At test time, our model can generate 3D shape and instance segmentation from a single depth view, probabilistically sampling proposals for the occluded region from the learned latent space.
In-Place Scene Labelling and Understanding with Implicit Scene Representation
Semantic labelling is highly correlated with geometry and radiance reconstruction, as scene entities with similar shape and appearance are more likely to come from similar classes.
iMAP: Implicit Mapping and Positioning in Real-Time
We show for the first time that a multilayer perceptron (MLP) can serve as the only scene representation in a real-time SLAM system for a handheld RGB-D camera.
End-to-End Egospheric Spatial Memory
Spatial memory, or the ability to remember and recall specific locations and objects, is central to autonomous agents' ability to carry out tasks in real environments.
Rearrangement: A Challenge for Embodied AI
In the rearrangement task, the goal is to bring a given physical environment into a specified state.
Deep Probabilistic Feature-metric Tracking
Dense image alignment from RGB-D images remains a critical issue for real-world applications, especially under challenging lighting conditions and in a wide baseline setting.
MoreFusion: Multi-object Reasoning for 6D Pose Estimation from Volumetric Fusion
Robots and other smart devices need efficient object-based scene representations from their on-board vision systems to reason about contact, physics and occlusion.
Bundle Adjustment on a Graph Processor
Graph processors such as Graphcore's Intelligence Processing Unit (IPU) are part of the major new wave of novel computer architecture for AI, and have a general design with massively parallel computation, distributed on-chip memory and very high inter-core communication bandwidth which allows breakthrough performance for message passing algorithms on arbitrary graphs.
DeepFactors: Real-Time Probabilistic Dense Monocular SLAM
The ability to estimate rich geometry and camera motion from monocular imagery is fundamental to future interactive robotics and augmented reality applications.
Learning One-Shot Imitation from Humans without Humans
But is there a way to remove the need for real world human demonstrations during training?
FutureMapping 2: Gaussian Belief Propagation for Spatial AI
We argue the case for Gaussian Belief Propagation (GBP) as a strong algorithmic framework for the distributed, generic and incremental probabilistic estimation we need in Spatial AI as we aim at high performance smart robots and devices which operate within the constraints of real products.
RLBench: The Robot Learning Benchmark & Learning Environment
We present a challenging new benchmark and learning-environment for robot learning: RLBench.
PyRep: Bringing V-REP to Deep Robot Learning
PyRep is a toolkit for robot learning research, built on top of the virtual robotics experimentation platform (V-REP).
SceneCode: Monocular Dense Semantic Reconstruction using Learned Encoded Scene Representations
Systems which incrementally create 3D semantic maps from image sequences must store and update representations of both geometry and semantic entities.
Self-Supervised Generalisation with Meta Auxiliary Learning
The loss for the label-generation network incorporates the loss of the multi-task network, and so this interaction between the two networks can be seen as a form of meta learning with a double gradient.
Task-Embedded Control Networks for Few-Shot Imitation Learning
Despite this, most robot learning approaches have focused on learning a single task, from scratch, with a limited notion of generalisation, and no way of leveraging the knowledge to learn other tasks more efficiently.
LS-Net: Learning to Solve Nonlinear Least Squares for Monocular Stereo
In this paper, we propose LS-Net, a neural nonlinear least squares optimization algorithm which learns to effectively optimize these cost functions even in the presence of adversities.
Fusion++: Volumetric Object-Level SLAM
Reconstructed objects are stored in an optimisable 6DoF pose graph which is our only persistent map representation.
Navigating the Landscape for Real-time Localisation and Mapping for Robotics and Virtual and Augmented Reality
Visual understanding of 3D environments in real-time, at low power, is a huge computational challenge.
Sim-to-Real Reinforcement Learning for Deformable Object Manipulation
Moreover, due to the large amount of data needed to learn these end-to-end solutions, an emerging trend is to learn control policies in simulation and then transfer them over to the real world.
CodeSLAM â Learning a Compact, Optimisable Representation for Dense Visual SLAM
Our approach is suitable for use in a keyframe-based monocular dense SLAM system: While each keyframe with a code can produce a depth map, the code can be optimised efficiently jointly with pose variables and together with the codes of overlapping keyframes to attain global consistency.
CodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAM
Our approach is suitable for use in a keyframe-based monocular dense SLAM system: While each keyframe with a code can produce a depth map, the code can be optimised efficiently jointly with pose variables and together with the codes of overlapping keyframes to attain global consistency.
FutureMapping: The Computational Structure of Spatial AI Systems
We discuss and predict the evolution of Simultaneous Localisation and Mapping (SLAM) into a general geometric and semantic `Spatial AI' perception capability for intelligent embodied devices.
End-to-End Multi-Task Learning with Attention
Our design, the Multi-Task Attention Network (MTAN), consists of a single shared network containing a global feature pool, together with a soft-attention module for each task.
SceneNet RGB-D: Can 5M Synthetic Images Beat Generic ImageNet Pre-Training on Indoor Segmentation?
We compare the semantic segmentation performance of network weights produced from pre-training on RGB images from our dataset against generic VGG-16 ImageNet weights.
Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task
End-to-end control for robot manipulation and grasping is emerging as an attractive alternative to traditional pipelined approaches.
Algorithmic Performance-Accuracy Trade-off in 3D Vision Applications Using HyperMapper
In this paper we investigate an emerging application, 3D scene understanding, likely to be significant in the mobile space in the near future.
SceneNet RGB-D: 5M Photorealistic Images of Synthetic Indoor Trajectories with Ground Truth
We introduce SceneNet RGB-D, expanding the previous work of SceneNet to enable large scale photorealistic rendering of indoor scene trajectories.
Deep Learning a Grasp Function for Grasping under Gripper Pose Uncertainty
With this, it is possible to achieve grasping robust to the gripper's pose uncertainty, by smoothing the grasp function with the pose uncertainty function.
Simultaneous Optical Flow and Intensity Estimation From an Event Camera
In a series of examples, we demonstrate the successful operation of our framework, including in situations where conventional cameras heavily suffer from dynamic range limitations or motion blur.
Pairwise Decomposition of Image Sequences for Active Multi-View Recognition
A multi-view image sequence provides a much richer capacity for object recognition than from a single image.
Effective Backscatter Approximation for Photometry in Murky Water
Backscatter corresponds to a complex term with several unknown variables, and makes the problem of normal estimation hard.
Comparative Design Space Exploration of Dense and Semi-Dense SLAM
SLAM has matured significantly over the past few years, and is beginning to appear in serious commercial products.
Introducing SLAMBench, a performance and accuracy benchmarking methodology for SLAM
Real-time dense computer vision and SLAM offer great potential for a new level of scene modelling, tracking and real environmental interaction for many types of robot, but their high computational requirements mean that use on mass market embedded platforms is challenging.
Backscatter Compensated Photometric Stereo with 3 Sources
We compare our method with previous approaches through extensive experimental results, where a variety of objects are imaged in a big water tank whose turbidity is systematically increased, and show reconstruction quality which degrades little relative to clean water results even with a very significant scattering level.
SLAM++: Simultaneous Localisation and Mapping at the Level of Objects
We present the major advantages of a new 'object oriented' 3D SLAM paradigm, which takes full advantage in the loop of prior knowledge that many scenes consist of repeated, domain-specific objects and structures.
KinectFusion: Real-Time Dense Surface Mapping and Tracking
We present a system for accurate real-time mapping of complex and arbitrary indoor scenes in variable lighting conditions, using only a moving low-cost depth camera and commodity graphics hardware.
