Search Results for author:
Found 71 papers, 34 papers with code
VINet: Visual-Inertial Odometry as a Sequence-to-Sequence Learning Problem
In this paper we present an on-manifold sequence-to-sequence learning approach to motion estimation using visual and inertial sensors.
VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization
Machine learning techniques, namely convolutional neural networks (CNN) and regression forests, have recently shown great promise in performing 6-DoF localization of monocular images.
3D Object Reconstruction from a Single Depth View with Adversarial Learning
In this paper, we propose a novel 3D-RecGAN approach, which reconstructs the complete 3D structure of a given object from a single arbitrary depth view using generative adversarial networks.
Learning from lions: inferring the utility of agents from their trajectories
We build a model using Gaussian processes to infer a spatio-temporal vector field from observed agent trajectories.
IONet: Learning to Cure the Curse of Drift in Inertial Odometry
Inertial sensors play a pivotal role in indoor localization, which in turn lays the foundation for pervasive personal applications.
Dense 3D Object Reconstruction from a Single Depth View
Unlike existing work which typically requires multiple views of the same object or class labels to recover the full 3D geometry, the proposed 3D-RecGAN++ only takes the voxel grid representation of a depth view of the object as input, and is able to generate the complete 3D occupancy grid with a high resolution of 256^3 by recovering the occluded/missing regions.
Identifying Sources and Sinks in the Presence of Multiple Agents with Gaussian Process Vector Calculus
In systems of multiple agents, identifying the cause of observed agent dynamics is challenging.
Defo-Net: Learning Body Deformation using Generative Adversarial Networks
Modelling the physical properties of everyday objects is a fundamental prerequisite for autonomous robots.
Robotics
3D-PhysNet: Learning the Intuitive Physics of Non-Rigid Object Deformations
This is further confounded by the fact that shape information about encountered objects in the real world is often impaired by occlusions, noise and missing regions e. g. a robot manipulating an object will only be able to observe a partial view of the entire solid.
Robust Attentional Aggregation of Deep Feature Sets for Multi-view 3D Reconstruction
However, GRU based approaches are unable to consistently estimate 3D shapes given different permutations of the same set of input images as the recurrent unit is permutation variant.
Ranked #1 on
3D Reconstruction
on Data3D−R2N2
Neural Allocentric Intuitive Physics Prediction from Real Videos
In this framework, real images are first converted to a synthetic domain representation that reduces complexity arising from lighting and texture.
GANVO: Unsupervised Deep Monocular Visual Odometry and Depth Estimation with Generative Adversarial Networks
In the last decade, supervised deep learning approaches have been extensively employed in visual odometry (VO) applications, which is not feasible in environments where labelled data is not abundant.
OxIOD: The Dataset for Deep Inertial Odometry
Advances in micro-electro-mechanical (MEMS) techniques enable inertial measurements units (IMUs) to be small, cheap, energy efficient, and widely used in smartphones, robots, and drones.
Transferring Physical Motion Between Domains for Neural Inertial Tracking
Inertial information processing plays a pivotal role in ego-motion awareness for mobile agents, as inertial measurements are entirely egocentric and not environment dependent.
Learning with Stochastic Guidance for Navigation
Due to the sparse rewards and high degree of environment variation, reinforcement learning approaches such as Deep Deterministic Policy Gradient (DDPG) are plagued by issues of high variance when applied in complex real world environments.
Robotics
Selective Sensor Fusion for Neural Visual-Inertial Odometry
Deep learning approaches for Visual-Inertial Odometry (VIO) have proven successful, but they rarely focus on incorporating robust fusion strategies for dealing with imperfect input sensory data.
Learning Monocular Visual Odometry through Geometry-Aware Curriculum Learning
Inspired by the cognitive process of humans and animals, Curriculum Learning (CL) trains a model by gradually increasing the difficulty of the training data.
Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds
The framework directly regresses 3D bounding boxes for all instances in a point cloud, while simultaneously predicting a point-level mask for each instance.
Ranked #13 on
3D Instance Segmentation
on S3DIS
(mPrec metric)
Distilling Knowledge From a Deep Pose Regressor Network
To the best of our knowledge, this is the first work which successfully distill the knowledge from a deep pose regression network.
DynaNet: Neural Kalman Dynamical Model for Motion Estimation and Prediction
In addition we show how DynaNet can indicate failures through investigation of properties such as the rate of innovation (Kalman Gain).
Autonomous Learning for Face Recognition in the Wild via Ambient Wireless Cues
Inspired by the fact that most people carry smart wireless devices with them, e. g. smartphones, we propose to use this wireless identifier as a supervisory label.
AtLoc: Attention Guided Camera Localization
Deep learning has achieved impressive results in camera localization, but current single-image techniques typically suffer from a lack of robustness, leading to large outliers.
Ranked #2 on
Visual Localization
on Oxford RobotCar Full
Milli-RIO: Ego-Motion Estimation with Millimetre-Wave Radar and Inertial Measurement Unit Sensor
With the fast-growing demand of location-based services in various indoor environments, robust indoor ego-motion estimation has attracted significant interest in the last decades.
DeepTIO: A Deep Thermal-Inertial Odometry with Visual Hallucination
The hallucination network is taught to predict fake visual features from thermal images by using Huber loss.
DeepPCO: End-to-End Point Cloud Odometry through Deep Parallel Neural Network
There is considerable work in the area of visual odometry (VO), and recent advances in deep learning have brought novel approaches to VO, which directly learn salient features from raw images.
Introducing an Explicit Symplectic Integration Scheme for Riemannian Manifold Hamiltonian Monte Carlo
We introduce a recent symplectic integration scheme derived for solving physically motivated systems with non-separable Hamiltonians.
See Through Smoke: Robust Indoor Mapping with Low-cost mmWave Radar
This paper presents the design, implementation and evaluation of milliMap, a single-chip millimetre wave (mmWave) radar based indoor mapping system targetted towards low-visibility environments to assist in emergency response.
SelfVIO: Self-Supervised Deep Monocular Visual-Inertial Odometry and Depth Estimation
In the last decade, numerous supervised deep learning approaches requiring large amounts of labeled data have been proposed for visual-inertial odometry (VIO) and depth map estimation.
RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds
We study the problem of efficient semantic segmentation for large-scale 3D point clouds.
Ranked #3 on
Semantic Segmentation
on Toronto-3D L002
Snoopy: Sniffing Your Smartwatch Passwords via Deep Sequence Learning
Demand for smartwatches has taken off in recent years with new models which can run independently from smartphones and provide more useful features, becoming first-class mobile platforms.
Learning Selective Sensor Fusion for States Estimation
By integrating the observations from different sensors, these mobile agents are able to perceive the environment and estimate system states, e. g. locations and orientations.
Deep Learning based Pedestrian Inertial Navigation: Methods, Dataset and On-Device Inference
Modern inertial measurements units (IMUs) are small, cheap, energy efficient, and widely employed in smart devices and mobile robots.
PointLoc: Deep Pose Regressor for LiDAR Point Cloud Localization
In this paper, we present a novel end-to-end learning-based LiDAR relocalization framework, termed PointLoc, which infers 6-DoF poses directly using only a single point cloud as input, without requiring a pre-built map.
Robotics
VMLoc: Variational Fusion For Learning-Based Multimodal Camera Localization
We conjecture that this is because of the naive approaches to feature space fusion through summation or concatenation which do not take into account the different strengths of each modality.
A Survey on Deep Learning for Localization and Mapping: Towards the Age of Spatial Machine Intelligence
Deep learning based localization and mapping has recently attracted significant attention.
Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges
An essential prerequisite for unleashing the potential of supervised deep learning algorithms in the area of 3D scene understanding is the availability of large-scale and richly annotated datasets.
Demo Abstract: Indoor Positioning System in Visually-Degraded Environments with Millimetre-Wave Radar and Inertial Sensors
In this demonstration, we present a real-time indoor positioning system which fuses millimetre-wave (mmWave) radar and IMU data via deep sensor fusion.
SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration
Extracting robust and general 3D local features is key to downstream tasks such as point cloud registration and reconstruction.
Ranked #2 on
Point Cloud Registration
on ETH (trained on 3DMatch)
P2-Net: Joint Description and Detection of Local Features for Pixel and Point Matching
Accurately describing and detecting 2D and 3D keypoints is crucial to establishing correspondences across images and point clouds.
RadarLoc: Learning to Relocalize in FMCW Radar
There is considerable work in the field of deep camera relocalization, which directly estimates poses from raw images.
SQN: Weakly-Supervised Semantic Segmentation of Large-Scale 3D Point Clouds
Labelling point clouds fully is highly time-consuming and costly.
Graph-based Thermal-Inertial SLAM with Probabilistic Neural Networks
Simultaneous Localization and Mapping (SLAM) system typically employ vision-based sensors to observe the surrounding environment.
SoundDet: Polyphonic Moving Sound Event Detection and Localization from Raw Waveform
Specifically, SoundDet consists of a backbone neural network and two parallel heads for temporal detection and spatial localization, respectively.
Learning Semantic Segmentation of Large-Scale Point Clouds with Random Sampling
We study the problem of efficient semantic segmentation of large-scale 3D point clouds.
Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations
FBINNs are designed to address the spectral bias of neural networks by using separate input normalisation over each subdomain, and reduce the complexity of the underlying optimisation problem by using many smaller neural networks in a parallel divide-and-conquer approach.
Scaling physics-informed neural networks to large domains by using domain decomposition
Recently, physics-informed neural networks (PINNs) have offered a powerful new paradigm for solving forward and inverse problems relating to differential equations.
CubeLearn: End-to-end Learning for Human Motion Recognition from Raw mmWave Radar Signals
To avoid the drawbacks of conventional DFT pre-processing, we propose a learnable pre-processing module, named CubeLearn, to directly extract features from raw radar signal and build an end-to-end deep neural network for mmWave FMCW radar motion recognition applications.
DeepAoANet: Learning Angle of Arrival from Software Defined Radios with Deep Neural Networks
To demonstrate the utility of our approach we have collected IQ (In-phase and Quadrature components) samples from a four-element Universal Linear Array (ULA) in various Light-of-Sight (LOS) and Non-Line-of-Sight (NLOS) environments, and published the dataset.
RADA: Robust Adversarial Data Augmentation for Camera Localization in Challenging Weather
As a result, it learns to generate minimal image perturbations that are still capable of perplexing the network.
SensatUrban: Learning Semantics from Urban-Scale Photogrammetric Point Clouds
Each point in the dataset has been labelled with fine-grained semantic annotations, resulting in a dataset that is three times the size of the previous existing largest photogrammetric point cloud dataset.
Real-Time Hybrid Mapping of Populated Indoor Scenes using a Low-Cost Monocular UAV
Monocular approaches to such tasks exist, and dense monocular mapping approaches have been successfully deployed for UAV applications.
Monocular 3D Human Pose Estimation
Monocular Depth Estimation
No Pain, Big Gain: Classify Dynamic Point Cloud Sequences with Static Models by Fitting Feature-level Space-time Surfaces
Scene flow is a powerful tool for capturing the motion field of 3D point clouds.
Ranked #1 on
3D Action Recognition
on NTU RGB+D
Meta-Sampler: Almost-Universal yet Task-Oriented Sampling for Point Clouds
In this work, we propose an almost-universal sampler, in our quest for a sampler that can learn to preserve the most useful points for a particular task, yet be inexpensive to adapt to different tasks, models, or datasets.
RangeUDF: Semantic Surface Reconstruction from 3D Point Clouds
We present RangeUDF, a new implicit representation based framework to recover the geometry and semantics of continuous 3D scene surfaces from point clouds.
When the Sun Goes Down: Repairing Photometric Losses for All-Day Depth Estimation
In this paper, we show how to use a combination of three techniques to allow the existing photometric losses to work for both day and nighttime images.
Sample, Crop, Track: Self-Supervised Mobile 3D Object Detection for Urban Driving LiDAR
Supervised approaches typically require the annotation of large training sets; there has thus been great interest in leveraging weakly, semi- or self-supervised methods to avoid this, with much success.
Tracking People in Highly Dynamic Industrial Environments
We have conducted extensive real-world experiments in a construction site showing significant accuracy improvement via cross-modality training and the use of social forces.
Fusion of Radio and Camera Sensor Data for Accurate Indoor Positioning
In this paper, we propose a novel positioning system, RAVEL (Radio And Vision Enhanced Localization), which fuses anonymous visual detections captured by widely available camera infrastructure, with radio readings (e. g. WiFi radio data).
Decoupling Skill Learning from Robotic Control for Generalizable Object Manipulation
Our experiments on manipulating complex articulated objects show that the proposed approach is more generalizable to unseen objects with large intra-class variations, outperforming previous approaches.
Multi-body SE(3) Equivariance for Unsupervised Rigid Segmentation and Motion Estimation
A truly generalizable approach to rigid segmentation and motion estimation is fundamental to 3D understanding of articulated objects and moving scenes.
Fast model inference and training on-board of Satellites
In this work we demonstrate the reliable use of RaVAEn onboard a satellite, achieving an encoding time of 0. 110s for tiles of a 4. 8x4. 8 km$^2$ area.
Deep Learning for Visual Localization and Mapping: A Survey
Deep learning based localization and mapping approaches have recently emerged as a new research direction and receive significant attentions from both industry and academia.
Simultaneous Localization and Mapping
Visual Localization
+1
DynPoint: Dynamic Neural Point For View Synthesis
The introduction of neural radiance fields has greatly improved the effectiveness of view synthesis for monocular videos.
3DMiner: Discovering Shapes from Large-Scale Unannotated Image Datasets
We present 3DMiner -- a pipeline for mining 3D shapes from challenging large-scale unannotated image datasets.
Spherical Mask: Coarse-to-Fine 3D Point Cloud Instance Segmentation with Spherical Representation
Coarse-to-fine 3D instance segmentation methods show weak performances compared to recent Grouping-based, Kernel-based and Transformer-based methods.
Ranked #1 on
3D Instance Segmentation
on ScanNet(v2)
MGDepth: Motion-Guided Cost Volume For Self-Supervised Monocular Depth In Dynamic Scenarios
Despite advancements in self-supervised monocular depth estimation, challenges persist in dynamic scenarios due to the dependence on assumptions about a static world.
Learning Continuous 3D Words for Text-to-Image Generation
We do this by engineering special sets of input tokens that can be transformed in a continuous manner -- we call them Continuous 3D Words.
Gen4Gen: Generative Data Pipeline for Generative Multi-Concept Composition
First, current personalization techniques fail to reliably extend to multiple concepts -- we hypothesize this to be due to the mismatch between complex scenes and simple text descriptions in the pre-training dataset (e. g., LAION).
See, Imagine, Plan: Discovering and Hallucinating Tasks from a Single Image
Humans can not only recognize and understand the world in its current state but also envision future scenarios that extend beyond immediate perception.
WSCLoc: Weakly-Supervised Sparse-View Camera Relocalization
Despite the advancements in deep learning for camera relocalization tasks, obtaining ground truth pose labels required for the training process remains a costly endeavor.
ZeST: Zero-Shot Material Transfer from a Single Image
We propose ZeST, a method for zero-shot material transfer to an object in the input image given a material exemplar image.
