Search Results for author:
Found 213 papers, 83 papers with code
Handcrafted Outlier Detection Revisited
As a result, outlier detection is a fundamental problem in computer vision and a wide range of approaches, from simple checks based on descriptor similarity to geometric verification, have been proposed over the last decades.
Calibration-free Structure-from-Motion with Calibrated Radial Trifocal Tensors
In this paper we consider the problem of Structure-from-Motion from images with unknown intrinsic calibration.
Privacy Preserving Structure-from-Motion
The recent trend towards cloud-based localization and mapping systems has raised significant privacy concerns.
Spot-Compose: A Framework for Open-Vocabulary Object Retrieval and Drawer Manipulation in Point Clouds
This allows for accurate detection directly in 3D scenes, object- and environment-aware grasp prediction, as well as robust and repeatable robotic manipulation.
OpenNeRF: Open Set 3D Neural Scene Segmentation with Pixel-Wise Features and Rendered Novel Views
Our OpenNeRF further leverages NeRF's ability to render novel views and extract open-set VLM features from areas that are not well observed in the initial posed images.
Know Your Neighbors: Improving Single-View Reconstruction via Spatial Vision-Language Reasoning
We propose KYN, a novel method for single-view scene reconstruction that reasons about semantic and spatial context to predict each point's density.
GeneAvatar: Generic Expression-Aware Volumetric Head Avatar Editing from a Single Image
Recently, we have witnessed the explosive growth of various volumetric representations in modeling animatable head avatars.
Creating a Digital Twin of Spinal Surgery: A Proof of Concept
In this paper, we present a proof of concept (PoC) for surgery digitalization that is applied to an ex-vivo spinal surgery performed in realistic conditions.
CR3DT: Camera-RADAR Fusion for 3D Detection and Tracking
Accurate detection and tracking of surrounding objects is essential to enable self-driving vehicles.
MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images
We propose MVSplat, an efficient feed-forward 3D Gaussian Splatting model learned from sparse multi-view images.
Ranked #1 on
Generalizable Novel View Synthesis
on ACID
F$^3$Loc: Fusion and Filtering for Floorplan Localization
In this paper we propose an efficient data-driven solution to self-localization within a floorplan.
OpenSUN3D: 1st Workshop Challenge on Open-Vocabulary 3D Scene Understanding
This report provides an overview of the challenge hosted at the OpenSUN3D Workshop on Open-Vocabulary 3D Scene Understanding held in conjunction with ICCV 2023.
Sat2Scene: 3D Urban Scene Generation from Satellite Images with Diffusion
Directly generating scenes from satellite imagery offers exciting possibilities for integration into applications like games and map services.
EgoGen: An Egocentric Synthetic Data Generator
To address this challenge, we introduce EgoGen, a new synthetic data generator that can produce accurate and rich ground-truth training data for egocentric perception tasks.
NeRFmentation: NeRF-based Augmentation for Monocular Depth Estimation
In the case of MDE models for autonomous driving, this issue is exacerbated by the linearity of the captured data trajectories.
LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometry
Visual odometry estimates the motion of a moving camera based on visual input.
Segment3D: Learning Fine-Grained Class-Agnostic 3D Segmentation without Manual Labels
Therefore, we explore the use of image segmentation foundation models to automatically generate training labels for 3D segmentation.
UniSDF: Unifying Neural Representations for High-Fidelity 3D Reconstruction of Complex Scenes with Reflections
In this work, we propose UniSDF, a general purpose 3D reconstruction method that can reconstruct large complex scenes with reflections.
MuRF: Multi-Baseline Radiance Fields
We present Multi-Baseline Radiance Fields (MuRF), a general feed-forward approach to solving sparse view synthesis under multiple different baseline settings (small and large baselines, and different number of input views).
ALSTER: A Local Spatio-Temporal Expert for Online 3D Semantic Reconstruction
Using these main contributions, our method can enable scenarios with real-time constraints and can scale to arbitrary scene sizes by processing and updating the scene only in a local region defined by the new measurement.
Spherical Frustum Sparse Convolution Network for LiDAR Point Cloud Semantic Segmentation
Through the hash-based representation, we propose the Spherical Frustum sparse Convolution (SFC) and Frustum Fast Point Sampling (F2PS) to convolve and sample the points stored in spherical frustums respectively.
LABELMAKER: Automatic Semantic Label Generation from RGB-D Trajectories
Semantic annotations are indispensable to train or evaluate perception models, yet very costly to acquire.
Nothing Stands Still: A Spatiotemporal Benchmark on 3D Point Cloud Registration Under Large Geometric and Temporal Change
To this end, we introduce the Nothing Stands Still (NSS) benchmark, which focuses on the spatiotemporal registration of 3D scenes undergoing large spatial and temporal change, ultimately creating one coherent spatiotemporal map.
Long-Term Invariant Local Features via Implicit Cross-Domain Correspondences
Our proposed method enhances cross-domain localization performance, significantly reducing the performance gap.
Leveraging Neural Radiance Fields for Uncertainty-Aware Visual Localization
As a promising fashion for visual localization, scene coordinate regression (SCR) has seen tremendous progress in the past decade.
Geometry Aware Field-to-field Transformations for 3D Semantic Segmentation
We present a novel approach to perform 3D semantic segmentation solely from 2D supervision by leveraging Neural Radiance Fields (NeRFs).
Active Visual Localization for Multi-Agent Collaboration: A Data-Driven Approach
However, localizing e. g. a ground robot in the map of a drone or head-mounted MR headset presents unique challenges due to viewpoint changes.
HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World
Building an interactive AI assistant that can perceive, reason, and collaborate with humans in the real world has been a long-standing pursuit in the AI community.
Q-REG: End-to-End Trainable Point Cloud Registration with Surface Curvature
Point cloud registration has seen recent success with several learning-based methods that focus on correspondence matching and, as such, optimize only for this objective.
Handbook on Leveraging Lines for Two-View Relative Pose Estimation
We propose an approach for estimating the relative pose between calibrated image pairs by jointly exploiting points, lines, and their coincidences in a hybrid manner.
Volumetric Semantically Consistent 3D Panoptic Mapping
We introduce an online 2D-to-3D semantic instance mapping algorithm aimed at generating comprehensive, accurate, and efficient semantic 3D maps suitable for autonomous agents in unstructured environments.
CaSAR: Contact-aware Skeletal Action Recognition
Skeletal Action recognition from an egocentric view is important for applications such as interfaces in AR/VR glasses and human-robot interaction, where the device has limited resources.
Learning Disentangled Avatars with Hybrid 3D Representations
Towards this end, both explicit and implicit 3D representations are heavily studied for a holistic modeling and capture of the whole human (e. g., body, clothing, face and hair), but neither representation is an optimal choice in terms of representation efficacy since different parts of the human avatar have different modeling desiderata.
ResFields: Residual Neural Fields for Spatiotemporal Signals
Neural fields, a category of neural networks trained to represent high-frequency signals, have gained significant attention in recent years due to their impressive performance in modeling complex 3D data, such as signed distance (SDFs) or radiance fields (NeRFs), via a single multi-layer perceptron (MLP).
R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras
We propose R3D3, a multi-camera system for dense 3D reconstruction and ego-motion estimation.
Vanishing Point Estimation in Uncalibrated Images with Prior Gravity Direction
We tackle the problem of estimating a Manhattan frame, i. e. three orthogonal vanishing points, and the unknown focal length of the camera, leveraging a prior vertical direction.
RLSAC: Reinforcement Learning enhanced Sample Consensus for End-to-End Robust Estimation
Therefore, RLSAC can avoid differentiating to learn the features and the feedback of downstream tasks for end-to-end robust estimation.
Automatic registration with continuous pose updates for marker-less surgical navigation in spine surgery
An intuitive surgical guidance is provided thanks to the integration into an augmented reality based navigation system.
Quantification of Predictive Uncertainty via Inference-Time Sampling
Predictive variability due to data ambiguities has typically been addressed via construction of dedicated models with built-in probabilistic capabilities that are trained to predict uncertainty estimates as variables of interest.
AffineGlue: Joint Matching and Robust Estimation
Moreover, we derive a new minimal solver for homography estimation, requiring only a single affine correspondence (AC) and a gravity prior.
Consensus-Adaptive RANSAC
The proposed attention mechanism and one-step transformer provide an adaptive behavior that enhances the performance of RANSAC, making it a more effective tool for robust estimation.
Lazy Visual Localization via Motion Averaging
Visual (re)localization is critical for various applications in computer vision and robotics.
The Drunkard's Odometry: Estimating Camera Motion in Deforming Scenes
Estimating camera motion in deformable scenes poses a complex and open research challenge.
LightGlue: Local Feature Matching at Light Speed
We introduce LightGlue, a deep neural network that learns to match local features across images.
SNAP: Self-Supervised Neural Maps for Visual Positioning and Semantic Understanding
Semantic 2D maps are commonly used by humans and machines for navigation purposes, whether it's walking or driving.
Next-generation Surgical Navigation: Marker-less Multi-view 6DoF Pose Estimation of Surgical Instruments
Third, we evaluate three state-of-the-art single-view and multi-view methods for the task of 6DoF pose estimation of surgical instruments and analyze the influence of camera configurations, training data, and occlusions on the pose accuracy and generalization ability.
Learning-based Relational Object Matching Across Views
Our approach compares favorably to previous state-of-the-art object-level matching approaches and achieves improved performance over a pure keypoint-based approach for large view-point changes.
SGAligner : 3D Scene Alignment with Scene Graphs
We propose SGAligner, the first method for aligning pairs of 3D scene graphs that is robust to in-the-wild scenarios (ie, unknown overlap -- if any -- and changes in the environment).
Ranked #1 on
Point Cloud Registration
on 3RScan
Tracking by 3D Model Estimation of Unknown Objects in Videos
We argue that this representation is limited and instead propose to guide and improve 2D tracking with an explicit object representation, namely the textured 3D shape and 6DoF pose in each video frame.
GlueStick: Robust Image Matching by Sticking Points and Lines Together
Line segments are powerful features complementary to points.
Human from Blur: Human Pose Tracking from Blurry Images
The key idea is to tackle the inverse problem of image deblurring by modeling the forward problem with a 3D human model, a texture map, and a sequence of poses to describe human motion.
3D Line Mapping Revisited
In contrast to sparse keypoints, a handful of line segments can concisely encode the high-level scene layout, as they often delineate the main structural elements.
RegFormer: An Efficient Projection-Aware Transformer Network for Large-Scale Point Cloud Registration
Specifically, a projection-aware hierarchical transformer is proposed to capture long-range dependencies and filter outliers by extracting point features globally.
NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAM
Neural implicit representations have recently become popular in simultaneous localization and mapping (SLAM), especially in dense visual SLAM.
Four-View Geometry With Unknown Radial Distortion
The minimal case for reconstruction requires 13 points in 4 views for both the calibrated and uncalibrated cameras.
Guiding Local Feature Matching with Surface Curvature
We propose a new method, named curvature similarity extractor (CSE), for improving local feature matching across images.
SGAligner: 3D Scene Alignment with Scene Graphs
We propose SGAligner, the first method for aligning pairs of 3D scene graphs that is robust to in-the-wild scenarios (i. e., unknown overlap - if any - and changes in the environment).
Privacy Preserving Localization via Coordinate Permutations
Recent methods on privacy-preserving image-based localization use a random line parameterization to protect the privacy of query images and database maps.
Removing Objects From Neural Radiance Fields
We validate our approach using a new and still-challenging dataset for the task of NeRF inpainting.
DeepLSD: Line Segment Detection and Refinement with Deep Image Gradients
Their learned counterparts are more repeatable and can handle challenging images, but at the cost of a lower accuracy and a bias towards wireframe lines.
VolRecon: Volume Rendering of Signed Ray Distance Functions for Generalizable Multi-View Reconstruction
The success of the Neural Radiance Fields (NeRF) in novel view synthesis has inspired researchers to propose neural implicit scene reconstruction.
OpenScene: 3D Scene Understanding with Open Vocabularies
Traditional 3D scene understanding approaches rely on labeled 3D datasets to train a model for a single task with supervision.
Ranked #5 on
3D Open-Vocabulary Instance Segmentation
on Replica
3D Open-Vocabulary Instance Segmentation
3D Semantic Segmentation
+1
LaMAR: Benchmarking Localization and Mapping for Augmented Reality
To close this gap, we introduce LaMAR, a new benchmark with a comprehensive capture and GT pipeline that co-registers realistic trajectories and sensor streams captured by heterogeneous AR devices in large, unconstrained scenes.
NeuralMeshing: Differentiable Meshing of Implicit Neural Representations
The generation of triangle meshes from point clouds, i. e. meshing, is a core task in computer graphics and computer vision.
Capturing and Animation of Body and Clothing from Monocular Video
Building on this insight, we propose SCARF (Segmented Clothed Avatar Radiance Field), a hybrid model combining a mesh-based body with a neural radiance field.
IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis
Existing inverse rendering combined with neural rendering methods can only perform editable novel view synthesis on object-specific scenes, while we present intrinsic neural radiance fields, dubbed IntrinsicNeRF, which introduce intrinsic decomposition into the NeRF-based neural rendering method and can extend its application to room-scale scenes.
Learning-Based Dimensionality Reduction for Computing Compact and Effective Local Feature Descriptors
A distinctive representation of image patches in form of features is a key component of many computer vision and robotics tasks, such as image matching, image retrieval, and visual localization.
3D Textured Shape Recovery with Learned Geometric Priors
3D textured shape recovery from partial scans is crucial for many real-world applications.
Visual Localization via Few-Shot Scene Region Classification
Visual (re)localization addresses the problem of estimating the 6-DoF (Degree of Freedom) camera pose of a query image captured in a known scene, which is a key building block of many computer vision and robotics applications.
CompNVS: Novel View Synthesis with Scene Completion
We introduce a scalable framework for novel view synthesis from RGB-D images with largely incomplete scene coverage.
NeFSAC: Neurally Filtered Minimal Samples
We tested NeFSAC on more than 100k image pairs from three publicly available real-world datasets and found that it leads to one order of magnitude speed-up, while often finding more accurate results than USAC alone.
Context-Aware Sequence Alignment using 4D Skeletal Augmentation
Temporal alignment of fine-grained human actions in videos is important for numerous applications in computer vision, robotics, and mixed reality.
Spatial Computing and Intuitive Interaction: Bringing Mixed Reality and Robotics Together
Spatial computing -- the ability of devices to be aware of their surroundings and to represent this digitally -- offers novel capabilities in human-robot interaction.
Camera Pose Estimation Using Implicit Distortion Models
Low-dimensional parametric models are the de-facto standard in computer vision for intrinsic camera calibration.
Privacy Preserving Partial Localization
We propose a principled approach overcoming these limitations, based on two observations.
Learning To Find Good Models in RANSAC
We propose the Model Quality Network, MQ-Net in short, for predicting the quality, e. g. the pose error of essential matrices, of models generated inside RANSAC.
NVS-MonoDepth: Improving Monocular Depth Prediction with Novel View Synthesis
Building upon the recent progress in novel view synthesis, we propose its application to improve monocular depth estimation.
Ranked #27 on
Monocular Depth Estimation
on KITTI Eigen split
NICE-SLAM: Neural Implicit Scalable Encoding for SLAM
Neural implicit representations have recently shown encouraging results in various domains, including promising progress in simultaneous localization and mapping (SLAM).
EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devices
Key to reasoning about interactions is to understand the body pose and motion of the interaction partner from the egocentric view.
IterMVS: Iterative Probability Estimation for Efficient Multi-View Stereo
We present IterMVS, a new data-driven method for high-resolution multi-view stereo.
LatentHuman: Shape-and-Pose Disentangled Latent Representation for Human Bodies
In this work, we propose a novel neural implicit representation for the human body, which is fully differentiable and optimizable with disentangled shape and pose latent spaces.
Motion-from-Blur: 3D Shape and Motion Estimation of Motion-blurred Objects in Videos
We propose a method for jointly estimating the 3D motion, 3D shape, and appearance of highly motion-blurred objects from a video.
Learning to Align Sequential Actions in the Wild
To this end, we propose an approach to enforce temporal priors on the optimal transport matrix, which leverages temporal consistency, while allowing for variations in the order of actions.
Controllable Data Augmentation Through Deep Relighting
At the heart of the success of deep learning is the quality of the data.
Non-local Recurrent Regularization Networks for Multi-view Stereo
However, existing recurrent methods only model the local dependencies in the depth domain, which greatly limits the capability of capturing the global scene context along the depth dimension.
Reconstructing and grounding narrated instructional videos in 3D
Contrary to the standard scenario of instance-level 3D reconstruction, where identical objects or scenes are present in all views, objects in different instructional videos may have large appearance variations given varying conditions and versions of the same product.
Learning Motion Priors for 4D Human Body Capture in 3D Scenes
To prove the effectiveness of the proposed motion priors, we combine them into a novel pipeline for 4D human body capture in 3D scenes.
Pixel-Perfect Structure-from-Motion with Featuremetric Refinement
Finding local features that are repeatable across multiple views is a cornerstone of sparse 3D reconstruction.
Privacy Preserving Localization and Mapping From Uncalibrated Cameras
In this paper, we propose a solution to the uncalibrated privacy preserving localization and mapping problem.
Shape from Blur: Recovering Textured 3D Shape and Motion of Fast Moving Objects
We address the novel task of jointly reconstructing the 3D shape, texture, and motion of an object from a single motion-blurred image.
Shape As Points: A Differentiable Poisson Solver
However, the implicit nature of neural implicit representations results in slow inference time and requires careful initialization.
H2O: Two Hands Manipulating Objects for First Person Interaction Recognition
To this end, we propose a method to create a unified dataset for egocentric 3D interaction recognition.
Towards Efficient Graph Convolutional Networks for Point Cloud Handling
In this paper, we aim at improving the computational efficiency of graph convolutional networks (GCNs) for learning on point clouds.
SOLD2: Self-supervised Occlusion-aware Line Description and Detection
We thus hereby introduce the first joint detection and description of line segments in a single deep network.
MBA-VO: Motion Blur Aware Visual Odometry
Motion blur is one of the major challenges remaining for visual odometry methods.
Back to the Feature: Learning Robust Camera Localization from Pixels to Pose
In this paper, we go Back to the Feature: we argue that deep networks should focus on learning robust and invariant visual features, while the geometric estimation should be left to principled algorithms.
Holistic 3D Scene Understanding from a Single Image with Implicit Representation
We not only propose an image-based local structured implicit network to improve the object shape estimation, but also refine the 3D object pose and scene layout via a novel implicit scene graph neural network that exploits the implicit local object features.
Ranked #1 on
Monocular 3D Object Detection
on SUN RGB-D
(using extra training data)
Localizing Unsynchronized Sensors with Unknown Sources
We look at the general case where neither the emission times of the sources nor the reference time frames of the receivers are known.
The Card Shuffling Hypotheses: Building a Time and Memory Efficient Graph Convolutional Network
State-of-the-art GCNs adopt $K$-nearest neighbor (KNN) searches for local feature aggregation and feature extraction operations from layer to layer.
Orthographic-Perspective Epipolar Geometry
In this paper we consider the epipolar geometry between orthographic and perspective cameras.
DeepSurfels: Learning Online Appearance Fusion
We present DeepSurfels, a novel hybrid scene representation for geometry and appearance information.
CodeVIO: Visual-Inertial Odometry with Learned Optimizable Dense Depth
In this work, we present a lightweight, tightly-coupled deep depth network and visual-inertial odometry (VIO) system, which can provide accurate state estimates and dense depth maps of the immediate surroundings.
FMODetect: Robust Detection of Fast Moving Objects
Compared to other methods, such as deblatting, the inference is of several orders of magnitude faster and allows applications such as real-time fast moving object detection and retrieval in large video collections.
Sat2Vid: Street-view Panoramic Video Synthesis from a Single Satellite Image
For geometrical and temporal consistency, our approach explicitly creates a 3D point cloud representation of the scene and maintains dense 3D-2D correspondences across frames that reflect the geometric scene configuration inferred from the satellite view.
DeepVideoMVS: Multi-View Stereo on Video with Recurrent Spatio-Temporal Fusion
We propose an online multi-view depth prediction approach on posed video streams, where the scene geometry information computed in the previous time steps is propagated to the current time step in an efficient and geometrically plausible way.
Cross-Descriptor Visual Localization and Mapping
Visual localization and mapping is the key technology underlying the majority of mixed reality and robotics systems.
PatchmatchNet: Learned Multi-View Patchmatch Stereo
We present PatchmatchNet, a novel and learnable cascade formulation of Patchmatch for high-resolution multi-view stereo.
Ranked #10 on
Point Clouds
on Tanks and Temples
DeFMO: Deblurring and Shape Recovery of Fast Moving Objects
We propose a method that, given a single image with its estimated background, outputs the object's appearance and position in a series of sub-frames as if captured by a high-speed camera (i. e. temporal super-resolution).
Ranked #1 on
Video Super-Resolution
on Falling Objects
NeuralFusion: Online Depth Fusion in Latent Space
We present a novel online depth map fusion approach that learns depth map aggregation in a latent feature space.
Freetures: Localization in Signed Distance Function Maps
Localization of a robotic system within a previously mapped environment is important for reducing estimation drift and for reusing previously built maps.
Robotics
Weakly Supervised Learning of Multi-Object 3D Scene Decompositions Using Deep Shape Priors
Our approach learns to decompose images of synthetic scenes with multiple objects on a planar surface into its constituent scene objects and to infer their 3D properties from a single view.
Probabilistic 3D surface reconstruction from sparse MRI information
Surface reconstruction from magnetic resonance (MR) imaging data is indispensable in medical image analysis and clinical research.
Semi-Supervised Learning of Multi-Object 3D Scene Representations
By differentiable rendering, we train our model to decompose scenes self-supervised from RGB-D images.
Self-Supervised Learning of Non-Rigid Residual Flow and Ego-Motion
Most of the current scene flow methods choose to model scene flow as a per point translation vector without differentiating between static and dynamic components of 3D motion.
HoloLens 2 Research Mode as a Tool for Computer Vision Research
Mixed reality headsets, such as the Microsoft HoloLens 2, are powerful sensing devices with integrated compute capabilities, which makes it an ideal platform for computer vision research.
LIC-Fusion 2.0: LiDAR-Inertial-Camera Odometry with Sliding-Window Plane-Feature Tracking
Only the tracked planar points belonging to the same plane will be used for plane initialization, which makes the plane extraction efficient and robust.
Robotics
KAPLAN: A 3D Point Descriptor for Shape Completion
We present a novel 3D shape completion method that operates directly on unstructured point clouds, thus avoiding resource-intensive data structures like voxel grids.
Infrastructure-based Multi-Camera Calibration using Radial Projections
In particular, our approach is more robust than the naive approach of first estimating intrinsic parameters and pose per camera before refining the extrinsic parameters of the system.
Online Invariance Selection for Local Feature Descriptors
To be invariant, or not to be invariant: that is the question formulated in this work about local descriptors.
Privacy-Preserving Image Features via Adversarial Affine Subspace Embeddings
Many computer vision systems require users to upload image features to the cloud for processing and storage.
AdaLAM: Revisiting Handcrafted Outlier Detection
Local feature matching is a critical component of many computer vision pipelines, including among others Structure-from-Motion, SLAM, and Visual Localization.
Self-Supervised Human Depth Estimation from Monocular Videos
Previous methods on estimating detailed human depth often require supervised training with `ground truth' depth data.
Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstruction
Modeling hand-object manipulations is essential for understanding how humans interact with their environment.
Ranked #9 on
hand-object pose
on HO-3D
OmniSLAM: Omnidirectional Localization and Dense Mapping for Wide-baseline Multi-camera Systems
In this paper, we present an omnidirectional localization and dense mapping system for a wide-baseline multiview stereo setup with ultra-wide field-of-view (FOV) fisheye cameras, which has a 360 degrees coverage of stereo observations of the environment.
Multi-View Optimization of Local Feature Geometry
In this work, we address the problem of refining the geometry of local image features from multiple views without known scene or camera geometry.
Convolutional Occupancy Networks
Recently, implicit neural representations have gained popularity for learning-based 3D reconstruction.
Self-Supervised Linear Motion Deblurring
Motion blurry images challenge many computer vision algorithms, e. g, feature detection, motion estimation, or object recognition.
Aerial Single-View Depth Completion with Image-Guided Uncertainty Estimation
In this paper, we propose a depth completion and uncertainty estimation approach that better handles the challenges of aerial platforms, such as large viewpoint and depth variations, and limited computing resources.
Learned Multi-View Texture Super-Resolution
We present a super-resolution method capable of creating a high-resolution texture map for a virtual 3D object from a set of lower-resolution images of that object.
RoutedFusion: Learning Real-time Depth Map Fusion
To this end, we present a novel real-time capable machine learning-based method for depth map fusion.
Why Having 10,000 Parameters in Your Camera Model is Better Than Twelve
In contrast, generic camera models allow for very accurate calibration due to their flexibility.
Reflection Separation using a Pair of Unpolarized and Polarized Images
When we take photos through glass windows or doors, the transmitted background scene is often blended with undesirable reflection.
DIST: Rendering Deep Implicit Signed Distance Function with Differentiable Sphere Tracing
We propose a differentiable sphere tracing algorithm to bridge the gap between inverse graphics methods and the recently proposed deep learning based implicit signed distance function.
Slanted Stixels: A way to represent steep streets
This work presents and evaluates a novel compact scene representation based on Stixels that infers geometric and semantic information.
Learned Semantic Multi-Sensor Depth Map Fusion
Our method learns sensor or algorithm properties jointly with semantic depth fusion and scene completion and can also be used as an expert system, e. g. to unify the strengths of various photometric stereo algorithms.
To Learn or Not to Learn: Visual Localization from Essential Matrices
Using a classical feature-based approach within this framework, we show state-of-the-art performance.
Large-scale, real-time visual-inertial localization revisited
Our approach spans from offline model building to real-time client-side pose fusion.
Discrete Optimization of Ray Potentials for Semantic 3D Reconstruction
The depth and semantic information is incorporated as a unary potential, smoothed by a pairwise regularizer.
3D Instance Segmentation via Multi-Task Metric Learning
The second goal is to learn instance information by densely estimating directional information of the instance's center of mass for each voxel.
Ranked #2 on
3D Semantic Instance Segmentation
on ScanNetV2
3D Appearance Super-Resolution with Deep Learning
Experimental results demonstrate that our proposed networks successfully incorporate the 3D geometric information and super-resolve the texture maps.
D2-Net: A Trainable CNN for Joint Detection and Description of Local Features
In this work we address the problem of finding reliable pixel-level correspondences under difficult imaging conditions.
Ranked #8 on
Image Matching
on IMC PhotoTourism
Robust Dense Mapping for Large-Scale Dynamic Environments
We use both instance-aware semantic segmentation and sparse scene flow to classify objects as either background, moving, or potentially moving, thereby ensuring that the system is able to model objects with the potential to transition from static to dynamic, such as parked cars.
H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactions
Given a single RGB image, our model jointly estimates the 3D hand and object poses, models their interactions, and recognizes the object and action classes with a single feed-forward pass through a neural network.
Understanding the Limitations of CNN-based Absolute Camera Pose Regression
We furthermore use our model to show that pose regression is more closely related to pose approximation via image retrieval than to accurate pose estimation via 3D structure.
Privacy Preserving Image-Based Localization
Current localization systems rely on the persistent storage of 3D point clouds of the scene to enable camera pose estimation, but such data reveals potentially sensitive scene information.
Incremental Visual-Inertial 3D Mesh Generation with Structural Regularities
We propose instead to tightly couple mesh regularization and state estimation by detecting and enforcing structural regularities in a novel factor-graph formulation.
DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image
In this paper, we propose a deep learning architecture that produces accurate dense depth for the outdoor scene from a single color image and a sparse depth.
DGC-Net: Dense Geometric Correspondence Network
This paper addresses the challenge of dense pixel correspondence estimation between two images.
Ranked #2 on
Dense Pixel Correspondence Estimation
on HPatches
Dense Pixel Correspondence Estimation
Optical Flow Estimation
+1
Episodic Curiosity through Reachability
One solution to this problem is to allow the agent to create rewards for itself - thus making rewards dense and more suitable for learning.
SurfelMeshing: Online Surfel-Based Mesh Reconstruction
In contrast to most existing approaches, we do not fuse depth measurements in a volume but in a dense surfel cloud.
Night-to-Day Image Translation for Retrieval-based Localization
We then compare the daytime and translated night images to obtain a pose estimate for the night image using the known 6-DOF position of the closest day image.
Efficient 2D-3D Matching for Multi-Camera Visual Localization
This results in a system that provides reliable and drift-less pose estimations for high speed autonomous driving.
Robotics
A Dataset of Flash and Ambient Illumination Pairs from the Crowd
We present a dataset of thousands of ambient and flash illumination pairs to enable studying flash photography and other applications that can benefit from having separate illuminations.
Learning Priors for Semantic 3D Reconstruction
In contrast to existing variational methods for semantic 3D reconstruction, our model is end-to-end trainable and captures more complex dependencies between the semantic labels and the 3D geometry.
VSO: Visual Semantic Odometry
Robust data association is a core problem of visual odometry, where image-to-image correspondences provide constraints for camera pose and map estimation.
Semantic Match Consistency for Long-Term Visual Localization
Robust and accurate visual localization across large appearance variations due to changes in time of day, seasons, or changes of the environment is a challenging problem which is of importance to application areas such as navigation of autonomous robots.
Uncertainty Quantification in CNN-Based Surface Prediction Using Shape Priors
Surface reconstruction is a vital tool in a wide range of areas of medical image analysis and clinical research.
Hybrid Scene Compression for Visual Localization
Besides outperforming previous compression techniques in terms of pose accuracy under the same memory constraints, our compression scheme itself is also more efficient.
Augmenting Crowd-Sourced 3D Reconstructions Using Semantic Detections
Image-based 3D reconstruction for Internet photo collections has become a robust technology to produce impressive virtual representations of real-world scenes.
Consensus Maximization for Semantic Region Correspondences
We propose a novel method for the geometric registration of semantically labeled regions.
Hybrid Camera Pose Estimation
A number of these new hybrid minimal solvers are also presented in this paper.
InLoc: Indoor Visual Localization with Dense Matching and View Synthesis
We seek to predict the 6 degree-of-freedom (6DoF) pose of a query photograph with respect to a large indoor 3D map.
Semantic Visual Localization
Robust visual localization under a wide range of viewing conditions is a fundamental problem in computer vision.
From Point Clouds to Mesh Using Regression
Surface reconstruction from a point cloud is a standard subproblem in many algorithms for dense 3D reconstruction from RGB images or depth maps.
An Exploration of 2D and 3D Deep Learning Techniques for Cardiac MR Image Segmentation
Accurate segmentation of the heart is an important step towards evaluating cardiac function.
3D Visual Perception for Self-Driving Cars using a Multi-Camera System: Calibration, Mapping, Localization, and Obstacle Detection
To minimize the number of cameras needed for surround perception, we utilize fisheye cameras.
Benchmarking 6DOF Outdoor Visual Localization in Changing Conditions
Visual localization enables autonomous vehicles to navigate in their surroundings and augmented reality applications to link virtual to real worlds.
Information-Flow Matting
Our resulting novel linear system formulation can be solved in closed-form and is robust against several fundamental challenges of natural matting such as holes and remote intricate structures.
Slanted Stixels: Representing San Francisco's Steepest Streets
In this work we present a novel compact scene representation based on Stixels that infers geometric and semantic information.
Designing Effective Inter-Pixel Information Flow for Natural Image Matting
Our resulting novel linear system formulation can be solved in closed-form and is robust against several fundamental challenges in natural matting such as holes and remote intricate structures.
Consensus Maximization With Linear Matrix Inequality Constraints
While randomized methods like RANSAC are fast, they do not guarantee global optimality and fail to manage large amounts of outliers.
SGM-Nets: Semi-Global Matching With Neural Networks
Moreover, we propose a novel SGM parameterization, which deploys different penalties depending on either positive or negative disparity changes in order to represent the object structures more discriminatively.
Are Large-Scale 3D Models Really Necessary for Accurate Visual Localization?
3D structure-based methods employ 3D models of the scene to estimate the full 6DOF pose of a camera very accurately.
A Multi-View Stereo Benchmark With High-Resolution Images and Multi-Camera Videos
Motivated by the limitations of existing multi-view stereo benchmarks, we present a novel dataset for this task.
Fast 3D Reconstruction of Faces With Glasses
We present a method for the fast 3D face reconstruction of people wearing glasses.
Toroidal Constraints for Two-Point Localization Under High Outlier Ratios
Adding the knowledge of direction of triangulation, we are able to approximate the position of the camera from two matches alone.
Comparative Evaluation of Hand-Crafted and Learned Local Features
In terms of matching performance, we evaluate the different descriptors regarding standard criteria. However, considering matching performance in isolation only provides an incomplete measure of a descriptor’s quality.
Semantically Informed Multiview Surface Refinement
We present a method to jointly refine the geometry and semantic segmentation of 3D surface meshes.
Matching neural paths: transfer from recognition to correspondence search
We propose to use a hierarchical semantic representation of the objects, coming from a convolutional neural network, to solve this ambiguity.
Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark
With the massive data set presented in this paper, we aim at closing this data gap to help unleash the full potential of deep learning methods for 3D labelling tasks.
The Stixel world: A medium-level representation of traffic scenes
We believe this challenge should be faced by introducing a representation of the sensory data that provides compressed and structured access to all relevant visual content of the scene.
Quad-networks: unsupervised learning to rank for interest point detection
In this paper, we ask a fundamental question: can we learn such detectors from scratch?
Semantically Guided Depth Upsampling
We present a novel method for accurate and efficient up- sampling of sparse depth data, guided by high-resolution imagery.
Do It Yourself Hyperspectral Imaging With Everyday Digital Cameras
In particular, due to the differences in spectral sensitivities of the cameras, different cameras yield different RGB measurements for the same spectral signal.
Large-Scale Location Recognition and the Geometric Burstiness Problem
Visual location recognition is the task of determining the place depicted in a query image from a given database of geo-tagged images.
Sparse to Dense 3D Reconstruction From Rolling Shutter Images
It is well known that the rolling shutter effect in images captured with a moving rolling shutter camera causes inaccuracies to 3D reconstructions.
Automatic 3D Reconstruction of Manifold Meshes via Delaunay Triangulation and Mesh Sweeping
In this paper we propose a new approach to incrementally initialize a manifold surface for automatic 3D reconstruction from images.
TI-POOLING: transformation-invariant pooling for feature learning in Convolutional Neural Networks
This more efficient use of training data results in better performance on popular benchmark datasets with smaller number of parameters when comparing to standard convolutional neural networks with dataset augmentation and to other baselines.
Semantic 3D Reconstruction with Continuous Regularization and Ray Potentials Using a Visibility Consistency Constraint
We propose an approach for dense semantic 3D reconstruction which uses a data term that is defined as potentials over viewing rays, combined with continuous surface area penalization.
Optimizing the Viewing Graph for Structure-From-Motion
The viewing graph represents a set of views that are related by pairwise relative geometries.
Entropy Minimization for Convex Relaxation Approaches
Despite their enormous success in solving hard combinatorial problems, convex relaxation approaches often suffer from the fact that the computed solutions are far from binary and that subsequent heuristic binarization may substantially degrade the quality of computed solutions.
Non-Parametric Structure-Based Calibration of Radially Symmetric Cameras
As a second step, we obtain the calibration by finding the translation of the camera center using an ordering constraint.
Merging the Unmatchable: Stitching Visually Disconnected SfM Models
An important variant of this problem is the case in which individual sides of a building can be reconstructed but not joined due to the missing visual overlap.
Camera Pose Voting for Large-Scale Image-Based Localization
Image-based localization approaches aim to determine the camera pose from which an image was taken.
Capturing Hands in Action using Discriminative Salient Points and Physics Simulation
Hand motion capture is a popular research field, recently gaining more attention due to the ubiquity of RGB-D sensors.
Scalable Structure From Motion for Densely Sampled Videos
Videos consisting of thousands of high resolution frames are challenging for existing structure from motion (SfM) and simultaneous-localization and mapping (SLAM) techniques.
Segment Based 3D Object Shape Priors
This leads to a semantic segmentation as a side product of our shape prior formulation.
Direction Matters: Depth Estimation With a Surface Normal Classifier
In this work we make use of recent advances in data driven classification to improve standard approaches for binocular stereo matching and single view depth estimation.
Learning the Matching Function
In this paper we propose a method, which learns the matching function, that automatically finds the space of allowed changes in visual appearance, such as due to the motion blur, chromatic distortions, different colour calibration or seasonal changes.
