no code implementations • 28 Feb 2023 • Ji Hou, Xiaoliang Dai, Zijian He, Angela Dai, Matthias Nießner
Current popular backbones in computer vision, such as Vision Transformers (ViT) and ResNets are trained to perceive the world from 2D images.
no code implementations • 7 Feb 2023 • Junwen Huang, Alexey Artemov, Yujin Chen, Shuaifeng Zhi, Kai Xu, Matthias Nießner
To this end, we design a trainable model that employs both incomplete 3D reconstructions and their corresponding source RGB-D images, fusing cross-domain features into volumetric embeddings to predict complete 3D geometry, color, and semantics.
no code implementations • 19 Dec 2022 • Yawar Siddiqui, Lorenzo Porzi, Samuel Rota Buló, Norman Müller, Matthias Nießner, Angela Dai, Peter Kontschieder
We propose Panoptic Lifting, a novel approach for learning panoptic 3D volumetric representations from images of in-the-wild scenes.
1 code implementation • 18 Dec 2022 • Abdullah Hamdi, Bernard Ghanem, Matthias Nießner
SuRFNet employs partial SRFs from few/one images and a specialized SRF loss to learn to generate high-quality sparse voxel radiance fields that can be rendered from novel views.
no code implementations • 6 Dec 2022 • Simon Giebenhain, Tobias Kirschstein, Markos Georgopoulos, Martin Rünz, Lourdes Agapito, Matthias Nießner
We propose a novel 3D morphable model for complete human heads based on hybrid neural fields.
no code implementations • 5 Dec 2022 • Can Gümeli, Angela Dai, Matthias Nießner
In a pairwise setting, our method improves registration recall of state-of-the-art feature matching from 77% to 87% overall and from 21% to 52% in pairs with 10% or less inter-frame overlap.
no code implementations • 2 Dec 2022 • Norman Müller, Yawar Siddiqui, Lorenzo Porzi, Samuel Rota Bulò, Peter Kontschieder, Matthias Nießner
We introduce DiffRF, a novel approach for 3D radiance field synthesis based on denoising diffusion probabilistic models.
no code implementations • 2 Dec 2022 • Shivangi Aneja, Justus Thies, Angela Dai, Matthias Nießner
Controllable editing and manipulation are given by language prompts to adapt texture and expression of the 3D morphable model.
no code implementations • 1 Dec 2022 • Dave Zhenyu Chen, Ronghang Hu, Xinlei Chen, Matthias Nießner, Angel X. Chang
Performing 3D dense captioning and visual grounding requires a common and shared understanding of the underlying multimodal relationships.
no code implementations • 25 Nov 2022 • Yinyu Nie, Angela Dai, Xiaoguang Han, Matthias Nießner
Holistic 3D scene understanding entails estimation of both layout configuration and object geometry in a 3D environment.
no code implementations • 25 Nov 2022 • Angela Dai, Matthias Nießner
Implicit neural field generating signed distance field representations (SDFs) of 3D shapes have shown remarkable progress in 3D shape reconstruction and generation.
1 code implementation • 18 Oct 2022 • Artem Sevastopolsky, Yury Malkov, Nikita Durasov, Luisa Verdoliva, Matthias Nießner
Our self-supervised strategy is the most useful with limited amounts of labeled training data, which can be beneficial for more tailored face recognition tasks and when facing privacy concerns.
no code implementations • 11 Oct 2022 • Jiapeng Tang, Lev Markhasin, Bi Wang, Justus Thies, Matthias Nießner
To this end, we introduce transformer-based deformation networks that represent a shape deformation as a composition of local surface deformations.
no code implementations • 11 Oct 2022 • Peter Kocsis, Peter Súkeník, Guillem Brasó, Matthias Nießner, Laura Leal-Taixé, Ismail Elezi
This allows us to improve the generalization of a CNN-based model without any increase in the number of weights at test time.
no code implementations • 28 Jun 2022 • Dominik Schmauser, Zeju Qiu, Norman Müller, Matthias Nießner
We propose a novel approach for joint 3D multi-object tracking and reconstruction from RGB-D sequences in indoor environments.
1 code implementation • 30 May 2022 • Jiemin Fang, Taoran Yi, Xinggang Wang, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Matthias Nießner, Qi Tian
A multi-distance interpolation method is proposed and applied on voxel features to model both small and large motions.
1 code implementation • 27 May 2022 • Biao Zhang, Matthias Nießner, Peter Wonka
All probabilistic experiments confirm that we are able to generate detailed and high quality shapes to yield the new state of the art in generative 3D shape modeling.
no code implementations • 3 May 2022 • Barbara Roessle, Matthias Nießner
Training feature matching with gradients from pose optimization naturally learns to down-weight outliers and boosts pose estimation on image pairs compared to SuperGlue by 6. 7% on ScanNet.
no code implementations • CVPR 2022 • Norman Müller, Andrea Simonelli, Lorenzo Porzi, Samuel Rota Bulò, Matthias Nießner, Peter Kontschieder
We introduce AutoRF - a new approach for learning neural 3D object representations where each object in the training set is observed by only a single view.
1 code implementation • 6 Apr 2022 • Davide Cozzolino, Matthias Nießner, Luisa Verdoliva
The aim of this work is to propose a deepfake detector that can cope with the wide variety of manipulation methods and scenarios encountered in the real world.
no code implementations • 5 Apr 2022 • Yawar Siddiqui, Justus Thies, Fangchang Ma, Qi Shan, Matthias Nießner, Angela Dai
Texture cues on 3D objects are key to compelling visual representations, with the possibility to create high visual fidelity with inherent spatial consistency across different views.
no code implementations • 31 Mar 2022 • Yunlu Chen, Basura Fernando, Hakan Bilen, Matthias Nießner, Efstratios Gavves
In this work, we address two key limitations of such representations, in failing to capture local 3D geometric fine details, and to learn from and generalize to shapes with unseen 3D transformations.
1 code implementation • 8 Mar 2022 • Di Chang, Aljaž Božič, Tong Zhang, Qingsong Yan, Yingcong Chen, Sabine Süsstrunk, Matthias Nießner
Finding accurate correspondences among different views is the Achilles' heel of unsupervised Multi-View Stereo (MVS).
no code implementations • 6 Dec 2021 • Yujin Chen, Matthias Nießner, Angela Dai
We present a new approach to instill 4D dynamic object priors into learned 3D representations by unsupervised pre-training.
Ranked #14 on
3D Instance Segmentation
on ScanNet(v2)
(mAP @ 50 metric)
1 code implementation • CVPR 2022 • Barbara Roessle, Jonathan T. Barron, Ben Mildenhall, Pratul P. Srinivasan, Matthias Nießner
To this end, we leverage dense depth priors in order to constrain the NeRF optimization.
1 code implementation • CVPR 2022 • Can Gümeli, Angela Dai, Matthias Nießner
We present ROCA, a novel end-to-end approach that retrieves and aligns 3D CAD models from a shape database to a single input image.
3D Dense Shape Correspondence
3D Object Detection From Monocular Images
+2
1 code implementation • CVPR 2022 • Lukas Höllein, Justin Johnson, Matthias Nießner
Style transfer typically operates on 2D images, making stylization of a mesh challenging.
no code implementations • 2 Dec 2021 • Dave Zhenyu Chen, Qirui Wu, Matthias Nießner, Angel X. Chang
Our D3Net unifies dense captioning and visual grounding in 3D in a self-critical manner.
no code implementations • CVPR 2022 • Philip-William Grassal, Malte Prinzler, Titus Leistner, Carsten Rother, Matthias Nießner, Justus Thies
We present Neural Head Avatars, a novel neural representation that explicitly models the surface geometry and appearance of an animatable human avatar that can be used for teleconferencing in AR/VR or other applications in the movie or games industry that rely on a digital human.
no code implementations • 1 Dec 2021 • Yinyu Nie, Angela Dai, Xiaoguang Han, Matthias Nießner
To this end, we propose P2R-Net to learn a probabilistic 3D model of the objects in a scene characterized by their class categories and oriented 3D bounding boxes, based on an input observed human trajectory in the environment.
1 code implementation • NeurIPS 2021 • Manuel Dahnert, Ji Hou, Matthias Nießner, Angela Dai
Inspired by 2D panoptic segmentation, we propose to unify the tasks of geometric reconstruction, 3D semantic segmentation, and 3D instance segmentation into the task of panoptic 3D scene reconstruction - from a single RGB image, predicting the complete geometric reconstruction of the scene in the camera frustum of the image, along with semantic and instance segmentations.
no code implementations • 7 Jul 2021 • Mohamed Elgharib, Mohit Mendiratta, Justus Thies, Matthias Nießner, Hans-Peter Seidel, Ayush Tewari, Vladislav Golyanik, Christian Theobalt
Even holding a mobile phone camera in the front of the face while sitting for a long duration is not convenient.
1 code implementation • NeurIPS 2021 • Aljaž Božič, Pablo Palafox, Justus Thies, Angela Dai, Matthias Nießner
We introduce TransformerFusion, a transformer-based 3D scene reconstruction approach.
1 code implementation • ICCV 2021 • Yang Li, Hikari Takehara, Takafumi Taketomi, Bo Zheng, Matthias Nießner
Tracking non-rigidly deforming scenes using range sensors has numerous applications including computer vision, AR/VR, and robotics.
1 code implementation • ICCV 2021 • Ji Hou, Saining Xie, Benjamin Graham, Angela Dai, Matthias Nießner
Inspired by these advances in geometric understanding, we aim to imbue image-based perception with representations learned under geometric constraints.
2 code implementations • CVPR 2022 • Dejan Azinović, Ricardo Martin-Brualla, Dan B Goldman, Matthias Nießner, Justus Thies
Obtaining high-quality 3D reconstructions of room-scale scenes is of paramount importance for upcoming applications in AR or VR.
1 code implementation • ICCV 2021 • Andrei Burov, Matthias Nießner, Justus Thies
To this end, we explicitly model the surface of the person using a multi-layer perceptron (MLP) which is embedded into the canonical space of the SMPL body model.
1 code implementation • ICCV 2021 • Pablo Palafox, Aljaž Božič, Justus Thies, Matthias Nießner, Angela Dai
Crucially, once learned, our neural parametric models of shape and pose enable optimization over the learned spaces to fit to new observations, similar to the fitting of a traditional parametric model, e. g., SMPL.
1 code implementation • ICCV 2021 • Yawar Siddiqui, Justus Thies, Fangchang Ma, Qi Shan, Matthias Nießner, Angela Dai
3D reconstruction of large scenes is a challenging problem due to the high-complexity nature of the solution space, in particular for generative neural networks.
3 code implementations • 15 Jan 2021 • Shivangi Aneja, Chris Bregler, Matthias Nießner
We propose a self-supervised training strategy where we only need a set of captioned images.
2 code implementations • 17 Dec 2020 • Xinpeng Wang, Chandan Yeshwanth, Matthias Nießner
In contrast, we do not use any appearance information, and implicitly learn object relations using the self-attention mechanism of transformers.
1 code implementation • CVPR 2021 • Ji Hou, Benjamin Graham, Matthias Nießner, Saining Xie
The rapid progress in 3D scene understanding has come with growing demand for data; however, collecting and annotating 3D scenes (e. g. point clouds) are notoriously hard.
Ranked #2 on
3D Semantic Segmentation
on ScanNet200
no code implementations • CVPR 2021 • Norman Müller, Yu-Shiang Wong, Niloy J. Mitra, Angela Dai, Matthias Nießner
From a sequence of RGB-D frames, we detect objects in each frame and learn to predict their complete object geometry as well as a dense correspondence mapping into a canonical space.
1 code implementation • 8 Dec 2020 • Kevis-Kokitsi Maninis, Stefan Popov, Matthias Nießner, Vittorio Ferrari
We address the task of aligning CAD models to a video sequence of a complex scene containing multiple objects.
1 code implementation • CVPR 2021 • Guy Gafni, Justus Thies, Michael Zollhöfer, Matthias Nießner
We present dynamic neural radiance fields for modeling the appearance and dynamics of a human face.
1 code implementation • ICCV 2021 • Davide Cozzolino, Andreas Rössler, Justus Thies, Matthias Nießner, Luisa Verdoliva
A major challenge in DeepFake forgery detection is that state-of-the-art algorithms are mostly trained to detect a specific fake method.
no code implementations • CVPR 2021 • Dave Zhenyu Chen, Ali Gholami, Matthias Nießner, Angel X. Chang
We introduce the task of dense captioning in 3D scans from commodity RGB-D sensors.
1 code implementation • CVPR 2021 • Aljaž Božič, Pablo Palafox, Michael Zollhöfer, Justus Thies, Angela Dai, Matthias Nießner
We introduce Neural Deformation Graphs for globally-consistent deformation tracking and 3D reconstruction of non-rigid objects.
1 code implementation • CVPR 2021 • Yinyu Nie, Ji Hou, Xiaoguang Han, Matthias Nießner
In this work, we introduce RfD-Net that jointly detects and reconstructs dense object surfaces directly from raw point clouds.
no code implementations • CVPR 2016 • Justus Thies, Michael Zollhöfer, Marc Stamminger, Christian Theobalt, Matthias Nießner
Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion.
no code implementations • 29 Jun 2020 • Hassan Abu Alhaija, Siva Karthik Mustikovela, Justus Thies, Varun Jampani, Matthias Nießner, Andreas Geiger, Carsten Rother
Neural rendering techniques promise efficient photo-realistic image synthesis while at the same time providing rich control over scene parameters by learning the physical image formation process.
1 code implementation • CVPR 2021 • Angela Dai, Yawar Siddiqui, Justus Thies, Julien Valentin, Matthias Nießner
We present SPSG, a novel approach to generate high-quality, colored 3D models of scenes from RGB-D scan observations by learning to infer unobserved scene geometry and color in a self-supervised fashion.
1 code implementation • NeurIPS 2020 • Aljaž Božič, Pablo Palafox, Michael Zollhöfer, Angela Dai, Justus Thies, Matthias Nießner
We introduce a novel, end-to-end learnable, differentiable non-rigid tracker that enables state-of-the-art non-rigid reconstruction by a learned robust optimization.
no code implementations • 21 Jun 2020 • Shivangi Aneja, Matthias Nießner
We propose Deep Distribution Transfer(DDT), a new transfer learning approach to address the problem of zero and few-shot transfer in the context of facial forgery detection.
no code implementations • 14 Jun 2020 • Andong Tan, Duc Tam Nguyen, Maximilian Dax, Matthias Nießner, Thomas Brox
Self-attention networks have shown remarkable progress in computer vision tasks such as image classification.
no code implementations • 8 Apr 2020 • Ayush Tewari, Ohad Fried, Justus Thies, Vincent Sitzmann, Stephen Lombardi, Kalyan Sunkavalli, Ricardo Martin-Brualla, Tomas Simon, Jason Saragih, Matthias Nießner, Rohit Pandey, Sean Fanello, Gordon Wetzstein, Jun-Yan Zhu, Christian Theobalt, Maneesh Agrawala, Eli Shechtman, Dan B. Goldman, Michael Zollhöfer
Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e. g., by the integration of differentiable rendering into network training.
1 code implementation • 30 Mar 2020 • Francis Engelmann, Martin Bokeloh, Alireza Fathi, Bastian Leibe, Matthias Nießner
We show that grouping proposals improves over NMS and outperforms previous state-of-the-art methods on the tasks of 3D object detection and semantic instance segmentation on the ScanNetV2 benchmark and the S3DIS dataset.
Ranked #1 on
3D Semantic Instance Segmentation
on ScanNetV2
no code implementations • CVPR 2020 • Yang Li, Aljaž Božič, Tianwei Zhang, Yanli Ji, Tatsuya Harada, Matthias Nießner
One of the widespread solutions for non-rigid tracking has a nested-loop structure: with Gauss-Newton to minimize a tracking objective in the outer loop, and Preconditioned Conjugate Gradient (PCG) to solve a sparse linear system in the inner loop.
no code implementations • ECCV 2020 • Armen Avetisyan, Tatiana Khanova, Christopher Choy, Denver Dash, Angela Dai, Matthias Nießner
We present a novel approach to reconstructing lightweight, CAD-based representations of scanned 3D environments from commodity RGB-D sensors.
2 code implementations • ECCV 2020 • Cheng Lin, Tingxiang Fan, Wenping Wang, Matthias Nießner
We explore how to enable machines to model 3D shapes like human modelers using deep reinforcement learning (RL).
1 code implementation • 19 Mar 2020 • Chiyu Max Jiang, Avneesh Sud, Ameesh Makadia, Jingwei Huang, Matthias Nießner, Thomas Funkhouser
Then, we use the decoder as a component in a shape optimization that solves for a set of latent codes on a regular grid of overlapping crops such that an interpolation of the decoded local shapes matches a partial or noisy observation.
1 code implementation • CVPR 2020 • Jingwei Huang, Justus Thies, Angela Dai, Abhijit Kundu, Chiyu Max Jiang, Leonidas Guibas, Matthias Nießner, Thomas Funkhouser
In this work, we present a novel approach for color texture generation using a conditional adversarial loss obtained from weakly-supervised views.
no code implementations • ICLR 2020 • Justus Thies, Michael Zollhöfer, Christian Theobalt, Marc Stamminger, Matthias Nießner
Based on this 3D proxy, the appearance of a captured view can be warped into a new target view as in classical image-based rendering.
3 code implementations • ECCV 2020 • Dave Zhenyu Chen, Angel X. Chang, Matthias Nießner
We introduce the task of 3D object localization in RGB-D scans using natural language descriptions.
1 code implementation • ECCV 2020 • Justus Thies, Mohamed Elgharib, Ayush Tewari, Christian Theobalt, Matthias Nießner
Neural Voice Puppetry has a variety of use-cases, including audio-driven video avatars, video dubbing, and text-driven video synthesis of a talking head.
1 code implementation • 9 Dec 2019 • Aljaž Božič, Michael Zollhöfer, Christian Theobalt, Matthias Nießner
Applying data-driven approaches to non-rigid 3D reconstruction has been difficult, which we believe can be attributed to the lack of a large-scale training corpus.
2 code implementations • CVPR 2020 • Angela Dai, Christian Diller, Matthias Nießner
We present a novel approach that converts partial and noisy RGB-D scans into high-quality 3D scene reconstructions by inferring unobserved scene geometry.
no code implementations • 27 Nov 2019 • Davide Cozzolino, Justus Thies, Andreas Rössler, Matthias Nießner, Luisa Verdoliva
Given a GAN-generated image, we insert the traces of a specific camera model into it and deceive state-of-the-art detectors into believing the image was acquired by that model.
1 code implementation • CVPR 2020 • Yawar Siddiqui, Julien Valentin, Matthias Nießner
To incorporate this uncertainty measure, we introduce a new viewpoint entropy formulation, which is the basis of our active learning strategy.
1 code implementation • ICCV 2019 • Manuel Dahnert, Angela Dai, Leonidas Guibas, Matthias Nießner
We propose a novel approach to learn a joint embedding space between scan and CAD geometry, where semantically similar objects from both domains lie close together.
1 code implementation • ICCV 2019 • Johanna Wald, Armen Avetisyan, Nassir Navab, Federico Tombari, Matthias Nießner
In this work, we introduce the task of 3D object instance re-localization (RIO): given one or multiple objects in an RGB-D scan, we want to estimate their corresponding 6DoF poses in another 3D scan of the same environment taken at a later point in time.
1 code implementation • ICCV 2019 • Armen Avetisyan, Angela Dai, Matthias Nießner
We present a novel, end-to-end approach to align CAD models to an 3D scan of a scene, enabling transformation of a noisy, incomplete 3D scan to a compact, CAD reconstruction with clean, complete object geometry.
3 code implementations • 28 Apr 2019 • Justus Thies, Michael Zollhöfer, Matthias Nießner
Similar to traditional textures, neural textures are stored as maps on top of 3D mesh proxies; however, the high-dimensional feature maps contain significantly more information, which can be interpreted by our new deferred neural rendering pipeline.
no code implementations • CVPR 2020 • Ji Hou, Angela Dai, Matthias Nießner
Thus, we introduce the task of semantic instance completion: from an incomplete RGB-D scan of a scene, we aim to detect the individual object instances and infer their complete object geometry.
no code implementations • 17 Mar 2019 • Dejan Azinović, Tzu-Mao Li, Anton Kaplanyan, Matthias Nießner
We introduce Inverse Path Tracing, a novel approach to jointly estimate the material properties of objects and light sources in indoor scenes by using an invertible light transport simulation.
1 code implementation • 5 Feb 2019 • Thomas Frerix, Matthias Nießner, Daniel Cremers
One way to achieve this task is by means of a projection step at test time after unconstrained training.
1 code implementation • ICCV 2019 • Chiyu "Max" Jiang, Dana Lynn Ona Lansigan, Philip Marcus, Matthias Nießner
We present a Deep Differentiable Simplex Layer (DDSL) for neural networks for geometric deep learning.
16 code implementations • 25 Jan 2019 • Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, Matthias Nießner
In particular, the benchmark is based on DeepFakes, Face2Face, FaceSwap and NeuralTextures as prominent representatives for facial manipulations at random compression level and size.
Ranked #1 on
DeepFake Detection
on FaceForensics
2 code implementations • ICLR 2019 • Chiyu "Max" Jiang, Dequan Wang, Jingwei Huang, Philip Marcus, Matthias Nießner
It has been challenging to analyze signals with mixed topologies (for example, point cloud with surface mesh).
no code implementations • 26 Dec 2018 • Lei Han, Mengqi Ji, Lu Fang, Matthias Nießner
Direct image-to-image alignment that relies on the optimization of photometric error metrics suffers from limited convergence range and sensitivity to lighting conditions.
1 code implementation • CVPR 2019 • Ji Hou, Angela Dai, Matthias Nießner
We introduce 3D-SIS, a novel neural network architecture for 3D semantic instance segmentation in commodity RGB-D scans.
Ranked #3 on
3D Semantic Instance Segmentation
on ScanNetV2
no code implementations • 6 Dec 2018 • Davide Cozzolino, Justus Thies, Andreas Rössler, Christian Riess, Matthias Nießner, Luisa Verdoliva
We devise a learning-based forensic detector which adapts well to new domains, i. e., novel manipulation methods and can handle scenarios where only a handful of fake examples are available during training.
1 code implementation • CVPR 2019 • Vincent Sitzmann, Justus Thies, Felix Heide, Matthias Nießner, Gordon Wetzstein, Michael Zollhöfer
In this work, we address the lack of 3D understanding of generative neural networks by introducing a persistent 3D feature embedding for view synthesis.
1 code implementation • CVPR 2019 • Jingwei Huang, Haotian Zhang, Li Yi, Thomas Funkhouser, Matthias Nießner, Leonidas Guibas
We introduce, TextureNet, a neural network architecture designed to extract features from high-resolution signals associated with 3D surface meshes (e. g., color texture maps).
Ranked #12 on
Semantic Segmentation
on ScanNet
2 code implementations • CVPR 2019 • Armen Avetisyan, Manuel Dahnert, Angela Dai, Manolis Savva, Angel X. Chang, Matthias Nießner
For a 3D reconstruction of an indoor scene, our method takes as input a set of CAD models, and predicts a 9DoF pose that aligns each model to the underlying scan geometry.
Ranked #1 on
3D Reconstruction
on Scan2CAD
no code implementations • 26 Nov 2018 • Justus Thies, Michael Zollhöfer, Christian Theobalt, Marc Stamminger, Matthias Nießner
Based on this 3D proxy, the appearance of a captured view can be warped into a new target view as in classical image-based rendering.
1 code implementation • CVPR 2019 • Angela Dai, Matthias Nießner
We introduce Scan2Mesh, a novel data-driven generative approach which transforms an unstructured and potentially incomplete range scan into a structured 3D mesh representation.
no code implementations • 29 May 2018 • Hyeongwoo Kim, Pablo Garrido, Ayush Tewari, Weipeng Xu, Justus Thies, Matthias Nießner, Patrick Pérez, Christian Richardt, Michael Zollhöfer, Christian Theobalt
In order to enable source-to-target video re-animation, we render a synthetic target video with the reconstructed head animation parameters from a source video, and feed it into the trained network -- thus taking full control of the target.
no code implementations • 29 May 2018 • Justus Thies, Michael Zollhöfer, Christian Theobalt, Marc Stamminger, Matthias Nießner
We propose HeadOn, the first real-time source-to-target reenactment approach for complete human portrait videos that enables transfer of torso and head motion, face expression, and eye gaze.
1 code implementation • ECCV 2018 • Angela Dai, Matthias Nießner
We present 3DMV, a novel method for 3D semantic scene segmentation of RGB-D scans in indoor environments using a joint 3D-multi-view prediction network.
Ranked #1 on
Scene Segmentation
on ScanNet
no code implementations • 24 Mar 2018 • Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, Matthias Nießner
Research on the detection of face manipulations has been seriously hampered by the lack of adequate datasets.
no code implementations • CVPR 2018 • Angela Dai, Daniel Ritchie, Martin Bokeloh, Scott Reed, Jürgen Sturm, Matthias Nießner
We introduce ScanComplete, a novel data-driven approach for taking an incomplete 3D scan of a scene as input and predicting a complete 3D model along with per-voxel semantic labels.
no code implementations • 18 Dec 2017 • Justus Thies, Michael Zollhöfer, Matthias Nießner
We present IMU2Face, a gesture-driven facial reenactment system.
no code implementations • 5 Oct 2017 • Vladislav Golyanik, Kihwan Kim, Robert Maier, Matthias Nießner, Didier Stricker, Jan Kautz
We introduce a novel multiframe scene flow approach that jointly optimizes the consistency of the patch appearances and their local rigid motions from RGB-D image sequences.
1 code implementation • 18 Sep 2017 • Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Nießner, Manolis Savva, Shuran Song, Andy Zeng, yinda zhang
Access to large, diverse RGB-D datasets is critical for training RGB-D scene understanding algorithms.
no code implementations • 12 Aug 2017 • Nazim Haouchine, Frederick Roy, Hadrien Courtecuisse, Matthias Nießner, Stephane Cotin
We present Calipso, an interactive method for editing images and videos in a physically-coherent manner.
1 code implementation • ICCV 2017 • Robert Maier, Kihwan Kim, Daniel Cremers, Jan Kautz, Matthias Nießner
We introduce a novel method to obtain high-quality 3D reconstructions from consumer RGB-D sensors.
no code implementations • 25 May 2017 • Benjamin Hepp, Matthias Nießner, Otmar Hilliges
We introduce a new method that efficiently computes a set of viewpoints and trajectories for high-quality 3D reconstructions in outdoor environments.
no code implementations • ICCV 2017 • Kihwan Kim, Jinwei Gu, Stephen Tyree, Pavlo Molchanov, Matthias Nießner, Jan Kautz
In addition, we have created a large synthetic dataset, SynBRDF, which comprises a total of $500$K RGBD images rendered with a physically-based ray tracer under a variety of natural illumination, covering $5000$ materials and $5000$ shapes.
1 code implementation • CVPR 2017 • Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nießner
A key requirement for leveraging supervised deep learning methods is the availability of large, labeled datasets.
Ranked #9 on
Semantic Segmentation
on ScanNetV2
2 code implementations • CVPR 2017 • Angela Dai, Charles Ruizhongtai Qi, Matthias Nießner
Although our 3D-EPN outperforms state-of-the-art completion method, the main contribution in our work lies in the combination of a data-driven shape predictor and analytic 3D shape synthesis.
no code implementations • 11 Oct 2016 • Justus Thies, Michael Zollhöfer, Marc Stamminger, Christian Theobalt, Matthias Nießner
Based on reenactment of a prerecorded stereo video of the person without the HMD, FaceVR incorporates photo-realistic re-rendering in real time, thus allowing artificial modifications of face and eye appearances.
no code implementations • 22 Apr 2016 • Zachary DeVito, Michael Mara, Michael Zollhöfer, Gilbert Bernstein, Jonathan Ragan-Kelley, Christian Theobalt, Pat Hanrahan, Matthew Fisher, Matthias Nießner
Many graphics and vision problems can be expressed as non-linear least squares optimizations of objective functions over visual data, such as images and meshes.
no code implementations • 5 Apr 2016 • Angela Dai, Matthias Nießner, Michael Zollhöfer, Shahram Izadi, Christian Theobalt
Our approach estimates globally optimized (i. e., bundle adjusted) poses in real-time, supports robust tracking with recovery from gross tracking failures (i. e., relocalization), and re-estimates the 3D model in real-time to ensure global consistency; all within a single framework.
no code implementations • 27 Mar 2016 • Matthias Innmann, Michael Zollhöfer, Matthias Nießner, Christian Theobalt, Marc Stamminger
We cast finding the optimal deformation of space as a non-linear regularized variational optimization problem by enforcing local smoothness and proximity to the input constraints.
2 code implementations • CVPR 2017 • Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao, Thomas Funkhouser
To amass training data for our model, we propose a self-supervised feature learning method that leverages the millions of correspondence labels found in existing RGB-D reconstructions.
Ranked #2 on
3D Reconstruction
on Scan2CAD
no code implementations • 18 Mar 2016 • Julien Valentin, Angela Dai, Matthias Nießner, Pushmeet Kohli, Philip Torr, Shahram Izadi, Cem Keskin
We demonstrate the efficacy of our approach on the challenging problem of RGB Camera Relocalization.