no code implementations • 17 Dec 2024 • Guoxing Sun, Rishabh Dabral, Heming Zhu, Pascal Fua, Christian Theobalt, Marc Habermann
Real-time free-view human rendering from sparse-view RGB inputs is a challenging task due to the sensor scarcity and the tight time budget.
no code implementations • 6 Dec 2024 • Wanyue Zhang, Rishabh Dabral, Vladislav Golyanik, Vasileios Choutas, Eduardo Alvarado, Thabo Beeler, Marc Habermann, Christian Theobalt
We present BimArt, a novel generative approach for synthesizing 3D bimanual hand interactions with articulated objects.
no code implementations • 21 Nov 2024 • Yumeng Liu, Xiaoxiao Long, Zemin Yang, YuAn Liu, Marc Habermann, Christian Theobalt, Yuexin Ma, Wenping Wang
Our work aims to reconstruct hand-object interactions from a single-view image, which is a fundamental but ill-posed task.
1 code implementation • 6 Nov 2024 • Siddharth Seth, Rishabh Dabral, Diogo Luvizon, Marc Habermann, Ming-Hsuan Yang, Christian Theobalt, Adam Kortylewski
We motivate our work by noting that most methods require a parametric model of the human body to ground pose-dependent deformations.
no code implementations • 22 Sep 2024 • Jianchun Chen, Jian Wang, yinda zhang, Rohit Pandey, Thabo Beeler, Marc Habermann, Christian Theobalt
Immersive VR telepresence ideally means being able to interact and communicate with digital avatars that are indistinguishable from and precisely reflect the behaviour of their real counterparts.
no code implementations • 19 Sep 2024 • Linjie Lyu, Ayush Tewari, Marc Habermann, Shunsuke Saito, Michael Zollhöfer, Thomas Leimkühler, Christian Theobalt
Radiance fields are powerful and, hence, popular models for representing the appearance of complex scenes.
no code implementations • 18 Sep 2024 • Kartik Teotia, Hyeongwoo Kim, Pablo Garrido, Marc Habermann, Mohamed Elgharib, Christian Theobalt
Real-time rendering of human head avatars is a cornerstone of many computer graphics applications, such as augmented reality, video games, and films, to name a few.
no code implementations • 10 Sep 2024 • Benoit Guillard, Marc Habermann, Christian Theobalt, Pascal Fua
Emulating low levels of detail can be achieved with shallow networks, but the generated shapes are typically not smooth.
no code implementations • 28 Aug 2024 • Basavaraj Sunagad, Heming Zhu, Mohit Mendiratta, Adam Kortylewski, Christian Theobalt, Marc Habermann
Over the past years, significant progress has been made in creating photorealistic and drivable 3D avatars solely from videos of real humans.
no code implementations • 27 Mar 2024 • Guoxing Sun, Rishabh Dabral, Pascal Fua, Christian Theobalt, Marc Habermann
Our key idea is to meta-learn the radiance field weights solely from potentially sparse multi-view videos, which can serve as a prior when fine-tuning them on sparse imagery depicting the human.
no code implementations • CVPR 2024 • Muhammad Hamza Mughal, Rishabh Dabral, Ikhsanul Habibie, Lucia Donatelli, Marc Habermann, Christian Theobalt
Gestures play a key role in human communication.
no code implementations • 18 Dec 2023 • Diogo Luvizon, Vladislav Golyanik, Adam Kortylewski, Marc Habermann, Christian Theobalt
Creating a controllable and relightable digital avatar from multi-view video with fixed illumination is a very challenging problem since humans are highly articulated, creating pose-dependent appearance effects, and skin as well as clothing require space-varying BRDF modeling.
no code implementations • CVPR 2024 • Ashwath Shetty, Marc Habermann, Guoxing Sun, Diogo Luvizon, Vladislav Golyanik, Christian Theobalt
At inference, our method only requires four camera views of the moving actor and the respective 3D skeletal pose.
1 code implementation • CVPR 2024 • Haokai Pang, Heming Zhu, Adam Kortylewski, Christian Theobalt, Marc Habermann
Real-time rendering of photorealistic and controllable human avatars stands as a cornerstone in Computer Vision and Graphics.
1 code implementation • 8 Dec 2023 • Heming Zhu, Fangneng Zhan, Christian Theobalt, Marc Habermann
Creating controllable, photorealistic, and geometrically detailed digital doubles of real humans solely from video data is a key challenge in Computer Graphics and Vision, especially when real-time performance is required.
no code implementations • 28 Nov 2023 • Zhengming Yu, Zhiyang Dou, Xiaoxiao Long, Cheng Lin, Zekun Li, YuAn Liu, Norman Müller, Taku Komura, Marc Habermann, Christian Theobalt, Xin Li, Wenping Wang
The experiments demonstrate the superior performance of Surf-D in shape generation across multiple modalities as conditions.
1 code implementation • CVPR 2024 • Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, YuAn Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Habermann, Christian Theobalt, Wenping Wang
In this work, we introduce Wonder3D, a novel method for efficiently generating high-fidelity textured meshes from single-view images. Recent methods based on Score Distillation Sampling (SDS) have shown the potential to recover 3D geometry from 2D diffusion priors, but they typically suffer from time-consuming per-shape optimization and inconsistent geometry.
Ranked #2 on
Single-View 3D Reconstruction
on GSO
1 code implementation • 30 Sep 2023 • Linjie Lyu, Ayush Tewari, Marc Habermann, Shunsuke Saito, Michael Zollhöfer, Thomas Leimkühler, Christian Theobalt
We further conduct an extensive comparative study of different priors on illumination used in previous work on inverse rendering.
no code implementations • 24 Aug 2023 • Wanyue Zhang, Rishabh Dabral, Thomas Leimkühler, Vladislav Golyanik, Marc Habermann, Christian Theobalt
Given an unseen object and a reference pose-object pair, we optimise for the object-aware pose that is closest in the feature space to the reference pose.
1 code implementation • 24 Aug 2023 • Navami Kairanda, Marc Habermann, Christian Theobalt, Vladislav Golyanik
Despite existing 3D cloth simulators producing realistic results, they predominantly operate on discrete surface representations (e. g. points and meshes) with a fixed spatial resolution, which often leads to large memory consumption and resolution-dependent simulations.
no code implementations • CVPR 2024 • Zhouyingcheng Liao, Vladislav Golyanik, Marc Habermann, Christian Theobalt
However, the former methods typically predict solely static skinning weights, which perform poorly for highly articulated poses, and the latter ones either require dense 3D character scans in different poses or cannot generate an explicit mesh with vertex correspondence over time.
no code implementations • 2 May 2023 • Xinyu Yi, Yuxiao Zhou, Marc Habermann, Vladislav Golyanik, Shaohua Pan, Christian Theobalt, Feng Xu
We integrate the two techniques together in EgoLocate, a system that simultaneously performs human motion capture (mocap), localization, and mapping in real time from sparse body-mounted sensors, including 6 inertial measurement units (IMUs) and a monocular phone camera.
1 code implementation • 12 Jan 2023 • Diogo Luvizon, Marc Habermann, Vladislav Golyanik, Adam Kortylewski, Christian Theobalt
In this work, we consider the problem of estimating the 3D position of multiple humans in a scene as well as their body shape and articulation from a single RGB video recorded with a static camera.
2 code implementations • ICCV 2023 • Yiming Wang, Qin Han, Marc Habermann, Kostas Daniilidis, Christian Theobalt, Lingjie Liu
Recent methods for neural surface representation and rendering, for example NeuS, have demonstrated the remarkably high-quality reconstruction of static scenes.
no code implementations • 27 Oct 2022 • Edith Tretschk, Navami Kairanda, Mallikarjun B R, Rishabh Dabral, Adam Kortylewski, Bernhard Egger, Marc Habermann, Pascal Fua, Christian Theobalt, Vladislav Golyanik
3D reconstruction of deformable (or non-rigid) scenes from a set of monocular 2D image observations is a long-standing and actively researched area of computer vision and graphics.
no code implementations • 21 Oct 2022 • Marc Habermann, Lingjie Liu, Weipeng Xu, Gerard Pons-Moll, Michael Zollhoefer, Christian Theobalt
Photo-real digital human avatars are of enormous importance in graphics, as they enable immersive communication over the globe, improve gaming and entertainment experiences, and can be particularly beneficial for AR and VR settings.
no code implementations • 11 Oct 2022 • Yue Jiang, Marc Habermann, Vladislav Golyanik, Christian Theobalt
Furthermore, we show that HiFECap outperforms the state-of-the-art human performance capture approaches qualitatively and quantitatively while for the first time capturing all aspects of the human.
no code implementations • 27 Jul 2022 • Linjie Lyu, Ayush Tewari, Thomas Leimkuehler, Marc Habermann, Christian Theobalt
Given a set of images of a scene, the re-rendering of this scene from novel views and lighting conditions is an important and challenging problem in Computer Vision and Graphics.
no code implementations • 16 Jun 2022 • Erik C. M. Johnson, Marc Habermann, Soshi Shimada, Vladislav Golyanik, Christian Theobalt
Capturing general deforming scenes from monocular RGB video is crucial for many computer graphics and vision applications.
no code implementations • 20 Nov 2021 • Marc Habermann, Weipeng Xu, Michael Zollhoefer, Gerard Pons-Moll, Christian Theobalt
Human performance capture is a highly important computer vision problem with many applications in movie production and virtual/augmented reality.
no code implementations • 6 Jul 2021 • Marc Habermann, Weipeng Xu, Helge Rhodin, Michael Zollhoefer, Gerard Pons-Moll, Christian Theobalt
Our texture term exploits the orientation information in the micro-structures of the objects, e. g., the yarn patterns of fabrics.
no code implementations • 3 Jun 2021 • Lingjie Liu, Marc Habermann, Viktor Rudnev, Kripasindhu Sarkar, Jiatao Gu, Christian Theobalt
To address this problem, we utilize a coarse body model as the proxy to unwarp the surrounding 3D space into a canonical pose.
no code implementations • 4 May 2021 • Marc Habermann, Lingjie Liu, Weipeng Xu, Michael Zollhoefer, Gerard Pons-Moll, Christian Theobalt
We propose a deep videorealistic 3D human character model displaying highly realistic shape, motion, and dynamic appearance learned in a new weakly supervised way from multi-view imagery.
no code implementations • ICCV 2021 • Linjie Lyu, Marc Habermann, Lingjie Liu, Mallikarjun B R, Ayush Tewari, Christian Theobalt
Differentiable rendering has received increasing interest for image-based inverse problems.
no code implementations • CVPR 2021 • Yuxiao Zhou, Marc Habermann, Ikhsanul Habibie, Ayush Tewari, Christian Theobalt, Feng Xu
We present the first method for real-time full body capture that estimates shape and motion of body and hands together with a dynamic 3D face model from a single color image.
Ranked #29 on
3D Hand Pose Estimation
on FreiHAND
no code implementations • 25 Nov 2020 • Yue Li, Marc Habermann, Bernhard Thomaszewski, Stelian Coros, Thabo Beeler, Christian Theobalt
Recent monocular human performance capture approaches have shown compelling dense tracking results of the full body from a single RGB camera.
2 code implementations • CVPR 2020 • Yuxiao Zhou, Marc Habermann, Weipeng Xu, Ikhsanul Habibie, Christian Theobalt, Feng Xu
We present a novel method for monocular hand shape and pose estimation at unprecedented runtime performance of 100fps and at state-of-the-art accuracy.
no code implementations • CVPR 2020 • Marc Habermann, Weipeng Xu, Michael Zollhoefer, Gerard Pons-Moll, Christian Theobalt
Human performance capture is a highly important computer vision problem with many applications in movie production and virtual/augmented reality.
no code implementations • 14 Jan 2020 • Lingjie Liu, Weipeng Xu, Marc Habermann, Michael Zollhoefer, Florian Bernard, Hyeongwoo Kim, Wenping Wang, Christian Theobalt
In this paper, we propose a novel human video synthesis method that approaches these limiting factors by explicitly disentangling the learning of time-coherent fine-scale details from the embedding of the human in 2D screen space.
no code implementations • CVPR 2020 • Lan Xu, Weipeng Xu, Vladislav Golyanik, Marc Habermann, Lu Fang, Christian Theobalt
The high frame rate is a critical requirement for capturing fast human motions.
no code implementations • 5 Oct 2018 • Marc Habermann, Weipeng Xu, Michael Zollhoefer, Gerard Pons-Moll, Christian Theobalt
Our method is the first real-time monocular approach for full-body performance capture.
no code implementations • 11 Sep 2018 • Lingjie Liu, Weipeng Xu, Michael Zollhoefer, Hyeongwoo Kim, Florian Bernard, Marc Habermann, Wenping Wang, Christian Theobalt
In contrast to conventional human character rendering, we do not require the availability of a production-quality photo-realistic 3D model of the human, but instead rely on a video sequence in conjunction with a (medium-quality) controllable 3D template model of the person.