Search Results for author: Helge Rhodin

Found 51 papers, 19 papers with code

Representing Animatable Avatar via Factorized Neural Fields

no code implementations2 Jun 2024 Chunjin Song, Zhijie Wu, Bastian Wandt, Leonid Sigal, Helge Rhodin

For reconstructing high-fidelity human 3D models from monocular videos, it is crucial to maintain consistent large-scale body shapes along with finely matched subtle wrinkles.

CasCalib: Cascaded Calibration for Motion Capture from Sparse Unsynchronized Cameras

1 code implementation10 May 2024 James Tang, Shashwat Suri, Daniel Ajisafe, Bastian Wandt, Helge Rhodin

We attain this generality by partitioning the high-dimensional time and calibration space into a cascade of subspaces and introduce tailored algorithms to optimize each efficiently and robustly.

Camera Calibration

Gaussian Shadow Casting for Neural Characters

no code implementations CVPR 2024 Luis Bolanos, Shih-Yang Su, Helge Rhodin

Combined with a deferred neural rendering model, our Gaussian shadows enable Lambertian shading and shadow casting with minimal overhead.

Neural Rendering

Mirror-Aware Neural Humans

1 code implementation9 Sep 2023 Daniel Ajisafe, James Tang, Shih-Yang Su, Bastian Wandt, Helge Rhodin

Human motion capture either requires multi-camera systems or is unreliable when using single-view input due to depth ambiguities.

3D Human Pose Estimation

Pose Modulated Avatars from Video

no code implementations23 Aug 2023 Chunjin Song, Bastian Wandt, Helge Rhodin

It is now possible to reconstruct dynamic human motion and shape from a sparse set of cameras using Neural Radiance Fields (NeRF) driven by an underlying skeleton.

Graph Neural Network

NPC: Neural Point Characters from Video

no code implementations ICCV 2023 Shih-Yang Su, Timur Bagautdinov, Helge Rhodin

Previous methods avoid using a template but rely on a costly or ill-posed mapping from observation to canonical space.

Few-shot Geometry-Aware Keypoint Localization

no code implementations CVPR 2023 Xingzhe He, Gaurav Bharaj, David Ferman, Helge Rhodin, Pablo Garrido

Supervised keypoint localization methods rely on large manually labeled image datasets, where objects can deform, articulate, or occlude.

Object Localization

Scaling Neural Face Synthesis to High FPS and Low Latency by Neural Caching

no code implementations10 Nov 2022 Frank Yu, Sid Fels, Helge Rhodin

The warping with a shallow network reduces latency and the caching operations can further be parallelized to improve the frame rate.

Face Generation Neural Rendering

UNeRF: Time and Memory Conscious U-Shaped Network for Training Neural Radiance Fields

no code implementations23 Jun 2022 Abiramy Kuganesan, Shih-Yang Su, James J. Little, Helge Rhodin

Neural Radiance Fields (NeRFs) increase reconstruction detail for novel view synthesis and scene reconstruction, with applications ranging from large static scenes to dynamic human motion.

Density Estimation Novel View Synthesis

AutoLink: Self-supervised Learning of Human Skeletons and Object Outlines by Linking Keypoints

1 code implementation21 May 2022 Xingzhe He, Bastian Wandt, Helge Rhodin

Our key ingredients are i) an encoder that predicts keypoint locations in an input image, ii) a shared graph as a latent variable that links the same pairs of keypoints in every image, iii) an intermediate edge map that combines the latent graph edge weights and keypoint locations in a soft, differentiable manner, and iv) an inpainting objective on randomly masked images.

Pose Estimation Self-Supervised Learning +6

LatentKeypointGAN: Controlling Images via Latent Keypoints -- Extended Abstract

no code implementations6 May 2022 Xingzhe He, Bastian Wandt, Helge Rhodin

Generative adversarial networks (GANs) can now generate photo-realistic images.

DANBO: Disentangled Articulated Neural Body Representations via Graph Neural Networks

no code implementations3 May 2022 Shih-Yang Su, Timur Bagautdinov, Helge Rhodin

While a few such approaches exist, those have limited generalization capabilities and are prone to learning spurious (chance) correlations between irrelevant body parts, resulting in implausible deformations and missing body parts on unseen poses.

Graph Neural Network Image Generation

A Simple Method to Boost Human Pose Estimation Accuracy by Correcting the Joint Regressor for the Human3.6m Dataset

2 code implementations29 Apr 2022 Eric Hedlin, Helge Rhodin, Kwang Moo Yi

While the quality of this pseudo-ground-truth is challenging to assess due to the lack of actual ground-truth SMPL, with the Human 3. 6m dataset, we qualitatively show that our joint locations are more accurate and that our regressor leads to improved pose estimations results on the test set without any need for retraining.

Pose Estimation

Domain Knowledge-Informed Self-Supervised Representations for Workout Form Assessment

1 code implementation28 Feb 2022 Paritosh Parmar, Amol Gharat, Helge Rhodin

To that end, we propose to learn exercise-oriented image and video representations from unlabeled samples such that a small dataset annotated by experts suffices for supervised error detection.

3D Action Recognition Action Analysis +11

NRST: Non-rigid Surface Tracking from Monocular Video

no code implementations6 Jul 2021 Marc Habermann, Weipeng Xu, Helge Rhodin, Michael Zollhoefer, Gerard Pons-Moll, Christian Theobalt

Our texture term exploits the orientation information in the micro-structures of the objects, e. g., the yarn patterns of fabrics.

AudioViewer: Learning to Visualize Sounds

1 code implementation22 Dec 2020 Chunjin Song, Yuchi Zhang, Willis Peng, Parmis Mohaghegh, Bastian Wandt, Helge Rhodin

Different from existing models that translate to hand sign language, between speech and text, or text and images, we target immediate and low-level audio to video translation that applies to generic environment sounds as well as human speech.

Translation

Temporal Representation Learning on Monocular Videos for 3D Human Pose Estimation

no code implementations2 Dec 2020 Sina Honari, Victor Constantin, Helge Rhodin, Mathieu Salzmann, Pascal Fua

In this paper we propose an unsupervised feature extraction method to capture temporal information on monocular videos, where we detect and encode subject of interest in each frame and leverage contrastive self-supervised (CSS) learning to extract rich latent vectors.

3D Human Pose Estimation 3D Pose Estimation +1

PCLs: Geometry-aware Neural Reconstruction of 3D Pose with Perspective Crop Layers

1 code implementation CVPR 2021 Frank Yu, Mathieu Salzmann, Pascal Fua, Helge Rhodin

Our conclusion is that it is important to utilize camera calibration information when available, for classical and deep-learning-based computer vision alike.

3D Reconstruction Camera Calibration

Self-supervised Segmentation via Background Inpainting

no code implementations11 Nov 2020 Isinsu Katircioglu, Helge Rhodin, Victor Constantin, Jörg Spörri, Mathieu Salzmann, Pascal Fua

While supervised object detection and segmentation methods achieve impressive accuracy, they generalize poorly to images whose appearance significantly differs from the data they have been trained on.

Human Detection Object +4

Ellipse Detection and Localization with Applications to Knots in Sawn Lumber Images

no code implementations10 Nov 2020 Shenyi Pan, Shuxian Fan, Samuel W. K. Wong, James V. Zidek, Helge Rhodin

Specific to the lumber application, we also propose an algorithm to correct any misalignment in the raw timber images during scanning, and contribute the first open-source lumber knot dataset by labeling the elliptical knots in the preprocessed images.

object-detection Object Detection +1

Deformation-aware Unpaired Image Translation for Pose Estimation on Laboratory Animals

no code implementations CVPR 2020 Siyuan Li, Semih Günel, Mirela Ostrek, Pavan Ramdya, Pascal Fua, Helge Rhodin

We compare our approach with existing domain transfer methods and demonstrate improved pose estimation accuracy on Drosophila melanogaster (fruit fly), Caenorhabditis elegans (worm) and Danio rerio (zebrafish), without requiring any manual annotation on the target domain and despite using simplistic off-the-shelf animal characters for simulation, or simple geometric shapes as models.

Pose Estimation Translation

Front2Back: Single View 3D Shape Reconstruction via Front to Back Prediction

1 code implementation CVPR 2020 Yuan Yao, Nico Schertler, Enrique Rosales, Helge Rhodin, Leonid Sigal, Alla Sheffer

Reconstruction of a 3D shape from a single 2D image is a classical computer vision problem, whose difficulty stems from the inherent ambiguity of recovering occluded or only partially observed surfaces.

3D Shape Reconstruction Surface Reconstruction

Gravity as a Reference for Estimating a Person's Height from Video

no code implementations ICCV 2019 Didier Bieler, Semih Günel, Pascal Fua, Helge Rhodin

We show theoretically and empirically that a simple motion trajectory analysis suffices to translate from pixel measurements to the person's metric height, reaching a MAE of up to 3. 9 cm on jumping motions, and that this works without camera and ground plane calibration.

Motion Capture from Pan-Tilt Cameras with Unknown Orientation

no code implementations30 Aug 2019 Roman Bachmann, Jörg Spörri, Pascal Fua, Helge Rhodin

We propose a method for estimating an athlete's global 3D position and articulated pose using multiple cameras.

Markerless Motion Capture

Self-supervised Training of Proposal-based Segmentation via Background Prediction

no code implementations18 Jul 2019 Isinsu Katircioglu, Helge Rhodin, Victor Constantin, Jörg Spörri, Mathieu Salzmann, Pascal Fua

While supervised object detection methods achieve impressive accuracy, they generalize poorly to images whose appearance significantly differs from the data they have been trained on.

Object object-detection +2

XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera

4 code implementations1 Jul 2019 Dushyant Mehta, Oleksandr Sotnychenko, Franziska Mueller, Weipeng Xu, Mohamed Elgharib, Pascal Fua, Hans-Peter Seidel, Helge Rhodin, Gerard Pons-Moll, Christian Theobalt

The first stage is a convolutional neural network (CNN) that estimates 2D and 3D pose features along with identity assignments for all visible joints of all individuals. We contribute a new architecture for this CNN, called SelecSLS Net, that uses novel selective long and short range skip connections to improve the information flow allowing for a drastically faster network without compromising accuracy.

3D Multi-Person Human Pose Estimation 3D Multi-Person Pose Estimation +1

Neural Scene Decomposition for Multi-Person Motion Capture

1 code implementation CVPR 2019 Helge Rhodin, Victor Constantin, Isinsu Katircioglu, Mathieu Salzmann, Pascal Fua

To this end, we introduce a self-supervised approach to learning what we call a neural scene decomposition (NSD) that can be exploited for 3D pose estimation.

3D Pose Estimation Instance Segmentation +1

What Face and Body Shapes Can Tell About Height

no code implementations25 May 2018 Semih Günel, Helge Rhodin, Pascal Fua

Recovering a person's height from a single image is important for virtual garment fitting, autonomous driving and surveillance, however, it is also very challenging due to the absence of absolute scale information.

Anatomy Autonomous Driving +1

Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera

no code implementations15 Mar 2018 Weipeng Xu, Avishek Chatterjee, Michael Zollhoefer, Helge Rhodin, Pascal Fua, Hans-Peter Seidel, Christian Theobalt

We tackle these challenges based on a novel lightweight setup that converts a standard baseball cap to a device for high-quality pose estimation based on a single cap-mounted fisheye camera.

Ranked #6 on Egocentric Pose Estimation on GlobalEgoMocap Test Dataset (using extra training data)

3D Pose Estimation Egocentric Pose Estimation

MonoPerfCap: Human Performance Capture from Monocular Video

no code implementations7 Aug 2017 Weipeng Xu, Avishek Chatterjee, Michael Zollhöfer, Helge Rhodin, Dushyant Mehta, Hans-Peter Seidel, Christian Theobalt

Reconstruction from monocular video alone is drastically more challenging, since strong occlusions and the inherent depth ambiguity lead to a highly ill-posed reconstruction problem.

Monocular Reconstruction Pose Estimation +1

VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera

1 code implementation3 May 2017 Dushyant Mehta, Srinath Sridhar, Oleksandr Sotnychenko, Helge Rhodin, Mohammad Shafiei, Hans-Peter Seidel, Weipeng Xu, Dan Casas, Christian Theobalt

A real-time kinematic skeleton fitting method uses the CNN output to yield temporally stable 3D global pose reconstructions on the basis of a coherent kinematic skeleton.

3D Human Pose Estimation

EgoCap: Egocentric Marker-less Motion Capture with Two Fisheye Cameras (Extended Abstract)

no code implementations31 Dec 2016 Helge Rhodin, Christian Richardt, Dan Casas, Eldar Insafutdinov, Mohammad Shafiei, Hans-Peter Seidel, Bernt Schiele, Christian Theobalt

Marker-based and marker-less optical skeletal motion-capture methods use an outside-in arrangement of cameras placed around a scene, with viewpoints converging on the center.

Pose Estimation

Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision

no code implementations29 Nov 2016 Dushyant Mehta, Helge Rhodin, Dan Casas, Pascal Fua, Oleksandr Sotnychenko, Weipeng Xu, Christian Theobalt

We propose a CNN-based approach for 3D human body pose estimation from single RGB images that addresses the issue of limited generalizability of models trained solely on the starkly limited publicly available 3D pose data.

Diversity Monocular 3D Human Pose Estimation +1

Model-based Outdoor Performance Capture

no code implementations21 Oct 2016 Nadia Robertini, Dan Casas, Helge Rhodin, Hans-Peter Seidel, Christian Theobalt

We propose a new model-based method to accurately reconstruct human performances captured outdoors in a multi-camera setup.

Edge Detection

EgoCap: Egocentric Marker-less Motion Capture with Two Fisheye Cameras

no code implementations23 Sep 2016 Helge Rhodin, Christian Richardt, Dan Casas, Eldar Insafutdinov, Mohammad Shafiei, Hans-Peter Seidel, Bernt Schiele, Christian Theobalt

We therefore propose a new method for real-time, marker-less and egocentric motion capture which estimates the full-body skeleton pose from a lightweight stereo pair of fisheye cameras that are attached to a helmet or virtual reality headset.

Pose Estimation Vocal Bursts Valence Prediction

Real-Time Hand Tracking Using a Sum of Anisotropic Gaussians Model

no code implementations11 Feb 2016 Srinath Sridhar, Helge Rhodin, Hans-Peter Seidel, Antti Oulasvirta, Christian Theobalt

In this paper, we propose a new approach that tracks the full skeleton motion of the hand from multiple RGB cameras in real-time.

A Versatile Scene Model with Differentiable Visibility Applied to Generative Pose Estimation

no code implementations ICCV 2015 Helge Rhodin, Nadia Robertini, Christian Richardt, Hans-Peter Seidel, Christian Theobalt

Generative reconstruction methods compute the 3D configuration (such as pose and/or geometry) of a shape by optimizing the overlap of the projected 3D shape model with images.

Occlusion Handling Pose Estimation

Cannot find the paper you are looking for? You can Submit a new open access paper.