Search Results for author:
Found 74 papers, 34 papers with code
NASA Neural Articulated Shape Approximation
Efficient representation of articulated objects such as human bodies is an important problem in computer vision and graphics.
GEARS: Local Geometry-aware Hand-object Interaction Synthesis
Generating realistic hand motion sequences in interaction with objects has gained increasing attention with the growing interest in digital humans.
Recent Trends in 3D Reconstruction of General Non-Rigid Scenes
Reconstructing models of the real world, including 3D geometry, appearance, and motion of real scenes, is essential for computer graphics and computer vision.
FORCE: Dataset and Method for Intuitive Physics Guided Human-object Interaction
Our key insight is that human motion is dictated by the interrelation between the force exerted by the human and the perceived resistance.
NRDF: Neural Riemannian Distance Fields for Learning Articulated Pose Priors
Faithfully modeling the space of articulations is a crucial task that allows recovery and generation of realistic poses, and remains a notorious challenge.
CloSe: A 3D Clothing Segmentation Dataset and Model
Additionally, we propose CloSe-Net, the first learning-based 3D clothing segmentation model for fine-grained segmentation from colored point clouds.
RHOBIN Challenge: Reconstruction of Human Object Interaction
Modeling the interaction between humans and objects has been an emerging research direction in recent years.
NICP: Neural ICP for 3D Human Registration at Scale
NSR achieves the state of the art over public benchmarks, and the release of its code and checkpoints will provide the community with a powerful tool useful for many downstream tasks like dataset alignments, cleaning, or asset animation.
Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Rendering
We present Paint-it, a text-driven high-fidelity texture map synthesis method for 3D meshes via neural re-parameterized texture optimization.
Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generation
We generate 1M+ human-object interaction pairs in 3D and leverage this large-scale data to train our HDM (Hierarchical Diffusion Model), a novel method to reconstruct interacting human and unseen objects, without any templates.
GAN-Avatar: Controllable Personalized GAN-based Human Head Avatar
For controlling the model, we learn a mapping from 3DMM facial expression parameters to the latent space of the generative model.
NSF: Neural Surface Fields for Human Modeling from Monocular Depth
Compared to existing approaches, our method eliminates the expensive per-frame surface extraction while maintaining mesh coherency, and is capable of reconstructing meshes with arbitrary resolution without retraining.
Object pop-up: Can we infer 3D objects and their poses from human interactions alone?
The intimate entanglement between objects affordances and human poses is of large interest, among others, for behavioural sciences, cognitive psychology, and Computer Vision communities.
Generating Continual Human Motion in Diverse 3D Scenes
We decompose the continual motion synthesis problem into walking along paths and transitioning in and out of the actions specified by the keypoints, which enables long generation of motions that satisfy scene constraints without explicitly incorporating scene information.
Visibility Aware Human-Object Interaction Tracking from Single RGB Camera
In this work, we propose a novel method to track the 3D human, object, contacts between them, and their relative translation across frames from a single RGB camera, while being robust to heavy occlusions.
HDHumans: A Hybrid Approach for High-fidelity Digital Humans
Photo-real digital human avatars are of enormous importance in graphics, as they enable immersive communication over the globe, improve gaming and entertainment experiences, and can be particularly beneficial for AR and VR settings.
Skeleton-free Pose Transfer for Stylized 3D Characters
We present the first method that automatically transfers poses between stylized 3D characters without skeletal rigging.
Pose-NDF: Modeling Human Pose Manifolds with Neural Distance Fields
The resulting high-dimensional implicit function can be differentiated with respect to the input poses and thus can be used to project arbitrary poses onto the manifold by using gradient descent on the set of 3-dimensional hyperspheres.
Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation Using Bounding Boxes
Indeed, we show that it is possible to train dense segmentation models using only bounding box labels.
3D Instance Segmentation
3D Semantic Instance Segmentation
+2
Learned Vertex Descent: A New Direction for 3D Human Model Fitting
An exhaustive evaluation demonstrates that our approach is able to capture the underlying body of clothed people with very different body shapes, achieving a significant improvement compared to state-of-the-art.
Interaction Replica: Tracking Human-Object Interaction and Scene Changes From Human Motion
In order for widespread adoption of such emerging applications, the sensor setup used to capture the interactions needs to be inexpensive and easy-to-use for non-expert users.
COUCH: Towards Controllable Human-Chair Interactions
In this work, we study the problem of synthesizing scene interactions conditioned on different contact positions on the object.
Control-NeRF: Editable Feature Volumes for Scene Rendering and Manipulation
With the aim of obtaining interpretable and controllable scene representations, our model couples learnt scene-specific feature volumes with a scene agnostic neural rendering network.
BEHAVE: Dataset and Method for Tracking Human Object Interactions
We present BEHAVE dataset, the first full body human- object interaction dataset with multi-view RGBD frames and corresponding 3D SMPL and object fits along with the annotated contacts between them.
CHORE: Contact, Human and Object REconstruction from a single RGB image
However, humans are constantly interacting with the surrounding objects, thus calling for models that can reason about not only the human but also the object and their interaction.
Ranked #2 on
3D Object Reconstruction
on BEHAVE
A Deeper Look into DeepCap
Human performance capture is a highly important computer vision problem with many applications in movie production and virtual/augmented reality.
Neural-GIF: Neural Generalized Implicit Functions for Animating People in Clothing
Neural-GIF can be trained on raw 3D scans and reconstructs detailed complex surface geometry and deformations.
NRST: Non-rigid Surface Tracking from Monocular Video
Our texture term exploits the orientation information in the micro-structures of the objects, e. g., the yarn patterns of fabrics.
Real-time Deep Dynamic Characters
We propose a deep videorealistic 3D human character model displaying highly realistic shape, motion, and dynamic appearance learned in a new weakly supervised way from multi-view imagery.
Stereo Radiance Fields (SRF): Learning View Synthesis for Sparse Views of Novel Scenes
Recent neural view synthesis methods have achieved impressive quality and realism, surpassing classical pipelines which rely on multi-view reconstruction.
Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors
We introduce (HPS) Human POSEitioning System, a method to recover the full 3D pose of a human registered with a 3D scan of the surrounding environment using wearable sensors.
SMPLicit: Topology-aware Generative Model for Clothed People
In this paper we introduce SMPLicit, a novel generative model to jointly represent body pose, shape and clothing geometry.
Learning Speech-driven 3D Conversational Gestures from Video
We propose the first approach to automatically and jointly synthesize both the synchronous 3D conversational body and hand gestures, as well as 3D face and head animations, of a virtual character from speech input.
Ranked #4 on
Gesture Generation
on BEAT2
Adjoint Rigid Transform Network: Task-conditioned Alignment of 3D Shapes
The remarkable result is that with only self-supervision, ART facilitates learning a unique canonical orientation for both rigid and nonrigid shapes, which leads to a notable boost in performance of aforementioned tasks.
D-NeRF: Neural Radiance Fields for Dynamic Scenes
In this paper we introduce D-NeRF, a method that extends neural radiance fields to a dynamic domain, allowing to reconstruct and render novel images of objects under rigid and non-rigid motions from a \emph{single} camera moving around the scene.
SelfPose: 3D Egocentric Pose Estimation from a Headset Mounted Camera
The quantitative evaluation, on synthetic and real-world datasets, shows that our strategy leads to substantial improvements in accuracy over state of the art egocentric approaches.
SHARP 2020: The 1st Shape Recovery from Partial Textured 3D Scans Challenge Results
Additionally, two unique datasets of 3D scans are proposed, to provide raw ground-truth data for the benchmarks.
Neural Unsigned Distance Fields for Implicit Function Learning
NDF represent surfaces at high resolutions as prior implicit models, but do not require closed surface data, and significantly broaden the class of representable shapes in the output.
LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration
Formulating this closed loop is not straightforward because it is not trivial to force the output of the NN to be on the surface of the human model - outside this surface the human model is not even defined.
Implicit Feature Networks for Texture Completion from Partial 3D Data
Instead, we focus on 3D texture and geometry completion from partial and incomplete 3D scans.
Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction
In this work, we present methodology that combines detail-rich implicit functions and parametric representations in order to reconstruct 3D models of people that remain controllable and accurate even in the presence of clothing.
SIZER: A Dataset and Model for Parsing 3D Clothing and Learning Size Sensitive 3D Clothing
SizerNet allows to estimate and visualize the dressing effect of a garment in various sizes, and ParserNet allows to edit clothing of an input mesh directly, removing the need for scan segmentation, which is a challenging problem in itself.
Unsupervised Shape and Pose Disentanglement for 3D Meshes
The experiments on datasets of 3D humans, faces, hands and animals demonstrate the generality of our approach.
Kinematic 3D Object Detection in Monocular Video
In this work, we propose a novel method for monocular video-based 3D object detection which carefully leverages kinematic motion to improve precision of 3D localization.
Ranked #6 on
3D Object Detection
on Rope3D
DeepCap: Monocular Human Performance Capture Using Weak Supervision
Human performance capture is a highly important computer vision problem with many applications in movie production and virtual/augmented reality.
TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style
While the low-frequency component is predicted from pose, shape and style parameters with an MLP, the high-frequency component is predicted with a mixture of shape-style specific pose models.
Learning to Transfer Texture from Clothing Images to 3D Humans
In this paper, we present a simple yet effective method to automatically transfer textures of clothing images (front and back) to 3D garments worn on top SMPL, in real time.
Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion
To solve this, we propose Implicit Feature Networks (IF-Nets), which deliver continuous outputs, can handle multiple topologies, and complete shapes for missing or sparse input data retaining the nice properties of recent learned implicit functions, but critically they can also retain detail when it is present in the input data, and can reconstruct articulated humans.
NASA: Neural Articulated Shape Approximation
Efficient representation of articulated objects such as human bodies is an important problem in computer vision and graphics.
360-Degree Textures of People in Clothing from a Single Image
In order to learn our model in a common UV-space, we non-rigidly register the SMPL model to thousands of 3D scans, effectively encoding textures and geometries as images in correspondence.
Multi-Garment Net: Learning to Dress 3D People from Images
We present Multi-Garment Network (MGN), a method to predict body shape and clothing, layered on top of the SMPL model from a few frames (1-8) of a video.
3D Human Pose Estimation
3D Shape Reconstruction From A Single 2D Image
Learning to Dress 3D People in Generative Clothing
To our knowledge, this is the first generative model that directly dresses 3D human body meshes and generalizes to different poses.
XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera
The first stage is a convolutional neural network (CNN) that estimates 2D and 3D pose features along with identity assignments for all visible joints of all individuals. We contribute a new architecture for this CNN, called SelecSLS Net, that uses novel selective long and short range skip connections to improve the information flow allowing for a drastically faster network without compromising accuracy.
Ranked #7 on
3D Multi-Person Pose Estimation
on MuPoTS-3D
3D Multi-Person Human Pose Estimation
3D Multi-Person Pose Estimation
+1
Body Shape Privacy in Images: Understanding Privacy and Preventing Automatic Shape Extraction
Modern approaches to pose and body shape estimation have recently achieved strong performance even under challenging real-world conditions.
Tex2Shape: Detailed Full Human Body Geometry From a Single Image
From a partial texture, we estimate detailed normal and vector displacement maps, which can be applied to a low-resolution smooth body model to add detail and clothing.
In the Wild Human Pose Estimation Using Explicit 2D Features and Intermediate 3D Representations
Convolutional Neural Network based approaches for monocular 3D human pose estimation usually require a large amount of training images with 3D pose annotations.
AMASS: Archive of Motion Capture as Surface Shapes
We achieve this using a new method, MoSh++, that converts mocap data into realistic 3D human meshes represented by a rigged body model; here we use SMPL [doi:10. 1145/2816795. 2818013], which is widely used and provides a standard skeletal representation as well as a fully rigged surface mesh.
SimulCap : Single-View Human Performance Capture with Cloth Simulation
This paper proposes a new method for live free-viewpoint human performance capture with dynamic details (e. g., cloth wrinkles) using a single RGBD camera.
Learning to Reconstruct People in Clothing from a Single RGB Camera
We present a learning-based model to infer the personalized 3D shape of people from a few frames (1-8) of a monocular video in which the person is moving, in less than 10 seconds with a reconstruction accuracy of 5mm.
Deep Inertial Poser: Learning to Reconstruct Human Pose from Sparse Inertial Measurements in Real Time
To learn from sufficient data, we synthesize IMU data from motion capture datasets.
LiveCap: Real-time Human Performance Capture from Monocular Video
Our method is the first real-time monocular approach for full-body performance capture.
Recovering Accurate 3D Human Pose in The Wild Using IMUs and a Moving Camera
In this work, we propose a method that combines a single hand-held camera and a set of Inertial Measurement Units (IMUs) attached at the body limbs to estimate accurate 3D poses in the wild.
Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimation
Direct prediction of 3D body pose and shape remains a challenge even for highly parameterized deep learning models.
Ranked #1 on
Monocular 3D Human Pose Estimation
on Human3.6M
(Use Video Sequence metric)
Detailed Human Avatars from Monocular Video
We present a novel method for high detail-preserving human avatar creation from monocular video.
Fashion is Taking Shape: Understanding Clothing Preference Based on Body Shape From Online Sources
To study the correlation between clothing garments and body shape, we collected a new dataset (Fashion Takes Shape), which includes images of users with clothing category annotations.
DoubleFusion: Real-time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor
We further propose a joint motion tracking method based on the double layer representation to enable robust and fast motion tracking performance.
Video Based Reconstruction of 3D People Models
This paper describes how to obtain accurate 3D body models and texture of arbitrary people from a single, monocular video in which a person is moving.
Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB
Our approach uses novel occlusion-robust pose-maps (ORPM) which enable full body pose inference even under strong partial occlusions by other people and objects in the scene.
Ranked #3 on
3D Multi-Person Pose Estimation (root-relative)
on MuPoTS-3D
(MPJPE metric)
3D Human Pose Estimation
3D Multi-Person Pose Estimation (absolute)
+2
Dynamic FAUST: Registering Human Bodies in Motion
We propose a new mesh registration method that uses both 3D geometry and texture information to register all scans in a sequence to a common reference topology.
A Generative Model of People in Clothing
We present the first image-based generative model of people in clothing for the full body.
Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs
We address the problem of making human motion capture in the wild more practical by using a small set of inertial sensors attached to the body.
Detailed, accurate, human shape estimation from clothed 3D scan sequences
We address the problem of estimating human pose and body shape from 3D scans over time.
Posebits for Monocular Human Pose Estimation
We advocate the inference of qualitative information about 3D human pose, called posebits, from images.
