Search Results for author:
Found 51 papers, 26 papers with code
Factor Fields: A Unified Framework for Neural Fields and Beyond
As evidenced by our experiments, CoBaFa leads to improvements over previous fast reconstruction methods in terms of the three critical goals in neural signal representation: approximation quality, compactness and efficiency.
HARP: Personalized Hand Reconstruction from a Monocular RGB Video
We present HARP (HAnd Reconstruction and Personalization), a personalized hand avatar creation approach that takes a short monocular RGB video of a human hand as input and reconstructs a faithful hand avatar exhibiting a high-fidelity appearance and geometry.
3D Segmentation of Humans in Point Clouds with Synthetic Data
Our analysis of different training schemes using a combination of synthetic and realistic data shows that synthetic data for pre-training improves performance in a wide variety of segmentation tasks and models.
ARAH: Animatable Volume Rendering of Articulated Human SDFs
We demonstrate that our proposed pipeline can generate clothed avatars with high-quality pose-dependent geometry and appearance from a sparse set of multi-view RGB videos.
Mask3D for 3D Semantic Instance Segmentation
Modern 3D semantic instance segmentation approaches predominantly rely on specialized voting mechanisms followed by carefully designed geometric clustering techniques.
Ranked #1 on
3D Instance Segmentation
on ScanNet200
Neural Point-based Shape Modeling of Humans in Challenging Clothing
Specifically, we extend point-based methods with a coarse stage, that replaces canonicalization with a learned pose-independent "coarse shape" that can capture the rough surface geometry of clothing like skirts.
Compositional Human-Scene Interaction Synthesis with Semantic Control
Furthermore, inspired by the compositional nature of interactions that humans can simultaneously interact with multiple objects, we define interaction semantics as the composition of varying numbers of atomic action-object pairs.
Accurate 3D Body Shape Regression using Metric and Semantic Attributes
Since paired data with images and 3D body shape are rare, we exploit two sources of information: (1) we collect internet images of diverse "fashion" models together with a small set of anthropometric measurements; (2) we collect linguistic shape attributes for a wide range of 3D body meshes and the model images.
Ranked #5 on
3D Human Shape Estimation
on SSP-3D
KeypointNeRF: Generalizing Image-based Volumetric Avatars using Relative Spatial Encoding of Keypoints
In this work, we investigate common issues with existing spatial encodings and propose a simple yet highly effective approach to modeling high-fidelity volumetric humans from sparse views.
Ranked #1 on
Generalizable Novel View Synthesis
on ZJU-MoCap
Context-Aware Sequence Alignment using 4D Skeletal Augmentation
Temporal alignment of fine-grained human actions in videos is important for numerous applications in computer vision, robotics, and mixed reality.
Interactive Object Segmentation in 3D Point Clouds
We propose an interactive approach for 3D instance segmentation, where users can iteratively collaborate with a deep learning model to segment objects in a 3D point cloud directly.
COAP: Compositional Articulated Occupancy of People
We present a novel neural implicit representation for articulated human bodies.
Human-Aware Object Placement for Visual Environment Reconstruction
In fact, we demonstrate that these human-scene interactions (HSIs) can be leveraged to improve the 3D reconstruction of a scene from a monocular RGB video.
SAGA: Stochastic Whole-Body Grasping with Contact
Given an initial pose and the generated whole-body grasping pose as the start and end of the motion respectively, we design a novel contact-aware generative motion infilling module to generate a diverse set of grasp-oriented motions.
The Wanderings of Odysseus in 3D Scenes
In our solution, we decompose the long-term motion into a time sequence of motion primitives.
EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devices
Key to reasoning about interactions is to understand the body pose and motion of the interaction partner from the egocentric view.
A Skeleton-Driven Neural Occupancy Representation for Articulated Hands
We present Hand ArticuLated Occupancy (HALO), a novel representation of articulated hands that bridges the advantages of 3D keypoints and neural implicit surfaces and can be used in end-to-end trainable architectures.
The Power of Points for Modeling Humans in Clothing
The geometry feature can be optimized to fit a previously unseen scan of a person in clothing, enabling the scan to be reposed realistically.
Learning Motion Priors for 4D Human Body Capture in 3D Scenes
To prove the effectiveness of the proposed motion priors, we combine them into a novel pipeline for 4D human body capture in 3D scenes.
Learning to Disambiguate Strongly Interacting Hands via Probabilistic Per-pixel Part Segmentation
In natural conversation and interaction, our hands often overlap or are in contact with each other.
Ranked #1 on
3D Interacting Hand Pose Estimation
on InterHand2.6M
MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Images
In contrast, we propose an approach that can quickly generate realistic clothed human avatars, represented as controllable neural SDFs, given only monocular depth images.
Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration
We combine PTF with multi-class occupancy networks, obtaining a novel learning-based framework that learns to simultaneously predict shape and per-point correspondences between the posed space and the canonical space for clothed human.
SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local Elements
We demonstrate the efficacy of our surface representation by learning models of complex clothing from point clouds.
LEAP: Learning Articulated Occupancy of People
Substantial progress has been made on modeling rigid 3D objects using deep implicit representations.
On Self-Contact and Human Pose
Third, we develop a novel HPS optimization method, SMPLify-XMC, that includes contact constraints and uses the known 3DCP body pose during fitting to create near ground-truth poses for MTP images.
Ranked #31 on
3D Human Pose Estimation
on 3DPW
(MPJPE metric)
We are More than Our Joints: Predicting how 3D Bodies Move
We note that motion prediction methods accumulate errors over time, resulting in joints or markers that diverge from true human bodies.
MATE: Plugging in Model Awareness to Task Embedding for Meta Learning
Meta-learning improves generalization of machine learning models when faced with previously unseen tasks by leveraging experiences from different, yet related prior tasks.
4D Human Body Capture from Egocentric Video via 3D Scene Grounding
We introduce a novel task of reconstructing a time series of second-person 3D human body meshes from monocular egocentric videos.
PLACE: Proximity Learning of Articulation and Contact in 3D Environments
To synthesize realistic human-scene interactions, it is essential to effectively represent the physical contact and proximity between the body and the world.
Grasping Field: Learning Implicit Representations for Human Grasps
Specifically, our generative model is able to synthesize high-quality human grasps, given only on a 3D object point cloud.
ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation
We show that our dataset can significantly improve the robustness of gaze estimation methods across different head poses and gaze angles.
Ranked #1 on
Gaze Estimation
on MPSGaze
Perpetual Motion: Generating Unbounded Human Motion
To address this problem, we propose a model to generate non-deterministic, \textit{ever-changing}, perpetual human motion, in which the global trajectory and the body pose are cross-conditioned.
Generating 3D People in Scenes without People
However, this is a challenging task for a computer as solving it requires that (1) the generated human bodies to be semantically plausible within the 3D environment (e. g. people sitting on the sofa or cooking near the stove), and (2) the generated human-scene interaction to be physically feasible such that the human body and scene do not interpenetrate while, at the same time, body-scene contact supports physical interactions.
Forecasting Human-Object Interaction: Joint Prediction of Motor Attention and Actions in First Person Video
Motivated by this, we adopt intentional hand movement as a future representation and propose a novel deep network that jointly models and predicts the egocentric hand motion, interaction hotspots and future action.
Learning Multi-Human Optical Flow
Therefore, we develop a dataset of multi-human optical flow and train optical flow networks on this dataset.
Learning to Train with Synthetic Humans
Here we explore two variations of synthetic data for this challenging problem; a dataset with purely synthetic humans and a real dataset augmented with synthetic humans.
Learning to Dress 3D People in Generative Clothing
To our knowledge, this is the first generative model that directly dresses 3D human body meshes and generalizes to different poses.
Frontal Low-rank Random Tensors for Fine-grained Action Segmentation
In this paper, we propose an approach to representing high-order information for temporal action segmentation via a simple yet effective bilinear form.
End-to-end Learning for Graph Decomposition
The new optimization problem can be viewed as a Conditional Random Field (CRF) in which the random variables are associated with the binary edge labels of the initial graph and the hard constraints are introduced in the CRF as high-order potentials.
Local Temporal Bilinear Pooling for Fine-grained Action Parsing
Fine-grained temporal action parsing is important in many applications, such as daily activity understanding, human motion analysis, surgical robotics and others requiring subtle and precise operations in a long-term period.
Part-Aligned Bilinear Representations for Person Re-identification
We propose a novel network that learns a part-aligned representation for person re-identification.
Ranked #4 on
Person Re-Identification
on UAV-Human
Temporal Human Action Segmentation via Dynamic Clustering
We present an effective dynamic clustering algorithm for the task of temporal human action segmentation, which has comprehensive applications such as robotics, motion analysis, and patient monitoring.
Multiple People Tracking by Lifted Multicut and Person Re-Identification
This allows us to reward tracks that assign detections of similar appearance to the same person in a way that does not introduce implausible solutions.
Joint Graph Decomposition & Node Labeling: Problem, Algorithms, Applications
In order to find feasible solutions efficiently, we define two local search algorithms that converge monotonously to a local optimum, offering a feasible solution at any time.
Generating Descriptions with Grounded and Co-Referenced People
At training time, we first learn how to localize characters by relating their visual appearance to mentions in the descriptions via a semi-supervised approach.
ArtTrack: Articulated Multi-person Tracking in the Wild
In this paper we propose an approach for articulated tracking of multiple people in unconstrained videos.
Ranked #7 on
Multi-Person Pose Estimation
on MPII Multi-Person
Joint Graph Decomposition and Node Labeling: Problem, Algorithms, Applications
In order to find feasible solutions efficiently, we define two local search algorithms that converge monotonously to a local optimum, offering a feasible solution at any time.
Multi-Person Tracking by Multicut and Deep Matching
In [1], we proposed a graph-based formulation that links and clusters person hypotheses over time by solving a minimum cost subgraph multicut problem.
A Multi-cut Formulation for Joint Segmentation and Tracking of Multiple Objects
Recently, Minimum Cost Multicut Formulations have been proposed and proven to be successful in both motion trajectory segmentation and multi-target tracking scenarios.
DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation
This paper considers the task of articulated human pose estimation of multiple people in real world images.
Ranked #2 on
Multi-Person Pose Estimation
on WAF
Subgraph Decomposition for Multi-Target Tracking
Tracking multiple targets in a video, based on a finite set of detection hypotheses, is a persistent problem in computer vision.
