Search Results for author:
Found 44 papers, 22 papers with code
Reconstructing Humans with a Biomechanically Accurate Skeleton
In this paper, we introduce a method for reconstructing 3D humans from a single image using a biomechanically accurate skeleton model.
MegaSynth: Scaling Up 3D Scene Reconstruction with Synthesized Data
We propose scaling up 3D scene reconstruction by training with synthesized data.
FIction: 4D Future Interaction Prediction from Video
Anticipating how a person will interact with objects in an environment is essential for activity understanding, but existing methods are limited to the 2D space of video frames-capturing physically ungrounded predictions of 'what' and ignoring the 'where' and 'how'.
OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitation
We study the problem of teaching humanoid robots manipulation skills by imitating from single video demonstrations.
Estimating Body and Hand Motion in an Ego-sensed World
We present EgoAllo, a system for human motion estimation from a head-mounted device.
Synergy and Synchrony in Couple Dances
Our contributions are a demonstration of the advantages of socially conditioned future motion prediction and an in-the-wild, couple dance video dataset to enable future research in this direction.
Atlas Gaussians Diffusion for 3D Generation
Using the latent diffusion model has proven effective in developing novel 3D generation techniques.
ExpertAF: Expert Actionable Feedback from Video
Our method takes a video demonstration and its accompanying 3D body pose and generates (1) free-form expert commentary describing what the person is doing well and what they could improve, and (2) a visual expert demonstration that incorporates the required corrections.
Expressive Gaussian Human Avatars from Monocular RGB Video
Nuanced expressiveness, particularly through fine-grained hand and facial expressions, is pivotal for enhancing the realism and vitality of digital human representations.
Real3D: Scaling Up Large Reconstruction Models with Real-World Images
Real3D introduces a novel self-training framework that can benefit from both the existing synthetic data and diverse single-view real images.
CoFie: Learning Compact Neural Surface Representations with Coordinate Fields
When using the same amount of parameters with prior works, CoFie reduces the shape error by 48% and 56% on novel instances of both training and unseen shape categories.
Evaluating Zero-Shot GPT-4V Performance on 3D Visual Question Answering Benchmarks
As interest in "reformulating" the 3D Visual Question Answering (VQA) problem in the context of foundation models grows, it is imperative to assess how these new paradigms influence existing closed-vocabulary datasets.
MultiPhys: Multi-Person Physics-aware 3D Motion Estimation
We introduce MultiPhys, a method designed for recovering multi-person motion from monocular videos.
Reconstructing Hand-Held Objects in 3D from Images and Videos
In order to obtain the best performing single frame model, we first present MCC-Hand-Object (MCC-HO), which jointly reconstructs hand and object geometry given a single RGB image and inferred 3D hand as inputs.
InstantSplat: Sparse-view SfM-free Gaussian Splatting in Seconds
InstantSplat adopts a self-supervised framework that bridges the gap between 2D images and 3D representations using Gaussian Bundle Adjustment (GauBA) and can be optimized in an end-to-end manner.
Reconstructing Hands in 3D with Transformers
The key to HaMeR's success lies in scaling up both the data used for training and the capacity of the deep network for hand reconstruction.
GART: Gaussian Articulated Template Models
We introduce Gaussian Articulated Template Model GART, an explicit, efficient, and expressive representation for non-rigid articulated subject capturing and rendering from monocular videos.
Generative Proxemics: A Prior for 3D Social Interaction from Images
To address this, we present a novel approach that learns a prior over the 3D proxemics two people in close social interaction and demonstrate its use for single-view 3D reconstruction.
Humans in 4D: Reconstructing and Tracking Humans with Transformers
To analyze video, we use 3D reconstructions from HMR 2. 0 as input to a tracking system that operates in 3D.
Ranked #3 on
Pose Tracking
on PoseTrack2018
Learning Articulated Shape with Keypoint Pseudo-labels from Web Images
It is based on two key insights: (1) 2D keypoint estimation networks trained on as few as 50-150 images of a given object category generalize well and generate reliable pseudo-labels; (2) a data selection mechanism can automatically create a "curated" subset of the unlabeled web images that can be used for training -- we evaluate four data selection methods.
On the Benefits of 3D Pose and Tracking for Human Action Recognition
Subsequently, we propose a Lagrangian Action Recognition model by fusing 3D pose and contextualized appearance over tracklets.
Ranked #1 on
Action Recognition
on AVA v2.2
(using extra training data)
Decoupling Human and Camera Motion from Videos in the Wild
Our method robustly recovers the global 3D trajectories of people in challenging in-the-wild videos, such as PoseTrack.
The One Where They Reconstructed 3D Humans and Environments in TV Shows
TV shows depict a wide variety of human behaviors and have been studied extensively for their potential to be a rich source of data for many applications.
Semantic keypoint-based pose estimation from single RGB frames
Empirically, we show that our approach can accurately recover the 6-DoF object pose for both instance- and class-based scenarios even against a cluttered background.
Tracking People by Predicting 3D Appearance, Location and Pose
For a future frame, we compute the similarity between the predicted state of a tracklet and the single frame observations in a probabilistic manner.
Tracking People by Predicting 3D Appearance, Location & Pose
For a future frame, we compute the similarity between the predicted state of a tracklet and the single frame observations in a probabilistic manner.
Tracking People with 3D Representations
We find that 3D representations are more effective than 2D representations for tracking in these settings, and we obtain state-of-the-art performance.
Probabilistic Modeling for Human Mesh Recovery
This paper focuses on the problem of 3D human reconstruction from 2D evidence.
Ranked #2 on
Multi-Hypotheses 3D Human Pose Estimation
on AH36M
Human Mesh Recovery from Multiple Shots
The tools we develop open the door to processing and analyzing in 3D content from a large library of edited media, which could be helpful for many downstream applications.
Independent Sign Language Recognition with 3D Body, Hands, and Face Reconstruction
Independent Sign Language Recognition is a complex visual recognition problem that combines several challenging tasks of Computer Vision due to the necessity to exploit and fuse information from hand gestures, body features and facial expressions.
Monocular Expressive Body Regression through Body-Driven Attention
To understand how people look, interact, or perform tasks, we need to quickly and accurately capture their 3D body, face, and hands together from an RGB image.
Coherent Reconstruction of Multiple Humans from a Single Image
Our goal is to train a single network that learns to avoid these problems and generate a coherent 3D reconstruction of all the humans in the scene.
Ranked #2 on
3D Human Reconstruction
on AGORA
Reactive Navigation in Partially Familiar Planar Environments Using Semantic Perceptual Feedback
This paper solves the planar navigation problem by recourse to an online reactive scheme that exploits recent advances in SLAM and visual object recognition to recast prior geometric knowledge in terms of an offline catalogue of familiar objects.
Robotics
TexturePose: Supervising Human Mesh Estimation with Texture Consistency
Assuming that the texture of the person does not change dramatically between frames, we can apply a novel texture consistency loss, which enforces that each point in the texture map has the same texture value across all frames.
Ranked #27 on
Weakly-supervised 3D Human Pose Estimation
on Human3.6M
Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop
Our approach is self-improving by nature, since better network estimates can lead the optimization to better solutions, while more accurate optimization fits provide better supervision for the network.
Convolutional Mesh Regression for Single-Image Human Shape Reconstruction
Image-based features are attached to the mesh vertices and the Graph-CNN is responsible to process them on the mesh structure, while the regression target for each vertex is its 3D location.
Ranked #35 on
Monocular 3D Human Pose Estimation
on Human3.6M
Expressive Body Capture: 3D Hands, Face, and Body from a Single Image
We use the new method, SMPLify-X, to fit SMPL-X to both controlled images and images in the wild.
Ranked #1 on
3D Human Reconstruction
on Expressive hands and faces dataset (EHF)
(TR V2V (mm), left hand metric)
Learning to Estimate 3D Human Pose and Shape from a Single Color Image
The proposed approach outperforms previous baselines on this task and offers an attractive solution for direct prediction of 3D shape from a single color image.
Ranked #127 on
3D Human Pose Estimation
on Human3.6M
(PA-MPJPE metric)
Ordinal Depth Supervision for 3D Human Pose Estimation
This information can be acquired by human annotators for a wide range of images and poses.
Ranked #1 on
Monocular 3D Human Pose Estimation
on Human3.6M
(Use Video Sequence metric)
Human Motion Capture Using a Drone
Current motion capture (MoCap) systems generally require markers and multiple calibrated cameras, which can be used only in constrained environments.
Harvesting Multiple Views for Marker-less 3D Human Pose Annotations
In this paper, we present a geometry-driven approach to automatically collect annotations for human pose prediction tasks.
Ranked #28 on
Weakly-supervised 3D Human Pose Estimation
on Human3.6M
6-DoF Object Pose from Semantic Keypoints
This paper presents a novel approach to estimating the continuous six degree of freedom (6-DoF) pose (3D translation and rotation) of an object from a single RGB image.
Ranked #1 on
Keypoint Detection
on Pascal3D+
MonoCap: Monocular Human Motion Capture using a CNN Coupled with a Geometric Prior
Recovering 3D full-body human pose is a challenging problem with many applications.
Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose
This paper addresses the challenge of 3D human pose estimation from a single color image.
Ranked #18 on
3D Human Pose Estimation
on HumanEva-I
