Search Results for author:
Found 32 papers, 6 papers with code
Learning Motion-Dependent Appearance for High-Fidelity Rendering of Dynamic Humans from a Single Camera
Appearance of dressed humans undergoes a complex geometric transformation induced not only by the static pose but also by its dynamics, i. e., there exists a number of cloth geometric configurations given a pose depending on the way it has moved.
PoseKernelLifter: Metric Lifting of 3D Human Pose using Sound
Based on this insight, we introduce a time-invariant transfer function called pose kernel -- the impulse response of audio signals induced by the body pose.
Ego4D: Around the World in 3,000 Hours of Egocentric Video
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite.
Self-supervised Secondary Landmark Detection via 3D Representation Learning
We hypothesize that there exists a shared representation between the primary and secondary landmarks because the range of motion of the secondary landmarks can be approximately spanned by that of the primary landmarks.
HUMBI: A Large Multiview Dataset of Human Body Expressions and Benchmark Challenge
We demonstrate that HUMBI is highly effective in learning and reconstructing a complete human model and is complementary to the existing datasets of human body expressions with limited views and subjects such as MPII-Gaze, Multi-PIE, Human3. 6M, and Panoptic Studio datasets.
Semi-supervised Dense Keypointsusing Unlabeled Multiview Images
(2) Geometric consistency: every point in the continuous correspondence fields must satisfy the multiview consistency collectively.
Automated Tracking of Primate Behavior
The availability of such data has in turn spurred developments in data analysis techniques.
Inverse Simulation: Reconstructing Dynamic Geometry of Clothed Humans via Optimal Control
Inspired by the theory of optimal control, we optimize the body states such that the simulated cloth motion is matched to the point cloud measurements, and the analytic gradient of the simulator is back-propagated to update the body states.
Self-supervised 3D Representation Learning of Dressed Humans from Social Media Videos
A key challenge of learning a visual representation for the 3D high fidelity geometry of dressed humans lies in the limited availability of the ground truth data (e. g., 3D scanned models), which results in the performance degradation of 3D human reconstruction when applying to real-world imagery.
Neural 3D Clothes Retargeting from a Single Image
In this paper, we present a method of clothes retargeting; generating the potential poses and deformations of a given 3D clothing template model to fit onto a person in a single RGB image.
Pose-Guided Human Animation from a Single Image in the Wild
We present a new pose transfer method for synthesizing a human animation from a single image of a person controlled by a sequence of body poses.
Surface Normal Estimation of Tilted Images via Spatial Rectifier
Our two main hypotheses are: (1) visual scene layout is indicative of the gravity direction; and (2) not all surfaces are equally represented by a learned estimator due to the structured distribution of the training data, thus, there exists a transformation for each tilted image that is more responsive to the learned estimator than others.
Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Camera
Our insight is that although its scale and quality are inconsistent with other views, the depth estimation from a single view can be used to reason about the globally coherent geometry of dynamic contents.
Self-Supervised Adaptation of High-Fidelity Face Models for Monocular Performance Tracking
In this paper, we propose a self-supervised domain adaptation approach to enable the animation of high-fidelity face models from a commodity camera.
Multiview Cross-supervision for Semantic Segmentation
We hypothesize that it is possible to leverage multiview image streams that are linked through the underlying 3D geometry, which can provide an additional supervisionary signal to train a segmentation model.
ECO: Egocentric Cognitive Mapping
We present a new method to localize a camera within a previously unseen environment perceived from an egocentric point of view.
HUMBI: A Large Multiview Dataset of Human Body Expressions
This paper presents a new large multiview dataset called HUMBI for human body expressions with natural clothing.
Multiview Supervision By Registration
This paper presents a semi-supervised learning framework to train a keypoint detector using multiview image streams given the limited labeled data (typically $<$4\%).
MONET: Multiview Semi-supervised Keypoint Detection via Epipolar Divergence
While multiview geometry can be used to self-supervise the unlabeled data, integrating the geometry into learning a keypoint detector is challenging due to representation mismatch.
3D Semantic Trajectory Reconstruction from 3D Pixel Continuum
This paper presents a method to reconstruct dense semantic trajectory stream of human interactions in 3D from synchronized multiple videos.
Predicting Behaviors of Basketball Players From First Person Videos
This paper presents a method to predict the future movements (location and gaze direction) of basketball players as a whole from their first person videos.
Customizing First Person Image Through Desired Actions
Our conjecture is that the spatial arrangement of a first person visual scene is deployed to afford an action, and therefore, the action can be inversely used to synthesize a new scene such that the action is feasible.
Social Behavior Prediction from First Person Videos
This paper presents a method to predict the future movements (location and gaze direction) of basketball players as a whole from their first person videos.
Am I a Baller? Basketball Performance Assessment from First-Person Videos
Finally, we use this feature to learn a basketball assessment model from pairs of labeled first-person basketball videos, for which a basketball expert indicates, which of the two players is better.
Unsupervised Learning of Important Objects from First-Person Videos
In this work, we show that we can detect important objects in first-person images without the supervision by the camera wearer or even third-person labelers.
Force From Motion: Decoding Physical Sensation in a First Person Video
In this paper, we focus on a problem of Force from Motion---decoding the sensation of 1) passive forces such as the gravity, 2) the physical scale of the motion (speed) and space, and 3) active forces exerted by the observer such as pedaling a bike or banking on a ski turn.
Egocentric Future Localization
We refine them by minimizing a cost function that describes compatibility between the obstacles in the EgoRetinal map and trajectories.
First Person Action-Object Detection with EgoNet
Unlike traditional third-person cameras mounted on robots, a first-person camera, captures a person's visual sensorimotor object interactions from up close.
Exploiting Egocentric Object Prior for 3D Saliency Detection
We empirically show that this representation can accurately characterize the egocentric object prior by testing it on an egocentric RGBD dataset for three tasks: the 3D saliency detection, future saliency prediction, and interaction classification.
Future Localization from an Egocentric Depth Image
As a byproduct of the predicted trajectories of ego-motion, we discover in the image the empty space occluded by foreground objects.
MAP Visibility Estimation for Large-Scale Dynamic 3D Reconstruction
Many traditional challenges in reconstructing 3D motion, such as matching across wide baselines and handling occlusion, reduce in significance as the number of unique viewpoints increases.
