Search Results for author:
Found 34 papers, 16 papers with code
IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generation
A mitigation is to fine-tune the 2D generator to be multi-view aware, which can help distillation or can be combined with reconstruction networks to output 3D objects directly.
Mosaic-SDF for 3D Generative Models
Current diffusion or flow-based generative models for 3D shapes divide to two: distilling pre-trained 2D image diffusion models, and training directly on 3D shapes.
Replay: Multi-modal Multi-view Acted Videos for Casual Holography
We introduce Replay, a collection of multi-view, multi-modal videos of humans interacting socially.
CoTracker: It is Better to Track Together
We introduce CoTracker, a transformer-based model that tracks dense points in a frame jointly across a video sequence.
DynamicStereo: Consistent Dynamic Depth from Stereo Videos
The network learns to pool information from neighboring frames to improve the temporal consistency of its predictions.
Real-time volumetric rendering of dynamic humans
We present a method for fast 3D reconstruction and real-time rendering of dynamic humans from monocular videos with accompanying parametric body fits.
Novel-View Acoustic Synthesis
We introduce the novel-view acoustic synthesis (NVAS) task: given the sight and sound observed at a source viewpoint, can we synthesize the sound of that scene from an unseen target viewpoint?
Self-Supervised Correspondence Estimation via Multiview Registration
To address this, we propose a self-supervised approach for correspondence estimation that learns from multiview consistency in short RGB-D video sequences.
Common Pets in 3D: Dynamic New-View Synthesis of Real-Life Deformable Categories
Obtaining photorealistic reconstructions of objects from sparse views is inherently ambiguous and can only be achieved by learning suitable reconstruction priors.
Filtered-CoPhy: Unsupervised Learning of Counterfactual Physics in Pixel Space
Learning causal relationships in high-dimensional data (images, videos) is a hard task, as they are often defined on low dimensional manifolds and must be extracted from complex signals dominated by appearance, lighting, textures and also spurious correlations in the data.
KeyTr: Keypoint Transporter for 3D Reconstruction of Deformable Objects in Videos
Compared to weaker deformation models, this significantly reduces the reconstruction ambiguity and, for dynamic objects, allows Keypoint Transporter to obtain reconstructions of the quality superior or at least comparable to prior approaches while being much faster and reliant on a pre-trained monocular depth estimator network.
BANMo: Building Animatable 3D Neural Models from Many Casual Videos
Our key insight is to merge three schools of thought; (1) classic deformable shape models that make use of articulated bones and blend skinning, (2) volumetric neural radiance fields (NeRFs) that are amenable to gradient-based optimization, and (3) canonical embeddings that generate correspondences between pixels and an articulated model.
Discovering Relationships between Object Categories via Universal Canonical Maps
Recent work has shown that it is possible to learn a unified dense pose predictor for several categories of related objects.
XCiT: Cross-Covariance Image Transformers
We propose a "transposed" version of self-attention that operates across feature channels rather than tokens, where the interactions are based on the cross-covariance matrix between keys and queries.
Ranked #55 on
Instance Segmentation
on COCO minival
NeuroMorph: Unsupervised Shape Interpolation and Correspondence in One Go
We present NeuroMorph, a new neural network architecture that takes as input two 3D shapes and produces in one go, i. e. in a single feed forward pass, a smooth interpolation and point-to-point correspondences between them.
Cross-Modal Retrieval Augmentation for Multi-Modal Classification
Recent advances in using retrieval components over external knowledge sources have shown impressive results for a variety of downstream tasks in natural language processing.
Training Vision Transformers for Image Retrieval
Transformers have shown outstanding results for natural language understanding and, more recently, for image classification.
Continuous Surface Embeddings
In this work, we focus on the task of learning and representing dense correspondences in deformable object categories.
Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimation
Remarkably, the resulting annotations are sufficient to train from scratch 3D pose regressor networks that outperform the current state-of-the-art on in-the-wild benchmarks such as 3DPW.
Ranked #22 on
3D Human Pose Estimation
on MPI-INF-3DHP
(PA-MPJPE metric)
Transferring Dense Pose to Proximal Animal Classes
Recent contributions have demonstrated that it is possible to recognize the pose of humans densely and accurately given a large dataset of poses annotated in detail.
Correlated Uncertainty for Learning Dense Correspondences from Noisy Labels
We show that these models, by understanding uncertainty better, can solve the original DensePose task more accurately, thus setting the new state-of-the-art accuracy in this benchmark.
CoPhy: Counterfactual Learning of Physical Dynamics
Understanding causes and effects in mechanical systems is an essential component of reasoning in the physical world.
C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion
We propose C3DPO, a method for extracting 3D models of deformable objects from 2D keypoint annotations in unconstrained images.
Slim DensePose: Thrifty Learning from Sparse Annotations and Motion Cues
DensePose supersedes traditional landmark detectors by densely mapping image pixels to body surface coordinates.
Dense Pose Transfer
In this work we integrate ideas from surface-based modeling with neural synthesis: we propose a combination of surface-based pose estimation and deep generative models that allows us to perform accurate pose transfer, i. e. synthesize a new image of a person based on a single image of that person and the image of a pose donor.
Object Level Visual Reasoning in Videos
Human activity recognition is typically addressed by detecting key concepts like global and local motion, features related to object classes present in the scene, as well as features related to the global context.
Ranked #1 on
Semantic Object Interaction Classification
on VLOG
DensePose: Dense Human Pose Estimation In The Wild
In this work, we establish dense correspondences between RGB image and a surface-based representation of the human body, a task we refer to as dense human pose estimation.
Ranked #2 on
Pose Estimation
on DensePose-COCO
Houdini: Fooling Deep Structured Visual and Speech Recognition Models with Adversarial Examples
Generating adversarial examples is a critical step for evaluating and improving the robustness of learning machines.
Mass Displacement Networks
Despite the large improvements in performance attained by using deep learning in computer vision, one can often further improve results with some additional post-processing that exploits the geometric nature of the underlying task.
Houdini: Fooling Deep Structured Prediction Models
Generating adversarial examples is a critical step for evaluating and improving the robustness of learning machines.
Predicting Deeper into the Future of Semantic Segmentation
The ability to predict and therefore to anticipate the future is an important attribute of intelligence.
Hand Pose Estimation through Semi-Supervised and Weakly-Supervised Learning
We propose a method for hand pose estimation based on a deep regressor trained on two different kinds of input.
Learning Human Identity from Motion Patterns
We present a large-scale study exploring the capability of temporal deep neural networks to interpret natural human kinematics and introduce the first method for active biometric authentication with mobile inertial sensors.
ModDrop: adaptive multi-modal gesture recognition
We present a method for gesture detection and localisation based on multi-scale and multi-modal deep learning.
