Search Results for author:
Found 176 papers, 35 papers with code
HTML: A Parametric Hand Texture Model for 3D Hand Reconstruction and Personalization
3D hand reconstruction from images is a widely-studied problem in computer vision and graphics, and has a particularly high relevance for virtual and augmented reality.
Neural Dense Non-Rigid Structure from Motion with Latent Space Constraints
We introduce the first dense neural non-rigid structure from motion (N-NRSfM) approach, which can be trained end-to-end in an unsupervised manner from 2D point tracks.
Regularized Vector Quantization for Tokenized Image Synthesis
The first is a prior distribution regularization which measures the discrepancy between a prior token distribution and the predicted token distribution to avoid codebook collapse and low codebook utilization.
NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusion
Novel view synthesis from a single image requires inferring occluded regions of objects and scenes whilst simultaneously maintaining semantic and physical consistency with the input.
Scene-Aware 3D Multi-Human Motion Capture from a Single Camera
In this work, we consider the problem of estimating the 3D position of multiple humans in a scene as well as their body shape and articulation from a single RGB video recorded with a static camera.
Imitator: Personalized Speech-driven 3D Facial Animation
To address this, we present Imitator, a speech-driven facial expression synthesis method, which learns identity-specific details from a short input video and produces novel facial expressions matching the identity-specific speaking style and facial idiosyncrasies of the target actor.
Scene-aware Egocentric 3D Human Pose Estimation
To this end, we propose an egocentric depth estimation network to predict the scene depth map from a wide-view egocentric fisheye camera while mitigating the occlusion of the human body with a depth-inpainting network.
IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions
Is it possible to synthesize such motion plausibly with a diverse set of objects and instructions?
NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstruction
To accelerate the training process, we integrate multi-resolution hash encodings into a neural surface representation and implement our whole algorithm in CUDA.
MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesis
Conventional methods for human motion synthesis are either deterministic or struggle with the trade-off between motion diversity and motion quality.
Fast Non-Rigid Radiance Fields from Monocularized Data
3D reconstruction and novel view synthesis of dynamic scenes from collections of single views recently gained increased attention.
NeuralUDF: Learning Unsigned Distance Fields for Multi-view Reconstruction of Surfaces with Arbitrary Topologies
In this paper, we propose to represent surfaces as the Unsigned Distance Function (UDF) and develop a new volume rendering scheme to learn the neural UDF representation.
GlowGAN: Unsupervised Learning of HDR Images from LDR Images in the Wild
Most in-the-wild images are stored in Low Dynamic Range (LDR) form, serving as a partial observation of the High Dynamic Range (HDR) visual world.
An Implicit Parametric Morphable Dental Model
Current dental models use an explicit mesh scene representation and model only the teeth, ignoring the gum.
FIRES: Fast Imaging and 3D Reconstruction of Archaeological Sherds
Extensive validation in labs and testing in excavation sites demonstrated that our FIRES system provides the first fast, accurate, portal, and cost-effective solution for the task of imaging and 3D reconstruction of sherds in archaeological excavations.
gCoRF: Generative Compositional Radiance Fields
We start with a global generative model (GAN) and learn to decompose it into different semantic parts using supervision from 2D segmentation masks.
State of the Art in Dense Monocular Non-Rigid 3D Reconstruction
3D reconstruction of deformable (or non-rigid) scenes from a set of monocular 2D image observations is a long-standing and actively researched area of computer vision and graphics.
HDHumans: A Hybrid Approach for High-fidelity Digital Humans
Photo-real digital human avatars are of enormous importance in graphics, as they enable immersive communication over the globe, improve gaming and entertainment experiences, and can be particularly beneficial for AR and VR settings.
HiFECap: Monocular High-Fidelity and Expressive Capture of Human Performances
Furthermore, we show that HiFECap outperforms the state-of-the-art human performance capture approaches qualitatively and quantitatively while for the first time capturing all aspects of the human.
HandFlow: Quantifying View-Dependent 3D Ambiguity in Two-Hand Reconstruction with Normalizing Flow
Through this, we demonstrate the quality of our probabilistic reconstruction and show that explicit ambiguity modeling is better-suited for this challenging problem.
Voxurf: Voxel-based Efficient and Accurate Neural Surface Reconstruction
Previous methods based on neural volume rendering mostly train a fully implicit model with MLPs, which typically require hours of training for a single scene.
Neural Novel Actor: Learning a Generalized Animatable Neural Representation for Human Actors
The learned representation can be used to synthesize novel view images of an arbitrary person from a sparse set of cameras, and further animate them with the user's pose control.
MoCapDeform: Monocular 3D Human Motion Capture in Deformable Scenes
3D human motion capture from monocular RGB images respecting interactions of a subject with complex and possibly deformable environments is a very challenging, ill-posed and under-explored problem.
UnrealEgo: A New Dataset for Robust Egocentric 3D Human Motion Capture
We present UnrealEgo, i. e., a new large-scale naturalistic dataset for egocentric 3D human pose estimation.
Neural Radiance Transfer Fields for Relightable Novel-view Synthesis with Global Illumination
Given a set of images of a scene, the re-rendering of this scene from novel views and lighting conditions is an important and challenging problem in Computer Vision and Graphics.
NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priors
The key idea of NeuRIS is to integrate estimated normal of indoor scenes as a prior in a neural rendering framework for reconstructing large texture-less shapes and, importantly, to do this in an adaptive manner to also enable the reconstruction of irregular shapes with fine details.
Learn to Predict How Humans Manipulate Large-sized Objects from Interactive Motions
We also observe that an object's intrinsic physical properties are useful for the object motion prediction, and thus design a set of object dynamic descriptors to encode such intrinsic properties.
EventNeRF: Neural Radiance Fields from a Single Colour Event Camera
At the core of our method is a neural radiance field trained entirely in a self-supervised manner from events while preserving the original resolution of the colour event channels.
GAN2X: Non-Lambertian Inverse Rendering of Image GANs
2D images are observations of the 3D physical world depicted with the geometry, material, and illumination components.
Unbiased 4D: Monocular 4D Reconstruction with a Neural Deformation Model
Capturing general deforming scenes from monocular RGB video is crucial for many computer graphics and vision applications.
Physics Informed Neural Fields for Smoke Reconstruction with Sparse Data
With a hybrid architecture that separates static and dynamic contents, fluid interactions with static obstacles are reconstructed for the first time without additional geometry input or human labeling.
HULC: 3D Human Motion Capture with Pose Manifold Sampling and Dense Contact Guidance
Marker-less monocular 3D human motion capture (MoCap) with scene interactions is a challenging research topic relevant for extended reality, robotics and virtual avatar generation.
BEHAVE: Dataset and Method for Tracking Human Object Interactions
We present BEHAVE dataset, the first full body human- object interaction dataset with multi-view RGBD frames and corresponding 3D SMPL and object fits along with the annotated contacts between them.
Direct Dense Pose Estimation
In addition, DDP is computationally more efficient than previous dense pose estimation methods, and it reduces jitters when applied to a video sequence, which is a problem plaguing the previous methods.
FIRe: Fast Inverse Rendering using Directional and Signed Distance Functions
To this end, we learn a signed distance function (SDF) along with our DDF model to represent a class of shapes.
Disentangled3D: Learning a 3D Generative Model with Disentangled Geometry and Appearance from Monocular Images
Our model can disentangle the geometry and appearance variations in the scene, i. e., we can independently sample from the geometry and appearance spaces of the generative model.
φ-SfT: Shape-from-Template with a Physics-Based Deformation Model
In contrast to previous works, this paper proposes a new SfT approach explaining 2D observations through physical simulations accounting for forces and material properties.
Playable Environments: Video Manipulation in Space and Time
We present Playable Environments - a new representation for interactive video generation and manipulation in space and time.
Estimating Egocentric 3D Human Pose in the Wild with External Weak Supervision
Specifically, we first generate pseudo labels for the EgoPW dataset with a spatio-temporal optimization method by incorporating the external-view supervision.
f-SfT: Shape-From-Template With a Physics-Based Deformation Model
In contrast to previous works, this paper proposes a new SfT approach explaining 2D observations through physical simulations accounting for forces and material properties.
Multimodal Image Synthesis and Editing: A Survey
As information exists in various modalities in real world, effective interaction and fusion among multimodal information plays a key role for the creation and perception of multimodal data in computer vision and deep learning research.
BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering
The wide span of viewing positions within these scenes yields multi-scale renderings with very different levels of detail, which poses great challenges to neural radiance field and biases it towards compromised results.
NeRF for Outdoor Scene Relighting
Photorealistic editing of outdoor scenes from photographs requires a profound understanding of the image formation process and an accurate estimation of the scene geometry, reflectance and illumination.
EgoRenderer: Rendering Human Avatars from Egocentric Camera Images
We next combine the target pose image and the textures into a combined feature image, which is transformed into the output color image using a neural image translation network.
A Deeper Look into DeepCap
Human performance capture is a highly important computer vision problem with many applications in movie production and virtual/augmented reality.
Advances in Neural Rendering
The reconstruction of such a scene representation from observations using differentiable rendering losses is known as inverse graphics or inverse rendering.
A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis
Motivated by the observation that a 3D object should look realistic from multiple viewpoints, these methods introduce a multi-view constraint as regularization to learn valid 3D radiance fields from 2D images.
StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis
We perform volume rendering only to produce a low-resolution feature map and progressively apply upsampling in 2D to address the first issue.
Gravity-Aware Monocular 3D Human-Object Reconstruction
We evaluate GraviCap on a new dataset with ground-truth annotations for persons and different objects undergoing free flights.
Neural Rays for Occlusion-aware Image-based Rendering
On such a 3D point, these generalization methods will include inconsistent image features from invisible views, which interfere with the radiance field construction.
StyleVideoGAN: A Temporal Generative Model using a Pretrained StyleGAN
We demonstrate that it suffices to train our temporal architecture on only 10 minutes of footage of 1 subject for about 6 hours.
Adiabatic Quantum Graph Matching with Permutation Matrix Constraints
In this work, we address such problems with emerging quantum computing technology and propose several reformulations of QAPs as unconstrained problems suitable for efficient execution on quantum hardware.
Egocentric Videoconferencing
Even holding a mobile phone camera in the front of the face while sitting for a long duration is not convenient.
Self-supervised Outdoor Scene Relighting
Outdoor scene relighting is a challenging problem that requires good understanding of the scene geometry, illumination and albedo.
NRST: Non-rigid Surface Tracking from Monocular Video
Our texture term exploits the orientation information in the micro-structures of the objects, e. g., the yarn patterns of fabrics.
HandVoxNet++: 3D Hand Shape and Pose Estimation using Voxel-Based Neural Networks
To address the limitations of the existing methods, we develop HandVoxNet++, i. e., a voxel-based deep network with 3D and graph convolutions trained in a fully supervised manner.
RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB Video
Moreover, we demonstrate that our approach offers previously unseen two-hand tracking performance from RGB, and quantitatively and qualitatively outperforms existing RGB-based methods that were not explicitly designed for two-hand interactions.
Fast Simultaneous Gravitational Alignment of Multiple Point Sets
The problem of simultaneous rigid alignment of multiple unordered point sets which is unbiased towards any of the inputs has recently attracted increasing interest, and several reliable methods have been newly proposed.
NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
In NeuS, we propose to represent a surface as the zero-level set of a signed distance function (SDF) and develop a new volume rendering method to train a neural SDF representation.
Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Camera
We present a novel method for real-time pose and shape reconstruction of two strongly interacting hands.
Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Control
To address this problem, we utilize a coarse body model as the proxy to unwarp the surrounding 3D space into a canonical pose.
Q-Match: Iterative Shape Matching via Quantum Annealing
Finding shape correspondences can be formulated as an NP-hard quadratic assignment problem (QAP) that becomes infeasible for shapes with high sampling density.
Real-time Deep Dynamic Characters
We propose a deep videorealistic 3D human character model displaying highly realistic shape, motion, and dynamic appearance learned in a new weakly supervised way from multi-view imagery.
Neural Monocular 3D Human Motion Capture with Physical Awareness
We present a new trainable system for physically plausible markerless 3D human motion capture, which achieves state-of-the-art results in a broad range of challenging scenarios.
Differentiable Event Stream Simulator for Non-Rigid 3D Tracking
This paper introduces the first differentiable simulator of event streams, i. e., streams of asynchronous brightness change signals recorded by event cameras.
Estimating Egocentric 3D Human Pose in Global Space
Furthermore, these methods suffer from limited accuracy and temporal instability due to ambiguities caused by the monocular setup and the severe occlusion in a strongly distorted egocentric perspective.
Efficient and Differentiable Shadow Computation for Inverse Problems
Differentiable rendering has received increasing interest for image-based inverse problems.
Towards High Fidelity Monocular Face Reconstruction with Rich Reflectance using Self-supervised Learning and Ray Tracing
In this paper, we build our work on the aforementioned approaches and propose a new method that greatly improves reconstruction quality and robustness in general scenes.
Ranked #10 on
3D Face Reconstruction
on NoW Benchmark
Adaptive Surface Normal Constraint for Depth Estimation
We present a novel method for single image depth estimation using surface normal constraints.
Synthesis of Compositional Animations from Textual Descriptions
Our model can generate plausible pose sequences for short sentences describing single actions as well as long compositional sentences describing multiple sequential and superimposed actions.
PhotoApp: Photorealistic Appearance Editing of Head Portraits
We present an approach for high-quality intuitive editing of the camera viewpoint and scene illumination in a portrait image.
HumanGAN: A Generative Model of Humans Images
We address these limitations and present a generative model for images of dressed humans offering control over pose, local body part appearance and garment style.
Style and Pose Control for Image Synthesis of Humans from a Single Monocular View
Photo-realistic re-rendering of a human from a single image with explicit control over body pose, shape and appearance enables a wide range of applications, such as human appearance transfer, virtual try-on, motion imitation, and novel view synthesis.
Learning Speech-driven 3D Conversational Gestures from Video
We propose the first approach to automatically and jointly synthesize both the synchronous 3D conversational body and hand gestures, as well as 3D face and head animations, of a virtual character from speech input.
Quantum Permutation Synchronization
We present QuantumSync, the first quantum algorithm for solving a synchronization problem in the context of computer vision.
Neural Re-Rendering of Humans from a Single Image
Human re-rendering from a single image is a starkly under-constrained problem, and state-of-the-art algorithms often exhibit undesired artefacts, such as over-smoothing, unrealistic distortions of the body parts and garments, or implausible changes of the texture.
Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video
We show that a single handheld consumer-grade camera is sufficient to synthesize sophisticated renderings of a dynamic scene from novel virtual camera views, e. g. a `bullet-time' video effect.
High-Fidelity Neural Human Motion Transfer from Monocular Video
We address these limitations for the first time in the literature and present a new framework which performs high-fidelity and temporally-consistent human motion transfer with natural pose-dependent non-rigid deformations, for several types of loose garments.
Monocular Real-time Full Body Capture with Inter-part Correlations
We present the first method for real-time full body capture that estimates shape and motion of body and hands together with a dynamic 3D face model from a single color image.
Ranked #11 on
3D Hand Pose Estimation
on FreiHAND
EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream
Due to the different data modality of event cameras compared to classical cameras, existing methods cannot be directly applied to and re-trained for event streams.
Pose-Guided Human Animation from a Single Image in the Wild
We present a new pose transfer method for synthesizing a human animation from a single image of a person controlled by a sequence of body poses.
i3DMM: Deep Implicit 3D Morphable Model of Human Heads
Our approach has the following favorable properties: (i) It is the first full head morphable model that includes hair.
Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks
We present a novel method for multi-view depth estimation from a single video, which is a critical task in various applications, such as perception, reconstruction and robot navigation.
Deep Physics-aware Inference of Cloth Deformation for Monocular Human Performance Capture
Recent monocular human performance capture approaches have shown compelling dense tracking results of the full body from a single RGB camera.
LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration
Formulating this closed loop is not straightforward because it is not trivial to force the output of the NN to be on the surface of the human model - outside this surface the human model is not even defined.
Learning Complete 3D Morphable Face Models from Images and Videos
Our network design and loss functions ensure a disentangled parameterization of not only identity and albedo, but also, for the first time, an expression basis.
Fast Gravitational Approach for Rigid Point Set Registration with Ordinary Differential Equations
This article introduces a new physics-based method for rigid point set alignment called Fast Gravitational Approach (FGA).
PIE: Portrait Image Embedding for Semantic Control
We present the first approach for embedding real portrait images in the latent space of StyleGAN, which allows for intuitive editing of the head pose, facial expression, and scene illumination in the image.
Monocular Reconstruction of Neural Face Reflectance Fields
The reflectance field of a face describes the reflectance properties responsible for complex lighting effects including diffuse, specular, inter-reflection and self shadowing.
PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Time
We, therefore, present PhysCap, the first algorithm for physically plausible, real-time and marker-less human 3D motion capture with a single colour camera at 25 fps.
PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations
At the level of patches, objects across different categories share similarities, which leads to more generalizable models.
Face2Face: Real-time Face Capture and Reenactment of RGB Videos
Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion.
Neural Sparse Voxel Fields
We also demonstrate several challenging tasks, including multi-scene learning, free-viewpoint rendering of a moving human, and large-scale scene rendering.
Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction
In this work, we present methodology that combines detail-rich implicit functions and parametric representations in order to reconstruct 3D models of people that remain controllable and accurate even in the presence of clothing.
Generative Model-Based Loss to the Rescue: A Method to Overcome Annotation Errors for Depth-Based Hand Pose Estimation
We propose to use a model-based generative loss for training hand pose estimators on depth images based on a volumetric hand model.
VideoForensicsHQ: Detecting High-quality Manipulated Face Videos
We introduce a new benchmark dataset for face video forgery detection, of unprecedented quality.
Vid2Curve: Simultaneous Camera Motion Estimation and Thin Structure Reconstruction from an RGB Video
We propose the first approach that simultaneously estimates camera motion and reconstructs the geometry of complex 3D thin structures in high quality from a color video captured by a handheld camera.
State of the Art on Neural Rendering
Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e. g., by the integration of differentiable rendering into network training.
HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map
The input to our method is a 3D voxelized depth map, and we rely on two hand shape representations.
Occlusion-Aware Depth Estimation with Adaptive Normal Constraints
We present a new learning-based method for multi-frame depth estimation from a color video, which is a fundamental problem in scene understanding, robot navigation or handheld 3D reconstruction.
StyleRig: Rigging StyleGAN for 3D Control over Portrait Images
StyleGAN generates photorealistic portrait images of faces with eyes, teeth, hair and context (neck, shoulders, background), but lacks a rig-like control over semantic face parameters that are interpretable in 3D, such as face pose, expressions, and scene illumination.
Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data
We present a novel method for monocular hand shape and pose estimation at unprecedented runtime performance of 100fps and at state-of-the-art accuracy.
DeepCap: Monocular Human Performance Capture Using Weak Supervision
Human performance capture is a highly important computer vision problem with many applications in movie production and virtual/augmented reality.
MINA: Convex Mixed-Integer Programming for Non-Rigid Shape Alignment
We present a convex mixed-integer programming formulation for non-rigid shape matching.
Neural Human Video Rendering by Learning Dynamic Textures and Rendering-to-Video Translation
In this paper, we propose a novel human video synthesis method that approaches these limiting factors by explicitly disentangling the learning of time-coherent fine-scale details from the embedding of the human in 2D screen space.
Image-guided Neural Object Rendering
Based on this 3D proxy, the appearance of a captured view can be warped into a new target view as in classical image-based rendering.
A Quantum Computational Approach to Correspondence Problems on Point Sets
Modern adiabatic quantum computers (AQC) are already used to solve difficult combinatorial optimisation problems in various domains of science.
Neural Voice Puppetry: Audio-driven Facial Reenactment
Neural Voice Puppetry has a variety of use-cases, including audio-driven video avatars, video dubbing, and text-driven video synthesis of a talking head.
DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Data
Applying data-driven approaches to non-rigid 3D reconstruction has been difficult, which we believe can be attributed to the lack of a large-scale training corpus.
Convex Optimisation for Inverse Kinematics
We consider the problem of inverse kinematics (IK), where one wants to find the parameters of a given kinematic skeleton that best explain a set of observed 3D joint locations.
Neural Style-Preserving Visual Dubbing
We present a style-preserving visual dubbing approach from single video inputs, which maintains the signature style of target actors when modifying facial expressions, including mouth motions, to match foreign languages.
Intrinsic Dynamic Shape Prior for Fast, Sequential and Dense Non-Rigid Structure from Motion with Detection of Temporally-Disjoint Rigidity
The reasons for the slow dissemination are the severe ill-posedness, high sensitivity to motion and deformation cues and the difficulty to obtain reliable point tracks in the vast majority of practical scenarios.
3D Morphable Face Models -- Past, Present and Future
In this paper, we provide a detailed survey of 3D Morphable Face Models over the 20 years since they were first proposed.
EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera
The high frame rate is a critical requirement for capturing fast human motions.
Multi-Garment Net: Learning to Dress 3D People from Images
We present Multi-Garment Network (MGN), a method to predict body shape and clothing, layered on top of the SMPL model from a few frames (1-8) of a video.
3D Human Pose Estimation
3D Shape Reconstruction From A Single 2D Image
Real-Time Global Illumination Decomposition of Videos
We propose the first approach for the decomposition of a monocular color video into direct and indirect illumination components in real time.
DispVoxNets: Non-Rigid Point Set Alignment with Supervised Learning Proxies
We introduce a supervised-learning framework for non-rigid point set alignment of a new kind - Displacements on Voxels Networks (DispVoxNets) - which abstracts away from the point set representation and regresses 3D displacement fields on regularly sampled proxy 3D voxel grids.
XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera
The first stage is a convolutional neural network (CNN) that estimates 2D and 3D pose features along with identity assignments for all visible joints of all individuals. We contribute a new architecture for this CNN, called SelecSLS Net, that uses novel selective long and short range skip connections to improve the information flow allowing for a drastically faster network without compromising accuracy.
Ranked #4 on
3D Multi-Person Pose Estimation
on MuPoTS-3D
3D Multi-Person Human Pose Estimation
3D Multi-Person Pose Estimation
+1
Text-based Editing of Talking-head Video
To edit a video, the user has to only edit the transcript, and an optimization strategy then chooses segments of the input corpus as base material.
EgoFace: Egocentric Face Performance Capture and Videorealistic Reenactment
Our lightweight setup allows operations in uncontrolled environments, and lends itself to telepresence applications such as video-conferencing from dynamic environments.
DEMEA: Deep Mesh Autoencoders for Non-Rigidly Deforming Objects
Mesh autoencoders are commonly used for dimensionality reduction, sampling and mesh modeling.
Emergence of Implicit Filter Sparsity in Convolutional Neural Networks
We show implicit filter level sparsity manifests in convolutional neural networks (CNNs) which employ Batch Normalization and ReLU activation, and are trained using adaptive gradient descent techniques with L2 regularization or weight decay.
Implicit Filter Sparsification In Convolutional Neural Networks
We show implicit filter level sparsity manifests in convolutional neural networks (CNNs) which employ Batch Normalization and ReLU activation, and are trained with adaptive gradient descent techniques and L2 regularization or weight decay.
IsMo-GAN: Adversarial Learning for Monocular Non-Rigid 3D Reconstruction
The majority of the existing methods for non-rigid 3D surface regression from monocular 2D images require an object template or point tracks over multiple frames as an input, and are still far from real-time processing rates.
Tex2Shape: Detailed Full Human Body Geometry From a Single Image
From a partial texture, we estimate detailed normal and vector displacement maps, which can be applied to a low-resolution smooth body model to add detail and clothing.
In the Wild Human Pose Estimation Using Explicit 2D Features and Intermediate 3D Representations
Convolutional Neural Network based approaches for monocular 3D human pose estimation usually require a large amount of training images with 3D pose annotations.
Learning to Reconstruct People in Clothing from a Single RGB Camera
We present a learning-based model to infer the personalized 3D shape of people from a few frames (1-8) of a monocular video in which the person is moving, in less than 10 seconds with a reconstruction accuracy of 5mm.
Learning a Disentangled Embedding for Monocular 3D Shape Retrieval and Pose Estimation
We propose a novel approach to jointly perform 3D shape retrieval and pose estimation from monocular images. In order to make the method robust to real-world image variations, e. g. complex textures and backgrounds, we learn an embedding space from 3D data that only includes the relevant information, namely the shape and pose.
FML: Face Model Learning from Videos
In contrast, we propose multi-frame video-based self-supervised training of a deep network that (i) learns a face identity model both in shape and appearance while (ii) jointly learning to reconstruct 3D faces.
On Implicit Filter Level Sparsity in Convolutional Neural Networks
We investigate filter level sparsity that emerges in convolutional neural networks (CNNs) which employ Batch Normalization and ReLU activation, and are trained with adaptive gradient descent techniques and L2 regularization or weight decay.
Higher-order Projected Power Iterations for Scalable Multi-Matching
The matching of multiple objects (e. g. shapes or images) is a fundamental problem in vision and graphics.
IGNOR: Image-guided Neural Object Rendering
Based on this 3D proxy, the appearance of a captured view can be warped into a new target view as in classical image-based rendering.
LiveCap: Real-time Human Performance Capture from Monocular Video
Our method is the first real-time monocular approach for full-body performance capture.
Neural Rendering and Reenactment of Human Actor Videos
In contrast to conventional human character rendering, we do not require the availability of a production-quality photo-realistic 3D model of the human, but instead rely on a video sequence in conjunction with a (medium-quality) controllable 3D template model of the person.
Detailed Human Avatars from Monocular Video
We present a novel method for high detail-preserving human avatar creation from monocular video.
HeadOn: Real-time Reenactment of Human Portrait Videos
We propose HeadOn, the first real-time source-to-target reenactment approach for complete human portrait videos that enables transfer of torso and head motion, face expression, and eye gaze.
Deep Video Portraits
In order to enable source-to-target video re-animation, we render a synthetic target video with the reconstructed head animation parameters from a source video, and feed it into the trained network -- thus taking full control of the target.
A Hybrid Model for Identity Obfuscation by Face Replacement
As more and more personal photos are shared and tagged in social media, avoiding privacy risks such as unintended recognition becomes increasingly challenging.
Synchronisation of Partial Multi-Matchings via Non-negative Factorisations
In order to deal with the inherent non-convexity of the permutation synchronisation problem, we use an initialisation procedure based on a novel rotation scheme applied to the solution of the spectral relaxation.
Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera
We tackle these challenges based on a novel lightweight setup that converts a standard baseball cap to a device for high-quality pose estimation based on a single cap-mounted fisheye camera.
Video Based Reconstruction of 3D People Models
This paper describes how to obtain accurate 3D body models and texture of arbitrary people from a single, monocular video in which a person is moving.
LIME: Live Intrinsic Material Estimation
We present the first end to end approach for real time material estimation for general object shapes with uniform material that only requires a single color image as input.
Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB
Our approach uses novel occlusion-robust pose-maps (ORPM) which enable full body pose inference even under strong partial occlusions by other people and objects in the scene.
Ranked #3 on
3D Multi-Person Pose Estimation (root-relative)
on MuPoTS-3D
(MPJPE metric)
3D Human Pose Estimation
3D Multi-Person Pose Estimation (absolute)
+2
Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hz
To alleviate this problem, we present the first approach that jointly learns 1) a regressor for face shape, expression, reflectance and illumination on the basis of 2) a concurrently learned parametric face model.
GANerated Hands for Real-time 3D Hand Tracking from Monocular RGB
We address the highly challenging problem of real-time 3D hand tracking based on a monocular RGB-only sequence.
DS*: Tighter Lifting-Free Convex Relaxations for Quadratic Matching Problems
In this work we study convex relaxations of quadratic optimisation problems over permutation matrices.
HandSeg: An Automatically Labeled Dataset for Hand Segmentation from Depth Images
We propose an automatic method for generating high-quality annotations for depth-based hand segmentation, and introduce a large-scale hand segmentation dataset.
MonoPerfCap: Human Performance Capture from Monocular Video
Reconstruction from monocular video alone is drastically more challenging, since strong occlusions and the inherent depth ambiguity lead to a highly ill-posed reconstruction problem.
Criteria Sliders: Learning Continuous Database Criteria via Interactive Ranking
Large databases are often organized by hand-labeled metadata, or criteria, which are expensive to collect.
VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera
A real-time kinematic skeleton fitting method uses the CNN output to yield temporally stable 3D global pose reconstructions on the basis of a coherent kinematic skeleton.
Ranked #16 on
Pose Estimation
on Leeds Sports Poses
Real-time Hand Tracking under Occlusion from an Egocentric RGB-D Sensor
We present an approach for real-time, robust and accurate hand pose estimation from moving egocentric RGB-D cameras in cluttered real environments.
InverseFaceNet: Deep Monocular Inverse Face Rendering
In contrast, we propose to recover high-quality facial pose, shape, expression, reflectance and illumination using a deep neural network that is trained using a large, synthetically created training corpus.
MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction
In this work we propose a novel model-based deep convolutional autoencoder that addresses the highly challenging problem of reconstructing a 3D human face from a single in-the-wild color image.
EgoCap: Egocentric Marker-less Motion Capture with Two Fisheye Cameras (Extended Abstract)
Marker-based and marker-less optical skeletal motion-capture methods use an outside-in arrangement of cameras placed around a scene, with viewpoints converging on the center.
Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision
We propose a CNN-based approach for 3D human body pose estimation from single RGB images that addresses the issue of limited generalizability of models trained solely on the starkly limited publicly available 3D pose data.
Ranked #17 on
Pose Estimation
on Leeds Sports Poses
Real-time Halfway Domain Reconstruction of Motion and Geometry
Our extensive experiments and evaluations show that our approach produces high-quality dense reconstructions of 3D geometry and scene flow at real-time frame rates, and compares favorably to the state of the art.
Model-based Outdoor Performance Capture
We propose a new model-based method to accurately reconstruct human performances captured outdoors in a multi-camera setup.
Real-time Joint Tracking of a Hand Manipulating an Object from RGB-D Input
However, due to difficult occlusions, fast motions, and uniform hand appearance, jointly tracking hand and object pose is more challenging than tracking either of the two separately.
Video Depth-From-Defocus
Many compelling video post-processing effects, in particular aesthetic focus editing and refocusing effects, are feasible if per-frame depth information is available.
FaceVR: Real-Time Facial Reenactment and Eye Gaze Control in Virtual Reality
Based on reenactment of a prerecorded stereo video of the person without the HMD, FaceVR incorporates photo-realistic re-rendering in real time, thus allowing artificial modifications of face and eye appearances.
EgoCap: Egocentric Marker-less Motion Capture with Two Fisheye Cameras
We therefore propose a new method for real-time, marker-less and egocentric motion capture which estimates the full-body skeleton pose from a lightweight stereo pair of fisheye cameras that are attached to a helmet or virtual reality headset.
Dense Wide-Baseline Scene Flow From Two Handheld Video Cameras
We finally refine the computed correspondence fields in a variational scene flow formulation.
General Automatic Human Shape and Motion Capture Using Volumetric Contour Cues
Our method uses a new image formation model with analytic visibility and analytically differentiable alignment energy.
Opt: A Domain Specific Language for Non-linear Least Squares Optimization in Graphics and Imaging
Many graphics and vision problems can be expressed as non-linear least squares optimizations of objective functions over visual data, such as images and meshes.
BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integration
Our approach estimates globally optimized (i. e., bundle adjusted) poses in real-time, supports robust tracking with recovery from gross tracking failures (i. e., relocalization), and re-estimates the 3D model in real-time to ensure global consistency; all within a single framework.
VolumeDeform: Real-time Volumetric Non-rigid Reconstruction
We cast finding the optimal deformation of space as a non-linear regularized variational optimization problem by enforcing local smoothness and proximity to the input constraints.
Context-guided diffusion for label propagation on graphs
Existing approaches for diffusion on graphs, e. g., for label propagation, are mainly focused on isotropic diffusion, which is induced by the commonly-used graph Laplacian regularizer.
Fast and Robust Hand Tracking Using Detection-Guided Optimization
In the optimization step, a novel objective function combines the detected part labels and a Gaussian mixture representation of the depth to estimate a pose that best fits the depth.
Local High-order Regularization on Data Manifolds
The iterated graph Laplacian enables high-order regularization, but it has a high computational complexity and so cannot be applied to large problems.
A Versatile Scene Model with Differentiable Visibility Applied to Generative Pose Estimation
Generative reconstruction methods compute the 3D configuration (such as pose and/or geometry) of a shape by optimizing the overlap of the projected 3D shape model with images.
Real-Time Hand Tracking Using a Sum of Anisotropic Gaussians Model
In this paper, we propose a new approach that tracks the full skeleton motion of the hand from multiple RGB cameras in real-time.
Semi-supervised Learning with Explicit Relationship Regularization
In many learning tasks, the structure of the target space of a function holds rich information about the relationships between evaluations of functions on different data points.
Automatic Face Reenactment
We propose an image-based, facial reenactment system that replaces the face of an actor in an existing target video with the face of a user from a source video, while preserving the original target performance.
Efficient Multi-view Performance Capture of Fine-Scale Surface Detail
We present a new effective way for performance capture of deforming meshes with fine-scale time-varying surface detail from multi-view video.
Efficient ConvNet-Based Marker-Less Motion Capture in General Scenes With a Low Number of Cameras
Our approach unites a discriminative image-based joint detection method with a model-based generative motion tracking algorithm through a combined pose optimization energy.
Building Statistical Shape Spaces for 3D Human Modeling
Statistical models of 3D human shape and pose learned from scan databases have developed into valuable tools to solve a variety of vision and graphics problems.
