Search Results for author:
Found 50 papers, 11 papers with code
XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera
The first stage is a convolutional neural network (CNN) that estimates 2D and 3D pose features along with identity assignments for all visible joints of all individuals. We contribute a new architecture for this CNN, called SelecSLS Net, that uses novel selective long and short range skip connections to improve the information flow allowing for a drastically faster network without compromising accuracy.
Ranked #7 on
3D Multi-Person Pose Estimation
on MuPoTS-3D
3D Multi-Person Human Pose Estimation
3D Multi-Person Pose Estimation
+1
VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera
A real-time kinematic skeleton fitting method uses the CNN output to yield temporally stable 3D global pose reconstructions on the basis of a coherent kinematic skeleton.
Ranked #16 on
Pose Estimation
on Leeds Sports Poses
High-Fidelity Neural Human Motion Transfer from Monocular Video
We address these limitations for the first time in the literature and present a new framework which performs high-fidelity and temporally-consistent human motion transfer with natural pose-dependent non-rigid deformations, for several types of loose garments.
i3DMM: Deep Implicit 3D Morphable Model of Human Heads
Our approach has the following favorable properties: (i) It is the first full head morphable model that includes hair.
Revisiting Image Deblurring with an Efficient ConvNet
In this work, we propose a unified lightweight CNN network that features a large effective receptive field (ERF) and demonstrates comparable or even better performance than Transformers while bearing less computational costs.
Ranked #2 on
Image Defocus Deblurring
on DPD
EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream
Due to the different data modality of event cameras compared to classical cameras, existing methods cannot be directly applied to and re-trained for event streams.
Towards a quality metric for dense light fields
We find that the existing image quality metrics provide good measures of light-field quality, but require dense reference light- fields for optimal performance.
PhotoApp: Photorealistic Appearance Editing of Head Portraits
We present an approach for high-quality intuitive editing of the camera viewpoint and scene illumination in a portrait image.
LIME: Live Intrinsic Material Estimation
We present the first end to end approach for real time material estimation for general object shapes with uniform material that only requires a single color image as input.
Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera
We tackle these challenges based on a novel lightweight setup that converts a standard baseball cap to a device for high-quality pose estimation based on a single cap-mounted fisheye camera.
Ranked #6 on
Egocentric Pose Estimation
on GlobalEgoMocap Test Dataset
(using extra training data)
MonoPerfCap: Human Performance Capture from Monocular Video
Reconstruction from monocular video alone is drastically more challenging, since strong occlusions and the inherent depth ambiguity lead to a highly ill-posed reconstruction problem.
EgoCap: Egocentric Marker-less Motion Capture with Two Fisheye Cameras (Extended Abstract)
Marker-based and marker-less optical skeletal motion-capture methods use an outside-in arrangement of cameras placed around a scene, with viewpoints converging on the center.
General Automatic Human Shape and Motion Capture Using Volumetric Contour Cues
Our method uses a new image formation model with analytic visibility and analytically differentiable alignment energy.
Model-based Outdoor Performance Capture
We propose a new model-based method to accurately reconstruct human performances captured outdoors in a multi-camera setup.
EgoCap: Egocentric Marker-less Motion Capture with Two Fisheye Cameras
We therefore propose a new method for real-time, marker-less and egocentric motion capture which estimates the full-body skeleton pose from a lightweight stereo pair of fisheye cameras that are attached to a helmet or virtual reality headset.
Deep Shading: Convolutional Neural Networks for Screen-Space Shading
In computer vision, convolutional neural networks (CNNs) have recently achieved new levels of performance for several inverse problems where RGB pixel appearance is mapped to attributes such as positions, normals or reflectance.
Real-Time Hand Tracking Using a Sum of Anisotropic Gaussians Model
In this paper, we propose a new approach that tracks the full skeleton motion of the hand from multiple RGB cameras in real-time.
A Versatile Scene Model with Differentiable Visibility Applied to Generative Pose Estimation
Generative reconstruction methods compute the 3D configuration (such as pose and/or geometry) of a shape by optimizing the overlap of the projected 3D shape model with images.
A Low-Dimensional Representation for Robust Partial Isometric Correspondences Computation
In this paper, we introduce a new approach to partial, intrinsic isometric matching.
FML: Face Model Learning from Videos
In contrast, we propose multi-frame video-based self-supervised training of a deep network that (i) learns a face identity model both in shape and appearance while (ii) jointly learning to reconstruct 3D faces.
Discovering the Structure of a Planar Mirror System from Multiple Observations of a Single Point
We investigate the problem of identifying the position of a viewer inside a room of planar mirrors with unknown geometry in conjunction with the room's shape parameters.
EgoFace: Egocentric Face Performance Capture and Videorealistic Reenactment
Our lightweight setup allows operations in uncontrolled environments, and lends itself to telepresence applications such as video-conferencing from dynamic environments.
Neural Style-Preserving Visual Dubbing
We present a style-preserving visual dubbing approach from single video inputs, which maintains the signature style of target actors when modifying facial expressions, including mouth motions, to match foreign languages.
Neural View-Interpolation for Sparse Light Field Video
We suggest representing light field (LF) videos as "one-off" neural networks (NN), i. e., a learned mapping from view-plus-time coordinates to high-resolution color values, trained on sparse views.
StyleRig: Rigging StyleGAN for 3D Control over Portrait Images
StyleGAN generates photorealistic portrait images of faces with eyes, teeth, hair and context (neck, shoulders, background), but lacks a rig-like control over semantic face parameters that are interpretable in 3D, such as face pose, expressions, and scene illumination.
VideoForensicsHQ: Detecting High-quality Manipulated Face Videos
We introduce a new benchmark dataset for face video forgery detection, of unprecedented quality.
Monocular Reconstruction of Neural Face Reflectance Fields
The reflectance field of a face describes the reflectance properties responsible for complex lighting effects including diffuse, specular, inter-reflection and self shadowing.
PIE: Portrait Image Embedding for Semantic Control
We present the first approach for embedding real portrait images in the latent space of StyleGAN, which allows for intuitive editing of the head pose, facial expression, and scene illumination in the image.
X-Fields: Implicit Neural View-, Light- and Time-Image Interpolation
We suggest to represent an X-Field -a set of 2D images taken across different view, time or illumination conditions, i. e., video, light field, reflectance fields or combinations thereof-by learning a neural network (NN) to map their view, time or light coordinates to 2D images.
Learning Complete 3D Morphable Face Models from Images and Videos
Our network design and loss functions ensure a disentangled parameterization of not only identity and albedo, but also, for the first time, an expression basis.
HDR Denoising and Deblurring by Learning Spatio-temporal Distortion Models
Regrettably, capturing DISTORTED sensor readings is time-consuming; as well, there is a lack of CLEAN HDR videos.
Learning Speech-driven 3D Conversational Gestures from Video
We propose the first approach to automatically and jointly synthesize both the synchronous 3D conversational body and hand gestures, as well as 3D face and head animations, of a virtual character from speech input.
Ranked #4 on
Gesture Generation
on BEAT2
Differentiable Event Stream Simulator for Non-Rigid 3D Tracking
This paper introduces the first differentiable simulator of event streams, i. e., streams of asynchronous brightness change signals recorded by event cameras.
Egocentric Videoconferencing
Even holding a mobile phone camera in the front of the face while sitting for a long duration is not convenient.
Self-supervised Outdoor Scene Relighting
Outdoor scene relighting is a challenging problem that requires good understanding of the scene geometry, illumination and albedo.
Mixed Integer Neural Inverse Design
Our mixed-integer inverse design uncovers globally optimal or near optimal solutions in a principled manner.
Learning a self-supervised tone mapping operator via feature contrast masking loss
High Dynamic Range (HDR) content is becoming ubiquitous due to the rapid development of capture technologies.
Neural Relightable Participating Media Rendering
We propose to learn neural representations for participating media with a complete simulation of global illumination.
Eikonal Fields for Refractive Novel-View Synthesis
We tackle the problem of generating novel-view images from collections of 2D images showing refractive and reflective objects.
Physics Informed Neural Fields for Smoke Reconstruction with Sparse Data
With a hybrid architecture that separates static and dynamic contents, fluid interactions with static obstacles are reconstructed for the first time without additional geometry input or human labeling.
Video frame interpolation for high dynamic range sequences captured with dual-exposure sensors
Video frame interpolation (VFI) enables many important applications that might involve the temporal domain, such as slow motion playback, or the spatial domain, such as stop motion sequences.
Autoinverse: Uncertainty Aware Inversion of Neural Networks
Neural networks are powerful surrogates for numerous forward processes.
GlowGAN: Unsupervised Learning of HDR Images from LDR Images in the Wild
Most in-the-wild images are stored in Low Dynamic Range (LDR) form, serving as a partial observation of the High Dynamic Range (HDR) visual world.
Neural Field Convolutions by Repeated Differentiation
Neural fields are evolving towards a general-purpose continuous representation for visual computing.
Enhancing image quality prediction with self-supervised visual masking
Full-reference image quality metrics (FR-IQMs) aim to measure the visual differences between a pair of reference and distorted images, with the goal of accurately predicting human judgments.
Large-Batch, Iteration-Efficient Neural Bayesian Design Optimization
Bayesian optimization (BO) provides a powerful framework for optimizing black-box, expensive-to-evaluate functions.
Dynamic Range Independent Image Quality Assessment
Current quality assessment metrics are not suitable for this task, as they assume that both reference and test images have the same dynamic range.
Video Quality Assessment for Computer Graphics Applications
We present a full-reference video quality metric geared specifically towards the requirements of Computer Graphics applications as a faster computational alternative to subjective evaluation.
Joint Sampling and Optimisation for Inverse Rendering
By combining proportional and finite-difference samples, we continuously reduce the variance of our novel gradient meta-estimators throughout the optimisation process.
TrustMol: Trustworthy Inverse Molecular Design via Alignment with Molecular Dynamics
Moreover, we propose a latent-property pairs acquisition method to effectively navigate the complexities of molecular latent optimization, a process that seems intuitive yet challenging due to the high-frequency and discontinuous nature of molecule space.
