Search Results for author:
Found 82 papers, 15 papers with code
HTML: A Parametric Hand Texture Model for 3D Hand Reconstruction and Personalization
3D hand reconstruction from images is a widely-studied problem in computer vision and graphics, and has a particularly high relevance for virtual and augmented reality.
Neural Dense Non-Rigid Structure from Motion with Latent Space Constraints
We introduce the first dense neural non-rigid structure from motion (N-NRSfM) approach, which can be trained end-to-end in an unsupervised manner from 2D point tracks.
Dynamic EventNeRF: Reconstructing General Dynamic Scenes from Multi-view Event Cameras
It is especially challenging in poor lighting and with fast motion.
BimArt: A Unified Approach for the Synthesis of 3D Bimanual Interaction with Articulated Objects
We present BimArt, a novel generative approach for synthesizing 3D bimanual hand interactions with articulated objects.
DICE: End-to-end Deformation Capture of Hand-Face Interactions from a Single Image
The first and only method for hand-face interaction recovery, Decaf, introduces a global fitting optimization guided by contact and deformation estimation networks trained on studio-collected data with 3D annotations.
D-NPC: Dynamic Neural Point Clouds for Non-Rigid View Synthesis from Monocular Video
By sampling a discrete point cloud from our model, we can efficiently render high-quality novel views using a fast differentiable rasterizer and neural rendering network.
EventEgo3D: 3D Human Motion Capture from Egocentric Event Streams
In response to the existing limitations, this paper 1) introduces a new problem, i. e., 3D human motion capture from an egocentric monocular event camera with a fisheye lens, and 2) proposes the first approach to it called EventEgo3D (EE3D).
Recent Trends in 3D Reconstruction of General Non-Rigid Scenes
Reconstructing models of the real world, including 3D geometry, appearance, and motion of real scenes, is essential for computer graphics and computer vision.
3D Human Pose Perception from Egocentric Stereo Videos
Hence, existing methods often fail to accurately estimate complex 3D poses from egocentric views.
Ranked #3 on
Egocentric Pose Estimation
on UnrealEgo
Quantum-Hybrid Stereo Matching With Nonlinear Regularization and Spatial Pyramids
Our approach is hybrid (i. e., quantum-classical) and is compatible with modern D-Wave quantum annealers, i. e., it includes a quadratic unconstrained binary optimization (QUBO) objective.
MACS: Mass Conditioned 3D Hand and Object Motion Synthesis
To improve the naturalness of the synthesized 3D hand object motions, this work proposes MACS the first MAss Conditioned 3D hand and object motion Synthesis approach.
3D Pose Estimation of Two Interacting Hands from a Monocular Event Camera
3D hand tracking from a monocular video is a very challenging problem due to hand interactions, occlusions, left-right hand ambiguity, and fast motion.
Relightable Neural Actor with Intrinsic Decomposition and Pose Control
Creating a controllable and relightable digital avatar from multi-view video with fixed illumination is a very challenging problem since humans are highly articulated, creating pose-dependent appearance effects, and skin as well as clothing require space-varying BRDF modeling.
Holoported Characters: Real-time Free-viewpoint Rendering of Humans from Sparse RGB Cameras
At inference, our method only requires four camera views of the moving actor and the respective 3D skeletal pose.
ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactions
Current approaches for 3D human motion synthesis generate high quality animations of digital humans performing a wide variety of actions and gestures.
3D-QAE: Fully Quantum Auto-Encoding of 3D Point Clouds
It is trained on collections of 3D point clouds to produce their compressed representations.
Projected Stochastic Gradient Descent with Quantum Annealed Binary Gradients
We present, QP-SBGD, a novel layer-wise stochastic optimiser tailored towards training neural networks with binary weights, known as binary neural networks (BNNs), on quantum hardware.
State of the Art on Diffusion Models for Visual Computing
The field of visual computing is rapidly advancing due to the emergence of generative artificial intelligence (AI), which unlocks unprecedented capabilities for the generation, editing, and reconstruction of images, videos, and 3D scenes.
Decaf: Monocular Deformation Capture for Face and Hand Interactions
At the core of our neural approach are a variational auto-encoder supplying the hand-face depth prior and modules that guide the 3D tracking by estimating the contacts and the deformations.
NeuralClothSim: Neural Deformation Fields Meet the Thin Shell Theory
Despite existing 3D cloth simulators producing realistic results, they predominantly operate on discrete surface representations (e. g. points and meshes) with a fixed spatial resolution, which often leads to large memory consumption and resolution-dependent simulations.
ROAM: Robust and Object-Aware Motion Generation Using Neural Pose Descriptors
Given an unseen object and a reference pose-object pair, we optimise for the object-aware pose that is closest in the feature space to the reference pose.
SceNeRFlow: Time-Consistent Reconstruction of General Dynamic Scenes
We propose SceNeRFlow to reconstruct a general, non-rigid scene in a time-consistent manner.
VINECS: Video-based Neural Character Skinning
However, the former methods typically predict solely static skinning weights, which perform poorly for highly articulated poses, and the latter ones either require dense 3D character scans in different poses or cannot generate an explicit mesh with vertex correspondence over time.
AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars
Our method edits the full head in a canonical space, and then propagates these edits to remaining time steps via a pretrained deformation network.
Self-supervised Pre-training with Masked Shape Prediction for 3D Scene Understanding
Masked signal modeling has greatly advanced self-supervised pre-training for language and 2D images.
EgoLocate: Real-time Motion Capture, Localization, and Mapping with Sparse Body-mounted Sensors
We integrate the two techniques together in EgoLocate, a system that simultaneously performs human motion capture (mocap), localization, and mapping in real time from sparse body-mounted sensors, including 6 inertial measurement units (IMUs) and a monocular phone camera.
CCuantuMM: Cycle-Consistent Quantum-Hybrid Matching of Multiple Shapes
Jointly matching multiple, non-rigidly deformed 3D shapes is a challenging, $\mathcal{NP}$-hard problem.
Quantum Multi-Model Fitting
Geometric model fitting is a challenging but fundamental computer vision problem.
Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Models
Most captivatingly, our PGM unlocks the director's mode, where the game is played by specifying goals for the agents in the form of a prompt.
Scene-Aware 3D Multi-Human Motion Capture from a Single Camera
In this work, we consider the problem of estimating the 3D position of multiple humans in a scene as well as their body shape and articulation from a single RGB video recorded with a static camera.
IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions
Is it possible to synthesize such motion plausibly with a diverse set of objects and instructions?
MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesis
Conventional methods for human motion synthesis are either deterministic or struggle with the trade-off between motion diversity and motion quality.
Ranked #7 on
Motion Synthesis
on AIST++
Fast Non-Rigid Radiance Fields from Monocularized Data
The reconstruction and novel view synthesis of dynamic scenes recently gained increased attention.
State of the Art in Dense Monocular Non-Rigid 3D Reconstruction
3D reconstruction of deformable (or non-rigid) scenes from a set of monocular 2D image observations is a long-standing and actively researched area of computer vision and graphics.
QuAnt: Quantum Annealing with Learnt Couplings
As a result, the solution encodings can be chosen flexibly and compactly.
HiFECap: Monocular High-Fidelity and Expressive Capture of Human Performances
Furthermore, we show that HiFECap outperforms the state-of-the-art human performance capture approaches qualitatively and quantitatively while for the first time capturing all aspects of the human.
MoCapDeform: Monocular 3D Human Motion Capture in Deformable Scenes
3D human motion capture from monocular RGB images respecting interactions of a subject with complex and possibly deformable environments is a very challenging, ill-posed and under-explored problem.
UnrealEgo: A New Dataset for Robust Egocentric 3D Human Motion Capture
We present UnrealEgo, i. e., a new large-scale naturalistic dataset for egocentric 3D human pose estimation.
Ranked #5 on
Egocentric Pose Estimation
on UnrealEgo
EventNeRF: Neural Radiance Fields from a Single Colour Event Camera
Asynchronously operating event cameras find many applications due to their high dynamic range, vanishingly low motion blur, low latency and low data bandwidth.
Unbiased 4D: Monocular 4D Reconstruction with a Neural Deformation Model
Capturing general deforming scenes from monocular RGB video is crucial for many computer graphics and vision applications.
HULC: 3D Human Motion Capture with Pose Manifold Sampling and Dense Contact Guidance
Marker-less monocular 3D human motion capture (MoCap) with scene interactions is a challenging research topic relevant for extended reality, robotics and virtual avatar generation.
Quantum Motion Segmentation
Motion segmentation is a challenging problem that seeks to identify independent motions in two or several input images.
Q-FW: A Hybrid Classical-Quantum Frank-Wolfe for Quadratic Binary Optimization
We present a hybrid classical-quantum framework based on the Frank-Wolfe algorithm, Q-FW, for solving quadratic, linearly-constrained, binary optimization problems on quantum annealers (QA).
φ-SfT: Shape-from-Template with a Physics-Based Deformation Model
In contrast to previous works, this paper proposes a new SfT approach explaining 2D observations through physical simulations accounting for forces and material properties.
Playable Environments: Video Manipulation in Space and Time
We present Playable Environments - a new representation for interactive video generation and manipulation in space and time.
f-SfT: Shape-From-Template With a Physics-Based Deformation Model
In contrast to previous works, this paper proposes a new SfT approach explaining 2D observations through physical simulations accounting for forces and material properties.
NeRF for Outdoor Scene Relighting
Photorealistic editing of outdoor scenes from photographs requires a profound understanding of the image formation process and an accurate estimation of the scene geometry, reflectance and illumination.
Advances in Neural Rendering
The reconstruction of such a scene representation from observations using differentiable rendering losses is known as inverse graphics or inverse rendering.
Convex Joint Graph Matching and Clustering via Semidefinite Relaxations
This paper proposes a new algorithm for simultaneous graph matching and clustering.
Gravity-Aware Monocular 3D Human-Object Reconstruction
We evaluate GraviCap on a new dataset with ground-truth annotations for persons and different objects undergoing free flights.
Adiabatic Quantum Graph Matching with Permutation Matrix Constraints
In this work, we address such problems with emerging quantum computing technology and propose several reformulations of QAPs as unconstrained problems suitable for efficient execution on quantum hardware.
Egocentric Videoconferencing
Even holding a mobile phone camera in the front of the face while sitting for a long duration is not convenient.
HandVoxNet++: 3D Hand Shape and Pose Estimation using Voxel-Based Neural Networks
To address the limitations of the existing methods, we develop HandVoxNet++, i. e., a voxel-based deep network with 3D and graph convolutions trained in a fully supervised manner.
Fast Simultaneous Gravitational Alignment of Multiple Point Sets
The problem of simultaneous rigid alignment of multiple unordered point sets which is unbiased towards any of the inputs has recently attracted increasing interest, and several reliable methods have been newly proposed.
Q-Match: Iterative Shape Matching via Quantum Annealing
Finding shape correspondences can be formulated as an NP-hard quadratic assignment problem (QAP) that becomes infeasible for shapes with high sampling density.
Neural Monocular 3D Human Motion Capture with Physical Awareness
We present a new trainable system for physically plausible markerless 3D human motion capture, which achieves state-of-the-art results in a broad range of challenging scenarios.
Differentiable Event Stream Simulator for Non-Rigid 3D Tracking
This paper introduces the first differentiable simulator of event streams, i. e., streams of asynchronous brightness change signals recorded by event cameras.
HumanGAN: A Generative Model of Humans Images
We address these limitations and present a generative model for images of dressed humans offering control over pose, local body part appearance and garment style.
Style and Pose Control for Image Synthesis of Humans from a Single Monocular View
Photo-realistic re-rendering of a human from a single image with explicit control over body pose, shape and appearance enables a wide range of applications, such as human appearance transfer, virtual try-on, motion imitation, and novel view synthesis.
Quantum Permutation Synchronization
We present QuantumSync, the first quantum algorithm for solving a synchronization problem in the context of computer vision.
Neural Re-Rendering of Humans from a Single Image
Human re-rendering from a single image is a starkly under-constrained problem, and state-of-the-art algorithms often exhibit undesired artefacts, such as over-smoothing, unrealistic distortions of the body parts and garments, or implausible changes of the texture.
Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video
We show that a single handheld consumer-grade camera is sufficient to synthesize sophisticated renderings of a dynamic scene from novel virtual camera views, e. g. a `bullet-time' video effect.
High-Fidelity Neural Human Motion Transfer from Monocular Video
We address these limitations for the first time in the literature and present a new framework which performs high-fidelity and temporally-consistent human motion transfer with natural pose-dependent non-rigid deformations, for several types of loose garments.
EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream
Due to the different data modality of event cameras compared to classical cameras, existing methods cannot be directly applied to and re-trained for event streams.
Pose-Guided Human Animation from a Single Image in the Wild
We present a new pose transfer method for synthesizing a human animation from a single image of a person controlled by a sequence of body poses.
Fast Gravitational Approach for Rigid Point Set Registration with Ordinary Differential Equations
This article introduces a new physics-based method for rigid point set alignment called Fast Gravitational Approach (FGA).
PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Time
We, therefore, present PhysCap, the first algorithm for physically plausible, real-time and marker-less human 3D motion capture with a single colour camera at 25 fps.
PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations
At the level of patches, objects across different categories share similarities, which leads to more generalizable models.
HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map
The input to our method is a 3D voxelized depth map, and we rely on two hand shape representations.
A Quantum Computational Approach to Correspondence Problems on Point Sets
Modern adiabatic quantum computers (AQC) are already used to solve difficult combinatorial optimisation problems in various domains of science.
Accelerated Gravitational Point Set Alignment With Altered Physical Laws
This work describes Barnes-Hut Rigid Gravitational Approach (BH-RGA) -- a new rigid point set registration method relying on principles of particle dynamics.
Intrinsic Dynamic Shape Prior for Fast, Sequential and Dense Non-Rigid Structure from Motion with Detection of Temporally-Disjoint Rigidity
The reasons for the slow dissemination are the severe ill-posedness, high sensitivity to motion and deformation cues and the difficulty to obtain reliable point tracks in the vast majority of practical scenarios.
EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera
The high frame rate is a critical requirement for capturing fast human motions.
DispVoxNets: Non-Rigid Point Set Alignment with Supervised Learning Proxies
We introduce a supervised-learning framework for non-rigid point set alignment of a new kind - Displacements on Voxels Networks (DispVoxNets) - which abstracts away from the point set representation and regresses 3D displacement fields on regularly sampled proxy 3D voxel grids.
DEMEA: Deep Mesh Autoencoders for Non-Rigidly Deforming Objects
Mesh autoencoders are commonly used for dimensionality reduction, sampling and mesh modeling.
Structure from Articulated Motion: Accurate and Stable Monocular 3D Reconstruction without Training Data
SfAM is highly robust to noisy 2D annotations, generalizes to arbitrary objects and does not rely on training data, which is shown in extensive experiments on public benchmarks and real video sequences.
IsMo-GAN: Adversarial Learning for Monocular Non-Rigid 3D Reconstruction
The majority of the existing methods for non-rigid 3D surface regression from monocular 2D images require an object template or point tracks over multiple frames as an input, and are still far from real-time processing rates.
HDM-Net: Monocular Non-Rigid 3D Reconstruction with Learned Deformation Model
Monocular dense 3D reconstruction of deformable objects is a hard ill-posed problem in computer vision.
Accurate 3D Reconstruction of Dynamic Scenes from Monocular Image Sequences with Severe Occlusions
We integrate a shape prior term into variational optimisation framework.
Scalable Dense Monocular Surface Reconstruction
This paper reports on a novel template-free monocular non-rigid surface reconstruction approach.
Multiframe Scene Flow with Piecewise Rigid Motion
We introduce a novel multiframe scene flow approach that jointly optimizes the consistency of the patch appearances and their local rigid motions from RGB-D image sequences.
Gravitational Approach for Point Set Registration
In this paper a new astrodynamics inspired rigid point set registration algorithm is introduced -- the Gravitational Approach (GA).
