Search Results for author: Vladislav Golyanik

Found 77 papers, 10 papers with code

Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video

2 code implementations ICCV 2021 Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollhöfer, Christoph Lassner, Christian Theobalt

We show that a single handheld consumer-grade camera is sufficient to synthesize sophisticated renderings of a dynamic scene from novel virtual camera views, e. g. a `bullet-time' video effect.

Novel View Synthesis Video Editing

Scene-Aware 3D Multi-Human Motion Capture from a Single Camera

1 code implementation12 Jan 2023 Diogo Luvizon, Marc Habermann, Vladislav Golyanik, Adam Kortylewski, Christian Theobalt

In this work, we consider the problem of estimating the 3D position of multiple humans in a scene as well as their body shape and articulation from a single RGB video recorded with a static camera.

Position

High-Fidelity Neural Human Motion Transfer from Monocular Video

1 code implementation CVPR 2021 Moritz Kappel, Vladislav Golyanik, Mohamed Elgharib, Jann-Ole Henningson, Hans-Peter Seidel, Susana Castillo, Christian Theobalt, Marcus Magnor

We address these limitations for the first time in the literature and present a new framework which performs high-fidelity and temporally-consistent human motion transfer with natural pose-dependent non-rigid deformations, for several types of loose garments.

Image Generation Vocal Bursts Intensity Prediction

EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream

1 code implementation ICCV 2021 Viktor Rudnev, Vladislav Golyanik, Jiayi Wang, Hans-Peter Seidel, Franziska Mueller, Mohamed Elgharib, Christian Theobalt

Due to the different data modality of event cameras compared to classical cameras, existing methods cannot be directly applied to and re-trained for event streams.

3D Hand Pose Estimation

MoCapDeform: Monocular 3D Human Motion Capture in Deformable Scenes

1 code implementation17 Aug 2022 Zhi Li, Soshi Shimada, Bernt Schiele, Christian Theobalt, Vladislav Golyanik

3D human motion capture from monocular RGB images respecting interactions of a subject with complex and possibly deformable environments is a very challenging, ill-posed and under-explored problem.

3D Human Pose Estimation

3D Pose Estimation of Two Interacting Hands from a Monocular Event Camera

1 code implementation21 Dec 2023 Christen Millerdurai, Diogo Luvizon, Viktor Rudnev, André Jonas, Jiayi Wang, Christian Theobalt, Vladislav Golyanik

3D hand tracking from a monocular video is a very challenging problem due to hand interactions, occlusions, left-right hand ambiguity, and fast motion.

3D Pose Estimation 3D Reconstruction

HDM-Net: Monocular Non-Rigid 3D Reconstruction with Learned Deformation Model

no code implementations27 Mar 2018 Vladislav Golyanik, Soshi Shimada, Kiran varanasi, Didier Stricker

Monocular dense 3D reconstruction of deformable objects is a hard ill-posed problem in computer vision.

3D Reconstruction

Scalable Dense Monocular Surface Reconstruction

no code implementations17 Oct 2017 Mohammad Dawud Ansari, Vladislav Golyanik, Didier Stricker

This paper reports on a novel template-free monocular non-rigid surface reconstruction approach.

Surface Reconstruction

Multiframe Scene Flow with Piecewise Rigid Motion

no code implementations5 Oct 2017 Vladislav Golyanik, Kihwan Kim, Robert Maier, Matthias Nießner, Didier Stricker, Jan Kautz

We introduce a novel multiframe scene flow approach that jointly optimizes the consistency of the patch appearances and their local rigid motions from RGB-D image sequences.

Scene Flow Estimation

Gravitational Approach for Point Set Registration

no code implementations CVPR 2016 Vladislav Golyanik, Sk Aziz Ali, Didier Stricker

In this paper a new astrodynamics inspired rigid point set registration algorithm is introduced -- the Gravitational Approach (GA).

IsMo-GAN: Adversarial Learning for Monocular Non-Rigid 3D Reconstruction

no code implementations27 Apr 2019 Soshi Shimada, Vladislav Golyanik, Christian Theobalt, Didier Stricker

The majority of the existing methods for non-rigid 3D surface regression from monocular 2D images require an object template or point tracks over multiple frames as an input, and are still far from real-time processing rates.

3D Reconstruction Generative Adversarial Network

Structure from Articulated Motion: Accurate and Stable Monocular 3D Reconstruction without Training Data

no code implementations12 May 2019 Onorina Kovalenko, Vladislav Golyanik, Jameel Malik, Ahmed Elhayek, Didier Stricker

SfAM is highly robust to noisy 2D annotations, generalizes to arbitrary objects and does not rely on training data, which is shown in extensive experiments on public benchmarks and real video sequences.

3D Reconstruction

DispVoxNets: Non-Rigid Point Set Alignment with Supervised Learning Proxies

no code implementations24 Jul 2019 Soshi Shimada, Vladislav Golyanik, Edgar Tretschk, Didier Stricker, Christian Theobalt

We introduce a supervised-learning framework for non-rigid point set alignment of a new kind - Displacements on Voxels Networks (DispVoxNets) - which abstracts away from the point set representation and regresses 3D displacement fields on regularly sampled proxy 3D voxel grids.

Intrinsic Dynamic Shape Prior for Fast, Sequential and Dense Non-Rigid Structure from Motion with Detection of Temporally-Disjoint Rigidity

no code implementations5 Sep 2019 Vladislav Golyanik, André Jonas, Didier Stricker, Christian Theobalt

The reasons for the slow dissemination are the severe ill-posedness, high sensitivity to motion and deformation cues and the difficulty to obtain reliable point tracks in the vast majority of practical scenarios.

Accelerated Gravitational Point Set Alignment With Altered Physical Laws

no code implementations ICCV 2019 Vladislav Golyanik, Christian Theobalt, Didier Stricker

This work describes Barnes-Hut Rigid Gravitational Approach (BH-RGA) -- a new rigid point set registration method relying on principles of particle dynamics.

A Quantum Computational Approach to Correspondence Problems on Point Sets

no code implementations CVPR 2020 Vladislav Golyanik, Christian Theobalt

Modern adiabatic quantum computers (AQC) are already used to solve difficult combinatorial optimisation problems in various domains of science.

PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Time

no code implementations20 Aug 2020 Soshi Shimada, Vladislav Golyanik, Weipeng Xu, Christian Theobalt

We, therefore, present PhysCap, the first algorithm for physically plausible, real-time and marker-less human 3D motion capture with a single colour camera at 25 fps.

HTML: A Parametric Hand Texture Model for 3D Hand Reconstruction and Personalization

no code implementations ECCV 2020 Neng Qian, Jiayi Wang, Franziska Mueller, Florian Bernard, Vladislav Golyanik, Christian Theobalt

3D hand reconstruction from images is a widely-studied problem in computer vision and graphics, and has a particularly high relevance for virtual and augmented reality.

Neural Rendering

Neural Dense Non-Rigid Structure from Motion with Latent Space Constraints

no code implementations ECCV 2020 Vikramjit Sidhu, Edgar Tretschk, Vladislav Golyanik, Antonio Agudo, Christian Theobalt

We introduce the first dense neural non-rigid structure from motion (N-NRSfM) approach, which can be trained end-to-end in an unsupervised manner from 2D point tracks.

3D Shape Reconstruction

Fast Gravitational Approach for Rigid Point Set Registration with Ordinary Differential Equations

no code implementations28 Sep 2020 Sk Aziz Ali, Kerem Kahraman, Christian Theobalt, Didier Stricker, Vladislav Golyanik

This article introduces a new physics-based method for rigid point set alignment called Fast Gravitational Approach (FGA).

Pose-Guided Human Animation from a Single Image in the Wild

no code implementations CVPR 2021 Jae Shin Yoon, Lingjie Liu, Vladislav Golyanik, Kripasindhu Sarkar, Hyun Soo Park, Christian Theobalt

We present a new pose transfer method for synthesizing a human animation from a single image of a person controlled by a sequence of body poses.

Pose Transfer

Neural Re-Rendering of Humans from a Single Image

no code implementations ECCV 2020 Kripasindhu Sarkar, Dushyant Mehta, Weipeng Xu, Vladislav Golyanik, Christian Theobalt

Human re-rendering from a single image is a starkly under-constrained problem, and state-of-the-art algorithms often exhibit undesired artefacts, such as over-smoothing, unrealistic distortions of the body parts and garments, or implausible changes of the texture.

Translation

Quantum Permutation Synchronization

no code implementations CVPR 2021 Tolga Birdal, Vladislav Golyanik, Christian Theobalt, Leonidas Guibas

We present QuantumSync, the first quantum algorithm for solving a synchronization problem in the context of computer vision.

Style and Pose Control for Image Synthesis of Humans from a Single Monocular View

no code implementations22 Feb 2021 Kripasindhu Sarkar, Vladislav Golyanik, Lingjie Liu, Christian Theobalt

Photo-realistic re-rendering of a human from a single image with explicit control over body pose, shape and appearance enables a wide range of applications, such as human appearance transfer, virtual try-on, motion imitation, and novel view synthesis.

Image Generation Novel View Synthesis +1

HumanGAN: A Generative Model of Humans Images

no code implementations11 Mar 2021 Kripasindhu Sarkar, Lingjie Liu, Vladislav Golyanik, Christian Theobalt

We address these limitations and present a generative model for images of dressed humans offering control over pose, local body part appearance and garment style.

Pose Transfer

Differentiable Event Stream Simulator for Non-Rigid 3D Tracking

no code implementations30 Apr 2021 Jalees Nehvi, Vladislav Golyanik, Franziska Mueller, Hans-Peter Seidel, Mohamed Elgharib, Christian Theobalt

This paper introduces the first differentiable simulator of event streams, i. e., streams of asynchronous brightness change signals recorded by event cameras.

Neural Monocular 3D Human Motion Capture with Physical Awareness

no code implementations3 May 2021 Soshi Shimada, Vladislav Golyanik, Weipeng Xu, Patrick Pérez, Christian Theobalt

We present a new trainable system for physically plausible markerless 3D human motion capture, which achieves state-of-the-art results in a broad range of challenging scenarios.

3D Pose Estimation

Q-Match: Iterative Shape Matching via Quantum Annealing

no code implementations ICCV 2021 Marcel Seelbach Benkner, Zorah Lähner, Vladislav Golyanik, Christof Wunderlich, Christian Theobalt, Michael Moeller

Finding shape correspondences can be formulated as an NP-hard quadratic assignment problem (QAP) that becomes infeasible for shapes with high sampling density.

Fast Simultaneous Gravitational Alignment of Multiple Point Sets

no code implementations21 Jun 2021 Vladislav Golyanik, Soshi Shimada, Christian Theobalt

The problem of simultaneous rigid alignment of multiple unordered point sets which is unbiased towards any of the inputs has recently attracted increasing interest, and several reliable methods have been newly proposed.

HandVoxNet++: 3D Hand Shape and Pose Estimation using Voxel-Based Neural Networks

no code implementations2 Jul 2021 Jameel Malik, Soshi Shimada, Ahmed Elhayek, Sk Aziz Ali, Christian Theobalt, Vladislav Golyanik, Didier Stricker

To address the limitations of the existing methods, we develop HandVoxNet++, i. e., a voxel-based deep network with 3D and graph convolutions trained in a fully supervised manner.

3D Hand Pose Estimation

Adiabatic Quantum Graph Matching with Permutation Matrix Constraints

no code implementations8 Jul 2021 Marcel Seelbach Benkner, Vladislav Golyanik, Christian Theobalt, Michael Moeller

In this work, we address such problems with emerging quantum computing technology and propose several reformulations of QAPs as unconstrained problems suitable for efficient execution on quantum hardware.

Graph Matching valid

NeRF for Outdoor Scene Relighting

no code implementations9 Dec 2021 Viktor Rudnev, Mohamed Elgharib, William Smith, Lingjie Liu, Vladislav Golyanik, Christian Theobalt

Photorealistic editing of outdoor scenes from photographs requires a profound understanding of the image formation process and an accurate estimation of the scene geometry, reflectance and illumination.

φ-SfT: Shape-from-Template with a Physics-Based Deformation Model

no code implementations22 Mar 2022 Navami Kairanda, Edith Tretschk, Mohamed Elgharib, Christian Theobalt, Vladislav Golyanik

In contrast to previous works, this paper proposes a new SfT approach explaining 2D observations through physical simulations accounting for forces and material properties.

3D Reconstruction Physical Simulations

Quantum Motion Segmentation

no code implementations24 Mar 2022 Federica Arrigoni, Willi Menapace, Marcel Seelbach Benkner, Elisa Ricci, Vladislav Golyanik

Motion segmentation is a challenging problem that seeks to identify independent motions in two or several input images.

Motion Segmentation Segmentation

Q-FW: A Hybrid Classical-Quantum Frank-Wolfe for Quadratic Binary Optimization

no code implementations23 Mar 2022 Alp Yurtsever, Tolga Birdal, Vladislav Golyanik

We present a hybrid classical-quantum framework based on the Frank-Wolfe algorithm, Q-FW, for solving quadratic, linearly-constrained, binary optimization problems on quantum annealers (QA).

Graph Matching

HULC: 3D Human Motion Capture with Pose Manifold Sampling and Dense Contact Guidance

no code implementations11 May 2022 Soshi Shimada, Vladislav Golyanik, Zhi Li, Patrick Pérez, Weipeng Xu, Christian Theobalt

Marker-less monocular 3D human motion capture (MoCap) with scene interactions is a challenging research topic relevant for extended reality, robotics and virtual avatar generation.

f-SfT: Shape-From-Template With a Physics-Based Deformation Model

no code implementations CVPR 2022 Navami Kairanda, Edith Tretschk, Mohamed Elgharib, Christian Theobalt, Vladislav Golyanik

In contrast to previous works, this paper proposes a new SfT approach explaining 2D observations through physical simulations accounting for forces and material properties.

3D Reconstruction Physical Simulations

EventNeRF: Neural Radiance Fields from a Single Colour Event Camera

no code implementations CVPR 2023 Viktor Rudnev, Mohamed Elgharib, Christian Theobalt, Vladislav Golyanik

Asynchronously operating event cameras find many applications due to their high dynamic range, vanishingly low motion blur, low latency and low data bandwidth.

3D Reconstruction Novel View Synthesis +1

HiFECap: Monocular High-Fidelity and Expressive Capture of Human Performances

no code implementations11 Oct 2022 Yue Jiang, Marc Habermann, Vladislav Golyanik, Christian Theobalt

Furthermore, we show that HiFECap outperforms the state-of-the-art human performance capture approaches qualitatively and quantitatively while for the first time capturing all aspects of the human.

Vocal Bursts Intensity Prediction

State of the Art in Dense Monocular Non-Rigid 3D Reconstruction

no code implementations27 Oct 2022 Edith Tretschk, Navami Kairanda, Mallikarjun B R, Rishabh Dabral, Adam Kortylewski, Bernhard Egger, Marc Habermann, Pascal Fua, Christian Theobalt, Vladislav Golyanik

3D reconstruction of deformable (or non-rigid) scenes from a set of monocular 2D image observations is a long-standing and actively researched area of computer vision and graphics.

3D Reconstruction

MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesis

no code implementations CVPR 2023 Rishabh Dabral, Muhammad Hamza Mughal, Vladislav Golyanik, Christian Theobalt

Conventional methods for human motion synthesis are either deterministic or struggle with the trade-off between motion diversity and motion quality.

Denoising Motion Synthesis

Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Models

no code implementations23 Mar 2023 Willi Menapace, Aliaksandr Siarohin, Stéphane Lathuilière, Panos Achlioptas, Vladislav Golyanik, Sergey Tulyakov, Elisa Ricci

Most captivatingly, our PGM unlocks the director's mode, where the game is played by specifying goals for the agents in the form of a prompt.

Navigate

EgoLocate: Real-time Motion Capture, Localization, and Mapping with Sparse Body-mounted Sensors

no code implementations2 May 2023 Xinyu Yi, Yuxiao Zhou, Marc Habermann, Vladislav Golyanik, Shaohua Pan, Christian Theobalt, Feng Xu

We integrate the two techniques together in EgoLocate, a system that simultaneously performs human motion capture (mocap), localization, and mapping in real time from sparse body-mounted sensors, including 6 inertial measurement units (IMUs) and a monocular phone camera.

Simultaneous Localization and Mapping

AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars

no code implementations1 Jun 2023 Mohit Mendiratta, Xingang Pan, Mohamed Elgharib, Kartik Teotia, Mallikarjun B R, Ayush Tewari, Vladislav Golyanik, Adam Kortylewski, Christian Theobalt

Our method edits the full head in a canonical space, and then propagates these edits to remaining time steps via a pretrained deformation network.

VINECS: Video-based Neural Character Skinning

no code implementations3 Jul 2023 Zhouyingcheng Liao, Vladislav Golyanik, Marc Habermann, Christian Theobalt

However, the former methods typically predict solely static skinning weights, which perform poorly for highly articulated poses, and the latter ones either require dense 3D character scans in different poses or cannot generate an explicit mesh with vertex correspondence over time.

ROAM: Robust and Object-Aware Motion Generation Using Neural Pose Descriptors

no code implementations24 Aug 2023 Wanyue Zhang, Rishabh Dabral, Thomas Leimkühler, Vladislav Golyanik, Marc Habermann, Christian Theobalt

Given an unseen object and a reference pose-object pair, we optimise for the object-aware pose that is closest in the feature space to the reference pose.

Motion Synthesis Object

NeuralClothSim: Neural Deformation Fields Meet the Kirchhoff-Love Thin Shell Theory

no code implementations24 Aug 2023 Navami Kairanda, Marc Habermann, Christian Theobalt, Vladislav Golyanik

Cloth simulation is an extensively studied problem, with a plethora of solutions available in computer graphics literature.

Decaf: Monocular Deformation Capture for Face and Hand Interactions

no code implementations28 Sep 2023 Soshi Shimada, Vladislav Golyanik, Patrick Pérez, Christian Theobalt

At the core of our neural approach are a variational auto-encoder supplying the hand-face depth prior and modules that guide the 3D tracking by estimating the contacts and the deformations.

State of the Art on Diffusion Models for Visual Computing

no code implementations11 Oct 2023 Ryan Po, Wang Yifan, Vladislav Golyanik, Kfir Aberman, Jonathan T. Barron, Amit H. Bermano, Eric Ryan Chan, Tali Dekel, Aleksander Holynski, Angjoo Kanazawa, C. Karen Liu, Lingjie Liu, Ben Mildenhall, Matthias Nießner, Björn Ommer, Christian Theobalt, Peter Wonka, Gordon Wetzstein

The field of visual computing is rapidly advancing due to the emergence of generative artificial intelligence (AI), which unlocks unprecedented capabilities for the generation, editing, and reconstruction of images, videos, and 3D scenes.

Projected Stochastic Gradient Descent with Quantum Annealed Binary Gradients

no code implementations23 Oct 2023 Maximilian Krahn, Michelle Sasdelli, Fengyi Yang, Vladislav Golyanik, Juho Kannala, Tat-Jun Chin, Tolga Birdal

We present, QP-SBGD, a novel layer-wise stochastic optimiser tailored towards training neural networks with binary weights, known as binary neural networks (BNNs), on quantum hardware.

ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactions

no code implementations28 Nov 2023 Anindita Ghosh, Rishabh Dabral, Vladislav Golyanik, Christian Theobalt, Philipp Slusallek

Current approaches for 3D human motion synthesis generate high-quality animations of digital humans performing a wide variety of actions and gestures.

Denoising Motion Synthesis

Holoported Characters: Real-time Free-viewpoint Rendering of Humans from Sparse RGB Cameras

no code implementations12 Dec 2023 Ashwath Shetty, Marc Habermann, Guoxing Sun, Diogo Luvizon, Vladislav Golyanik, Christian Theobalt

At inference, our method only requires four camera views of the moving actor and the respective 3D skeletal pose.

Relightable Neural Actor with Intrinsic Decomposition and Pose Control

no code implementations18 Dec 2023 Diogo Luvizon, Vladislav Golyanik, Adam Kortylewski, Marc Habermann, Christian Theobalt

Creating a digital human avatar that is relightable, drivable, and photorealistic is a challenging and important problem in Vision and Graphics.

MACS: Mass Conditioned 3D Hand and Object Motion Synthesis

no code implementations22 Dec 2023 Soshi Shimada, Franziska Mueller, Jan Bednarik, Bardia Doosti, Bernd Bickel, Danhang Tang, Vladislav Golyanik, Jonathan Taylor, Christian Theobalt, Thabo Beeler

To improve the naturalness of the synthesized 3D hand object motions, this work proposes MACS the first MAss Conditioned 3D hand and object motion Synthesis approach.

Motion Synthesis Object

Quantum-Hybrid Stereo Matching With Nonlinear Regularization and Spatial Pyramids

no code implementations26 Dec 2023 Cameron Braunstein, Eddy Ilg, Vladislav Golyanik

Our approach is hybrid (i. e., quantum-classical) and is compatible with modern D-Wave quantum annealers, i. e., it includes a quadratic unconstrained binary optimization (QUBO) objective.

Combinatorial Optimization Stereo Matching

Recent Trends in 3D Reconstruction of General Non-Rigid Scenes

no code implementations22 Mar 2024 Raza Yunus, Jan Eric Lenssen, Michael Niemeyer, Yiyi Liao, Christian Rupprecht, Christian Theobalt, Gerard Pons-Moll, Jia-Bin Huang, Vladislav Golyanik, Eddy Ilg

Reconstructing models of the real world, including 3D geometry, appearance, and motion of real scenes, is essential for computer graphics and computer vision.

3D Reconstruction Navigate

Cannot find the paper you are looking for? You can Submit a new open access paper.