Search Results for author: Mohamed Elgharib

Found 37 papers, 9 papers with code

Retrieval in Long Surveillance Videos using User Described Motion and Object Attributes

no code implementations1 May 2014 Greg Castanon, Mohamed Elgharib, Venkatesh Saligrama, Pierre-Marc Jodoin

We present a content-based retrieval method for long surveillance videos both for wide-area (Airborne) as well as near-field imagery (CCTV).

Retrieval

Learning-based Video Motion Magnification

2 code implementations ECCV 2018 Tae-Hyun Oh, Ronnachai Jaroensri, Changil Kim, Mohamed Elgharib, Frédo Durand, William T. Freeman, Wojciech Matusik

We show that the learned filters achieve high-quality results on real videos, with less ringing artifacts and better noise characteristics than previous methods.

Motion Magnification

On Learning Associations of Faces and Voices

1 code implementation15 May 2018 Changil Kim, Hijung Valentina Shin, Tae-Hyun Oh, Alexandre Kaspar, Mohamed Elgharib, Wojciech Matusik

We computationally model the overlapping information between faces and voices and show that the learned cross-modal representation contains enough information to identify matching faces and voices with performance similar to that of humans.

Speaker Identification

A Dataset of Flash and Ambient Illumination Pairs from the Crowd

no code implementations ECCV 2018 Yagiz Aksoy, Changil Kim, Petr Kellnhofer, Sylvain Paris, Mohamed Elgharib, Marc Pollefeys, Wojciech Matusik

We present a dataset of thousands of ambient and flash illumination pairs to enable studying flash photography and other applications that can benefit from having separate illuminations.

FML: Face Model Learning from Videos

no code implementations CVPR 2019 Ayush Tewari, Florian Bernard, Pablo Garrido, Gaurav Bharaj, Mohamed Elgharib, Hans-Peter Seidel, Patrick Pérez, Michael Zollhöfer, Christian Theobalt

In contrast, we propose multi-frame video-based self-supervised training of a deep network that (i) learns a face identity model both in shape and appearance while (ii) jointly learning to reconstruct 3D faces.

3D Reconstruction Face Model

EgoFace: Egocentric Face Performance Capture and Videorealistic Reenactment

no code implementations26 May 2019 Mohamed Elgharib, Mallikarjun BR, Ayush Tewari, Hyeongwoo Kim, Wentao Liu, Hans-Peter Seidel, Christian Theobalt

Our lightweight setup allows operations in uncontrolled environments, and lends itself to telepresence applications such as video-conferencing from dynamic environments.

XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera

4 code implementations1 Jul 2019 Dushyant Mehta, Oleksandr Sotnychenko, Franziska Mueller, Weipeng Xu, Mohamed Elgharib, Pascal Fua, Hans-Peter Seidel, Helge Rhodin, Gerard Pons-Moll, Christian Theobalt

The first stage is a convolutional neural network (CNN) that estimates 2D and 3D pose features along with identity assignments for all visible joints of all individuals. We contribute a new architecture for this CNN, called SelecSLS Net, that uses novel selective long and short range skip connections to improve the information flow allowing for a drastically faster network without compromising accuracy.

3D Multi-Person Human Pose Estimation 3D Multi-Person Pose Estimation +1

Neural Style-Preserving Visual Dubbing

no code implementations5 Sep 2019 Hyeongwoo Kim, Mohamed Elgharib, Michael Zollhöfer, Hans-Peter Seidel, Thabo Beeler, Christian Richardt, Christian Theobalt

We present a style-preserving visual dubbing approach from single video inputs, which maintains the signature style of target actors when modifying facial expressions, including mouth motions, to match foreign languages.

Generative Adversarial Network

Neural Voice Puppetry: Audio-driven Facial Reenactment

1 code implementation ECCV 2020 Justus Thies, Mohamed Elgharib, Ayush Tewari, Christian Theobalt, Matthias Nießner

Neural Voice Puppetry has a variety of use-cases, including audio-driven video avatars, video dubbing, and text-driven video synthesis of a talking head.

Face Model Neural Rendering +2

StyleRig: Rigging StyleGAN for 3D Control over Portrait Images

no code implementations CVPR 2020 Ayush Tewari, Mohamed Elgharib, Gaurav Bharaj, Florian Bernard, Hans-Peter Seidel, Patrick Pérez, Michael Zollhöfer, Christian Theobalt

StyleGAN generates photorealistic portrait images of faces with eyes, teeth, hair and context (neck, shoulders, background), but lacks a rig-like control over semantic face parameters that are interpretable in 3D, such as face pose, expressions, and scene illumination.

Monocular Reconstruction of Neural Face Reflectance Fields

no code implementations CVPR 2021 Mallikarjun B R., Ayush Tewari, Tae-Hyun Oh, Tim Weyrich, Bernd Bickel, Hans-Peter Seidel, Hanspeter Pfister, Wojciech Matusik, Mohamed Elgharib, Christian Theobalt

The reflectance field of a face describes the reflectance properties responsible for complex lighting effects including diffuse, specular, inter-reflection and self shadowing.

Monocular Reconstruction

User-assisted Video Reflection Removal

no code implementations7 Sep 2020 Amgad Ahmed, Suhong Kim, Mohamed Elgharib, Mohamed Hefeeda

We show that user-assistance significantly improves the layer separation results.

Reflection Removal valid

PIE: Portrait Image Embedding for Semantic Control

no code implementations20 Sep 2020 Ayush Tewari, Mohamed Elgharib, Mallikarjun B R., Florian Bernard, Hans-Peter Seidel, Patrick Pérez, Michael Zollhöfer, Christian Theobalt

We present the first approach for embedding real portrait images in the latent space of StyleGAN, which allows for intuitive editing of the head pose, facial expression, and scene illumination in the image.

Face Model

Learning Complete 3D Morphable Face Models from Images and Videos

no code implementations CVPR 2021 Mallikarjun B R, Ayush Tewari, Hans-Peter Seidel, Mohamed Elgharib, Christian Theobalt

Our network design and loss functions ensure a disentangled parameterization of not only identity and albedo, but also, for the first time, an expression basis.

3D Face Reconstruction Monocular Reconstruction +1

i3DMM: Deep Implicit 3D Morphable Model of Human Heads

1 code implementation CVPR 2021 Tarun Yenamandra, Ayush Tewari, Florian Bernard, Hans-Peter Seidel, Mohamed Elgharib, Daniel Cremers, Christian Theobalt

Our approach has the following favorable properties: (i) It is the first full head morphable model that includes hair.

EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream

1 code implementation ICCV 2021 Viktor Rudnev, Vladislav Golyanik, Jiayi Wang, Hans-Peter Seidel, Franziska Mueller, Mohamed Elgharib, Christian Theobalt

Due to the different data modality of event cameras compared to classical cameras, existing methods cannot be directly applied to and re-trained for event streams.

3D Hand Pose Estimation

High-Fidelity Neural Human Motion Transfer from Monocular Video

1 code implementation CVPR 2021 Moritz Kappel, Vladislav Golyanik, Mohamed Elgharib, Jann-Ole Henningson, Hans-Peter Seidel, Susana Castillo, Christian Theobalt, Marcus Magnor

We address these limitations for the first time in the literature and present a new framework which performs high-fidelity and temporally-consistent human motion transfer with natural pose-dependent non-rigid deformations, for several types of loose garments.

Image Generation Vocal Bursts Intensity Prediction

Learning Speech-driven 3D Conversational Gestures from Video

no code implementations13 Feb 2021 Ikhsanul Habibie, Weipeng Xu, Dushyant Mehta, Lingjie Liu, Hans-Peter Seidel, Gerard Pons-Moll, Mohamed Elgharib, Christian Theobalt

We propose the first approach to automatically and jointly synthesize both the synchronous 3D conversational body and hand gestures, as well as 3D face and head animations, of a virtual character from speech input.

3D Face Animation Generative Adversarial Network +2

Differentiable Event Stream Simulator for Non-Rigid 3D Tracking

no code implementations30 Apr 2021 Jalees Nehvi, Vladislav Golyanik, Franziska Mueller, Hans-Peter Seidel, Mohamed Elgharib, Christian Theobalt

This paper introduces the first differentiable simulator of event streams, i. e., streams of asynchronous brightness change signals recorded by event cameras.

Self-supervised Outdoor Scene Relighting

no code implementations ECCV 2020 Ye Yu, Abhimitra Meka, Mohamed Elgharib, Hans-Peter Seidel, Christian Theobalt, William A. P. Smith

Outdoor scene relighting is a challenging problem that requires good understanding of the scene geometry, illumination and albedo.

StyleVideoGAN: A Temporal Generative Model using a Pretrained StyleGAN

no code implementations15 Jul 2021 Gereon Fox, Ayush Tewari, Mohamed Elgharib, Christian Theobalt

We demonstrate that it suffices to train our temporal architecture on only 10 minutes of footage of 1 subject for about 6 hours.

NeRF for Outdoor Scene Relighting

no code implementations9 Dec 2021 Viktor Rudnev, Mohamed Elgharib, William Smith, Lingjie Liu, Vladislav Golyanik, Christian Theobalt

Photorealistic editing of outdoor scenes from photographs requires a profound understanding of the image formation process and an accurate estimation of the scene geometry, reflectance and illumination.

f-SfT: Shape-From-Template With a Physics-Based Deformation Model

no code implementations CVPR 2022 Navami Kairanda, Edith Tretschk, Mohamed Elgharib, Christian Theobalt, Vladislav Golyanik

In contrast to previous works, this paper proposes a new SfT approach explaining 2D observations through physical simulations accounting for forces and material properties.

3D Reconstruction Physical Simulations

φ-SfT: Shape-from-Template with a Physics-Based Deformation Model

no code implementations22 Mar 2022 Navami Kairanda, Edith Tretschk, Mohamed Elgharib, Christian Theobalt, Vladislav Golyanik

In contrast to previous works, this paper proposes a new SfT approach explaining 2D observations through physical simulations accounting for forces and material properties.

3D Reconstruction Physical Simulations

EventNeRF: Neural Radiance Fields from a Single Colour Event Camera

no code implementations CVPR 2023 Viktor Rudnev, Mohamed Elgharib, Christian Theobalt, Vladislav Golyanik

Asynchronously operating event cameras find many applications due to their high dynamic range, vanishingly low motion blur, low latency and low data bandwidth.

3D Reconstruction Novel View Synthesis +1

gCoRF: Generative Compositional Radiance Fields

no code implementations31 Oct 2022 Mallikarjun BR, Ayush Tewari, Xingang Pan, Mohamed Elgharib, Christian Theobalt

We start with a global generative model (GAN) and learn to decompose it into different semantic parts using supervision from 2D segmentation masks.

Image Generation

An Implicit Parametric Morphable Dental Model

no code implementations21 Nov 2022 Congyi Zhang, Mohamed Elgharib, Gereon Fox, Min Gu, Christian Theobalt, Wenping Wang

Current dental models use an explicit mesh scene representation and model only the teeth, ignoring the gum.

GVP: Generative Volumetric Primitives

no code implementations31 Mar 2023 Mallikarjun B R, Xingang Pan, Mohamed Elgharib, Christian Theobalt

Advances in 3D-aware generative models have pushed the boundary of image synthesis with explicit camera control.

Image Generation Knowledge Distillation

AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars

no code implementations1 Jun 2023 Mohit Mendiratta, Xingang Pan, Mohamed Elgharib, Kartik Teotia, Mallikarjun B R, Ayush Tewari, Vladislav Golyanik, Adam Kortylewski, Christian Theobalt

Our method edits the full head in a canonical space, and then propagates these edits to remaining time steps via a pretrained deformation network.

Cannot find the paper you are looking for? You can Submit a new open access paper.