1 code implementation • ECCV 2020 • Jan Brejcha, Michal Lukáč, Yannick Hold-Geoffroy, Oliver Wang, Martin Čadík
We introduce a solution to large scale Augmented Reality for outdoor scenes by registering camera images to textured Digital Elevation Models (DEMs).
Ranked #2 on
Patch Matching
on HPatches
(using extra training data)
no code implementations • 22 Jan 2025 • Edurne Bernal-Berdun, Ana Serrano, Belen Masia, Matheus Gadelha, Yannick Hold-Geoffroy, Xin Sun, Diego Gutierrez
Images as an artistic medium often rely on specific camera angles and lens distortions to convey ideas or emotions; however, such precise control is missing in current text-to-image models.
no code implementations • 16 Jan 2025 • Sumit Chaturvedi, Mengwei Ren, Yannick Hold-Geoffroy, Jingyuan Liu, Julie Dorsey, Zhixin Shu
Our method generalizes to diverse real photographs and produces realistic illumination effects, including specular highlights and cast shadows, while preserving the subject's identity.
no code implementations • 18 Dec 2024 • Junuk Cha, Mengwei Ren, Krishna Kumar Singh, He Zhang, Yannick Hold-Geoffroy, Seunghyun Yoon, HyunJoon Jung, Jae Shin Yoon, Seungryul Baek
As a condition of the lighting images, we perform image-based relighting for both foreground and background using a single portrait image or a set of OLAT (One-Light-at-A-Time) images captured from lightstage system.
no code implementations • 7 Oct 2024 • Jae Shin Yoon, Zhixin Shu, Mengwei Ren, Xuaner Zhang, Yannick Hold-Geoffroy, Krishna Kumar Singh, He Zhang
For robust and natural shadow removal, we propose to train the diffusion model with a compositional repurposing framework: a pre-trained text-guided image generation model is first fine-tuned to harmonize the lighting and color of the foreground with a background scene by using a background harmonization dataset; and then the model is further fine-tuned to generate a shadow-free portrait image via a shadow-paired dataset.
no code implementations • 25 Aug 2024 • Andrew Hou, Zhixin Shu, Xuaner Zhang, He Zhang, Yannick Hold-Geoffroy, Jae Shin Yoon, Xiaoming Liu
Existing portrait relighting methods struggle with precise control over facial shadows, particularly when faced with challenges such as handling hard shadows from directional light sources or adjusting shadows while remaining in harmony with existing lighting conditions.
no code implementations • 8 Jul 2024 • Mohammad Reza Karimi Dastjerdi, Frédéric Fortier-Chouinard, Yannick Hold-Geoffroy, Marc Hébert, Claude Demers, Nima Kalantari, Jean-François Lalonde
Most novel view synthesis methods such as NeRF are unable to capture the true high dynamic range (HDR) radiance of scenes since they are typically trained on photos captured with standard low dynamic range (LDR) cameras.
no code implementations • 1 May 2024 • Zheng Zeng, Valentin Deschaintre, Iliyan Georgiev, Yannick Hold-Geoffroy, Yiwei Hu, Fujun Luan, Ling-Qi Yan, Miloš Hašan
Our X$\rightarrow$RGB model explores a middle ground between traditional rendering and generative models: we can specify only certain appearance properties that should be followed, and give freedom to the model to hallucinate a plausible version of the rest.
no code implementations • CVPR 2024 • Peter Kocsis, Julien Philip, Kalyan Sunkavalli, Matthias Nießner, Yannick Hold-Geoffroy
Our method is the first that enables the generation of images with controllable, consistent lighting and performs on par with specialized relighting state-of-the-art methods.
no code implementations • CVPR 2024 • Karran Pandey, Paul Guerrero, Matheus Gadelha, Yannick Hold-Geoffroy, Karan Singh, Niloy J. Mitra
Diffusion handles is a novel approach to enable 3D object edits on diffusion images requiring only existing pre-trained diffusion models depth estimation without any fine-tuning or 3D object retrieval.
1 code implementation • CVPR 2024 • Justine Giroux, Mohammad Reza Karimi Dastjerdi, Yannick Hold-Geoffroy, Javier Vazquez-Corral, Jean-François Lalonde
Progress in lighting estimation is tracked by computing existing image quality assessment (IQA) metrics on images from standard datasets.
no code implementations • 2 Dec 2023 • Karran Pandey, Paul Guerrero, Matheus Gadelha, Yannick Hold-Geoffroy, Karan Singh, Niloy Mitra
Our key insight is to lift diffusion activations for an object to 3D using a proxy depth, 3D-transform the depth and associated activations, and project them back to image space.
no code implementations • 20 May 2023 • Xilong Zhou, Miloš Hašan, Valentin Deschaintre, Paul Guerrero, Yannick Hold-Geoffroy, Kalyan Sunkavalli, Nima Khademi Kalantari
Instead, we train a generator for a neural material representation that is rendered with a learned relighting module to create arbitrarily lit RGB images; these are compared against real photos using a discriminator.
no code implementations • ICCV 2023 • Mohammad Reza Karimi Dastjerdi, Jonathan Eisenmann, Yannick Hold-Geoffroy, Jean-François Lalonde
In this work, we propose to bridge the gap between these recent trends in the literature, and propose a method which combines a parametric light model with 360{\deg} panoramas, ready to use as HDRI in rendering engines.
no code implementations • CVPR 2023 • Yichen Sheng, Jianming Zhang, Julien Philip, Yannick Hold-Geoffroy, Xin Sun, He Zhang, Lu Ling, Bedrich Benes
To compensate for the lack of geometry in 2D Image compositing, recent deep learning-based approaches introduced a pixel height representation to generate soft shadows and reflections.
no code implementations • ICCV 2023 • Dominique Piché-Meunier, Yannick Hold-Geoffroy, Jianming Zhang, Jean-François Lalonde
Instead, we go further and propose to use a lens-based representation that models the depth of field using two parameters: the blur factor and focus disparity.
no code implementations • ICCV 2023 • Manuel Ladron De Guevara, Jose Echevarria, Yijun Li, Yannick Hold-Geoffroy, Cameron Smith, Daichi Ito
We present a novel method for automatic vectorized avatar generation from a single portrait image.
no code implementations • CVPR 2023 • Byeong-Uk Lee, Jianming Zhang, Yannick Hold-Geoffroy, In So Kweon
In this paper, we propose a single image scale estimation method based on a novel scale field representation.
1 code implementation • CVPR 2023 • Linyi Jin, Jianming Zhang, Yannick Hold-Geoffroy, Oliver Wang, Kevin Matzen, Matthew Sticha, David F. Fouhey
We propose perspective fields as a representation that models the local perspective properties of an image.
no code implementations • 25 Aug 2022 • Yannick Hold-Geoffroy, Dominique Piché-Meunier, Kalyan Sunkavalli, Jean-Charles Bazin, François Rameau, Jean-François Lalonde
Image editing and compositing have become ubiquitous in entertainment, from digital art to AR and VR experiences.
1 code implementation • CVPR 2022 • Yu-Ying Yeh, Zhengqin Li, Yannick Hold-Geoffroy, Rui Zhu, Zexiang Xu, Miloš Hašan, Kalyan Sunkavalli, Manmohan Chandraker
Most indoor 3D scene reconstruction methods focus on recovering 3D geometry and scene layout.
no code implementations • 15 Apr 2022 • Mohammad Reza Karimi Dastjerdi, Yannick Hold-Geoffroy, Jonathan Eisenmann, Siavash Khodadadeh, Jean-François Lalonde
We propose a method to extrapolate a 360{\deg} field of view from a single image that allows for user-controlled synthesis of the out-painted content.
1 code implementation • CVPR 2021 • Fanbo Xiang, Zexiang Xu, Miloš Hašan, Yannick Hold-Geoffroy, Kalyan Sunkavalli, Hao Su
We achieve this by introducing a 3D-to-2D texture mapping (or surface parameterization) network into volumetric representations.
no code implementations • 9 Aug 2020 • Sai Bi, Zexiang Xu, Pratul Srinivasan, Ben Mildenhall, Kalyan Sunkavalli, Miloš Hašan, Yannick Hold-Geoffroy, David Kriegman, Ravi Ramamoorthi
We combine this representation with a physically-based differentiable ray marching framework that can render images from a neural reflectance field under any viewpoint and light.
no code implementations • ECCV 2020 • Kai-En Lin, Zexiang Xu, Ben Mildenhall, Pratul P. Srinivasan, Yannick Hold-Geoffroy, Stephen DiVerdi, Qi Sun, Kalyan Sunkavalli, Ravi Ramamoorthi
We propose a learning-based approach for novel view synthesis for multi-camera 360$^{\circ}$ panorama capture rigs.
no code implementations • ECCV 2020 • Sai Bi, Zexiang Xu, Kalyan Sunkavalli, Miloš Hašan, Yannick Hold-Geoffroy, David Kriegman, Ravi Ramamoorthi
We also show that our learned reflectance volumes are editable, allowing for modifying the materials of the captured scenes.
1 code implementation • ECCV 2020 • Rui Zhu, Xingyi Yang, Yannick Hold-Geoffroy, Federico Perazzi, Jonathan Eisenmann, Kalyan Sunkavalli, Manmohan Chandraker
Most 3D reconstruction methods may only recover scene properties up to a global scale ambiguity.
1 code implementation • 6 Apr 2020 • Julia Gong, Yannick Hold-Geoffroy, Jingwan Lu
Caricature, a type of exaggerated artistic portrait, amplifies the distinctive, yet nuanced traits of human faces.
no code implementations • 19 Oct 2019 • Marc-André Gardner, Yannick Hold-Geoffroy, Kalyan Sunkavalli, Christian Gagné, Jean-François Lalonde
We present a method to estimate lighting from a single image of an indoor scene.
no code implementations • CVPR 2019 • Jinsong Zhang, Kalyan Sunkavalli, Yannick Hold-Geoffroy, Sunil Hadap, Jonathan Eisenmann, Jean-François Lalonde
We use this network to label a large-scale dataset of LDR panoramas with lighting parameters and use them to train our single image outdoor lighting estimation network.
no code implementations • CVPR 2019 • Yannick Hold-Geoffroy, Akshaya Athawale, Jean-François Lalonde
We propose a data-driven learned sky model, which we use for outdoor lighting estimation from a single image.
no code implementations • 28 Mar 2018 • Yannick Hold-Geoffroy, Paulo F. U. Gotardo, Jean-François Lalonde
Our analysis reveals that partially cloudy days improve the conditioning of the outdoor PS problem while sunny days do not allow the unambiguous recovery of surface normals from photometric cues alone.
no code implementations • CVPR 2018 • Yannick Hold-Geoffroy, Kalyan Sunkavalli, Jonathan Eisenmann, Matt Fisher, Emiliano Gambaretto, Sunil Hadap, Jean-François Lalonde
This network is trained using automatically generated samples from a large-scale panorama dataset, and considerably outperforms other methods, including recent deep learning-based approaches, in terms of standard L2 error.
1 code implementation • CVPR 2017 • Yannick Hold-Geoffroy, Kalyan Sunkavalli, Sunil Hadap, Emiliano Gambaretto, Jean-François Lalonde
We present a CNN-based technique to estimate high-dynamic range outdoor illumination from a single low dynamic range image.
Ranked #1 on
Outdoor Light Source Estimation
on SUN360