Search Results for author: Zan Gojcic

Found 27 papers, 13 papers with code

Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Models

no code implementations3 Mar 2025 Jay Zhangjie Wu, Yuxuan Zhang, Haithem Turki, Xuanchi Ren, Jun Gao, Mike Zheng Shou, Sanja Fidler, Zan Gojcic, Huan Ling

At the core of our approach is Difix, a single-step image diffusion model trained to enhance and remove artifacts in rendered novel views caused by underconstrained regions of the 3D representation.

3DGS 3D Reconstruction +2

DiffusionRenderer: Neural Inverse and Forward Rendering with Video Diffusion Models

no code implementations30 Jan 2025 Ruofan Liang, Zan Gojcic, Huan Ling, Jacob Munkberg, Jon Hasselgren, Zhi-Hao Lin, Jun Gao, Alexander Keller, Nandita Vijaykumar, Sanja Fidler, Zian Wang

Classic physically-based rendering (PBR) accurately simulates the light transport, but relies on precise scene representations--explicit 3D geometry, high-quality material properties, and lighting conditions--that are often impractical to obtain in real-world scenarios.

3D geometry Inverse Rendering

3DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splatting

1 code implementation17 Dec 2024 Qi Wu, Janick Martinez Esturo, Ashkan Mirzaei, Nicolas Moenne-Loccoz, Zan Gojcic

3D Gaussian Splatting (3DGS) enables efficient reconstruction and high-fidelity real-time rendering of complex scenes on consumer hardware.

3DGS

OmniRe: Omni Urban Scene Reconstruction

no code implementations29 Aug 2024 Ziyu Chen, Jiawei Yang, Jiahui Huang, Riccardo de Lutio, Janick Martinez Esturo, Boris Ivanovic, Or Litany, Zan Gojcic, Sanja Fidler, Marco Pavone, Li Song, Yue Wang

To that end, we propose a comprehensive 3DGS framework for driving scenes, named OmniRe, that allows for accurate, full-length reconstruction of diverse dynamic objects in a driving log.

3DGS

Photorealistic Object Insertion with Diffusion-Guided Inverse Rendering

no code implementations19 Aug 2024 Ruofan Liang, Zan Gojcic, Merlin Nimier-David, David Acuna, Nandita Vijaykumar, Sanja Fidler, Zian Wang

The correct insertion of virtual objects in images of real-world scenes requires a deep understanding of the scene's lighting, geometry and materials, as well as the image formation process.

Inverse Rendering Object +1

3D Gaussian Ray Tracing: Fast Tracing of Particle Scenes

no code implementations9 Jul 2024 Nicolas Moenne-Loccoz, Ashkan Mirzaei, Or Perel, Riccardo de Lutio, Janick Martinez Esturo, Gavriel State, Sanja Fidler, Nicholas Sharp, Zan Gojcic

The benefits of ray tracing are well-known in computer graphics: processing incoherent rays for secondary lighting effects such as shadows and reflections, rendering from highly-distorted cameras common in robotics, stochastically sampling rays, and more.

Memorize What Matters: Emergent Scene Decomposition from Multitraverse

1 code implementation27 May 2024 Yiming Li, Zehong Wang, Yue Wang, Zhiding Yu, Zan Gojcic, Marco Pavone, Chen Feng, Jose M. Alvarez

Humans naturally retain memories of permanent elements, while ephemeral moments often slip through the cracks of memory.

3D Reconstruction Neural Rendering +2

RefFusion: Reference Adapted Diffusion Models for 3D Scene Inpainting

no code implementations16 Apr 2024 Ashkan Mirzaei, Riccardo de Lutio, Seung Wook Kim, David Acuna, Jonathan Kelly, Sanja Fidler, Igor Gilitschenski, Zan Gojcic

In this work, we propose an approach for 3D scene inpainting -- the task of coherently replacing parts of the reconstructed scene with desired content.

3D Inpainting Image Inpainting

Adaptive Shells for Efficient Neural Radiance Field Rendering

no code implementations16 Nov 2023 Zian Wang, Tianchang Shen, Merlin Nimier-David, Nicholas Sharp, Jun Gao, Alexander Keller, Sanja Fidler, Thomas Müller, Zan Gojcic

We then extract an explicit mesh of a narrow band around the surface, with width determined by the kernel size, and fine-tune the radiance field within this band.

Novel View Synthesis Stochastic Optimization

Towards Viewpoint Robustness in Bird's Eye View Segmentation

no code implementations ICCV 2023 Tzofi Klinghoffer, Jonah Philion, Wenzheng Chen, Or Litany, Zan Gojcic, Jungseock Joo, Ramesh Raskar, Sanja Fidler, Jose M. Alvarez

We introduce a technique for novel view synthesis and use it to transform collected data to the viewpoint of target rigs, allowing us to train BEV segmentation models for diverse target rigs without any additional data collection or labeling cost.

Autonomous Vehicles BEV Segmentation +1

Flexible Isosurface Extraction for Gradient-Based Mesh Optimization

1 code implementation10 Aug 2023 Tianchang Shen, Jacob Munkberg, Jon Hasselgren, Kangxue Yin, Zian Wang, Wenzheng Chen, Zan Gojcic, Sanja Fidler, Nicholas Sharp, Jun Gao

This work considers gradient-based mesh optimization, where we iteratively optimize for a 3D surface mesh by representing it as the isosurface of a scalar field, an increasingly common paradigm in applications including photogrammetry, generative modeling, and inverse physics.

Neural Kernel Surface Reconstruction

1 code implementation CVPR 2023 Jiahui Huang, Zan Gojcic, Matan Atzmon, Or Litany, Sanja Fidler, Francis Williams

We present a novel method for reconstructing a 3D implicit surface from a large-scale, sparse, and noisy point cloud.

Surface Reconstruction

Neural LiDAR Fields for Novel View Synthesis

no code implementations ICCV 2023 Shengyu Huang, Zan Gojcic, Zian Wang, Francis Williams, Yoni Kasten, Sanja Fidler, Konrad Schindler, Or Litany

We present Neural Fields for LiDAR (NFL), a method to optimise a neural field scene representation from LiDAR measurements, with the goal of synthesizing realistic LiDAR scans from novel viewpoints.

NeRF Novel LiDAR View Synthesis +1

Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenes

no code implementations6 Apr 2023 Zian Wang, Tianchang Shen, Jun Gao, Shengyu Huang, Jacob Munkberg, Jon Hasselgren, Zan Gojcic, Wenzheng Chen, Sanja Fidler

Reconstruction and intrinsic decomposition of scenes from captured imagery would enable many applications such as relighting and virtual object insertion.

3D Reconstruction Inverse Rendering +1

LION: Latent Point Diffusion Models for 3D Shape Generation

2 code implementations12 Oct 2022 Xiaohui Zeng, Arash Vahdat, Francis Williams, Zan Gojcic, Or Litany, Sanja Fidler, Karsten Kreis

To advance 3D DDMs and make them useful for digital artists, we require (i) high generation quality, (ii) flexibility for manipulation and applications such as conditional synthesis and shape interpolation, and (iii) the ability to output smooth surfaces or meshes.

3D Generation 3D Shape Generation +3

GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images

3 code implementations22 Sep 2022 Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, Kangxue Yin, Daiqing Li, Or Litany, Zan Gojcic, Sanja Fidler

As several industries are moving towards modeling massive 3D virtual worlds, the need for content creation tools that can scale in terms of the quantity, quality, and diversity of 3D content is becoming evident.

Diversity

Dynamic 3D Scene Analysis by Point Cloud Accumulation

1 code implementation25 Jul 2022 Shengyu Huang, Zan Gojcic, Jiahui Huang, Andreas Wieser, Konrad Schindler

Compared to state-of-the-art scene flow estimators, our proposed approach aims to align all 3D points in a common reference frame correctly accumulating the points on the individual objects.

Autonomous Vehicles Semantic Segmentation +1

Multiway Non-rigid Point Cloud Registration via Learned Functional Map Synchronization

1 code implementation25 Nov 2021 Jiahui Huang, Tolga Birdal, Zan Gojcic, Leonidas J. Guibas, Shi-Min Hu

We present SyNoRiM, a novel way to jointly register multiple non-rigid shapes by synchronizing the maps relating learned functions defined on the point clouds.

Point Cloud Registration

Weakly Supervised Learning of Rigid 3D Scene Flow

1 code implementation CVPR 2021 Zan Gojcic, Or Litany, Andreas Wieser, Leonidas J. Guibas, Tolga Birdal

We propose a data-driven scene flow estimation algorithm exploiting the observation that many 3D scenes can be explained by a collection of agents moving as rigid bodies.

Autonomous Driving Scene Flow Estimation +2

CaSPR: Learning Canonical Spatiotemporal Point Cloud Representations

1 code implementation NeurIPS 2020 Davis Rempe, Tolga Birdal, Yongheng Zhao, Zan Gojcic, Srinath Sridhar, Leonidas J. Guibas

We propose CaSPR, a method to learn object-centric Canonical Spatiotemporal Point Cloud Representations of dynamically moving or evolving objects.

Camera Pose Estimation Object +1

Cannot find the paper you are looking for? You can Submit a new open access paper.