Search Results for author: Vitor Guizilini

Found 36 papers, 14 papers with code

NeRF-MAE : Masked AutoEncoders for Self Supervised 3D representation Learning for Neural Radiance Fields

no code implementations1 Apr 2024 Muhammad Zubair Irshad, Sergey Zakahrov, Vitor Guizilini, Adrien Gaidon, Zsolt Kira, Rares Ambrus

Given the capabilities of neural fields in densely representing a 3D scene from 2D images, we ask the question: Can we scale their self-supervised pretraining, specifically using masked autoencoders, to generate effective 3D representations from posed RGB images.

3D Object Detection object-detection +3

Towards Realistic Scene Generation with LiDAR Diffusion Models

1 code implementation31 Mar 2024 Haoxi Ran, Vitor Guizilini, Yue Wang

In this paper, we propose LiDAR Diffusion Models (LiDMs) to generate LiDAR-realistic scenes from a latent space tailored to capture the realism of LiDAR scenes by incorporating geometric priors into the learning pipeline.

Image Generation Scene Generation

Zero-Shot Multi-Object Shape Completion

no code implementations21 Mar 2024 Shun Iwase, Katherine Liu, Vitor Guizilini, Adrien Gaidon, Kris Kitani, Rares Ambrus, Sergey Zakharov

We present a 3D shape completion method that recovers the complete geometry of multiple objects in complex scenes from a single RGB-D image.


NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenes

1 code implementation ICCV 2023 Muhammad Zubair Irshad, Sergey Zakharov, Katherine Liu, Vitor Guizilini, Thomas Kollar, Adrien Gaidon, Zsolt Kira, Rares Ambrus

NeO 360's representation allows us to learn from a large collection of unbounded 3D scenes while offering generalizability to new views and novel scenes from as few as a single image during inference.

Generalizable Novel View Synthesis Novel View Synthesis

Robust Self-Supervised Extrinsic Self-Calibration

no code implementations4 Aug 2023 Takayuki Kanai, Igor Vasiljevic, Vitor Guizilini, Adrien Gaidon, Rares Ambrus

Autonomous vehicles and robots need to operate over a wide variety of scenarios in order to complete tasks efficiently and safely.

Autonomous Vehicles Depth Prediction +2

NeRFuser: Large-Scale Scene Representation by NeRF Fusion

1 code implementation22 May 2023 Jiading Fang, Shengjie Lin, Igor Vasiljevic, Vitor Guizilini, Rares Ambrus, Adrien Gaidon, Gregory Shakhnarovich, Matthew R. Walter

A practical benefit of implicit visual representations like Neural Radiance Fields (NeRFs) is their memory efficiency: large scenes can be efficiently stored and shared as small neural nets instead of collections of images.

DeLiRa: Self-Supervised Depth, Light, and Radiance Fields

no code implementations ICCV 2023 Vitor Guizilini, Igor Vasiljevic, Jiading Fang, Rares Ambrus, Sergey Zakharov, Vincent Sitzmann, Adrien Gaidon

In this work, we propose to use the multi-view photometric objective from the self-supervised depth estimation literature as a geometric regularizer for volumetric rendering, significantly improving novel view synthesis without requiring additional information.

3D Reconstruction Depth Estimation +1

Viewpoint Equivariance for Multi-View 3D Object Detection

1 code implementation CVPR 2023 Dian Chen, Jie Li, Vitor Guizilini, Rares Ambrus, Adrien Gaidon

We design view-conditioned queries at the output level, which enables the generation of multiple virtual frames during training to learn viewpoint equivariance by enforcing multi-view consistency.

3D Object Detection Object +2

Photo-realistic Neural Domain Randomization

no code implementations23 Oct 2022 Sergey Zakharov, Rares Ambrus, Vitor Guizilini, Wadim Kehl, Adrien Gaidon

In this paper, we show that the recent progress in neural rendering enables a new unified approach we call Photo-realistic Neural Domain Randomization (PNDR).

Image Generation Monocular Depth Estimation +3

Depth Is All You Need for Monocular 3D Detection

no code implementations5 Oct 2022 Dennis Park, Jie Li, Dian Chen, Vitor Guizilini, Adrien Gaidon

Our methods leverage commonly available LiDAR or RGB videos during training time to fine-tune the depth representation, which leads to improved 3D detectors.

Depth Prediction Monocular Depth Estimation +1

Depth Field Networks for Generalizable Multi-view Scene Representation

no code implementations28 Jul 2022 Vitor Guizilini, Igor Vasiljevic, Jiading Fang, Rares Ambrus, Greg Shakhnarovich, Matthew Walter, Adrien Gaidon

Modern 3D computer vision leverages learning to boost geometric reasoning, mapping image data to classical structures such as cost volumes or epipolar constraints to improve matching.

Data Augmentation Depth Estimation +2

SpOT: Spatiotemporal Modeling for 3D Object Tracking

no code implementations12 Jul 2022 Colton Stearns, Davis Rempe, Jie Li, Rares Ambrus, Sergey Zakharov, Vitor Guizilini, Yanchao Yang, Leonidas J Guibas

In this work, we develop a holistic representation of traffic scenes that leverages both spatial and temporal information of the actors in the scene.

3D Multi-Object Tracking 3D Object Tracking +1

Multi-Frame Self-Supervised Depth with Transformers

no code implementations CVPR 2022 Vitor Guizilini, Rares Ambrus, Dian Chen, Sergey Zakharov, Adrien Gaidon

Experiments on the KITTI and DDAD datasets show that our DepthFormer architecture establishes a new state of the art in self-supervised monocular depth estimation, and is even competitive with highly specialized supervised single-frame architectures.

Monocular Depth Estimation

Learning Optical Flow, Depth, and Scene Flow without Real-World Labels

no code implementations28 Mar 2022 Vitor Guizilini, Kuan-Hui Lee, Rares Ambrus, Adrien Gaidon

However, the simultaneous self-supervised learning of depth and scene flow is ill-posed, as there are infinitely many combinations that result in the same 3D point.

Autonomous Driving Monocular Depth Estimation +3

Self-Supervised Camera Self-Calibration from Video

no code implementations6 Dec 2021 Jiading Fang, Igor Vasiljevic, Vitor Guizilini, Rares Ambrus, Greg Shakhnarovich, Adrien Gaidon, Matthew R. Walter

Camera calibration is integral to robotics and computer vision algorithms that seek to infer geometric properties of the scene from visual input streams.

Autonomous Vehicles Camera Calibration +3

DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries

1 code implementation13 Oct 2021 Yue Wang, Vitor Guizilini, Tianyuan Zhang, Yilun Wang, Hang Zhao, Justin Solomon

This top-down approach outperforms its bottom-up counterpart in which object bounding box prediction follows per-pixel depth estimation, since it does not suffer from the compounding error introduced by a depth prediction model.

3D Object Detection Autonomous Driving +5

Is Pseudo-Lidar needed for Monocular 3D Object detection?

2 code implementations ICCV 2021 Dennis Park, Rares Ambrus, Vitor Guizilini, Jie Li, Adrien Gaidon

Recent progress in 3D object detection from single images leverages monocular depth estimation as a way to produce 3D pointclouds, turning cameras into pseudo-lidar sensors.

 Ranked #1 on Monocular 3D Object Detection on KITTI Pedestrian Moderate (using extra training data)

Monocular 3D Object Detection Monocular Depth Estimation +2

MarioNette: Self-Supervised Sprite Learning

1 code implementation NeurIPS 2021 Dmitriy Smirnov, Michael Gharbi, Matthew Fisher, Vitor Guizilini, Alexei A. Efros, Justin Solomon

Artists and video game designers often construct 2D animations using libraries of sprites -- textured patches of objects and characters.

Full Surround Monodepth from Multiple Cameras

no code implementations31 Mar 2021 Vitor Guizilini, Igor Vasiljevic, Rares Ambrus, Greg Shakhnarovich, Adrien Gaidon

In this work, we extend monocular self-supervised depth and ego-motion estimation to large-baseline multi-camera rigs.

Autonomous Driving Motion Estimation

Geometric Unsupervised Domain Adaptation for Semantic Segmentation

no code implementations ICCV 2021 Vitor Guizilini, Jie Li, Rares Ambrus, Adrien Gaidon

Simulators can efficiently generate large amounts of labeled synthetic data with perfect supervision for hard-to-label tasks like semantic segmentation.

Depth Prediction Monocular Depth Estimation +3

Monocular Depth Estimation for Soft Visuotactile Sensors

no code implementations5 Jan 2021 Rares Ambrus, Vitor Guizilini, Naveen Kuppuswamy, Andrew Beaulieu, Adrien Gaidon, Alex Alspach

Fluid-filled soft visuotactile sensors such as the Soft-bubbles alleviate key challenges for robust manipulation, as they enable reliable grasps along with the ability to obtain high-resolution sensory feedback on contact geometry and forces.

Monocular Depth Estimation

Neural Ray Surfaces for Self-Supervised Learning of Depth and Ego-motion

1 code implementation15 Aug 2020 Igor Vasiljevic, Vitor Guizilini, Rares Ambrus, Sudeep Pillai, Wolfram Burgard, Greg Shakhnarovich, Adrien Gaidon

Self-supervised learning has emerged as a powerful tool for depth and ego-motion estimation, leading to state-of-the-art results on benchmark datasets.

Depth Estimation Motion Estimation +2

Semantically-Guided Representation Learning for Self-Supervised Monocular Depth

1 code implementation ICLR 2020 Vitor Guizilini, Rui Hou, Jie Li, Rares Ambrus, Adrien Gaidon

Instead of using semantic labels and proxy losses in a multi-task approach, we propose a new architecture leveraging fixed pretrained semantic segmentation networks to guide self-supervised representation learning via pixel-adaptive convolutions.

Depth Prediction Monocular Depth Estimation +3

Neural Outlier Rejection for Self-Supervised Keypoint Learning

2 code implementations ICLR 2020 Jiexiong Tang, Hanme Kim, Vitor Guizilini, Sudeep Pillai, Rares Ambrus

By making the sampling of inlier-outlier sets from point-pair correspondences fully differentiable within the keypoint learning framework, we show that are able to simultaneously self-supervise keypoint description and improve keypoint matching.

Homography Estimation Keypoint Detection +1

Dynamic Hilbert Maps: Real-Time Occupancy Predictions in Changing Environment

no code implementations4 Dec 2019 Vitor Guizilini, Ransalu Senanayake, Fabio Ramos

This paper addresses the problem of learning instantaneous occupancy levels of dynamic environments and predicting future occupancy levels.

Real-Time Panoptic Segmentation from Dense Detections

no code implementations CVPR 2020 Rui Hou, Jie Li, Arjun Bhargava, Allan Raventos, Vitor Guizilini, Chao Fang, Jerome Lynch, Adrien Gaidon

Panoptic segmentation is a complex full scene parsing task requiring simultaneous instance and semantic segmentation at high resolution.

Clustering object-detection +4

Two Stream Networks for Self-Supervised Ego-Motion Estimation

no code implementations4 Oct 2019 Rares Ambrus, Vitor Guizilini, Jie Li, Sudeep Pillai, Adrien Gaidon

Learning depth and camera ego-motion from raw unlabeled RGB video streams is seeing exciting progress through self-supervision from strong geometric cues.

Data Augmentation Motion Estimation +2

Robust Semi-Supervised Monocular Depth Estimation with Reprojected Distances

no code implementations4 Oct 2019 Vitor Guizilini, Jie Li, Rares Ambrus, Sudeep Pillai, Adrien Gaidon

Dense depth estimation from a single image is a key problem in computer vision, with exciting applications in a multitude of robotic tasks.

Monocular Depth Estimation valid

Sparse-to-Continuous: Enhancing Monocular Depth Estimation using Occupancy Maps

1 code implementation24 Sep 2018 Nícolas Rosa, Vitor Guizilini, Valdir Grassi Jr

This paper addresses the problem of single image depth estimation (SIDE), focusing on improving the quality of deep neural network predictions.

Monocular Depth Estimation

Learning to Race through Coordinate Descent Bayesian Optimisation

no code implementations17 Feb 2018 Rafael Oliveira, Fernando H. M. Rocha, Lionel Ott, Vitor Guizilini, Fabio Ramos, Valdir Grassi Jr

On the other hand, the cost to evaluate the policy's performance might also be high, being desirable that a solution can be found with as few interactions as possible with the real system.

Bayesian Optimisation Car Racing +1

Continuous Convolutional Neural Networks for Image Classification

no code implementations ICLR 2018 Vitor Guizilini, Fabio Ramos

This paper introduces the concept of continuous convolution to neural networks and deep learning applications in general.

Classification General Classification +1

Bayesian Optimisation for Safe Navigation under Localisation Uncertainty

no code implementations7 Sep 2017 Rafael Oliveira, Lionel Ott, Vitor Guizilini, Fabio Ramos

In outdoor environments, mobile robots are required to navigate through terrain with varying characteristics, some of which might significantly affect the integrity of the platform.

Bayesian Optimisation Navigate

Cannot find the paper you are looking for? You can Submit a new open access paper.