Search Results for author: Adrien Gaidon

Found 78 papers, 29 papers with code

NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenes

1 code implementation ICCV 2023 Muhammad Zubair Irshad, Sergey Zakharov, Katherine Liu, Vitor Guizilini, Thomas Kollar, Adrien Gaidon, Zsolt Kira, Rares Ambrus

NeO 360's representation allows us to learn from a large collection of unbounded 3D scenes while offering generalizability to new views and novel scenes from as few as a single image during inference.

Generalizable Novel View Synthesis Novel View Synthesis

Robust Self-Supervised Extrinsic Self-Calibration

no code implementations4 Aug 2023 Takayuki Kanai, Igor Vasiljevic, Vitor Guizilini, Adrien Gaidon, Rares Ambrus

Autonomous vehicles and robots need to operate over a wide variety of scenarios in order to complete tasks efficiently and safely.

Autonomous Vehicles Depth Prediction +2

HomE: Homography-Equivariant Video Representation Learning

1 code implementation2 Jun 2023 Anirudh Sriram, Adrien Gaidon, Jiajun Wu, Juan Carlos Niebles, Li Fei-Fei, Ehsan Adeli

In this work, we propose a novel method for representation learning of multi-view videos, where we explicitly model the representation space to maintain Homography Equivariance (HomE).

Action Classification Action Recognition +2

NeRFuser: Large-Scale Scene Representation by NeRF Fusion

1 code implementation22 May 2023 Jiading Fang, Shengjie Lin, Igor Vasiljevic, Vitor Guizilini, Rares Ambrus, Adrien Gaidon, Gregory Shakhnarovich, Matthew R. Walter

A practical benefit of implicit visual representations like Neural Radiance Fields (NeRFs) is their memory efficiency: large scenes can be efficiently stored and shared as small neural nets instead of collections of images.

DeLiRa: Self-Supervised Depth, Light, and Radiance Fields

no code implementations ICCV 2023 Vitor Guizilini, Igor Vasiljevic, Jiading Fang, Rares Ambrus, Sergey Zakharov, Vincent Sitzmann, Adrien Gaidon

In this work, we propose to use the multi-view photometric objective from the self-supervised depth estimation literature as a geometric regularizer for volumetric rendering, significantly improving novel view synthesis without requiring additional information.

3D Reconstruction Depth Estimation +1

Object Discovery from Motion-Guided Tokens

2 code implementations CVPR 2023 Zhipeng Bao, Pavel Tokmakov, Yu-Xiong Wang, Adrien Gaidon, Martial Hebert

Object discovery -- separating objects from the background without manual labels -- is a fundamental open challenge in computer vision.

Object Discovery Quantization +1

Viewpoint Equivariance for Multi-View 3D Object Detection

1 code implementation CVPR 2023 Dian Chen, Jie Li, Vitor Guizilini, Rares Ambrus, Adrien Gaidon

We design view-conditioned queries at the output level, which enables the generation of multiple virtual frames during training to learn viewpoint equivariance by enforcing multi-view consistency.

3D Object Detection object-detection +1

In-Distribution Barrier Functions: Self-Supervised Policy Filters that Avoid Out-of-Distribution States

no code implementations27 Jan 2023 Fernando Castañeda, Haruki Nishimura, Rowan Mcallister, Koushil Sreenath, Adrien Gaidon

Learning-based control approaches have shown great promise in performing complex tasks directly from high-dimensional perception data for real robotic systems.

ROAD: Learning an Implicit Recursive Octree Auto-Decoder to Efficiently Encode 3D Shapes

no code implementations12 Dec 2022 Sergey Zakharov, Rares Ambrus, Katherine Liu, Adrien Gaidon

Compact and accurate representations of 3D shapes are central to many perception and robotics tasks.

Photo-realistic Neural Domain Randomization

no code implementations23 Oct 2022 Sergey Zakharov, Rares Ambrus, Vitor Guizilini, Wadim Kehl, Adrien Gaidon

In this paper, we show that the recent progress in neural rendering enables a new unified approach we call Photo-realistic Neural Domain Randomization (PNDR).

Image Generation Monocular Depth Estimation +3

Depth Is All You Need for Monocular 3D Detection

no code implementations5 Oct 2022 Dennis Park, Jie Li, Dian Chen, Vitor Guizilini, Adrien Gaidon

Our methods leverage commonly available LiDAR or RGB videos during training time to fine-tune the depth representation, which leads to improved 3D detectors.

Depth Prediction Monocular Depth Estimation +1

RAP: Risk-Aware Prediction for Robust Planning

1 code implementation4 Oct 2022 Haruki Nishimura, Jean Mercat, Blake Wulfe, Rowan Mcallister, Adrien Gaidon

Robust planning in interactive scenarios requires predicting the uncertain future to make risk-aware decisions.

Depth Field Networks for Generalizable Multi-view Scene Representation

no code implementations28 Jul 2022 Vitor Guizilini, Igor Vasiljevic, Jiading Fang, Rares Ambrus, Greg Shakhnarovich, Matthew Walter, Adrien Gaidon

Modern 3D computer vision leverages learning to boost geometric reasoning, mapping image data to classical structures such as cost volumes or epipolar constraints to improve matching.

Data Augmentation Depth Estimation +2

Neural Groundplans: Persistent Neural Scene Representations from a Single Image

no code implementations22 Jul 2022 Prafull Sharma, Ayush Tewari, Yilun Du, Sergey Zakharov, Rares Ambrus, Adrien Gaidon, William T. Freeman, Fredo Durand, Joshua B. Tenenbaum, Vincent Sitzmann

We present a method to map 2D image observations of a scene to a persistent 3D scene representation, enabling novel view synthesis and disentangled representation of the movable and immovable components of the scene.

Disentanglement Instance Segmentation +4

Control-Aware Prediction Objectives for Autonomous Driving

no code implementations28 Apr 2022 Rowan Mcallister, Blake Wulfe, Jean Mercat, Logan Ellis, Sergey Levine, Adrien Gaidon

Autonomous vehicle software is typically structured as a modular pipeline of individual components (e. g., perception, prediction, and planning) to help separate concerns into interpretable sub-tasks.

Autonomous Driving Trajectory Prediction

Multi-Frame Self-Supervised Depth with Transformers

no code implementations CVPR 2022 Vitor Guizilini, Rares Ambrus, Dian Chen, Sergey Zakharov, Adrien Gaidon

Experiments on the KITTI and DDAD datasets show that our DepthFormer architecture establishes a new state of the art in self-supervised monocular depth estimation, and is even competitive with highly specialized supervised single-frame architectures.

Monocular Depth Estimation

Object Permanence Emerges in a Random Walk along Memory

1 code implementation4 Apr 2022 Pavel Tokmakov, Allan Jabri, Jie Li, Adrien Gaidon

This paper proposes a self-supervised objective for learning representations that localize objects under occlusion - a property known as object permanence.

Learning Optical Flow, Depth, and Scene Flow without Real-World Labels

no code implementations28 Mar 2022 Vitor Guizilini, Kuan-Hui Lee, Rares Ambrus, Adrien Gaidon

However, the simultaneous self-supervised learning of depth and scene flow is ill-posed, as there are infinitely many combinations that result in the same 3D point.

Autonomous Driving Monocular Depth Estimation +3

Discovering Objects that Can Move

1 code implementation CVPR 2022 Zhipeng Bao, Pavel Tokmakov, Allan Jabri, Yu-Xiong Wang, Adrien Gaidon, Martial Hebert

Our experiments demonstrate that, despite only capturing a small subset of the objects that move, this signal is enough to generalize to segment both moving and static instances of dynamic objects.

Motion Segmentation Object Discovery

Dynamics-Aware Comparison of Learned Reward Functions

no code implementations ICLR 2022 Blake Wulfe, Ashwin Balakrishna, Logan Ellis, Jean Mercat, Rowan Mcallister, Adrien Gaidon

The ability to learn reward functions plays an important role in enabling the deployment of intelligent agents in the real world.

Self-Supervised Camera Self-Calibration from Video

no code implementations6 Dec 2021 Jiading Fang, Igor Vasiljevic, Vitor Guizilini, Rares Ambrus, Greg Shakhnarovich, Adrien Gaidon, Matthew R. Walter

Camera calibration is integral to robotics and computer vision algorithms that seek to infer geometric properties of the scene from visual input streams.

Autonomous Vehicles Camera Calibration +2

Self-supervised Learning is More Robust to Dataset Imbalance

1 code implementation ICLR 2022 Hong Liu, Jeff Z. HaoChen, Adrien Gaidon, Tengyu Ma

Third, inspired by the theoretical insights, we devise a re-weighted regularization technique that consistently improves the SSL representation quality on imbalanced datasets with several evaluation criteria, closing the small gap between balanced and imbalanced datasets with the same number of examples.

Long-tail Learning Self-Supervised Learning

Is Pseudo-Lidar needed for Monocular 3D Object detection?

2 code implementations ICCV 2021 Dennis Park, Rares Ambrus, Vitor Guizilini, Jie Li, Adrien Gaidon

Recent progress in 3D object detection from single images leverages monocular depth estimation as a way to produce 3D pointclouds, turning cameras into pseudo-lidar sensors.

 Ranked #1 on Monocular 3D Object Detection on KITTI Pedestrian Hard (using extra training data)

Monocular 3D Object Detection Monocular Depth Estimation +2

Hierarchical Lovasz Embeddings for Proposal-Free Panoptic Segmentation

no code implementations CVPR 2021 Tommi Kerola, Jie Li, Atsushi Kanehira, Yasunori Kudo, Alexis Vallet, Adrien Gaidon

We use a hierarchical Lovasz hinge loss to learn a low-dimensional embedding space structured into a unified semantic and instance hierarchy without requiring separate network branches or object proposals.

Instance Segmentation Panoptic Segmentation

Hierarchical Lovász Embeddings for Proposal-free Panoptic Segmentation

no code implementations8 Jun 2021 Tommi Kerola, Jie Li, Atsushi Kanehira, Yasunori Kudo, Alexis Vallet, Adrien Gaidon

We use a hierarchical Lov\'asz hinge loss to learn a low-dimensional embedding space structured into a unified semantic and instance hierarchy without requiring separate network branches or object proposals.

Instance Segmentation Panoptic Segmentation

Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss

1 code implementation NeurIPS 2021 Jeff Z. HaoChen, Colin Wei, Adrien Gaidon, Tengyu Ma

Despite the empirical successes, theoretical foundations are limited -- prior analyses assume conditional independence of the positive pairs given the same class label, but recent empirical applications use heavily correlated positive pairs (i. e., data augmentations of the same image).

Contrastive Learning Generalization Bounds +1

Full Surround Monodepth from Multiple Cameras

no code implementations31 Mar 2021 Vitor Guizilini, Igor Vasiljevic, Rares Ambrus, Greg Shakhnarovich, Adrien Gaidon

In this work, we extend monocular self-supervised depth and ego-motion estimation to large-baseline multi-camera rigs.

Autonomous Driving Motion Estimation

Geometric Unsupervised Domain Adaptation for Semantic Segmentation

no code implementations ICCV 2021 Vitor Guizilini, Jie Li, Rares Ambrus, Adrien Gaidon

Simulators can efficiently generate large amounts of labeled synthetic data with perfect supervision for hard-to-label tasks like semantic segmentation.

Depth Prediction Monocular Depth Estimation +2

Learning to Track with Object Permanence

1 code implementation ICCV 2021 Pavel Tokmakov, Jie Li, Wolfram Burgard, Adrien Gaidon

In this work, we introduce an end-to-end trainable approach for joint object detection and tracking that is capable of such reasoning.

Multi-Object Tracking object-detection +2

Monocular Depth Estimation for Soft Visuotactile Sensors

no code implementations5 Jan 2021 Rares Ambrus, Vitor Guizilini, Naveen Kuppuswamy, Andrew Beaulieu, Adrien Gaidon, Alex Alspach

Fluid-filled soft visuotactile sensors such as the Soft-bubbles alleviate key challenges for robust manipulation, as they enable reliable grasps along with the ability to obtain high-resolution sensory feedback on contact geometry and forces.

Monocular Depth Estimation

Risk-Sensitive Sequential Action Control with Multi-Modal Human Trajectory Forecasting for Safe Crowd-Robot Interaction

no code implementations12 Sep 2020 Haruki Nishimura, Boris Ivanovic, Adrien Gaidon, Marco Pavone, Mac Schwager

This paper presents a novel online framework for safe crowd-robot interaction based on risk-sensitive stochastic optimal control, wherein the risk is modeled by the entropic risk measure.

Trajectory Forecasting

Driving Through Ghosts: Behavioral Cloning with False Positives

no code implementations29 Aug 2020 Andreas Bühler, Adrien Gaidon, Andrei Cramariuc, Rares Ambrus, Guy Rosman, Wolfram Burgard

In this work, we propose a behavioral cloning approach that can safely leverage imperfect perception without being conservative.

Autonomous Driving

Neural Ray Surfaces for Self-Supervised Learning of Depth and Ego-motion

1 code implementation15 Aug 2020 Igor Vasiljevic, Vitor Guizilini, Rares Ambrus, Sudeep Pillai, Wolfram Burgard, Greg Shakhnarovich, Adrien Gaidon

Self-supervised learning has emerged as a powerful tool for depth and ego-motion estimation, leading to state-of-the-art results on benchmark datasets.

Depth Estimation Motion Estimation +2

PillarFlow: End-to-end Birds-eye-view Flow Estimation for Autonomous Driving

no code implementations3 Aug 2020 Kuan-Hui Lee, Matthew Kliemann, Adrien Gaidon, Jie Li, Chao Fang, Sudeep Pillai, Wolfram Burgard

In autonomous driving, accurately estimating the state of surrounding obstacles is critical for safe and robust path planning.

Autonomous Driving

Reinforcement Learning based Control of Imitative Policies for Near-Accident Driving

1 code implementation1 Jul 2020 Zhangjie Cao, Erdem Biyik, Woodrow Z. Wang, Allan Raventos, Adrien Gaidon, Guy Rosman, Dorsa Sadigh

To address driving in near-accident scenarios, we propose a hierarchical reinforcement and imitation learning (H-ReIL) approach that consists of low-level policies learned by IL for discrete driving modes, and a high-level policy learned by RL that switches between different driving modes.

Autonomous Driving Imitation Learning +2

Heteroskedastic and Imbalanced Deep Learning with Adaptive Regularization

1 code implementation ICLR 2021 Kaidi Cao, Yining Chen, Junwei Lu, Nikos Arechiga, Adrien Gaidon, Tengyu Ma

Real-world large-scale datasets are heteroskedastic and imbalanced -- labels have varying levels of uncertainty and label distributions are long-tailed.

Image Classification

Differentiable Rendering: A Survey

no code implementations22 Jun 2020 Hiroharu Kato, Deniz Beker, Mihai Morariu, Takahiro Ando, Toru Matsuoka, Wadim Kehl, Adrien Gaidon

Deep neural networks (DNNs) have shown remarkable performance improvements on vision-related tasks such as object detection or image segmentation.

Image Segmentation object-detection +2

Semantically-Guided Representation Learning for Self-Supervised Monocular Depth

1 code implementation ICLR 2020 Vitor Guizilini, Rui Hou, Jie Li, Rares Ambrus, Adrien Gaidon

Instead of using semantic labels and proxy losses in a multi-task approach, we propose a new architecture leveraging fixed pretrained semantic segmentation networks to guide self-supervised representation learning via pixel-adaptive convolutions.

Depth Prediction Monocular Depth Estimation +3

Spatiotemporal Relationship Reasoning for Pedestrian Intent Prediction

1 code implementation20 Feb 2020 Bingbin Liu, Ehsan Adeli, Zhangjie Cao, Kuan-Hui Lee, Abhijeet Shenoi, Adrien Gaidon, Juan Carlos Niebles

In addition, we introduce a new dataset designed specifically for autonomous-driving scenarios in areas with dense pedestrian populations: the Stanford-TRI Intent Prediction (STIP) dataset.

Autonomous Driving Navigate

Real-Time Panoptic Segmentation from Dense Detections

no code implementations CVPR 2020 Rui Hou, Jie Li, Arjun Bhargava, Allan Raventos, Vitor Guizilini, Chao Fang, Jerome Lynch, Adrien Gaidon

Panoptic segmentation is a complex full scene parsing task requiring simultaneous instance and semantic segmentation at high resolution.

Clustering object-detection +3

Autolabeling 3D Objects with Differentiable Rendering of SDF Shape Priors

1 code implementation CVPR 2020 Sergey Zakharov, Wadim Kehl, Arjun Bhargava, Adrien Gaidon

We present an automatic annotation pipeline to recover 9D cuboids and 3D shapes from pre-trained off-the-shelf 2D detectors and sparse LIDAR data.

Disentangling Human Dynamics for Pedestrian Locomotion Forecasting with Noisy Supervision

no code implementations4 Nov 2019 Karttikeya Mangalam, Ehsan Adeli, Kuan-Hui Lee, Adrien Gaidon, Juan Carlos Niebles

In contrast to the previous work that aims to solve either the task of pose prediction or trajectory forecasting in isolation, we propose a framework to unify the two problems and address the practically useful task of pedestrian locomotion prediction in the wild.

Human Dynamics Pose Prediction +1

Generating Human Action Videos by Coupling 3D Game Engines and Probabilistic Graphical Models

no code implementations12 Oct 2019 César Roberto de Souza, Adrien Gaidon, Yohann Cabon, Naila Murray, Antonio Manuel López

With this model we generate a diverse, realistic, and physically plausible dataset of human action videos, called PHAV for "Procedural Human Action Videos".

Action Recognition Optical Flow Estimation +3

Two Stream Networks for Self-Supervised Ego-Motion Estimation

no code implementations4 Oct 2019 Rares Ambrus, Vitor Guizilini, Jie Li, Sudeep Pillai, Adrien Gaidon

Learning depth and camera ego-motion from raw unlabeled RGB video streams is seeing exciting progress through self-supervision from strong geometric cues.

Data Augmentation Motion Estimation +2

Robust Semi-Supervised Monocular Depth Estimation with Reprojected Distances

no code implementations4 Oct 2019 Vitor Guizilini, Jie Li, Rares Ambrus, Sudeep Pillai, Adrien Gaidon

Dense depth estimation from a single image is a key problem in computer vision, with exciting applications in a multitude of robotic tasks.

Monocular Depth Estimation

Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss

6 code implementations NeurIPS 2019 Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, Tengyu Ma

Deep learning algorithms can fare poorly when the training dataset suffers from heavy class-imbalance but the testing criterion requires good generalization on less frequent classes.

Long-tail learning with class descriptors

An Attention-based Recurrent Convolutional Network for Vehicle Taillight Recognition

no code implementations9 Jun 2019 Kuan-Hui Lee, Takaaki Tagawa, Jia-En M. Pan, Adrien Gaidon, Bertrand Douillard

Vehicle taillight recognition is an important application for automated driving, especially for intent prediction of ado vehicles and trajectory planning of the ego vehicle.

Trajectory Planning

Learning to Fuse Things and Stuff

no code implementations4 Dec 2018 Jie Li, Allan Raventos, Arjun Bhargava, Takaaki Tagawa, Adrien Gaidon

We propose an end-to-end learning approach for panoptic segmentation, a novel task unifying instance (things) and semantic (stuff) segmentation.

Ranked #26 on Panoptic Segmentation on Cityscapes val (using extra training data)

Instance Segmentation Panoptic Segmentation

SPIGAN: Privileged Adversarial Learning from Simulation

no code implementations ICLR 2019 Kuan-Hui Lee, German Ros, Jie Li, Adrien Gaidon

Deep Learning for Computer Vision depends mainly on the source of supervision. Photo-realistic simulators can generate large-scale automatically labeled syntheticdata, but introduce a domain gap negatively impacting performance.

Image-to-Image Translation Semantic Segmentation +1

SuperDepth: Self-Supervised, Super-Resolved Monocular Depth Estimation

no code implementations3 Oct 2018 Sudeep Pillai, Rares Ambrus, Adrien Gaidon

Both contributions provide significant performance gains over the state-of-the-art in self-supervised depth and pose estimation on the public KITTI benchmark.

Depth Prediction Image Super-Resolution +2

Procedural Generation of Videos to Train Deep Action Recognition Networks

no code implementations CVPR 2017 César Roberto de Souza, Adrien Gaidon, Yohann Cabon, Antonio Manuel López Peña

Deep learning for human action recognition in videos is making significant progress, but is slowed down by its dependency on expensive manual labeling of large video collections.

Action Recognition In Videos Representation Learning +1

Sympathy for the Details: Dense Trajectories and Hybrid Classification Architectures for Action Recognition

no code implementations25 Aug 2016 César Roberto de Souza, Adrien Gaidon, Eleonora Vig, Antonio Manuel López

Action recognition in videos is a challenging task due to the complexity of the spatio-temporal patterns to model and the difficulty to acquire and learn on large quantities of video data.

Action Recognition In Videos General Classification +3

Virtual Worlds as Proxy for Multi-Object Tracking Analysis

no code implementations CVPR 2016 Adrien Gaidon, Qiao Wang, Yohann Cabon, Eleonora Vig

We provide quantitative experimental evidence suggesting that (i) modern deep learning algorithms pre-trained on real data behave similarly in real and virtual worlds, and (ii) pre-training on virtual data improves performance.

Instance Segmentation Multi-Object Tracking +4

Online Domain Adaptation for Multi-Object Tracking

no code implementations4 Aug 2015 Adrien Gaidon, Eleonora Vig

We quantitatively measure the benefit of our domain adaptation strategy on the KITTI tracking benchmark and on a new dataset (PASCAL-to-KITTI) we introduce to study the domain mismatch problem in MOT.

Multi-Object Tracking Multi-Task Learning +1

Deep Fishing: Gradient Features from Deep Nets

no code implementations23 Jul 2015 Albert Gordo, Adrien Gaidon, Florent Perronnin

Convolutional Networks (ConvNets) have recently improved image recognition performance thanks to end-to-end learning of deep feed-forward models from raw pixels.

Online Learning to Sample

no code implementations30 Jun 2015 Guillaume Bouchard, Théo Trouillon, Julien Perez, Adrien Gaidon

Stochastic Gradient Descent (SGD) is one of the most widely used techniques for online optimization in machine learning.

Image Classification

Self-Learning Camera: Autonomous Adaptation of Object Detectors to Unlabeled Video Streams

no code implementations17 Jun 2014 Adrien Gaidon, Gloria Zen, Jose A. Rodriguez-Serrano

In this paper, we address the problem of self-learning detectors in an autonomous manner, i. e. (i) detectors continuously updating themselves to efficiently adapt to streaming data sources (contrary to transductive algorithms), (ii) without any labeled data strongly related to the target data stream (contrary to self-paced learning), and (iii) without manual intervention to set and update hyper-parameters.

Multi-Task Learning Self-Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.