no code implementations • CVPR 2022 • Shyamal Buch, Cristóbal Eyzaguirre, Adrien Gaidon, Jiajun Wu, Li Fei-Fei, Juan Carlos Niebles
Building on recent progress in self-supervised image-language models, we revisit this question in the context of video and language tasks.
no code implementations • 28 Apr 2022 • Rowan Mcallister, Blake Wulfe, Jean Mercat, Logan Ellis, Sergey Levine, Adrien Gaidon
Autonomous vehicle software is typically structured as a modular pipeline of individual components (e. g., perception, prediction, and planning) to help separate concerns into interpretable sub-tasks.
no code implementations • CVPR 2022 • Vitor Guizilini, Rares Ambrus, Dian Chen, Sergey Zakharov, Adrien Gaidon
Experiments on the KITTI and DDAD datasets show that our DepthFormer architecture establishes a new state of the art in self-supervised monocular depth estimation, and is even competitive with highly specialized supervised single-frame architectures.
1 code implementation • 4 Apr 2022 • Pavel Tokmakov, Allan Jabri, Jie Li, Adrien Gaidon
This paper proposes a self-supervised objective for learning representations that localize objects under occlusion - a property known as object permanence.
no code implementations • 28 Mar 2022 • Vitor Guizilini, Kuan-Hui Lee, Rares Ambrus, Adrien Gaidon
However, the simultaneous self-supervised learning of depth and scene flow is ill-posed, as there are infinitely many combinations that result in the same 3D point.
1 code implementation • CVPR 2022 • Zhipeng Bao, Pavel Tokmakov, Allan Jabri, Yu-Xiong Wang, Adrien Gaidon, Martial Hebert
Our experiments demonstrate that, despite only capturing a small subset of the objects that move, this signal is enough to generalize to segment both moving and static instances of dynamic objects.
no code implementations • ICLR 2022 • Blake Wulfe, Ashwin Balakrishna, Logan Ellis, Jean Mercat, Rowan Mcallister, Adrien Gaidon
The ability to learn reward functions plays an important role in enabling the deployment of intelligent agents in the real world.
no code implementations • 6 Dec 2021 • Jiading Fang, Igor Vasiljevic, Vitor Guizilini, Rares Ambrus, Greg Shakhnarovich, Adrien Gaidon, Matthew R. Walter
Camera calibration is integral to robotics and computer vision algorithms that seek to infer geometric properties of the scene from visual input streams.
1 code implementation • ICLR 2022 • Hong Liu, Jeff Z. HaoChen, Adrien Gaidon, Tengyu Ma
Third, inspired by the theoretical insights, we devise a re-weighted regularization technique that consistently improves the SSL representation quality on imbalanced datasets with several evaluation criteria, closing the small gap between balanced and imbalanced datasets with the same number of examples.
no code implementations • ICCV 2021 • Aditya Ganeshan, Alexis Vallet, Yasunori Kudo, Shin-ichi Maeda, Tommi Kerola, Rares Ambrus, Dennis Park, Adrien Gaidon
Deep learning models for semantic segmentation rely on expensive, large-scale, manually annotated datasets.
Ranked #13 on
Semantic Segmentation
on NYU Depth v2
1 code implementation • ICCV 2021 • Dennis Park, Rares Ambrus, Vitor Guizilini, Jie Li, Adrien Gaidon
Recent progress in 3D object detection from single images leverages monocular depth estimation as a way to produce 3D pointclouds, turning cameras into pseudo-lidar sensors.
Ranked #1 on
Monocular 3D Object Detection
on KITTI Pedestrian Hard
(using extra training data)
no code implementations • CVPR 2021 • Tommi Kerola, Jie Li, Atsushi Kanehira, Yasunori Kudo, Alexis Vallet, Adrien Gaidon
We use a hierarchical Lovasz hinge loss to learn a low-dimensional embedding space structured into a unified semantic and instance hierarchy without requiring separate network branches or object proposals.
no code implementations • NeurIPS 2021 • Jeff Z. HaoChen, Colin Wei, Adrien Gaidon, Tengyu Ma
Despite the empirical successes, theoretical foundations are limited -- prior analyses assume conditional independence of the positive pairs given the same class label, but recent empirical applications use heavily correlated positive pairs (i. e., data augmentations of the same image).
no code implementations • 8 Jun 2021 • Tommi Kerola, Jie Li, Atsushi Kanehira, Yasunori Kudo, Alexis Vallet, Adrien Gaidon
We use a hierarchical Lov\'asz hinge loss to learn a low-dimensional embedding space structured into a unified semantic and instance hierarchy without requiring separate network branches or object proposals.
1 code implementation • 30 Apr 2021 • Nishant Rai, Ehsan Adeli, Kuan-Hui Lee, Adrien Gaidon, Juan Carlos Niebles
Labeling videos at scale is impractical.
1 code implementation • 26 Apr 2021 • Boris Ivanovic, Kuan-Hui Lee, Pavel Tokmakov, Blake Wulfe, Rowan Mcallister, Adrien Gaidon, Marco Pavone
Reasoning about the future behavior of other agents is critical to safe robot navigation.
no code implementations • 31 Mar 2021 • Vitor Guizilini, Igor Vasiljevic, Rares Ambrus, Greg Shakhnarovich, Adrien Gaidon
In this work, we extend monocular self-supervised depth and ego-motion estimation to large-baseline multi-camera rigs.
1 code implementation • CVPR 2021 • Vitor Guizilini, Rares Ambrus, Wolfram Burgard, Adrien Gaidon
Estimating scene geometry from data obtained with cost-effective sensors is key for robots and self-driving cars.
no code implementations • ICCV 2021 • Vitor Guizilini, Jie Li, Rares Ambrus, Adrien Gaidon
Simulators can efficiently generate large amounts of labeled synthetic data with perfect supervision for hard-to-label tasks like semantic segmentation.
no code implementations • 29 Mar 2021 • Sharada Mohanty, Jyotish Poonganam, Adrien Gaidon, Andrey Kolobov, Blake Wulfe, Dipam Chakraborty, Gražvydas Šemetulskis, João Schapke, Jonas Kubilius, Jurgis Pašukonis, Linas Klimas, Matthew Hausknecht, Patrick MacAlpine, Quang Nhat Tran, Thomas Tumiel, Xiaocheng Tang, Xinwei Chen, Christopher Hesse, Jacob Hilton, William Hebgen Guss, Sahika Genc, John Schulman, Karl Cobbe
We present the design of a centralized benchmark for Reinforcement Learning which can help measure Sample Efficiency and Generalization in Reinforcement Learning by doing end to end evaluation of the training and rollout phases of thousands of user submitted code bases in a scalable way.
1 code implementation • ICCV 2021 • Pavel Tokmakov, Jie Li, Wolfram Burgard, Adrien Gaidon
In this work, we introduce an end-to-end trainable approach for joint object detection and tracking that is capable of such reasoning.
no code implementations • 5 Jan 2021 • Rares Ambrus, Vitor Guizilini, Naveen Kuppuswamy, Andrew Beaulieu, Adrien Gaidon, Alex Alspach
Fluid-filled soft visuotactile sensors such as the Soft-bubbles alleviate key challenges for robust manipulation, as they enable reliable grasps along with the ability to obtain high-resolution sensory feedback on contact geometry and forces.
no code implementations • 1 Jan 2021 • Masanori Koyama, Toshiki Nakanishi, Shin-ichi Maeda, Vitor Campagnolo Guizilini, Adrien Gaidon
Estimating the 3D shape of real-world objects is a key perceptual challenge.
no code implementations • 24 Nov 2020 • Daisuke Nishiyama, Mario Ynocente Castro, Shirou Maruyama, Shinya Shiroshita, Karim Hamzaoui, Yi Ouyang, Guy Rosman, Jonathan DeCastro, Kuan-Hui Lee, Adrien Gaidon
Automated Vehicles require exhaustive testing in simulation to detect as many safety-critical failures as possible before deployment on public roads.
no code implementations • 11 Nov 2020 • Shinya Shiroshita, Shirou Maruyama, Daisuke Nishiyama, Mario Ynocente Castro, Karim Hamzaoui, Guy Rosman, Jonathan DeCastro, Kuan-Hui Lee, Adrien Gaidon
Traffic simulators are important tools in autonomous driving development.
no code implementations • ECCV 2020 • Deniz Beker, Hiroharu Kato, Mihai Adrian Morariu, Takahiro Ando, Toru Matsuoka, Wadim Kehl, Adrien Gaidon
3D object detection from monocular images is an ill-posed problem due to the projective entanglement of depth and scale.
3D Object Detection
3D Object Detection From Monocular Images
+4
1 code implementation • 16 Sep 2020 • Boris Ivanovic, Amine Elhafsi, Guy Rosman, Adrien Gaidon, Marco Pavone
Reasoning about human motion is a core component of modern human-robot interactive systems.
no code implementations • 12 Sep 2020 • Haruki Nishimura, Boris Ivanovic, Adrien Gaidon, Marco Pavone, Mac Schwager
This paper presents a novel online framework for safe crowd-robot interaction based on risk-sensitive stochastic optimal control, wherein the risk is modeled by the entropic risk measure.
no code implementations • 29 Aug 2020 • Andreas Bühler, Adrien Gaidon, Andrei Cramariuc, Rares Ambrus, Guy Rosman, Wolfram Burgard
In this work, we propose a behavioral cloning approach that can safely leverage imperfect perception without being conservative.
1 code implementation • 15 Aug 2020 • Igor Vasiljevic, Vitor Guizilini, Rares Ambrus, Sudeep Pillai, Wolfram Burgard, Greg Shakhnarovich, Adrien Gaidon
Self-supervised learning has emerged as a powerful tool for depth and ego-motion estimation, leading to state-of-the-art results on benchmark datasets.
no code implementations • 3 Aug 2020 • Kuan-Hui Lee, Matthew Kliemann, Adrien Gaidon, Jie Li, Chao Fang, Sudeep Pillai, Wolfram Burgard
In autonomous driving, accurately estimating the state of surrounding obstacles is critical for safe and robust path planning.
1 code implementation • 1 Jul 2020 • Zhangjie Cao, Erdem Biyik, Woodrow Z. Wang, Allan Raventos, Adrien Gaidon, Guy Rosman, Dorsa Sadigh
To address driving in near-accident scenarios, we propose a hierarchical reinforcement and imitation learning (H-ReIL) approach that consists of low-level policies learned by IL for discrete driving modes, and a high-level policy learned by RL that switches between different driving modes.
1 code implementation • ICLR 2021 • Kaidi Cao, Yining Chen, Junwei Lu, Nikos Arechiga, Adrien Gaidon, Tengyu Ma
Real-world large-scale datasets are heteroskedastic and imbalanced -- labels have varying levels of uncertainty and label distributions are long-tailed.
Ranked #8 on
Image Classification
on WebVision-1000
no code implementations • 22 Jun 2020 • Hiroharu Kato, Deniz Beker, Mihai Morariu, Takahiro Ando, Toru Matsuoka, Wadim Kehl, Adrien Gaidon
Deep neural networks (DNNs) have shown remarkable performance improvements on vision-related tasks such as object detection or image segmentation.
3 code implementations • ECCV 2020 • Karttikeya Mangalam, Harshayu Girase, Shreyas Agarwal, Kuan-Hui Lee, Ehsan Adeli, Jitendra Malik, Adrien Gaidon
In this work, we present Predicted Endpoint Conditioned Network (PECNet) for flexible human trajectory prediction.
Ranked #1 on
Multi Future Trajectory Prediction
on ETH/UCY
no code implementations • CVPR 2020 • Boxiao Pan, Haoye Cai, De-An Huang, Kuan-Hui Lee, Adrien Gaidon, Ehsan Adeli, Juan Carlos Niebles
In this paper, we propose a novel spatio-temporal graph model for video captioning that exploits object interactions in space and time.
1 code implementation • ICLR 2020 • Vitor Guizilini, Rui Hou, Jie Li, Rares Ambrus, Adrien Gaidon
Instead of using semantic labels and proxy losses in a multi-task approach, we propose a new architecture leveraging fixed pretrained semantic segmentation networks to guide self-supervised representation learning via pixel-adaptive convolutions.
1 code implementation • 20 Feb 2020 • Bingbin Liu, Ehsan Adeli, Zhangjie Cao, Kuan-Hui Lee, Abhijeet Shenoi, Adrien Gaidon, Juan Carlos Niebles
In addition, we introduce a new dataset designed specifically for autonomous-driving scenarios in areas with dense pedestrian populations: the Stanford-TRI Intent Prediction (STIP) dataset.
1 code implementation • 7 Dec 2019 • Jiexiong Tang, Rares Ambrus, Vitor Guizilini, Sudeep Pillai, Hanme Kim, Patric Jensfelt, Adrien Gaidon
Detecting and matching robust viewpoint-invariant keypoints is critical for visual SLAM and Structure-from-Motion.
no code implementations • CVPR 2020 • Rui Hou, Jie Li, Arjun Bhargava, Allan Raventos, Vitor Guizilini, Chao Fang, Jerome Lynch, Adrien Gaidon
Panoptic segmentation is a complex full scene parsing task requiring simultaneous instance and semantic segmentation at high resolution.
1 code implementation • CVPR 2020 • Sergey Zakharov, Wadim Kehl, Arjun Bhargava, Adrien Gaidon
We present an automatic annotation pipeline to recover 9D cuboids and 3D shapes from pre-trained off-the-shelf 2D detectors and sparse LIDAR data.
no code implementations • 4 Nov 2019 • Karttikeya Mangalam, Ehsan Adeli, Kuan-Hui Lee, Adrien Gaidon, Juan Carlos Niebles
In contrast to the previous work that aims to solve either the task of pose prediction or trajectory forecasting in isolation, we propose a framework to unify the two problems and address the practically useful task of pedestrian locomotion prediction in the wild.
no code implementations • 12 Oct 2019 • César Roberto de Souza, Adrien Gaidon, Yohann Cabon, Naila Murray, Antonio Manuel López
With this model we generate a diverse, realistic, and physically plausible dataset of human action videos, called PHAV for "Procedural Human Action Videos".
no code implementations • 4 Oct 2019 • Rares Ambrus, Vitor Guizilini, Jie Li, Sudeep Pillai, Adrien Gaidon
Learning depth and camera ego-motion from raw unlabeled RGB video streams is seeing exciting progress through self-supervision from strong geometric cues.
no code implementations • 4 Oct 2019 • Vitor Guizilini, Jie Li, Rares Ambrus, Sudeep Pillai, Adrien Gaidon
Dense depth estimation from a single image is a key problem in computer vision, with exciting applications in a multitude of robotic tasks.
6 code implementations • NeurIPS 2019 • Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, Tengyu Ma
Deep learning algorithms can fare poorly when the training dataset suffers from heavy class-imbalance but the testing criterion requires good generalization on less frequent classes.
Ranked #4 on
Long-tail Learning
on CIFAR-10-LT (ρ=10)
no code implementations • 9 Jun 2019 • Kuan-Hui Lee, Takaaki Tagawa, Jia-En M. Pan, Adrien Gaidon, Bertrand Douillard
Vehicle taillight recognition is an important application for automated driving, especially for intent prediction of ado vehicles and trajectory planning of the ego vehicle.
3 code implementations • CVPR 2020 • Vitor Guizilini, Rares Ambrus, Sudeep Pillai, Allan Raventos, Adrien Gaidon
Although cameras are ubiquitous, robotic platforms typically rely on active sensors like LiDAR for direct 3D perception.
2 code implementations • ICCV 2019 • Felipe Codevilla, Eder Santana, Antonio M. López, Adrien Gaidon
Driving requires reacting to a wide variety of complex environment conditions and agent behaviors.
Ranked #12 on
Autonomous Driving
on CARLA Leaderboard
no code implementations • CVPR 2019 • Fabian Manhardt, Wadim Kehl, Adrien Gaidon
We present a deep learning method for end-to-end monocular 3D object detection and metric shape retrieval.
no code implementations • 4 Dec 2018 • Jie Li, Allan Raventos, Arjun Bhargava, Takaaki Tagawa, Adrien Gaidon
We propose an end-to-end learning approach for panoptic segmentation, a novel task unifying instance (things) and semantic (stuff) segmentation.
Ranked #13 on
Panoptic Segmentation
on Cityscapes val
(using extra training data)
no code implementations • ICLR 2019 • Kuan-Hui Lee, German Ros, Jie Li, Adrien Gaidon
Deep Learning for Computer Vision depends mainly on the source of supervision. Photo-realistic simulators can generate large-scale automatically labeled syntheticdata, but introduce a domain gap negatively impacting performance.
no code implementations • 3 Oct 2018 • Sudeep Pillai, Rares Ambrus, Adrien Gaidon
Both contributions provide significant performance gains over the state-of-the-art in self-supervised depth and pose estimation on the public KITTI benchmark.
no code implementations • CVPR 2017 • César Roberto de Souza, Adrien Gaidon, Yohann Cabon, Antonio Manuel López Peña
Deep learning for human action recognition in videos is making significant progress, but is slowed down by its dependency on expensive manual labeling of large video collections.
no code implementations • 25 Aug 2016 • César Roberto de Souza, Adrien Gaidon, Eleonora Vig, Antonio Manuel López
Action recognition in videos is a challenging task due to the complexity of the spatio-temporal patterns to model and the difficulty to acquire and learn on large quantities of video data.
no code implementations • CVPR 2016 • Adrien Gaidon, Qiao Wang, Yohann Cabon, Eleonora Vig
We provide quantitative experimental evidence suggesting that (i) modern deep learning algorithms pre-trained on real data behave similarly in real and virtual worlds, and (ii) pre-training on virtual data improves performance.
no code implementations • 4 Aug 2015 • Adrien Gaidon, Eleonora Vig
We quantitatively measure the benefit of our domain adaptation strategy on the KITTI tracking benchmark and on a new dataset (PASCAL-to-KITTI) we introduce to study the domain mismatch problem in MOT.
no code implementations • 23 Jul 2015 • Albert Gordo, Adrien Gaidon, Florent Perronnin
Convolutional Networks (ConvNets) have recently improved image recognition performance thanks to end-to-end learning of deep feed-forward models from raw pixels.
no code implementations • 30 Jun 2015 • Guillaume Bouchard, Théo Trouillon, Julien Perez, Adrien Gaidon
Stochastic Gradient Descent (SGD) is one of the most widely used techniques for online optimization in machine learning.
no code implementations • 17 Jun 2014 • Adrien Gaidon, Gloria Zen, Jose A. Rodriguez-Serrano
In this paper, we address the problem of self-learning detectors in an autonomous manner, i. e. (i) detectors continuously updating themselves to efficiently adapt to streaming data sources (contrary to transductive algorithms), (ii) without any labeled data strongly related to the target data stream (contrary to self-paced learning), and (iii) without manual intervention to set and update hyper-parameters.