Search Results for author:
Found 90 papers, 43 papers with code
Nighttime Defogging Using High-Low Frequency Decomposition and Grayscale-Color Networks
Given an RGB foggy nighttime image, our grayscale module takes the grayscale version of the image as input, and decomposes it into high and low frequency layers.
TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction
We present TrafficBots, a multi-agent policy built upon motion prediction and end-to-end driving, and based on TrafficBots we obtain a world model tailored for the planning module of autonomous vehicles.
No One Left Behind: Real-World Federated Class-Incremental Learning
To tackle the above issues, we propose a novel Local-Global Anti-forgetting (LGA) model to address local and global catastrophic forgetting on old categories, which is a pioneering work to explore a global class-incremental model in the FL feld.
Benchmarking the Robustness of LiDAR Semantic Segmentation Models
In this paper, we aim to comprehensively analyze the robustness of LiDAR semantic segmentation models under various corruptions.
Simulating Road Spray Effects in Automotive Lidar Sensor Models
The model conforms to the Open Simulation Interface (OSI) standard and is based on the formation of detection clusters within a spray plume.
One-Shot Domain Adaptive and Generalizable Semantic Segmentation with Class-Aware Cross-Domain Transformers
Thus, we aim to relieve this need on a large number of real data, and explore the one-shot unsupervised sim-to-real domain adaptation (OSUDA) and generalization (OSDG) problem, where only one real-world data sample is available.
MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation
MIC significantly improves the state-of-the-art performance across the different recognition tasks for synthetic-to-real, day-to-nighttime, and clear-to-adverse-weather UDA.
Normalization Perturbation: A Simple Domain Generalization Method for Real-World Domain Shifts
Thus, we propose to perturb the channel statistics of source domain features to synthesize various latent styles, so that the trained deep model can perceive diverse potential domains and generalizes well even without observations of target domain data in training.
Motion Transformer with Global Intention Localization and Local Movement Refinement
Predicting multimodal future behavior of traffic participants is essential for robotic vehicles to make safe decisions.
MTR-A: 1st Place Solution for 2022 Waymo Open Dataset Challenge -- Motion Prediction
In this report, we present the 1st place solution for motion prediction track in 2022 Waymo Open Dataset Challenges.
Ret3D: Rethinking Object Relations for Efficient 3D Object Detection in Driving Scenes
Current efficient LiDAR-based detection frameworks are lacking in exploiting object relations, which naturally present in both spatial and temporal manners.
HRFuser: A Multi-resolution Sensor Fusion Architecture for 2D Object Detection
Besides standard cameras, autonomous vehicles typically include multiple additional sensors, such as lidars and radars, which help acquire richer information for perceiving the content of the driving scene.
Ranked #1 on
2D object detection
on Clear Weather
Deep Gradient Learning for Efficient Camouflaged Object Detection
This paper introduces DGNet, a novel deep framework that exploits object gradient supervision for camouflaged object detection (COD).
Bi-level Alignment for Cross-Domain Crowd Counting
The main challenge for this task is to achieve high-quality manual annotations on a large amount of training data.
HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation
Therefore, we propose HRDA, a multi-resolution training approach for UDA, that combines the strengths of small high-resolution crops to preserve fine segmentation details and large low-resolution crops to capture long-range context dependencies with a learned scale attention, while maintaining a manageable GPU memory footprint.
LiDAR Snowfall Simulation for Robust 3D Object Detection
Due to the difficulty of collecting and annotating training data in this setting, we propose a physically based method to simulate the effect of snowfall on real clear-weather LiDAR point clouds.
Ranked #1 on
3D Object Detection
on Heavy Snowfall
Continual Test-Time Domain Adaptation
However, real-world machine perception systems are running in non-stationary and continually changing environments where the target domain distribution can change over time.
Scribble-Supervised LiDAR Semantic Segmentation
Densely annotating LiDAR point clouds remains too expensive and time-consuming to keep up with the ever growing volume of data.
Ranked #1 on
3D Semantic Segmentation
on ScribbleKITTI
Pix2NeRF: Unsupervised Conditional $π$-GAN for Single Image to Neural Radiance Fields Translation
We propose a pipeline to generate Neural Radiance Fields~(NeRF) of an object or a scene of a specific class, conditioned on a single input image.
Adiabatic Quantum Computing for Multi Object Tracking
Multi-Object Tracking (MOT) is most often approached in the tracking-by-detection paradigm, where object detections are associated through time.
End-To-End Optimization of LiDAR Beam Configuration for 3D Object Detection and Localization
In this work, we take a new route to learn to optimize the LiDAR beam configuration for a given application.
Sound and Visual Representation Learning with Multiple Pretraining Tasks
Specifically, for this study, we investigate binaural sounds and image data in isolation.
Pix2NeRF: Unsupervised Conditional p-GAN for Single Image to Neural Radiance Fields Translation
We propose a pipeline to generate Neural Radiance Fields (NeRF) of an object or a scene of a specific class, conditioned on a single input image.
CoSSL: Co-Learning of Representation and Classifier for Imbalanced Semi-Supervised Learning
In this paper, we propose a novel co-learning framework (CoSSL) with decoupled representation learning and classifier learning for imbalanced SSL.
Both Style and Fog Matter: Cumulative Domain Adaptation for Semantic Foggy Scene Understanding
Unlike previous methods that mainly focus on closing the domain gap caused by fog -- defogging the foggy images or fogging the clear images, we propose to alleviate the domain gap by considering fog influence and style variation simultaneously.
Ranked #2 on
Domain Adaptation
on Cityscapes-to-FoggyZurich
DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation
It improves the state of the art by 10. 8 mIoU for GTA-to-Cityscapes and 5. 4 mIoU for Synthia-to-Cityscapes and enables learning even difficult classes such as train, bus, and truck well.
Ranked #5 on
Domain Adaptation
on Cityscapes to ACDC
TACS: Taxonomy Adaptive Cross-Domain Semantic Segmentation
In many real-world settings, the target domain task requires a different taxonomy than the one imposed by the source domain.
Binaural SoundNet: Predicting Semantics, Depth and Motion with Binaural Sounds
Humans can robustly recognize and localize objects by using visual and/or auditory cues.
Improving Semi-Supervised and Domain-Adaptive Semantic Segmentation with Self-Supervised Depth Estimation
Training deep networks for semantic segmentation requires large amounts of labeled training data, which presents a major challenge in practice, as labeling segmentation masks is a highly labor-intensive process.
End-to-End Urban Driving by Imitating a Reinforcement Learning Coach
Our end-to-end agent achieves a 78% success rate while generalizing to a new town and new weather on the NoCrash-dense benchmark and state-of-the-art performance on the challenging public routes of the CARLA LeaderBoard.
Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather
2) Through extensive experiments with several state-of-the-art detection approaches, we show that our fog simulation can be leveraged to significantly improve the performance for 3D object detection in the presence of fog.
Ranked #1 on
3D Object Detection
on Dense Fog
Self-Aligned Video Deraining With Transmission-Depth Consistency
In this paper, we address the problems of rain streaks and rain accumulation removal in video, by developing a self-aligned network with transmission-depth consistency.
Domain Adaptive Semantic Segmentation with Self-Supervised Depth Estimation
However, such a supervision is not always available.
Ranked #11 on
Domain Adaptation
on SYNTHIA-to-Cityscapes
(using extra training data)
ACDC: The Adverse Conditions Dataset with Correspondences for Semantic Driving Scene Understanding
To address this, we introduce ACDC, the Adverse Conditions Dataset with Correspondences for training and testing semantic segmentation methods on adverse visual conditions.
Learnable Online Graph Representations for 3D Multi-Object Tracking
Tracking of objects in 3D is a fundamental task in computer vision that finds use in a wide range of applications such as autonomous driving, robotics or augmented reality.
Spectral Tensor Train Parameterization of Deep Learning Layers
We study low-rank parameterizations of weight matrices with embedded spectral properties in the Deep Learning context.
Hyperspectral Image Super-Resolution with Spectral Mixup and Heterogeneous Datasets
With these contributions, our method is able to learn from heterogeneous datasets and lift the requirement for having a large amount of HD HSI training samples.
Task Switching Network for Multi-Task Learning
Multiple tasks are performed by switching between them, performing one task at a time.
Three Ways to Improve Semantic Segmentation with Self-Supervised Depth Estimation
Training deep networks for semantic segmentation requires large amounts of labeled training data, which presents a major challenge in practice, as labeling segmentation masks is a highly labor-intensive process.
Ranked #2 on
Semi-Supervised Semantic Segmentation
on Cityscapes 100 samples labeled
(using extra training data)
Cluster, Split, Fuse, and Update: Meta-Learning for Open Compound Domain Adaptive Semantic Segmentation
Open compound domain adaptation (OCDA) is a domain adaptation setting, where target domain is modeled as a compound of multiple unknown homogeneous domains, which brings the advantage of improved generalization to unseen domains.
mDALU: Multi-Source Domain Adaptation and Label Unification with Partial Datasets
One challenge of object recognition is to generalize to new domains, to more classes and/or to new modalities.
Depth Estimation from Monocular Images and Sparse Radar Data
We give a comprehensive study of the fusion between RGB images and Radar measurements from different aspects and proposed a working solution based on the observations.
Improving Point Cloud Semantic Segmentation by Learning 3D Object Detection
Point cloud semantic segmentation plays an essential role in autonomous driving, providing vital information about drivable surfaces and nearby objects that can aid higher level tasks such as path planning and collision avoidance.
Multi-scale Interaction for Real-time LiDAR Data Segmentation on an Embedded Platform
Real-time semantic segmentation of LiDAR data is crucial for autonomously driving vehicles, which are usually equipped with an embedded platform and have limited computational resources.
Ranked #2 on
Real-Time 3D Semantic Segmentation
on SemanticKITTI
Weakly Supervised 3D Object Detection from Lidar Point Cloud
This work proposes a weakly supervised approach for 3D object detection, only requiring a small set of weakly annotated scenes, associated with a few precisely labeled object instances.
Off-Policy Reinforcement Learning for Efficient and Effective GAN Architecture Search
In this paper, we introduce a new reinforcement learning (RL) based neural architecture search (NAS) methodology for effective and efficient generative adversarial network (GAN) architecture search.
Ranked #12 on
Image Generation
on STL-10
T-Basis: a Compact Representation for Neural Networks
Each of the tensors in the set is modeled using Tensor Rings, though the concept applies to other Tensor Networks.
Learning Accurate and Human-Like Driving using Semantic Maps and Attention
This paper investigates how end-to-end driving models can be improved to drive more accurately and human-like.
Analogical Image Translation for Fog Generation
AIT achieves this zero-shot image translation capability by coupling a supervised training scheme in the synthetic domain, a cycle consistency strategy in the real domain, an adversarial training scheme between the two domains, and a novel network design.
Map-Guided Curriculum Domain Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation
Our central contributions are: 1) a curriculum framework to gradually adapt semantic segmentation models from day to night through progressively darker times of day, exploiting cross-time-of-day correspondences between daytime images from a reference map and dark images to guide the label inference in the dark domains; 2) a novel uncertainty-aware annotation and evaluation framework and metric for semantic segmentation, including image regions beyond human recognition capability in the evaluation in a principled fashion; 3) the Dark Zurich dataset, comprising 2416 unlabeled nighttime and 2920 unlabeled twilight images with correspondences to their daytime counterparts plus a set of 201 nighttime images with fine pixel-level annotations created with our protocol, which serves as a first benchmark for our novel evaluation.
Ranked #3 on
Semantic Segmentation
on Nighttime Driving
Action Sequence Predictions of Vehicles in Urban Environments using Map and Social Context
Our second contribution lies in applying the method to the well-known traffic agent tracking and prediction dataset Argoverse, resulting in 228, 000 action sequences.
Multi-Task Learning for Dense Prediction Tasks: A Survey
In this survey, we provide a well-rounded view on state-of-the-art deep learning approaches for MTL in computer vision, explicitly emphasizing on dense prediction tasks.
Quantifying Data Augmentation for LiDAR based 3D Object Detection
In this work, we shed light on different data augmentation techniques commonly used in Light Detection and Ranging (LiDAR) based 3D Object Detection.
Semantic Object Prediction and Spatial Sound Super-Resolution with Binaural Sounds
We also propose two auxiliary tasks namely, a) a novel task on Spatial Sound Super-resolution to increase the spatial resolution of sounds, and b) dense depth prediction of the scene.
Matching Neuromorphic Events and Color Images via Adversarial Learning
The event camera has appealing properties: high dynamic range, low latency, low power consumption and low memory usage, and thus provides complementariness to conventional frame-based cameras.
Don't Forget The Past: Recurrent Depth Estimation from Monocular Video
In particular, we put three different types of depth estimation (supervised depth prediction, self-supervised depth prediction, and self-supervised depth completion) into a common framework.
Self-supervised Object Motion and Depth Estimation from Video
We present a self-supervised learning framework to estimate the individual object motion and monocular depth from video.
Semantic Understanding of Foggy Scenes with Purely Synthetic Data
This work addresses the problem of semantic scene understanding under foggy road conditions.
Talk2Nav: Long-Range Vision-and-Language Navigation with Dual Attention and Spatial Memory
Our first contribution is the creation of a large-scale dataset with verbal navigation instructions.
Texture Underfitting for Domain Adaptation
Comprehensive semantic segmentation is one of the key components for robust scene understanding and a requirement to enable autonomous driving.
Learning a Curve Guardian for Motorcycles
Our contributes are fourfold: 1) we predict the motorcycle's intra-lane position using a convolutional neural network (CNN), 2) we predict the motorcycle roll angle using a CNN, 3) we use an upgraded controller model that incorporates road incline for a more realistic model and prediction, 4) we design a scale-able system by utilizing HERE Technologies map database to obtain the accurate road geometry of the future path.
Gated CRF Loss for Weakly Supervised Semantic Image Segmentation
State-of-the-art approaches for semantic segmentation rely on deep convolutional neural networks trained on fully annotated datasets, that have been shown to be notoriously expensive to collect, both in terms of time and money.
Image Segmentation
Weakly supervised Semantic Segmentation
+1
Learning Accurate, Comfortable and Human-like Driving
Our model is trained and evaluated on the Drive360 dataset, which features 60 hours and 3000 km of real-world driving data.
Real-time 3D Traffic Cone Detection for Autonomous Driving
In this work, we leverage the unique structure of traffic cones and propose a pipelined approach to the problem.
Guided Curriculum Model Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation
Our central contributions are: 1) a curriculum framework to gradually adapt semantic segmentation models from day to night via labeled synthetic images and unlabeled real images, both for progressively darker times of day, which exploits cross-time-of-day correspondences for the real images to guide the inference of their labels; 2) a novel uncertainty-aware annotation and evaluation framework and metric for semantic segmentation, designed for adverse conditions and including image regions beyond human recognition capability in the evaluation in a principled fashion; 3) the Dark Zurich dataset, which comprises 2416 unlabeled nighttime and 2920 unlabeled twilight images with correspondences to their daytime counterparts plus a set of 151 nighttime images with fine pixel-level annotations created with our protocol, which serves as a first benchmark to perform our novel evaluation.
Ranked #5 on
Semantic Segmentation
on Nighttime Driving
Curriculum Model Adaptation with Synthetic and Real Data for Semantic Foggy Scene Understanding
The method is based on the fact that the results of semantic segmentation in moderately adverse conditions (light fog) can be bootstrapped to solve the same problem in highly adverse conditions (dense fog).
Ranked #3 on
Domain Adaptation
on Cityscapes-to-FoggyDriving
Dark Model Adaptation: Semantic Image Segmentation from Daytime to Nighttime
This work addresses the problem of semantic image segmentation of nighttime scenes.
Ranked #10 on
Semantic Segmentation
on Nighttime Driving
Model Adaptation with Synthetic and Real Data for Semantic Dense Foggy Scene Understanding
In addition, we present three other main stand-alone contributions: 1) a novel method to add synthetic fog to real, clear-weather scenes using semantic input; 2) a new fog density estimator; 3) the Foggy Zurich dataset comprising $3808$ real foggy images, with pixel-level semantic annotations for $16$ images with dense fog.
Unified Hypersphere Embedding for Speaker Recognition
Incremental improvements in accuracy of Convolutional Neural Networks are usually achieved through use of deeper and more complex models trained on larger datasets.
Failure Prediction for Autonomous Driving
This work presents a method to learn to predict the occurrence of these failures, i. e. to assess how difficult a scene is to a given driving model and to possibly give the human driver an early headsup.
Ensemble Manifold Segmentation for Model Distillation and Semi-supervised Learning
We mitigate this by introducing ensemble manifold segmentation (EMS).
End-to-End Learning of Driving Models with Surround-View Cameras and Route Planners
In particular, we develop a sensor setup that provides data for a 360-degree view of the area surrounding the vehicle, the driving route to the destination, and low-level driving maneuvers (e. g. steering angle and speed) by human drivers.
Domain Adaptive Faster R-CNN for Object Detection in the Wild
The results demonstrate the effectiveness of our proposed approach for robust object detection in various domain shift scenarios.
Learning the Synthesizability of Dynamic Texture Samples
In this paper, we investigate the synthesizability of dynamic texture samples: {\em given a dynamic texture sample, how synthesizable it is by using EDTS, and which EDTS method is the most suitable to synthesize it?}
Object Referring in Videos with Language and Human Gaze
To that end, we present a new video dataset for OR, with 30, 000 objects over 5, 000 stereo video sequences annotated for their descriptions and gaze.
Object Referring in Visual Scene with Spoken Language
This paper investigates Object Referring with Spoken Language (ORSpoken) by presenting two datasets and one novel approach.
Semantic Foggy Scene Understanding with Synthetic Data
Due to the difficulty of collecting and annotating foggy images, we choose to generate synthetic fog on real images that depict clear-weather outdoor scenes, and then leverage these partially synthetic data for SFSU by employing state-of-the-art convolutional neural networks (CNN).
Deep Domain Adaptation by Geodesic Distance Minimization
Our work builds on the recently proposed Deep CORAL method, which proposed to train a convolutional neural network and simultaneously minimize the Euclidean distance of convariance matrices between the source and target domains.
Speech-Based Visual Question Answering
This paper introduces speech-based visual question answering (VQA), the task of generating an answer given an image and a spoken question.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
PathTrack: Fast Trajectory Annotation with Path Supervision
We further validate our approach by crowdsourcing the PathTrack dataset, with more than 15, 000 person trajectories in 720 sequences.
Scale-Aware Alignment of Hierarchical Image Segmentation
To demonstrate the power of our method, we perform comprehensive experiments, which show that our method, as a post-processing step, can significantly improve the quality of the hierarchical segmentation representations, and ease the usage of hierarchical image segmentation to high-level vision tasks such as object segmentation.
Fast Algorithms for Linear and Kernel SVM+
The SVM+ approach has shown excellent performance in visual recognition tasks for exploiting privileged information in the training data.
Fast Optical Flow using Dense Inverse Search
Most recent works in optical flow extraction focus on the accuracy and neglect the time complexity.
Unsupervised High-level Feature Learning by Ensemble Projection for Semi-supervised Image Classification and Image Clustering
Hence, in the spirit of ensemble learning we create a set of such training sets which are all diverse, leading to diverse classifiers.
Is Image Super-resolution Helpful for Other Vision Tasks?
In this paper, we present the first comprehensive study and analysis of the usefulness of ISR for other vision applications.
Metric Imitation by Manifold Transfer for Efficient Vision Applications
In particular, MI consists of: 1) quantifying the properties of source metrics as manifold geometry, 2) transferring the manifold from source domain to target domain, and 3) learning a mapping of TFs so that the manifold is approximated as well as possible in the mapped feature domain.
Joint Vanishing Point Extraction and Tracking
Although the method is designed for unknown camera poses, it is also helpful in scenarios with known poses, since a multi-frame approach in VP detection helps to regularize in frames with weak VP line support.
Latent Dictionary Learning for Sparse Representation based Classification
Each dictionary atom is jointly learned with a latent vector, which associates this atom to the representation of different classes.
The Synthesizability of Texture Examples
This work is the first attempt to quantify this image property, and we find that texture synthesizability can be learned and predicted.
