Search Results for author:
Found 84 papers, 44 papers with code
Centaur: Robust End-to-End Autonomous Driving with Test-Time Training
In this work, we propose Centaur (Cluster Entropy for Test-time trAining using Uncertainty) which updates a planner's behavior via test-time training, without relying on hand-engineered rules or cost functions.
Enhancing Autonomous Driving Safety with Collision Scenario Integration
In this paper, we propose SafeFusion, a training framework to learn from collision data.
Counterfactual Situation Testing: From Single to Multidimensional Discrimination
ST finds for each complainant similar protected and non-protected instances in the dataset; constructs, respectively, a control and test group; and compares the groups such that a difference in outcomes implies a potential case of individual discrimination.
Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Models
Recently, promising progress has been made by open-source vision-language models (VLMs) in bringing their capabilities closer to those of proprietary frontier models.
Global Average Feature Augmentation for Robust Semantic Segmentation with Transformers
Robustness to out-of-distribution data is crucial for deploying modern neural networks.
Uncertainty Estimation for 3D Object Detection via Evidential Learning
We introduce a framework for quantifying uncertainty in 3D object detection by leveraging an evidential learning loss on Bird's Eye View representations in the 3D detector.
SSE: Multimodal Semantic Data Selection and Enrichment for Industrial-scale Data Assimilation
We quantitatively show that our Semantic Selection and Enrichment framework (SSE) can a) successfully maintain model performance with a smaller training dataset and b) improve model performance by enriching the smaller dataset without exceeding the original dataset size.
Exploring Camera Encoder Designs for Autonomous Driving Perception
The cornerstone of autonomous vehicles (AV) is a solid perception system, where camera encoders play a crucial role.
Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillation
We propose Hydra-MDP, a novel paradigm employing multiple teachers in a teacher-student model.
Step Out and Seek Around: On Warm-Start Training with Incremental Data
First, a novel feature regularization (FeatReg) to retain and refine knowledge from existing checkpoints; Second, we propose adaptive knowledge distillation (AdaKD), a novel approach to forget mitigation and knowledge transfer.
Memorize What Matters: Emergent Scene Decomposition from Multitraverse
Humans naturally retain memories of permanent elements, while ephemeral moments often slip through the cracks of memory.
Mutatis Mutandis: Revisiting the Comparator in Discrimination Testing
The CP comparator is the standard one among discrimination testing, representing an idealized comparison as it aims for having a complainant-comparator pair that only differs on membership to the protected attribute.
OmniDrive: A Holistic LLM-Agent Framework for Autonomous Driving with 3D Perception, Reasoning and Planning
We further propose OmniDrive-nuScenes, a new visual question-answering dataset challenging the true 3D situational awareness of a model with comprehensive visual question-answering (VQA) tasks, including scene description, traffic regulation, 3D grounding, counterfactual reasoning, decision making and planning.
Mining Supervision for Dynamic Regions in Self-Supervised Monocular Depth Estimation
In the next stage, we use an object network to estimate the depth of those moving objects assuming rigid motions.
What is Point Supervision Worth in Video Instance Segmentation?
Video instance segmentation (VIS) is a challenging vision task that aims to detect, segment, and track objects in videos.
Improving Distant 3D Object Detection Using 2D Box Supervision
This mapping allows the depth estimation of distant objects conditioned on their 2D boxes, making long-range 3D detection with 2D supervision feasible.
Causal Perception
We present two kinds of causal perception, unfaithful and inconsistent, based on the SCM properties of faithfulness and consistency.
Fully Attentional Networks with Self-emerging Token Labeling
With the proposed STL framework, our best model based on FAN-L-Hybrid (77. 3M parameters) achieves 84. 8% Top-1 accuracy and 42. 1% mCE on ImageNet-1K and ImageNet-C, and sets a new state-of-the-art for ImageNet-A (46. 1%) and ImageNet-R (56. 6%) without using extra data, outperforming the original FAN counterpart by significant margins.
Ranked #16 on
Domain Generalization
on ImageNet-C
Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?
We initially observed that the nuScenes dataset, characterized by relatively simple driving scenarios, leads to an under-utilization of perception information in end-to-end models incorporating ego status, such as the ego vehicle's velocity.
BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detection
Recently, the rise of query-based Transformer decoders is reshaping camera-based 3D object detection.
SEGIC: Unleashing the Emergent Correspondence for In-Context Segmentation
In-context segmentation aims at segmenting novel images using a few labeled example images, termed as "in-context examples", exploring content similarities between examples and the target.
ViR: Towards Efficient Vision Retention Backbones
Inspired by this trend, we propose a new class of computer vision models, dubbed Vision Retention Networks (ViR), with dual parallel and recurrent formulations, which strike an optimal balance between fast inference and parallel training with competitive performance.
Towards Viewpoint Robustness in Bird's Eye View Segmentation
We introduce a technique for novel view synthesis and use it to transform collected data to the viewpoint of target rigs, allowing us to train BEV segmentation models for diverse target rigs without any additional data collection or labeling cost.
FB-BEV: BEV Representation from Forward-Backward View Transformations
Currently, the two most prominent VTM paradigms are forward projection and backward projection.
The Initial Screening Order Problem
To study the impact of the ISO, we introduce a human-like screener and compare it to its algorithmic counterpart, where the human-like screener is conceived to be inconsistent over time.
Parametric Depth Based Feature Representation Learning for Object Detection and Segmentation in Bird's Eye View
In contrast, we propose to use parametric depth distribution modeling for feature transformation.
FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformation
This technical report summarizes the winning solution for the 3D Occupancy Prediction Challenge, which is held in conjunction with the CVPR 2023 Workshop on End-to-End Autonomous Driving and CVPR 23 Workshop on Vision-Centric Autonomous Driving Workshop.
Ranked #2 on
Prediction Of Occupancy Grid Maps
on Occ3D-nuScenes
FasterViT: Fast Vision Transformers with Hierarchical Attention
At a high level, global self-attentions enable the efficient cross-window communication at lower costs.
Ranked #188 on
Image Classification
on ImageNet
Domain Adaptive Decision Trees: Implications for Accuracy and Fairness
In uses of pre-trained machine learning models, it is a known issue that the target population in which the model is being deployed may not have been reflected in the source population with which the model was trained.
VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion
To enable such capability in AI systems, we propose VoxFormer, a Transformer-based semantic scene completion framework that can output complete 3D volumetric semantics from only 2D images.
3D geometry
3D Semantic Scene Completion from a single RGB image
+1
Counterfactual Situation Testing: Uncovering Discrimination under Fairness given the Difference
For any complainant, we find and compare similar protected and non-protected instances in the dataset used by the classifier to construct a control and test group, where a difference between the decision outcomes of the two groups implies potential individual discrimination.
Vision Transformers Are Good Mask Auto-Labelers
We propose Mask Auto-Labeler (MAL), a high-quality Transformer-based mask auto-labeling framework for instance segmentation using only box annotations.
FocalFormer3D: Focusing on Hard Instance for 3D Object Detection
For 3D object detection, we instantiate this method as FocalFormer3D, a simple yet effective detector that excels at excavating difficult objects and improving prediction recall.
Parametric Depth Based Feature Representation Learning for Object Detection and Segmentation in Bird's-Eye View
In contrast, we propose to use parametric depth distribution modeling for feature transformation.
Soft Masking for Cost-Constrained Channel Pruning
Structured channel pruning has been shown to significantly accelerate inference time for convolution neural networks (CNNs) on modern hardware, with a relatively minor loss of network accuracy.
Structural Pruning via Latency-Saliency Knapsack
We propose Hardware-Aware Latency Pruning (HALP) that formulates structural pruning as a global resource allocation optimization problem, aiming at maximizing the accuracy while constraining latency under a predefined budget on targeting device.
Optimizing Data Collection for Machine Learning
Modern deep learning systems require huge data sets to achieve impressive performance, but there is little guidance on how much or what kind of data to collect.
Object-Level Targeted Selection via Deep Template Matching
Existing semantic image retrieval methods often focus on mining for larger sized geographical landmarks, and/or require extra labeled data, such as images/image-pairs with similar objects, for mining images with generic objects.
How Much More Data Do I Need? Estimating Requirements for Downstream Tasks
Given a small training data set and a learning algorithm, how much more data is necessary to reach a target validation or test performance?
Knowledge Distillation for 6D Pose Estimation by Aligning Distributions of Local Predictions
Knowledge distillation facilitates the training of a compact student network by using a deep teacher one.
Non-parametric Depth Distribution Modelling based Depth Inference for Multi-view Stereo
Boundary pixels usually follow a multi-modal distribution as they represent different depths; Therefore, the assumption results in an erroneous depth prediction at the coarser level of the cost volume pyramid and can not be corrected in the refinement levels leading to wrong depth predictions.
Understanding The Robustness in Vision Transformers
Our study is motivated by the intriguing properties of the emerging visual grouping in Vision Transformers, which indicates that self-attention may promote robustness through improved mid-level representations.
Ranked #4 on
Domain Generalization
on ImageNet-R
(using extra training data)
M$^2$BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representation
In this paper, we propose M$^2$BEV, a unified framework that jointly performs 3D object detection and map segmentation in the Birds Eye View~(BEV) space with multi-camera image inputs.
FreeSOLO: Learning to Segment Objects without Annotations
FreeSOLO further demonstrates superiority as a strong pre-training method, outperforming state-of-the-art self-supervised pre-training methods by +9. 8% AP when fine-tuning instance segmentation with only 5% COCO masks.
Fairness Implications of Encoding Protected Categorical Attributes
We investigate the interaction between categorical encodings and target encoding regularization methods that reduce unfairness.
Boosting Supervised Learning Performance with Co-training
In this paper, we propose a new light-weight self-supervised learning framework that could boost supervised learning performance with minimum additional computation cost.
When to Prune? A Policy towards Early Structural Pruning
Through extensive experiments on ImageNet, we show that EPI empowers a quick tracking of early training epochs suitable for pruning, offering same efficacy as an otherwise ``oracle'' grid-search that scans through epochs and requires orders of magnitude more compute.
HALP: Hardware-Aware Latency Pruning
We propose Hardware-Aware Latency Pruning (HALP) that formulates structural pruning as a global resource allocation optimization problem, aiming at maximizing the accuracy while constraining latency under a predefined budget.
Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformers
Specifically, we supervise the attention modules in the mask decoder in a layer-wise manner.
Ranked #4 on
Panoptic Segmentation
on COCO test-dev
Privacy Vulnerability of Split Computing to Data-Free Model Inversion Attacks
Prior works usually assume that SC offers privacy benefits as only intermediate features, instead of private data, are shared from devices to the cloud.
Not All Labels Are Equal: Rationalizing The Labeling Costs for Training Object Detection
Deep neural networks have reached high accuracy on object detection but their success hinges on large amounts of labeled data.
Optimal Quantization Using Scaled Codebook
We study the problem of quantizing N sorted, scalar datapoints with a fixed codebook containing K entries that are allowed to be rescaled.
Distilling Image Classifiers in Object Detectors
Knowledge distillation constitutes a simple yet effective way to improve the performance of a compact student network by exploiting the knowledge of a more powerful teacher.
SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
We present SegFormer, a simple, efficient yet powerful semantic segmentation framework which unifies Transformers with lightweight multilayer perception (MLP) decoders.
Ranked #1 on
Semantic Segmentation
on COCO-Stuff full
See through Gradients: Image Batch Recovery via GradInversion
In this work, we introduce GradInversion, using which input images from a larger batch (8 - 48 images) can also be recovered for large networks such as ResNets (50 layers), on complex datasets such as ImageNet (1000 classes, 224x224 px).
Image-Level or Object-Level? A Tale of Two Resampling Strategies for Long-Tailed Detection
As a result, image resampling alone is not enough to yield a sufficiently balanced distribution at the object level.
Self-supervised Learning of Depth Inference for Multi-view Stereo
Here, we propose a self-supervised learning framework for multi-view stereo that exploit pseudo labels from the input data.
Contrastive Syn-to-Real Generalization
Training on synthetic data can be beneficial for label or data-scarce scenarios.
Active Learning for Deep Object Detection via Probabilistic Modeling
Most of these methods are based on multiple models or are straightforward extensions of classification methods, hence estimate an image's informativeness using only the classification head.
Deep Active Learning for Object Detection with Mixture Density Networks
For active learning, we propose a scoring function that aggregates uncertainties from both the classification and the localization outputs of the network.
Personalized Federated Learning with First Order Model Optimization
While federated learning traditionally aims to train a single global model across decentralized local datasets, one model may not always be ideal for all participating clients.
Scalable Active Learning for Object Detection
We have built a scalable production system for active learning in the domain of autonomous driving.
Context Based Emotion Recognition using EMOTIC Dataset
In this paper we present EMOTIC, a dataset of images of people in a diverse set of natural situations, annotated with their apparent emotion.
Ranked #5 on
Emotion Recognition in Context
on EMOTIC
(using extra training data)
Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion
We introduce DeepInversion, a new method for synthesizing images from the image distribution used to train a deep neural network.
Cost Volume Pyramid Based Depth Inference for Multi-View Stereo
We propose a cost volume-based neural network for depth inference from multi-view images.
Ranked #15 on
3D Reconstruction
on DTU
Training Data Distribution Search with Ensemble Active Learning
In this paper, we propose to scale up ensemble Active Learning methods to perform acquisition at a large scale (10k to 500k samples at a time).
VACL: Variance-Aware Cross-Layer Regularization for Pruning Deep Residual Networks
Our approach can effectively improve the structural sparsity of residual models.
Quadtree Generating Networks: Efficient Hierarchical Scene Parsing with Sparse Convolutions
Semantic segmentation with Convolutional Neural Networks is a memory-intensive task due to the high spatial resolution of feature maps and output predictions.
Training Data Subset Search with Ensemble Active Learning
In this paper, we propose to scale up ensemble Active Learning (AL) methods to perform acquisition at a large scale (10k to 500k samples at a time).
The Relevance of Bayesian Layer Positioning to Model Uncertainty in Deep Bayesian Active Learning
One of the main challenges of deep learning tools is their inability to capture model uncertainty.
ExpandNets: Linear Over-parameterization to Train Compact Convolutional Networks
As evidenced by our experiments, our approach outperforms both training the compact network from scratch and performing knowledge distillation from a teacher.
Large-Scale Visual Active Learning with Deep Probabilistic Ensembles
Annotating the right data for training deep neural networks is an important challenge.
Deep Probabilistic Ensembles: Approximate Variational Inference through KL Regularization
In this paper, we introduce Deep Probabilistic Ensembles (DPEs), a scalable technique that uses a regularized ensemble to approximate a deep Bayesian Neural Network (BNN).
Effective Use of Synthetic Data for Urban Scene Semantic Segmentation
Our approach builds on the observation that foreground and background classes are not affected in the same manner by the domain shift, and thus should be treated differently.
Compression-aware Training of Deep Networks
In recent years, great progress has been made in a variety of application domains thanks to the development of increasingly deeper neural networks.
Domain-adaptive deep network compression
We show that domain transfer leads to large shifts in network activations and that it is desirable to take this into account when compressing.
Bringing Background into the Foreground: Making All Classes Equal in Weakly-supervised Video Semantic Segmentation
Our experiments demonstrate the benefits of our classifier heatmaps and of our two-stream architecture on challenging urban scene datasets and on the YouTube-Objects benchmark, where we obtain state-of-the-art results.
Class-Weighted Convolutional Features for Visual Instance Search
In this paper, we go beyond this spatial information and propose a local-aware encoding of convolutional features based on semantic information predicted in the target image.
Emotion Recognition in Context
In this paper we present the Emotions in Context Database (EMCO), a dataset of images containing people in context in non-controlled environments.
Incorporating Network Built-in Priors in Weakly-supervised Semantic Segmentation
We then show how to obtain multi-class masks by the fusion of foreground/background ones with information extracted from a weakly-supervised localization network.
Learning the Number of Neurons in Deep Networks
In this paper, we introduce an approach to automatically determining the number of neurons in each layer of a deep network during learning.
Built-in Foreground/Background Prior for Weakly-Supervised Semantic Segmentation
Hence, weak supervision using only image tags could have a significant impact in semantic segmentation.
Learning Image Matching by Simply Watching Video
This work presents an unsupervised learning based approach to the ubiquitous computer vision problem of image matching.
Road Detection by One-Class Color Classification: Dataset and Experiments
These algorithms reduce the effect of lighting variations and weather conditions by exploiting the discriminant/invariant properties of different color representations.
