Search Results for author:
Found 76 papers, 56 papers with code
TAPTR: Tracking Any Point with Transformers as Detection
Based on the observation that point tracking bears a great resemblance to object detection and tracking, we borrow designs from DETR-like algorithms to address the task of TAP.
SparseFusion: Efficient Sparse Multi-Modal Fusion Framework for Long-Range 3D Perception
The versatility of SparseFusion is also validated in the temporal object detection task and 3D lane detection task.
Generalized Predictive Model for Autonomous Driving
In this paper, we introduce the first large-scale video prediction model in the autonomous driving discipline.
FastMAC: Stochastic Spectral Sampling of Correspondence Graph
As such, the core of our method is the stochastic spectral sampling of correspondence graph.
Embodied Understanding of Driving Scenarios
Hereby, we introduce the Embodied Language Model (ELM), a comprehensive framework tailored for agents' understanding of driving scenes with large spatial and temporal spans.
Enhancing Generalization in Medical Visual Question Answering Tasks via Gradient-Guided Model Perturbation
In this paper, we introduce a method that incorporates gradient-guided parameter perturbations to the visual encoder of the multimodality model during both pre-training and fine-tuning phases, to improve model generalization for downstream medical VQA tasks.
Translating Images to Road Network:A Non-Autoregressive Sequence-to-Sequence Approach
Instead, our work establishes a unified representation of both types of data domain by projecting both Euclidean and non-Euclidean data into an integer series called RoadNet Sequence.
Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks
We introduce Grounded SAM, which uses Grounding DINO as an open-set object detector to combine with the segment anything model (SAM).
Visual Point Cloud Forecasting enables Scalable Autonomous Driving
To resolve this, we bring up a new pre-training task termed as visual point cloud forecasting - predicting future point clouds from historical visual input.
Fully Sparse 3D Occupancy Prediction
Occupancy prediction plays a pivotal role in autonomous driving.
LaneSegNet: Map Learning with Lane Segment Perception for Autonomous Driving
A map, as crucial information for downstream applications of an autonomous driving system, is usually represented in lanelines or centerlines.
DriveLM: Driving with Graph Visual Question Answering
The experiments demonstrate that Graph VQA provides a simple, principled framework for reasoning about a driving scene, and DriveLM-Data provides a challenging benchmark for this task.
A Survey of Reasoning with Foundation Models
Reasoning, a crucial ability for complex problem-solving, plays a pivotal role in various real-world settings such as negotiation, medical diagnosis, and criminal investigation.
Open-sourced Data Ecosystem in Autonomous Driving: the Present and Future
With the continuous maturation and application of autonomous driving technology, a systematic examination of open-source autonomous driving datasets becomes instrumental in fostering the robust evolution of the industry ecosystem.
LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models
To address this issue, we have created GVC data that allows for the combination of grounding and chat capabilities.
Visual In-Context Prompting
In-context prompting in large language models (LLMs) has become a prevalent approach to improve zero-shot capabilities, but this idea is less explored in the vision domain.
LLM4Drive: A Survey of Large Language Models for Autonomous Driving
Recently, large language models (LLMs) have demonstrated abilities including understanding context, logical reasoning, and generating answers.
Leveraging Vision-Centric Multi-Modal Expertise for 3D Object Detection
Current research is primarily dedicated to advancing the accuracy of camera-only 3D object detectors (apprentice) through the knowledge transferred from LiDAR- or multi-modal-based counterparts (expert).
Ranked #6 on
3D Object Detection
on nuScenes Camera Only
DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Driving
We find that even equipped with a SOTA perception model, directly letting the student model learn the required inputs of the teacher model leads to poor driving performance, which comes from the large distribution gap between predicted privileged inputs and the ground-truth.
Ranked #2 on
CARLA longest6
on CARLA
DFA3D: 3D Deformable Attention For 2D-to-3D Feature Lifting
Existing feature lifting approaches, such as Lift-Splat-based and 2D attention-based, either use estimated depth to get pseudo LiDAR features and then splat them to a 3D space, which is a one-pass operation without feature refinement, or ignore depth and lift features by 2D attention mechanisms, which achieve finer semantics while suffering from a depth ambiguity problem.
Density-invariant Features for Distant Point Cloud Registration
Registration of distant outdoor LiDAR point clouds is crucial to extending the 3D vision of collaborative autonomous vehicles, and yet is challenging due to small overlapping area and a huge disparity between observed point densities.
Ranked #1 on
Point Cloud Registration
on nuScenes (Distant PCR)
End-to-end Autonomous Driving: Challenges and Frontiers
The autonomous driving community has witnessed a rapid growth in approaches that embrace an end-to-end algorithm framework, utilizing raw sensor input to generate vehicle motion plans, instead of concentrating on individual tasks such as detection and motion prediction.
detrex: Benchmarking Detection Transformers
To address this issue, we develop a unified, highly modular, and lightweight codebase called detrex, which supports a majority of the mainstream DETR-based instance recognition algorithms, covering various fundamental tasks, including object detection, segmentation, and pose estimation.
Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentation
In this paper, we propose MUTR, a Multi-modal Unified Temporal transformer for Referring video object segmentation.
Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving
End-to-end autonomous driving has made impressive progress in recent years.
Ranked #4 on
CARLA longest6
on CARLA
A Strong and Reproducible Object Detector with Only Public Datasets
This work presents Focal-Stable-DINO, a strong and reproducible object detection model which achieves 64. 6 AP on COCO val2017 and 64. 8 AP on COCO test-dev using only 700M parameters without any test time augmentation.
Ranked #5 on
Object Detection
on COCO minival
(using extra training data)
OpenLane-V2: A Topology Reasoning Benchmark for Unified 3D HD Mapping
Accurately depicting the complex traffic scene is a vital component for autonomous vehicles to execute correct judgments.
Graph-based Topology Reasoning for Driving Scenes
Understanding the road genome is essential to realize autonomous driving.
Ranked #5 on
3D Lane Detection
on OpenLane-V2 val
Detection Transformer with Stable Matching
We point out that the unstable matching in DETR is caused by a multi-optimization path problem, which is highlighted by the one-to-one matching design in DETR.
Sparse Dense Fusion for 3D Object Detection
With the prevalence of multimodal learning, camera-LiDAR fusion has gained popularity in 3D object detection.
Geometric-aware Pretraining for Vision-centric 3D Object Detection
We also conduct experiments on various image backbones and view transformations to validate the efficacy of our approach.
3D Data Augmentation for Driving Scenes on Camera
The proposed data augmentation approach contributes to a gain of 1. 7% and 1. 4% in terms of detection accuracy, on Waymo and nuScences respectively.
Lite DETR : An Interleaved Multi-Scale Encoder for Efficient DETR
Recent DEtection TRansformer-based (DETR) models have obtained remarkable performance.
Mimic before Reconstruct: Enhancing Masked Autoencoders with Feature Mimicking
To alleviate this, previous methods simply replace the pixel reconstruction targets of 75% masked tokens by encoded features from pre-trained image-image (DINO) or image-language (CLIP) contrastive learning.
Introducing Depth into Transformer-based 3D Object Detection
To address the second issue, we introduce an auxiliary learning task called Depth-aware Negative Suppression loss.
Policy Pre-training for Autonomous Driving via Self-supervised Geometric Modeling
Witnessing the impressive achievements of pre-training techniques on large-scale data in the field of computer vision and natural language processing, we wonder whether this idea could be adapted in a grab-and-go spirit, and mitigate the sample inefficiency problem for visuomotor driving.
Translating Images to Road Network: A Non-Autoregressive Sequence-to-Sequence Approach
The extraction of road network is essential for the generation of high-definition maps since it enables the precise localization of road landmarks and their interconnections.
Distilling Focal Knowledge From Imperfect Expert for 3D Object Detection
Specifically, a set of queries are leveraged to locate the instance-level areas for masked feature generation, to intensify feature representation ability in these areas.
Lite DETR: An Interleaved Multi-Scale Encoder for Efficient DETR
Recent DEtection TRansformer-based (DETR) models have obtained remarkable performance.
Planning-oriented Autonomous Driving
Oriented at this, we revisit the key components within perception and prediction, and prioritize the tasks such that all these tasks contribute to planning.
BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision
The proposed method is verified with a wide spectrum of traditional and modern image backbones and achieves new SoTA results on the large-scale nuScenes dataset.
Ranked #5 on
3D Object Detection
on Rope3D
Stare at What You See: Masked Image Modeling without Reconstruction
However, unlike the low-level features such as pixel values, we argue the features extracted by powerful teacher models already encode rich semantic correlation across regions in an intact image. This raises one question: is reconstruction necessary in Masked Image Modeling (MIM) with a teacher model?
DCL-Net: Deep Correspondence Learning Network for 6D Pose Estimation
Establishment of point correspondence between camera and object coordinate systems is a promising way to solve 6D object poses.
Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipe
As sensor configurations get more complex, integrating multi-source information from different sensors and representing features in a unified view come of vital importance.
ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning
In particular, we propose a spatial-temporal feature learning scheme towards a set of more representative features for perception, prediction and planning tasks simultaneously, which is called ST-P3.
Ranked #7 on
Bird's-Eye View Semantic Segmentation
on nuScenes
(IoU ped - 224x480 - Vis filter. - 100x100 at 0.5 metric)
Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline
The two branches are connected so that the control branch receives corresponding guidance from the trajectory branch at each time step.
Ranked #3 on
Autonomous Driving
on CARLA Leaderboard
Level 2 Autonomous Driving on a Single Device: Diving into the Devils of Openpilot
Equipped with a wide span of sensors, predominant autonomous driving solutions are becoming more modular-oriented for safe system design.
HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encoding
Based on these observations, we propose Heterogeneous Driving Graph Transformer (HDGT), a backbone modelling the driving scene as a heterogeneous graph with different types of nodes and edges.
BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers
In a nutshell, BEVFormer exploits both spatial and temporal information by interacting with spatial and temporal space through predefined grid-shaped BEV queries.
PersFormer: 3D Lane Detection via Perspective Transformer and the OpenLane Benchmark
Methods for 3D lane detection have been recently proposed to address the issue of inaccurate lane layouts in many autonomous driving scenarios (uphill/downhill, bump, etc.).
Ranked #5 on
3D Lane Detection
on Apollo Synthetic 3D Lane
Align Representations With Base: A New Approach to Self-Supervised Learning
Existing symmetric contrastive learning methods suffer from collapses (complete and dimensional) or quadratic complexity of objectives.
Sparse Steerable Convolutions: An Efficient Learning of SE(3)-Equivariant Features for Estimation and Tracking of Object Poses in 3D Space
In this paper, we propose a novel design of Sparse Steerable Convolution (SS-Conv) to address the shortcoming; SS-Conv greatly accelerates steerable convolution with sparse tensors, while strictly preserving the property of SE(3)-equivariance.
Monocular 3D Object Detection: An Extrinsic Parameter Free Approach
Due to the lack of insight in industrial application, existing methods on open datasets neglect the camera pose information, which inevitably results in the detector being susceptible to camera extrinsic parameters.
Ranked #9 on
Monocular 3D Object Detection
on KITTI Cars Moderate
(using extra training data)
Towards Improving the Consistency, Efficiency, and Flexibility of Differentiable Neural Architecture Search
Our method enables differentiable sparsification, and keeps the derived architecture equivalent to that of Engine-cell, which further improves the consistency between search and evaluation.
EnTranNAS: Towards Closing the Gap between the Architectures in Search and Evaluation
The Engine-cell is differentiable for architecture search, while the Transit-cell only transits the current sub-graph by architecture derivation.
Exploring intermediate representation for monocular vehicle pose estimation
The latter question motivates us to incorporate geometry knowledge with a new loss function based on a projective invariant.
Ranked #1 on
Vehicle Pose Estimation
on KITTI Cars Hard
ISTA-NAS: Efficient and Consistent Neural Architecture Search by Sparse Coding
By doing so, our network for search at each update satisfies the sparsity constraint and is efficient to train.
Point-Set Anchors for Object Detection, Instance Segmentation and Pose Estimation
A recent approach for object detection and human pose estimation is to regress bounding boxes or human keypoints from a central point on the object or person.
Dynamical System Inspired Adaptive Time Stepping Controller for Residual Network Families
We establish a stability condition for ResNets with step sizes and weight parameters, and point out the effects of step sizes on the stability and performance.
SOGNet: Scene Overlap Graph Network for Panoptic Segmentation
In order to overcome the lack of supervision, we introduce a differentiable module to resolve the overlap between any pair of instances.
Ranked #8 on
Panoptic Segmentation
on Cityscapes test
Deepsleep: Fast and Accurate Delineation of Sleep Arousals at Millisecond Resolution by Deep Learning
Background: Sleep arousals are transient periods of wakefulness punctuated into sleep.
Finding Task-Relevant Features for Few-Shot Learning by Category Traversal
Few-shot learning is an important area of research.
Feature Intertwiner for Object Detection
We argue that the reliable set could guide the feature learning of the less reliable set during training - in spirit of student mimicking teacher behavior and thus pushing towards a more compact class centroid in the feature space.
Ranked #134 on
Object Detection
on COCO test-dev
Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge
This study assesses the state-of-the-art machine learning (ML) methods used for brain tumor image analysis in mpMRI scans, during the last seven instances of the International Brain Tumor Segmentation (BraTS) challenge, i. e., 2012-2018.
Neural Network Encapsulation
Motivated by the routing to make higher capsule have agreement with lower capsule, we extend the mechanism as a compensation for the rapid loss of information in nearby layers.
Zoom Out-and-In Network with Map Attention Decision for Region Proposal and Object Detection
A key observation is that it is difficult to classify anchors of different sizes with the same set of features.
Ranked #2 on
Region Proposal
on COCO test-dev
Recurrent Scale Approximation for Object Detection in CNN
To further increase efficiency and accuracy, we (a): design a scale-forecast network to globally predict potential scales in the image since there is no need to compute maps on all levels of the pyramid.
Ranked #3 on
Face Detection
on Annotated Faces in the Wild
Learning Deep Features via Congenerous Cosine Loss for Person Recognition
Person recognition aims at recognizing the same identity across time and space with complicated scenes and similar appearance.
Zoom Out-and-In Network with Recursive Training for Object Proposal
In this paper, we propose a zoom-out-and-in network for generating object proposals.
Dual Deep Network for Visual Tracking
In this paper, we propose a dual network to better utilize features among layers for visual tracking.
Multi-Bias Non-linear Activation in Deep Neural Networks
It provides great flexibility of selecting responses to different visual patterns in different magnitude ranges to form rich representations in higher layers.
Learning Deep Representation With Large-Scale Attributes
Experimental results show that the attributes are helpful in learning better features and improving the object detection accuracy by 2. 6% in mAP on the ILSVRC 2014 object detection dataset and 2. 4% in mAP on PASCAL VOC 2007 object detection dataset.
LCNN: Low-level Feature Embedded CNN for Salient Object Detection
In this paper, we propose a novel deep neural network framework embedded with low-level features (LCNN) for salient object detection in complex images.
Inner and Inter Label Propagation: Salient Object Detection in the Wild
For most natural images, some boundary superpixels serve as the background labels and the saliency of other superpixels are determined by ranking their similarities to the boundary labels based on an inner propagation scheme.
