Search Results for author:
Found 19 papers, 17 papers with code
MaTVLM: Hybrid Mamba-Transformer for Efficient Vision-Language Modeling
In this work, we present a hybrid model MaTVLM by substituting a portion of the transformer decoder layers in a pre-trained VLM with Mamba-2 layers.
OmniMamba: Efficient and Unified Multimodal Understanding and Generation via State Space Models
The model fully leverages Mamba-2's high computational and memory efficiency, extending its capabilities from text generation to multimodal generation.
RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learning
By leveraging 3DGS techniques, we construct a photorealistic digital replica of the real physical world, enabling the AD policy to extensively explore the state space and learn to handle out-of-distribution scenarios through large-scale trial and error.
Multimodal Mamba: Decoder-only Multimodal State Space Model via Quadratic to Linear Distillation
We propose an seeding strategy to carve Mamba from trained Transformer and a three-stage distillation recipe, which can effectively transfer the knowledge from Transformer to Mamba while preserving multimodal capabilities.
DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving
However, the numerous denoising steps in the robotic diffusion policy and the more dynamic, open-world nature of traffic scenes pose substantial challenges for generating diverse driving actions at a real-time speed.
Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving
In contrast, Large Vision-Language Models (LVLMs) excel in scene understanding and reasoning.
DiG: Scalable and Efficient Diffusion Models with Gated Linear Attention
In this paper, we aim to incorporate the sub-quadratic modeling capability of Gated Linear Attention (GLA) into the 2D diffusion backbone.
ViG: Linear-complexity Visual Sequence Learning with Gated Linear Attention
Recently, linear complexity sequence modeling networks have achieved modeling capabilities similar to Vision Transformers on a variety of computer vision tasks, while using fewer FLOPs and less memory.
MIM4D: Masked Modeling with Multi-View Video for Autonomous Driving Representation Learning
Learning robust and scalable visual representations from massive multi-view video data remains a challenge in computer vision and autonomous driving.
VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning
Learning a human-like driving policy from large-scale driving demonstrations is promising, but the uncertainty and non-deterministic nature of planning make it challenging.
Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model
The results demonstrate that Vim is capable of overcoming the computation & memory constraints on performing Transformer-style understanding for high-resolution images and it has great potential to be the next-generation backbone for vision foundation models.
MapTRv2: An End-to-End Framework for Online Vectorized HD Map Construction
We propose a unified permutation-equivalent modeling approach, \ie, modeling map element as a point set with a group of equivalent permutations, which accurately describes the shape of map element and stabilizes the learning process.
VMA: Divide-and-Conquer Vectorized Map Annotation System for Large-Scale Driving Scene
We design a divide-and-conquer annotation scheme to solve the spatial extensibility problem of HD map generation, and abstract map elements with a variety of geometric patterns as unified point sequence representation, which can be extended to most map elements in the driving scene.
VAD: Vectorized Scene Representation for Efficient Autonomous Driving
In this paper, we propose VAD, an end-to-end vectorized paradigm for autonomous driving, which models the driving scene as a fully vectorized representation.
Ranked #15 on
Bench2Drive
on Bench2Drive
Lane Graph as Path: Continuity-preserving Path-wise Modeling for Online Lane Graph Construction
Motivated by this, we propose to model the lane graph in a novel path-wise manner, which well preserves the continuity of the lane and encodes traffic information for planning.
Perceive, Interact, Predict: Learning Dynamic and Static Clues for End-to-End Motion Prediction
Motion prediction is highly relevant to the perception of dynamic objects and static map elements in the scenarios of autonomous driving.
MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction
High-definition (HD) map provides abundant and precise environmental information of the driving scene, serving as a fundamental and indispensable component for planning in autonomous driving system.
Ranked #9 on
3D Lane Detection
on OpenLane-V2 val
You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detection
Can Transformer perform 2D object- and region-level recognition from a pure sequence-to-sequence perspective with minimal knowledge about the 2D spatial structure?
Ranked #30 on
Object Detection
on COCO-O
Real-Time and Accurate Object Detection in Compressed Video by Long Short-term Feature Aggregation
Based on deep networks, video object detection is actively studied for pushing the limits of detection speed and accuracy.
