Search Results for author:
Found 75 papers, 38 papers with code
Genesis: Multimodal Driving Scene Generation with Spatio-Temporal and Cross-Modal Consistency
We present Genesis, a unified framework for joint generation of multi-view driving videos and LiDAR sequences with spatio-temporal and cross-modal consistency.
ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving
Recent approaches attempt to address this challenge by leveraging the rich world knowledge of Vision-Language Models (VLMs), but these methods suffer from several limitations: (1) a significant domain gap between the pre-training data of VLMs and real-world driving data, (2) a dimensionality mismatch between the discrete language space and the continuous action space, and (3) imitation learning tends to capture the average behavior present in the dataset, which may be suboptimal even dangerous.
Ranked #9 on
NavSim
on OpenScene
UrbanCraft: Urban View Extrapolation via Hierarchical Sem-Geometric Priors
Existing neural rendering-based urban scene reconstruction methods mainly focus on the Interpolated View Synthesis (IVS) setting that synthesizes from views close to training camera trajectory.
AgentThink: A Unified Framework for Tool-Augmented Chain-of-Thought Reasoning in Vision-Language Models for Autonomous Driving
Vision-Language Models (VLMs) show promise for autonomous driving, yet their struggle with hallucinations, inefficient reasoning, and limited real-world validation hinders accurate perception and robust step-by-step reasoning.
Beyond Intermediate States: Explaining Visual Redundancy through Language
Multi-modal Large Langue Models (MLLMs) often process thousands of visual tokens, which consume a significant portion of the context window and impose a substantial computational burden.
ChatBEV: A Visual Language Model that Understands BEV Maps
Traffic scene understanding is essential for intelligent transportation systems and autonomous driving, ensuring safe and efficient vehicle operation.
EmoDiffusion: Enhancing Emotional 3D Facial Animation with Latent Diffusion Models
Speech-driven 3D facial animation seeks to produce lifelike facial expressions that are synchronized with the speech content and its emotional nuances, finding applications in various multimedia fields.
Generative Multi-Agent Collaboration in Embodied AI: A Systematic Review
Embodied multi-agent systems (EMAS) have attracted growing attention for their potential to address complex, real-world challenges in areas such as logistics and robotics.
Range and Bird's Eye View Fused Cross-Modal Visual Place Recognition
Compared to single-modal VPR, this approach benefits from the widespread availability of RGB cameras and the robustness of point clouds in providing accurate spatial geometry and distance information.
RCP-Bench: Benchmarking Robustness for Collaborative Perception Under Diverse Corruptions
Extensive experiments on 10 leading collaborative perception models reveal that, while these models perform well under ideal conditions, they are significantly affected by corruptions.
GSGTrack: Gaussian Splatting-Guided Object Pose Tracking from RGB Videos
Tracking the 6DoF pose of unknown objects in monocular RGB video sequences is crucial for robotic manipulation.
Towards Low-Resource Harmful Meme Detection with LMM Agents
The proliferation of Internet memes in the age of social media necessitates effective identification of harmful ones.
AIPatient: Simulating Patients with EHRs and LLM Powered Agentic Workflow
Here, we developed AIPatient, an advanced simulated patient system with AIPatient Knowledge Graph (AIPatient KG) as the input and the Reasoning Retrieval-Augmented Generation (Reasoning RAG) agentic workflow as the generation backbone.
Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey
Reinforcement Learning (RL) is a potent tool for sequential decision-making and has achieved performance surpassing human capabilities across many challenging real-world tasks.
WPN: An Unlearning Method Based on N-pair Contrastive Learning in Language Models
Generative language models (LMs) offer numerous advantages but may produce inappropriate or harmful outputs due to the harmful knowledge acquired during pre-training.
HGL: Hierarchical Geometry Learning for Test-time Adaptation in 3D Point Cloud Segmentation
HGL comprises three complementary modules from local, global to temporal learning in a bottom-up manner. Technically, we first construct a local geometry learning module for pseudo-label generation.
Embracing Events and Frames with Hierarchical Feature Refinement Network for Object Detection
The core concept is the design of the coarse-to-fine fusion module, denoted as the cross-modality adaptive feature refinement (CAFR) module.
Ranked #1 on
Object Detection
on PKU-DDD17-Car
GeoNLF: Geometry guided Pose-Free Neural LiDAR Fields
Based on this, we propose Geometry guided Neural LiDAR Fields(GeoNLF), a hybrid framework performing alternately global neural reconstruction and pure geometric pose optimization.
MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Models
Large vision-language models (LVLMs) have significantly improved multimodal reasoning tasks, such as visual question answering and image captioning.
Beyond Raw Videos: Understanding Edited Videos with Large Multimodal Model
However, a large portion of videos in real-world applications are edited videos, \textit{e. g.}, users usually cut and add effects/modifications to the raw video before publishing it on social media platforms.
ELF-UA: Efficient Label-Free User Adaptation in Gaze Estimation
Our model only needs a few unlabeled images of a target user for the model adaptation.
RCDN: Towards Robust Camera-Insensitivity Collaborative Perception via Dynamic Feature-based 3D Neural Modeling
To address above problems, we propose RCDN, a Robust Camera-insensitivity collaborative perception with a novel Dynamic feature-based 3D Neural modeling mechanism.
Risk Assessment for Nonlinear Cyber-Physical Systems under Stealth Attacks
To comprehensively quantify these risks, we propose a framework that considers both the reachability of a system and the risk distribution of a scenario.
CofiPara: A Coarse-to-fine Paradigm for Multimodal Sarcasm Target Identification with Large Multimodal Models
Current methods for Multimodal Sarcasm Target Identification (MSTI) predominantly focus on superficial indicators in an end-to-end manner, overlooking the nuanced understanding of multimodal sarcasm conveyed through both the text and image.
Urban Architect: Steerable 3D Urban Scene Generation with Layout Prior
(2) To handle the unbounded nature of urban scenes, we represent 3D scene with a Scalable Hash Grid structure, incrementally adapting to the growing scale of urban scenes.
LiDAR4D: Dynamic Neural Fields for Novel Space-time View LiDAR Synthesis
In light of this, we propose LiDAR4D, a differentiable LiDAR-only framework for novel space-time LiDAR view synthesis.
Pensieve: Retrospect-then-Compare Mitigates Visual Hallucination
Multi-modal Large Language Models (MLLMs) demonstrate remarkable success across various vision-language tasks.
GLC++: Source-Free Universal Domain Adaptation through Global-Local Clustering and Contrastive Affinity Learning
GLC++ enhances the novel category clustering accuracy of GLC by 4. 3% in open-set scenarios on Office-Home.
MAP: MAsk-Pruning for Source-Free Model Intellectual Property Protection
In this paper, we target a practical setting where only a well-trained source model is available and investigate how we can realize IP protection.
LEAD: Learning Decomposition for Source-free Universal Domain Adaptation
Besides, LEAD is also appealing in that it is complementary to most existing methods.
Ranked #2 on
Universal Domain Adaptation
on VisDA2017
PCDepth: Pattern-based Complementary Learning for Monocular Depth Estimation by Best of Both Worlds
Therefore, existing complementary learning approaches for MDE fuse intensity information from images and scene details from event data for better scene understanding.
GarchingSim: An Autonomous Driving Simulator with Photorealistic Scenes and Minimalist Workflow
However, the availability of free and open-source simulators is limited, and the installation and configuration process can be daunting for beginners and interdisciplinary researchers.
POCE: Primal Policy Optimization with Conservative Estimation for Multi-constraint Offline Reinforcement Learning
Multi-constraint offline reinforcement learning (RL) promises to learn policies that satisfy both cumulative and state-wise costs from offline datasets.
Spreeze: High-Throughput Parallel Reinforcement Learning Framework
While existing parallel RL frameworks encompass a variety of RL algorithms and parallelization techniques, the excessively burdensome communication frameworks hinder the attainment of the hardware's limit for final throughput and training effects on a single desktop.
HDMNet: A Hierarchical Matching Network with Double Attention for Large-scale Outdoor LiDAR Point Cloud Registration
Specifically, A novel feature consistency enhanced double-soft matching network is introduced to achieve two-stage matching with high flexibility while enlarging the receptive field with high efficiency in a patch-to patch manner, which significantly improves the registration performance.
Dual-Scale Interest Extraction Framework with Self-Supervision for Sequential Recommendation
Nevertheless, the existing approaches just extract each interest independently for the corresponding sub-sequence while ignoring the global correlation of the entire interaction sequence, which may fail to capture the user's inherent preference for the potential interests generalization and unavoidably make the recommended items homogeneous with the historical behaviors.
Urban Radiance Field Representation with Deformable Neural Mesh Primitives
To construct urban-level radiance fields efficiently, we design Deformable Neural Mesh Primitive~(DNMP), and propose to parameterize the entire scene with such primitives.
NeuralPCI: Spatio-temporal Neural Field for 3D Point Cloud Multi-frame Non-linear Interpolation
In light of these issues, we present NeuralPCI: an end-to-end 4D spatio-temporal Neural field for 3D Point Cloud Interpolation, which implicitly integrates multi-frame information to handle nonlinear large motions for both indoor and outdoor scenarios.
Ranked #1 on
3D Point Cloud Interpolation
on NL-Drive
UMC: A Unified Bandwidth-efficient and Multi-resolution based Collaborative Perception Framework
To verify our algorithm, we conducted experiments on the V2X-Sim and OPV2V datasets.
Text with Knowledge Graph Augmented Transformer for Video Captioning
Meanwhile, the internal stream is designed to exploit the multi-modality information in videos (e. g., the appearance of video frames, speech transcripts, and video captions) to ensure the quality of caption results.
Ranked #7 on
Video Captioning
on YouCook2
TMA: Temporal Motion Aggregation for Event-based Optical Flow
In this paper, we argue that temporal continuity is a vital element of event-based optical flow and propose a novel Temporal Motion Aggregation (TMA) approach to unlock its potential.
Upcycling Models under Domain and Category Shift
We examine the superiority of our GLC on multiple benchmarks with different category shift scenarios, including partial-set, open-set, and open-partial-set DA.
Ranked #3 on
Universal Domain Adaptation
on VisDA2017
Modality-Agnostic Debiasing for Single Domain Generalization
We validate the superiority of our MAD in a variety of single-DG scenarios with different modalities, including recognition on 1D texts, 2D images, 3D point clouds, and semantic segmentation on 2D images.
SUPS: A Simulated Underground Parking Scenario Dataset for Autonomous Driving
However, there is a lack of underground parking scenario datasets with multiple sensors and well-labeled images that support both SLAM tasks and perception tasks, such as semantic segmentation and parking slot detection.
A Human-Centered Safe Robot Reinforcement Learning Framework with Interactive Behaviors
Deployment of Reinforcement Learning (RL) algorithms for robotics applications in the real world requires ensuring the safety of the robot and its environment.
Dual-Stream Transformer for Generic Event Boundary Captioning
This paper describes our champion solution for the CVPR2022 Generic Event Boundary Captioning (GEBC) competition.
A Review of Safe Reinforcement Learning: Methods, Theory and Applications
To establish a good foundation for future safe RL research, in this paper, we provide a review of safe RL from the perspectives of methods, theories, and applications.
Detect Rumors in Microblog Posts for Low-Resource Domains via Adversarial Contrastive Learning
Massive false rumors emerging along with breaking news or trending topics severely hinder the truth.
BMD: A General Class-balanced Multicentric Dynamic Prototype Strategy for Source-free Domain Adaptation
Source-free Domain Adaptation (SFDA) aims to adapt a pre-trained source model to the unlabeled target domain without accessing the well-labeled source data, which is a much more practical setting due to the data privacy, security, and transmission issues.
Unsupervised Domain Adaptation for Nighttime Aerial Tracking
Previous advances in object tracking mostly reported on favorable illumination circumstances while neglecting performance at nighttime, which significantly impeded the development of related aerial robot applications.
Rumor Detection on Twitter with Claim-Guided Hierarchical Graph Attention Networks
Rumors are rampant in the era of social media.
HRegNet: A Hierarchical Network for Large-scale Outdoor LiDAR Point Cloud Registration
Extensive experiments are conducted on two large-scale outdoor LiDAR point cloud datasets to demonstrate the high accuracy and efficiency of the proposed HRegNet.
DMInet: An Accurate and Highly Flexible Deep Learning Framework for Drug Membrane Interaction with Membrane Selectivity
Inheriting from coarse-grained Martini representation of organic molecules and combined with deep learning, DMInet has the potential for more accelerated high throughput screening in drug discovery across a much larger chemical space than that can be explored by physics-based simulations alone.
ACM-Net: Action Context Modeling Network for Weakly-Supervised Temporal Action Localization
Traditional methods mainly focus on foreground and background frames separation with only a single attention branch and class activation sequence.
Ranked #5 on
Weakly Supervised Action Localization
on THUMOS14
Data Augmentation for Object Detection via Differentiable Neural Rendering
We propose a new offline data augmentation method for object detection, which semantically interpolates the training data with novel views.
NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting
Although Transformer has made breakthrough success in widespread domains especially in Natural Language Processing (NLP), applying it to time series forecasting is still a great challenge.
Lightweight Convolutional Neural Network with Gaussian-based Grasping Representation for Robotic Grasping Detection
Extensive experiments on two public grasping datasets, Cornell and Jacquard demonstrate the state-of-the-art performance of our method in balancing accuracy and inference speed.
Ranked #1 on
Robotic Grasping
on Jacquard dataset
PointINet: Point Cloud Frame Interpolation Network
Generally, the frame rates of mechanical LiDAR sensors are 10 to 20 Hz, which is much lower than other commonly used sensors like cameras.
MoNet: Motion-based Point Cloud Prediction Network
3D point clouds accurately model 3D information of surrounding environment and are crucial for intelligent vehicles to perceive the scene.
LAP-Net: Adaptive Features Sampling via Learning Action Progression for Online Action Detection
At each time step, this sampling strategy first estimates current action progression and then decide what temporal ranges should be used to aggregate the optimal supplementary features.
RSKDD-Net: Random Sample-based Keypoint Detector and Descriptor
To tackle the information loss of random sampling, we exploit a novel random dilation cluster strategy to enlarge the receptive field of each sampled point and an attention mechanism to aggregate the positions and features of neighbor points.
PAN: Towards Fast Action Recognition via Learning Persistence of Appearance
In contrast to optical flow, our PA focuses more on distilling the motion information at boundaries.
Ranked #2 on
Action Recognition
on Jester (Gesture Recognition)
Efficient Pig Counting in Crowds with Keypoints Tracking and Spatial-aware Temporal Response Filtering
Existing methods only focused on pig counting using single image, and its accuracy is challenged by several factors, including pig movements, occlusion and overlapping.
Gen-LaneNet: A Generalized and Scalable Approach for 3D Lane Detection
The method, inspired by the latest state-of-the-art 3D-LaneNet, is a unified framework solving image encoding, spatial transform of features and 3D lane prediction in a single network.
Ranked #9 on
3D Lane Detection
on Apollo Synthetic 3D Lane
Indirect and Direct Training of Spiking Neural Networks for End-to-End Control of a Lane-Keeping Vehicle
Building spiking neural networks (SNNs) based on biological synaptic plasticities holds a promising potential for accomplishing fast and energy-efficient computing, which is beneficial to mobile robotic applications.
Neuromorphic Visual Odometry System for Intelligent Vehicle Application with Bio-inspired Vision Sensor
The neuromorphic camera is a brand new vision sensor that has emerged in recent years.
Robotics
Mechanisms underlying the response of mouse cortical networks to optogenetic manipulation
Networks with three inhibitory populations and V1-like architecture account for the data in ALM layer 2/3.
Globally optimal vertical direction estimation in Atlanta World
Accordingly, we propose a vertical direction estimation method by considering the relationship between the vertical frame and horizontal frames.
A Novel Method for the Absolute Pose Problem with Pairwise Constraints
In this paper, we consider pairwise constraints and propose a globally optimal algorithm for solving the absolute pose estimation problem.
Salience Biased Loss for Object Detection in Aerial Images
Experimental results show our proposed loss function with the RetinaNet architecture outperformed other state-of-art object detection models by at least 4. 31 mAP, and RetinaNet by 2. 26 mAP with the same inference speed of RetinaNet.
Deep Anticipation: Light Weight Intelligent Mobile Sensing in IoT by Recurrent Architecture
However, the mobile sensing perception brings new challenges for how to efficiently analyze and intelligently interpret the deluge of IoT data in mission- critical services.
Hierarchical Latent Semantic Mapping for Automated Topic Generation
Inspired by these algorithms, in this paper, we propose a novel method named Hierarchical Latent Semantic Mapping (HLSM), which automatically generates topics from corpus.
Large-Scale Visual Font Recognition
This paper addresses the large-scale visual font recognition (VFR) problem, which aims at automatic identification of the typeface, weight, and slope of the text in an image or photo without any knowledge of content.
Ranked #1 on
Font Recognition
on VFR-447
Detection Evolution with Multi-order Contextual Co-occurrence
The so-called (1 st -order) context feature is computed as a set of randomized binary comparisons on the response map of the baseline object detector.
