Search Results for author: Runsheng Xu

Found 33 papers, 18 papers with code

EMMA: End-to-End Multimodal Model for Autonomous Driving

no code implementations30 Oct 2024 Jyh-Jing Hwang, Runsheng Xu, Hubert Lin, Wei-Chih Hung, Jingwei Ji, Kristy Choi, Di Huang, Tong He, Paul Covington, Benjamin Sapp, Yin Zhou, James Guo, Dragomir Anguelov, Mingxing Tan

We show that co-training EMMA with planner trajectories, object detection, and road graph tasks yields improvements across all three domains, highlighting EMMA's potential as a generalist model for autonomous driving applications.

3D Object Detection Autonomous Driving +4

Light the Night: A Multi-Condition Diffusion Framework for Unpaired Low-Light Enhancement in Autonomous Driving

no code implementations CVPR 2024 Jinlong Li, Baolu Li, Zhengzhong Tu, Xinyu Liu, Qing Guo, Felix Juefei-Xu, Runsheng Xu, Hongkai Yu

Vision-centric perception systems for autonomous driving have gained considerable attention recently due to their cost-effectiveness and scalability, especially compared to LiDAR-based systems.

Autonomous Driving

V2X-Real: a Largs-Scale Dataset for Vehicle-to-Everything Cooperative Perception

no code implementations24 Mar 2024 Hao Xiang, Zhaoliang Zheng, Xin Xia, Runsheng Xu, Letian Gao, Zewei Zhou, Xu Han, Xinkai Ji, Mingxi Li, Zonglin Meng, Li Jin, Mingyue Lei, Zhaoyang Ma, Zihang He, Haoxuan Ma, Yunshuang Yuan, Yingqian Zhao, Jiaqi Ma

Recent advancements in Vehicle-to-Everything (V2X) technologies have enabled autonomous vehicles to share sensing information to see through occlusions, greatly boosting the perception capability.

Autonomous Vehicles

V2X-DGW: Domain Generalization for Multi-agent Perception under Adverse Weather Conditions

no code implementations17 Mar 2024 Baolu Li, Jinlong Li, Xinyu Liu, Runsheng Xu, Zhengzhong Tu, Jiacheng Guo, Xiaopeng Li, Hongkai Yu

In this paper, we propose a Domain Generalization based approach, named V2X-DGW, for LiDAR-based 3D object detection on multi-agent perception system under adverse weather conditions.

3D Object Detection Domain Generalization +2

Breaking Data Silos: Cross-Domain Learning for Multi-Agent Perception from Independent Private Sources

1 code implementation6 Feb 2024 Jinlong Li, Baolu Li, Xinyu Liu, Runsheng Xu, Jiaqi Ma, Hongkai Yu

However, the data source to train the various agents is independent and private in each company, leading to the Distribution Gap of different private data for training distinct agents in multi-agent perception system.

3D Object Detection object-detection

DUSA: Decoupled Unsupervised Sim2Real Adaptation for Vehicle-to-Everything Collaborative Perception

1 code implementation12 Oct 2023 Xianghao Kong, Wentao Jiang, Jinrang Jia, Yifeng Shi, Runsheng Xu, Si Liu

To take full advantage of simulated data, we present a new unsupervised sim2real domain adaptation method for V2X collaborative detection named Decoupled Unsupervised Sim2Real Adaptation (DUSA).

Autonomous Driving Domain Adaptation

Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perception

no code implementations31 Aug 2023 Si Liu, Chen Gao, Yuan Chen, Xingyu Peng, Xianghao Kong, Kun Wang, Runsheng Xu, Wentao Jiang, Hao Xiang, Jiaqi Ma, Miao Wang

Specifically, we analyze the performance changes of different methods under different bandwidths, providing a deep insight into the performance-bandwidth trade-off issue.

Autonomous Driving

Domain Adaptation based Object Detection for Autonomous Driving in Foggy and Rainy Weather

1 code implementation18 Jul 2023 Jinlong Li, Runsheng Xu, Xinyu Liu, Jin Ma, Baolu Li, Qin Zou, Jiaqi Ma, Hongkai Yu

To bridge the domain gap and improve the performance of object detection in foggy and rainy weather, this paper presents a novel framework for domain-adaptive object detection.

Autonomous Driving Data Augmentation +4

S2R-ViT for Multi-Agent Cooperative Perception: Bridging the Gap from Simulation to Reality

no code implementations16 Jul 2023 Jinlong Li, Runsheng Xu, Xinyu Liu, Baolu Li, Qin Zou, Jiaqi Ma, Hongkai Yu

We investigate the effects of these two types of domain gaps and propose a novel uncertainty-aware vision transformer to effectively relief the Deployment Gap and an agent-based feature adaptation module with inter-agent and ego-agent discriminators to reduce the Feature Gap.

3D Object Detection object-detection +1

HM-ViT: Hetero-modal Vehicle-to-Vehicle Cooperative perception with vision transformer

1 code implementation ICCV 2023 Hao Xiang, Runsheng Xu, Jiaqi Ma

We present HM-ViT, the first unified multi-agent hetero-modal cooperative perception framework that can collaboratively predict 3D objects for highly dynamic vehicle-to-vehicle (V2V) collaborations with varying numbers and types of agents.

Autonomous Vehicles

FedBEVT: Federated Learning Bird's Eye View Perception Transformer in Road Traffic Systems

1 code implementation4 Apr 2023 Rui Song, Runsheng Xu, Andreas Festag, Jiaqi Ma, Alois Knoll

Our findings suggest that FedBEVT outperforms the baseline approaches in all four use cases, demonstrating the potential of our approach for improving BEV perception in autonomous driving.

Autonomous Driving Federated Learning

Collaboration Helps Camera Overtake LiDAR in 3D Detection

1 code implementation CVPR 2023 Yue Hu, Yifan Lu, Runsheng Xu, Weidi Xie, Siheng Chen, Yanfeng Wang

Camera-only 3D detection provides an economical solution with a simple configuration for localizing objects in 3D space compared to LiDAR-based detection systems.

Depth Estimation

Street-View Image Generation from a Bird's-Eye View Layout

1 code implementation11 Jan 2023 Alexander Swerdlow, Runsheng Xu, Bolei Zhou

Instead of using perception data from real-life scenarios, an ideal model for simulation would generate realistic street-view images that align with a given HD map and traffic layout, a task that is critical for visualizing complex traffic scenarios and developing robust perception models for autonomous driving.

Autonomous Driving Image Generation

Learning for Vehicle-to-Vehicle Cooperative Perception under Lossy Communication

1 code implementation16 Dec 2022 Jinlong Li, Runsheng Xu, Xinyu Liu, Jin Ma, Zicheng Chi, Jiaqi Ma, Hongkai Yu

Due to the beneficial Vehicle-to-Vehicle (V2V) communication, the deep learning based features from other agents can be shared to the ego vehicle so as to improve the perception of the ego vehicle.

3D Object Detection object-detection

Analyzing Infrastructure LiDAR Placement with Realistic LiDAR Simulation Library

2 code implementations29 Nov 2022 Xinyu Cai, Wentao Jiang, Runsheng Xu, Wenquan Zhao, Jiaqi Ma, Si Liu, Yikang Li

Through simulating point cloud data in different LiDAR placements, we can evaluate the perception accuracy of these placements using multiple detection models.

Domain Adaptive Object Detection for Autonomous Driving under Foggy Weather

1 code implementation27 Oct 2022 Jinlong Li, Runsheng Xu, Jin Ma, Qin Zou, Jiaqi Ma, Hongkai Yu

This paper proposes a novel domain adaptive object detection framework for autonomous driving under foggy weather.

Autonomous Driving Data Augmentation +4

Bridging the Domain Gap for Multi-Agent Perception

1 code implementation16 Oct 2022 Runsheng Xu, Jinlong Li, Xiaoyu Dong, Hongkai Yu, Jiaqi Ma

Existing multi-agent perception algorithms usually select to share deep neural features extracted from raw sensing data between agents, achieving a trade-off between accuracy and communication bandwidth limit.

3D Object Detection Domain Adaptation +1

V2XP-ASG: Generating Adversarial Scenes for Vehicle-to-Everything Perception

1 code implementation27 Sep 2022 Hao Xiang, Runsheng Xu, Xin Xia, Zhaoliang Zheng, Bolei Zhou, Jiaqi Ma

Recent advancements in Vehicle-to-Everything communication technology have enabled autonomous vehicles to share sensory information to obtain better perception performance.

Autonomous Vehicles

Diffusion Models: A Comprehensive Survey of Methods and Applications

2 code implementations2 Sep 2022 Ling Yang, Zhilong Zhang, Yang song, Shenda Hong, Runsheng Xu, Yue Zhao, Yingxia Shao, Wentao Zhang, Bin Cui, Ming-Hsuan Yang

This survey aims to provide a contextualized, in-depth look at the state of diffusion models, identifying the key areas of focus and pointing to potential areas for further exploration.

Image Super-Resolution Survey +2

CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers

2 code implementations5 Jul 2022 Runsheng Xu, Zhengzhong Tu, Hao Xiang, Wei Shao, Bolei Zhou, Jiaqi Ma

The extensive experiments on the V2V perception dataset, OPV2V, demonstrate that CoBEVT achieves state-of-the-art performance for cooperative BEV semantic segmentation.

3D Object Detection Autonomous Driving +2

Pik-Fix: Restoring and Colorizing Old Photos

1 code implementation4 May 2022 Runsheng Xu, Zhengzhong Tu, Yuanqi Du, Xiaoyu Dong, Jinlong Li, Zibo Meng, Jiaqi Ma, Alan Bovik, Hongkai Yu

Our proposed framework consists of three modules: a restoration sub-network that conducts restoration from degradations, a similarity network that performs color histogram matching and color transfer, and a colorization subnet that learns to predict the chroma elements of images conditioned on chromatic reference signals.

Colorization

V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer

1 code implementation20 Mar 2022 Runsheng Xu, Hao Xiang, Zhengzhong Tu, Xin Xia, Ming-Hsuan Yang, Jiaqi Ma

In this paper, we investigate the application of Vehicle-to-Everything (V2X) communication to improve the perception performance of autonomous vehicles.

3D Object Detection Autonomous Vehicles +1

ROMNet: Renovate the Old Memories

no code implementations5 Feb 2022 Runsheng Xu, Zhengzhong Tu, Yuanqi Du, Xiaoyu Dong, Jinlong Li, Zibo Meng, Jiaqi Ma, Hongkai Yu

Renovating the memories in old photos is an intriguing research topic in computer vision fields.

Colorization

OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication

2 code implementations16 Sep 2021 Runsheng Xu, Hao Xiang, Xin Xia, Xu Han, Jinlong Li, Jiaqi Ma

We then construct a comprehensive benchmark with a total of 16 implemented models to evaluate several information fusion strategies~(i. e. early, late, and intermediate fusion) with state-of-the-art LiDAR detection algorithms.

3D Object Detection Benchmarking

Hierarchical Road Topology Learning for Urban Map-less Driving

no code implementations31 Mar 2021 Li Zhang, Faezeh Tafazzoli, Gunther Krehl, Runsheng Xu, Timo Rehfeld, Manuel Schier, Arunava Seal

The majority of current approaches in autonomous driving rely on High-Definition (HD) maps which detail the road geometry and surrounding area.

Autonomous Driving

Holistic Grid Fusion Based Stop Line Estimation

no code implementations18 Sep 2020 Runsheng Xu, Faezeh Tafazzoli, Li Zhang, Timo Rehfeld, Gunther Krehl, Arunava Seal

Intersection scenarios provide the most complex traffic situations in Autonomous Driving and Driving Assistance Systems.

Autonomous Driving

GIA-Net: Global Information Aware Network for Low-light Imaging

no code implementations14 Sep 2020 Zibo Meng, Runsheng Xu, Chiu Man Ho

In this paper, we propose a global information aware (GIA) module, which is capable of extracting and integrating the global information into the network to improve the performance of low-light imaging.

Towards Better Driver Safety: Empowering Personal Navigation Technologies with Road Safety Awareness

no code implementations5 Jun 2020 Runsheng Xu, Allen Yilun Lin, Shibo Zhang, Peixi Xiong, Brent Hecht

Recent research has found that navigation systems usually assume that all roads are equally safe, directing drivers to dangerous routes, which led to catastrophic consequences.

Human-Computer Interaction

Lane Boundary Geometry Extraction from Satellite Imagery

no code implementations6 Feb 2020 Andi Zang, Runsheng Xu, Zichen Li, David Doria

Autonomous driving car is becoming more of a reality, as a key component, high-definition(HD) maps shows its value in both market place and industry.

Autonomous Driving

NeckSense: A Multi-Sensor Necklace for Detecting Eating Activities in Free-Living Conditions

no code implementations17 Nov 2019 Shibo Zhang, Yuqi Zhao, Dzung Tri Nguyen, Runsheng Xu, Sougata Sen, Josiah Hester, Nabil Alshurafa

Moreover, our system can achieve a F1-score of 77. 1% for episodes even in an all-day-around free-living setting.

Cannot find the paper you are looking for? You can Submit a new open access paper.