Search Results for author: Wei Zhan

Found 71 papers, 17 papers with code

Q-SLAM: Quadric Representations for Monocular SLAM

no code implementations12 Mar 2024 Chensheng Peng, Chenfeng Xu, Yue Wang, Mingyu Ding, Heng Yang, Masayoshi Tomizuka, Kurt Keutzer, Marco Pavone, Wei Zhan

This focus results in a significant disconnect between NeRF applications, i. e., novel-view synthesis and the requirements of SLAM.

3D Reconstruction Depth Estimation +2

Towards Generalizable and Interpretable Motion Prediction: A Deep Variational Bayes Approach

no code implementations10 Mar 2024 Juanwu Lu, Wei Zhan, Masayoshi Tomizuka, Yeping Hu

For interpretability, the model achieves target-driven motion prediction by estimating the spatial distribution of long-term destinations with a variational mixture of Gaussians.

Autonomous Vehicles motion prediction

PhyGrasp: Generalizing Robotic Grasping with Physics-informed Large Multimodal Models

no code implementations26 Feb 2024 Dingkun Guo, Yuqi Xiang, Shuqi Zhao, Xinghao Zhu, Masayoshi Tomizuka, Mingyu Ding, Wei Zhan

With these two capabilities, PhyGrasp is able to accurately assess the physical properties of object parts and determine optimal grasping poses.

Object Physical Commonsense Reasoning +1

BeTAIL: Behavior Transformer Adversarial Imitation Learning from Human Racing Gameplay

no code implementations22 Feb 2024 Catherine Weaver, Chen Tang, Ce Hao, Kenta Kawamoto, Masayoshi Tomizuka, Wei Zhan

Thus, we propose BeTAIL: Behavior Transformer Adversarial Imitation Learning, which combines a Behavior Transformer (BeT) policy from human demonstrations with online AIL.

Imitation Learning

Depth-aware Volume Attention for Texture-less Stereo Matching

1 code implementation14 Feb 2024 Tong Zhao, Mingyu Ding, Wei Zhan, Masayoshi Tomizuka, Yintao Wei

Furthermore, we propose a more rigorous evaluation metric that considers depth-wise relative error, providing comprehensive evaluations for universal stereo matching and depth estimation models.

Depth Estimation Stereo Matching

Controllable Safety-Critical Closed-loop Traffic Simulation via Guided Diffusion

no code implementations31 Dec 2023 Wei-Jer Chang, Francesco Pittaluga, Masayoshi Tomizuka, Wei Zhan, Manmohan Chandraker

These findings affirm that guided diffusion models provide a robust and versatile foundation for safety-critical, interactive traffic simulation, extending their utility across the broader landscape of autonomous driving.

Autonomous Driving Denoising

Quantifying Agent Interaction in Multi-agent Reinforcement Learning for Cost-efficient Generalization

no code implementations11 Oct 2023 Yuxin Chen, Chen Tang, Ran Tian, Chenran Li, Jinning Li, Masayoshi Tomizuka, Wei Zhan

We observe that, generally, a more diverse set of co-play agents during training enhances the generalization performance of the ego agent; however, this improvement varies across distinct scenarios and environments.

Multi-agent Reinforcement Learning

Human-oriented Representation Learning for Robotic Manipulation

no code implementations4 Oct 2023 Mingxiao Huo, Mingyu Ding, Chenfeng Xu, Thomas Tian, Xinghao Zhu, Yao Mu, Lingfeng Sun, Masayoshi Tomizuka, Wei Zhan

We introduce Task Fusion Decoder as a plug-and-play embedding translator that utilizes the underlying relationships among these perceptual skills to guide the representation learning towards encoding meaningful structure for what's important for all perceptual skills, ultimately empowering learning of downstream robotic manipulation tasks.

Hand Detection Representation Learning +1

LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving

no code implementations4 Oct 2023 Hao Sha, Yao Mu, YuXuan Jiang, Li Chen, Chenfeng Xu, Ping Luo, Shengbo Eben Li, Masayoshi Tomizuka, Wei Zhan, Mingyu Ding

Existing learning-based autonomous driving (AD) systems face challenges in comprehending high-level information, generalizing to rare events, and providing interpretability.

Autonomous Driving Decision Making

RSRD: A Road Surface Reconstruction Dataset and Benchmark for Safe and Comfortable Autonomous Driving

no code implementations3 Oct 2023 Tong Zhao, Chenfeng Xu, Mingyu Ding, Masayoshi Tomizuka, Wei Zhan, Yintao Wei

This paper addresses the growing demands for safety and comfort in intelligent robot systems, particularly autonomous vehicles, where road conditions play a pivotal role in overall driving performance.

Autonomous Driving Depth Estimation +3

Towards Free Data Selection with General-Purpose Models

1 code implementation NeurIPS 2023 Yichen Xie, Mingyu Ding, Masayoshi Tomizuka, Wei Zhan

However, current approaches, represented by active learning methods, typically follow a cumbersome pipeline that iterates the time-consuming model training and batch data selection repeatedly.

Active Learning

Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstration

no code implementations18 Sep 2023 Jinning Li, Xinyi Liu, Banghua Zhu, Jiantao Jiao, Masayoshi Tomizuka, Chen Tang, Wei Zhan

GOLD distills an offline DT policy into a lightweight policy network through guided online safe RL training, which outperforms both the offline DT policy and online safe RL algorithms.

Autonomous Driving Decision Making +3

DELFlow: Dense Efficient Learning of Scene Flow for Large-Scale Point Clouds

no code implementations ICCV 2023 Chensheng Peng, Guangming Wang, Xian Wan Lo, Xinrui Wu, Chenfeng Xu, Masayoshi Tomizuka, Wei Zhan, Hesheng Wang

Previous methods rarely predict scene flow from the entire point clouds of the scene with one-time inference due to the memory inefficiency and heavy overhead from distance calculation and sorting involved in commonly used farthest point sampling, KNN, and ball query algorithms for local feature aggregation.

Scene Flow Estimation

Residual Q-Learning: Offline and Online Policy Customization without Value

no code implementations NeurIPS 2023 Chenran Li, Chen Tang, Haruki Nishimura, Jean Mercat, Masayoshi Tomizuka, Wei Zhan

Specifically, we formulate the customization problem as a Markov Decision Process (MDP) with a reward function that combines 1) the inherent reward of the demonstration; and 2) the add-on reward specified by the downstream task.

Imitation Learning Q-Learning

Skill-Critic: Refining Learned Skills for Reinforcement Learning

no code implementations14 Jun 2023 Ce Hao, Catherine Weaver, Chen Tang, Kenta Kawamoto, Masayoshi Tomizuka, Wei Zhan

Hierarchical reinforcement learning (RL) can accelerate long-horizon decision-making by temporally abstracting a policy into multiple levels.

Decision Making Hierarchical Reinforcement Learning +2

Quadric Representations for LiDAR Odometry, Mapping and Localization

no code implementations27 Apr 2023 Chao Xia, Chenfeng Xu, Patrick Rim, Mingyu Ding, Nanning Zheng, Kurt Keutzer, Masayoshi Tomizuka, Wei Zhan

Current LiDAR odometry, mapping and localization methods leverage point-wise representations of 3D scenes and achieve high accuracy in autonomous driving tasks.

Autonomous Driving

Active Finetuning: Exploiting Annotation Budget in the Pretraining-Finetuning Paradigm

1 code implementation CVPR 2023 Yichen Xie, Han Lu, Junchi Yan, Xiaokang Yang, Masayoshi Tomizuka, Wei Zhan

We propose a novel method called ActiveFT for active finetuning task to select a subset of data distributing similarly with the entire unlabeled pool and maintaining enough diversity by optimizing a parametric model in the continuous space.

Image Classification Semantic Segmentation

Editing Driver Character: Socially-Controllable Behavior Generation for Interactive Traffic Simulation

no code implementations24 Mar 2023 Wei-Jer Chang, Chen Tang, Chenran Li, Yeping Hu, Masayoshi Tomizuka, Wei Zhan

To ensure that autonomous vehicles take safe and efficient maneuvers in different interactive traffic scenarios, we should be able to evaluate autonomous vehicles against reactive agents with different social characteristics in the simulation environment.

Autonomous Driving

UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling

2 code implementations13 Feb 2023 Haoyu Lu, Yuqi Huo, Guoxing Yang, Zhiwu Lu, Wei Zhan, Masayoshi Tomizuka, Mingyu Ding

Particularly, on the MSRVTT retrieval task, UniAdapter achieves 49. 7% recall@1 with 2. 2% model parameters, outperforming the latest competitors by 2. 0%.

Retrieval Text Retrieval +3

Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detection

1 code implementation5 Oct 2022 Jinhyung Park, Chenfeng Xu, Shijia Yang, Kurt Keutzer, Kris Kitani, Masayoshi Tomizuka, Wei Zhan

While recent camera-only 3D detection methods leverage multiple timesteps, the limited history they use significantly hampers the extent to which temporal fusion can improve object perception.

3D Object Detection object-detection +2

Center Feature Fusion: Selective Multi-Sensor Fusion of Center-based Objects

no code implementations26 Sep 2022 Philip Jacobson, Yiyang Zhou, Wei Zhan, Masayoshi Tomizuka, Ming C. Wu

In this work, we propose a novel approach Center Feature Fusion (CFF), in which we leverage center-based detection networks in both the camera and LiDAR streams to identify relevant object locations.

Autonomous Vehicles Object +3

Analyzing and Enhancing Closed-loop Stability in Reactive Simulation

no code implementations9 Aug 2022 Wei-Jer Chang, Yeping Hu, Chenran Li, Wei Zhan, Masayoshi Tomizuka

In this paper, we aim to provide a thorough stability analysis of the reactive simulation and propose a solution to enhance the stability.

Generalizability Analysis of Graph-based Trajectory Predictor with Vectorized Representation

no code implementations6 Aug 2022 Juanwu Lu, Wei Zhan, Masayoshi Tomizuka, Yeping Hu

Results show significant performance degradation due to domain shift, and feature attribution provides insights to identify potential causes of these problems.

Autonomous Vehicles Trajectory Prediction

What Matters for 3D Scene Flow Network

1 code implementation19 Jul 2022 Guangming Wang, Yunzhe Hu, Zhe Liu, Yiyang Zhou, Masayoshi Tomizuka, Wei Zhan, Hesheng Wang

Our proposed model surpasses all existing methods by at least 38. 2% on FlyingThings3D dataset and 24. 7% on KITTI Scene Flow dataset for EPE3D metric.

Scene Flow Estimation

SST-Calib: Simultaneous Spatial-Temporal Parameter Calibration between LIDAR and Camera

no code implementations8 Jul 2022 Akio Kodaira, Yiyang Zhou, Pengwei Zang, Wei Zhan, Masayoshi Tomizuka

With information from multiple input modalities, sensor fusion-based algorithms usually out-perform their single-modality counterparts in robotics.

Optical Flow Estimation Segmentation +2

PreTraM: Self-Supervised Pre-training via Connecting Trajectory and Map

1 code implementation21 Apr 2022 Chenfeng Xu, Tian Li, Chen Tang, Lingfeng Sun, Kurt Keutzer, Masayoshi Tomizuka, Alireza Fathi, Wei Zhan

It is hard to replicate these approaches in trajectory forecasting due to the lack of adequate trajectory data (e. g., 34K samples in the nuScenes dataset).

Contrastive Learning Representation Learning +1

Interventional Behavior Prediction: Avoiding Overly Confident Anticipation in Interactive Prediction

no code implementations19 Apr 2022 Chen Tang, Wei Zhan, Masayoshi Tomizuka

Moreover, to properly evaluate an IBP model with offline datasets, we propose a Shapley-value-based metric to verify if the prediction model satisfies the inherent temporal independence of an interventional distribution.

DetMatch: Two Teachers are Better Than One for Joint 2D and 3D Semi-Supervised Object Detection

1 code implementation17 Mar 2022 Jinhyung Park, Chenfeng Xu, Yiyang Zhou, Masayoshi Tomizuka, Wei Zhan

While numerous 3D detection works leverage the complementary relationship between RGB images and point clouds, developments in the broader framework of semi-supervised object recognition remain uninfluenced by multi-modal fusion.

object-detection Object Detection +2

Transferable and Adaptable Driving Behavior Prediction

no code implementations10 Feb 2022 Letian Wang, Yeping Hu, Liting Sun, Wei Zhan, Masayoshi Tomizuka, Changliu Liu

By mimicking humans' cognition model and semantic understanding during driving, we propose HATN, a hierarchical framework to generate high-quality, transferable, and adaptable predictions for driving behaviors in multi-agent dense-traffic environments.

Autonomous Vehicles Trajectory Prediction

Towards General and Efficient Active Learning

3 code implementations15 Dec 2021 Yichen Xie, Masayoshi Tomizuka, Wei Zhan

Existing work follows a cumbersome pipeline that repeats the time-consuming model training and batch data selection multiple times.

Active Learning Depth Estimation +4

Causal-based Time Series Domain Generalization for Vehicle Intention Prediction

no code implementations3 Dec 2021 Yeping Hu, Xiaogang Jia, Masayoshi Tomizuka, Wei Zhan

In this paper, we aim to address the domain generalization problem for vehicle intention prediction tasks and a causal-based time series domain generalization (CTSDG) model is proposed.

Autonomous Vehicles Domain Generalization +3

Exploring Social Posterior Collapse in Variational Autoencoder for Interaction Modeling

no code implementations NeurIPS 2021 Chen Tang, Wei Zhan, Masayoshi Tomizuka

In this work, we argue that one of the typical formulations of VAEs in multi-agent modeling suffers from an issue we refer to as social posterior collapse, i. e., the model is prone to ignoring historical social context when predicting the future trajectory of an agent.

Graph Attention Trajectory Forecasting

Dealing with the Unknown: Pessimistic Offline Reinforcement Learning

no code implementations9 Nov 2021 Jinning Li, Chen Tang, Masayoshi Tomizuka, Wei Zhan

Reinforcement Learning (RL) has been shown effective in domains where the agent can learn policies by actively interacting with its operating environment.

reinforcement-learning Reinforcement Learning (RL)

A Simple and Efficient Multi-task Network for 3D Object Detection and Road Understanding

1 code implementation6 Mar 2021 Di Feng, Yiyang Zhou, Chenfeng Xu, Masayoshi Tomizuka, Wei Zhan

Detecting dynamic objects and predicting static road information such as drivable areas and ground heights are crucial for safe autonomous driving.

3D Object Detection Autonomous Driving +1

A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning

no code implementations17 Jan 2021 Jinning Li, Liting Sun, Jianyu Chen, Masayoshi Tomizuka, Wei Zhan

To address this challenge, we propose a hierarchical behavior planning framework with a set of low-level safe controllers and a high-level reinforcement learning algorithm (H-CtRL) as a coordinator for the low-level controllers.

Autonomous Vehicles reinforcement-learning +1

Labels Are Not Perfect: Inferring Spatial Uncertainty in Object Detection

no code implementations18 Dec 2020 Di Feng, Zining Wang, Yiyang Zhou, Lars Rosenbaum, Fabian Timm, Klaus Dietmayer, Masayoshi Tomizuka, Wei Zhan

As a result, an in-depth evaluation among different object detection methods remains challenging, and the training process of object detectors is sub-optimal, especially in probabilistic object detection.

Autonomous Driving Object +2

Sparse R-CNN: End-to-End Object Detection with Learnable Proposals

6 code implementations CVPR 2021 Peize Sun, Rufeng Zhang, Yi Jiang, Tao Kong, Chenfeng Xu, Wei Zhan, Masayoshi Tomizuka, Lei LI, Zehuan Yuan, Changhu Wang, Ping Luo

In our method, however, a fixed sparse set of learned object proposals, total length of $N$, are provided to object recognition head to perform classification and location.

Object object-detection +2

IDE-Net: Interactive Driving Event and Pattern Extraction from Human Data

no code implementations4 Nov 2020 Xiaosong Jia, Liting Sun, Masayoshi Tomizuka, Wei Zhan

We find three interpretable patterns of interactions, bringing insights for driver behavior representation, modeling and comprehension.

Autonomous Vehicles Multi-Task Learning

Socially-Compatible Behavior Design of Autonomous Vehicles with Verification on Real Human Data

no code implementations28 Oct 2020 Letian Wang, Liting Sun, Masayoshi Tomizuka, Wei Zhan

It allows the AVs to infer the characteristics of other road users online and generate behaviors optimizing not only their own rewards, but also their courtesy to others, and their confidence regarding the prediction uncertainties.

Autonomous Vehicles

Towards Better Performance and More Explainable Uncertainty for 3D Object Detection of Autonomous Vehicles

no code implementations22 Jun 2020 Hujie Pan, Zining Wang, Wei Zhan, Masayoshi Tomizuka

In this paper, we propose a novel form of the loss function to increase the performance of LiDAR-based 3d object detection and obtain more explainable and convincing uncertainty for the prediction.

3D Object Detection Autonomous Vehicles +1

Efficient Sampling-Based Maximum Entropy Inverse Reinforcement Learning with Application to Autonomous Driving

no code implementations22 Jun 2020 Zheng Wu, Liting Sun, Wei Zhan, Chenyu Yang, Masayoshi Tomizuka

Different from existing IRL algorithms, by introducing an efficient continuous-domain trajectory sampler, the proposed algorithm can directly learn the reward functions in the continuous domain while considering the uncertainties in demonstrated trajectories from human drivers.

Autonomous Driving reinforcement-learning +1

Scenario-Transferable Semantic Graph Reasoning for Interaction-Aware Probabilistic Prediction

no code implementations7 Apr 2020 Yeping Hu, Wei Zhan, Masayoshi Tomizuka

Accurately predicting the possible behaviors of traffic participants is an essential capability for autonomous vehicles.

Autonomous Driving Navigate

SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentation

3 code implementations ECCV 2020 Chenfeng Xu, Bichen Wu, Zining Wang, Wei Zhan, Peter Vajda, Kurt Keutzer, Masayoshi Tomizuka

Using standard convolutions to process such LiDAR images is problematic, as convolution filters pick up local features that are only active in specific regions in the image.

3D Semantic Segmentation Point Cloud Segmentation +1

Inferring Spatial Uncertainty in Object Detection

no code implementations7 Mar 2020 Zining Wang, Di Feng, Yiyang Zhou, Lars Rosenbaum, Fabian Timm, Klaus Dietmayer, Masayoshi Tomizuka, Wei Zhan

Based on the spatial distribution, we further propose an extension of IoU, called the Jaccard IoU (JIoU), as a new evaluation metric that incorporates label uncertainty.

Autonomous Driving Object +2

AutoScale: Learning to Scale for Crowd Counting and Localization

2 code implementations20 Dec 2019 Chenfeng Xu, Dingkang Liang, Yongchao Xu, Song Bai, Wei Zhan, Xiang Bai, Masayoshi Tomizuka

A major issue is that the density map on dense regions usually accumulates density values from a number of nearby Gaussian blobs, yielding different large density values on a small set of pixels.

Crowd Counting Model Optimization

Interpretable Modelling of Driving Behaviors in Interactive Driving Scenarios based on Cumulative Prospect Theory

no code implementations19 Jul 2019 Liting Sun, Wei Zhan, Yeping Hu, Masayoshi Tomizuka

Hence, the goal of this work is to formulate the human drivers' behavior generation model with CPT so that some ``irrational'' behavior or decisions of human can be better captured and predicted.

Autonomous Vehicles Decision Making

Behavior Planning of Autonomous Cars with Social Perception

no code implementations2 May 2019 Liting Sun, Wei Zhan, Ching-Yao Chan, Masayoshi Tomizuka

The uncertainties can come either from sensor limitations such as occlusions and limited sensor range, or from probabilistic prediction of other road participants, or from unknown social behavior in a new area.

Autonomous Vehicles Decision Making +2

Coordination and Trajectory Prediction for Vehicle Interactions via Bayesian Generative Modeling

no code implementations2 May 2019 Jiachen Li, Hengbo Ma, Wei Zhan, Masayoshi Tomizuka

In order to tackle the task of probabilistic prediction for multiple, interactive entities, we propose a coordination and trajectory prediction system (CTPS), which has a hierarchical structure including a macro-level coordination recognition module and a micro-level subtle pattern prediction module which solves a probabilistic generation task.

Trajectory Prediction

Multi-modal Probabilistic Prediction of Interactive Behavior via an Interpretable Model

no code implementations22 Mar 2019 Yeping Hu, Wei Zhan, Liting Sun, Masayoshi Tomizuka

The proposed method is based on a generative model and is capable of jointly predicting sequential motions of each pair of interacting agents.

valid

A Framework for Probabilistic Generic Traffic Scene Prediction

no code implementations30 Oct 2018 Yeping Hu, Wei Zhan, Masayoshi Tomizuka

In a given scenario, simultaneously and accurately predicting every possible interaction of traffic participants is an important capability for autonomous vehicles.

Autonomous Vehicles Decision Making +1

Towards a Fatality-Aware Benchmark of Probabilistic Reaction Prediction in Highly Interactive Driving Scenarios

no code implementations10 Sep 2018 Wei Zhan, Liting Sun, Yeping Hu, Jiachen Li, Masayoshi Tomizuka

Modified methods based on PGM, NN and IRL are provided to generate probabilistic reaction predictions in an exemplar scenario of nudging from a highway ramp.

Autonomous Vehicles Decision Making

Generic Probabilistic Interactive Situation Recognition and Prediction: From Virtual to Real

no code implementations9 Sep 2018 Jiachen Li, Hengbo Ma, Wei Zhan, Masayoshi Tomizuka

Accurate and robust recognition and prediction of traffic situation plays an important role in autonomous driving, which is a prerequisite for risk assessment and effective decision making.

Autonomous Driving Decision Making +1

Probabilistic Prediction of Interactive Driving Behavior via Hierarchical Inverse Reinforcement Learning

no code implementations9 Sep 2018 Liting Sun, Wei Zhan, Masayoshi Tomizuka

To safely and efficiently interact with other road participants, AVs have to accurately predict the behavior of surrounding vehicles and plan accordingly.

Autonomous Vehicles reinforcement-learning +1

Courteous Autonomous Cars

no code implementations8 Aug 2018 Liting Sun, Wei Zhan, Masayoshi Tomizuka, Anca D. Dragan

Such a courtesy term enables the robot car to be aware of possible irrationality of the human behavior, and plan accordingly.

Probabilistic Prediction of Vehicle Semantic Intention and Motion

no code implementations10 Apr 2018 Yeping Hu, Wei Zhan, Masayoshi Tomizuka

Accurately predicting the possible behaviors of traffic participants is an essential capability for future autonomous vehicles.

Autonomous Vehicles motion prediction

Fusing Bird View LIDAR Point Cloud and Front View Camera Image for Deep Object Detection

no code implementations17 Nov 2017 Zining Wang, Wei Zhan, Masayoshi Tomizuka

The fusion method shows particular benefit for detection of pedestrians in the bird view compared to other fusion-based object detection networks.

3D Object Detection Autonomous Driving +2

A Fast Integrated Planning and Control Framework for Autonomous Driving via Imitation Learning

no code implementations9 Jul 2017 Liting Sun, Cheng Peng, Wei Zhan, Masayoshi Tomizuka

For safe and efficient planning and control in autonomous driving, we need a driving policy which can achieve desirable driving quality in long-term horizon with guaranteed safety and feasibility.

Autonomous Driving Imitation Learning +1

Cannot find the paper you are looking for? You can Submit a new open access paper.