Search Results for author: Masayoshi Tomizuka

Found 128 papers, 30 papers with code

The Feasibility of Constrained Reinforcement Learning Algorithms: A Tutorial Study

no code implementations15 Apr 2024 Yujie Yang, Zhilong Zheng, Shengbo Eben Li, Masayoshi Tomizuka, Changliu Liu

We demonstrate our feasibility theory by visualizing different feasible regions under both MPC and RL policies in an emergency braking control task.

Model Predictive Control reinforcement-learning +1

RoadBEV: Road Surface Reconstruction in Bird's Eye View

1 code implementation9 Apr 2024 Tong Zhao, Lei Yang, Yichen Xie, Mingyu Ding, Masayoshi Tomizuka, Yintao Wei

This paper uniformly proposes two simple yet effective models for road elevation reconstruction in BEV named RoadBEV-mono and RoadBEV-stereo, which estimate road elevation with monocular and stereo images, respectively.

Autonomous Driving Monocular Depth Estimation +2

Q-SLAM: Quadric Representations for Monocular SLAM

no code implementations12 Mar 2024 Chensheng Peng, Chenfeng Xu, Yue Wang, Mingyu Ding, Heng Yang, Masayoshi Tomizuka, Kurt Keutzer, Marco Pavone, Wei Zhan

This focus results in a significant disconnect between NeRF applications, i. e., novel-view synthesis and the requirements of SLAM.

3D Reconstruction Depth Estimation +2

Towards Generalizable and Interpretable Motion Prediction: A Deep Variational Bayes Approach

no code implementations10 Mar 2024 Juanwu Lu, Wei Zhan, Masayoshi Tomizuka, Yeping Hu

For interpretability, the model achieves target-driven motion prediction by estimating the spatial distribution of long-term destinations with a variational mixture of Gaussians.

Autonomous Vehicles motion prediction

MATRIX: Multi-Agent Trajectory Generation with Diverse Contexts

no code implementations9 Mar 2024 Zhuo Xu, Rui Zhou, Yida Yin, Huidong Gao, Masayoshi Tomizuka, Jiachen Li

Data-driven methods have great advantages in modeling complicated human behavioral dynamics and dealing with many human-robot interaction applications.

Data Augmentation Motion Planning

PhyGrasp: Generalizing Robotic Grasping with Physics-informed Large Multimodal Models

no code implementations26 Feb 2024 Dingkun Guo, Yuqi Xiang, Shuqi Zhao, Xinghao Zhu, Masayoshi Tomizuka, Mingyu Ding, Wei Zhan

With these two capabilities, PhyGrasp is able to accurately assess the physical properties of object parts and determine optimal grasping poses.

Object Physical Commonsense Reasoning +1

BeTAIL: Behavior Transformer Adversarial Imitation Learning from Human Racing Gameplay

no code implementations22 Feb 2024 Catherine Weaver, Chen Tang, Ce Hao, Kenta Kawamoto, Masayoshi Tomizuka, Wei Zhan

Thus, we propose BeTAIL: Behavior Transformer Adversarial Imitation Learning, which combines a Behavior Transformer (BeT) policy from human demonstrations with online AIL.

Imitation Learning

Depth-aware Volume Attention for Texture-less Stereo Matching

1 code implementation14 Feb 2024 Tong Zhao, Mingyu Ding, Wei Zhan, Masayoshi Tomizuka, Yintao Wei

Furthermore, we propose a more rigorous evaluation metric that considers depth-wise relative error, providing comprehensive evaluations for universal stereo matching and depth estimation models.

Depth Estimation Stereo Matching

Controllable Safety-Critical Closed-loop Traffic Simulation via Guided Diffusion

no code implementations31 Dec 2023 Wei-Jer Chang, Francesco Pittaluga, Masayoshi Tomizuka, Wei Zhan, Manmohan Chandraker

These findings affirm that guided diffusion models provide a robust and versatile foundation for safety-critical, interactive traffic simulation, extending their utility across the broader landscape of autonomous driving.

Autonomous Driving Denoising

SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Execution

1 code implementation18 Dec 2023 Zhixuan Liang, Yao Mu, Hengbo Ma, Masayoshi Tomizuka, Mingyu Ding, Ping Luo

Experiments on multi-task robotic manipulation benchmarks like Meta-World and LOReL demonstrate state-of-the-art performance and human-interpretable skill representations from SkillDiffuser.

Trajectory Planning

Multi-level Reasoning for Robotic Assembly: From Sequence Inference to Contact Selection

no code implementations17 Dec 2023 Xinghao Zhu, Devesh K. Jha, Diego Romeres, Lingfeng Sun, Masayoshi Tomizuka, Anoop Cherian

Automating the assembly of objects from their parts is a complex problem with innumerable applications in manufacturing, maintenance, and recycling.

Motion Planning valid

Quantifying Agent Interaction in Multi-agent Reinforcement Learning for Cost-efficient Generalization

no code implementations11 Oct 2023 Yuxin Chen, Chen Tang, Ran Tian, Chenran Li, Jinning Li, Masayoshi Tomizuka, Wei Zhan

We observe that, generally, a more diverse set of co-play agents during training enhances the generalization performance of the ego agent; however, this improvement varies across distinct scenarios and environments.

Multi-agent Reinforcement Learning

Human-oriented Representation Learning for Robotic Manipulation

no code implementations4 Oct 2023 Mingxiao Huo, Mingyu Ding, Chenfeng Xu, Thomas Tian, Xinghao Zhu, Yao Mu, Lingfeng Sun, Masayoshi Tomizuka, Wei Zhan

We introduce Task Fusion Decoder as a plug-and-play embedding translator that utilizes the underlying relationships among these perceptual skills to guide the representation learning towards encoding meaningful structure for what's important for all perceptual skills, ultimately empowering learning of downstream robotic manipulation tasks.

Hand Detection Representation Learning +1

LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving

no code implementations4 Oct 2023 Hao Sha, Yao Mu, YuXuan Jiang, Li Chen, Chenfeng Xu, Ping Luo, Shengbo Eben Li, Masayoshi Tomizuka, Wei Zhan, Mingyu Ding

Existing learning-based autonomous driving (AD) systems face challenges in comprehending high-level information, generalizing to rare events, and providing interpretability.

Autonomous Driving Decision Making

Generalizable Long-Horizon Manipulations with Large Language Models

no code implementations3 Oct 2023 Haoyu Zhou, Mingyu Ding, Weikun Peng, Masayoshi Tomizuka, Lin Shao, Chuang Gan

This work introduces a framework harnessing the capabilities of Large Language Models (LLMs) to generate primitive task conditions for generalizable long-horizon manipulations with novel objects and unseen tasks.

RSRD: A Road Surface Reconstruction Dataset and Benchmark for Safe and Comfortable Autonomous Driving

no code implementations3 Oct 2023 Tong Zhao, Chenfeng Xu, Mingyu Ding, Masayoshi Tomizuka, Wei Zhan, Yintao Wei

This paper addresses the growing demands for safety and comfort in intelligent robot systems, particularly autonomous vehicles, where road conditions play a pivotal role in overall driving performance.

Autonomous Driving Depth Estimation +3

Towards Free Data Selection with General-Purpose Models

1 code implementation NeurIPS 2023 Yichen Xie, Mingyu Ding, Masayoshi Tomizuka, Wei Zhan

However, current approaches, represented by active learning methods, typically follow a cumbersome pipeline that iterates the time-consuming model training and batch data selection repeatedly.

Active Learning

Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstration

no code implementations18 Sep 2023 Jinning Li, Xinyi Liu, Banghua Zhu, Jiantao Jiao, Masayoshi Tomizuka, Chen Tang, Wei Zhan

GOLD distills an offline DT policy into a lightweight policy network through guided online safe RL training, which outperforms both the offline DT policy and online safe RL algorithms.

Autonomous Driving Decision Making +3

DELFlow: Dense Efficient Learning of Scene Flow for Large-Scale Point Clouds

1 code implementation ICCV 2023 Chensheng Peng, Guangming Wang, Xian Wan Lo, Xinrui Wu, Chenfeng Xu, Masayoshi Tomizuka, Wei Zhan, Hesheng Wang

Previous methods rarely predict scene flow from the entire point clouds of the scene with one-time inference due to the memory inefficiency and heavy overhead from distance calculation and sorting involved in commonly used farthest point sampling, KNN, and ball query algorithms for local feature aggregation.

Scene Flow Estimation

Residual Q-Learning: Offline and Online Policy Customization without Value

no code implementations NeurIPS 2023 Chenran Li, Chen Tang, Haruki Nishimura, Jean Mercat, Masayoshi Tomizuka, Wei Zhan

Specifically, we formulate the customization problem as a Markov Decision Process (MDP) with a reward function that combines 1) the inherent reward of the demonstration; and 2) the add-on reward specified by the downstream task.

Imitation Learning Q-Learning

Skill-Critic: Refining Learned Skills for Reinforcement Learning

no code implementations14 Jun 2023 Ce Hao, Catherine Weaver, Chen Tang, Kenta Kawamoto, Masayoshi Tomizuka, Wei Zhan

Hierarchical reinforcement learning (RL) can accelerate long-horizon decision-making by temporally abstracting a policy into multiple levels.

Decision Making Hierarchical Reinforcement Learning +2

Efficient Multi-Task and Transfer Reinforcement Learning with Parameter-Compositional Framework

no code implementations2 Jun 2023 Lingfeng Sun, Haichao Zhang, Wei Xu, Masayoshi Tomizuka

In this work, we investigate the potential of improving multi-task training and also leveraging it for transferring in the reinforcement learning setting.

reinforcement-learning Transfer Reinforcement Learning

Quadric Representations for LiDAR Odometry, Mapping and Localization

no code implementations27 Apr 2023 Chao Xia, Chenfeng Xu, Patrick Rim, Mingyu Ding, Nanning Zheng, Kurt Keutzer, Masayoshi Tomizuka, Wei Zhan

Current LiDAR odometry, mapping and localization methods leverage point-wise representations of 3D scenes and achieve high accuracy in autonomous driving tasks.

Autonomous Driving

Open-Vocabulary Point-Cloud Object Detection without 3D Annotation

1 code implementation CVPR 2023 Yuheng Lu, Chenfeng Xu, Xiaobao Wei, Xiaodong Xie, Masayoshi Tomizuka, Kurt Keutzer, Shanghang Zhang

In this paper, we address open-vocabulary 3D point-cloud detection by a dividing-and-conquering strategy, which involves: 1) developing a point-cloud detector that can learn a general representation for localizing various objects, and 2) connecting textual and point-cloud representations to enable the detector to classify novel object categories based on text prompting.

3D Object Detection 3D Open-Vocabulary Object Detection +3

Active Finetuning: Exploiting Annotation Budget in the Pretraining-Finetuning Paradigm

1 code implementation CVPR 2023 Yichen Xie, Han Lu, Junchi Yan, Xiaokang Yang, Masayoshi Tomizuka, Wei Zhan

We propose a novel method called ActiveFT for active finetuning task to select a subset of data distributing similarly with the entire unlabeled pool and maintaining enough diversity by optimizing a parametric model in the continuous space.

Image Classification Semantic Segmentation

Editing Driver Character: Socially-Controllable Behavior Generation for Interactive Traffic Simulation

no code implementations24 Mar 2023 Wei-Jer Chang, Chen Tang, Chenran Li, Yeping Hu, Masayoshi Tomizuka, Wei Zhan

To ensure that autonomous vehicles take safe and efficient maneuvers in different interactive traffic scenarios, we should be able to evaluate autonomous vehicles against reactive agents with different social characteristics in the simulation environment.

Autonomous Driving

UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling

2 code implementations13 Feb 2023 Haoyu Lu, Yuqi Huo, Guoxing Yang, Zhiwu Lu, Wei Zhan, Masayoshi Tomizuka, Mingyu Ding

Particularly, on the MSRVTT retrieval task, UniAdapter achieves 49. 7% recall@1 with 2. 2% model parameters, outperforming the latest competitors by 2. 0%.

Retrieval Text Retrieval +3

AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners

1 code implementation3 Feb 2023 Zhixuan Liang, Yao Mu, Mingyu Ding, Fei Ni, Masayoshi Tomizuka, Ping Luo

For example, AdaptDiffuser not only outperforms the previous art Diffuser by 20. 8% on Maze2D and 7. 5% on MuJoCo locomotion, but also adapts better to new tasks, e. g., KUKA pick-and-place, by 27. 9% without requiring additional expert data.

Towards Modeling and Influencing the Dynamics of Human Learning

no code implementations2 Jan 2023 Ran Tian, Masayoshi Tomizuka, Anca Dragan, Andrea Bajcsy

Interestingly, robot actions influence what this experience is, and therefore influence how people's internal models change.

PaCo: Parameter-Compositional Multi-Task Reinforcement Learning

1 code implementation21 Oct 2022 Lingfeng Sun, Haichao Zhang, Wei Xu, Masayoshi Tomizuka

However, the gaps between contents and difficulties of different tasks bring us challenges on both which tasks should share the parameters and what parameters should be shared, as well as the optimization challenges due to parameter sharing.

reinforcement-learning Reinforcement Learning (RL)

Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detection

1 code implementation5 Oct 2022 Jinhyung Park, Chenfeng Xu, Shijia Yang, Kurt Keutzer, Kris Kitani, Masayoshi Tomizuka, Wei Zhan

While recent camera-only 3D detection methods leverage multiple timesteps, the limited history they use significantly hampers the extent to which temporal fusion can improve object perception.

3D Object Detection object-detection +2

Center Feature Fusion: Selective Multi-Sensor Fusion of Center-based Objects

no code implementations26 Sep 2022 Philip Jacobson, Yiyang Zhou, Wei Zhan, Masayoshi Tomizuka, Ming C. Wu

In this work, we propose a novel approach Center Feature Fusion (CFF), in which we leverage center-based detection networks in both the camera and LiDAR streams to identify relevant object locations.

Autonomous Vehicles Object +3

Analyzing and Enhancing Closed-loop Stability in Reactive Simulation

no code implementations9 Aug 2022 Wei-Jer Chang, Yeping Hu, Chenran Li, Wei Zhan, Masayoshi Tomizuka

In this paper, we aim to provide a thorough stability analysis of the reactive simulation and propose a solution to enhance the stability.

Generalizability Analysis of Graph-based Trajectory Predictor with Vectorized Representation

no code implementations6 Aug 2022 Juanwu Lu, Wei Zhan, Masayoshi Tomizuka, Yeping Hu

Results show significant performance degradation due to domain shift, and feature attribution provides insights to identify potential causes of these problems.

Autonomous Vehicles Trajectory Prediction

What Matters for 3D Scene Flow Network

1 code implementation19 Jul 2022 Guangming Wang, Yunzhe Hu, Zhe Liu, Yiyang Zhou, Masayoshi Tomizuka, Wei Zhan, Hesheng Wang

Our proposed model surpasses all existing methods by at least 38. 2% on FlyingThings3D dataset and 24. 7% on KITTI Scene Flow dataset for EPE3D metric.

Scene Flow Estimation

SST-Calib: Simultaneous Spatial-Temporal Parameter Calibration between LIDAR and Camera

no code implementations8 Jul 2022 Akio Kodaira, Yiyang Zhou, Pengwei Zang, Wei Zhan, Masayoshi Tomizuka

With information from multiple input modalities, sensor fusion-based algorithms usually out-perform their single-modality counterparts in robotics.

Optical Flow Estimation Segmentation +2

Open-Vocabulary 3D Detection via Image-level Class and Debiased Cross-modal Contrastive Learning

no code implementations5 Jul 2022 Yuheng Lu, Chenfeng Xu, Xiaobao Wei, Xiaodong Xie, Masayoshi Tomizuka, Kurt Keutzer, Shanghang Zhang

Current point-cloud detection methods have difficulty detecting the open-vocabulary objects in the real world, due to their limited generalization capability.

Cloud Detection Contrastive Learning

PreTraM: Self-Supervised Pre-training via Connecting Trajectory and Map

1 code implementation21 Apr 2022 Chenfeng Xu, Tian Li, Chen Tang, Lingfeng Sun, Kurt Keutzer, Masayoshi Tomizuka, Alireza Fathi, Wei Zhan

It is hard to replicate these approaches in trajectory forecasting due to the lack of adequate trajectory data (e. g., 34K samples in the nuScenes dataset).

Contrastive Learning Representation Learning +1

Interventional Behavior Prediction: Avoiding Overly Confident Anticipation in Interactive Prediction

no code implementations19 Apr 2022 Chen Tang, Wei Zhan, Masayoshi Tomizuka

Moreover, to properly evaluate an IBP model with offline datasets, we propose a Shapley-value-based metric to verify if the prediction model satisfies the inherent temporal independence of an interventional distribution.

Learning to Synthesize Volumetric Meshes from Vision-based Tactile Imprints

no code implementations29 Mar 2022 Xinghao Zhu, Siddarth Jain, Masayoshi Tomizuka, Jeroen van Baar

Vision-based tactile sensors typically utilize a deformable elastomer and a camera mounted above to provide high-resolution image observations of contacts.

Image Augmentation Robotic Grasping

DetMatch: Two Teachers are Better Than One for Joint 2D and 3D Semi-Supervised Object Detection

1 code implementation17 Mar 2022 Jinhyung Park, Chenfeng Xu, Yiyang Zhou, Masayoshi Tomizuka, Wei Zhan

While numerous 3D detection works leverage the complementary relationship between RGB images and point clouds, developments in the broader framework of semi-supervised object recognition remain uninfluenced by multi-modal fusion.

object-detection Object Detection +2

Important Object Identification with Semi-Supervised Learning for Autonomous Driving

no code implementations5 Mar 2022 Jiachen Li, Haiming Gang, Hengbo Ma, Masayoshi Tomizuka, Chiho Choi

We propose a novel approach for important object identification in egocentric driving scenarios with relational reasoning on the objects in the scene.

Autonomous Driving Binary Classification +5

Transferable and Adaptable Driving Behavior Prediction

no code implementations10 Feb 2022 Letian Wang, Yeping Hu, Liting Sun, Wei Zhan, Masayoshi Tomizuka, Changliu Liu

By mimicking humans' cognition model and semantic understanding during driving, we propose HATN, a hierarchical framework to generate high-quality, transferable, and adaptable predictions for driving behaviors in multi-agent dense-traffic environments.

Autonomous Vehicles Trajectory Prediction

Learning Differentiable Safety-Critical Control using Control Barrier Functions for Generalization to Novel Environments

no code implementations4 Jan 2022 Hengbo Ma, Bike Zhang, Masayoshi Tomizuka, Koushil Sreenath

By embedding the optimization procedure of the exponential control barrier function based quadratic program (ECBF-QP) as a differentiable layer within a deep learning architecture, we propose a differentiable safety-critical control framework that enables generalization to new environments for high relative-degree systems with forward invariance guarantees.

Multi-Objective Diverse Human Motion Prediction With Knowledge Distillation

no code implementations CVPR 2022 Hengbo Ma, Jiachen Li, Ramtin Hosseini, Masayoshi Tomizuka, Chiho Choi

Obtaining accurate and diverse human motion prediction is essential to many industrial applications, especially robotics and autonomous driving.

Autonomous Driving Human motion prediction +3

Towards General and Efficient Active Learning

3 code implementations15 Dec 2021 Yichen Xie, Masayoshi Tomizuka, Wei Zhan

Existing work follows a cumbersome pipeline that repeats the time-consuming model training and batch data selection multiple times.

Active Learning Depth Estimation +4

Causal-based Time Series Domain Generalization for Vehicle Intention Prediction

no code implementations3 Dec 2021 Yeping Hu, Xiaogang Jia, Masayoshi Tomizuka, Wei Zhan

In this paper, we aim to address the domain generalization problem for vehicle intention prediction tasks and a causal-based time series domain generalization (CTSDG) model is proposed.

Autonomous Vehicles Domain Generalization +3

Exploring Social Posterior Collapse in Variational Autoencoder for Interaction Modeling

no code implementations NeurIPS 2021 Chen Tang, Wei Zhan, Masayoshi Tomizuka

In this work, we argue that one of the typical formulations of VAEs in multi-agent modeling suffers from an issue we refer to as social posterior collapse, i. e., the model is prone to ignoring historical social context when predicting the future trajectory of an agent.

Graph Attention Trajectory Forecasting

Dealing with the Unknown: Pessimistic Offline Reinforcement Learning

no code implementations9 Nov 2021 Jinning Li, Chen Tang, Masayoshi Tomizuka, Wei Zhan

Reinforcement Learning (RL) has been shown effective in domains where the agent can learn policies by actively interacting with its operating environment.

reinforcement-learning Reinforcement Learning (RL)

Grouptron: Dynamic Multi-Scale Graph Convolutional Networks for Group-Aware Dense Crowd Trajectory Forecasting

no code implementations29 Sep 2021 Rui Zhou, HongYu Zhou, Huidong Gao, Masayoshi Tomizuka, Jiachen Li, Zhuo Xu

Accurate, long-term forecasting of pedestrian trajectories in highly dynamic and interactive scenes is a long-standing challenge.

Trajectory Forecasting

RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecasting

no code implementations ICCV 2021 Jiachen Li, Fan Yang, Hengbo Ma, Srikanth Malla, Masayoshi Tomizuka, Chiho Choi

Motion forecasting plays a significant role in various domains (e. g., autonomous driving, human-robot interaction), which aims to predict future motion sequences given a set of historical observations.

Motion Forecasting Trajectory Prediction

Spectral Temporal Graph Neural Network for Trajectory Prediction

no code implementations5 Jun 2021 Defu Cao, Jiachen Li, Hengbo Ma, Masayoshi Tomizuka

To this end, we propose a Spectral Temporal Graph Neural Network (SpecTGNN), which can capture inter-agent correlations and temporal dependency simultaneously in frequency domain in addition to time domain.

Autonomous Vehicles Motion Forecasting +1

History Encoding Representation Design for Human Intention Inference

no code implementations4 Jun 2021 Zhuo Xu, Masayoshi Tomizuka

In this extended abstract, we investigate the design of learning representation for human intention inference.

Convex Parameterization and Optimization for Robust Tracking of a Magnetically Levitated Planar Positioning System

no code implementations22 Mar 2021 Jun Ma, Zilong Cheng, Haiyue Zhu, Xiaocong Li, Masayoshi Tomizuka, Tong Heng Lee

Magnetic levitation positioning technology has attracted considerable research efforts and dedicated attention due to its extremely attractive features.

Learning Human Rewards by Inferring Their Latent Intelligence Levels in Multi-Agent Games: A Theory-of-Mind Approach with Application to Driving Data

no code implementations7 Mar 2021 Ran Tian, Masayoshi Tomizuka, Liting Sun

In this work, we advocate that humans are bounded rational and have different intelligence levels when reasoning about others' decision-making process, and such an inherent and latent characteristic should be accounted for in reward learning algorithms.

Decision Making reinforcement-learning +1

A Simple and Efficient Multi-task Network for 3D Object Detection and Road Understanding

1 code implementation6 Mar 2021 Di Feng, Yiyang Zhou, Chenfeng Xu, Masayoshi Tomizuka, Wei Zhan

Detecting dynamic objects and predicting static road information such as drivable areas and ground heights are crucial for safe autonomous driving.

3D Object Detection Autonomous Driving +1

Grounded Relational Inference: Domain Knowledge Driven Explainable Autonomous Driving

no code implementations23 Feb 2021 Chen Tang, Nishan Srishankar, Sujitha Martin, Masayoshi Tomizuka

Explainability is essential for autonomous vehicles and other robotics systems interacting with humans and other objects during operation.

Autonomous Driving

Spatio-Temporal Graph Dual-Attention Network for Multi-Agent Prediction and Tracking

no code implementations18 Feb 2021 Jiachen Li, Hengbo Ma, Zhihao Zhang, Jinning Li, Masayoshi Tomizuka

Due to the existence of frequent interactions and uncertainty in the scene evolution, it is desired for the prediction system to enable relational reasoning on different entities and provide a distribution of future trajectories for each agent.

Autonomous Vehicles Navigate +2

A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning

no code implementations17 Jan 2021 Jinning Li, Liting Sun, Jianyu Chen, Masayoshi Tomizuka, Wei Zhan

To address this challenge, we propose a hierarchical behavior planning framework with a set of low-level safe controllers and a high-level reinforcement learning algorithm (H-CtRL) as a coordinator for the low-level controllers.

Autonomous Vehicles reinforcement-learning +1

Labels Are Not Perfect: Inferring Spatial Uncertainty in Object Detection

no code implementations18 Dec 2020 Di Feng, Zining Wang, Yiyang Zhou, Lars Rosenbaum, Fabian Timm, Klaus Dietmayer, Masayoshi Tomizuka, Wei Zhan

As a result, an in-depth evaluation among different object detection methods remains challenging, and the training process of object detectors is sub-optimal, especially in probabilistic object detection.

Autonomous Driving Object +2

Sparse R-CNN: End-to-End Object Detection with Learnable Proposals

6 code implementations CVPR 2021 Peize Sun, Rufeng Zhang, Yi Jiang, Tao Kong, Chenfeng Xu, Wei Zhan, Masayoshi Tomizuka, Lei LI, Zehuan Yuan, Changhu Wang, Ping Luo

In our method, however, a fixed sparse set of learned object proposals, total length of $N$, are provided to object recognition head to perform classification and location.

Object object-detection +2

COCOI: Contact-aware Online Context Inference for Generalizable Non-planar Pushing

no code implementations23 Nov 2020 Zhuo Xu, Wenhao Yu, Alexander Herzog, Wenlong Lu, Chuyuan Fu, Masayoshi Tomizuka, Yunfei Bai, C. Karen Liu, Daniel Ho

General contact-rich manipulation problems are long-standing challenges in robotics due to the difficulty of understanding complicated contact physics.

Reinforcement Learning (RL) Robot Manipulation

IDE-Net: Interactive Driving Event and Pattern Extraction from Human Data

no code implementations4 Nov 2020 Xiaosong Jia, Liting Sun, Masayoshi Tomizuka, Wei Zhan

We find three interpretable patterns of interactions, bringing insights for driver behavior representation, modeling and comprehension.

Autonomous Vehicles Multi-Task Learning

Socially-Compatible Behavior Design of Autonomous Vehicles with Verification on Real Human Data

no code implementations28 Oct 2020 Letian Wang, Liting Sun, Masayoshi Tomizuka, Wei Zhan

It allows the AVs to infer the characteristics of other road users online and generate behaviors optimizing not only their own rewards, but also their courtesy to others, and their confidence regarding the prediction uncertainties.

Autonomous Vehicles

Bounded Risk-Sensitive Markov Games: Forward Policy Design and Inverse Reward Learning with Iterative Reasoning and Cumulative Prospect Theory

no code implementations3 Sep 2020 Ran Tian, Liting Sun, Masayoshi Tomizuka

Classical game-theoretic approaches for multi-agent systems in both the forward policy design problem and the inverse reward learning problem often make strong rationality assumptions: agents perfectly maximize expected utilities under uncertainties.

Expressing Diverse Human Driving Behavior with Probabilistic Rewards and Online Inference

no code implementations20 Aug 2020 Liting Sun, Zheng Wu, Hengbo Ma, Masayoshi Tomizuka

In human-robot interaction (HRI) systems, such as autonomous vehicles, understanding and representing human behavior are important.

Autonomous Vehicles

Efficient Sampling-Based Maximum Entropy Inverse Reinforcement Learning with Application to Autonomous Driving

no code implementations22 Jun 2020 Zheng Wu, Liting Sun, Wei Zhan, Chenyu Yang, Masayoshi Tomizuka

Different from existing IRL algorithms, by introducing an efficient continuous-domain trajectory sampler, the proposed algorithm can directly learn the reward functions in the continuous domain while considering the uncertainties in demonstrated trajectories from human drivers.

Autonomous Driving reinforcement-learning +1

Towards Better Performance and More Explainable Uncertainty for 3D Object Detection of Autonomous Vehicles

no code implementations22 Jun 2020 Hujie Pan, Zining Wang, Wei Zhan, Masayoshi Tomizuka

In this paper, we propose a novel form of the loss function to increase the performance of LiDAR-based 3d object detection and obtain more explainable and convincing uncertainty for the prediction.

3D Object Detection Autonomous Vehicles +1

In Proximity of ReLU DNN, PWA Function, and Explicit MPC

no code implementations9 Jun 2020 Saman Fahandezh-Saadi, Masayoshi Tomizuka

Rectifier (ReLU) deep neural networks (DNN) and their connection with piecewise affine (PWA) functions is analyzed.

Model Predictive Control

Visual Transformers: Token-based Image Representation and Processing for Computer Vision

8 code implementations5 Jun 2020 Bichen Wu, Chenfeng Xu, Xiaoliang Dai, Alvin Wan, Peizhao Zhang, Zhicheng Yan, Masayoshi Tomizuka, Joseph Gonzalez, Kurt Keutzer, Peter Vajda

In this work, we challenge this paradigm by (a) representing images as semantic visual tokens and (b) running transformers to densely model token relationships.

General Classification Image Classification +1

Scenario-Transferable Semantic Graph Reasoning for Interaction-Aware Probabilistic Prediction

no code implementations7 Apr 2020 Yeping Hu, Wei Zhan, Masayoshi Tomizuka

Accurately predicting the possible behaviors of traffic participants is an essential capability for autonomous vehicles.

Autonomous Driving Navigate

SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentation

3 code implementations ECCV 2020 Chenfeng Xu, Bichen Wu, Zining Wang, Wei Zhan, Peter Vajda, Kurt Keutzer, Masayoshi Tomizuka

Using standard convolutions to process such LiDAR images is problematic, as convolution filters pick up local features that are only active in specific regions in the image.

3D Semantic Segmentation Point Cloud Segmentation +1

EvolveGraph: Multi-Agent Trajectory Prediction with Dynamic Relational Reasoning

no code implementations NeurIPS 2020 Jiachen Li, Fan Yang, Masayoshi Tomizuka, Chiho Choi

In this paper, we propose a generic trajectory forecasting framework (named EvolveGraph) with explicit relational structure recognition and prediction via latent interaction graphs among multiple heterogeneous, interactive agents.

Autonomous Driving Decision Making +2

Inferring Spatial Uncertainty in Object Detection

no code implementations7 Mar 2020 Zining Wang, Di Feng, Yiyang Zhou, Lars Rosenbaum, Fabian Timm, Klaus Dietmayer, Masayoshi Tomizuka, Wei Zhan

Based on the spatial distribution, we further propose an extension of IoU, called the Jaccard IoU (JIoU), as a new evaluation metric that incorporates label uncertainty.

Autonomous Driving Object +2

Social-WaGDAT: Interaction-aware Trajectory Prediction via Wasserstein Graph Double-Attention Network

no code implementations14 Feb 2020 Jiachen Li, Hengbo Ma, Zhihao Zhang, Masayoshi Tomizuka

Effective understanding of the environment and accurate trajectory prediction of surrounding dynamic obstacles are indispensable for intelligent mobile systems (like autonomous vehicles and social robots) to achieve safe and high-quality planning when they navigate in highly interactive and crowded scenarios.

Autonomous Vehicles Navigate +2

AutoScale: Learning to Scale for Crowd Counting and Localization

2 code implementations20 Dec 2019 Chenfeng Xu, Dingkang Liang, Yongchao Xu, Song Bai, Wei Zhan, Xiang Bai, Masayoshi Tomizuka

A major issue is that the density map on dense regions usually accumulates density values from a number of nearby Gaussian blobs, yielding different large density values on a small set of pixels.

Crowd Counting Model Optimization

Robust Feature-Based Point Registration Using Directional Mixture Model

no code implementations25 Nov 2019 Saman Fahandezh-Saadi, Di Wang, Masayoshi Tomizuka

This paper presents a robust probabilistic point registration method for estimating the rigid transformation (i. e. rotation matrix and translation vector) between two pointcloud dataset.

Translation

Online Learning in Planar Pushing with Combined Prediction Model

no code implementations17 Oct 2019 Huidong Gao, Yi Ouyang, Masayoshi Tomizuka

In this paper, we propose a combined prediction model and an online learning framework for planar push prediction.

Trajectory Planning

epBRM: Improving a Quality of 3D Object Detection using End Point Box Regression Module

no code implementations27 Sep 2019 Kiwoo Shin, Masayoshi Tomizuka

Our approach can improve a 3D object detection performance by predicting more precise 3D bounding box coordinates.

3D Object Detection Autonomous Vehicles +3

Generic Prediction Architecture Considering both Rational and Irrational Driving Behaviors

no code implementations23 Jul 2019 Yeping Hu, Liting Sun, Masayoshi Tomizuka

Both rational and irrational behaviors exist, and the autonomous vehicles need to be aware of this in their prediction module.

Autonomous Vehicles

Interpretable Modelling of Driving Behaviors in Interactive Driving Scenarios based on Cumulative Prospect Theory

no code implementations19 Jul 2019 Liting Sun, Wei Zhan, Yeping Hu, Masayoshi Tomizuka

Hence, the goal of this work is to formulate the human drivers' behavior generation model with CPT so that some ``irrational'' behavior or decisions of human can be better captured and predicted.

Autonomous Vehicles Decision Making

Conditional Generative Neural System for Probabilistic Trajectory Prediction

no code implementations5 May 2019 Jiachen Li, Hengbo Ma, Masayoshi Tomizuka

Effective understanding of the environment and accurate trajectory prediction of surrounding dynamic obstacles are critical for intelligent systems such as autonomous vehicles and wheeled mobile robotics navigating in complex scenarios to achieve safe and high-quality decision making, motion planning and control.

Autonomous Vehicles Decision Making +3

Behavior Planning of Autonomous Cars with Social Perception

no code implementations2 May 2019 Liting Sun, Wei Zhan, Ching-Yao Chan, Masayoshi Tomizuka

The uncertainties can come either from sensor limitations such as occlusions and limited sensor range, or from probabilistic prediction of other road participants, or from unknown social behavior in a new area.

Autonomous Vehicles Decision Making +2

Coordination and Trajectory Prediction for Vehicle Interactions via Bayesian Generative Modeling

no code implementations2 May 2019 Jiachen Li, Hengbo Ma, Wei Zhan, Masayoshi Tomizuka

In order to tackle the task of probabilistic prediction for multiple, interactive entities, we propose a coordination and trajectory prediction system (CTPS), which has a hierarchical structure including a macro-level coordination recognition module and a micro-level subtle pattern prediction module which solves a probabilistic generation task.

Trajectory Prediction

Model-free Deep Reinforcement Learning for Urban Autonomous Driving

2 code implementations20 Apr 2019 Jianyu Chen, Bodi Yuan, Masayoshi Tomizuka

Urban autonomous driving decision making is challenging due to complex road geometry and multi-agent interactions.

Autonomous Driving Decision Making +2

Optimization Model for Planning Precision Grasps with Multi-Fingered Hands

no code implementations15 Apr 2019 Yongxiang Fan, Xinghao Zhu, Masayoshi Tomizuka

Searching precision grasps on the object represented by point cloud, is challenging due to the complex object shape, high-dimensionality, collision and undesired properties of the sensing and positioning.

Robotics

Interaction-aware Decision Making with Adaptive Strategies under Merging Scenarios

no code implementations12 Apr 2019 Yeping Hu, Alireza Nakhaei, Masayoshi Tomizuka, Kikuo Fujimura

In this paper, we proposed an interaction-aware decision making with adaptive strategies (IDAS) approach that can let the autonomous vehicle negotiate the road with other drivers by leveraging their cooperativeness under merging scenarios.

Autonomous Vehicles Common Sense Reasoning +2

Interaction-aware Multi-agent Tracking and Probabilistic Behavior Prediction via Adversarial Learning

no code implementations4 Apr 2019 Jiachen Li, Hengbo Ma, Masayoshi Tomizuka

In order to enable high-quality decision making and motion planning of intelligent systems such as robotics and autonomous vehicles, accurate probabilistic predictions for surrounding interactive objects is a crucial prerequisite.

Autonomous Vehicles Decision Making +2

Multi-modal Probabilistic Prediction of Interactive Behavior via an Interpretable Model

no code implementations22 Mar 2019 Yeping Hu, Wei Zhan, Liting Sun, Masayoshi Tomizuka

The proposed method is based on a generative model and is capable of jointly predicting sequential motions of each pair of interacting agents.

valid

Efficient Grasp Planning and Execution with Multi-Fingered Hands by Surface Fitting

no code implementations28 Feb 2019 Yongxiang Fan, Masayoshi Tomizuka

The framework includes a multi-dimensional iterative surface fitting (MDISF) for grasp planning and a grasp trajectory optimization (GTO) for grasp imagination.

Robotics

Zero-shot Deep Reinforcement Learning Driving Policy Transfer for Autonomous Vehicles based on Robust Control

no code implementations7 Dec 2018 Zhuo Xu, Chen Tang, Masayoshi Tomizuka

Although deep reinforcement learning (deep RL) methods have lots of strengths that are favorable if applied to autonomous driving, real deep RL applications in autonomous driving have been slowed down by the modeling gap between the source (training) domain and the target (deployment) domain.

Autonomous Driving

A Framework for Probabilistic Generic Traffic Scene Prediction

no code implementations30 Oct 2018 Yeping Hu, Wei Zhan, Masayoshi Tomizuka

In a given scenario, simultaneously and accurately predicting every possible interaction of traffic participants is an important capability for autonomous vehicles.

Autonomous Vehicles Decision Making +1

A Learning Framework for Robust Bin Picking by Customized Grippers

no code implementations23 Sep 2018 Yongxiang Fan, Hsien-Chung Lin, Te Tang, Masayoshi Tomizuka

In this paper, we propose a learning framework to plan robust grasps for customized grippers in real-time.

Collision Avoidance

Towards a Fatality-Aware Benchmark of Probabilistic Reaction Prediction in Highly Interactive Driving Scenarios

no code implementations10 Sep 2018 Wei Zhan, Liting Sun, Yeping Hu, Jiachen Li, Masayoshi Tomizuka

Modified methods based on PGM, NN and IRL are provided to generate probabilistic reaction predictions in an exemplar scenario of nudging from a highway ramp.

Autonomous Vehicles Decision Making

Probabilistic Prediction of Interactive Driving Behavior via Hierarchical Inverse Reinforcement Learning

no code implementations9 Sep 2018 Liting Sun, Wei Zhan, Masayoshi Tomizuka

To safely and efficiently interact with other road participants, AVs have to accurately predict the behavior of surrounding vehicles and plan accordingly.

Autonomous Vehicles reinforcement-learning +1

Generic Probabilistic Interactive Situation Recognition and Prediction: From Virtual to Real

no code implementations9 Sep 2018 Jiachen Li, Hengbo Ma, Wei Zhan, Masayoshi Tomizuka

Accurate and robust recognition and prediction of traffic situation plays an important role in autonomous driving, which is a prerequisite for risk assessment and effective decision making.

Autonomous Driving Decision Making +1

Robot Safe Interaction System for Intelligent Industrial Co-Robots

1 code implementation12 Aug 2018 Changliu Liu, Masayoshi Tomizuka

Human-robot interactions have been recognized to be a key element of future industrial collaborative robots (co-robots).

Robotics Systems and Control

Courteous Autonomous Cars

no code implementations8 Aug 2018 Liting Sun, Wei Zhan, Masayoshi Tomizuka, Anca D. Dragan

Such a courtesy term enables the robot car to be aware of possible irrationality of the human behavior, and plan accordingly.

Probabilistic Prediction of Vehicle Semantic Intention and Motion

no code implementations10 Apr 2018 Yeping Hu, Wei Zhan, Masayoshi Tomizuka

Accurately predicting the possible behaviors of traffic participants is an essential capability for future autonomous vehicles.

Autonomous Vehicles motion prediction

Grasp Planning for Customized Grippers by Iterative Surface Fitting

no code implementations30 Mar 2018 Yongxiang Fan, Hsien-Chung Lin, Te Tang, Masayoshi Tomizuka

The proposed algorithm is able to consider the structural constraints of the gripper and plan optimal grasps in real-time.

Robotics

Cascade Attribute Learning Network

no code implementations24 Nov 2017 Zhuo Xu, Haonan Chang, Masayoshi Tomizuka

We propose the cascade attribute learning network (CALNet), which can learn attributes in a control task separately and assemble them together.

Attribute Position +1

Fusing Bird View LIDAR Point Cloud and Front View Camera Image for Deep Object Detection

no code implementations17 Nov 2017 Zining Wang, Wei Zhan, Masayoshi Tomizuka

The fusion method shows particular benefit for detection of pedestrians in the bird view compared to other fusion-based object detection networks.

3D Object Detection Autonomous Driving +2

The Convex Feasible Set Algorithm for Real Time Optimization in Motion Planning

1 code implementation2 Sep 2017 Changliu Liu, Chung-Yen Lin, Masayoshi Tomizuka

The idea is to find a convex feasible set for the original problem and iteratively solve a sequence of subproblems using the convex constraints.

Optimization and Control Robotics

A Fast Integrated Planning and Control Framework for Autonomous Driving via Imitation Learning

no code implementations9 Jul 2017 Liting Sun, Cheng Peng, Wei Zhan, Masayoshi Tomizuka

For safe and efficient planning and control in autonomous driving, we need a driving policy which can achieve desirable driving quality in long-term horizon with guaranteed safety and feasibility.

Autonomous Driving Imitation Learning +1

Cannot find the paper you are looking for? You can Submit a new open access paper.