MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction

Predicting the future behavior of road users is one of the most challenging and important problems in autonomous driving. Applying deep learning to this problem requires fusing heterogeneous world state in the form of rich perception signals and map information, and inferring highly multi-modal distributions over possible futures. In this paper, we present MultiPath++, a future prediction model that achieves state-of-the-art performance on popular benchmarks. MultiPath++ improves the MultiPath architecture by revisiting many design choices. The first key design difference is a departure from dense image-based encoding of the input world state in favor of a sparse encoding of heterogeneous scene elements: MultiPath++ consumes compact and efficient polylines to describe road features, and raw agent state information directly (e.g., position, velocity, acceleration). We propose a context-aware fusion of these elements and develop a reusable multi-context gating fusion component. Second, we reconsider the choice of pre-defined, static anchors, and develop a way to learn latent anchor embeddings end-to-end in the model. Lastly, we explore ensembling and output aggregation techniques -- common in other ML domains -- and find effective variants for our probabilistic multimodal output representation. We perform an extensive ablation on these design choices, and show that our proposed model achieves state-of-the-art performance on the Argoverse Motion Forecasting Competition and the Waymo Open Dataset Motion Prediction Challenge.

PDF Abstract


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Motion Forecasting Argoverse CVPR 2020 multipath++ MR (K=6) 0.1324 # 206
minADE (K=1) 1.6235 # 216
minFDE (K=1) 3.6141 # 207
MR (K=1) 0.5645 # 215
minADE (K=6) 0.7897 # 232
minFDE (K=6) 1.2144 # 224
DAC (K=6) 0.9876 # 37
brier-minFDE (K=6) 1.7932 # 9


No methods listed for this paper. Add relevant methods here