MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Mask

Feature warping is a core technique in optical flow estimation; however, the ambiguity caused by occluded areas during warping is a major problem that remains unsolved. In this paper, we propose an asymmetric occlusion-aware feature matching module, which can learn a rough occlusion mask that filters useless (occluded) areas immediately after feature warping without any explicit supervision. The proposed module can be easily integrated into end-to-end network architectures and enjoys performance gains while introducing negligible computational cost. The learned occlusion mask can be further fed into a subsequent network cascade with dual feature pyramids with which we achieve state-of-the-art performance. At the time of submission, our method, called MaskFlownet, surpasses all published optical flow methods on the MPI Sintel, KITTI 2012 and 2015 benchmarks. Code is available at

PDF Abstract CVPR 2020 PDF CVPR 2020 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Optical Flow Estimation KITTI 2012 MaskFlownet-S Average End-Point Error 1.1 # 2
Optical Flow Estimation KITTI 2012 MaskFlownet Average End-Point Error 1.1 # 2
Optical Flow Estimation KITTI 2015 MaskFlownet-S Fl-all 6.81 # 5
Optical Flow Estimation KITTI 2015 MaskFlownet Fl-all 6.11 # 4
Optical Flow Estimation KITTI 2015 (train) MaskFlowNet F1-all 23.1 # 9
Optical Flow Estimation Sintel-clean MaskFlownet-S Average End-Point Error 2.77 # 11
Optical Flow Estimation Sintel-clean MaskFlownet Average End-Point Error 2.52 # 9
Optical Flow Estimation Sintel-final MaskFlownet-S Average End-Point Error 4.38 # 12
Optical Flow Estimation Sintel-final MaskFlownet Average End-Point Error 4.17 # 10