BMN: Boundary-Matching Network for Temporal Action Proposal Generation

ICCV 2019  Â·  Tianwei Lin, Xiao Liu, Xin Li, Errui Ding, Shilei Wen ·

Temporal action proposal generation is an challenging and promising task which aims to locate temporal regions in real-world videos where action or event may occur. Current bottom-up proposal generation methods can generate proposals with precise boundary, but cannot efficiently generate adequately reliable confidence scores for retrieving proposals. To address these difficulties, we introduce the Boundary-Matching (BM) mechanism to evaluate confidence scores of densely distributed proposals, which denote a proposal as a matching pair of starting and ending boundaries and combine all densely distributed BM pairs into the BM confidence map. Based on BM mechanism, we propose an effective, efficient and end-to-end proposal generation method, named Boundary-Matching Network (BMN), which generates proposals with precise temporal boundaries as well as reliable confidence scores simultaneously. The two-branches of BMN are jointly trained in an unified framework. We conduct experiments on two challenging datasets: THUMOS-14 and ActivityNet-1.3, where BMN shows significant performance improvement with remarkable efficiency and generalizability. Further, combining with existing action classifier, BMN can achieve state-of-the-art temporal action detection performance.

PDF Abstract ICCV 2019 PDF ICCV 2019 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Temporal Action Localization ActivityNet-1.3 BMN mAP IOU@0.5 50.07 # 21
mAP 33.85 # 22
mAP IOU@0.75 34.78 # 15
mAP IOU@0.95 8.29 # 13
Temporal Action Proposal Generation ActivityNet-1.3 BMN AUC (val) 67.1 # 6
AR@100 75.01 # 6
Temporal Action Localization EPIC-KITCHENS-100 BMN (verb) Avg mAP (0.1-0.5) 8.4 # 5
mAP IOU@0.1 10.8 # 5
mAP IOU@0.2 9.8 # 5
mAP IOU@0.3 8.4 # 5
mAP IOU@0.4 7.1 # 5
mAP IOU@0.5 5.6 # 5
Temporal Action Localization FineAction BMN (i3d feaure) mAP 9.25 # 3
mAP IOU@0.5 14.44 # 2
mAP IOU@0.75 8.92 # 2
mAP IOU@0.95 3.12 # 2
Action Recognition THUMOS’14 BMN mAP@0.3 56.0 # 1
mAP@0.4 47.4 # 1
mAP@0.5 38.8 # 1
Temporal Action Localization THUMOS’14 BMN mAP IOU@0.5 32.2 # 27


No methods listed for this paper. Add relevant methods here