Proposal Relation Network for Temporal Action Detection

This technical report presents our solution for temporal action detection task in AcitivityNet Challenge 2021. The purpose of this task is to locate and identify actions of interest in long untrimmed videos. The crucial challenge of the task comes from that the temporal duration of action varies dramatically, and the target actions are typically embedded in a background of irrelevant activities. Our solution builds on BMN, and mainly contains three steps: 1) action classification and feature encoding by Slowfast, CSN and ViViT; 2) proposal generation. We improve BMN by embedding the proposed Proposal Relation Network (PRN), by which we can generate proposals of high quality; 3) action detection. We calculate the detection results by assigning the proposals with corresponding classification results. Finally, we ensemble the results under different settings and achieve 44.7% on the test set, which improves the champion result in ActivityNet 2020 by 1.9% in terms of average mAP.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Temporal Action Localization ActivityNet-1.3 PRN (CSN) mAP IOU@0.5 57.9 # 2
mAP 39.4 # 2
Temporal Action Localization ActivityNet-1.3 PRN (ViViT) mAP IOU@0.5 55.5 # 3
mAP 37.5 # 4
Temporal Action Localization ActivityNet-1.3 PRN+BMN (ensemble) mAP IOU@0.5 59.7 # 1
mAP 42.0 # 1

Methods


No methods listed for this paper. Add relevant methods here