Learning Action Completeness from Points for Weakly-supervised Temporal Action Localization

ICCV 2021  ·  Pilhyeon Lee, Hyeran Byun ·

We tackle the problem of localizing temporal intervals of actions with only a single frame label for each action instance for training. Owing to label sparsity, existing work fails to learn action completeness, resulting in fragmentary action predictions. In this paper, we propose a novel framework, where dense pseudo-labels are generated to provide completeness guidance for the model. Concretely, we first select pseudo background points to supplement point-level action labels. Then, by taking the points as seeds, we search for the optimal sequence that is likely to contain complete action instances while agreeing with the seeds. To learn completeness from the obtained sequence, we introduce two novel losses that contrast action instances with background ones in terms of action score and feature similarity, respectively. Experimental results demonstrate that our completeness guidance indeed helps the model to locate complete action instances, leading to large performance gains especially under high IoU thresholds. Moreover, we demonstrate the superiority of our method over existing state-of-the-art methods on four benchmarks: THUMOS'14, GTEA, BEOID, and ActivityNet. Notably, our method even performs comparably to recent fully-supervised methods, at the 6 times cheaper annotation cost. Our code is available at https://github.com/Pilhyeon.

PDF Abstract ICCV 2021 PDF ICCV 2021 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Weakly Supervised Action Localization ActivityNet-1.2 LACP mAP@0.5 44 # 1
Mean mAP 26.8 # 1
Weakly Supervised Action Localization ActivityNet-1.3 LACP mAP@0.5 40.4 # 2
mAP@0.5:0.95 25.1 # 1
Weakly Supervised Action Localization BEOID LACP mAP@0.1:0.7 51.8 # 1
mAP@0.5 42.7 # 1
Weakly Supervised Action Localization GTEA LACP mAP@0.1:0.7 43.5 # 1
mAP@0.5 33.9 # 1
Weakly Supervised Action Localization THUMOS14 LACP avg-mAP (0.1-0.5) 62.7 # 1
avg-mAP (0.3-0.7) 44.5 # 1
avg-mAP (0.1:0.7) 52.8 # 1
Weakly Supervised Action Localization THUMOS’14 LACP mAP@0.5 45.3 # 1
Weakly Supervised Action Localization THUMOS 2014 LACP mAP@0.5 45.3 # 1
mAP@0.1:0.7 52.8 # 1
mAP@0.1:0.5 62.7 # 1

Methods


No methods listed for this paper. Add relevant methods here