Weakly Supervised Action Localization by Sparse Temporal Pooling Network

CVPR 2018  ·  Phuc Nguyen, Ting Liu, Gautam Prasad, Bohyung Han ·

We propose a weakly supervised temporal action localization algorithm on untrimmed videos using convolutional neural networks. Our algorithm learns from video-level class labels and predicts temporal intervals of human actions with no requirement of temporal localization annotations. We design our network to identify a sparse subset of key segments associated with target actions in a video using an attention module and fuse the key segments through adaptive temporal pooling. Our loss function is comprised of two terms that minimize the video-level action classification error and enforce the sparsity of the segment selection. At inference time, we extract and score temporal proposals using temporal class activations and class-agnostic attentions to estimate the time intervals that correspond to target actions. The proposed algorithm attains state-of-the-art results on the THUMOS14 dataset and outstanding performance on ActivityNet1.3 even with its weak supervision.

PDF Abstract CVPR 2018 PDF CVPR 2018 Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Weakly Supervised Action Localization ActivityNet-1.3 STPN mAP@0.5 29.3 # 13
Weakly Supervised Action Localization THUMOS 2014 STPN mAP@0.5 16.9 # 24
mAP@0.1:0.7 27.0 # 20


No methods listed for this paper. Add relevant methods here