R-C3D: Region Convolutional 3D Network for Temporal Activity Detection

ICCV 2017  ·  Huijuan Xu, Abir Das, Kate Saenko ·

We address the problem of activity detection in continuous, untrimmed video streams. This is a difficult task that requires extracting meaningful spatio-temporal features to capture activities, accurately localizing the start and end times of each activity... We introduce a new model, Region Convolutional 3D Network (R-C3D), which encodes the video streams using a three-dimensional fully convolutional network, then generates candidate temporal regions containing activities, and finally classifies selected regions into specific activities. Computation is saved due to the sharing of convolutional features between the proposal and the classification pipelines. The entire model is trained end-to-end with jointly optimized localization and classification losses. R-C3D is faster than existing methods (569 frames per second on a single Titan X Maxwell GPU) and achieves state-of-the-art results on THUMOS'14. We further demonstrate that our model is a general activity detection framework that does not rely on assumptions about particular dataset properties by evaluating our approach on ActivityNet and Charades. Our code is available at http://ai.bu.edu/r-c3d/. read more

PDF Abstract ICCV 2017 PDF ICCV 2017 Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Temporal Action Localization ActivityNet-1.3 R-C3D mAP IOU@0.5 26.8 # 12
Action Detection Charades R-C3D mAP 12.4 # 8
Action Recognition THUMOS’14 Single-stream R-C3D (two-way buffer) mAP@0.1 54.5 # 4
mAP@0.2 51.5 # 4
mAP@0.3 44.8 # 8
mAP@0.4 35.6 # 7
mAP@0.5 28.9 # 7
Action Recognition THUMOS’14 Single-stream R-C3D (one-way buffer) mAP@0.1 51.6 # 5
mAP@0.2 49.2 # 5
mAP@0.3 42.8 # 9
mAP@0.4 33.4 # 9
mAP@0.5 27.0 # 8
Temporal Action Localization THUMOS’14 R-C3D mAP IOU@0.5 28.9 # 14
mAP IOU@0.1 54.5 # 7
mAP IOU@0.2 51.5 # 6
mAP IOU@0.3 44.8 # 12
mAP IOU@0.4 35.6 # 11


No methods listed for this paper. Add relevant methods here