Resource Efficient 3D Convolutional Neural Networks

4 Apr 2019  ·  Okan Köpüklü, Neslihan Kose, Ahmet Gunduz, Gerhard Rigoll ·

Recently, convolutional neural networks with 3D kernels (3D CNNs) have been very popular in computer vision community as a result of their superior ability of extracting spatio-temporal features within video frames compared to 2D CNNs. Although there has been great advances recently to build resource efficient 2D CNN architectures considering memory and power budget, there is hardly any similar resource efficient architectures for 3D CNNs. In this paper, we have converted various well-known resource efficient 2D CNNs to 3D CNNs and evaluated their performance on three major benchmarks in terms of classification accuracy for different complexity levels. We have experimented on (1) Kinetics-600 dataset to inspect their capacity to learn, (2) Jester dataset to inspect their ability to capture motion patterns, and (3) UCF-101 to inspect the applicability of transfer learning. We have evaluated the run-time performance of each model on a single Titan XP GPU and a Jetson TX2 embedded system. The results of this study show that these models can be utilized for different types of real-world applications since they provide real-time performance with considerable accuracies and memory usage. Our analysis on different complexity levels shows that the resource efficient 3D CNNs should not be designed too shallow or narrow in order to save complexity. The codes and pretrained models used in this work are publicly available.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Action Recognition In Videos Jester (Gesture Recognition) 3D-SqueezeNet Val 90.77 # 7
Action Recognition In Videos Jester (Gesture Recognition) 3D-MobileNetV2 0.2x Val 86.43 # 9
Action Recognition In Videos Jester (Gesture Recognition) 3D-ShuffleNetV2 0.25x Val 86.91 # 8
Action Recognition In Videos UCF101 3D-MobileNetV2 0.2x 3-fold Accuracy 55.56 # 4
Action Recognition In Videos UCF101 3D-ShuffleNetV2 0.25x 3-fold Accuracy 56.52 # 3
Action Recognition In Videos UCF101 3D-SqueezeNet 3-fold Accuracy 74.94 # 2


No methods listed for this paper. Add relevant methods here