X3D: Expanding Architectures for Efficient Video Recognition

CVPR 2020  ·  Christoph Feichtenhofer ·

This paper presents X3D, a family of efficient video networks that progressively expand a tiny 2D image classification architecture along multiple network axes, in space, time, width and depth. Inspired by feature selection methods in machine learning, a simple stepwise network expansion approach is employed that expands a single axis in each step, such that good accuracy to complexity trade-off is achieved. To expand X3D to a specific target complexity, we perform progressive forward expansion followed by backward contraction. X3D achieves state-of-the-art performance while requiring 4.8x and 5.5x fewer multiply-adds and parameters for similar accuracy as previous work. Our most surprising finding is that networks with high spatiotemporal resolution can perform well, while being extremely light in terms of network width and parameters. We report competitive accuracy at unprecedented efficiency on video classification and detection benchmarks. Code will be available at: https://github.com/facebookresearch/SlowFast

PDF Abstract CVPR 2020 PDF CVPR 2020 Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Action Classification Kinetics-400 X3D-XXL Acc@1 80.4 # 86
Acc@5 94.6 # 60
Action Classification Kinetics-400 X3D-M Acc@1 76 # 135
Acc@5 92.3 # 103
Action Classification Kinetics-400 X3D-L Acc@1 77.5 # 122
Acc@5 92.9 # 99
Action Classification Kinetics-400 X3D-XL Acc@1 79.1 # 102
Acc@5 93.9 # 79