Large-Scale Video Classification with Convolutional Neural Networks
Convolutional Neural Networks (CNNs) have been established as a powerful class of models for image recognition problems. Encouraged by these results, we provide an extensive empirical evaluation of CNNs on large-scale video classification using a new dataset of 1 million YouTube videos belonging to 487 classes. We study multiple approaches for extending the connectivity of a CNN in time domain to take advantage of local spatio-temporal information and suggest a multiresolution, foveated architecture as a promising way of speeding up the training. Our best spatio-temporal networks display significant performance improvements compared to strong feature-based baselines (55.3% to 63.9%), but only a surprisingly modest improvement compared to single-frame models (59.3% to 60.9%). We further study the generalization performance of our best model by retraining the top layers on the UCF-101 Action Recognition dataset and observe significant performance improvements compared to the UCF-101 baseline model (63.3% up from 43.9%).
PDF Abstract 2014 IEEE 2014 PDF 2014 IEEE 2014 AbstractCode
Sports-1M
ImageNet
UCF101
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Uses Extra Training Data |
Benchmark |
|---|---|---|---|---|---|---|---|
| Action Recognition | Sports-1M | DeepVideo’s Slow Fusion | Clip Hit@1 | 41.9 | # 5 | ||
| Video hit@1 | 60.9 | # 9 | |||||
| Video hit@5 | 80.2 | # 9 | |||||
| Action Recognition | UCF101 | Slow Fusion + Finetune top 3 layers | 3-fold Accuracy | 65.4 | # 83 |
