Video Classification

Edit

90 papers with code • 8 benchmarks • 10 datasets

Video Classification is the task of producing a label that is relevant to the video given its frames. A good video level classifier is one that not only provides accurate frame labels, but also best describes the entire video given the features and the annotations of the various frames in the video. For example, a video might contain a tree in some frame, but the label that is central to the video might be something else (e.g., “hiking”). The granularity of the labels that are needed to describe the frames and the video depends on the task. Typical tasks include assigning one or more global labels to the video, and assigning one or more labels for each frame inside the video.

Source: Efficient Large Scale Video Classification

Benchmarks

Add a Result

Dataset	Best Model	Paper Title	Compare
YouTube-8M	DCGN (self-attention graph pooling)	Hierarchical Video Frame Sequence Representation with Deep Convolutional Graph Network	See all
Kinetics	Multigrid	A Multigrid Method for Efficiently Training Video Models	See all
Charades	Multigrid	A Multigrid Method for Efficiently Training Video Models	See all
Something-Something V1	MSNet-R50En (ours)	MotionSqueeze: Neural Motion Feature Learning for Video Understanding	See all
Something-Something V2	MSNet-R50En (ours)	MotionSqueeze: Neural Motion Feature Learning for Video Understanding	See all
Multimodal PISA	MMDL	Piano Skills Assessment	See all
Home Action Genome	Cooperative Ours (3rd-person)	Home Action Genome: Cooperative Compositional Action Understanding	See all
Hockey Fight Detection Dataset	CNN+LSTM	Robust Real-Time Violence Detection in Video Using CNN And LSTM	See all

Datasets

Subtasks

Action Recognition

Greatest papers with code

AssembleNet: Searching for Multi-Stream Neural Connectivity in Video Architectures

tensorflow/models • • ICLR 2020

Learning to represent videos is a very challenging task both algorithmically and computationally.

Ranked #1 on Multimodal Activity Recognition on Moments in Time Dataset

Action Classification Action Recognition +3

70,697

Paper
Code

Group Normalization

facebookresearch/detectron • • ECCV 2018

FAIR's research platform for object detection research, implementing popular algorithms like Mask R-CNN and RetinaNet.

Ranked #73 on Object Detection on COCO minival

Object Detection Platform +1

24,519

Paper
Code

Non-local Neural Networks

facebookresearch/detectron • • CVPR 2018

Both convolutional and recurrent operations are building blocks that process one local neighborhood at a time.

Ranked #7 on Action Classification on Toyota Smarthome dataset (using extra training data)

Action Classification Action Recognition +3

24,519

Paper
Code

Revisiting ResNets: Improved Training and Scaling Strategies

rwightman/pytorch-image-models • • 13 Mar 2021

Using improved training and scaling strategies, we design a family of ResNet architectures, ResNet-RS, which are 1. 7x - 2. 7x faster than EfficientNets on TPUs, while achieving similar accuracies on ImageNet.

Ranked #27 on Image Classification on CIFAR-100 (using extra training data)

Action Classification Image Classification +1

11,782

Paper
Code

X3D: Expanding Architectures for Efficient Video Recognition

facebookresearch/SlowFast • • CVPR 2020

This paper presents X3D, a family of efficient video networks that progressively expand a tiny 2D image classification architecture along multiple network axes, in space, time, width and depth.

Ranked #23 on Action Classification on Kinetics-400

Action Classification Feature Selection +4

3,944

Paper
Code

A Multigrid Method for Efficiently Training Video Models

facebookresearch/SlowFast • • CVPR 2020

We empirically demonstrate a general and robust grid schedule that yields a significant out-of-the-box training speedup without a loss in accuracy for different models (I3D, non-local, SlowFast), datasets (Kinetics, Something-Something, Charades), and training settings (with and without pre-training, 128 GPUs or 1 GPU).

Ranked #1 on Video Classification on Kinetics

Action Detection Action Recognition +1

3,944

Paper
Code

Would Mega-scale Datasets Further Enhance Spatiotemporal 3D CNNs?

kenshohara/3D-ResNets-PyTorch • • 10 Apr 2020

Therefore, in the present paper, we conduct exploration study in order to improve spatiotemporal 3D CNNs as follows: (i) Recently proposed large-scale video datasets help improve spatiotemporal 3D CNNs in terms of video classification accuracy.

General Classification Video Classification +1

2,978

Paper
Code

YouTube-8M: A Large-Scale Video Classification Benchmark

google/youtube-8m • • 27 Sep 2016

Despite the size of the dataset, some of our models train to convergence in less than a day on a single machine using TensorFlow.

Ranked #1 on Action Recognition In Videos on Sports-1M (Video hit@1 metric)

3D Face Reconstruction Action Recognition +2

2,076

Paper
Code

Temporal Segment Networks for Action Recognition in Videos

open-mmlab/mmaction • • 8 May 2017

Furthermore, based on the temporal segment networks, we won the video classification track at the ActivityNet challenge 2016 among 24 teams, which demonstrates the effectiveness of TSN and the proposed good practices.

Ranked #12 on Action Classification on Moments in Time (Top 5 Accuracy metric)