Temporal Relational Reasoning in Videos

ECCV 2018 Bolei ZhouAlex AndonianAude OlivaAntonio Torralba

Temporal relational reasoning, the ability to link meaningful transformations of objects or entities over time, is a fundamental property of intelligent species. In this paper, we introduce an effective and interpretable network module, the Temporal Relation Network (TRN), designed to learn and reason about temporal dependencies between video frames at multiple time scales... (read more)

PDF Abstract

Evaluation results from the paper


Task Dataset Model Metric name Metric value Global rank Uses extra
training data
Compare
Action Classification Charades MultiScale TRN MAP 25.2 # 7
Action Recognition In Videos Charades MultiScale TRN MAP 24.1 # 9
Action Recognition In Videos Jester MultiScale TRN Val 95.31 # 2
Action Classification Moments in Time TRN-Multiscale Top 1 Accuracy 28.27% # 5
Action Classification Moments in Time TRN-Multiscale Top 5 Accuracy 53.87% # 4
Action Recognition In Videos Something-Something V1 2-Stream TRN Top 1 Accuracy 42.01 # 14
Action Recognition In Videos Something-Something V2 2-Stream TRN Top-1 Accuracy 55.52 # 6
Action Recognition In Videos Something-Something V2 2-Stream TRN Top-5 Accuracy 83.06 # 6