Group-aware Contrastive Regression for Action Quality Assessment

Assessing action quality is challenging due to the subtle differences between videos and large variations in scores. Most existing approaches tackle this problem by regressing a quality score from a single video, suffering a lot from the large inter-video score variations. In this paper, we show that the relations among videos can provide important clues for more accurate action quality assessment during both training and inference. Specifically, we reformulate the problem of action quality assessment as regressing the relative scores with reference to another video that has shared attributes (e.g., category and difficulty), instead of learning unreferenced scores. Following this formulation, we propose a new Contrastive Regression (CoRe) framework to learn the relative scores by pair-wise comparison, which highlights the differences between videos and guides the models to learn the key hints for assessment. In order to further exploit the relative information between two videos, we devise a group-aware regression tree to convert the conventional score regression into two easier sub-problems: coarse-to-fine classification and regression in small intervals. To demonstrate the effectiveness of CoRe, we conduct extensive experiments on three mainstream AQA datasets including AQA-7, MTL-AQA and JIGSAWS. Our approach outperforms previous methods by a large margin and establishes new state-of-the-art on all three benchmarks.

PDF Abstract ICCV 2021 PDF ICCV 2021 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Action Quality Assessment AQA-7 CoRe Spearman Correlation 84.01% # 1
RL2(*100) 2.12 # 1
Action Quality Assessment AQA-7 I3D+MLP Spearman Correlation 76.01% # 3
RL2(*100) 3.20 # 3
Action Quality Assessment MTL-AQA I3D+MLP(w/ DD) Spearman Correlation 93.81 # 2
RL2(*100) 0.394 # 3
Action Quality Assessment MTL-AQA CoRe Spearman Correlation 93.41 # 3
RL2(*100) 0.365 # 2
Action Quality Assessment MTL-AQA I3D+MLP Spearman Correlation 91.96 # 7
RL2(*100) 0.465 # 5
Action Quality Assessment MTL-AQA CoRe(w/ DD) Spearman Correlation 95.12 # 1
RL2(*100) 0.260 # 1

Methods


No methods listed for this paper. Add relevant methods here