What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment

CVPR 2019  ·  Paritosh Parmar, Brendan Tran Morris ·

Can performance on the task of action quality assessment (AQA) be improved by exploiting a description of the action and its quality? Current AQA and skills assessment approaches propose to learn features that serve only one task - estimating the final score. In this paper, we propose to learn spatio-temporal features that explain three related tasks - fine-grained action recognition, commentary generation, and estimating the AQA score. A new multitask-AQA dataset, the largest to date, comprising of 1412 diving samples was collected to evaluate our approach (https://github.com/ParitoshParmar/MTL-AQA). We show that our MTL approach outperforms STL approach using two different kinds of architectures: C3D-AVG and MSCADC. The C3D-AVG-MTL approach achieves the new state-of-the-art performance with a rank correlation of 90.44%. Detailed experiments were performed to show that MTL offers better generalization than STL, and representations from action recognition models are not sufficient for the AQA task and instead should be learned.

PDF Abstract CVPR 2019 PDF CVPR 2019 Abstract


Introduced in the Paper:


Used in the Paper:


Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Action Recognition MTL-AQA C3D-AVG Position Accuracy 96.32 % # 1
Armstand Accuracy 99.72 % # 1
Rotation Type Accuracy 97.45 % # 1
No. of Somersaults Accuracy 96.88 % # 1
No. of Twists Accuracy 93.20 % # 1
Action Quality Assessment MTL-AQA C3D-AVG-MTL Spearman Correlation 90.44 # 13
Action Quality Assessment MTL-AQA MSCADC-MTL Spearman Correlation 86.12 # 16
Action Quality Assessment MTL-AQA MSCADC-STL Spearman Correlation 84.72 # 17
Action Quality Assessment MTL-AQA C3D-AVG-STL Spearman Correlation 89.60 # 14


No methods listed for this paper. Add relevant methods here