FIFA: Fast Inference Approximation for Action Segmentation

9 Aug 2021  ·  Yaser Souri, Yazan Abu Farha, Fabien Despinoy, Gianpiero Francesca, Juergen Gall ·

We introduce FIFA, a fast approximate inference method for action segmentation and alignment. Unlike previous approaches, FIFA does not rely on expensive dynamic programming for inference. Instead, it uses an approximate differentiable energy function that can be minimized using gradient-descent. FIFA is a general approach that can replace exact inference improving its speed by more than 5 times while maintaining its performance. FIFA is an anytime inference algorithm that provides a better speed vs. accuracy trade-off compared to exact inference. We apply FIFA on top of state-of-the-art approaches for weakly supervised action segmentation and alignment as well as fully supervised action segmentation. FIFA achieves state-of-the-art results on most metrics on two action segmentation datasets.

PDF Abstract

Datasets


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Action Segmentation Breakfast FIFA + MS-TCN F1@10% 75.5 # 13
F1@50% 54.8 # 19
Acc 68.6 # 20
Edit 78.5 # 2
F1@25% 70.2 # 12
Weakly Supervised Action Segmentation (Transcript) Breakfast FIFA + MuCon Acc 51.3 # 1

Methods


No methods listed for this paper. Add relevant methods here