Time-Mapping Using Space-Time Saliency
We describe a new approach for generating regular-speed, low-frame-rate (LFR) video from a high-frame-rate (HFR) input while preserving the important moments in the original. We call this time-mapping, a time-based analogy to high dynamic range to low dynamic range spatial tone-mapping. Our approach makes these contributions: (1) a robust space-time saliency method for evaluating visual importance, (2) a re-timing technique to temporally resample based on frame importance, and (3) temporal filters to enhance the rendering of salient motion. Results of our space-time saliency method on a benchmark dataset show it is state-of-the-art. In addition, the benefits of our approach to HFR-to-LFR time-mapping over more direct methods are demonstrated in a user study.
PDF Abstract
FBMS
FBMS-59
Results from the Paper
Ranked #6 on
Video Salient Object Detection
on DAVSOD-Difficult20
(using extra training data)
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Uses Extra Training Data |
Benchmark |
|---|---|---|---|---|---|---|---|
| Video Salient Object Detection | DAVIS-2016 | TIMP | S-Measure | 0.574 | # 9 | ||
| MAX E-MEASURE | 0.680 | # 8 | |||||
| AVERAGE MAE | 0.185 | # 1 | |||||
| Video Salient Object Detection | DAVSOD-Difficult20 | TIMP | S-Measure | 0.530 | # 6 | ||
| max E-measure | 0.665 | # 6 | |||||
| Average MAE | 0.190 | # 6 | |||||
| Video Salient Object Detection | DAVSOD-easy35 | TIMP | S-Measure | 0.534 | # 7 | ||
| max E-Measure | 0.582 | # 8 | |||||
| Average MAE | 0.206 | # 6 | |||||
| Video Salient Object Detection | DAVSOD-Normal25 | TIMP | S-Measure | 0.503 | # 7 | ||
| max E-measure | 0.616 | # 5 | |||||
| Average MAE | 0.245 | # 6 | |||||
| Video Salient Object Detection | FBMS-59 | TIMP | S-Measure | 0.576 | # 15 | ||
| AVERAGE MAE | 0.192 | # 14 | |||||
| MAX F-MEASURE | 0.465 | # 15 | |||||
| Video Salient Object Detection | UVSD | TIMP | S-Measure | 0.541 | # 8 | ||
| max E-measure | 0.662 | # 8 | |||||
| Average MAE | 0.171 | # 8 | |||||
| Video Salient Object Detection | ViSal | TIMP | S-Measure | 0.612 | # 9 | ||
| max E-measure | 0.743 | # 9 | |||||
| Average MAE | 0.170 | # 9 | |||||
| Video Salient Object Detection | VOS-T | TIMP | S-Measure | 0.546 | # 9 | ||
| max E-measure | 0.640 | # 9 | |||||
| Average MAE | 0.192 | # 9 |
