Unsupervised Offline Changepoint Detection Ensembles

9 May 2021  ยท  Katser I, Kozitsin V, Lobachev V, Maksimov I. ยท

Offline changepoint detection (CPD) algorithms are used for signal segmentation in an optimal way. Generally, these algorithms are based on the assumption that signalโ€™s changed statistical properties are known, and the appropriate models (metrics, cost functions) for changepoint detection are used. Otherwise, the process of proper model selection can become laborious and time-consuming with uncertain results. Although an ensemble approach is well known for increasing the robustness of the individual algorithms and dealing with mentioned challenges, it is weakly formalized and much less highlighted for CPD problems than for outlier detection or classification problems. This paper proposes an unsupervised CPD ensemble (CPDE) procedure with the pseudocode of the particular proposed ensemble algorithms and the link to their Python realization. The approachโ€™s novelty is in aggregating several cost functions before the changepoint search procedure running during the offline analysis. The numerical experiment showed that the proposed CPDE outperforms non-ensemble CPD procedures. Additionally, we focused on analyzing common CPD algorithms, scaling, and aggregation functions, comparing them during the numerical experiment. The results were obtained on the two anomaly benchmarks that contain industrial faults and failuresโ€”Tennessee Eastman Process (TEP) and Skoltech Anomaly Benchmark (SKAB). One of the possible applications of our research is the estimation of the failure time for fault identification and isolation problems of the technical diagnostics.

PDF Abstract

Datasets


Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Change Point Detection SKAB Opt CPD algorithm (Mahalanobis metric) NAB (standard) 22.37 # 4
NAB (lowFP) 19.9 # 3
NAB (LowFN) 23.37 # 4
Change Point Detection SKAB OptEnsemble CPDE algorithm (WeightedSum+Rank) NAB (standard) 23.07 # 3
NAB (lowFP) 20.52 # 2
NAB (LowFN) 24.35 # 3
Change Point Detection SKAB BinSegEnsemble CPDE algorithm (WeightedSum+Rank) NAB (standard) 18.1 # 7
NAB (lowFP) 15.36 # 7
NAB (LowFN) 19.51 # 6
Change Point Detection SKAB WinEnsemble CPDE algorithm (Sum+MinAbs) NAB (standard) 19.38 # 5
NAB (lowFP) 17.03 # 5
NAB (LowFN) 20.35 # 5
Change Point Detection SKAB BinSeg CPD algorithm (Mahalanobis metric) NAB (standard) 24.1 # 2
NAB (lowFP) 21.69 # 1
NAB (LowFN) 25.04 # 2
Change Point Detection SKAB Win CPD algorithm (l1 metric) NAB (standard) 18.4 # 6
NAB (lowFP) 16.22 # 6
NAB (LowFN) 19.19 # 7
Change Point Detection TEP BinSegEnsemble CPDE algorithm (Min+MinMax/Rank) NAB (standard) 41.81 # 1
NAB (lowFP) 41 # 1
NAB (LowFN) 42.16 # 1
Change Point Detection TEP OptEnsemble CPDE algorithm (Min+MinMax/Rank) NAB (standard) 41.81 # 1
NAB (lowFP) 41 # 1
NAB (LowFN) 42.16 # 1
Change Point Detection TEP Opt CPD algorithm (Mahalanobis metric) NAB (standard) 36.88 # 3
NAB (lowFP) 35.82 # 3
NAB (LowFN) 37.29 # 3
Change Point Detection TEP Win CPD algorithm (Mahalanobis metric) NAB (standard) 27.79 # 5
NAB (lowFP) 27 # 5
NAB (LowFN) 28.05 # 5
Change Point Detection TEP BinSeg CPD algorithm (Mahalanobis metric) NAB (standard) 36.88 # 3
NAB (lowFP) 35.82 # 3
NAB (LowFN) 37.29 # 3
Change Point Detection TEP WinEnsemble CPDE algorithm (WeightedSum+MinAbs) NAB (standard) 25.14 # 6
NAB (lowFP) 24.33 # 6
NAB (LowFN) 26.29 # 6

Methods


No methods listed for this paper. Add relevant methods here