An empirical study of the effect of background data size on the stability of SHapley Additive exPlanations (SHAP) for deep learning models

no code implementations • 24 Apr 2022 • Han Yuan, Mingxuan Liu, Lican Kang, Chenkui Miao, Ying Wu

In our empirical study on the MIMIC-III dataset, we show that the two core explanations - SHAP values and variable rankings fluctuate when using different background datasets acquired from random sampling, indicating that users cannot unquestioningly trust the one-shot interpretation from SHAP.