Mitigating the Inconsistency Between Word Saliency and Model Confidence with Pathological Contrastive Training

no code implementations • Findings (ACL) 2022 • Pengwei Zhan, Yang Wu, Shaolei Zhou, Yunjian Zhang, Liming Wang

We show that the pathological inconsistency is caused by the representation collapse issue, which means that the representation of the sentences with tokens in different saliency reduced is somehow collapsed, and thus the important words cannot be distinguished from unimportant words in terms of model confidence changing.

Contrastive Learning