A Unified Positive-Unlabeled Learning Framework for Document-Level Relation Extraction with Different Levels of Labeling

17 Oct 2022  ·  Ye Wang, Xinxin Liu, Wenxin Hu, Tao Zhang ·

Document-level relation extraction (RE) aims to identify relations between entities across multiple sentences. Most previous methods focused on document-level RE under full supervision. However, in real-world scenario, it is expensive and difficult to completely label all relations in a document because the number of entity pairs in document-level RE grows quadratically with the number of entities. To solve the common incomplete labeling problem, we propose a unified positive-unlabeled learning framework - shift and squared ranking loss positive-unlabeled (SSR-PU) learning. We use positive-unlabeled (PU) learning on document-level RE for the first time. Considering that labeled data of a dataset may lead to prior shift of unlabeled data, we introduce a PU learning under prior shift of training data. Also, using none-class score as an adaptive threshold, we propose squared ranking loss and prove its Bayesian consistency with multi-label ranking metrics. Extensive experiments demonstrate that our method achieves an improvement of about 14 F1 points relative to the previous baseline with incomplete labeling. In addition, it outperforms previous state-of-the-art results under both fully supervised and extremely unlabeled settings as well.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Document-level RE with incomplete labeling ChemDisGene SSR-PU F1 48.56 # 1
Document-level RE with incomplete labeling ChemDisGene ATLOP F1 42.73 # 2
Relation Extraction ReDocRED SSR-PU F1 78.86 # 3
Ign F1 77.67 # 3
Document-level RE with incomplete labeling Re-DocRED SSR-PU Ign F1 58.68 # 1
F1 59.50 # 1
Document-level RE with incomplete labeling Re-DocRED ATLOP Ign F1 45.09 # 2
F1 45.19 # 2

Methods


No methods listed for this paper. Add relevant methods here