Few-Shot Document-Level Relation Extraction

NAACL 2022  ยท  Nicholas Popovic, Michael Fรคrber ยท

We present FREDo, a few-shot document-level relation extraction (FSDLRE) benchmark. As opposed to existing benchmarks which are built on sentence-level relation extraction corpora, we argue that document-level corpora provide more realism, particularly regarding none-of-the-above (NOTA) distributions. Therefore, we propose a set of FSDLRE tasks and construct a benchmark based on two existing supervised learning data sets, DocRED and sciERC. We adapt the state-of-the-art sentence-level method MNAV to the document-level and develop it further for improved domain adaptation. We find FSDLRE to be a challenging setting with interesting new characteristics such as the ability to sample NOTA instances from the support set. The data, code, and trained models are available online (https://github.com/nicpopovic/FREDo).

PDF Abstract NAACL 2022 PDF NAACL 2022 Abstract

Datasets


Introduced in the Paper:

FREDo

Used in the Paper:

DocRED SciERC
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Few-Shot Relation Classification DocRED DL-MNAV F1 (1-Doc) 7.05 # 1
F1 (3-Doc) 8.42 # 1
Few-Shot Relation Classification FREDo DL-MNAV F1 (1-Doc) 7.05 # 1
F1 (3-Doc) 8.42 # 1
Few-Shot Relation Classification FREDo (cross-domain) DL-MNAV+SIE+SBN F1 (1-Doc) 2.85 # 1
F1 (3-Doc) 3.72 # 1
Few-Shot Relation Classification SciERC DL-MNAV+SIE+SBN F1 (1-Doc) 2.85 # 1
F1 (3-Doc) 3.72 # 1

Methods


No methods listed for this paper. Add relevant methods here