KILT (KILT Benchmark)

Introduced by Petroni et al. in KILT: a Benchmark for Knowledge Intensive Language Tasks

KILT (Knowledge Intensive Language Tasks) is a benchmark consisting of 11 datasets representing 5 types of tasks:

Fact-checking (FEVER),
Entity linking (AIDA CoNLL-YAGO, WNED-WIKI, WNED-CWEB),
Slot filling (T-Rex, Zero Shot RE),
Open domain QA (Natural Questions, HotpotQA, TriviaQA, ELI5),
Dialog generation (Wizard of Wikipedia).

All these datasets have been grounded in a single pre-processed wikipedia snapshot, allowing for fairer and more consistent evaluation as well as enabling new task setups such as multitask and transfer learning.

Source: KILT Benchmarking

Homepage

Benchmarks

Add a new result Link an existing benchmark

Task	Dataset Variant	Best Model
Fact Verification	KILT: FEVER	Re2G
Open-Domain Dialog	KILT: Wizard of Wikipedia	Hindsight
Slot Filling	KILT: Zero Shot RE	Multitask DPR + BART
Slot Filling	KILT: T-REx	Re2G
Open-Domain Question Answering	KILT: Natural Questions	Re2G
Open-Domain Question Answering	KILT: ELI5	arxiv.org/abs/2103.06332
Open-Domain Question Answering	KILT: TriviaQA	Re2G
Open-Domain Question Answering	KILT: HotpotQA	Multitask DPR + BART
Entity Linking	KILT: AIDA-YAGO2	GENRE
Entity Linking	KILT: WNED-WIKI	GENRE
Entity Linking	KILT: WNED-CWEB	GENRE
Question Answering	KILT: ELI5	RBG