CTLR@WiC-TSV: Target Sense Verification using Marked Inputs andPre-trained Models

This paper describes the CTRL participation in the Target Sense Verification of the Words in Context challenge (WiC-TSV) at SemDeep6. Our strategy is based on a simplistic annotation scheme of the target words to later be classified by well-known pre-trained neural models. In particular, the marker allows to include position information to help models to correctly identify the word to disambiguate. Results on the challenge show that our strategy outperforms other participants (+11, 4 Accuracy points) and strong baselines (+1, 7 Accuracy points).

PDF Abstract SemDeep 2021 PDF SemDeep 2021 Abstract
No code implementations yet. Submit your code now

Datasets


Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Entity Linking WiC-TSV CTLR Task 1 Accuracy: all 76.8 # 2
Task 1 Accuracy: general purpose 74.5 # 3
Task 1 Accuracy: domain specific 79.6 # 2
Task 2 Accuracy: all 72.7 # 1
Task 2 Accuracy: general purpose 65.6 # 2
Task 2 Accuracy: domain specific 81.5 # 1
Task 3 Accuracy: all 78.3 # 2
Task 3 Accuracy: general purpose 72.1 # 4
Task 3 Accuracy: domain specific 85.7 # 2

Methods