CTLR@WiC-TSV: Target Sense Verification using Marked Inputs andPre-trained Models
This paper describes the CTRL participation in the Target Sense Verification of the Words in Context challenge (WiC-TSV) at SemDeep6. Our strategy is based on a simplistic annotation scheme of the target words to later be classified by well-known pre-trained neural models. In particular, the marker allows to include position information to help models to correctly identify the word to disambiguate. Results on the challenge show that our strategy outperforms other participants (+11, 4 Accuracy points) and strong baselines (+1, 7 Accuracy points).
PDF Abstract SemDeep 2021 PDF SemDeep 2021 AbstractTasks
Datasets
Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
---|---|---|---|---|---|---|
Entity Linking | WiC-TSV | CTLR | Task 1 Accuracy: all | 76.8 | # 2 | |
Task 1 Accuracy: general purpose | 74.5 | # 3 | ||||
Task 1 Accuracy: domain specific | 79.6 | # 2 | ||||
Task 2 Accuracy: all | 72.7 | # 1 | ||||
Task 2 Accuracy: general purpose | 65.6 | # 2 | ||||
Task 2 Accuracy: domain specific | 81.5 | # 1 | ||||
Task 3 Accuracy: all | 78.3 | # 2 | ||||
Task 3 Accuracy: general purpose | 72.1 | # 4 | ||||
Task 3 Accuracy: domain specific | 85.7 | # 2 |