Resolving the Scope of Speculation and Negation using Transformer-Based Architectures

Speculation is a naturally occurring phenomena in textual data, forming an integral component of many systems, especially in the biomedical information retrieval domain. Previous work addressing cue detection and scope resolution (the two subtasks of speculation detection) have ranged from rule-based systems to deep learning-based approaches... (read more)

PDF Abstract

Datasets


  Add Datasets introduced or used in this paper

Results from the Paper


 Ranked #1 on Negation Scope Resolution on BioScope : Abstracts (using extra training data)

     Get a GitHub badge
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK USES EXTRA
TRAINING DATA
BENCHMARK
Negation Scope Resolution BioScope : Abstracts XLNet F1 95.74 # 1
Speculation Scope Resolution BioScope : Abstracts XLNet F1 97.87 # 1
Speculation Scope Resolution BioScope : Full Papers XLNet F1 96.91 # 1
Negation Scope Resolution BioScope : Full Papers XLNet F1 94.4 # 1
Negation Scope Resolution *sem 2012 Shared Task: Sherlock Dataset RoBERTa F1 91.59 # 2
Negation Scope Resolution SFU Review Corpus XLNet F1 91.25 # 1
Speculation Scope Resolution SFU Review Corpus XLNet F1 91.00 # 1

Methods used in the Paper


METHOD TYPE
Residual Connection
Skip Connections
Attention Dropout
Regularization
Linear Warmup With Linear Decay
Learning Rate Schedules
BPE
Subword Segmentation
Weight Decay
Regularization
RoBERTa
Transformers
SentencePiece
Tokenizers
GELU
Activation Functions
Dense Connections
Feedforward Networks
Adam
Stochastic Optimization
WordPiece
Subword Segmentation
Softmax
Output Functions
Dropout
Regularization
Multi-Head Attention
Attention Modules
Layer Normalization
Normalization
Scaled Dot-Product Attention
Attention Mechanisms
XLNet
Transformers
BERT
Language Models