SKEP is a self-supervised pre-training method for sentiment analysis. With the help of automatically-mined knowledge, SKEP conducts sentiment masking and constructs three sentiment knowledge prediction objectives, so as to embed sentiment information at the word, polarity and aspect level into pre-trained sentiment representation. In particular, the prediction of aspect-sentiment pairs is converted into multi-label classification, aiming to capture the dependency between words in a pair.
SKEP contains two parts: (1) Sentiment masking recognizes the sentiment information of an input sequence based on automatically-mined sentiment knowledge, and produces a corrupted version by removing these informations. (2) Sentiment pre-training objectives require the transformer to recover the removed information from the corrupted version. The three prediction objectives on top are jointly optimized: Sentiment Word (SW) prediction (on $\left.\mathrm{x}_{9}\right)$, Word Polarity (SP) prediction (on $\mathrm{x}_{6}$ and $\mathbf{x}_{9}$ ), Aspect-Sentiment pairs (AP) prediction (on $\mathbf{x}_{1}$ ). Here, the smiley denotes positive polarity. Notably, on $\mathrm{x}_{6}$, only SP is calculated without SW, as its original word has been predicted in the pair prediction on $\mathbf{x}_{1}$.
Source: SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment AnalysisPaper | Code | Results | Date | Stars |
---|
Task | Papers | Share |
---|---|---|
Multi-Label Classification | 1 | 33.33% |
Sentiment Analysis | 1 | 33.33% |
Stock Market Prediction | 1 | 33.33% |
Component | Type |
|
---|---|---|
🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |