Tag Recommendation for Online Q&A Communities based on BERT Pre-Training Technique
Online Q&A and open source communities use tags and keywords to index, categorize, and search for specific content. The most obvious advantage of tag recommendation is the correct classification of information. In this study, we used the BERT pre-training technique in tag recommendation task for online Q&A and open-source communities for the first time. Our evaluation on freecode datasets show that the proposed method, called TagBERT, is more accurate compared to deep learning and other baseline methods. Moreover, our model achieved a high stability by solving the problem of previous researches, where increasing the number of tag recommendations significantly reduced model performance.
PDF Abstract
Datasets
Add Datasets
introduced or used in this paper
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
|---|---|---|---|---|---|---|
| Multi-Label Text Classification | Freecode | TagBERT | F1-score | 46 | # 1 | |
| Multi-Label Text Classification | Freecode | TagCNN | F1-score | 45.3 | # 2 | |
| Multi-Label Text Classification | Freecode | FastTagRec | F1-score | 33.2 | # 5 | |
| Multi-Label Text Classification | Freecode | EnTagRec | F1-score | 36 | # 4 | |
| Multi-Label Text Classification | Freecode | TagMulRec | F1-score | 36.4 | # 3 |
Methods
Adam •
Attention Dropout •
BERT •
Dense Connections •
Dropout •
GELU •
Layer Normalization •
Linear Layer •
Linear Warmup With Linear Decay •
Multi-Head Attention •
Residual Connection •
Scaled Dot-Product Attention •
Softmax •
Weight Decay •
WordPiece
