Figure Eight at SemEval-2019 Task 3: Ensemble of Transfer Learning Methods for Contextual Emotion Detection
This paper describes our transfer learning-based approach to contextual emotion detection as part of SemEval-2019 Task 3. We experiment with transfer learning using pre-trained language models (ULMFiT, OpenAI GPT, and BERT) and fine-tune them on this task. We also train a deep learning model from scratch using pre-trained word embeddings and BiLSTM architecture with attention mechanism. The ensembled model achieves competitive result, ranking ninth out of 165 teams. The result reveals that ULMFiT performs best due to its superior fine-tuning techniques. We propose improvements for future work.
PDF Abstract
Datasets
Add Datasets
introduced or used in this paper
Results from the Paper
Submit
results from this paper
to get state-of-the-art GitHub badges and help the
community compare results to other papers.
Methods
Activation Regularization •
Adam •
Attention Dropout •
AWD-LSTM •
BERT •
BiLSTM •
BPE •
Cosine Annealing •
Dense Connections •
Discriminative Fine-Tuning •
DropConnect •
Dropout •
Embedding Dropout •
GELU •
GPT •
Layer Normalization •
Linear Layer •
Linear Warmup With Cosine Annealing •
Linear Warmup With Linear Decay •
LSTM •
Multi-Head Attention •
Residual Connection •
Scaled Dot-Product Attention •
Sigmoid Activation •
Slanted Triangular Learning Rates •
Softmax •
Tanh Activation •
Temporal Activation Regularization •
ULMFiT •
Variational Dropout •
Weight Decay •
Weight Tying •
WordPiece
