MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scale

We study the zero-shot transfer capabilities of text matching models on a massive scale, by self-supervised training on 140 source domains from community question answering forums in English. We investigate the model performances on nine benchmarks of answer selection and question similarity tasks, and show that all 140 models transfer surprisingly well, where the large majority of models substantially outperforms common IR baselines... (read more)

PDF Abstract EMNLP 2020 PDF EMNLP 2020 Abstract

Datasets


Results from the Paper


  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.

Methods used in the Paper


METHOD TYPE
Adam
Stochastic Optimization
Softmax
Output Functions
Residual Connection
Skip Connections
GELU
Activation Functions
Dense Connections
Feedforward Networks
Layer Normalization
Normalization
WordPiece
Subword Segmentation
Multi-Head Attention
Attention Modules
Weight Decay
Regularization
Dropout
Regularization
Linear Warmup With Linear Decay
Learning Rate Schedules
Attention Dropout
Regularization
Scaled Dot-Product Attention
Attention Mechanisms
BERT
Language Models