SimCSE: Simple Contrastive Learning of Sentence Embeddings

18 Apr 2021 Tianyu Gao Xingcheng Yao Danqi Chen

This paper presents SimCSE, a simple contrastive learning framework that greatly advances the state-of-the-art sentence embeddings. We first describe an unsupervised approach, which takes an input sentence and predicts itself in a contrastive objective, with only standard dropout used as noise... (read more)

PDF Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Semantic Textual Similarity SICK SimCSE-RoBERTalarge Spearman Correlation 0.8195 # 1
Semantic Textual Similarity STS12 SimCSE-RoBERTalarge Spearman Correlation 0.7746 # 1
Semantic Textual Similarity STS13 SimCSE-RoBERTalarge Spearman Correlation 0.8727 # 1
Semantic Textual Similarity STS14 SimCSE-RoBERTalarge Spearman Correlation 0.8236 # 1
Semantic Textual Similarity STS15 SimCSE-RoBERTalarge Spearman Correlation 0.8666 # 1
Semantic Textual Similarity STS16 SimCSE-RoBERTalarge Spearman Correlation 0.8393 # 1
Semantic Textual Similarity STS Benchmark SimCSE-RoBERTalarge Spearman Correlation 0.867 # 8

Methods used in the Paper


METHOD TYPE
Dropout
Regularization