SimCSE is a contrastive learning framework for generating sentence embeddings. It utilizes an unsupervised approach, which takes an input sentence and predicts itself in contrastive objective, with only standard dropout used as noise. The authors find that dropout acts as minimal “data augmentation” of hidden representations, while removing it leads to a representation collapse. Afterwards a supervised approach is used, which incorporates annotated pairs from natural language inference datasets into the contrastive framework, by using “entailment” pairs as positives and “contradiction
Source: SimCSE: Simple Contrastive Learning of Sentence EmbeddingsPaper | Code | Results | Date | Stars |
---|
Task | Papers | Share |
---|---|---|
Sentence | 36 | 26.09% |
Sentence Embeddings | 28 | 20.29% |
Semantic Textual Similarity | 18 | 13.04% |
Sentence Embedding | 13 | 9.42% |
Language Modelling | 7 | 5.07% |
Retrieval | 6 | 4.35% |
Semantic Similarity | 4 | 2.90% |
Question Answering | 2 | 1.45% |
Information Retrieval | 2 | 1.45% |
Component | Type |
|
---|---|---|
🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |