Semantic Textual Similarity
557 papers with code • 13 benchmarks • 17 datasets
Semantic textual similarity deals with determining how similar two pieces of texts are. This can take the form of assigning a score from 1 to 5. Related tasks are paraphrase or duplicate identification.
Image source: Learning Semantic Textual Similarity from Conversations
Benchmarks
These leaderboards are used to track progress in Semantic Textual Similarity
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Semantic Textual Similarity models and implementations
GLUE
MRPC
SICK
SentEval
CARER
MTEB
STS Benchmark
EVALution
CxC
Most implemented papers
ERNIE: Enhanced Representation through Knowledge Integration
We present a novel language representation model enhanced by knowledge called ERNIE (Enhanced Representation through kNowledge IntEgration).
FNet: Mixing Tokens with Fourier Transforms
google-research/google-research
•
•
NAACL 2022
At longer input lengths, our FNet model is significantly faster: when compared to the "efficient" Transformers on the Long Range Arena benchmark, FNet matches the accuracy of the most accurate models, while outpacing the fastest models across all sequence lengths on GPUs (and across relatively shorter lengths on TPUs).
Improving Language Understanding by Generative Pre-Training
huggingface/transformers
•
•
Preprint 2018
We demonstrate that large gains on these tasks can be realized by generative pre-training of a language model on a diverse corpus of unlabeled text, followed by discriminative fine-tuning on each specific task.
Big Bird: Transformers for Longer Sequences
google-research/bigbird
•
•
NeurIPS 2020
To remedy this, we propose, BigBird, a sparse attention mechanism that reduces this quadratic dependency to linear.
DeBERTa: Decoding-enhanced BERT with Disentangled Attention
microsoft/DeBERTa
•
•
ICLR 2021
Recent progress in pre-trained neural language models has significantly improved the performance of many natural language processing (NLP) tasks.
TinyBERT: Distilling BERT for Natural Language Understanding
huawei-noah/Pretrained-Language-Model
•
•
Findings of the Association for Computational Linguistics 2020
To accelerate inference and reduce model size while maintaining accuracy, we first propose a novel Transformer distillation method that is specially designed for knowledge distillation (KD) of the Transformer-based models.
TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learning
UKPLab/sentence-transformers
•
•
Learning sentence embeddings often requires a large amount of labeled data.
SpanBERT: Improving Pre-training by Representing and Predicting Spans
facebookresearch/SpanBERT
•
•
TACL 2020
We present SpanBERT, a pre-training method that is designed to better represent and predict spans of text.
SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization
However, due to limited data resources from downstream tasks and the extremely large capacity of pre-trained models, aggressive fine-tuning often causes the adapted model to overfit the data of downstream tasks and forget the knowledge of the pre-trained model.
Language-agnostic BERT Sentence Embedding
FreddeFrallan/Multilingual-CLIP
•
•
ACL 2022
While BERT is an effective method for learning monolingual sentence embeddings for semantic similarity and embedding based transfer learning (Reimers and Gurevych, 2019), BERT based cross-lingual sentence embeddings have yet to be explored.
