OpenCodePapers

semantic-textual-similarity-on-sts-benchmark

Semantic Textual Similarity
Dataset Link
Results over time
Click legend items to toggle metrics. Hover points for model names.
Leaderboard
Click a sortable column header to sort.
PaperCodePearson CorrelationSpearman CorrelationAccuracyDev Pearson CorrelationDev Spearman CorrelationModelNameReleaseDate
SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization✓ Link0.9290.925MT-DNN-SMART2019-11-08
StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding0.9280.924StructBERTRoBERTa ensemble2019-08-13
MNet-Sim: A Multi-layered Semantic Similarity Network to Evaluate Sentence Similarity0.9270.931Mnet-Sim2021-11-09
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer✓ Link0.9250.921T5-11B2019-10-23
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations✓ Link0.925ALBERT2019-09-26
XLNet: Generalized Autoregressive Pretraining for Language Understanding✓ Link0.925XLNet (single model)2019-06-19
RoBERTa: A Robustly Optimized BERT Pretraining Approach✓ Link0.922RoBERTa2019-07-26
[]()0.921ELECTRA
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale✓ Link0.919RoBERTa-large 355M (MLP quantized vector-wise, fine-tuned)2022-08-15
A Statistical Framework for Low-bitwidth Training of Deep Neural Networks✓ Link0.919PSQ (Chen et al., 2020)2020-10-27
Entailment as Few-Shot Learner✓ Link0.918RoBERTa-large 355M + Entailment as Few-shot Learner2021-04-29
ERNIE 2.0: A Continual Pre-training Framework for Language Understanding✓ Link0.912ERNIE 2.0 Large2019-07-29
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT0.911Q-BERT (Shen et al., 2020)2019-09-12
Q8BERT: Quantized 8Bit BERT✓ Link0.911Q8BERT (Zafrir et al., 2019)2019-10-14
[]()0.910ELECTRA (no tricks)
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter✓ Link0.907DistilBERT 66M2019-10-02
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer✓ Link0.9060.898T5-3B2019-10-23
CLEAR: Contrastive Learning for Sentence Representation0.905MLM+ del-word2020-12-31
RealFormer: Transformer Likes Residual Attention✓ Link0.90110.8988RealFormer2020-12-21
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer✓ Link0.899T5-Large2019-10-23
SpanBERT: Improving Pre-training by Representing and Predicting Spans✓ Link0.899SpanBERT2019-07-24
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer✓ Link0.894T5-Base2019-10-23
ERNIE 2.0: A Continual Pre-training Framework for Language Understanding✓ Link0.876ERNIE 2.0 Base2019-07-29
Charformer: Fast Character Transformers via Gradient-based Subword Tokenization✓ Link0.873Charformer-Tall2021-06-23
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer✓ Link0.8560.85T5-Small2019-10-23
ERNIE: Enhanced Language Representation with Informative Entities✓ Link0.832ERNIE2019-05-17
How to Train BERT with an Academic Budget✓ Link0.82024hBERT2021-04-15
TinyBERT: Distilling BERT for Natural Language Understanding✓ Link0.799TinyBERT-4 14.5M2019-09-23
Universal Sentence Encoder✓ Link0.782USE_T2018-03-29
AnglE-optimized Text Embeddings✓ Link0.8969AnglE-LLaMA-13B2023-09-22
Adversarial Self-Attention for Language Understanding✓ Link0.892ASA + RoBERTa2022-06-25
Scaling Sentence Embeddings with Large Language Models✓ Link0.8914PromptEOL+CSE+LLaMA-30B2023-07-31
AnglE-optimized Text Embeddings✓ Link0.8897AnglE-LLaMA-7B2023-09-22
AnglE-optimized Text Embeddings✓ Link0.8897AnglE-LLaMA-7B-v22023-09-22
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer✓ Link0.886T5-Large 770M2019-10-23
Scaling Sentence Embeddings with Large Language Models✓ Link0.8856PromptEOL+CSE+OPT-13B2023-07-31
Scaling Sentence Embeddings with Large Language Models✓ Link0.8833PromptEOL+CSE+OPT-2.7B2023-07-31
Improved Universal Sentence Embeddings with Prompt-based Contrastive Learning and Energy-based Learning✓ Link0.8787PromCSE-RoBERTa-large (0.355B)2022-03-14
Big Bird: Transformers for Longer Sequences✓ Link.878BigBird2020-07-28
SimCSE: Simple Contrastive Learning of Sentence Embeddings✓ Link0.867SimCSE-RoBERTalarge2021-04-18
Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations✓ Link0.867Trans-Encoder-RoBERTa-large-cross (unsup.)2021-09-27
Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations✓ Link0.8655Trans-Encoder-RoBERTa-large-bi (unsup.)2021-09-27
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding✓ Link0.865BERT-LARGE2018-10-11
Adversarial Self-Attention for Language Understanding✓ Link0.865ASA + BERT-base2022-06-25
Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations✓ Link0.8616Trans-Encoder-BERT-large-bi (unsup.)2021-09-27
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks✓ Link0.8615SRoBERTa-NLI-STSb-large2019-08-27
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks✓ Link0.8479SBERT-STSb-base2019-08-27
Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations✓ Link0.8465Trans-Encoder-RoBERTa-base-cross (unsup.)2021-09-27
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks✓ Link0.8445SBERT-STSb-large2019-08-27
FNet: Mixing Tokens with Fourier Transforms✓ Link0.84FNet-Large2021-05-09
Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations✓ Link0.839Trans-Encoder-BERT-base-bi (unsup.)2021-09-27
[]()0.7981Pearl
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks✓ Link0.79SBERT-NLI-large2019-08-27
Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders✓ Link0.787Mirror-RoBERTa-base (unsup.)2021-04-16
Generating Datasets with Pretrained Language Models✓ Link0.7782Dino (STSb/̄🦕)2021-04-15
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks✓ Link0.7777SRoBERTa-NLI-base2019-08-27
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks✓ Link0.7703SBERT-NLI-base2019-08-27
Generating Datasets with Pretrained Language Models✓ Link0.7651Dino (STS/̄🦕)2021-04-15
Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders✓ Link0.764Mirror-BERT-base (unsup.)2021-04-16
On the Sentence Embeddings from Pre-trained Language Models✓ Link0.7226BERTlarge-flow (target)2020-11-02
An Unsupervised Sentence Embedding Method by Mutual Information Maximization✓ Link0.6921IS-BERT-NLI2020-09-25
Rematch: Robust and Efficient Matching of Local Knowledge Graphs to Improve Structural and Semantic Similarity✓ Link0.6652Rematch2024-04-02
Def2Vec: Extensible Word Embeddings from Dictionary Definitions✓ Link0.6372Def2Vec2023-12-16
DeBERTa: Decoding-enhanced BERT with Disentangled Attention✓ Link92.5DeBERTa (large)2020-06-05
SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization✓ Link92.892.6SMARTRoBERTa2019-11-08
SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization✓ Link90.089.4SMART-BERT2019-11-08
Check out how to contribute new results.