2-Tier SimCSE: Elevating BERT for Robust Sentence Embeddings

Junjin Wang; Yumeng Wang; Ziran Zhou

REVIEW 4 major objections 7 minor 13 references

2-Tier SimCSE: Elevating BERT for Robust Sentence Embeddings

T0 review · 4 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that stacking unsupervised SimCSE followed by supervised SimCSE to fine-tune minBERT yields the best sentence-embedding model for semantic textual similarity, with a dev Pearson score of 0.811 and a test correlation of…

desk verdict A clear, honest course-project write-up whose single-run 0.005 STS gain is too thin to support the 'superior performance' claim, though the stacking recipe is genuinely new. read the letter →

arxiv 2501.13758 v1 pith:M4QFFR7Q submitted 2025-01-23 cs.CL cs.AIcs.LG

Yumeng Wang , Ziran Zhou , Junjin Wang This is my paper · ORCID

classification cs.CLcs.AIcs.LG

keywords sentenceembeddingsSimCSEcontrastivelearningsemantictextualsimilarityminBERTdropouttransfer2-Tierfine-tuning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that combining the two known SimCSE objectives in sequence—unsupervised first, supervised second—produces sentence embeddings from minBERT that are more accurate for semantic textual similarity than either objective alone. If true, it would mean that a cheap unlabeled-data contrastive stage can usefully prime a model before a labeled contrastive stage, without requiring new architectural components. The authors report a development Pearson correlation of 0.811 on STS, against 0.806 for supervised SimCSE alone and 0.716 for unsupervised SimCSE alone, and a test correlation of 0.788. The paper also claims that transfer of STS-trained SimCSE models to paraphrase and sentiment tasks does not help, and that adaptive dropout hurts SimCSE training through added parameters.

What carries the argument

The load-bearing mechanism is the 2-Tier SimCSE Fine-tuning Procedure, a sequential fine-tuning schedule: a minBERT model is first trained on the STS dataset, then fine-tuned with the unsupervised SimCSE contrastive objective (two dropout-masked passes of the same sentence as positives, in-batch sentences as negatives), and then fine-tuned again with the supervised SimCSE objective (NLI entailment pairs as positives, contradiction pairs as hard negatives). The contrastive objective is a cross-entropy loss with in-batch negatives, and the procedure's work is to combine a general alignment objective from unlabeled data with a task-aligned, label-driven objective; the paper attributes the STS gain to this stacking.

What would settle it

Run both the supervised-only SimCSE and the 2-Tier SimCSE from the same minBERT initialization for at least 10 random seeds with identical total compute and the paper's hyperparameters; if the mean test Pearson correlation of the 2-Tier model does not exceed that of supervised SimCSE by more than the standard error, the claimed superiority is not established.

Watch

Extended reading notes

Core claim

The central discovery claimed by the paper is that a two-tier fine-tuning procedure—first unsupervised SimCSE (dropout as augmentation on the STS corpus), then supervised SimCSE (NLI entailment pairs as positives and contradiction pairs as hard negatives)—raises the STS Pearson correlation of minBERT beyond either stage alone, reaching a dev score of 0.811 and a test correlation of 0.788. The paper further reports that transferring this model to paraphrase and sentiment tasks does not improve those tasks, and that removing adaptive dropout from the unsupervised SimCSE single-task model improves its STS score.

Load-bearing premise

The central claim rests on a single training run: the 0.005 point advantage of the 2-Tier model over supervised SimCSE alone on the development set is assumed to be a real effect rather than random seed or hyperparameter variation.

Editorial extensions

If this is right

The 2-Tier SimCSE model achieves a test Pearson correlation of 0.788 on STS and an average test score of 0.742 across SST, paraphrase, and STS.
Removing adaptive dropout from the single-task unsupervised SimCSE model improves its STS dev score from 0.656 to 0.716, indicating the added dropout parameters cause overfitting.
Transfer learning from STS-trained SimCSE models does not improve paraphrase or SST dev accuracy; the 2-Tier transfer reaches 0.785 on paraphrase and 0.490 on SST, near or below the single-task baselines.
The supervised SimCSE stage alone (0.806 dev Pearson) is close to the 2-Tier result (0.811), so the marginal gain of the unsupervised first stage is small on the development set.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the 0.005 dev gain is reproducible across seeds, the two-tier ordering may serve as a general recipe: an unsupervised contrastive pre-fine-tune could align the representation space before a supervised contrastive stage shapes it for a target task.
A testable extension is to apply the same two-tier schedule to other backbone models and other supervised contrastive datasets, where the gap could be larger when the unsupervised stage uses in-domain unlabeled text.
Since the paper reports one run, a fair reading is that the 2-Tier versus supervised-only difference is within seed noise until multi-seed confidence intervals are presented.
The adaptive-dropout ablation suggests parameter-count growth, not the dropout mechanism itself, drives the overfitting; comparing adaptive dropout with matched parameter counts would separate these effects.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know this one before reading: it is a Stanford CS224N project paper, clearly written and unusually detailed for the genre, but the central claim rests on a 0.005 dev Pearson difference from a single run. The proposed 2-Tier SimCSE—unsupervised SimCSE on STS, then supervised SimCSE on NLI—is not in the original SimCSE paper, so the configuration is new. What the paper does well: the experimental write-up is transparent about hyperparameters, the dropout ablations are sensible, and the error analysis is genuinely thoughtful (the SST and paraphrase examples are well chosen). The authors also report a negative transfer result, which is honest and useful. Credit where due: this is a solid course project. The soft spots are exactly where the stress-test note lands. The 2-Tier baseline is confounded: the supervised SimCSE baseline starts from vanilla minBERT, while 2-Tier starts from an STS-pretrained minBERT, so the 0.005 gap could just be extra task-specific pretraining, not the two-tier stacking. No ablation isolates the unsupervised tier. All numbers come from a single run with no error bars, and the abstract's 0.742 average test score mixes the best model per task, which muddies rather than supports the claim. The paper even says the 2-Tier result was 'better than expected,' which reads as cherry-picking rather than a pre-registered comparison. On the citation pattern: they cite the standard SimCSE, dropout, and NLI sources; nothing looks inflated. The math is simple contrastive learning, correctly stated. The empirical evidence is what it is: a small, possibly real, possibly noise-level gain. For whom is this useful? Someone teaching or learning SimCSE on a small budget, or anyone curious about how a two-stage contrastive recipe behaves on the STS benchmark. It does not deserve a full journal review, but it does deserve a serious referee if submitted to a workshop or as a short paper, because the stacking idea is testable and the failure analysis is above average. My own verdict is skeptical: the load-bearing claim is not established by the evidence presented. If I were advising the authors, the next step would be multi-seed runs, error bars, a clean ablation (STS-pretrained minBERT + supervised SimCSE without the unsupervised tier), and code release. Recommendation: send to peer review only as a workshop-level empirical report; the claim needs to be de-risked before it carries any weight.

Referee Report

4 major / 7 minor

Summary. The paper applies SimCSE-style contrastive fine-tuning to the minBERT model for three downstream tasks: sentiment analysis (SST), semantic textual similarity (STS), and paraphrase detection (QQP). It compares single-task and multitask baselines, experiments with standard, curriculum, and adaptive dropout, and proposes a '2-Tier SimCSE Fine-tuning Model' that first applies unsupervised SimCSE on STS and then supervised SimCSE on NLI data. The central empirical claim, in Section 4.3.6 and Table 7, is that this 2-Tier model achieves a dev Pearson score of 0.811 on STS, surpassing the supervised-SimCSE-only baseline (0.806). The paper also reports transfer-learning results on paraphrase and SST tasks, an error analysis, and an ablation of adaptive dropout. The abstract summarizes an average test score of 0.742 across the three tasks, assembled from the best model per task.

Significance. If the 2-Tier result were robust, it would be a simple and potentially useful recipe: sequentially combining unsupervised and supervised SimCSE objectives improves sentence embeddings for STS beyond either stage alone. The paper also provides a thoughtful error analysis and an honest ablation showing that adaptive dropout hurts in the unsupervised SimCSE setting, which is a useful negative result. However, the load-bearing evidence is thin: the headline improvement is a 0.005 dev Pearson difference from a single training run with no error bars or statistical testing, and the comparison is confounded by initialization differences between the 2-Tier model and the supervised-SimCSE baseline. The contribution is incremental relative to the existing SimCSE framework, and the evaluation does not yet establish the claimed superiority.

major comments (4)

[Section 4.3.6, Table 7] The central claim of 'superior performance' rests on a single training run: the 2-Tier model yields a dev Pearson score of 0.811 versus 0.806 for the supervised-SimCSE baseline (Section 4.3.4, Table 5). No standard deviations, multiple seeds, or statistical tests are reported anywhere in the paper. A gap of 0.005 on one run is within the range of typical seed-to-seed variation for contrastive fine-tuning, so the paper does not currently provide evidence that the 2-Tier stacking is reliably better than supervised SimCSE alone. The authors should report mean and standard deviation over at least 3-5 random seeds for both models and, ideally, a paired significance test.
[Section 4.3.6 vs. Section 4.3.4] The comparison between the 2-Tier model and the supervised-SimCSE baseline is confounded by initialization. The supervised baseline (Section 4.3.4) is described as starting from a single-task minBERT and then fine-tuning on NLI, whereas the 2-Tier model (Section 4.3.6) starts from a minBERT already pre-trained on STS, then applies unsupervised SimCSE on STS, and then supervised SimCSE on NLI. Thus the 0.005 dev gap cannot be attributed to the two-tier stacking of contrastive objectives; it may be entirely due to the extra STS pre-training stage. A missing ablation, such as STS-pretrained minBERT followed by supervised SimCSE without the unsupervised tier (or equivalently, supervised SimCSE initialized from the same checkpoint used for the 2-Tier model), is needed to isolate the effect of the unsupervised tier. Without this control, the paper's specific claim that combining two contrastive stages is beneficial is not established.
[Abstract and Section 4.3.7] The abstract's 'average test score of 0.742' is not a score of the 2-Tier model: it combines the best test scores from different models per task, namely the single-task baseline with standard dropout on paraphrase and SST, and the 2-Tier model on STS. This is stated explicitly in Section 4.3.7. Reporting a per-task 'best model' average obscures the actual performance of the proposed 2-Tier model, which is worse than the baseline on paraphrase (0.785 dev vs. 0.794) and much worse on SST (0.490 dev vs. 0.534). The average score should either be reported for a single consistent model, or the per-task best scores should be clearly labeled as an upper bound, not as the 2-Tier model's achievement.
[Section 4.3.7] The test leaderboard reports a single STS test correlation of 0.788 for the 2-Tier model but provides no test-set numbers for the supervised-SimCSE baseline or any other comparison model. Even if the dev-set gap were accepted as real, the paper does not demonstrate that the improvement transfers to the test set, because the baseline's test correlation is never given. The authors should report test results for all competing models under identical evaluation conditions.

minor comments (7)

[Section 4.3.1] The section is numbered '5.3.1' in the text, which appears to be a typo; the surrounding sections are numbered 4.3.x. Please renumber.
[Table 1] The description of the five scaling methods is unclear. For example, the first row ('Sum of two sentence pair embeddings with linear transformation') and the fifth row ('Cross-attention between sentence pair embeddings') report results of 0.381 and 0.338, but the text does not explain what these numbers mean (Pearson correlation on the dev set, presumably). Please define the evaluation protocol for Table 1 in a complete sentence.
[Section 3.2.1.1] The reference to Morerio et al. (2017) is appropriate for curriculum dropout, but the described implementation ('dynamically increasing dropout rates during training') is not precisely the 'standout' method cited; please clarify whether the implementation follows the original paper or a simplified variant.
[References] The reference 'Chuang Zhai, Yeyun Gong, Heng Zhang, Yuqing Kong, Weikang Wang, Guozhong Tang, Longbiao Wang, and Hanghang Tong. 2021. Contrastive learning for unsupervised sentence embedding enhancement.' is missing publication venue or arXiv identifier. The same applies to the Dasgupta et al. (2019) and Yin et al. (2019) entries, which have arXiv identifiers but no 'arXiv:' prefix.
[Figure 2 and Figure 3] The figures have no captions or axis labels in the text, making the accuracy comparison and the heatmap difficult to interpret. Please add proper captions and label the axes in Figure 3.
[Section 4.3.2] The sentence 'We observed the best-performing dropout strategies being adaptive dropout (highest performance on STS task) and standard dropout (highest performance on paraphrase and SST tasks)' is contradicted by Table 3, where adaptive dropout gives 0.579 on STS but standard dropout gives 0.502; the sentence should state the values explicitly so the reader can verify the claim.
[Appendix Table 8] The appendix table reports SST and CFIMDB pre-train/finetune results but is never referenced in the main text; either integrate it into the experimental narrative or remove it.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the STS result is an empirical benchmark evaluation against external data, not a derivation that reduces to its own inputs.

full rationale

This is an empirical evaluation paper rather than a derivation chain, so equation-level circularity is not present. The central claim in Section 4.3.6 / Table 7 — that the 2-Tier SimCSE model reaches a 0.811 dev Pearson score on STS, outperforming the single supervised SimCSE at 0.806 — is a measured comparison on the external SemEval STS benchmark, not a quantity reconstructed from fitted parameters or definitions. No parameter is fitted so that the reported metric equals its own input: the dev correlation is computed from model outputs and gold labels, and the test score of 0.788 is an independent leaderboard result. The method imports SimCSE from Gao et al. (2021), but the authors do not overlap with that prior work, and the paper does not rest its claim on that citation alone; the 2-Tier result is directly measured in-house. There is no self-citation chain, no imported uniqueness theorem, no ansatz smuggled in via citation, and no renaming of a known result as a new contribution. The skeptic's concern about missing an ablation that controls for the initial STS pre-training is a validity or attribution issue, not circularity: even if the gain were entirely due to extra STS pre-training, the reported number would still be an honest empirical measurement rather than an input to the method. Similarly, the absence of error bars and the mixing of best-per-task test scores are robustness and reporting weaknesses, not circular reductions. Therefore the paper is self-contained with respect to circularity, and the score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new theoretical entities; its contributions are configurations of existing techniques. The free parameters are hyperparameters and model choices selected using dev-set performance, and the axioms are standard assumptions from SimCSE and the course-project setup. The dev-set selection of the scaling method and the 2-Tier stages is the main source of potential bias in the reported gains.

free parameters (5)

STS similarity scaling method = sigmoid(cosine_similarity) * 5
Selected as best among five scaling variants in Table 1 using dev STS correlation; this choice is part of the method being evaluated.
Default training hyperparameters = lr=1e-5, dropout=0.3, batch_size=8, epochs=10
Used for baselines, transfer, and 2-Tier SST/paraphrase stages; chosen by course default rather than by fitting to data.
Unsupervised SimCSE hyperparameters = batch_size=64, lr=3e-5, dropout=0.1
Used for unsupervised SimCSE pre-training/fine-tuning; one configuration reported, likely tuned on dev.
Supervised SimCSE hyperparameters = batch_size=24, lr=5e-5, dropout=0.1, epochs=5
Used for supervised SimCSE on NLI; selected on dev and then reused in the 2-Tier model.
2-Tier stage selection = best unsupervised model then best supervised model, chosen by dev STS correlation
The composition is selected post hoc on the same dev set used to report the 0.811 result, adding selection optimism.

assumptions (5)

domain assumption Standard dropout masks act as sufficient data augmentation for contrastive sentence embedding learning.
Core SimCSE assumption from Gao et al. (2021), adopted in Section 3.2.2.1 without revalidation.
domain assumption NLI entailment/contradiction pairs provide useful positives and hard negatives for supervised SimCSE.
Used in Section 3.2.2.2 and Section 4.3.4; relies on SNLI/MNLI supervision rather than task-specific STS labels.
domain assumption Pearson correlation on the STS-dev set is a valid proxy for sentence embedding quality.
All model choices are ranked by this metric in Sections 4.3.2 through 4.3.6.
domain assumption minBERT is a strong enough base model that gains from SimCSE transfer to it.
The whole study is a fine-tuning exercise on minBERT, as stated in Section 3.1.
ad hoc to paper The best cosine scaling method can be chosen on the dev set without invalidating the final comparison.
Table 1 selects sigmoid scaling based on dev correlation; this is a model choice made on the same data used to report the central result.

how reviews work

0 comments

Cite this review

Pith. "Pith review of 2-Tier SimCSE: Elevating BERT for Robust Sentence Embeddings." pith.science (2026). https://pith.science/paper/M4QFFR7Q

@misc{pith2026250113758,
  author       = {Pith},
  title        = {Pith review of: 2-Tier SimCSE: Elevating BERT for Robust Sentence Embeddings},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M4QFFR7Q}},
  note         = {Machine review of arXiv:2501.13758}
}

read the original abstract

Effective sentence embeddings that capture semantic nuances and generalize well across diverse contexts are crucial for natural language processing tasks. We address this challenge by applying SimCSE (Simple Contrastive Learning of Sentence Embeddings) using contrastive learning to fine-tune the minBERT model for sentiment analysis, semantic textual similarity (STS), and paraphrase detection. Our contributions include experimenting with three different dropout techniques, namely standard dropout, curriculum dropout, and adaptive dropout, to tackle overfitting, proposing a novel 2-Tier SimCSE Fine-tuning Model that combines both unsupervised and supervised SimCSE on STS task, and exploring transfer learning potential for Paraphrase and SST tasks. Our findings demonstrate the effectiveness of SimCSE, with the 2-Tier model achieving superior performance on the STS task, with an average test score of 0.742 across all three downstream tasks. The results of error analysis reveals challenges in handling complex sentiments and reliance on lexical overlap for paraphrase detection, highlighting areas for future research. The ablation study revealed that removing Adaptive Dropout in the Single-Task Unsupervised SimCSE Model led to improved performance on the STS task, indicating overfitting due to added parameters. Transfer learning from SimCSE models on Paraphrase and SST tasks did not enhance performance, suggesting limited transferability of knowledge from the STS task.

Figures

Figures reproduced from arXiv: 2501.13758 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 6 canonical work pages

[1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Jimmy Ba and Brendan Frey. 2013. https://proceedings.neurips.cc/paper_files/paper/2013/file/7b5b23f4aadf9513306bcd59afb6e4c9-Paper.pdf Adaptive dropout . In Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc

work page 2013
[4]

Bowman, Gabor Angeli, Christopher Potts, and Christopher D

Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. Stanford natural language inference (snli) corpus. https://nlp.stanford.edu/projects/snli/. Website

work page 2015
[5]

Dynamic Attention Networks for Task Oriented Grounding

Soumik Dasgupta, Badri N. Patro, and Vinay P. Namboodiri. 2019. http://arxiv.org/abs/1910.06315 Dynamic attention networks for task oriented grounding . arXiv preprint arXiv:1910.06315

work page Pith review arXiv 2019
[6]

Tianyu Gao, Xuancheng Wang, Changyou Chen, and Jian Tang. 2019. Anisotropic vector spaces learning for natural language processing. arXiv preprint arXiv:1907.12009

arXiv 2019
[7]

Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.552 Simcse: Simple contrastive learning of sentence embeddings . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894--6910, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics

doi:10.18653/v1/2021.emnlp-main.552 2021
[8]

Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9729--9738

work page 2020

Show all 13 references

[9]

Kingma and Jimmy Ba

Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

2014 arXiv
[10]

Pietro Morerio, Jacopo Cavazza, Riccardo Volpi, Ren\'e Vidal, and Vittorio Murino. 2017. Curriculum dropout. In ICCV

2017
[11]

Adina Williams, Nikita Nangia, and Samuel R. Bowman. 2018. Multinli: A corpus of multi-genre natural language inference. https://cims.nyu.edu/ sbowman/multinli/. Website

2018
[12]

Wenpeng Yin, Karl Kann, Mo Yu, and Hinrich Schütze. 2019. Comparative study of cnn and rnn for natural language processing. arXiv preprint arXiv:1702.01923

2019 arXiv
[13]

Chuang Zhai, Yeyun Gong, Heng Zhang, Yuqing Kong, Weikang Wang, Guozhong Tang, Longbiao Wang, and Hanghang Tong. 2021. http://arxiv.org/abs/2104.08821 Contrastive learning for unsupervised sentence embedding enhancement

2021 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Jimmy Ba and Brendan Frey. 2013. https://proceedings.neurips.cc/paper_files/paper/2013/file/7b5b23f4aadf9513306bcd59afb6e4c9-Paper.pdf Adaptive dropout . In Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc

work page 2013

[4] [4]

Bowman, Gabor Angeli, Christopher Potts, and Christopher D

Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. Stanford natural language inference (snli) corpus. https://nlp.stanford.edu/projects/snli/. Website

work page 2015

[5] [5]

Dynamic Attention Networks for Task Oriented Grounding

Soumik Dasgupta, Badri N. Patro, and Vinay P. Namboodiri. 2019. http://arxiv.org/abs/1910.06315 Dynamic attention networks for task oriented grounding . arXiv preprint arXiv:1910.06315

work page Pith review arXiv 2019

[6] [6]

Tianyu Gao, Xuancheng Wang, Changyou Chen, and Jian Tang. 2019. Anisotropic vector spaces learning for natural language processing. arXiv preprint arXiv:1907.12009

arXiv 2019

[7] [7]

Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.552 Simcse: Simple contrastive learning of sentence embeddings . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894--6910, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics

doi:10.18653/v1/2021.emnlp-main.552 2021

[8] [8]

Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9729--9738

work page 2020