{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2021:JDVYZVMEZKIKYMTR5Y7GAZ2ICM","short_pith_number":"pith:JDVYZVME","schema_version":"1.0","canonical_sha256":"48eb8cd584ca90ac3271ee3e606748130a9b5d5b1e43938a82e86154b8b68519","source":{"kind":"arxiv","id":"2104.08821","version":4},"attestation_state":"computed","paper":{"title":"SimCSE: Simple Contrastive Learning of Sentence Embeddings","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"Contrastive learning with standard dropout as the only noise produces sentence embeddings that match or beat prior supervised results.","cross_cats":["cs.LG"],"primary_cat":"cs.CL","authors_text":"Danqi Chen, Tianyu Gao, Xingcheng Yao","submitted_at":"2021-04-18T11:27:08Z","abstract_excerpt":"This paper presents SimCSE, a simple contrastive learning framework that greatly advances state-of-the-art sentence embeddings. We first describe an unsupervised approach, which takes an input sentence and predicts itself in a contrastive objective, with only standard dropout used as noise. This simple method works surprisingly well, performing on par with previous supervised counterparts. We find that dropout acts as minimal data augmentation, and removing it leads to a representation collapse. Then, we propose a supervised approach, which incorporates annotated pairs from natural language in"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":true,"formal_links_present":true},"canonical_record":{"source":{"id":"2104.08821","kind":"arxiv","version":4},"metadata":{"license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","primary_cat":"cs.CL","submitted_at":"2021-04-18T11:27:08Z","cross_cats_sorted":["cs.LG"],"title_canon_sha256":"521eac38e6d56e5902f623741973ee63232a52df03edcbb497fbb9d6cc2c02e0","abstract_canon_sha256":"0ba08acab8d5daef298eda40b969a5252fbad3b10cd5421f91a7fa217ca5a3b2"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-05-17T23:38:53.096185Z","signature_b64":"07T3FCm7zNBHOYdisHdxPUYmiobpHJ6WhkRZebKrpdF2prXV0yoyCE9ZyY/hHRNzUXG1cLihiGj7XTIFRE0iDQ==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"48eb8cd584ca90ac3271ee3e606748130a9b5d5b1e43938a82e86154b8b68519","last_reissued_at":"2026-05-17T23:38:53.095688Z","signature_status":"signed_v1","first_computed_at":"2026-05-17T23:38:53.095688Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"SimCSE: Simple Contrastive Learning of Sentence Embeddings","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"Contrastive learning with standard dropout as the only noise produces sentence embeddings that match or beat prior supervised results.","cross_cats":["cs.LG"],"primary_cat":"cs.CL","authors_text":"Danqi Chen, Tianyu Gao, Xingcheng Yao","submitted_at":"2021-04-18T11:27:08Z","abstract_excerpt":"This paper presents SimCSE, a simple contrastive learning framework that greatly advances state-of-the-art sentence embeddings. We first describe an unsupervised approach, which takes an input sentence and predicts itself in a contrastive objective, with only standard dropout used as noise. This simple method works surprisingly well, performing on par with previous supervised counterparts. We find that dropout acts as minimal data augmentation, and removing it leads to a representation collapse. Then, we propose a supervised approach, which incorporates annotated pairs from natural language in"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"our unsupervised and supervised models using BERT base achieve an average of 76.3% and 81.6% Spearman's correlation respectively, a 4.2% and 2.2% improvement compared to the previous best results. We also show -- both theoretically and empirically -- that the contrastive learning objective regularizes pre-trained embeddings' anisotropic space to be more uniform","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"that standard dropout is sufficient as data augmentation to prevent representation collapse in the unsupervised contrastive objective, and that NLI entailment/contradiction pairs form appropriate positive and hard-negative pairs for learning general sentence embeddings.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"SimCSE achieves 76.3% unsupervised and 81.6% supervised Spearman's correlation on STS tasks with BERT-base, improving prior best results by 4.2% and 2.2% via simple contrastive learning.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Contrastive learning with standard dropout as the only noise produces sentence embeddings that match or beat prior supervised results.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"d23982583bab21067a7b361fe4fdb8ba245a7c8e32535fc7425b7316d1e1f02c"},"source":{"id":"2104.08821","kind":"arxiv","version":4},"verdict":{"id":"9386c89c-7b0c-4e0d-9757-f550bca43508","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-15T07:44:10.339562Z","strongest_claim":"our unsupervised and supervised models using BERT base achieve an average of 76.3% and 81.6% Spearman's correlation respectively, a 4.2% and 2.2% improvement compared to the previous best results. We also show -- both theoretically and empirically -- that the contrastive learning objective regularizes pre-trained embeddings' anisotropic space to be more uniform","one_line_summary":"SimCSE achieves 76.3% unsupervised and 81.6% supervised Spearman's correlation on STS tasks with BERT-base, improving prior best results by 4.2% and 2.2% via simple contrastive learning.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"that standard dropout is sufficient as data augmentation to prevent representation collapse in the unsupervised contrastive objective, and that NLI entailment/contradiction pairs form appropriate positive and hard-negative pairs for learning general sentence embeddings.","pith_extraction_headline":"Contrastive learning with standard dropout as the only noise produces sentence embeddings that match or beat prior supervised results."},"references":{"count":109,"sample":[{"doi":"","year":2012,"title":"Eneko Agirre, Daniel Cer, Mona Diab, and Aitor Gonzalez-Agirre. 2012. https://www.aclweb.org/anthology/S12-1051 S em E val-2012 task 6: A pilot on semantic textual similarity . In * SEM 2012: The Firs","work_id":"5c5985e9-1814-48b7-bcb4-7729a1c061f6","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2013,"title":"Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, and Weiwei Guo. 2013. https://www.aclweb.org/anthology/S13-1004 * SEM 2013 shared task: Semantic textual similarity . In Second Joint Confer","work_id":"bbbddcb5-789b-4d66-8a0e-ca0c3d63d8c3","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2017,"title":"Sanjeev Arora, Yingyu Liang, and Tengyu Ma. 2017. https://openreview.net/forum?id=SyK00v5xx A simple but tough-to-beat baseline for sentence embeddings . In International Conference on Learning Repres","work_id":"134c4c07-53b7-4f57-a9fc-8cec09d1cfc2","ref_index":6,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2021,"title":"Fredrik Carlsson, Amaru Cuba Gyllensten, Evangelia Gogoulou, Erik Ylip \\\"a \\\"a Hellqvist, and Magnus Sahlgren. 2021. https://openreview.net/forum?id=Ov_sMNau-PF Semantic re-tuning with contrastive ten","work_id":"24bac75d-ff61-4dd9-9ec9-9902d815cc3f","ref_index":8,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2020,"title":"Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. http://proceedings.mlr.press/v119/chen20j.html A simple framework for contrastive learning of visual representations . In Inter","work_id":"643fd409-2cee-4151-8cfb-7a1f881694a1","ref_index":11,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":109,"snapshot_sha256":"da93f7ddf56dd0c77df7511b7cd2b2e3cdc8033f95f12136db020b7086eb270e","internal_anchors":4},"formal_canon":{"evidence_count":2,"snapshot_sha256":"948452d7b3a30d0f4065d16892d7691db9b52fabea99a1d4f53ea8f85569c3da"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2104.08821","created_at":"2026-05-17T23:38:53.095781+00:00"},{"alias_kind":"arxiv_version","alias_value":"2104.08821v4","created_at":"2026-05-17T23:38:53.095781+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2104.08821","created_at":"2026-05-17T23:38:53.095781+00:00"},{"alias_kind":"pith_short_12","alias_value":"JDVYZVMEZKIK","created_at":"2026-05-18T12:33:33.725879+00:00"},{"alias_kind":"pith_short_16","alias_value":"JDVYZVMEZKIKYMTR","created_at":"2026-05-18T12:33:33.725879+00:00"},{"alias_kind":"pith_short_8","alias_value":"JDVYZVME","created_at":"2026-05-18T12:33:33.725879+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":32,"internal_anchor_count":32,"sample":[{"citing_arxiv_id":"2605.23780","citing_title":"Beyond Binary Edits Robust Multimodal Knowledge Editing with Adversarial Subspace Alignment","ref_index":14,"is_internal_anchor":true},{"citing_arxiv_id":"2402.19088","citing_title":"Survey in Characterizing Semantic Change","ref_index":29,"is_internal_anchor":true},{"citing_arxiv_id":"2410.22240","citing_title":"Are Decoder-Only Large Language Models the Silver Bullet for Code Search?","ref_index":72,"is_internal_anchor":true},{"citing_arxiv_id":"2410.16431","citing_title":"Conjuring Semantic Similarity","ref_index":11,"is_internal_anchor":true},{"citing_arxiv_id":"2502.12911","citing_title":"Knapsack Optimization-based Schema Linking for LLM-based Text-to-SQL Generation","ref_index":18,"is_internal_anchor":true},{"citing_arxiv_id":"2605.18007","citing_title":"Semantic Reranking at Inference Time for Hard Examples in Rhetorical Role Labeling","ref_index":25,"is_internal_anchor":true},{"citing_arxiv_id":"2605.16767","citing_title":"Retrieval-Based Multi-Label Legal Annotation: Extensible, Data-Efficient and Hallucination-Free","ref_index":27,"is_internal_anchor":true},{"citing_arxiv_id":"2511.07969","citing_title":"Unified Work Embeddings: Contrastive Learning of a Bidirectional Multi-task Ranker","ref_index":17,"is_internal_anchor":true},{"citing_arxiv_id":"2407.12580","citing_title":"E5-V: Universal Embeddings with Multimodal Large Language Models","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2512.13511","citing_title":"Adapting MLLMs for Nuanced Video Retrieval","ref_index":27,"is_internal_anchor":true},{"citing_arxiv_id":"2208.03299","citing_title":"Atlas: Few-shot Learning with Retrieval Augmented Language Models","ref_index":40,"is_internal_anchor":true},{"citing_arxiv_id":"2602.00616","citing_title":"SPOT: Selective Prompt Projection via Total Variation for Inference-Only Safe Text-to-Image Generation","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2509.20354","citing_title":"EmbeddingGemma: Powerful and Lightweight Text Representations","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2405.17428","citing_title":"NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models","ref_index":119,"is_internal_anchor":true},{"citing_arxiv_id":"2604.26955","citing_title":"Policy-Governed LLM Routing with Intent Matching for Instrument Laboratories","ref_index":12,"is_internal_anchor":true},{"citing_arxiv_id":"2311.05232","citing_title":"A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions","ref_index":99,"is_internal_anchor":true},{"citing_arxiv_id":"2112.09118","citing_title":"Unsupervised Dense Information Retrieval with Contrastive Learning","ref_index":131,"is_internal_anchor":true},{"citing_arxiv_id":"2605.08809","citing_title":"SimReg: Achieving Higher Performance in the Pretraining via Embedding Similarity Regularization","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2604.25853","citing_title":"G-Loss: Graph-Guided Fine-Tuning of Language Models","ref_index":12,"is_internal_anchor":true},{"citing_arxiv_id":"2402.03216","citing_title":"M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation","ref_index":56,"is_internal_anchor":true},{"citing_arxiv_id":"2604.24374","citing_title":"MIPIC: Matryoshka Representation Learning via Self-Distilled Intra-Relational and Progressive Information Chaining","ref_index":9,"is_internal_anchor":true},{"citing_arxiv_id":"2605.04962","citing_title":"TabEmbed: Benchmarking and Learning Generalist Embeddings for Tabular Understanding","ref_index":8,"is_internal_anchor":true},{"citing_arxiv_id":"2605.04295","citing_title":"LLMs Uncertainty Quantification via Adaptive Conformal Semantic Entropy","ref_index":6,"is_internal_anchor":true},{"citing_arxiv_id":"2604.20417","citing_title":"Semantic Recall for Vector Search","ref_index":11,"is_internal_anchor":true},{"citing_arxiv_id":"2604.10590","citing_title":"Bridging Linguistic Gaps: Cross-Lingual Mapping in Pre-Training and Dataset for Enhanced Multilingual LLM Performance","ref_index":13,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":2,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/JDVYZVMEZKIKYMTR5Y7GAZ2ICM","json":"https://pith.science/pith/JDVYZVMEZKIKYMTR5Y7GAZ2ICM.json","graph_json":"https://pith.science/api/pith-number/JDVYZVMEZKIKYMTR5Y7GAZ2ICM/graph.json","events_json":"https://pith.science/api/pith-number/JDVYZVMEZKIKYMTR5Y7GAZ2ICM/events.json","paper":"https://pith.science/paper/JDVYZVME"},"agent_actions":{"view_html":"https://pith.science/pith/JDVYZVMEZKIKYMTR5Y7GAZ2ICM","download_json":"https://pith.science/pith/JDVYZVMEZKIKYMTR5Y7GAZ2ICM.json","view_paper":"https://pith.science/paper/JDVYZVME","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2104.08821&json=true","fetch_graph":"https://pith.science/api/pith-number/JDVYZVMEZKIKYMTR5Y7GAZ2ICM/graph.json","fetch_events":"https://pith.science/api/pith-number/JDVYZVMEZKIKYMTR5Y7GAZ2ICM/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/JDVYZVMEZKIKYMTR5Y7GAZ2ICM/action/timestamp_anchor","attest_storage":"https://pith.science/pith/JDVYZVMEZKIKYMTR5Y7GAZ2ICM/action/storage_attestation","attest_author":"https://pith.science/pith/JDVYZVMEZKIKYMTR5Y7GAZ2ICM/action/author_attestation","sign_citation":"https://pith.science/pith/JDVYZVMEZKIKYMTR5Y7GAZ2ICM/action/citation_signature","submit_replication":"https://pith.science/pith/JDVYZVMEZKIKYMTR5Y7GAZ2ICM/action/replication_record"}},"created_at":"2026-05-17T23:38:53.095781+00:00","updated_at":"2026-05-17T23:38:53.095781+00:00"}