Pith. sign in

REVIEW 9 cited by

GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.16829 v1 pith:DYCC66IH submitted 2024-02-26 cs.LG cs.CL

Aivin V. Solatorio This is my paper
classification cs.LGcs.CL
keywords datamodeltrainingembeddinggistembedmodelsacrossapplications
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Embedding models are integral to AI applications like semantic search, personalized recommendations, and retrieval augmented generation for LLMs, necessitating high-quality training data. However, the limited scalability of manual data curation prompts the need for automated methods to ensure data integrity. Traditional unsupervised triplet mining automates training data generation, crucial for embedding model training, yet inadvertently injects biases and noise, thereby degrading model performance. Addressing this, we introduce GISTEmbed, a novel strategy that enhances in-batch negative selection during contrastive training through a guide model. This approach departs from reliance on random sampling and equal utility assumption of batch negatives, significantly reducing noise from data quality issues and improving model fine-tuning. Benchmarked against the Massive Text Embedding Benchmark (MTEB), GISTEmbed showcases consistent performance improvements across various model sizes and achieves state-of-the-art results in select categories. This framework enables significant enhancements for smaller models by leveraging the capabilities of powerful yet resource-intensive large models. GISTEmbed can potentially revolutionize the creation of highly efficient, smaller models, democratizing access to advanced AI technologies. Making these technologies more accessible and cost-effective, especially for applications constrained by resources, significantly expands the impact and accessibility of state-of-the-art AI solutions across diverse sectors.

Discussion (0). Sign in to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Omnilingual SONAR: Cross-Lingual and Cross-Modal Sentence Embeddings Bridging Massively Multilingual Text and Speech

    cs.CL 2026-03 accept novelty 7.0 of 10

    Progressive LLM-initialized training plus teacher-student distillation yields omnilingual sentence embeddings that halve FLORES error and cut BIBLE error 15× while supporting speech and zero-shot Spectrum transfer.

  2. STEP: Career-Path Recommendation via Temporal and Educational Trajectory Modeling

    cs.CL 2026-07 conditional novelty 6.0 of 10

    STEP, with ROUTE embeddings and JobHop v2, sets new next-job prediction SOTA on four ESCO career-trajectory benchmarks by modeling inter-job time and education.

  3. LLM generation novelty through the lens of semantic similarity

    cs.LG 2025-10 conditional novelty 6.0 of 10

    An output is called novel when no semantically similar pretraining context can be retrieved; using this embedding-based measure, SmolLM models show longer-range reuse and instruction tuning increases novelty.

  4. Exploiting Leaderboards for Large-Scale Distribution of Malicious Models

    cs.LG 2025-07 conditional novelty 6.0 of 10

    A new attack framework, TrojanClimb, shows that adversaries can place models with embedded backdoors or biases on public leaderboards while retaining competitive rankings, across text embeddings, text generation, spee...

  5. ExploreGS: Explorable 3D Scene Reconstruction with Virtual Camera Samplings and Diffusion Priors

    cs.CV 2025-08 unverdicted novelty 5.0 of 10

    Adding information-gain-selected virtual views refined by video diffusion priors to 3D Gaussian Splatting improves arbitrary-view rendering quality.

  6. Overview of the TalentCLEF 2025: Skill and Job Title Intelligence for Human Capital Management

    cs.CL 2025-07 conditional novelty 5.0 of 10

    TalentCLEF 2025 publishes the first public multilingual benchmark for job title matching and skill prediction, with results showing training strategy matters more than model size.

  7. Towards Applying Large Language Models to Complement Single-Cell Foundation Models

    cs.LG 2025-07 conditional novelty 5.0 of 10

    A fusion model called scMPT, combining scGPT with an LLM text encoder, improves single-cell cell type classification on most tested datasets, and the paper shows the LLM relies on marker genes and simple expression patterns.

  8. SACL: Understanding and Combating Textual Bias in Code Retrieval with Semantic-Augmented Reranking and Localization

    cs.CL 2025-06 reject novelty 5.0 of 10

    Code retrievers rely on surface text like identifiers and docstrings rather than code semantics, and a description-based reranking pipeline, SACL, compensates for that bias but its headline gains come mostly from arti...

  9. Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaning

    cs.CL 2025-06 conditional novelty 5.0 of 10

    State-of-the-art text embeddings lag far behind on tasks requiring pragmatic inference, stance detection, and social meaning, relative to their strong performance on surface semantic benchmarks.

Pith tools