Mapping Similarity Spaces across Embedding Models with Synthetic Query Probing

Marcin Rozmus; Peter van der Putten

REVIEW 3 major objections 4 minor 17 references

Mapping Similarity Spaces across Embedding Models with Synthetic Query Probing

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Cosine similarity scores from different embedding models cannot be directly compared, but learned monotone mappings fitted on synthetic query–chunk pairs can partially transfer retrieval thresholds across models, with isotonic regression…

desk verdict Useful empirical study of cross-model similarity score distributions, but its headline claim about threshold portability is not actually tested: the mappings are fit and evaluated on the same full dataset, with no held-out transfer experiment. read the letter →

arxiv 2608.05857 v1 pith:XRGEFSJO submitted 2026-08-06 cs.CL

Marcin Rozmus , Peter van der Putten This is my paper

classification cs.CL

keywords embeddingmodelscosinesimilaritythresholdcalibrationisotonicregressionsyntheticqueryprobingretrieval-augmentedgenerationscoredistributionmappingquantile

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that cosine similarity scores from different embedding models are not directly comparable, and that naive threshold reuse across models fails. It introduces Synthetic Query Probing (SQP), a reference-free method that generates controlled query–chunk pairs labeled by semantic relatedness to analyze cross-model score distributions. The authors show that learned score conversion functions—especially isotonic regression—partially align these distributions and improve threshold portability. If correct, this provides a practical calibration step for migrating retrieval-augmented generation systems between embedding models and corpora.

What carries the argument

Synthetic Query Probing (SQP) is the central mechanism: from a chunk of a corpus, an LLM generates queries labeled as PARAPHRASE, RELEVANT, or IRRELEVANT, producing paired similarity scores across embedding models. The cross-model conversion functions are fitted on these paired scores using linear regression, isotonic regression, and quantile (CDF) mapping, and threshold analysis sweeps a binary classifier over cosine similarity to find operating points at target precision levels.

What would settle it

Fit an SQP conversion on a corpus that also has human relevance judgments or logged real user queries, apply the converted threshold, and measure precision at the target operating point on the real queries; if precision falls systematically below the synthetic performance, the reference-free assumption is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that while embedding models largely agree on the relative ranking of query–chunk pairs, their absolute similarity scores exhibit systematic, model-dependent and corpus-dependent distortions. These distortions can be captured by monotone mapping functions learned from synthetic query probing data: linear regression works for near-linear within-family pairs, while isotonic regression best models the S-shaped non-linearity between different model families. The resulting mappings partially align similarity spaces and make precision-targeted thresholds more portable, though conversion accuracy degrades on heterogeneous corpora and is directionally asymmetric.

Load-bearing premise

The entire threshold and mapping analysis treats the LLM-generated synthetic queries and their automatic labels (PARAPHRASE, RELEVANT, IRRELEVANT) as a faithful substitute for real user queries and true relevance on the target corpus.

Editorial extensions

If this is right

Within a model family, dimensionality changes are nearly lossless for score conversion, with $R^2 \geq 0.97$ on both corpora, so lightweight linear mappings suffice for such migrations.
Cross-model threshold reuse without calibration can lead to large precision drops; the paper's mappings reduce this gap, with isotonic regression achieving the highest $R^2$ and lowest MAE across all twelve directional pairs.
Thresholds are primarily corpus-dependent: changing the corpus induces larger threshold shifts than changing the embedding model within a family, implying per-corpus calibration remains necessary even when the model is fixed.
Conversion reliability is predictable from class separability: when relevant and irrelevant queries are cleanly separated, SQP-derived mappings are tighter, suggesting SQP statistics can serve as a prior indicator of transfer quality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

SQP could serve as a continuous drift-monitoring tool: periodically regenerated synthetic probes on a live corpus could flag when an embedding model's score distribution has shifted enough to require recalibration, an application the paper lists as future work.
The S-shaped Titan–Ada relationship likely reflects differences in how the models normalize embedding magnitudes; testing other model families beyond these two vendors would reveal whether such non-linearity is generic or idiosyncratic.
The label-free nature of SQP means it could be extended beyond cosine similarity to other distance metrics, and the method's dependence on LLM-generated labels suggests a testable extension: compare SQP-fitted thresholds against thresholds derived from real user query logs on the same corpus to quantify synthetic-to-real transfer.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing to know: this is a genuinely useful empirical study of how cosine similarity scores differ across embedding models, and it introduces a practical methodology for studying that. But the headline claim—that learned mappings "improve threshold portability"—is not directly supported by the experiments as run. The paper itself concedes this in Section 5, which is honest but means the abstract overstates what was shown.

What's new and good: Synthetic Query Probing is a simple, reference-free way to generate controlled query-chunk pairs with paraphrase/relevant/irrelevant labels, and the paper uses it to characterize score distributions across four embedding configurations on two corpora. The distributional findings are solid and clearly presented: Titan dimensionalities produce a consistent upward shift as dimensions decrease, Ada compresses scores into a narrow high band, ranking structure is preserved across models, and class separability drives conversion quality. The cross-model S-shaped nonlinearity and the directional asymmetry in error (Ada to Titan has higher MAE than the reverse) are real observations, and the comparison of linear, isotonic, and quantile mappings is reasonable. Within the Titan family, the near-lossless conversions are a useful practical result.

The soft spots are concentrated in the evaluation protocol. The conversion functions are fitted and evaluated on the full dataset, so the reported R^2 and MAE are in-sample goodness of fit, not predictive accuracy. There is no experiment that takes a threshold calibrated on one model, converts it to another model's scale via the learned mapping, and measures precision/recall on held-out data. That is the load-bearing experiment for the portability claim, and it is missing. The paper's own Section 5 admits this: "conversion functions are fitted on the full dataset and should be extended with proper train-test protocols." This is an explicit limitation that directly weakens the abstract's claim. I therefore agree with the stress-test note that the claim should be toned down to "mappings can be fit with high accuracy on SQP data" until the transfer experiment is done.

Other concerns are real but secondary: synthetic queries are a proxy for real user traffic (the paper cites prior work suggesting they are predictive, but it is still a proxy), the sample is small (100 chunks per corpus, 3,000 pairs total), there are no error bars, and no code or data are released. None of these are fatal; they are addressable with additional experiments.

Who is this for? RAG practitioners and anyone doing model migration who wants a lightweight way to reason about score scales and thresholds. It will not settle the field, but it provides a useful framework and initial evidence. It deserves serious refereeing, because the problem is real and the methodology is new, but it needs major revision: add a held-out threshold-transfer experiment, report variability, and ideally release the SQP data or code. I would not cite the portability claim as established, but I would cite the distributional characterization and the SQP methodology.

Referee Report

3 major / 4 minor

Summary. The paper introduces Synthetic Query Probing (SQP), a reference-free method that uses an LLM to generate PARAPHRASE, RELEVANT, and IRRELEVANT queries for sampled chunks, yielding 3,000 labeled query–chunk pairs per corpus. Using these pairs, the authors compute cosine scores for four embedding configurations (Amazon Titan Text Embeddings V2 at 256/512/1024 dimensions and OpenAI text-embedding-ada-002) on SciFact and a proprietary Pegasystems corpus. They analyze per-class score distributions, derive precision-based thresholds for each configuration, and fit linear, isotonic, and quantile mappings between all 12 directional model pairs, reporting R^2 and MAE. The central claim is that learned mappings partially align similarity spaces and improve threshold portability, with isotonic regression performing best.

Significance. The problem of cross-model score comparability is real and practically important for RAG migrations, and SQP is a simple, scalable, annotation-free way to collect paired scores across models. The distributional results (e.g., Ada's compressed score range, the Titan dimensionality gradient, and preserved class ordering) are informative and could be useful to practitioners. However, the headline claim of improved threshold portability is not directly tested: all conversion functions are fitted and evaluated on the same full dataset, and no threshold-transfer experiment is performed. The paper's contribution is therefore a promising methodology and a set of descriptive findings, not yet a validated solution for threshold reuse.

major comments (3)

[§4.5, Table 3, §5] The conversion functions in Section 4.5 are fitted on the full 3,000 query–chunk pairs per corpus and evaluated on the same data; Table 3 therefore reports in-sample R^2 and MAE, not predictive accuracy. The abstract's claim that mappings 'improve threshold portability' is never tested directly: no experiment converts a threshold calibrated on one model through a learned mapping and measures precision/recall on held-out data. Section 5 explicitly concedes this ('conversion functions are fitted on the full dataset and should be extended with proper train-test protocols for reusable calibration'). The authors should add a held-out or cross-validated threshold-transfer experiment, or weaken the claim to 'mappings can be fit accurately on SQP data.'
[§4.3, Table 2] The precision operating thresholds in Table 2 are selected by sweeping Equation (1) over the full SQP set, and the reported precision/recall/F1 are computed on the same data. This makes the thresholds in-sample by construction. If the goal is to demonstrate portability, the threshold calibration and evaluation should be separated (e.g., fit thresholds on a training split, evaluate on a test split, and report variability). As it stands, the argument that 'naively reusing a 1024-d threshold at lower dimensionalities would lead to a noticeable drop in precision' is based on thresholds and precisions both measured on the same set.
[§4.1, §5] The ground-truth relevance labels are generated automatically by Claude Sonnet 4.6 with no human verification, and all subsequent analyses (distributions, thresholds, mappings) treat these labels as ground truth. The paper cites [2] for the predictive validity of synthetic benchmarks, but a small human-checked subset or a comparison against a standard retrieval benchmark would reduce the risk that the observed score relationships are specific to the query generator rather than to real RAG traffic. I raise this as a correctness-risk concern, not as a claim that the method is circular.

minor comments (4)

[Table 3] The quantile mapping columns report only MAE, not R^2, although the text claims isotonic regression achieves the highest R^2 across all methods. Please include R^2 for quantile mapping or explain the omission.
[Equation (1)] The set-builder notation for the threshold sweep is garbled; please correct the LaTeX so that T_mi is clearly defined.
[§4.5] The statement that R^2 is 'identical in both directions' for the 1024 ↔ Ada pair holds for linear regression (0.841) but not for isotonic regression (0.945 vs. 0.889 in Table 3a). Please clarify that the symmetry claim applies only to the linear model.
[Overall] A data/code availability statement is missing; providing the SQP dataset or a reproducible pipeline would strengthen the paper.

Circularity Check

1 steps flagged · score 6.0 of 10

Threshold-portability claim rests on in-sample mapping fit, not on any transferred-threshold experiment.

fitted input called prediction [Section 4.5 (Table 3) and Section 5 (Limitations)]
"For every directional pair and method we fit the corresponding parameters: the slope and intercept (a, b) for linear OLS, the monotone step function (knot set) for isotonic regression, and the empirical percentile lookup for quantile mapping. To keep this paper readable we omit the full per-pair parameter sets and report only aggregate accuracy (MAE and R2) here. ... Finally, conversion functions are fitted on the full dataset and should be extended with proper traintest protocols for reusable calibration."

The abstract claims 'Learned mappings partially align these spaces and improve threshold portability, with isotonic regression performing best,' but the evidence is MAE and R2 computed on the exact same full dataset used to fit linear, isotonic, and quantile mappings. No threshold is calibrated on one model, converted through a learned mapping, and evaluated for retrieval precision or recall on another model or on held-out data. The paper's own Section 5 concedes that the conversion functions are fitted on the full dataset. Thus the 'portability' result is the in-sample fit quality itself, renamed as a prediction, and the 'isotonic performs best' comparison is an in-sample model-selection outcome forced by the training data.

full rationale

The distributional analysis (per-class statistics, ranking preservation, threshold differences) is an independent empirical contribution and does not reduce to a fitted parameter. The circularity is confined to the headline portability claim: Section 4.5 fits and evaluates the conversion functions on the same 3,000 query-chunk pairs, and the abstract's 'improve threshold portability' is asserted from those in-sample R2/MAE numbers rather than from any actual threshold-transfer experiment. The Section 5 limitation statement explicitly acknowledges this gap, which corroborates the reduction. The synthetic-query validity caveat is supported only by reference [2], which shares an author with this paper; I do not count that as circular because [2] is a prior empirical study and the present observations stand independently of it by construction, though it does mean the proxy's validity rests heavily on that citation. Overall, the central distributional finding is self-contained, but the central portability claim is partially circular, warranting a score of 6 rather than a lower score.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central empirical analysis rests on cosine similarity as the relevance metric, on synthetic query labels treated as ground truth, and on a single 100-chunk sample per corpus. Conversion parameters are fitted to the full dataset, so their reported accuracy is in-sample. No new physical entities are introduced.

free parameters (5)

Linear conversion slope and intercept (a, b) = Not reported (omitted for readability)
Fitted by ordinary least squares for each of the 12 directional pairs on the full dataset; used to compute the MAE and R2 reported in Table 3.
Isotonic regression step function knots = Not reported (omitted for readability)
Monotone step functions fitted per directional pair on the full dataset; reported as best method in Table 3.
Quantile mapping empirical percentiles = Not reported (omitted for readability)
Empirical percentile lookup fitted per directional pair on the full dataset; MAE reported in Table 3.
Precision operating thresholds = e.g. 0.063 to 0.821 depending on model and corpus
Selected per model and corpus to achieve precision 0.93, 0.95, or 0.97; used for the threshold comparison in Table 2.
SQP sample sizes and query counts = 100 chunks, 10 queries per class per chunk
Design choices in Section 4.1 that determine distribution stability and cost; no sensitivity analysis is provided.

assumptions (4)

domain assumption Cosine similarity is the relevant scoring function for retrieval.
Adopted in Section 3 as the dominant production metric, while acknowledging limitations; all comparisons and mappings are defined on cosine scores.
ad hoc to paper Synthetic queries labeled as PARAPHRASE, RELEVANT, and IRRELEVANT are valid ground truth for relevance.
Section 4.1 uses Claude Sonnet 4.6 output as ground truth without human validation; the paper notes in Section 5 that synthetic queries may introduce bias.
domain assumption A uniform random sample of 100 chunks is representative of each corpus for score distribution analysis.
Section 4.1 samples 100 chunks from corpora of 5,183 and 114,648 chunks; distribution stability is asserted rather than measured.
domain assumption Embeddings are normalized, so cosine similarity behaves as assumed.
Stated in the Section 5 limitations list; it is a precondition for comparing cosine scores across configurations.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Mapping Similarity Spaces across Embedding Models with Synthetic Query Probing." pith.science (2026). https://pith.science/paper/XRGEFSJO

@misc{pith2026260805857,
  author       = {Pith},
  title        = {Pith review of: Mapping Similarity Spaces across Embedding Models with Synthetic Query Probing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XRGEFSJO}},
  note         = {Machine review of arXiv:2608.05857}
}

read the original abstract

Retrieval-Augmented Generation systems rely on similarity scores to retrieve relevant content, yet scores are not directly comparable across embedding models due to differing geometric properties, complicating model migration and limiting threshold reuse. We study how similarity scores can be related by learning mappings between score distributions rather than embeddings. We introduce Synthetic Query Probing, generating queries from documents to create controlled query-chunk pairs, enabling large-scale, reference-free analysis of cross-model similarity behavior. We evaluate the approach on multiple embedding configurations and learn score conversion functions using linear, isotonic, and quantile mappings. Experiments on SciFact and a proprietary corpus show that while models largely agree on rankings, their absolute scores exhibit systematic distortions. Learned mappings partially align these spaces and improve threshold portability, with isotonic regression performing best. Our results highlight the need for cross-model calibration and position Synthetic Query Probing as a scalable framework for analyzing embedding comparability.

Figures

Figures reproduced from arXiv: 2608.05857 by the authors.

**Figure 1.** Cosine similarity distributions by semantic class for all four embedding configurations SciFact corpus. Top row: KDE density plots showing each configuration independently. Titan variants show near-perfect class separation with IRRELEVANT scores clustered near zero (mean 0.018 at 1024-d, rising to 0.061 at 256-d), while Ada compresses all classes into a 0.626-0.930 band yet keeps IRRELEVANT fully disjoint from the … view at source ↗

**Figure 2.** Cosine similarity distributions by semantic class for all four embedding configurations - Enterprise corpus. Top row: KDE density plots showing each configuration independently. Titan variants show clear class separation with IRRELEVANT scores clustered around 0.2, while Ada compresses all classes into a narrow 0.603-0.963 band. Bottom row: Box-plots confirming the upward shift in Titan scores at lower dimensions a… view at source ↗

**Figure 3.** Precision, Recall, and F1 versus threshold for all four embedding configurations on the SciFact and Enterprise corpora. The top row shows SciFact and the bottom row shows Enterprise. (MAE and R2 ) here [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: SciFact corpus: cross-configuration score relationships for all six pairs. Top row: Scatter plots colored by semantic class—PARAPHRASE (orange) in the upperright, IRRELEVANT (red) in the lower-left. The Titan cross-dimension pairs show near-linear relationships, refle…

**Figure 5.** Figure 5: Enterprise corpus: cross-configuration score relationships for all six pairs. Top row: Scatter plots colored by semantic class—PARAPHRASE (orange) in the upperright, IRRELEVANT (red) in the lower-left. The Titan cross-dimension pairs (256–512, 256–1024, 512–1024) show…

**Figure 6.** Figure 6: Cross-configuration conversion fits on the SciFact corpus for two representative dimension pairs, each shown with the linear OLS, isotonic regression, and quantile mapping conversion functions overlaid on similarity-score pairs. neous corpora (SciFact) than on heteroge…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

17 extracted references · 8 canonical work pages

[2]

(eds.) Machine Learning and Principles and Practice of Knowledge Discovery in Databases

van Elburg, J., van der Putten, P., Marx, M.: Can we evaluate RAGs with synthetic data? In: Koprinska, I., Mendes-Moreira, J., Branco, P. (eds.) Machine Learning and Principles and Practice of Knowledge Discovery in Databases. SynDAiTE workshop at ECML PKDD 2025. pp. 177–192. Springer Nature Switzerland, Cham (2026)

work page 2026
[1]

In: Petroni, F., Siciliano, F., Silvestri, F., Trappolini, G

Caspari, L., Dastidar, K.G., Zerhoudi, S., Mitrovic, J., Granitzer, M.: Beyond benchmarks: Evaluating embedding model similarity for retrieval augmented gen- eration systems. In: Petroni, F., Siciliano, F., Silvestri, F., Trappolini, G. (eds.) Proceedings of the Workshop Information Retrieval’s Role in RAG Systems (IR- RAG 2024) co-located with the 47th I...

work page 2024
[3]

In: Yue, Y., Garg, A., Peng, N., Sha, F., Yu, R

Enevoldsen, K., Chung, I., Kerboua, I., Kardos, M., Mathur, A., Stap, D., Gala, J., Siblini, W., Krzemiński, D., Winata, G., Sturua, S., Utpala, S., Ciancone, M., Schaeﬀer, M., Misra, D., Dhakal, S., Rystrø m, J., Solomatin, R., Çağatan, O., Kundu, A., Bernstorﬀ, M., Xiao, S., Sukhlecha, A., Pahwa, B., Poświata, R., GV, K.K., Ashraf, S., Auras, D., Plüste...

work page 2025
[4]

In: Demberg, V., Inui, K., Mar- quez, L

Frank, M., Aﬂi, H.: PTEB: Towards robust text embedding evaluation via stochas- tic paraphrasing at evaluation time with LLMs. In: Demberg, V., Inui, K., Mar- quez, L. (eds.) Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 2832–

work page
[5]

Frank, M., Aﬂi, H.: The Harder Text Embedding Benchmark (HTEB): Beyond one-dimensional static robustness (2026), https://arxiv.org/abs/2605.28190

work page Pith review arXiv 2026
[6]

Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., Wang, H.: Retrieval-augmented generation for large language models: A survey (2024), https://arxiv.org/abs/2312.10997

arXiv 2024
[7]

In: Webber, B., Cohn, T., He, Y., Liu, Y

Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., Yih, W.t.: Dense passage retrieval for open-domain question answering. In: Webber, B., Cohn, T., He, Y., Liu, Y. (eds.) Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 6769–6781. Association for Computational Linguistics, Online (...

doi:10.18653/v1/2020.emnlp-main.550 2020
[8]

In: Chaudhuri, K., Salakhutdinov, R

Kornblith, S., Norouzi, M., Lee, H., Hinton, G.: Similarity of neural network rep- resentations revisited. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 97, pp. 3519–3529. PMLR (09–15 Jun 2019)

2019

Show all 17 references

[9]

In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.t., Rocktäschel, T., Riedel, S., Kiela, D.: Retrieval-augmented generation for knowledge-intensive NLP tasks. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, ...

2020
[10]

In: Vlachos, A., Augenstein, I

Muennighoﬀ, N., Tazi, N., Magne, L., Reimers, N.: MTEB: Massive text embed- ding benchmark. In: Vlachos, A., Augenstein, I. (eds.) Proceedings of the 17th Conference of the European Chapter of the Association for Computational Lin- guistics. pp. 2014–2037. Association for Comp...

2014 doi
[11]

Plaat, A., van Duijn, M., Van Stein, N., Preuss, M., van der Putten, P., Batenburg, K.J.: Agentic large language models, a survey. J. Artif. Int. Res. 84 (Jan 2026). https://doi.org/10.1613/jair.1.18675

2026 doi
[12]

In: Inui, K., Jiang, J., Ng, V., Wan, X

Reimers, N., Gurevych, I.: Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In: Inui, K., Jiang, J., Ng, V., Wan, X. (eds.) Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natur...

2019 doi
[13]

Tacheny, N.: Calibrated similarity for reliable geometric analysis of embedding spaces (2026), https://arxiv.org/abs/2601.16907

2026
[14]

In: Thirty-ﬁfth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) (2021)

Thakur, N., Reimers, N., Rücklé, A., Srivastava, A., Gurevych, I.: BEIR: A het- erogeneous benchmark for zero-shot evaluation of information retrieval models. In: Thirty-ﬁfth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) (2021)

2021
[15]

In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

Wadden, D., Lin, S., Lo, K., Wang, L.L., van Zuylen, M., Cohan, A., Ha- jishirzi, H.: Fact or ﬁction: Verifying scientiﬁc claims. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 7534–7550. Association for Computational Li...

2020 doi
[16]

You, K.: Semantics at an angle: When cosine similarity works until it doesn’t (2025), https://arxiv.org/abs/2504.16318

2025 arXiv
[2851]

https://doi.org/10.18653/v1/2026.eacl-long.130 Mapping Similarity Spaces across Embedding Models with SQP 15

Association for Computational Linguistics, Rabat, Morocco (Mar 2026). https://doi.org/10.18653/v1/2026.eacl-long.130 Mapping Similarity Spaces across Embedding Models with SQP 15

2026 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [2]

(eds.) Machine Learning and Principles and Practice of Knowledge Discovery in Databases

van Elburg, J., van der Putten, P., Marx, M.: Can we evaluate RAGs with synthetic data? In: Koprinska, I., Mendes-Moreira, J., Branco, P. (eds.) Machine Learning and Principles and Practice of Knowledge Discovery in Databases. SynDAiTE workshop at ECML PKDD 2025. pp. 177–192. Springer Nature Switzerland, Cham (2026)

work page 2026

[2] [1]

In: Petroni, F., Siciliano, F., Silvestri, F., Trappolini, G

Caspari, L., Dastidar, K.G., Zerhoudi, S., Mitrovic, J., Granitzer, M.: Beyond benchmarks: Evaluating embedding model similarity for retrieval augmented gen- eration systems. In: Petroni, F., Siciliano, F., Silvestri, F., Trappolini, G. (eds.) Proceedings of the Workshop Information Retrieval’s Role in RAG Systems (IR- RAG 2024) co-located with the 47th I...

work page 2024

[3] [3]

In: Yue, Y., Garg, A., Peng, N., Sha, F., Yu, R

Enevoldsen, K., Chung, I., Kerboua, I., Kardos, M., Mathur, A., Stap, D., Gala, J., Siblini, W., Krzemiński, D., Winata, G., Sturua, S., Utpala, S., Ciancone, M., Schaeﬀer, M., Misra, D., Dhakal, S., Rystrø m, J., Solomatin, R., Çağatan, O., Kundu, A., Bernstorﬀ, M., Xiao, S., Sukhlecha, A., Pahwa, B., Poświata, R., GV, K.K., Ashraf, S., Auras, D., Plüste...

work page 2025

[4] [4]

In: Demberg, V., Inui, K., Mar- quez, L

Frank, M., Aﬂi, H.: PTEB: Towards robust text embedding evaluation via stochas- tic paraphrasing at evaluation time with LLMs. In: Demberg, V., Inui, K., Mar- quez, L. (eds.) Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 2832–

work page

[5] [5]

Frank, M., Aﬂi, H.: The Harder Text Embedding Benchmark (HTEB): Beyond one-dimensional static robustness (2026), https://arxiv.org/abs/2605.28190

work page Pith review arXiv 2026

[6] [6]

Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., Wang, H.: Retrieval-augmented generation for large language models: A survey (2024), https://arxiv.org/abs/2312.10997

arXiv 2024

[7] [7]

In: Webber, B., Cohn, T., He, Y., Liu, Y

Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., Yih, W.t.: Dense passage retrieval for open-domain question answering. In: Webber, B., Cohn, T., He, Y., Liu, Y. (eds.) Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 6769–6781. Association for Computational Linguistics, Online (...

doi:10.18653/v1/2020.emnlp-main.550 2020

[8] [8]

In: Chaudhuri, K., Salakhutdinov, R

Kornblith, S., Norouzi, M., Lee, H., Hinton, G.: Similarity of neural network rep- resentations revisited. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 97, pp. 3519–3529. PMLR (09–15 Jun 2019)

2019