Pith. sign in

REVIEW 3 major objections 1 minor 68 references

Cosine similarity between label unembeddings tells us nothing about the probabilities a softmax model assigns.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · grok-4.5

2026-07-13 15:42 UTC pith:GGHRL4VD

load-bearing objection Abstract claims a clean softmax reparameterization result on unembeddings; the supplied full text is an unrelated nuclear-physics paper, so the claim is unverifiable. the 3 major comments →

arxiv 2603.29488 v2 pith:GGHRL4VD submitted 2026-03-31 cs.LG

What Cosine Similarity of Label Representations Can and Cannot Tell us

classification cs.LG
keywords cosine similarityunembeddingssoftmax classifierlabel representationsneural network interpretabilitysigmoid classifierranking geometry
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Cosine similarity is routinely used to compare vector representations inside neural networks, including the vectors that map internal states to class or token scores (unembeddings). This paper shows that for a softmax classifier—whether an image classifier or an autoregressive language model—that similarity carries no information about the probabilities the model actually produces. Given any two unembeddings, one can construct another model that assigns exactly the same probability to every label for every input, yet forces the cosine similarity of those two vectors to be either +1 or −1. For multi-label sigmoid classifiers the picture is different: the full set of pairwise cosine similarities among unembeddings does constrain which label combinations can occur. For softmax models, however, one needs the cosine similarities among all differences of unembeddings to recover the rankings the model can output. The authors conclude that reading meaning into unembedding cosine similarity without reference to the classifier that produced the vectors is misleading.

Core claim

For any softmax classifier, the cosine similarity between any two label unembeddings can be driven to +1 or to −1 by a reparameterization that leaves the model’s output probabilities completely unchanged for every input. Consequently raw cosine similarity of unembeddings conveys no information about model probabilities.

What carries the argument

A probability-preserving reparameterization of the unembedding vectors (and the map into them) that forces the cosine of any chosen pair to ±1 while the softmax distribution over labels remains identical for all inputs.

Load-bearing premise

The model class is closed under free reparameterizations of the unembeddings and the preceding map, so that softmax probabilities can stay fixed while cosine similarity is forced to plus or minus one.

What would settle it

Produce a concrete softmax architecture whose final linear layer cannot be reparameterized so as to keep every input’s probability vector identical while driving the cosine of two chosen unembeddings to +1 or −1; or prove that every probability-preserving map necessarily preserves that cosine.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • High cosine similarity between class or token unembeddings cannot be read as evidence that the model treats those labels as similar in probability space.
  • Interpretability analyses of language-model unembeddings that rely only on cosine similarity do not constrain next-token ranking behavior.
  • For multi-label sigmoid classifiers, pairwise unembedding cosines do determine the set of realizable label combinations.
  • Recovering the rankings a softmax model can produce requires the geometry of all differences of unembeddings, not the unembeddings themselves.
  • Any claim that two labels are ‘similar’ because their unembeddings have high cosine must be re-checked against the actual classifier head.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Many published plots that cluster or color labels by unembedding cosine may be visualizing an arbitrary gauge freedom rather than model behavior.
  • Regularizing unembedding geometry during training may leave softmax probabilities almost untouched unless earlier layers or residual-stream maps are also constrained.
  • Classifier-aware metrics (for example induced ranking geometry from differences of unembeddings) are more faithful reportables than raw cosine tables for large language models.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 1 minor

Summary. The abstract claims that, for any softmax classifier (image classifier or autoregressive LM), cosine similarity between label unembeddings conveys no information about model probabilities: given any two unembeddings one can construct another model with identical input-to-probability maps for all inputs yet with cosine similarity of those two unembeddings forced to +1 or −1. It further contrasts sigmoid classifiers (pairwise unembedding cosines determine feasible multi-label sets) with softmax classifiers (pairwise cosines of all unembedding differences are needed to determine feasible rankings). The supplied full manuscript body, however, is an unrelated nuclear-physics paper on the Ghent single-pion-production model: multipole expansion, Watson’s theorem via K-matrix unitarization, Delta form-factor updates from MAID, ρ/ω exchanges, and comparisons to CLAS, MAID and DCC data. No definitions of unembeddings, no reparameterization maps, no statements about softmax invariance, and no theorems on cosine similarity appear.

Significance. If the abstract’s constructive reparameterization were present and correct, it would be a useful cautionary result for representation-similarity practice in interpretability and model analysis, clarifying that unembedding geometry is not an invariant of the induced probability map. Because the body contains none of that material, the claimed contribution cannot be evaluated and currently has no verifiable significance as an ML paper.

major comments (3)
  1. Title/abstract vs. body mismatch: the manuscript body (Sections II–VII and Appendices A–H) is a complete nuclear-physics paper on unitarizing the Ghent SPP model (Watson’s theorem, multipoles E/M/S, Delta width and C3–C5 form factors, CLAS comparisons). None of the abstract’s objects (unembeddings, softmax/sigmoid classifiers, cosine of label representations, reparameterizations preserving probabilities) appear. The central claim is therefore unsupported by the submitted text.
  2. Absent constructive proof: the abstract’s strongest claim—that for any two unembeddings one can build a model with identical probabilities for all inputs but cosine similarity ±1—has no statement, no map on the representation/unembedding pair, and no conditions on biases or residual freedom. Without that construction the invariance argument cannot be checked (including closure of the model class under the reparameterization).
  3. Absent secondary claims: the sigmoid vs. softmax distinction (pairwise cosines of unembeddings vs. of all differences of unembeddings determining feasible label sets/rankings) is likewise missing from every section and appendix; no supporting lemmas or counter-examples are present.
minor comments (1)
  1. Even as a nuclear-physics manuscript the supplied text has OCR/encoding artifacts (garbled section headers, missing characters in equations) that would need cleanup if that were the intended submission; they are secondary to the abstract–body mismatch.

Circularity Check

0 steps flagged

No circularity found: the claimed cosine-similarity reparameterization is absent from the supplied manuscript, and the nuclear-physics text that is present is a standard model-update paper without circular derivation.

full rationale

The abstract asserts a constructive invariance result for softmax unembeddings (same probabilities for all inputs, cosine similarity forced to ±1). That style of claim is not circular by nature—it is an existence/reparameterization argument. However, the FULL MANUSCRIPT TEXT provided is an unrelated nuclear-physics paper on the Ghent single-pion-production model (Watson's theorem, multipole expansion, K-matrix unitarization, Delta form factors, CLAS comparisons). No definitions of unembeddings, no reparameterization maps, and no softmax-probability theorems appear, so the claimed derivation chain cannot be walked or reduced to its inputs. Within the nuclear-physics content that is actually present, form factors and couplings are taken from MAID and external πN analyses and then compared to CLAS data and other models; self-citations to prior Ghent-model papers are ordinary model development, not load-bearing uniqueness theorems that force the result. No step reduces a 'prediction' to a fitted input or self-definition by construction. Score 0 with empty steps is therefore the correct outcome.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 0 invented entities

Abstract-only review of a theoretical ML claim; the supplied full text is a mismatched physics manuscript and contributes no axioms for the cosine-similarity result. Load-bearing ingredients visible from the abstract are standard softmax/sigmoid definitions and the freedom to reparameterize unembeddings while preserving induced probabilities.

axioms (3)
  • domain assumption Softmax probabilities are determined by logits; transformations of unembeddings that leave all logits (or all logit differences) unchanged leave all model probabilities unchanged.
    Standard property of softmax classifiers; the abstract's construction relies on it.
  • ad hoc to paper One may construct an alternative model with the same input-to-probability map but different unembedding geometry (in particular cosine ±1 for a chosen pair).
    This is the paper's constructive claim; treated as an axiom of the argument until the proof is available.
  • domain assumption Sigmoid multi-label decisions depend on signs of independent scores; pairwise geometry of unembeddings can therefore constrain feasible label combinations.
    Standard for independent sigmoid heads; used for the contrast with softmax.

pith-pipeline@v1.1.0-grok45 · 27518 in / 2422 out tokens · 31094 ms · 2026-07-13T15:42:26.930300+00:00 · methodology

0 comments
read the original abstract

Cosine similarity is often used to measure the similarity of vector representations of neural network models. However, the cosine similarity of representations is not guaranteed to tell us anything about model probabilities. In this paper we show that for a softmax classifier, be it an image classifier or an autoregressive language model, the cosine similarity between label representations (called unembeddings in the paper) does not give any information on the probabilities assigned by the model. Specifically, we prove that given two unembeddings, it is possible to create another model which assigns the same probabilities for all inputs, but where the cosine similarity between the representations is now either 1 or -1. We also show that for a sigmoid classifier (where each input can be assigned multiple labels), all pairwise cosine similarities between the unembeddings define the set of possible label combinations. However, for softmax classifiers (where each input is assigned a ranking of the labels from most to least likely), we need all pairwise cosine similarities between all differences of unembeddings to know which rankings the model can predict. We conclude that it is misleading to interpret the cosine similarity between unembeddings without reference to the classifier that produced them.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

68 extracted references · 24 linked inside Pith

  1. [1]

    These are the amplitudes of Eq

    These multipoles can now be combined to obtain multipoles with a fixed isospinI, as explained in Appendix D. These are the amplitudes of Eq. (25) and are then used to compute the elements of the hadron tensor. Expressions are given in Appendix F. In particular for the angle-integrated hadron tensor elements of Eq. (19) one obtains � dΩ� H11 +H 22 2 = 2π �...

  2. [2]

    Reducing the notation of the amplitudes to �� �J, M, l,1/2�T�J, M, l,1/2��� �T � ��� and �� �J, M, l,1/2�T�J, M, λ���� �T � ���, Eq. (C13) implies � 2l+ 1 2J+ 1 �l,0,1/2, λ � �J, λ� �(T� ���)� T � ��� + � 2l+ 3 2J+ 1 �l+ 1,0,1/2, λ � �J, λ� �(T� ���+1)� T � ���+1 ��.(C14) This holds for arbitraryλ � =� 1 2: � 2l+ 1 2J+ 1 �l,0,1/2,1/2�J,1/2�(T � ���)� T � ...

  3. [3]

    Alvarez-Ruso�� ���(NuSTEC), Prog

    L. Alvarez-Ruso�� ���(NuSTEC), Prog. Part. Nucl. Phys.���, 1 (2018), arXiv:1706.03621 [hep-ph]

  4. [4]

    Katori and M

    T. Katori and M. Martini, J. Phys. G��, 013001 (2018), arXiv:1611.07770 [hep-ph]

  5. [5]

    Acciarri�� ���(DUNE), (2016)

    R. Acciarri�� ���(DUNE), (2016)

  6. [6]

    M. A. Acero�� ���(NOvA, R. Group), Eur. Phys. J. C ��, 1119 (2020)

  7. [7]

    M. A. Acero�� ���(NOvA), Phys. Rev. D���, 052011 (2023)

  8. [8]

    Abe�� ���(T2K), Nucl

    K. Abe�� ���(T2K), Nucl. Instrum. Meth. A���, 106 (2011), arXiv:1106.1238 [physics.ins-det]

  9. [9]

    Abe�� ���, (2011)

    K. Abe�� ���, (2011)

  10. [10]

    Acciarri�� ���(MicroBooNE, LAr1-ND, ICARUS- WA104), (2015)

    R. Acciarri�� ���(MicroBooNE, LAr1-ND, ICARUS- WA104), (2015)

  11. [11]

    Isaacson, W

    J. Isaacson, W. Jay, A. Lovato, P. Machado, A. Niko- lakopoulos, N. Rocco, and N. Steinberg, (2025), arXiv:2508.19213 [hep-ph]

  12. [12]

    Hayato and L

    Y. Hayato and L. Pickering, Eur. Phys. J. ST���, 4469 (2021), arXiv:2106.15809 [hep-ph]

  13. [13]

    Leitner and U

    T. Leitner and U. Mosel, Phys. Rev. C��, 064614 (2010), arXiv:1004.4433 [nucl-th]

  14. [14]

    Khachatryan�� ���(CLAS, e4v), Nature���, 565 (2021)

    M. Khachatryan�� ���(CLAS, e4v), Nature���, 565 (2021)

  15. [15]

    D¨ opper and N

    N. D¨ opper and N. Kaiser, Nucl. Phys. A����, 123323 (2026)

  16. [16]

    D.-L. Yao, L. Alvarez-Ruso, A. N. Hiller Blin, and M. J. Vicente Vacas, Phys. Rev. D��, 076004 (2018), arXiv:1806.09364 [hep-ph]

  17. [17]

    D.-L. Yao, L. Alvarez-Ruso, A. N. Hiller Blin, and M. J. Vicente Vacas, PoS������, 048 (2019)

  18. [18]

    Hern´ andez, J

    E. Hern´ andez, J. Nieves, and M. Valverde, Phys. Rev. D��, 033005 (2007)

  19. [19]

    Gonz´ alez-Jim´ enez, K

    R. Gonz´ alez-Jim´ enez, K. Niewczas, and N. Jachowicz, Phys. Rev. D��, 013004 (2018), arXiv:1710.08374 [nucl- th]

  20. [20]

    Kabirnezhad, (2022), arXiv:2203.15594 [hep-ph]

    M. Kabirnezhad, (2022), arXiv:2203.15594 [hep-ph]

  21. [21]

    Lalakulich, E

    O. Lalakulich, E. A. Paschos, and G. Piranishvili, Phys. Rev. D��, 014009 (2006)

  22. [22]

    Gonz´ alez-Jim´ enez, N

    R. Gonz´ alez-Jim´ enez, N. Jachowicz, K. Niewczas, J. Nys, V. Pandey, T. Van Cuyck, and N. Van Dessel, Phys. Rev. D��, 113007 (2017), arXiv:1612.05511 [nucl-th]

  23. [23]

    Hern´ andez, J

    E. Hern´ andez, J. Nieves, S. K. Singh, M. Valverde, and M. J. Vicente Vacas, Phys. Rev. D��, 053009 (2008)

  24. [24]

    Hern´ andez, J

    E. Hern´ andez, J. Nieves, M. Valverde, and M. J. Vicente Vacas, Phys. Rev. D��, 085046 (2010)

  25. [25]

    Hern´ andez, J

    E. Hern´ andez, J. Nieves, and M. J. V. Vacas, Phys. Rev. D��, 113009 (2013)

  26. [26]

    Alvarez-Ruso, E

    L. Alvarez-Ruso, E. Hern´ andez, J. Nieves, and M. J. Vicente Vacas, Phys. Rev. D��, 014016 (2016)

  27. [27]

    Olsson, Nuclear Physics B��, 55 (1974)

    M. Olsson, Nuclear Physics B��, 55 (1974)

  28. [28]

    I. G. Aznauryan, Phys. Rev. C��, 015209 (2003)

  29. [29]

    I. G. Aznauryan, V. D. Burkert, H. Egiyan, K. Joo, R. Minehart, and L. C. Smith, Phys. Rev. C��, 015201 (2005)

  30. [30]

    De Cruz, T

    L. De Cruz, T. Vrancx, P. Vancraeyveld, and J. Rycke- busch, Phys. Rev. Lett.���, 182002 (2012)

  31. [31]

    Corthals, J

    T. Corthals, J. Ryckebusch, and T. Van Cauteren, Phys. Rev. C��, 045207 (2006)

  32. [32]

    Garc´ ıa-Marcos, T

    J. Garc´ ıa-Marcos, T. Franco-Munoz, R. Gonz´ alez- Jim´ enez, A. Nikolakopoulos, N. Jachowicz, and J. M. Ud´ ıas, Phys. Rev. C���, 024608 (2024), arXiv:2310.18056 [nucl-th]

  33. [33]

    Nikolakopoulos, R

    A. Nikolakopoulos, R. Gonz´ alez-Jim´ enez, N. Jachowicz, and J. M. Ud´ ıas, Phys. Rev. D���, 053007 (2023)

  34. [34]

    Gonz´ alez-Jim´ enez, A

    R. Gonz´ alez-Jim´ enez, A. Nikolakopoulos, N. Jachowicz, and J. M. Ud´ ıas, Phys. Rev. C���, 045501 (2019), arXiv:1904.10696 [nucl-th]

  35. [35]

    Nikolakopoulos, R

    A. Nikolakopoulos, R. Gonz´ alez-Jim´ enez, K. Niewczas, J. Sobczyk, and N. Jachowicz, Phys. Rev. D��, 093008 (2018), arXiv:1803.03163 [nucl-th]

  36. [36]

    Q. Yan, K. Niewczas, A. Nikolakopoulos, R. Gonz´ alez- Jim´ enez, N. Jachowicz, X. Lu, J. Sobczyk, and Y. Zheng, JHEP��, 141 (2024), arXiv:2405.05212 [hep-ph]

  37. [37]

    Niewczas, A

    K. Niewczas, A. Nikolakopoulos, J. T. Sobczyk, N. Ja- chowicz, and R. Gonz´ alez-Jim´ enez, Phys. Rev. D���, 053003 (2021), arXiv:2011.05269 [hep-ph]

  38. [38]

    F. A. Berends, A. Donnachie, and D. L. Weaver, Nucl. Phys. B�, 1 (1967)

  39. [39]

    D¨ oring, J

    M. D¨ oring, J. Haidenbauer, M. Mai, and T. Sato, Prog. Part. Nucl. Phys.���, 104213 (2026), arXiv:2505.02745 [nucl-th]

  40. [40]

    Kamano, S

    H. Kamano, S. X. Nakamura, T. S. H. Lee, and T. Sato, Phys. Rev. C��, 035209 (2013), arXiv:1305.4351 [nucl- th]

  41. [41]

    Kamano, S

    H. Kamano, S. X. Nakamura, T. S. H. Lee, and T. Sato, Phys. Rev. C��, 015201 (2016), arXiv:1605.00363 [nucl- th]

  42. [42]

    S. X. Nakamura, H. Kamano, and T. Sato, Phys. Rev. D��, 074024 (2015), arXiv:1506.03403 [hep-ph]

  43. [43]

    S. U. Chung, J. Brose, R. Hackmann, E. Klempt, S. Spanier, and C. Strassburger, Annalen Phys.�, 404 (1995)

  44. [44]

    Drechsel, S

    D. Drechsel, S. S. Kamalov, and L. Tiator, Eur. Phys. J. A��, 69 (2007)

  45. [45]

    Drechsel, O

    D. Drechsel, O. Hanstein, S. S. Kamalov, and L. Tiator, Nucl. Phys. A���, 145 (1999)

  46. [46]

    K. M. Watson, Phys. Rev.��, 1163 (1952)

  47. [47]

    Alekou�� ���, Eur

    A. Alekou�� ���, Eur. Phys. J. ST���, 3779 (2022), [Erratum: Eur.Phys.J.ST 232, 15–16 (2023)], arXiv:2206.01208 [hep-ex]

  48. [48]

    J. E. Sobczyk, E. Hern´ andez, S. X. Nakamura, J. Nieves, and T. Sato, Phys. Rev. D��, 073001 (2018), arXiv:1807.11281 [hep-ph]

  49. [49]

    Guidal, J

    M. Guidal, J. M. Laget, and M. Vanderhaeghen, Nucl. Phys. A���, 645 (1997)

  50. [50]

    S. S. Kamalov, S. N. Yang, D. Drechsel, O. Hanstein, and L. Tiator, Phys. Rev. C��, 032201 (2001), arXiv:nucl- th/0006068

  51. [51]

    Ronchen, M

    D. Ronchen, M. Doring, F. Huang, H. Haberzettl, J. Haidenbauer, C. Hanhart, S. Krewald, U. G. Meiss- ner, and K. Nakayama, Eur. Phys. J. A��, 44 (2013), arXiv:1211.6998 [nucl-th]

  52. [52]

    Workman, Phys

    R. Workman, Phys. Rev. C��, 055207 (2006)

  53. [53]

    Tiator, D

    L. Tiator, D. Drechsel, S. S. Kamalov, and M. Van- derhaeghen, Eur. Phys. J. ST���, 141 (2011), arXiv:1109.6745 [nucl-th]

  54. [54]

    I. G. Aznauryan�� ���(CLAS), Phys. Rev. C��, 055203 (2009), arXiv:0909.2349 [nucl-ex]

  55. [55]

    Rarita and J

    W. Rarita and J. Schwinger, Phys. Rev.��, 61 (1941)

  56. [56]

    R. L. Workman�� ���(Particle Data Group), PTEP ����, 083C01 (2022)

  57. [57]

    T. J. Leitner,���������������� ������������ �� � �������� 26 ������� �������� ��������� �����, thesis (2009)

  58. [58]

    Pascalutsa and R

    V. Pascalutsa and R. Timmermans, Phys. Rev. C��, 042201 (1999), arXiv:nucl-th/9905065

  59. [59]

    I. G. Aznauryan, V. D. Burkert, and T. S. H. Lee, (2008), arXiv:0810.0997 [nucl-th]

  60. [60]

    Leitner, O

    T. Leitner, O. Buss, L. Alvarez-Ruso, and U. Mosel, Phys. Rev. C��, 034601 (2009), arXiv:0812.0587 [nucl- th]

  61. [61]

    CLAS Physics Database,

    Skobeltsyn Institute of Nuclear Physics, “CLAS Physics Database,”�������������������������������������� ������(n.d.), moscow State University

  62. [62]

    Garcilazo and E

    H. Garcilazo and E. Moya de Guerra, Nucl. Phys. A���, 521 (1993)

  63. [63]

    Nozawa, B

    S. Nozawa, B. Blankleider, and T. S. H. Lee, Nucl. Phys. A���, 459 (1990)

  64. [64]

    Matsuyama, T

    A. Matsuyama, T. Sato, and T. S. H. Lee, Phys. Rept. ���, 193 (2007)

  65. [65]

    Allasia�� ���, Nucl

    D. Allasia�� ���, Nucl. Phys. B���, 285 (1990)

  66. [66]

    Jacob and G

    M. Jacob and G. C. Wick, Annals Phys.�, 404 (1959)

  67. [67]

    Dennery, Phys

    P. Dennery, Phys. Rev.���, 2000 (1961)

  68. [68]

    G. F. Chew, M. L. Goldberger, F. E. Low, and Y. Nambu, Phys. Rev.���, 1345 (1957)