REVIEW 3 major objections 1 minor 68 references
Cosine similarity between label unembeddings tells us nothing about the probabilities a softmax model assigns.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-13 15:42 UTC pith:GGHRL4VD
load-bearing objection Abstract claims a clean softmax reparameterization result on unembeddings; the supplied full text is an unrelated nuclear-physics paper, so the claim is unverifiable. the 3 major comments →
What Cosine Similarity of Label Representations Can and Cannot Tell us
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
For any softmax classifier, the cosine similarity between any two label unembeddings can be driven to +1 or to −1 by a reparameterization that leaves the model’s output probabilities completely unchanged for every input. Consequently raw cosine similarity of unembeddings conveys no information about model probabilities.
What carries the argument
A probability-preserving reparameterization of the unembedding vectors (and the map into them) that forces the cosine of any chosen pair to ±1 while the softmax distribution over labels remains identical for all inputs.
Load-bearing premise
The model class is closed under free reparameterizations of the unembeddings and the preceding map, so that softmax probabilities can stay fixed while cosine similarity is forced to plus or minus one.
What would settle it
Produce a concrete softmax architecture whose final linear layer cannot be reparameterized so as to keep every input’s probability vector identical while driving the cosine of two chosen unembeddings to +1 or −1; or prove that every probability-preserving map necessarily preserves that cosine.
If this is right
- High cosine similarity between class or token unembeddings cannot be read as evidence that the model treats those labels as similar in probability space.
- Interpretability analyses of language-model unembeddings that rely only on cosine similarity do not constrain next-token ranking behavior.
- For multi-label sigmoid classifiers, pairwise unembedding cosines do determine the set of realizable label combinations.
- Recovering the rankings a softmax model can produce requires the geometry of all differences of unembeddings, not the unembeddings themselves.
- Any claim that two labels are ‘similar’ because their unembeddings have high cosine must be re-checked against the actual classifier head.
Where Pith is reading between the lines
- Many published plots that cluster or color labels by unembedding cosine may be visualizing an arbitrary gauge freedom rather than model behavior.
- Regularizing unembedding geometry during training may leave softmax probabilities almost untouched unless earlier layers or residual-stream maps are also constrained.
- Classifier-aware metrics (for example induced ranking geometry from differences of unembeddings) are more faithful reportables than raw cosine tables for large language models.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The abstract claims that, for any softmax classifier (image classifier or autoregressive LM), cosine similarity between label unembeddings conveys no information about model probabilities: given any two unembeddings one can construct another model with identical input-to-probability maps for all inputs yet with cosine similarity of those two unembeddings forced to +1 or −1. It further contrasts sigmoid classifiers (pairwise unembedding cosines determine feasible multi-label sets) with softmax classifiers (pairwise cosines of all unembedding differences are needed to determine feasible rankings). The supplied full manuscript body, however, is an unrelated nuclear-physics paper on the Ghent single-pion-production model: multipole expansion, Watson’s theorem via K-matrix unitarization, Delta form-factor updates from MAID, ρ/ω exchanges, and comparisons to CLAS, MAID and DCC data. No definitions of unembeddings, no reparameterization maps, no statements about softmax invariance, and no theorems on cosine similarity appear.
Significance. If the abstract’s constructive reparameterization were present and correct, it would be a useful cautionary result for representation-similarity practice in interpretability and model analysis, clarifying that unembedding geometry is not an invariant of the induced probability map. Because the body contains none of that material, the claimed contribution cannot be evaluated and currently has no verifiable significance as an ML paper.
major comments (3)
- Title/abstract vs. body mismatch: the manuscript body (Sections II–VII and Appendices A–H) is a complete nuclear-physics paper on unitarizing the Ghent SPP model (Watson’s theorem, multipoles E/M/S, Delta width and C3–C5 form factors, CLAS comparisons). None of the abstract’s objects (unembeddings, softmax/sigmoid classifiers, cosine of label representations, reparameterizations preserving probabilities) appear. The central claim is therefore unsupported by the submitted text.
- Absent constructive proof: the abstract’s strongest claim—that for any two unembeddings one can build a model with identical probabilities for all inputs but cosine similarity ±1—has no statement, no map on the representation/unembedding pair, and no conditions on biases or residual freedom. Without that construction the invariance argument cannot be checked (including closure of the model class under the reparameterization).
- Absent secondary claims: the sigmoid vs. softmax distinction (pairwise cosines of unembeddings vs. of all differences of unembeddings determining feasible label sets/rankings) is likewise missing from every section and appendix; no supporting lemmas or counter-examples are present.
minor comments (1)
- Even as a nuclear-physics manuscript the supplied text has OCR/encoding artifacts (garbled section headers, missing characters in equations) that would need cleanup if that were the intended submission; they are secondary to the abstract–body mismatch.
Circularity Check
No circularity found: the claimed cosine-similarity reparameterization is absent from the supplied manuscript, and the nuclear-physics text that is present is a standard model-update paper without circular derivation.
full rationale
The abstract asserts a constructive invariance result for softmax unembeddings (same probabilities for all inputs, cosine similarity forced to ±1). That style of claim is not circular by nature—it is an existence/reparameterization argument. However, the FULL MANUSCRIPT TEXT provided is an unrelated nuclear-physics paper on the Ghent single-pion-production model (Watson's theorem, multipole expansion, K-matrix unitarization, Delta form factors, CLAS comparisons). No definitions of unembeddings, no reparameterization maps, and no softmax-probability theorems appear, so the claimed derivation chain cannot be walked or reduced to its inputs. Within the nuclear-physics content that is actually present, form factors and couplings are taken from MAID and external πN analyses and then compared to CLAS data and other models; self-citations to prior Ghent-model papers are ordinary model development, not load-bearing uniqueness theorems that force the result. No step reduces a 'prediction' to a fitted input or self-definition by construction. Score 0 with empty steps is therefore the correct outcome.
Axiom & Free-Parameter Ledger
axioms (3)
- domain assumption Softmax probabilities are determined by logits; transformations of unembeddings that leave all logits (or all logit differences) unchanged leave all model probabilities unchanged.
- ad hoc to paper One may construct an alternative model with the same input-to-probability map but different unembedding geometry (in particular cosine ±1 for a chosen pair).
- domain assumption Sigmoid multi-label decisions depend on signs of independent scores; pairwise geometry of unembeddings can therefore constrain feasible label combinations.
read the original abstract
Cosine similarity is often used to measure the similarity of vector representations of neural network models. However, the cosine similarity of representations is not guaranteed to tell us anything about model probabilities. In this paper we show that for a softmax classifier, be it an image classifier or an autoregressive language model, the cosine similarity between label representations (called unembeddings in the paper) does not give any information on the probabilities assigned by the model. Specifically, we prove that given two unembeddings, it is possible to create another model which assigns the same probabilities for all inputs, but where the cosine similarity between the representations is now either 1 or -1. We also show that for a sigmoid classifier (where each input can be assigned multiple labels), all pairwise cosine similarities between the unembeddings define the set of possible label combinations. However, for softmax classifiers (where each input is assigned a ranking of the labels from most to least likely), we need all pairwise cosine similarities between all differences of unembeddings to know which rankings the model can predict. We conclude that it is misleading to interpret the cosine similarity between unembeddings without reference to the classifier that produced them.
Reference graph
Works this paper leans on
-
[1]
These are the amplitudes of Eq
These multipoles can now be combined to obtain multipoles with a fixed isospinI, as explained in Appendix D. These are the amplitudes of Eq. (25) and are then used to compute the elements of the hadron tensor. Expressions are given in Appendix F. In particular for the angle-integrated hadron tensor elements of Eq. (19) one obtains � dΩ� H11 +H 22 2 = 2π �...
-
[2]
Reducing the notation of the amplitudes to �� �J, M, l,1/2�T�J, M, l,1/2��� �T � ��� and �� �J, M, l,1/2�T�J, M, λ���� �T � ���, Eq. (C13) implies � 2l+ 1 2J+ 1 �l,0,1/2, λ � �J, λ� �(T� ���)� T � ��� + � 2l+ 3 2J+ 1 �l+ 1,0,1/2, λ � �J, λ� �(T� ���+1)� T � ���+1 ��.(C14) This holds for arbitraryλ � =� 1 2: � 2l+ 1 2J+ 1 �l,0,1/2,1/2�J,1/2�(T � ���)� T � ...
-
[3]
Alvarez-Ruso�� ���(NuSTEC), Prog
L. Alvarez-Ruso�� ���(NuSTEC), Prog. Part. Nucl. Phys.���, 1 (2018), arXiv:1706.03621 [hep-ph]
Pith/arXiv arXiv 2018
-
[4]
T. Katori and M. Martini, J. Phys. G��, 013001 (2018), arXiv:1611.07770 [hep-ph]
Pith/arXiv arXiv 2018
-
[5]
Acciarri�� ���(DUNE), (2016)
R. Acciarri�� ���(DUNE), (2016)
2016
-
[6]
M. A. Acero�� ���(NOvA, R. Group), Eur. Phys. J. C ��, 1119 (2020)
2020
-
[7]
M. A. Acero�� ���(NOvA), Phys. Rev. D���, 052011 (2023)
2023
-
[8]
K. Abe�� ���(T2K), Nucl. Instrum. Meth. A���, 106 (2011), arXiv:1106.1238 [physics.ins-det]
arXiv 2011
-
[9]
Abe�� ���, (2011)
K. Abe�� ���, (2011)
2011
-
[10]
Acciarri�� ���(MicroBooNE, LAr1-ND, ICARUS- WA104), (2015)
R. Acciarri�� ���(MicroBooNE, LAr1-ND, ICARUS- WA104), (2015)
2015
-
[11]
J. Isaacson, W. Jay, A. Lovato, P. Machado, A. Niko- lakopoulos, N. Rocco, and N. Steinberg, (2025), arXiv:2508.19213 [hep-ph]
arXiv 2025
-
[12]
Y. Hayato and L. Pickering, Eur. Phys. J. ST���, 4469 (2021), arXiv:2106.15809 [hep-ph]
Pith/arXiv arXiv 2021
-
[13]
T. Leitner and U. Mosel, Phys. Rev. C��, 064614 (2010), arXiv:1004.4433 [nucl-th]
Pith/arXiv arXiv 2010
-
[14]
Khachatryan�� ���(CLAS, e4v), Nature���, 565 (2021)
M. Khachatryan�� ���(CLAS, e4v), Nature���, 565 (2021)
2021
-
[15]
D¨ opper and N
N. D¨ opper and N. Kaiser, Nucl. Phys. A����, 123323 (2026)
2026
-
[16]
D.-L. Yao, L. Alvarez-Ruso, A. N. Hiller Blin, and M. J. Vicente Vacas, Phys. Rev. D��, 076004 (2018), arXiv:1806.09364 [hep-ph]
Pith/arXiv arXiv 2018
-
[17]
D.-L. Yao, L. Alvarez-Ruso, A. N. Hiller Blin, and M. J. Vicente Vacas, PoS������, 048 (2019)
2019
-
[18]
Hern´ andez, J
E. Hern´ andez, J. Nieves, and M. Valverde, Phys. Rev. D��, 033005 (2007)
2007
-
[19]
R. Gonz´ alez-Jim´ enez, K. Niewczas, and N. Jachowicz, Phys. Rev. D��, 013004 (2018), arXiv:1710.08374 [nucl- th]
Pith/arXiv arXiv 2018
-
[20]
Kabirnezhad, (2022), arXiv:2203.15594 [hep-ph]
M. Kabirnezhad, (2022), arXiv:2203.15594 [hep-ph]
Pith/arXiv arXiv 2022
-
[21]
Lalakulich, E
O. Lalakulich, E. A. Paschos, and G. Piranishvili, Phys. Rev. D��, 014009 (2006)
2006
-
[22]
R. Gonz´ alez-Jim´ enez, N. Jachowicz, K. Niewczas, J. Nys, V. Pandey, T. Van Cuyck, and N. Van Dessel, Phys. Rev. D��, 113007 (2017), arXiv:1612.05511 [nucl-th]
Pith/arXiv arXiv 2017
-
[23]
Hern´ andez, J
E. Hern´ andez, J. Nieves, S. K. Singh, M. Valverde, and M. J. Vicente Vacas, Phys. Rev. D��, 053009 (2008)
2008
-
[24]
Hern´ andez, J
E. Hern´ andez, J. Nieves, M. Valverde, and M. J. Vicente Vacas, Phys. Rev. D��, 085046 (2010)
2010
-
[25]
Hern´ andez, J
E. Hern´ andez, J. Nieves, and M. J. V. Vacas, Phys. Rev. D��, 113009 (2013)
2013
-
[26]
Alvarez-Ruso, E
L. Alvarez-Ruso, E. Hern´ andez, J. Nieves, and M. J. Vicente Vacas, Phys. Rev. D��, 014016 (2016)
2016
-
[27]
Olsson, Nuclear Physics B��, 55 (1974)
M. Olsson, Nuclear Physics B��, 55 (1974)
1974
-
[28]
I. G. Aznauryan, Phys. Rev. C��, 015209 (2003)
2003
-
[29]
I. G. Aznauryan, V. D. Burkert, H. Egiyan, K. Joo, R. Minehart, and L. C. Smith, Phys. Rev. C��, 015201 (2005)
2005
-
[30]
De Cruz, T
L. De Cruz, T. Vrancx, P. Vancraeyveld, and J. Rycke- busch, Phys. Rev. Lett.���, 182002 (2012)
2012
-
[31]
Corthals, J
T. Corthals, J. Ryckebusch, and T. Van Cauteren, Phys. Rev. C��, 045207 (2006)
2006
-
[32]
J. Garc´ ıa-Marcos, T. Franco-Munoz, R. Gonz´ alez- Jim´ enez, A. Nikolakopoulos, N. Jachowicz, and J. M. Ud´ ıas, Phys. Rev. C���, 024608 (2024), arXiv:2310.18056 [nucl-th]
Pith/arXiv arXiv 2024
-
[33]
Nikolakopoulos, R
A. Nikolakopoulos, R. Gonz´ alez-Jim´ enez, N. Jachowicz, and J. M. Ud´ ıas, Phys. Rev. D���, 053007 (2023)
2023
-
[34]
R. Gonz´ alez-Jim´ enez, A. Nikolakopoulos, N. Jachowicz, and J. M. Ud´ ıas, Phys. Rev. C���, 045501 (2019), arXiv:1904.10696 [nucl-th]
Pith/arXiv arXiv 2019
-
[35]
A. Nikolakopoulos, R. Gonz´ alez-Jim´ enez, K. Niewczas, J. Sobczyk, and N. Jachowicz, Phys. Rev. D��, 093008 (2018), arXiv:1803.03163 [nucl-th]
Pith/arXiv arXiv 2018
-
[36]
Q. Yan, K. Niewczas, A. Nikolakopoulos, R. Gonz´ alez- Jim´ enez, N. Jachowicz, X. Lu, J. Sobczyk, and Y. Zheng, JHEP��, 141 (2024), arXiv:2405.05212 [hep-ph]
Pith/arXiv arXiv 2024
-
[37]
K. Niewczas, A. Nikolakopoulos, J. T. Sobczyk, N. Ja- chowicz, and R. Gonz´ alez-Jim´ enez, Phys. Rev. D���, 053003 (2021), arXiv:2011.05269 [hep-ph]
Pith/arXiv arXiv 2021
-
[38]
F. A. Berends, A. Donnachie, and D. L. Weaver, Nucl. Phys. B�, 1 (1967)
1967
-
[39]
M. D¨ oring, J. Haidenbauer, M. Mai, and T. Sato, Prog. Part. Nucl. Phys.���, 104213 (2026), arXiv:2505.02745 [nucl-th]
arXiv 2026
-
[40]
H. Kamano, S. X. Nakamura, T. S. H. Lee, and T. Sato, Phys. Rev. C��, 035209 (2013), arXiv:1305.4351 [nucl- th]
Pith/arXiv arXiv 2013
-
[41]
H. Kamano, S. X. Nakamura, T. S. H. Lee, and T. Sato, Phys. Rev. C��, 015201 (2016), arXiv:1605.00363 [nucl- th]
Pith/arXiv arXiv 2016
-
[42]
S. X. Nakamura, H. Kamano, and T. Sato, Phys. Rev. D��, 074024 (2015), arXiv:1506.03403 [hep-ph]
Pith/arXiv arXiv 2015
-
[43]
S. U. Chung, J. Brose, R. Hackmann, E. Klempt, S. Spanier, and C. Strassburger, Annalen Phys.�, 404 (1995)
1995
-
[44]
Drechsel, S
D. Drechsel, S. S. Kamalov, and L. Tiator, Eur. Phys. J. A��, 69 (2007)
2007
-
[45]
Drechsel, O
D. Drechsel, O. Hanstein, S. S. Kamalov, and L. Tiator, Nucl. Phys. A���, 145 (1999)
1999
-
[46]
K. M. Watson, Phys. Rev.��, 1163 (1952)
1952
-
[47]
A. Alekou�� ���, Eur. Phys. J. ST���, 3779 (2022), [Erratum: Eur.Phys.J.ST 232, 15–16 (2023)], arXiv:2206.01208 [hep-ex]
Pith/arXiv arXiv 2022
-
[48]
J. E. Sobczyk, E. Hern´ andez, S. X. Nakamura, J. Nieves, and T. Sato, Phys. Rev. D��, 073001 (2018), arXiv:1807.11281 [hep-ph]
Pith/arXiv arXiv 2018
-
[49]
Guidal, J
M. Guidal, J. M. Laget, and M. Vanderhaeghen, Nucl. Phys. A���, 645 (1997)
1997
-
[50]
S. S. Kamalov, S. N. Yang, D. Drechsel, O. Hanstein, and L. Tiator, Phys. Rev. C��, 032201 (2001), arXiv:nucl- th/0006068
arXiv 2001
-
[51]
D. Ronchen, M. Doring, F. Huang, H. Haberzettl, J. Haidenbauer, C. Hanhart, S. Krewald, U. G. Meiss- ner, and K. Nakayama, Eur. Phys. J. A��, 44 (2013), arXiv:1211.6998 [nucl-th]
Pith/arXiv arXiv 2013
-
[52]
Workman, Phys
R. Workman, Phys. Rev. C��, 055207 (2006)
2006
-
[53]
L. Tiator, D. Drechsel, S. S. Kamalov, and M. Van- derhaeghen, Eur. Phys. J. ST���, 141 (2011), arXiv:1109.6745 [nucl-th]
Pith/arXiv arXiv 2011
-
[54]
I. G. Aznauryan�� ���(CLAS), Phys. Rev. C��, 055203 (2009), arXiv:0909.2349 [nucl-ex]
Pith/arXiv arXiv 2009
-
[55]
Rarita and J
W. Rarita and J. Schwinger, Phys. Rev.��, 61 (1941)
1941
-
[56]
R. L. Workman�� ���(Particle Data Group), PTEP ����, 083C01 (2022)
2022
-
[57]
T. J. Leitner,���������������� ������������ �� � �������� 26 ������� �������� ��������� �����, thesis (2009)
2009
-
[58]
V. Pascalutsa and R. Timmermans, Phys. Rev. C��, 042201 (1999), arXiv:nucl-th/9905065
Pith/arXiv arXiv 1999
-
[59]
I. G. Aznauryan, V. D. Burkert, and T. S. H. Lee, (2008), arXiv:0810.0997 [nucl-th]
Pith/arXiv arXiv 2008
-
[60]
T. Leitner, O. Buss, L. Alvarez-Ruso, and U. Mosel, Phys. Rev. C��, 034601 (2009), arXiv:0812.0587 [nucl- th]
Pith/arXiv arXiv 2009
-
[61]
CLAS Physics Database,
Skobeltsyn Institute of Nuclear Physics, “CLAS Physics Database,”�������������������������������������� ������(n.d.), moscow State University
-
[62]
Garcilazo and E
H. Garcilazo and E. Moya de Guerra, Nucl. Phys. A���, 521 (1993)
1993
-
[63]
Nozawa, B
S. Nozawa, B. Blankleider, and T. S. H. Lee, Nucl. Phys. A���, 459 (1990)
1990
-
[64]
Matsuyama, T
A. Matsuyama, T. Sato, and T. S. H. Lee, Phys. Rept. ���, 193 (2007)
2007
-
[65]
Allasia�� ���, Nucl
D. Allasia�� ���, Nucl. Phys. B���, 285 (1990)
1990
-
[66]
Jacob and G
M. Jacob and G. C. Wick, Annals Phys.�, 404 (1959)
1959
-
[67]
Dennery, Phys
P. Dennery, Phys. Rev.���, 2000 (1961)
2000
-
[68]
G. F. Chew, M. L. Goldberger, F. E. Low, and Y. Nambu, Phys. Rev.���, 1345 (1957)
1957
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.