A Method for Estimating the Proximity of Vector Representation Groups in Multidimensional Space. On the Example of the Paraphrase Task

Artem Artemov; Boris Alekseev

REVIEW 4 major objections 5 minor 8 references

A Method for Estimating the Proximity of Vector Representation Groups in Multidimensional Space. On the Example of the Paraphrase Task

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a training-free projection cosine, averaged symmetrically over two word-vector groups, measures sentence similarity and scores 0.81 accuracy on Russian paraphrase classification.

desk verdict The paper's headline measure is unnormalized as printed—identical groups score 3, not 1—so the empirical thresholds and accuracies built on it don't support the claims. read the letter →

arxiv 1908.09341 v2 pith:DWHZ3XE4 submitted 2019-08-25 cs.LG cs.CLstat.ML

Artem Artemov , Boris Alekseev This is my paper · ORCID

classification cs.LGcs.CLstat.ML

keywords NeuralNetworksMachineLearningNLPParaphrasingTextSimilarityVectorRepresentationsentenceorthogonalprojection

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that two groups of vectors can be compared by the angle each vector makes with the linear span of the other group, rather than by averaging each group into a single vector or by computing all pairwise cosines. The proposed text-proximity measure symmetrizes those angles and, on the Russian paraphrase task, is reported to reach 0.81 accuracy and 0.5169 F-measure, better in accuracy than the trained semantic-cores baseline it compares against and the published competition results it cites. If the claim is right, a parameter-free geometric score can serve as a general set-to-set similarity for any object that has vector representations, at a cost of only $n+m$ projection computations instead of $n\times m$ pairwise comparisons. The paper also claims the measure preserves semantic characteristics of individual words and can be combined with syntactic or morphological vector components.

What carries the argument

The load-bearing object is the projection cosine: $\cos(b,A)=\sqrt{b\cdot A^T(A\cdot A^T)^{-1}A\cdot b^T}/|b|$. It measures the cosine of the angle between a vector $b$ and its orthogonal projection onto the linear span, which the paper calls a plane, formed by the rows of $A$. The group-to-group similarity is the symmetrized average $\mathrm{SIM}(A,B)=\left(\sum_i \cos(b_i,A)+\sum_j \cos(a_j,B)\right)/2$, computed in $n+m$ projection evaluations rather than $n\times m$ pairwise vector cosines. This formula only works when $A\cdot A^T$ is invertible, meaning the group's vectors must be linearly independent.

What would settle it

Take a sentence pair from the paraphrase task in which one sentence has more words than the embedding dimension or contains repeated words, so the rows of $A$ are linearly dependent; the formula in the paper is then undefined as written. Tracking how often this happens in the reported test set, and what accuracy a pseudoinverse or regularized variant achieves, would directly show whether the 0.81 result generalizes to ordinary text.

Watch

Extended reading notes

Core claim

The paper's central proposal is to define the cosine of the angle between one vector $b$ and a group of vectors $A$ as the ratio of the length of the orthogonal projection of $b$ onto the subspace spanned by $A$ to the length of $b$: $\cos(b,A)=\sqrt{b\cdot A^T(A\cdot A^T)^{-1}A\cdot b^T}/|b|$. The proximity of two groups $A$ and $B$ is then the symmetrized average of $\cos(b_i,A)$ over all $b_i$ in $B$ and $\cos(a_j,B)$ over all $a_j$ in $A$. The authors argue that this measure preserves the contribution of each word's vector rather than collapsing a sentence into one aggregate, that it is invariant under adding or removing vectors linearly dependent on the group's basis, and that it can be combined with extra vector components such as syntactic or morphological features. They report that on a Russian paraphrase corpus the method achieves 0.81 accuracy and 0.5169 F-measure for two-class classification, outperforming the semantic-cores method on the same data.

Load-bearing premise

The whole measure is only defined when the word vectors in each group are linearly independent, because the formula requires inverting $A A^T$; real sentences often violate this, and the paper leaves that singular case to future work.

Editorial extensions

If this is right

Set-to-set similarity becomes a parameter-free geometric computation: no trained classifier is needed, only embeddings and orthogonal projections.
The score can be merged with any kind of vector feature, including syntactic and morphological annotations, without changing the formula.
Because the cosine is invariant under adding vectors linearly dependent on the group's basis, redundant words do not tilt the score toward mere repetition.
The cost structure makes the method practical for large collections, since the matrix $A^T(A A^T)^{-1}A$ for a fixed group can be reused across many query vectors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial extension: if singular groups are handled by a pseudoinverse, the measure likely becomes a canonical angle between subspaces; comparing it with the sum of squared principal angles between the two spans would place it in standard subspace geometry.
Editorial extension: the reported comparison with competition results uses different protocols, so a direct head-to-head run on the same 1000-pair test set would be a straightforward way to confirm the claimed advantage over trained systems.
Editorial extension: the method measures content coverage rather than word order, so combining it with a small order-sensitive feature, such as positional weighting, is a natural testable variant.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the core projection-cosine derivation is standard and correct, but the paper's symmetrized formula divides by 2 instead of by group sizes, so SIM(A,A)=3 for two copies of a 3-vector identity group. The stress-test is right. That single issue guts the empirical section, because the thresholds a and b in Eq. (14) and all Table 2 numbers are fit to that particular scale. If the code silently normalized by group size, then the evaluated method is not the formula in the paper; if it didn't, the thresholds are meaningless. Either way the headline result is unsupported.

What's actually here: the vector-to-group cosine in Eq. (13) is just the standard cosine between a vector and its orthogonal projection onto a subspace, written as sqrt(b A^T(AA^T)^{-1} A b^T)/|b|. The derivation is correct for invertible AA^T, and the paper deserves credit for presenting it cleanly and for noting the O(n+m) complexity versus O(nm) pairwise cosines. The comparison against their own semantic-cores method on the same 70/30 split is at least honest, though the held-out set is small (about 570 pairs) and there are no error bars.

The soft spots beyond normalization: (1) The abstract and intro claim the method is sensitive to word order, but projection onto a span is permutation-invariant, so it cannot distinguish word order any better than a bag-of-words method. That undercuts the stated motivation. (2) The external comparison against 'competition results' uses different training and test data, so the 0.81 vs 0.59 accuracy claim is not a valid comparison. (3) The singular-matrix caveat is acknowledged, but the normalization bug is more basic—it fails on well-conditioned equal-size inputs. (4) The paper cites its own prior work for the baseline, which is fine, but the 'Alekseev's method' naming without citing the underlying linear algebra is a bit cheeky.

Who this is for: someone looking for a simple set-to-set similarity for sentence embeddings might glance at it, but in the current form the math doesn't hold up where it counts. The idea is salvageable—normalize properly, rerun, and make the word-order claim honest—but the preprint as written is not a reliable research contribution.

Recommendation: reject in current form. If the authors fix the normalization and re-report results, it could be worth a referee; as it stands, a desk reject with a clear explanation of the normalization error would be appropriate.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes a proximity measure between two sets of vectors. For a vector b and a matrix A, it defines cos(b, A) as the Euclidean length of the orthogonal projection of b onto the row space of A divided by |b|, as given in Eq. (13). For two groups A and B, it then proposes a symmetrized average of vector-to-group cosines. This measure is applied to Russian paraphrase classification, where thresholds a and b are fit on a 70/30 split and results are compared with the authors' semantic-cores method and with a public competition leaderboard. The paper reports accuracy 0.81 and F1 0.5169 for Alekseev's method.

Significance. The core projection-cosine formula is geometrically natural, and the algebraic derivation of Eq. (13) is correct under the stated invertibility condition. The method is also essentially parameter-free apart from two classification thresholds, and the authors are explicit that singular matrices are deferred to future work. If the measure were correctly defined and carefully evaluated, a simple vector-group similarity of this kind could be useful for text-comparison tasks. However, the symmetrization formula as printed is not normalized, and the experimental evaluation has serious flaws, so the paper's main claims are not currently supported.

major comments (4)

[Section 2.1, definition of SIM(A,B)] The symmetrization formula immediately after Eq. (13) is not a normalized similarity. As printed, SIM(A,B) = (Σ_i cos(b_i,A) + Σ_j cos(a_j,B)) / 2 divides only by 2, not by the group sizes |B| and |A|. For A = B = I_3 (the 3×3 identity matrix), every cosine term is 1 and the formula gives SIM = 3, not 1. This violates the paper's own requirement in Section 1 that the proximity coefficient lie in [0,1] and equal 1 for a full match. Consequently, the thresholds a and b in Eq. (14) and all entries in Table 2 are calibrated to an unnormalized scale; if the implementation used a properly normalized version, then the evaluated method differs from the formula presented. Either way, the headline empirical results are unsupported and the definition must be corrected.
[Section 2.1 and Conclusion (singular matrices)] Eq. (13) requires the square matrix A·A^T to be invertible, which holds only when the rows of A are linearly independent. Real sentence groups frequently have linearly dependent word vectors, or more words than embedding dimensions, making A·A^T singular. The conclusion explicitly defers this case to future work, but Section 3.2 does not report how singular matrices were handled in the experiments, what fraction of inputs were singular, or whether a pseudoinverse or regularization was used. Without this information, the method as defined is inapplicable to many real inputs and the experiments cannot be reproduced.
[Section 3.2 and Table 2 (evaluation validity)] The comparison with public competition results is not valid. The cited competition used 7,000 training pairs and 1,000 test pairs, while the reported 0.81 accuracy is obtained on a 30% holdout (roughly 573 records) of 1,913 records, with thresholds optimized on the training part. No confidence intervals, error bars, or significance tests are provided, and the two-class and three-class results in Table 2 vary widely across adjacent rows. The statement that Alekseev's method scored 0.81 accuracy versus the competition's 0.59 is therefore not an apples-to-apples comparison and does not support the paper's central claim.
[Section 3.2 (reproducibility)] The experimental section does not specify the pre-trained word embeddings, the exact composition of the semantic and grammatical feature vectors, or the precise construction of the 1,913-record dataset. Since the behavior of the proposed measure depends entirely on the vector representations used, the results cannot be replicated or meaningfully compared with other methods without these details.

minor comments (5)

[Abstract and Section 1] The abstract claims the measure 'does not lose any semantic characteristics and takes account of the words' traits,' but no formal definition or empirical evidence is given for this statement; it should be either substantiated or removed.
[Section 1, Eq. (1)] The matrix notation with F, C, and M is confusing and is not used later in the paper; consider removing it or clearly connecting it to the word-vector group formulation.
[Section 2.1] The sentence 'This will no longer require nm/2 calculations, but only m' is unclear, especially since the symmetrized version is state to require n+m calculations; please clarify the intended comparison.
[Table 2] The column heading 'Bound Evaluation' is ambiguous, and it is not explained how the lower and upper thresholds a and b are optimized or whether the reported accuracies are on the training or validation set.
[References] Reference [5] is a blog post; for a survey of text similarity methods, the authors should cite a peer-reviewed primary source, e.g., [3] and [4], and use those consistently.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the proximity formula is derived from elementary projection geometry without fitted parameters, and the reported accuracy comes from held-out data.

full rationale

The paper's derivation of cos(b,A) in Eqs. (5)-(13) is a standard least-squares projection: the coefficient vector lambda is obtained from lambda A A^T = b A^T, giving lambda = b A^T (A A^T)^-1 and projection p = lambda A; the cosine is then |p|/|b|. This is unparameterized linear algebra and does not assume the result it claims to predict. The group similarity SIM(A,B) is built from these per-vector cosines by symmetrizing the two directional averages; it contains no learned constants. The only fitted quantities in the evaluation are the clipping thresholds a and b in Eq. (14), and those are fitted on a 70% training split and applied to the remaining 30% (Section 3.2), so the reported 0.81 accuracy is a held-out number rather than a re-statement of the training fit. The two self-citations ([1] and [2]) describe the semantic-cores baseline and are not used to derive Alekseev's method; even if those citations were removed, the SIM formula and its evaluation stand on the paper's own equations. The acknowledged singularity limitation of A A^T is a correctness/robustness gap, not a circularity, and the unnormalized printed form of SIM(A,B) is a potential implementation inconsistency but does not make the derivation circular. No load-bearing step reduces to its own input.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central formula itself has no fitted parameters. The only declared fitted values are the classification thresholds a and b. The method additionally assumes invertibility of A·A^T and assumes that spans of proprietary embeddings are meaningful semantic objects, both of which are load-bearing and unverified.

free parameters (1)

classification thresholds a and b = a=0.47, b=0.96 for the 3-class model; other values in Table 2
These thresholds convert the proximity score into a paraphrase class and are chosen by maximizing precision on the 70% training split; every reported accuracy and F1 depends on them.

assumptions (3)

domain assumption For every compared group A, the matrix A·A^T is invertible.
Eq. (9) and Eq. (13) require this; the Conclusion defers the singular case to future work. In text data, word vectors are often linearly dependent or exceed the embedding dimension.
domain assumption The linear span of word vectors is a meaningful representation of a sentence.
The whole method compares a vector with the plane generated by the other group; no evidence is given that this subspace captures sentence meaning or ordering.
domain assumption The BRAIN2NLP / Big Semantic Model embeddings carry valid semantic and grammatical information.
Sections 2.2 and 3.2 rely on this proprietary system without specifying its architecture, training data, or vector details.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Method for Estimating the Proximity of Vector Representation Groups in Multidimensional Space. On the Example of the Paraphrase Task." pith.science (2026). https://pith.science/paper/DWHZ3XE4

@misc{pith2026190809341,
  author       = {Pith},
  title        = {Pith review of: A Method for Estimating the Proximity of Vector Representation Groups in Multidimensional Space. On the Example of the Paraphrase Task},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DWHZ3XE4}},
  note         = {Machine review of arXiv:1908.09341}
}

read the original abstract

The following paper presents a method of comparing two sets of vectors. The method can be applied in all tasks, where it is necessary to measure the closeness of two objects presented as sets of vectors. It may be applicable when we compare the meanings of two sentences as part of the problem of paraphrasing. This is the problem of measuring semantic similarity of two sentences (group of words). The existing methods are not sensible for the word order or syntactic connections in the considered sentences. The method appears to be advantageous because it neither presents a group of words as one scalar value, nor does it try to show the closeness through an aggregation vector, which is mean for the set of vectors. Instead of that we measure the cosine of the angle as the mean for the first group vectors projections (the context) on one side and each vector of the second group on the other side. The similarity of two sentences defined by these means does not lose any semantic characteristics and takes account of the words traits. The method was verified on the comparison of sentence pairs in Russian.

Figures

Figures reproduced from arXiv: 1908.09341 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

8 extracted references · 6 canonical work pages

[1]

The Training of Neuromodels for Machine Comprehension of Text. Brain2Text Algorithm

A. Artemov, A. Sergeev, I. Khasenevich, A. Yuzhakov, M. Chugunov. The Training of Neuromodels for Machine Comprehension of Text. BRAIN2Text Algorithm. 2018. https://arxiv.org/abs/1804.00551

work page Pith review arXiv 2018
[2]

Neural Network-based Object Classification by Known and Unknown Features (Based on Text Queries)

A.Artemov, I.Bolokhov, D.Kem, I.Khasenevich. Neural Network-based Object Classiﬁcation by Known and Unknown Features (Based on Text Queries). 2019. https://arxiv.org/abs/1906.00800

work page Pith review arXiv 2019
[3]

A survey of text similarity approaches,

W. H. Gomaa and A. A. Fahmy, “A survey of text similarity approaches,” Int. J. Comput. Appl., vol. 68, no. 13, 2013, doi: https://doi.org/10.5120/11638-7118. 7 A PREPRINT - AUGUST 30, 2019

doi:10.5120/11638-7118 2013
[4]

, Corley C

Mihalcea R. , Corley C. & Strapparava C. (2006). Corpus-based and Knowledge-based Measures of Text Semantic Similarity. In Proceedings, The Twenty-First National Conference on Artiﬁcial Intelligence and the Eighteenth Innovative Applications of Artiﬁcial Intelligence Conference, July 16-20, 2006, Boston, Massachusetts, USA

work page 2006
[5]

A. Sieg. Text Similarities : Estimate the degree of similarity between two texts. July, 2018. https://medium.com/@adriensieg/text-similarities-da019229c894

work page 2018
[6]

Fast-join: An efﬁcient method for fuzzy token matching based string sim- ilarity join,

J. Wang, G. Li, and J. Fe, “Fast-join: An efﬁcient method for fuzzy token matching based string sim- ilarity join,” in 2011 IEEE 27th International Conference on Data Engineering, 2011, pp. 458–469, doi: https://doi.org/10.1109/ICDE.2011.5767865

arXiv 2011
[7]

Wang S., Wang W., Zhuang Y . & Fei X. (2015). An ontology evolution method based on folksonomy. In Journal of Applied Research and Technology. JART

work page 2015
[8]

Semantic data mapping technology to solve semantic data problem on heterogeneity aspect,

A. Yunianta, O. M. Barukab, N. Yusof, N. Dengen, H. Haviluddin, and M. S. Othman, “Semantic data mapping technology to solve semantic data problem on heterogeneity aspect,” Int. J. Adv. Intell. Informatics, vol. 3, no. 3, pp. 161–172, Dec. 2017, doi: https://doi.org/10.26555/ijain.v3i3.131. 8

work page doi:10.26555/ijain.v3i3.131 2017

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

The Training of Neuromodels for Machine Comprehension of Text. Brain2Text Algorithm

A. Artemov, A. Sergeev, I. Khasenevich, A. Yuzhakov, M. Chugunov. The Training of Neuromodels for Machine Comprehension of Text. BRAIN2Text Algorithm. 2018. https://arxiv.org/abs/1804.00551

work page Pith review arXiv 2018

[2] [2]

Neural Network-based Object Classification by Known and Unknown Features (Based on Text Queries)

A.Artemov, I.Bolokhov, D.Kem, I.Khasenevich. Neural Network-based Object Classiﬁcation by Known and Unknown Features (Based on Text Queries). 2019. https://arxiv.org/abs/1906.00800

work page Pith review arXiv 2019

[3] [3]

A survey of text similarity approaches,

W. H. Gomaa and A. A. Fahmy, “A survey of text similarity approaches,” Int. J. Comput. Appl., vol. 68, no. 13, 2013, doi: https://doi.org/10.5120/11638-7118. 7 A PREPRINT - AUGUST 30, 2019

doi:10.5120/11638-7118 2013

[4] [4]

, Corley C

Mihalcea R. , Corley C. & Strapparava C. (2006). Corpus-based and Knowledge-based Measures of Text Semantic Similarity. In Proceedings, The Twenty-First National Conference on Artiﬁcial Intelligence and the Eighteenth Innovative Applications of Artiﬁcial Intelligence Conference, July 16-20, 2006, Boston, Massachusetts, USA

work page 2006

[5] [5]

A. Sieg. Text Similarities : Estimate the degree of similarity between two texts. July, 2018. https://medium.com/@adriensieg/text-similarities-da019229c894

work page 2018

[6] [6]

Fast-join: An efﬁcient method for fuzzy token matching based string sim- ilarity join,

J. Wang, G. Li, and J. Fe, “Fast-join: An efﬁcient method for fuzzy token matching based string sim- ilarity join,” in 2011 IEEE 27th International Conference on Data Engineering, 2011, pp. 458–469, doi: https://doi.org/10.1109/ICDE.2011.5767865

arXiv 2011

[7] [7]

Wang S., Wang W., Zhuang Y . & Fei X. (2015). An ontology evolution method based on folksonomy. In Journal of Applied Research and Technology. JART

work page 2015

[8] [8]

Semantic data mapping technology to solve semantic data problem on heterogeneity aspect,

A. Yunianta, O. M. Barukab, N. Yusof, N. Dengen, H. Haviluddin, and M. S. Othman, “Semantic data mapping technology to solve semantic data problem on heterogeneity aspect,” Int. J. Adv. Intell. Informatics, vol. 3, no. 3, pp. 161–172, Dec. 2017, doi: https://doi.org/10.26555/ijain.v3i3.131. 8

work page doi:10.26555/ijain.v3i3.131 2017