{"id":"52de01eb-045e-4fc6-93aa-c772b5d18903","arxiv_id":"2505.03532","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"The authors define a joint generalized cosine similarity from the Gram determinant and use it in a contrastive loss called GHA to align three or more modalities in one step.","lead":"The paper introduces a way to measure similarity among any number of feature vectors at once, using an angle derived from a matrix determinant, and turns it into a contrastive learning loss. It promises a scalable alternative to pairwise losses for multi-modal models, but the experiments do not consistently support the claimed advantage.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"JGCS degenerates for n>D: Eq. (5) yields cosΘ=1 for every n-tuple once the number of vectors exceeds the embedding dimension, so the advertised 'arbitrary number of vectors' claim fails precisely in the high-modality regime the paper motivates.","rationale":"I read the paper in good faith. The Gram-determinant construction itself is standard and the GHA loss is a reasonable InfoNCE-style adaptation. The paper deserves credit for the determinant identity and for naming a concrete n-way geometric quantity. However, the central advertised property—'arbitrary number of vectors'—is internally contradicted by Eq. (5) and the paper's own n≤D condition in the extremal property. This is a correctness issue, not a disagreement with consensus: it is a mathematical consequence of rank. The reader's weakest_assumption identified exactly this, and I agree. I did not base the objection on the mixed Derm7pt results or missing code; those affect the empirical superiority claim, but the n>D degeneracy undermines the definition itself. The paper already received CONDITIONAL; my stress-test does not move that verdict, since the concern is already on record. If the authors add an explicit n≤D caveat and describe a valid strategy for n>D (or restrict the claim), the mathematical contribution can stand.","tokens_in":12152,"tokens_out":7059,"duration_ms":70130,"concrete_test":"Construct a 2D example with n=3: M1 = [(1,0); (0,1); (1,1)/√2] and M2 = [(1,0); (0,1); (1,-1)/√2]. Compute cosΘ via Eq. (5) for both; because rank(M)≤2<3, det(MM^T)=0 in both cases, so cosΘ=1 identically. If the implementation returns 1 for all such triples, the n>D degeneracy is confirmed and the 'arbitrary number of vectors' claim is false. A second check: run the GHA loss (Eq. 7) with D=128, n=129, random Gaussian features; the loss should be constant across training steps, showing no learning signal.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing claim is that JGCS handles an arbitrary number of vectors (Abstract, §I). Inspection of Eq. (5) shows this is false as stated. For M ∈ R^{n×D}, the Gram matrix MM^T is n×n of rank at most D. Whenever n > D, det(MM^T)=0, so Θ=0 and cosΘ=1 for every n-tuple. Thus the similarity carries no information: all positive and negative tuples in the GHA loss (Eq. 7) have identical logits exp(1/τ), so the contrastive loss is a constant and no gradient signal remains. The paper itself restricts to n≤D only in the extremal property in §IV-A, not in the abstract, the definition, or the 'arbitrary number' language. This is not an exotic corner case: the introduction motivates the method with 8-, 10-, and 12-modal datasets, and if those embeddings are produced at dimension below the modality count (e.g., D=8 for 12 modalities), the method collapses exactly where it was promised to scale. The fix is either to state n≤D prominently and provide a fallback for n>D (e.g., projection to a D'-dimensional subspace), or to withdraw the 'arbitrary' claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a multi-vector similarity measure, the Joint Generalized Cosine Similarity (JGCS), defined through the Gram-hypervolume angle between n feature vectors, together with a contrastive loss (GHA Loss) built on it. The authors claim this is the first similarity method capable of handling an arbitrary number of vectors, that the associated loss outperforms pairwise contrastive baselines for n-modal semantic alignment, and that it is noise-robust, computationally efficient, and scalable. The method is derived analytically in Section IV-A, and experiments are reported on the Derm7pt three-modal dataset and on simulated data in Section V.","tokens_in":1575,"tokens_out":1911,"duration_ms":56856,"significance":"If the method is restricted to the regime n ≤ D and the ReLU non-negativity constraint is accepted, the construction is a mathematically valid generalization of cosine similarity to n vectors: the Gram-determinant angle is rotation-invariant, permutation-symmetric, and reduces to the planar angle for n = 2. The computational-efficiency comparison in Figure 6 is a useful concrete observation, and the idea of replacing pairwise losses with a single n-way contrastive term is well motivated. However, the paper's headline claim of handling an arbitrary number of vectors is not correct as stated, because the Gram determinant vanishes whenever the number of vectors exceeds the embedding dimension; the empirical evidence for superiority over pairwise baselines is also incomplete, as Table I contains only single runs without error bars and the simulated contrastive experiment trains and evaluates on the same 4,000 samples. The central mathematical identity is sound, but the paper needs substantial clarification and additional experiments before its claims can be accepted.","major_comments":[{"comment":"The central claim that JGCS can handle an arbitrary number of vectors is false for n > D. For M ∈ R^{n×D}, the Gram matrix MM^T has rank at most D, so when n > D, det(MM^T) = 0, Eq. (5) gives Θ = 0, and cos Θ = 1 for every n-tuple. Consequently, in Eq. (7) all positive and negative logits equal exp(1/τ), the GHA loss is constant, and no gradient signal reaches the encoders. The paper mentions the condition n ≤ D only in the extremal property in §IV-A, not as a general constraint on the method, even though the introduction motivates the approach with 8-, 10-, and 12-modal datasets. Please either state the n ≤ D limitation prominently and provide a fallback for n > D (for example, projecting the n vectors into a subspace of dimension D' < n before computing the determinant), or remove the 'arbitrary number of vectors' claim from the abstract, introduction, and conclusion.","section":"§I, §IV-A, Eq. (5)"},{"comment":"Eq. (1) writes V = |det(M)| = sqrt(det(MM^T)) for a non-square matrix M ∈ R^{n×D}, but det(M) is undefined when M is not square. The correct expression is V = sqrt(det(MM^T)). This is not merely a typographical issue, because the erroneous equality obscures the rank condition behind the n > D collapse and should be corrected for mathematical rigor.","section":"§IV-A, Eq. (1)"},{"comment":"The method requires all feature vectors to be nonnegative, which the paper imposes by adding a ReLU activation to the final layer. This is an architectural constraint whose cost is not analyzed: it restricts the feasible representation space, prevents the representation of negative semantic directions, and can introduce dead ReLU units during training. No experiment isolates the effect of this constraint or compares against alternatives such as using squared norms or orientation-corrected angles. Since the non-negativity assumption is a prerequisite for the angle to be meaningful, the paper should either justify it more carefully or ablate it empirically.","section":"§IV-A, final paragraph; §IV-C, Eqs. (10)–(12)"},{"comment":"The empirical claim that GHA Loss achieves superior performance is not fully supported. In Table I, the GHA setting loses to the Dual setting in several configurations (for example, Res101+Gate: GHA 59.90 vs. Dual 63.02; Res50+Gate: GHA 63.54 vs. Dual 65.89), and all reported numbers appear to come from single runs without error bars or significance tests, so the statement in §I that the results 'consistently demonstrate the high efficiency of the proposed method' is too strong. In §V-C, the simulated contrastive experiment trains the encoder on the same 4,000 samples used for evaluation, which demonstrates training-set alignment but not generalization. Please report repeated runs with standard deviations or confidence intervals, perform a meaningful train/test split, and temper the claims to match the observed differences.","section":"§V-B, Table I; §V-C"}],"minor_comments":[{"comment":"The sentence 'The convergence analysis of the model is depicted in Figure 1' appears to reference the wrong figure; the relevant convergence plots are shown in Figure 5, while Figure 1 depicts the method comparison diagram.","section":"§V-B"},{"comment":"The permutation-symmetry property states that swapping feature vectors 'only alters the sign of the Gram matrix determinant,' but the Gram determinant det(MM^T) is invariant under row permutations because the permutation contributes a squared sign; only det(M) changes sign. The conclusion that Θ is unaffected is correct, but the stated reason is not.","section":"§IV-A, item 3"},{"comment":"The sentence 'Models such as OSCAR and ViLBERT place a greater emphasis on modality interaction...' is repeated verbatim within the same subsection; one occurrence should be removed.","section":"§II-A"},{"comment":"The sentence 'Contrastive learning was first applied in the field of computer science and subsequently extended to the domain of natural language analysis' is imprecise; contrastive ideas have deep roots in linguistics and psycholinguistics, and the sentence should be rephrased to describe the machine-learning history more carefully.","section":"§II-B"},{"comment":"The hyperparameters τ = 0.005 and λ = 1 are reported, but no sensitivity analysis or discussion of how these values were chosen is provided; this is especially relevant because temperature strongly affects contrastive learning behavior.","section":"§V-A"}],"recommendation":"major_revision","confidential_remarks":"The n > D collapse is the load-bearing issue: the paper's main advertised capability does not hold in the high-modality regime it uses for motivation. The fix is straightforward in principle (state and handle the rank constraint), and the empirical section needs repeated runs and a held-out evaluation. With these changes the paper could be publishable, but the current version overclaims in both the methodological and experimental dimensions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is legitimate and worth building on: a Gram-determinant angle gives a single n-way similarity that reduces to cosine similarity for n=2 and to the triangle area in [37] for n=3, and the GHA loss is a clean way to do n-modal contrastive alignment without pairwise aggregation. The math is mostly correct, and the invariance properties are nice. If you need a joint n-tuple contrastive objective, this is a reasonable template.\n\nThe problems are real but concentrated. The most serious is the n>D degeneracy. When the number of vectors exceeds the embedding dimension, det(MM^T) is identically zero, so cosΘ collapses to 1 for every tuple. The loss then has no gradient signal at all. The paper advertises 'arbitrary number of vectors' but only qualifies n≤D in one extremal property, buried in Section IV. The introduction motivates the method with 8-, 10-, and 12-modal datasets; if any of those use embeddings below the modality count, the method silently fails. This needs to be stated prominently and a fallback (e.g., subspace projection or penalizing the rank) provided.\n\nThere are also soft spots in presentation and evidence. Equation (1) writes |det(M)| for non-square M, which is undefined; only the Gram-determinant form makes sense. The ReLU positivity constraint is mentioned but its cost to embedding expressiveness is never discussed. The experimental section is the weakest part: single runs with no error bars, simulated alignment trained and evaluated on the same 4,000 samples (so it only shows memorization), and the Derm7pt results are mixed—GHA loses to Dual in several configurations. The claim of 'superior performance' doesn't hold uniformly.\n\nThe paper also undersells the relationship to [37], which is essentially the n=3 special case. A more explicit acknowledgment would keep the novelty honest.\n\nStill, the central construction is sound and genuinely useful as a building block. I'd send it to review—the method deserves exposure and critique—but it needs a major revision: fix the n>D issue, add error bars, test on a real >3-modal dataset, and release code. As is, I wouldn't cite it for the empirical claims, but I'd keep it in mind for the loss formulation.","headline":"A sound but over-scoped determinant-based n-way similarity with a useful contrastive loss; the 'arbitrary number of vectors' claim degenerates when n exceeds the embedding dimension, and the experiments are too thin to support the superiority claims.","tokens_in":12941,"tokens_out":2736,"would_cite":false,"duration_ms":27565,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Gram-determinant angle yields a single similarity score for any number of modality vectors and powers the GHA contrastive loss.","keywords":["Joint Generalized Cosine Similarity","GHA Loss","multi-modal contrastive learning","semantic alignment","Gram determinant","n-modal similarity","Derm7pt"],"falsifier":"Run the GHA contrastive loss on synthetic n-tuples with $n=D+1$ and $D$-dimensional embeddings: every positive and negative tuple satisfies $\\det(MM^T)=0$, so $\\cos\\Theta=1$ for all of them and the loss cannot separate positives from negatives. That directly tests whether the advertised 'arbitrary number of vectors' holds beyond the dimension.","tokens_in":11952,"feed_emoji":"📐","tokens_out":12349,"duration_ms":110799,"temperature":0.7,"pith_summary":"Most multi-modal contrastive learning is stuck with pairwise similarity: with three or more modalities, losses are computed between every pair and then summed. This paper proposes a single n-way similarity, the Joint Generalized Cosine Similarity (JGCS), defined as the cosine of the Gram hypervolume angle spanned by all modality vectors at once. It then builds a contrastive loss, GHA Loss, on this joint similarity and reports that on the three-modal Derm7pt dataset and on simulations it aligns modalities at least as well as pairwise baselines while running faster and tolerating added noise. If the method holds, multi-modal alignment with three, five, or twelve modalities no longer needs to be stitched together from pairwise terms.","feed_headline":"One angle, not pairs: a new n-way similarity for multi-modal learning","feed_subtitle":"A Gram-determinant angle and its GHA loss align n modalities in one step, cutting pairwise computation and noise.","key_machinery":"The load-bearing object is the Gram hypervolume angle (GHA): for n row vectors $f_i$ forming matrix $M$, the volume of the parallelepiped they span is $\\sqrt{\\det(MM^T)}$, and dividing that volume by the product of the vector lengths gives the sine of one generalized angle $\\Theta$. JGCS is defined as $\\cos\\Theta$, and the loss plugs it into an InfoNCE-style contrastive term plus an angular equilibrium regularizer that penalizes variance among pairwise cosine similarities so no modality collapses onto the plane of another. The construction is rotation-invariant and permutation-symmetric, degenerates to ordinary cosine similarity for $n=2$, and requires non-negative embeddings (a ReLU final layer) to break the sign symmetry inherited from the squared determinant.","core_discovery":"The central claim is that the Gram-determinant angle $\\Theta$ given by $\\Theta_{f_1,\\dots,f_n}=\\arcsin(\\sqrt{\\det(MM^T)}/\\prod_{i=1}^n\\|f_i\\|_2)$, with $M=[f_1,\\dots,f_n]^T$, is a valid and useful similarity for an n-tuple of vectors: $\\cos\\Theta=1$ for linearly dependent vectors and $\\cos\\Theta=0$ for pairwise orthogonal vectors. Replacing the pairwise cosine or dot product in a contrastive loss with $\\cos\\Theta$ yields the GHA Loss, which adds an angular equilibrium term that equalizes the pairwise cosine similarities inside each n-tuple. The paper claims this is the first similarity measurement that works for an arbitrary number of vectors, and its experiments show the joint objective outperforming or matching pairwise summed InfoNCE baselines across model configurations on Derm7pt while visibly clustering the three modalities of each sample in two-dimensional projections.","pith_inferences":["The paper advertises an arbitrary number of vectors, but the formula only discriminates when n does not exceed the embedding dimension D; at n > D every n-tuple has determinant zero and $\\cos\\Theta=1$, so a projection or higher-dimensional embedding is required before the method can be used on the 12-modal datasets cited in the introduction.","Replacing the ReLU positivity constraint with a reference axis or a signed angle would make JGCS usable in domains where non-negative embeddings are unnatural.","The angular equilibrium term is a diversity regularizer in disguise; it could be lifted out of contrastive learning into ensemble and feature-union problems where redundant signal is penalized."],"forward_implications":["Three or more modalities can be trained with one joint contrastive term instead of $\\binom{n}{2}$ pairwise terms, removing hand-chosen weights for aggregated losses.","GHA Loss computation grows roughly linearly with the number of modalities, so high-modality pretraining avoids the exponential pairwise blow-up measured in the paper.","The same $\\cos\\Theta$ can serve outside contrastive learning as a general n-way similarity for clustering, retrieval, or diversity scoring of feature sets.","Because the angle is invariant to rotations, the alignment objective no longer depends on arbitrary coordinate choices in the shared embedding space."],"supporting_citations":[{"why":"Supplies the InfoNCE contrastive formulation that GHA Loss generalizes by substituting the joint cosine similarity for a pairwise similarity.","marker":"[28]"},{"why":"Provides the three-modal Derm7pt dataset and the official train/validation/test split used for the real-data comparisons.","marker":"[38]"},{"why":"The closest prior non-pairwise joint contrastive loss, based on triangle area and limited to three modalities.","marker":"[37]"},{"why":"The two-modality contrastive pretraining approach that motivates the shared embedding space and represents the pairwise paradigm.","marker":"[20]"},{"why":"A three-modal contrastive framework built on pairwise InfoNCE aggregation, the main approach GHA Loss is designed to replace.","marker":"[32]"}],"fun_headline_variants":["Gram-angle similarity aligns any number of modalities in one shot","Beyond pairwise: a joint cosine for n-modal contrastive learning","First similarity for n vectors: JGCS makes multi-modal alignment simpler","JGCS: the first n-way similarity for contrastive learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method silently requires that the number of vectors not exceed the embedding dimension; if there are more vectors than dimensions, every n-tuple has the same Gram-determinant volume and the similarity carries no information.","fun_headline_variants_meta":{"raw":{"variants":["Gram-angle similarity aligns any number of modalities in one shot","Beyond pairwise: a joint cosine for n-modal contrastive learning","First similarity for n vectors: JGCS makes multi-modal alignment simpler","JGCS: the first n-way similarity for contrastive learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000625,"raw_usage":{"total_tokens":2912,"prompt_tokens":986,"completion_tokens":1926,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":602,"completion_tokens_details":{"reasoning_tokens":1855}},"tokens_in":602,"tokens_out":1926,"duration_ms":14203,"temperature":1.0,"reasoning_tokens":1855,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:50:11.534213+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the GHA contrastive loss on synthetic n-tuples with $n=D+1$ and $D$-dimensional embeddings: every positive and negative tuple satisfies $\\det(MM^T)=0$, so $\\cos\\Theta=1$ for all of them and the loss cannot separate positives from negatives. That directly tests whether the advertised 'arbitrary number of vectors' holds beyond the dimension.","supporting_citations":[{"cited_title":"Triangular Contrastive Learning on Molecular Graphs","cited_arxiv_id":"2205.13279","evidence_quote":"Provides the three-modal Derm7pt dataset and the official train/validation/test split used for the real-data comparisons."},{"cited_title":"Con- trastive learning of medical visual representations from paired images and text,","cited_arxiv_id":null,"evidence_quote":"The closest prior non-pairwise joint contrastive loss, based on triangle area and limited to three modalities."},{"cited_title":"Semantic alignment network for multi-modal emotion recognition,","cited_arxiv_id":null,"evidence_quote":"A three-modal contrastive framework built on pairwise InfoNCE aggregation, the main approach GHA Loss is designed to replace."}],"review_version":1}