{"id":"0ffd768a-b43c-42d6-ba6f-f942a01f00e0","arxiv_id":"2504.16798","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"M2M-AlignNet fuses 3D brain structure, 4D brain activity, and tabular biomarkers with a contrastive alignment loss and reports improved AD risk classification on EHBS, ADNI, and HCP.","lead":"Researchers combine brain scans that show structure (sMRI) with scans that show activity over time (fMRI) plus tabular biomarkers in one deep-learning model to detect early Alzheimer's risk. The paper claims a new contrastive alignment and co-attention fusion improves diagnostic accuracy compared with several baselines.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 6 does not implement the claimed multi-patch-to-multi-patch alignment: the numerator contains only one positive per row and the self-weighting multiplies only negative-pair terms, so the central contribution reduces to a weighted InfoNCE.","rationale":"The reader's weakest assumption identifies the same internal inconsistency that I find most load-bearing: Eq. 6 does not implement multi-patch-to-multi-patch alignment. This is not a disagreement with an external consensus; it is a mismatch between the paper's central claim and its own equations. The numerator selects one positive per row, and the adaptive weight appears only in the denominator, so the loss is a weighted InfoNCE with diagonal positives. The paper's own text confirms this by saying the constraint is loosened through weighting of negative pairs. The off-diagonal form also has the wrong exclusion index (k≠i instead of k≠j), making any claimed extension to multiple positives internally inconsistent. Secondary issues such as missing significance tests, missing hyperparameters, and lack of released code are real but do not need to be reached; they weaken confidence in Table I rather than refute the mechanism. Because the alignment loss is the stated core contribution, the REJECT verdict is appropriate and unchanged.","tokens_in":11128,"tokens_out":10821,"duration_ms":103683,"concrete_test":"Re-derive the gradient of Eq. 6 with respect to each logit S_{t,(i,j)} for a fixed fMRI row i. Show that the attractive term comes only from the single positive numerator exp(S_{t,(i,j)}/τ), while all w_{t,(i,k)} terms occur only in the denominator, so for a row there is no positive pull toward more than one sMRI column. Then check the off-diagonal case: with i≠j, the denominator indicator k≠i retains k=j, so the positive pair is also penalized; a many-to-many formulation would need indicators k≠j or an explicit set of positive pairs. This two-line derivation settles whether Eq. 6 can implement the claimed alignment.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central novelty is the M2M contrastive loss (Sec. III-D, Fig. 2) claimed to align multiple fMRI patches with multiple sMRI patches without one-to-one constraints. Eq. 6 does not support that claim. For a fixed fMRI row i, the loss is −log( exp(S_{t,(i,j)}/τ) / Σ_{k=1}^C w_{t,(i,k)} 1_{k≠i} exp(S_{t,(i,k)}/τ) ). The numerator is exactly one positive pair (i,j); the self-weight w appears only in the denominator, on terms with k≠i. Consequently, for each row there is an attractive gradient toward only one sMRI column; every other column is merely down-weighted as a negative, never pulled closer. The paper's own justification (\"we loose the constraint by introducing adaptive self-weighting for the negative pairs\") confirms that this is a relaxation of negative repulsion, not an addition of positive correspondences. The off-diagonal case is internally inconsistent as well: if i≠j, the indicator k≠i leaves the positive column j inside the denominator, so the same pair is simultaneously attracted and repelled; a correct off-diagonal formulation would exclude k≠j. Eq. 5-6 therefore compute an ordinary weighted InfoNCE with diagonal positives, and the central claim of geometry-weighted many-to-many alignment is not established by the equations. The good classification results in Table I may still come from the co-attention and bottleneck design, but the paper's stated core contribution is the alignment loss, so the central claim fails as written.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes M2M-AlignNet, a multimodal framework for Alzheimer's disease diagnosis that fuses 4D fMRI, 3D sMRI, and tabular data. The method uses a SwiFT 4D Transformer backbone, a latent-as-query co-attention fusion module with bottleneck refinement, and a contrastive alignment loss that the authors claim performs multi-patch-to-multi-patch (M2M) alignment between fMRI and sMRI patches via geometry-weighted self-weighting. Experiments are reported on the EHBS, ADNI, and HCP datasets, with comparisons to several multimodal baselines, ablated modules, and qualitative visualizations of discriminative brain regions and latent embeddings.","tokens_in":11399,"tokens_out":5940,"duration_ms":52153,"significance":"If the central claims were supported, the paper would address an underexplored and clinically relevant fusion setting, namely combining 4D spatiotemporal fMRI with 3D structural MRI for early AD diagnosis. The architecture-level ideas, including latent queries for modality-agnostic fusion and bottleneck-based refinement, are reasonable, and the multi-dataset evaluation is a strength. The paper also engages with a clinically meaningful question about function-structure correspondence. However, the central mechanistic claim about the M2M loss is not supported by the equations, and the experimental evidence lacks significance testing and contains high-variance ablation results. No code or reproducibility artifacts are provided, which further limits verification.","major_comments":[{"comment":"The loss in Eq. (6) does not implement multi-patch-to-multi-patch alignment. For a fixed fMRI row i, the numerator contains exactly one positive pair exp(S_{t,(i,j)}/tau), and the self-weight w_{t,(i,k)} multiplies only the negative-pair terms in the denominator (k != i). Thus each fMRI patch is attracted to exactly one sMRI patch, and the weighting merely down-weights some negatives. The text's claim that the method 'allows correspondence between a single patch from the fMRI embedding and multiple patches from the sMRI embedding, and vice versa' is not realized by the equation. Moreover, for any off-diagonal positive pair (i != j), the indicator 1_{k != i} does not exclude k = j, so the same pair is simultaneously attracted in the numerator and repelled in the denominator. This is a load-bearing error because the M2M loss is the paper's core contribution.","section":"Sec. III-D, Eq. (6)"},{"comment":"The 'geometry-weighted' and 'distance-adjusted' characterization is not supported. D(.) in Eq. (7) is a dot product, cosine similarity, KL divergence, JSD, or MMD computed on latent embeddings, not a spatial distance between brain regions. The weighting therefore reflects embedding similarity, not brain geometry. Additionally, the negative correlation function T is never defined, so the mapping from D to w is unspecified. This affects the framing of the method and the claimed novelty of geometry-aware alignment.","section":"Sec. III-D, Eq. (7)"},{"comment":"No statistical significance tests are reported, and several differences are within one standard deviation of the mean. For example, on EHBS PR-AUC the proposed method is 64.49 +/- 3.9 versus mmFormer at 64.06 +/- 5.8, and on HCP PR-AUC it is 97.59 +/- 1.2 versus mmFormer at 96.62 +/- 1.7. The phrase 'significantly outperforms' is not justified. Also, on EHBS accuracy the proposed method (78.01) is lower than SwiFT-EF (79.74), and this is not discussed. Paired tests across folds or bootstrap tests should be reported for all metrics.","section":"Table I"},{"comment":"The ablation table is difficult to interpret. The header lists modules, but the rows are not labeled with which module is removed, and several accuracy values have very large standard deviations (e.g., 54.36 +/- 31.8 and 83.72 +/- 11.2). The claim that removing spatial fusion or alignment causes the most substantial performance degradation needs a clearly labeled table and a statistical comparison; as presented, the evidence is inconclusive.","section":"Table II"}],"minor_comments":[{"comment":"The word 'correspondance' is misspelled as 'correspondence' in the abstract and the introduction; the typo should be corrected.","section":"Abstract and Introduction"},{"comment":"The section heading contains a typo: 'Proprocessing' should be 'Preprocessing'.","section":"Sec. III-A"},{"comment":"The function T in Eq. (7) is left unspecified. The authors should state explicitly what T is and how the weights w are normalized (for example, whether a softmax is applied over k).","section":"Sec. III-D"},{"comment":"Several hyperparameters are not reported, including the temperature tau, the relative weight of the M2M contrastive loss versus the classification loss, the number of latent co-attention queries, and the bottleneck dimensions. These are needed for reproducibility.","section":"Sec. IV-A"},{"comment":"The phrase 'to verify the reliability and interpretability of the proposed DRL' uses the undefined abbreviation 'DRL'; this should be replaced with the model name.","section":"Sec. IV-E"},{"comment":"The claim that distributions appearing nearly 'orthogonal' to each other indicate 'effective alignment' is conceptually confusing, since alignment in a shared space should decrease distributional distance. The interpretation should be clarified.","section":"Sec. IV-F and Fig. 5"}],"recommendation":"reject","confidential_remarks":"The central contribution is contradicted by the equations in Sec. III-D, and the experimental evidence does not establish significance. A straightforward revision path is not apparent because the loss would need to be redefined to actually implement many-to-many alignment and all experiments would need to be rerun. The architecture-level ideas may have some value, but as written the manuscript does not meet the bar for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The paper has real empirical content and the architecture is a reasonable engineering contribution. But the central conceptual claim—that Eq. 6 implements a multi-patch-to-multi-patch (M2M), geometry-weighted contrastive alignment—does not survive a close read of the equations. The numerator in Eq. 6 still contains exactly one positive pair per row, and the self-weight w only reweights negative-pair terms in the denominator. That is a weighted InfoNCE with diagonal positives, not many-to-many attraction. Worse, if you ever use an off-diagonal positive (i≠j), the indicator 1_{k≠i} keeps that positive in the denominator, so the same pair is both pulled and pushed. The text's own justification ('we loose the constraint by introducing adaptive self-weighting for the negative pairs') confirms the loss relaxes negative repulsion, not positive correspondence.\n\nWhat is new and good: applying a 4D Swin Transformer to both sMRI (with a dummy temporal axis) and fMRI, then fusing through latent-query co-attention and bottleneck refinement, is a sensible combination I have not seen exactly elsewhere. The evaluation spans three datasets—EHBS, ADNI, HCP—with ablations and an interpretability analysis. The authors also engage the clinical literature on function–structure coupling. That is a credible empirical package.\n\nSoft spots, in proportion. The classification gains over baselines are modest and some are within one standard deviation; there are no significance tests. Ablation Table II has accuracy standard deviations up to 31.8, so several component 'contributions' are not trustworthy. No code, data, or key hyperparameters (τ, loss weight) are released. The 'geometry-aware' label is also misleading: D in Eq. 7 is a dot product or divergence on embeddings, not a spatial distance. The t-SNE claim that aligned distributions are nearly 'orthogonal' is confusing—orthogonality is not alignment.\n\nThe citation pattern is fine; the self-citations to Khalilullah et al. and Wei et al. are relevant prior work.\n\nBottom line: as submitted, I would reject. The stated core contribution is not what the equations implement, and the empirical superiority is not rigorously established. But this is not a throwaway. The architecture and the idea of patch-level alignment for fMRI-sMRI have merit. A revision that fixes the loss (or honestly reframes it as weighted InfoNCE), adds significance testing, and releases code could change my mind.\n\nRecommendation: if this lands on your desk, do not desk-reject it. Send it to peer review with a referee who will check the math; the flaws are fixable and the empirical effort deserves a serious look.","headline":"Solid engineering, overstated novelty: the M2M loss in Eq. 6 is a weighted InfoNCE, not the many-to-many geometry-aware alignment the paper advertises.","tokens_in":12006,"tokens_out":3833,"would_cite":false,"duration_ms":34637,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"M2M-AlignNet claims that a geometry-aware, many-to-many patch contrastive loss plus a latent-query co-attention module fuses 4D fMRI and 3D sMRI to outperform existing multimodal methods for Alzheimer's diagnosis.","keywords":["Alzheimer's disease","multimodal fusion","fMRI","sMRI","contrastive learning","co-attention","latent alignment","M2M-AlignNet"],"falsifier":"Compute the gradient of Eq. 6 with respect to an off-diagonal pair: if off-diagonal pairs never receive a positive pull, the many-to-many claim is false. Then retrain with several off-diagonal pairs designated as positives and check whether the reported PR-AUC and ROC-AUC improve.","tokens_in":2206,"feed_emoji":"🧠","tokens_out":6833,"duration_ms":112938,"temperature":0.7,"pith_summary":"This paper tries to establish that 4D fMRI and 3D sMRI can be fused for early Alzheimer's diagnosis by aligning many patches across modalities in a shared latent space, rather than forcing one-to-one correspondences or letting one modality dominate. The proposed M2M-AlignNet couples a patch-wise contrastive loss with adaptive self-weighting of negative pairs to a co-attention module that uses learnable latent queries instead of a fixed modality-as-query design. On three imaging cohorts, it reports higher PR-AUC and ROC-AUC than comparison multimodal methods, with the largest gains on presymptomatic risk classification. If correct, it would give clinicians a practical way to exploit complementary structural and functional MRI signals for earlier detection.","feed_headline":"Patch-level fMRI-sMRI alignment lifts Alzheimer's detection","feed_subtitle":"Co-attention fusion plus a contrastive alignment beat existing multimodal baselines on three imaging cohorts.","key_machinery":"The central mechanism is the M2M contrastive loss in Eq. 6: a reweighted InfoNCE-style loss computed patch-wise over the fMRI-sMRI similarity matrix $S_t$. Each positive pair contributes a numerator $\\exp(S_{t,(i,j)}/\\tau)$, while the denominator sums over negative partners $k \\neq i$ with weights $w_{t,(i,k)} = T(D(L_f^{(t,i)}, L_s^k))$, where $D$ is a divergence or similarity measure and $T$ is a negative-correlation function. The claimed effect is that patches with similar semantics are penalized less as negatives, allowing one patch to align with several patches across modalities. The co-attention module uses a learnable latent query $J_{\\mathrm{sp}}$ initialized from a normal distribution, attending over concatenated modality keys and values, followed by modality-specific refinement with a down-up bottleneck.","core_discovery":"The paper's central claim is that a multi-patch-to-multi-patch (M2M) contrastive loss, with per-pair weights derived from the discrepancy between fMRI and sMRI latents, creates a geometry-aware alignment that improves multimodal Alzheimer's diagnosis. The authors argue this goes beyond CLIP-style pairwise alignment and beyond co-attention with a fixed modality hierarchy. The latent-as-query co-attention fuses sMRI, fMRI spatial, and fMRI temporal features through learnable query vectors, and a bottleneck refinement module suppresses redundancy. On the presymptomatic at-risk cohort, the full model reaches PR-AUC 64.49 and ROC-AUC 71.55 with sMRI and fMRI alone; adding tabular biomarker features raises PR-AUC to 67.66 and ROC-AUC to 73.46. Ablations show that removing the contrastive alignment or the spatial co-attention fusion causes the largest performance drops.","pith_inferences":["As written, Eq. 6 keeps one positive pair per row; the adaptive weight only down-weights negative-pair denominators. A true many-to-many positive alignment would require several sMRI patches to receive positive gradients from one fMRI patch, which the equations do not show.","The 'geometry' in 'geometry-aware' does not enter via spatial distances; $D$ is dot product, cosine similarity, KL divergence, JSD, or MMD on embeddings. Weighting by actual anatomical distance between patches would be a direct test of the geometric claim the name implies.","The paper notes the loss extends to more than two modalities by summing over modality pairs; a natural test is adding PET or DTI and checking whether the alignment gains persist or saturate.","A decisive experiment is to replace the self-weighting in Eq. 6 with a standard InfoNCE denominator and hold everything else fixed; if the reported performance does not drop, the M2M mechanism is not what drives the results."],"forward_implications":["If the central claim is correct, early Alzheimer's risk can be detected from resting fMRI and sMRI with PR-AUC 64.49 and ROC-AUC 71.55 on the presymptomatic at-risk cohort, beating all listed baselines on ROC-AUC.","Adding tabular biomarker features raises PR-AUC to 67.66 and ROC-AUC to 73.46, indicating that imaging and blood-based biomarkers can be combined within the same fusion architecture.","Ablating either the contrastive alignment or the spatial co-attention fusion causes the largest performance drops, so both modules are load-bearing for the reported gains.","The choice of discrepancy measure for the self-weighting is decisive: dot-product weighting gives the best results while Jensen-Shannon divergence performs worst.","The model is relatively lightweight (1.24M parameters) compared to the strongest baseline (35.82M parameters) while achieving higher ROC-AUC, which may matter for clinical deployment."],"supporting_citations":[{"why":"Supplies the InfoNCE formulation that the M2M contrastive loss is built on.","marker":"[Oord et al.(2018)]"},{"why":"Provides the contrastive-learning framework for maximizing agreement between positive pairs, which the M2M loss adapts.","marker":"[Chen et al.(2020)]"},{"why":"Defines the pairwise cross-modal alignment (CLIP) that the paper explicitly distinguishes from its multi-patch-to-multi-patch approach.","marker":"[Radford et al.(2021)]"},{"why":"Supplies the SwiFT 4D Swin Transformer backbone used to encode both fMRI and sMRI.","marker":"[Kim et al.(2023)]"},{"why":"Provides the clinical evidence of multi-to-multi fMRI-sMRI coupling that motivates the many-to-many alignment design.","marker":"[Khalilullah et al.(2023)]"},{"why":"Introduces the attention bottleneck / modality refinement module used to condense fused cross-modal features.","marker":"[Nagrani et al.(2021)]"},{"why":"The co-attention fusion approach the paper extends by replacing modality-as-query with latent-as-query.","marker":"[Liu et al.(2024)]"},{"why":"Provides a 3D multimodal fusion baseline (MDL-Net) that the proposed method is compared against.","marker":"[Qiu et al.(2024)]"},{"why":"Describes the presymptomatic at-risk cohort used for the primary Alzheimer's diagnosis experiments.","marker":"[Goetz et al.(2019)]"},{"why":"Describes the AD-vs-control neuroimaging dataset used as a second benchmark.","marker":"[Jack Jr et al.(2008)]"}],"fun_headline_variants":["Geometry-weighted fMRI-sMRI alignment boosts Alzheimer's diagnosis","M2M contrastive loss aligns patches for early Alzheimer's detection","Latent-as-query co-attention fuses multimodal data for AD diagnosis","Patch-level contrastive alignment improves multimodal Alzheimer's screening","Aligning fMRI patches to sMRI structure lifts AD diagnostic accuracy"],"cache_read_input_tokens":13952,"weakest_assumption_plain":"The load-bearing premise is that Eq. 6 implements multi-patch-to-multi-patch alignment; if it is actually a weighted one-to-one contrastive loss with only diagonal positives, the named 'M2M' mechanism is not what drives the reported gains.","fun_headline_variants_meta":{"raw":{"variants":["Geometry-weighted fMRI-sMRI alignment boosts Alzheimer's diagnosis","M2M contrastive loss aligns patches for early Alzheimer's detection","Latent-as-query co-attention fuses multimodal data for AD diagnosis","Patch-level contrastive alignment improves multimodal Alzheimer's screening","Aligning fMRI patches to sMRI structure lifts AD diagnostic accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000372,"raw_usage":{"total_tokens":1992,"prompt_tokens":950,"completion_tokens":1042,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":566,"completion_tokens_details":{"reasoning_tokens":956}},"tokens_in":566,"tokens_out":1042,"duration_ms":9738,"temperature":1.0,"reasoning_tokens":956,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:56:02.462673+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the gradient of Eq. 6 with respect to an off-diagonal pair: if off-diagonal pairs never receive a positive pull, the many-to-many claim is false. Then retrain with several off-diagonal pairs designated as positives and check whether the reported PR-AUC and ROC-AUC improve.","supporting_citations":[],"review_version":1}