{"id":"9c12842a-5c6a-4d62-ac12-9ff46b207fe4","arxiv_id":"2507.04999","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A labeled optimal transport alignment plus asymmetric fusion keeps multimodal eye disease grading accurate even when fundus or OCT is missing.","lead":"This paper introduces a deep learning framework that keeps eye disease grading accurate when one of two imaging modalities is missing, by aligning fundus and OCT images with optimal transport. It reports state-of-the-art results on three large public eye datasets, which matters because many clinics cannot always obtain both scans.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Inter-modality missing results are not reproducible because the test-time protocol for single-modality inputs is unspecified; the claimed robustness from OT alignment cannot be attributed without that protocol.","rationale":"The reader's conditional verdict is appropriate. My stress-test converges on the same load-bearing gap: the inter-modality missing experiments are defined by a protocol ('train on complete training set, evaluate on single-modality test set') but the forward pass for a single-modality input is never specified. Since losses and fusion require two encoders, the paper must define a missing-feature substitution; without it, the central robustness result is not reproducible. The public code makes this checkable, so this is a condition for acceptance rather than a fatal flaw. I also noted the shape/index inconsistency in Eq. 3, which is secondary but concrete: the feature-wise alignment, one of the two claimed contributions, cannot be implemented as written. The ablations and t-SNE are supportive but do not resolve the test-time protocol question. There is no evidence of fabrication, and the paper's motivation and overall structure are coherent. Therefore the appropriate response remains CONDITIONAL: the manuscript should be accepted only after the missing-modality inference path is specified and verified, and Eq. 3 is corrected or clarified.","tokens_in":8536,"tokens_out":6879,"duration_ms":80158,"concrete_test":"Using the released code, run the exact inference path that produces the Inter-Modality Missing rows of Table 1 with only fundus inputs and only OCT inputs, and trace how absent-modality features are produced (zero tensor, learned missing encoder, dropped stream, or imputed prototype). Then rerun the same rows with cross-modal fusion terms hard-masked whenever one modality is absent. If average AUC changes by more than about 2 points relative to the reported values, the robustness claim depends on an unspecified test-time substitution rather than on the OT alignment. Separately, type-check Eq. 3 by computing its loss literally on one batch; if the shapes mismatch, the paper's description does not match the evaluated implementation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—SOTA under inter-modality incompleteness—requires that a model trained only on complete pairs (Section 3, Inter-Modality Missing) classify a single-modality test input. The manuscript does not describe how the network behaves when one encoder is absent. The full model fuses backbone features, class-prototype features, and an 'OT fundus' feature; when OCT is missing there is no e_o, no T_v, and no soft prototype, so the OCT stream and the asymmetric fusion cannot be computed as written. Figure 1 labels a 'Missing Feature' but no text defines its role. If the code substitutes zero tensors, a learned imputation branch, or drops cross-modal terms, the reported robustness may come from that unspecified mechanism rather than from labeled OT alignment. This is load-bearing because the paper's central robustness claim, exceeding IMDR by 4-5% AUC when one modality is missing, is not reproducible without this protocol. A secondary internal inconsistency supports the same concern: Eq. 3 declares T_v in R^{D_o x D_f} while the GW objective sums over sample indices i,j,k,l and multiplies by T_v_{k,l}, and 'OT fundus = T_v^T e_o' has mismatched shapes when e_o is in R^{N x D_o}. As printed, the feature-wise alignment cannot be implemented, so the main cross-modal mechanism is not fully specified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes RIMA, a multimodal alignment and fusion framework for ophthalmic disease grading and diagnosis from color fundus images and OCT scans. The core idea is to use labeled optimal transport to align features across modalities at two scales: class-wise alignment via soft prototypes, and feature-wise alignment via a Gromov-Wasserstein transport plan, followed by an asymmetric fusion strategy that treats the two modalities differently because fundus features are global while OCT features are noisy and localized. The authors report experiments on three Harvard-30k datasets (AMD, DR, Glaucoma) under three settings: complete-modality fusion, inter-modality missing (one modality entirely absent at test time), and proportional random missing (partial missing during both training and test). They claim state-of-the-art performance in the first two settings, with the abstract highlighting a 4–5% AUC improvement over the IMDR baseline when one modality is missing.","tokens_in":8852,"tokens_out":4333,"duration_ms":44158,"significance":"If the reported results are reproducible, the paper addresses a clinically important problem: robust ophthalmic diagnosis when one of two imaging modalities is unavailable. The labeled-OT formulation with class constraints is a reasonable and interesting extension of standard GWOT, and the asymmetric fusion is motivated by domain-specific characteristics of fundus and OCT. The paper also makes a concrete effort to evaluate on three real-world ophthalmic datasets and reports improvements over several existing methods. However, the current manuscript has a load-bearing reproducibility gap: the test-time protocol for a fully missing modality is never specified, and the feature-wise alignment equation contains a dimensional inconsistency that prevents implementation as written. The lack of standard deviations or significance tests further weakens the empirical claims. These issues need to be resolved before the central claim of robustness under inter-modality incompleteness can be accepted.","major_comments":[{"comment":"The test-time protocol for a single-modality input is unspecified. The model description in Section 2 assumes both encoders produce features e_f and e_o, and the asymmetric fusion in Section 2.3 concatenates backbone, class-prototype, and feature-transport features from both streams. When one modality is absent at test time, there is no e_o (or e_f), no transport plan T_v, and no soft prototype, so the fusion cannot be computed as written. The figure labels a 'Missing Feature' but no text defines its role. The central robustness claim—exceeding IMDR by 4–5% AUC when one modality is missing—cannot be reproduced or attributed to OT alignment without a clear statement of how the network handles the absent encoder (e.g., zero-padding its output, a learned imputation branch, or dropping cross-modal terms). Please specify the exact inference path and, if the model is trained only on complete pairs, explain how the fusion adapts to a single-modality input.","section":"Section 3, Inter-Modality Missing; Section 2.3"},{"comment":"Equation (3) is not implementable as written. T_v is defined as belonging to R^{D_o × D_f}, but the objective sums over sample indices i,j,k,l and multiplies by T_of_{ci,j} T_v_{k,l}, which implies T_v is indexed by samples. Moreover, the expression 'OT fundus = T_v^T e_o' is dimensionally inconsistent when e_o ∈ R^{N × D_o} and T_v ∈ R^{D_o × D_f}. Please provide the correct dimensions and index conventions for T_v, and clarify how the feature-wise transport plan is used to produce the OT fundus feature. This is not a cosmetic issue; without a consistent definition, the feature-wise alignment mechanism cannot be implemented from the paper.","section":"Equation (3)"},{"comment":"The empirical evaluation lacks measures of variability and statistical testing. The paper states that five-fold cross-validation was used, but Table 1 reports only point estimates. The baseline comparisons are taken from a prior paper (Liu et al. [7]) without any variance information, so the claimed improvements (e.g., exceeding IMDR by 4–5% AUC in the inter-modality missing setting) cannot be assessed for statistical significance. Please report per-fold means and standard deviations for all methods, and, where feasible, paired significance tests for the key comparisons.","section":"Table 1"},{"comment":"The training protocol for the proportional random missing setting is not described. The text says 'Train on the training set with a partial proportion of missing modalities,' but it does not specify how the network is trained when a modality is missing (e.g., whether the same imputation/handling mechanism as in the inter-modality missing setting is used). This matters because the proportional random missing experiments are used to claim robustness in Figure 3, which only compares against B-CNN and does not include other state-of-the-art methods. Please clarify the training procedure for this setting and include comparisons with at least one more baseline.","section":"Section 3, Proportional Random Missing"}],"minor_comments":[{"comment":"The notation is confusing: the feasible set T_{µ,ν} is defined earlier, then ̅T_{µ,ν} is introduced with the same symbol T used for both the set and the transport plan. Please use distinct notation to avoid ambiguity.","section":"Section 2.1, Equation (1)"},{"comment":"The ablation table is garbled; rows for 'w/o Class-wise Alignment' and 'w/o Feature-wise Alignment' do not clearly indicate which modality is being evaluated, and the numbers seem misaligned. Please reformat the table so that each row clearly indicates the modality and the metrics.","section":"Table 2"},{"comment":"The 'Missing Feature' label in Figure 1 is not defined anywhere in the text. Please either remove it or explain its role in the architecture.","section":"Figure 1"},{"comment":"The OCT input is resized to 96×96×96, but the paper does not specify how the volume is sampled (number of B-scans, slice selection, or preprocessing). Please add these details for reproducibility.","section":"Section 3, Implementation Details"},{"comment":"There is a typo in the summation index: 'nX' should be 'N'. Also, the cosine-similarity loss should clarify whether the sum is over all samples in the batch.","section":"Equation (5)"}],"recommendation":"major_revision","confidential_remarks":"The paper's GitHub link suggests that code may be available, but I did not verify the repository during review. The heavy reliance on baseline numbers from a prior paper (Liu et al. [7]) without variance is a concern; please ensure the authors provide their own re-runs or at least the per-fold statistics for the baselines they report."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this is a reasonable, well-motivated paper on handling missing OCT/fundus at test time, built around labeled optimal transport and an asymmetric fusion. The genuinely new bits are the combination of class-wise prototype alignment with feature-wise transport and the asymmetry between modalities; the labeled OT core is borrowed from Ryu et al. [11] and GWOT is standard. The reported numbers are consistently strong across three Harvard-30k subsets, and the ablation study shows the components matter. Code is promised.\n\nThe soft spots, in order of importance. First, the test-time protocol for inter-modality missing is unspecified. Section 3 says only 'Train on a complete training set and evaluate on test set with single modality.' The model as drawn has two encoders and a fusion block; when OCT is absent, the OCT stream has no input. The caption mentions a 'Missing Feature' but the text never says what role it plays. If the missing stream is zero-filled, or the cross-modal terms are dropped, the robustness could be coming from that choice, not from the OT alignment. This is load-bearing for the central claim.\n\nSecond, Eq. 3 and the 'OT fundus' definition don't type-check as printed. T_v is declared in R^{Do x Df}, then used with indices k,l that would have to be sample indices in the GW distance, and T_v^T e_o is a shape mismatch when e_o is N x Do. This may be a notation slip, but as written the feature-wise alignment cannot be implemented, and feature-wise alignment contributes a measurable chunk in Table 2.\n\nThird, the results tables report no standard deviations or significance tests, and the baselines are taken wholesale from Liu et al. [7]. That's a real limitation but not fatal; the gap to IMDR is large enough that it probably holds.\n\nOn circularity: I'm with the reader — this is supervised self-referential training, not a circular evaluation. The class constraints use labels, but the method isn't relabeling fitted constants as predictions.\n\nWho this is for: researchers working on missing-modality medical imaging, especially ophthalmology, and people building on labeled OT for cross-modal alignment. It deserves a serious referee; the empirical story is plausible and the released code will let reviewers resolve the test-time question. My recommendation: send to peer review, but the revision must specify the missing-modality input handling and fix Eq. 3, or the robustness claim stays unverifiable.","headline":"Plausible and well-motivated missing-modality framework with strong reported numbers, but the test-time protocol for a single modality is unspecified and the feature-wise OT equation doesn't type-check.","tokens_in":9339,"tokens_out":5282,"would_cite":true,"duration_ms":51932,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A class-constrained optimal transport alignment between fundus and OCT images lets a model keep its diagnostic accuracy when one eye-imaging modality is entirely missing at test time.","keywords":["missing modality","optimal transport","ophthalmic disease grading","fundus imaging","optical coherence tomography","multimodal alignment","asymmetric fusion","Gromov-Wasserstein"],"falsifier":"Train two ResNet-50 baselines, one on fundus only and one on OCT only, using the same Harvard-30k splits, and compare against the proposed model trained only on paired data then tested with one modality missing. If the paired-trained model does not beat the single-modality baselines on the respective single-modality test sets, the learned alignment is not transferring and the robustness claim fails.","tokens_in":8372,"feed_emoji":"👁️","tokens_out":8225,"duration_ms":86630,"temperature":0.7,"pith_summary":"This paper argues that a multimodal ophthalmic diagnosis model can be made robust to a fully missing modality by aligning fundus and OCT features with a class-constrained optimal transport plan, rather than by imputing images or distilling from a teacher. The proposed framework learns soft same-class matches between modalities, builds class prototypes from those matches, and uses an asymmetric fusion strategy that treats the fundus as global and OCT as localized. On three Harvard-30k datasets, the authors report the best accuracy, AUC, and F1 in both complete-modality fusion and inter-modality missing settings, including gains of 4-5% AUC over the IMDR baseline when one modality is absent. The point of the work is that paired training data can be exploited so strongly that at test time one modality alone suffices.","feed_headline":"Missing one eye scan no longer sinks diagnosis accuracy","feed_subtitle":"Optimal transport aligns fundus and OCT features so one modality alone can still grade eye disease.","key_machinery":"The central object is Labeled Optimal Transport, a Gromov-Wasserstein transport plan constrained by $T_{ij}>0 \\Rightarrow y_i^f=y_j^o$, so mass only flows between features of the same class. It is solved with entropy-regularized Sinkhorn iterations and then drives two alignment levels: class-wise alignment, where repeated sampling from the plan's row probabilities gives a soft prototype $c_i^o = \\sum_j p(j|i) e_j^o$ used in a cosine loss, and feature-wise alignment, where the plan produces an 'OT fundus' feature $T_v^\\top e_o$ for distribution matching. On top of this, an asymmetric fusion path concatenates cross-modal, semantic, and backbone features for the fundus through an attention layer, while OCT uses only semantic and backbone features through MLPs to avoid overfitting noise. The machinery's job is to replace image-level imputation and strict one-to-one matching with a soft, class-aware correspondence that can be exploited when one modality is absent.","core_discovery":"The central claim is that label-constrained optimal transport alignment is enough to carry ophthalmic diagnosis when one modality is absent. The transport plan is restricted so that mass moves only between same-class samples, $T_{ij}>0 \\Rightarrow y_i^f=y_j^o$, and then used to build soft class prototypes and a feature-wise 'OT fundus' representation. The authors report that this framework beats the compared methods in complete-modality fusion and in inter-modality missing settings on Harvard-30k AMD, DR, and Glaucoma, and the ablation attributes the largest share of the gain to class-wise alignment, which costs 7-8% AUC on DR when removed.","pith_inferences":["An implication the authors leave implicit is that the same asymmetric optimal-transport alignment could transfer to any paired-modality medical task where one view is global and the other is localized or noisy, such as MRI with pathology slides or CT with endoscopy.","A simpler representation-level imputer that learns a conditional mean of the target modality given the source could serve as a baseline to isolate what the optimal-transport constraint adds, since the soft prototype is essentially a class-conditioned expected match.","The class constraint depends on training labels, so label noise would corrupt the transport plan; testing under noisy or partial labels would reveal whether the claimed robustness degrades gracefully.","The paper does not report single-modality-only training baselines, so an external comparison against fundus-only and OCT-only ResNet-50 models would clarify whether the reported gains come from the alignment or from the shared training recipe."],"forward_implications":["A model trained once on paired fundus/OCT data could be deployed in clinics that have only a fundus camera or only an OCT scanner, without retraining on single-modality data.","Because the alignment is class-constrained and soft, it captures within-class variation in lesions rather than forcing one-to-one matches, which the ablation attributes 7-8% AUC on DR to when removed.","The asymmetry principle, global feature transport toward fundus and class-only transport toward OCT, offers a design rule for other paired modalities with different noise and coverage profiles.","Under proportional random missing rates from 0% to 100%, the framework's AUC and F1 stay relatively flat compared with the B-CNN baseline, so the method is intended for real clinics with partially missing data."],"supporting_citations":[{"why":"Baseline IMDR whose inter-modality missing and complete-fusion results the paper compares against, and the source of the other compared methods' reported numbers.","marker":"[7]"},{"why":"Supplies the Harvard-30k AMD, DR, and Glaucoma datasets used for all experiments and ablations.","marker":"[8]"},{"why":"Defines the Gromov-Wasserstein distance used as the cost in the class-wise and feature-wise transport objectives.","marker":"[9]"},{"why":"Provides the Sinkhorn iterations used to solve the entropy-regularized optimal transport plans.","marker":"[13]"},{"why":"The labeled Gromov-Wasserstein formulation that the proposed same-class transport constraint extends.","marker":"[11]"},{"why":"Background on computational optimal transport and entropy regularization that justifies the solver and the Gromov-Wasserstein distance.","marker":"[10]"},{"why":"ResNet-50 backbone shared by all compared models for both fundus and OCT encoders.","marker":"[5]"}],"fun_headline_variants":["Optimal transport aligns missing eye scans for diagnosis","Missing OCT or fundus? Label-aware OT still grades","One eye scan suffices with class-constrained OT","Label-constrained transport fixes incomplete eye data","Robust eye grading from single modality via OT"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's robustness rests on the assumption that the class-constrained optimal transport alignment learned from fully paired training data transfers to a classifier at test time when an entire modality is absent, even though the inter-modality missing training protocol never shows the model a missing-modality input.","fun_headline_variants_meta":{"raw":{"variants":["Optimal transport aligns missing eye scans for diagnosis","Missing OCT or fundus? Label-aware OT still grades","One eye scan suffices with class-constrained OT","Label-constrained transport fixes incomplete eye data","Robust eye grading from single modality via OT"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000206,"raw_usage":{"total_tokens":1406,"prompt_tokens":964,"completion_tokens":442,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":580,"completion_tokens_details":{"reasoning_tokens":369}},"tokens_in":580,"tokens_out":442,"duration_ms":5363,"temperature":1.0,"reasoning_tokens":369,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:34:06.379066+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train two ResNet-50 baselines, one on fundus only and one on OCT only, using the same Harvard-30k splits, and compare against the proposed model trained only on paired data then tested with one modality missing. If the paired-trained model does not beat the single-modality baselines on the respective single-modality test sets, the learned alignment is not transferring and the robustness claim fails.","supporting_citations":[{"cited_title":"In: Proceedings of the AAAI Conference on Artificial Intelli- gence","cited_arxiv_id":null,"evidence_quote":"Baseline IMDR whose inter-modality missing and complete-fusion results the paper compares against, and the source of the other compared methods' reported numbers."},{"cited_title":"In: International conference on machine learning","cited_arxiv_id":null,"evidence_quote":"Defines the Gromov-Wasserstein distance used as the cost in the class-wise and feature-wise transport objectives."},{"cited_title":"Cross-modality Matching and Prediction of Perturbation Responses with Labeled Gromov-Wasserstein Optimal Transport","cited_arxiv_id":"2405.00838","evidence_quote":"The labeled Gromov-Wasserstein formulation that the proposed same-class transport constraint extends."}],"review_version":1}