{"id":"a14b9d21-fb2b-4f58-813b-86e5c14d75c6","arxiv_id":"2607.06356","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":7,"one_line_summary":"TMF-RSE combines image, mask, and VLM features with evidential regression to achieve state-of-the-art lung severity scores on Per-COVID-19 CT and RALO benchmarks.","lead":"The paper builds a deep learning system that fuses chest images, lung segmentation masks, and vision-language model features to predict lung disease severity scores. A smart generalist might read it to understand how combining visual, structural, and semantic AI signals can improve automated medical imaging diagnostics.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"The paper's central architectural contribution—hierarchical tri-modal fusion—is never compared against simple concatenation of the same three modalities, leaving the core novelty unvalidated.","rationale":"The reader identified the missing concatenation baseline as one of several conditions (point 1 in their rationale), but I elevate it to the single most load-bearing concern because it directly targets the paper's core novelty. The reader's weakest_assumption focused on the VLM's semantic contribution, which is a valid concern but secondary: even if the VLM acts as a generic encoder, the ablation in Table IV shows it improves performance, and the paper does not strongly claim the VLM provides interpretable clinical semantics (it claims it acts as a 'semantic prior'). The more fundamental question is whether the elaborate fusion architecture—semantic gating, structural prior modulation, hierarchical three-stage fusion—provides any benefit over trivial concatenation. The paper's own acknowledgment that this baseline is missing ('left for supplementary material if needed') indicates awareness of the gap. The small effect sizes (3-4% relative MAE improvements) without significance testing make this gap more consequential: if concatenation matches performance, the paper's contribution reduces to 'three modalities are better than one,' which is less novel than 'a specific tri-modal fusion architecture outperforms baselines.' The authorship overlap (Dornaika appears on the proposed method, baselines [8-11], and benchmarks [5-6]) compounds this concern but is not itself a technical flaw—it means independent replication is especially important. The reader's verdict of CONDITIONAL is appropriate; I would maintain it with the specific recommendation that the concatenation baseline is the single most important additional experiment needed.","tokens_in":9527,"tokens_out":2245,"duration_ms":227848,"concrete_test":"Train a single additional variant: concatenate the projected features from all three encoders (image 512-d, mask 256-d, VLM 3×768-d) into a single vector, pass through the same MLP regression head (with evidential loss), and evaluate on both benchmarks using identical training protocol. If this concatenation baseline achieves MAE within 2% of TMF-RSE on RALO GE (i.e., ≤0.346) or Per-COVID-19 CT validation (≤4.10), the hierarchical fusion mechanism's contribution is not demonstrably necessary, and the paper's central architectural claim weakens substantially.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's novelty rests on three fusion mechanisms: semantic gating (Eq. 1), structural prior modulation (Eq. 2), and hierarchical three-stage fusion (Sec. II-E). Table IV ablates modality *subsets* (image-only, image+mask, image+VLM, full tri-modal) but never ablates *fusion strategies*. The paper explicitly acknowledges this: 'We do not tabulate a concatenation-only baseline here; such a comparison is left for supplementary material if needed' (Sec. V-B). This is the single most load-bearing gap because: (1) the architectural contribution IS the fusion mechanism—if simple concatenation of all three modalities achieves similar performance, the semantic gating, structural prior modulation, and hierarchical fusion add complexity without benefit; (2) the improvements being claimed are small (RALO GE: 0.339 vs 0.351 MAE for PViTGAtt-IP, a 3.4% relative improvement; RALO LO: 0.310 vs 0.322, 3.7% relative), and without significance testing or a concatenation baseline, these margins could easily be within noise or achievable with trivial fusion; (3) on Per-COVID-19 CT test phase (Table III), TMF-RSE does NOT achieve the best MAE (3.69 vs 3.55 for two methods), undermining the uniform 'outperforms' claim. The ablation from image+VLM (0.395) to full TMF-RSE (0.339) on RALO GE is the only evidence that the fusion mechanism itself adds value, but this conflates adding the mask modality with the fusion strategy. A concatenation baseline would separate these factors.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"The paper proposes TMF-RSE, a tri-modal fusion framework for lung severity scoring that combines appearance features (DINOv3-ViT), structural features (SAM3-derived masks via a lightweight CNN), and semantic features (frozen LLaVA-Med VLM with region-specific clinical prompts for upper/middle/lower lung zones). Three fusion mechanisms are introduced: semantic gating (Eq. 1), structural prior modulation (Eq. 2), and a three-stage hierarchical fusion (Sec. II-E). An evidential regression head (NIG distribution) provides aleatoric and epistemic uncertainty estimates. Experiments are conducted on the RALO chest radiograph benchmark and the Per-COVID-19 CT benchmark (validation and test phases), with modality ablations reported in Table IV.","tokens_in":9735,"tokens_out":1926,"duration_ms":139448,"significance":"The integration of a medical VLM as a semantic prior for continuous severity regression is a reasonable and relatively novel idea, and the dual-level uncertainty modeling (structural prior modulation at the representation level plus evidential regression at the output level) is a thoughtful design choice. The modality ablation (Table IV) shows consistent monotonic improvements across image-only, bi-modal, and full tri-modal configurations on both benchmarks, which provides evidence that each modality contributes. The uncertainty sparsification analysis (Fig. 2) is a useful sanity check for the evidential head. The work addresses a clinically relevant problem with a principled architecture.","major_comments":[{"comment":"Sec. V-B: The paper's central architectural contribution is the fusion mechanism (semantic gating, structural prior modulation, hierarchical fusion), yet no concatenation-only baseline for the same three modalities is reported. The authors explicitly defer this: 'We do not tabulate a concatenation-only baseline here; such a comparison is left for supplementary material if needed.' Table IV ablates modality subsets but not fusion strategies. Without a concatenation baseline, the reader cannot distinguish whether the performance gains over bi-modal variants come from the proposed fusion mechanisms or simply from having access to all three modalities. This is load-bearing because the fusion mechanism IS the claimed novelty. A simple late-fusion or feature-concatenation baseline using the same three encoders would directly test whether semantic gating, structural prior modulation, and the 3×","section":null},{"comment":"Tables I–III: The improvements over the strongest baselines are small (RALO GE: 0.339 vs. 0.351 MAE for PViTGAtt-IP, a 3.4% relative improvement; RALO LO: 0.310 vs. 0.322, 3.7% relative). No statistical significance testing (e.g., paired t-test, bootstrap confidence intervals) is reported for any comparison. Given the modest margins and the fact that on Per-COVID-19 CT test phase (Table III), TMF-RSE does not achieve the best MAE (3.69 vs. 3.55 for two methods), significance testing is needed to support the 'outperforms' claim. The claim should also be qualified: on the test phase, TMF-RSE achieves the best PC and RMSE but not MAE.","section":null},{"comment":"Sec. II-D and Table IV: The VLM encoder's contribution is validated only through aggregate performance differences (image-only vs. image+VLM). No analysis is provided of what the VLM embeddings actually encode, whether the region-specific prompts elicit distinct pathological signatures across zones, or whether the semantic gating (Eq. 1) selectively passes clinically relevant features. The VLM could be functioning as a generic image encoder. A simple diagnostic (e.g., t-SNE of VLM embeddings by zone, gate activation statistics, or comparison against a non-medical VLM) would strengthen the claim that clinical semantics are being captured.","section":null}],"minor_comments":[{"comment":"Sec. II-E, Stage 3: The 'synergy gate' is mentioned but not formally defined with an equation, unlike semantic gating (Eq. 1) and structural prior modulation (Eq. 2). Adding the equation would improve reproducibility.","section":null},{"comment":"Sec. III-C: Batch size 1 with gradient accumulation is used, but the effective batch size is not stated. This affects reproducibility.","section":null},{"comment":"Table III: The 'outperforms' language in the abstract and conclusion should be qualified, as TMF-RSE does not achieve the best MAE on the Per-COVID-19 CT test phase.","section":null},{"comment":"Sec. II-D: The clinical prompts for each lung zone are described in prose but not quoted verbatim. Providing the exact prompt strings would aid reproducibility.","section":null},{"comment":"Fig. 2: The y-axis range (1.0–4.0) makes it difficult to assess the sparsification behavior quantitatively. Consider reporting the MAE at specific removal fractions (e.g., 10%, 25%, 50%) in a table or adjusting the axis.","section":null},{"comment":"References [8]–[11] share authorship with the submitted paper. This should be disclosed explicitly in the text or acknowledgments for transparency.","section":null}],"recommendation":"major_revision","confidential_remarks":"The authorship overlap with baselines [8]–[11] is substantial (same research group). The comparisons are not necessarily compromised by this, but the absence of a concatenation baseline combined with the modest improvement margins raises the bar for demonstrating that the proposed fusion mechanisms are the source of the gains rather than simply the addition of a third modality. The concatenation baseline is a straightforward experiment that should have been included; its absence is the primary reason for major revision rather than minor."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful and constructive review. The referee correctly identifies that our central architectural contribution is the fusion mechanism, and we agree that a concatenation-only baseline is needed to validate it. We also accept the need for statistical significance testing and a more precise characterization of our results on the Per-COVID-19 test phase. For the VLM diagnostic analysis, we will provide what we can within our computational constraints. Below we address each major comment in turn.","responses":[{"response":"The referee is correct. The absence of a concatenation-only baseline using the same three encoders is a genuine gap, and our deferral to supplementary material was insufficient given that the fusion mechanism is the claimed novelty. We will add a late-fusion / feature-concatenation baseline (all three encoders, identical training protocol, simple concatenation followed by MLP regression head) to Table IV in the revised manuscript. This will directly test whether semantic gating, structural prior modulation, and hierarchical fusion provide benefits beyond mere access to all three modalities. We expect the gap to be meaningful—particularly because our semantic gating and structural prior modulation are designed to condition representations rather than simply aggregating them—but we agree the experiment is necessary to support the claim.","revision_made":"yes","referee_comment":"Sec. V-B: No concatenation-only baseline for the same three modalities is reported. The authors explicitly defer this. Without a concatenation baseline, the reader cannot distinguish whether the performance gains come from the proposed fusion mechanisms or simply from having access to all three modalities."},{"response":"We accept both points. (1) We will add paired bootstrap confidence intervals (1000 resamples) for MAE and Pearson correlation on both benchmarks, reporting p-values against the strongest baseline in each table. We will present these in a supplementary table or as annotations on the existing tables. (2) The referee is correct that on the Per-COVID-19 CT test phase (Table III), TMF-RSE achieves the best PC (0.9122) and RMSE (6.15) but not MAE (3.69 vs. 3.55 for Taiyuan_university_lab713 and PViTGAtt-IP). We will revise the manuscript text to qualify the claim accordingly, stating that TMF-RSE achieves the best PC and RMSE on the test phase but not MAE, and noting that the different metrics rank methods differently. We will avoid the unqualified word 'outperforms' where it is not supported across all metrics.","revision_made":"yes","referee_comment":"Tables I–III: Improvements over baselines are small (3.4–3.7% relative on RALO). No statistical significance testing is reported. On Per-COVID-19 CT test phase (Table III), TMF-RSE does not achieve the best MAE (3.69 vs. 3.55). Significance testing is needed, and the 'outperforms' claim should be qualified."},{"response":"We agree that diagnostic analysis of the VLM embeddings would strengthen the paper and we will add what is feasible within our computational constraints. Specifically: (1) We will provide t-SNE visualizations of the per-zone VLM embeddings (upper, middle, lower) to show whether region-specific prompts produce separable representations. (2) We will report gate activation statistics (mean, variance, distribution) from Eq. 1, broken down by lung zone, to show whether the semantic gating is selectively modulating image features rather than acting as a uniform pass-through. (3) We will add a comparison against a non-medical VLM (standard LLaVA) as an additional ablation row, which directly tests whether the medical pretraining of LLaVA-Med contributes beyond generic vision-language features. We note one limitation: a full probing analysis of what the frozen VLM embeddings encode at a semantic concept level would require additional labeled pathological data that is not available in the Per-COVID-19 or RALO benchmarks, so our diagnostics will be representational rather than concept-level. We will be transparent about this scope.","revision_made":"partial","referee_comment":"Sec. II-D and Table IV: The VLM encoder's contribution is validated only through aggregate performance differences. No analysis of what the VLM embeddings actually encode, whether region-specific prompts elicit distinct pathological signatures, or whether semantic gating selectively passes clinically relevant features. A simple diagnostic (t-SNE, gate activation statistics, comparison against non-medical VLM) would strengthen the claim."}],"tokens_in":9419,"tokens_out":943,"duration_ms":117906,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"The paper combines three pretrained encoders — DINOv3 for appearance, a lightweight CNN on SAM3 masks for structure, and frozen LLaVA-Med for regional clinical semantics — into a hierarchical fusion architecture with evidential regression for lung severity scoring. The ablation in Table IV is the strongest part: each modality adds measurable value going from image-only (0.469 MAE on RALO GE) to full tri-modal (0.339), and the pattern holds across both benchmarks. The evidential uncertainty sparsification curve (Fig. 2) is a nice touch, showing the uncertainty scores actually track prediction difficulty. The architecture is sensible and the experiments are run on real benchmarks with external reference standards, not toy data. Credit for that. The stress-test concern about the missing concatenation baseline is the real issue, and it lands. The paper's novelty is the fusion mechanism — semantic gating, structural prior modulation, hierarchical three-stage fusion — but Table IV only ablates modality subsets, not fusion strategies. The authors explicitly defer concatenation comparison to supplementary material. If concatenating all three modalities matches the hierarchical fusion, the architectural contribution evaporates. The improvements over baselines are also modest (3-4% relative on RALO), with no significance testing, and on Per-COVID-19 test phase (Table III) TMF-RSE does not achieve the best MAE — 3.69 vs 3.55 for two leaderboard entries. The claim of uniform outperformance is overstated. The VLM concern is valid but secondary. The paper never analyzes what the LLaVA-Med embeddings encode or whether the three-zone clinical prompts elicit the intended pathological signatures. The VLM could be functioning as a generic image encoder. That said, the ablation does show the VLM branch contributes beyond image-only, so it is doing something — just not necessarily what the authors claim. The authorship overlap between baselines [8,9,10,11] and the proposed method is worth noting but does not invalidate the comparison since the benchmarks are externally defined with independent reference standards. No code or data is released, which limits reproducibility given the seven or so hyperparameters involved. This paper is for researchers working on multi-modal fusion in medical image regression. It deserves a serious referee who should require: (1) the concatenation baseline, (2) significance testing or at least confidence intervals, and (3) some analysis of what the VLM embeddings capture. If the concatenation baseline holds up, this is a solid contribution; if not, the paper needs significant revision.","headline":"Tri-modal fusion (DINOv3 + SAM3 + LLaVA-Med) for lung severity regression shows consistent ablation gains but never validates the fusion mechanism itself against simple concatenation.","tokens_in":10587,"tokens_out":612,"would_cite":false,"duration_ms":149607,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Three views beat one for lung severity scoring","keywords":["multi-modal fusion","lung severity scoring","vision-language models","evidential regression","chest CT","chest radiography","semantic gating","uncertainty estimation"],"falsifier":"If the image+VLM ablation variant performed no better than image+mask, the claim that VLM-derived clinical semantics add value beyond generic feature extraction would be undermined.","tokens_in":9607,"feed_emoji":"🫁","tokens_out":1030,"duration_ms":144944,"temperature":0.7,"pith_summary":"This paper argues that automated lung disease severity scoring from chest imaging improves substantially when the model fuses three distinct information channels — raw image appearance, anatomical structure from segmentation masks, and clinical semantics from a medical vision-language model — rather than relying on image features alone. The architecture, called TMF-RSE, routes each chest image through three parallel encoders: a DINOv3 vision transformer for visual texture, a lightweight CNN for lung-mask boundaries, and a frozen LLaVA-Med model that extracts semantic embeddings from upper, middle, and lower lung zones using targeted clinical prompts. These three streams are combined through a hierarchical fusion module with semantic gating (VLM features control which image evidence passes forward), structural prior modulation (mask confidence shapes attention sharpness), and a three-stage progressive integration. An evidential regression head then outputs both a severity score and separate aleatoric and epistemic uncertainty estimates. On two benchmarks — Per-COVID-19 CT and RALO chest radiographs — the full tri-modal model achieves lower mean absolute error and higher Pearson correlation than recent transformer-based baselines, with ablations showing each modality contributes incrementally. The uncertainty estimates are validated through a sparsification analysis showing that removing the highest-uncertainty predictions progressively reduces error on the remaining cases.","feed_headline":"Tri-modal fusion beats transformers at lung severity scoring","feed_subtitle":"Combining image texture, lung structure, and clinical language yields lower error and useful uncertainty on CT and X-ray benchmarks.","key_machinery":"TMF-RSE architecture: DINOv3-ViT image encoder (appearance), lightweight CNN mask encoder (structure), frozen LLaVA-Med VLM encoder (semantics), tri-modal fusion with semantic gating + structural prior modulation + hierarchical fusion, evidential regression head (Normal-Inverse-Gamma distribution)","core_discovery":"The central finding is that combining appearance, structural, and semantic features in a hierarchical fusion architecture yields measurable improvements over image-only and pairwise bi-modal approaches for continuous lung severity regression. The VLM encoder, queried with region-specific clinical prompts for three lung zones, contributes beyond what a generic image encoder provides, as shown by the gap between image+VLM and image-only ablation variants. The dual-level uncertainty design — structural prior modulation at the representation level and evidential regression at the output level — produces uncertainty scores that track prediction difficulty, confirmed by the monotonic decrease inMA","pith_inferences":["The paper validates VLM contribution only through aggregate performance gains, not through analysis of what the VLM embeddings encode. If the VLM is acting as a generic image encoder, a lighter-weight frozen image backbone might achieve similar gains at lower computational cost.","The three-zone lung division may not align with actual pathological distributions in diseases beyond COVID-19; finer anatomical parcellation could matter for pathologies with different spatial patterns.","The batch-size-1 training constraint imposed by VLM processing raises questions about optimization stability and whether the reported gains would persist with larger-scale training or different optimizer configurations.","The uncertainty sparsification analysis shows correlation between uncertainty and error but does not establish calibration — whether the uncertainty magnitudes are clinically actionable for threshold-based triage remains untested."],"forward_implications":["Clinical deployment of automated severity scoring could use the evidential uncertainty estimates to flag cases requiring radiologist review, enabling triage rather than full replacement of human assessment.","The region-specific VLM prompting strategy (upper/middle/lower lung zones with tailored clinical queries) could generalize to other anatomical scoring tasks where pathology distribution varies by region.","If the VLM embeddings prove to encode clinically meaningful features rather than acting as a generic image encoder, the semantic gating mechanism could be extended to provide interpretable clinical reasoning alongside severity predictions.","The dual-level uncertainty design (representation-level + output-level) could be adopted in other medical imaging regression tasks where both segmentation quality and prediction confidence matter."],"fun_headline_variants":["Adding clinical language to CT texture and lung structure improves severity scores","Region-specific semantic prompts and evidential regression outperform transformer baseline","Tri-modal VLM fusion lowers error on Per-COVID-19 and RALO lung severity tasks","Appearance, structure, and clinical semantics combine for better lung severity regression","Hierarchical tri-modal fusion with uncertainty estimates beats image-only baselines"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The paper assumes that the frozen medical vision-language model, when queried with region-specific clinical prompts about lung pathology, extracts semantically meaningful clinical features that contribute to severity regression — but it only validates this through aggregate performance improvements, without examining what the VLM embeddings actually encode.","fun_headline_variants_meta":{"raw":{"variants":["Adding clinical language to CT texture and lung structure improves severity scores","Region-specific semantic prompts and evidential regression outperform transformer baselines","Tri-modal VLM fusion lowers error on Per-COVID-19 and RALO lung severity tasks","Appearance, structure, and clinical semantics combine for better lung severity regression","Hierarchical tri-modal fusion with uncertainty estimates beats image-only baselines"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":617,"prompt_tokens":520,"completion_tokens":97,"prompt_tokens_details":null},"tokens_in":520,"tokens_out":97,"duration_ms":48559,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-08T08:26:04.583704+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If the image+VLM ablation variant performed no better than image+mask, the claim that VLM-derived clinical semantics add value beyond generic feature extraction would be undermined.","supporting_citations":[],"review_version":1}