{"id":"e0eb252e-abdc-4342-8b63-b16c1942a669","arxiv_id":"2502.04794","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A pretrained-feature plus self-attention fusion model classifies fever-of-unknown-origin causes from PET/CT and clinical data with macro-AUROC 0.865 to 0.929 on a single-center retrospective cohort.","lead":"This study builds a model that combines PET/CT scans with lab results to classify the cause of fever of unknown origin in 416 patients, reporting area-under-the-curve scores between 0.87 and 0.93. A generalist reader might care because FUO is a slow and expensive diagnosis, and this is a concrete attempt to automate part of that work.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline AUROC range is a post-hoc maximum over four feature extractors and multiple hyperparameter settings, so the reported 0.8654–0.9291 may not reflect a single reproducible model.","rationale":"The reader's verdict is CONDITIONAL, and the concerns raised—missing confidence intervals, training-set hyperparameter selection, no external validation—are real. My stress-test identifies the most load-bearing version of those concerns: the headline AUROC range is constructed by picking the best feature extractor for each task after seeing the test-fold results with no held-out validation. This is directly checkable from Tables IV–X: the best configuration is not constant across tasks, so the reported range cannot be attributed to a single MedMimic model. The reader's weakest_assumption points to label validity and task boundaries, which is a legitimate interpretation-level concern, especially for the 'early diagnosis' framing; however, the selection-bias concern is more immediately fatal to the numerical claim because it can be demonstrated from the published tables without access to clinical records or data. The paper also lacks code/data release, which prevents checking whether the reported metrics are stable; that reinforces the need for the concrete test above. I do not see an internal mathematical inconsistency in the fusion equations, but the evaluation protocol makes the headline claim non-reproducible as stated. The verdict should remain CONDITIONAL: the authors can address this by fixing the model, using nested cross-validation, and reporting fold-level variance and significance tests.","tokens_in":20462,"tokens_out":7716,"duration_ms":84126,"concrete_test":"Fix a single feature extractor (e.g., DINOv2) and a single hidden dimension before evaluation; rerun all seven tasks with nested 5-fold cross-validation (hyperparameters chosen inside each training fold) and report mean ± std per fold for each modality configuration. Then test full multimodal vs Clinical+PET with a paired DeLong test per fold. If the fixed-model range drops below the headline range or the fusion gain is not significant, the reported AUROC is an artifact of post-hoc selection.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central performance claim is not tied to a fixed model. In Tables IV–X, the MFCN is evaluated under four feature extractors (PCA, ResNet-18, ViT, DINOv2), and the abstract's '0.8654–0.9291' takes the best extractor per task: Task 3 uses ViT (0.8654), Task 5 uses DINOv2 (0.9291), while the same DINOv2 model gives 0.8556 on Task 3 and ViT gives 0.8342 on Task 6 (Tables VIII, X). Section V also states that 'the hyperparameter configuration that yielded the best performance on the training sets was selected as optimal,' with no held-out validation or nested cross-validation; the ablation varies hidden dimension (16–256) and ResNet depth (3–6). With 416 patients and rare classes such as hematologic malignancies (n=18, Table II), per-fold macro-AUROC will have large variance, yet no confidence intervals or significance tests are reported. Therefore the claim that 'Clinical Data + CT + PET consistently achieved the best performance' may reflect selection over many configurations rather than a reproducible diagnostic model.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MedMimic, a multimodal fusion framework for classifying causes of fever of unknown origin (FUO). Imaging features are extracted from 18F-FDG PET/CT slices with four encoders (PCA, ResNet-18, ViT, DINOv2), and a learnable self-attention network (MFCN) fuses these features with clinical data. The method is evaluated on 416 patients from a single center across seven diagnostic tasks defined by regrouping final clinical diagnoses. Using five-fold cross-validation and macro-AUROC, the authors report a range of 0.8654 to 0.9291 for the Clinical Data + CT + PET configuration and conclude that multimodal fusion consistently outperforms single-modality ML and DL baselines. Ablation studies examine hidden dimensions and network components.","tokens_in":20664,"tokens_out":4486,"duration_ms":48554,"significance":"The manuscript addresses a clinically meaningful problem, uses a real hospital cohort, and systematically compares several input configurations and feature extractors. The five-fold protocol and the use of out-of-sample test folds are appropriate, and the ablation study is a useful sanity check. If the reported performance corresponded to a fixed, pre-specified model with reported uncertainty, the result would be of interest to the medical-imaging community. At present, however, the central claim is weakened by selection over multiple feature extractors and hyperparameter settings, by the absence of confidence intervals or significance tests, and by the gap between the reported AUROC range and the actual values of any single model configuration.","major_comments":[{"comment":"The headline macro-AUROC range 0.8654–0.9291 is a per-task maximum over four feature extractors, not the performance of one fixed MFCN model. For example, in Table VI (Task 3) the best Clinical Data + CT + PET result is ViT at 0.8654 while DINOv2 gives 0.8556; in Table VIII (Task 5) the best is DINOv2 at 0.9291; in Table IX (Task 6) the best is DINOv2 at 0.8899 while ViT gives 0.8342. The abstract and Section V-C should state which extractor and architecture produce each number, or should report results for a single pre-specified model, otherwise the central claim is not reproducible as stated.","section":"Abstract and Section V-C, Tables IV-X"},{"comment":"No confidence intervals, standard deviations, or significance tests are reported for any macro-AUROC value. With 416 patients and rare classes such as hematologic malignancies (n=18 in Table II), per-fold macro-AUROC will have substantial variability. In addition, the paper states that 'the hyperparameter configuration that yielded the best performance on the training sets was selected as optimal' and the ablation varies hidden dimension from 16 to 256 and ResNet depth from 3 to 6; selecting the best configuration on the same study data and then reporting the best result can inflate apparent performance. The authors should report per-fold results, confidence intervals, and use nested cross-validation or a held-out validation set for model selection, or clearly separate model-selection performance from final test performance.","section":"Section V (Experimental Setup) and Tables IV-X"},{"comment":"The seven classification tasks are constructed by regrouping final clinical diagnoses, and patients without a definitive diagnosis were excluded. This makes the reported AUROC values a measure of classifying already-resolved diagnoses, not necessarily of early FUO diagnosis as claimed. The number of excluded undiagnosed patients and their characteristics are not reported. The authors should quantify the exclusions, discuss how label noise or ambiguous final diagnoses could affect the tasks, and temper the 'early diagnosis' claim accordingly.","section":"Section IV-A, Fig. 2, and Task Definitions"},{"comment":"The zero-padding mask tensor Z introduced in Section IV-C.2 is never applied as an attention mask in Algorithm 2; zero-padded slice positions appear to participate in the self-attention computation on the same footing as real slices. The paper should clarify whether padded positions are masked out, and if they are not, justify this design choice and provide an ablation comparing masked versus unmasked attention. Without this, the contribution of the learnable self-attention mechanism is not precisely established.","section":"Section IV-C.2 and Algorithm 2"},{"comment":"The comparison is not symmetric: the MFCN is evaluated with four imaging feature extractors, while the baseline ML and single-modality DL methods are not given the same fusion architecture. The statement that 'Clinical Data + CT + PET consistently achieved the best performance' is therefore a claim about MFCN with per-task best extractor selection, not about the fusion strategy in general. A fairer comparison would couple the same fusion module to all extractors or report MFCN results for each extractor separately as the primary result.","section":"Section V-A and Table III"}],"minor_comments":[{"comment":"The text says 'accuracy ranging from 0.8654 to 0.9291,' but the tables report macro-AUROC, not accuracy; please use the correct metric name consistently.","section":"Section V-C"},{"comment":"Line 20 of Algorithm 1 says 'FPET ← Pad(FCT)' but should almost certainly be 'FPET ← Pad(FPET)'; this pseudocode error should be corrected.","section":"Algorithm 1"},{"comment":"The notation for y is inconsistent: Eq. (1) calls y 'a scalar' while the problem formulation defines y as a one-hot encoded label vector. Please unify the notation.","section":"Section III-B and Eq. (1)"},{"comment":"The number of PCA components b1 is never specified in the experimental setup, making the PCA baseline difficult to reproduce.","section":"Section IV-C.1 and Figure 5"},{"comment":"Reference [18] cites a survey on large language model datasets, but the text discusses a multicenter FUO study in Japan; the citation appears incorrect.","section":"References"},{"comment":"The index term 'Self-Attenton' contains a typo and should read 'Self-Attention'.","section":"Index Terms"}],"recommendation":"major_revision","confidential_remarks":"The manuscript has a genuine clinical application and a reasonable experimental skeleton, but the main performance claim is not tied to a single reproducible configuration and lacks uncertainty quantification. These are fixable in revision: report a fixed model or extractor-specific results, add confidence intervals and significance tests, and clearly separate model selection from evaluation. The label-construction issue also needs a frank discussion. I do not see grounds for rejection, but the current form is not ready for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a legitimate clinical ML extension—a new single-center FUO cohort with 416 PET/CT cases, seven clinically sensible classification tasks, and a broad comparison of ML and DL baselines. What it is not, as written, is a demonstration that one reproducible MFCN model hits 0.8654–0.9291 macro-AUROC. That range is assembled by taking the best of four feature extractors per task, and the selection is made after seeing test-fold results.\n\nWhat's genuinely new and good: the dataset and task decomposition are useful. FUO is under-served by ML benchmarks, and the seven tasks (benign vs malignant, infectious vs non-infectious, three-way etiologies, etc.) match real clinical questions. The paper is transparent about the cohort exclusions and reports complete tables for all configurations. DINOv2 and ViT beating PCA in most cases is a credible, publishable finding. The ablation studies, while not decisive, do show the components matter.\n\nNow the soft spots, in proportion. The load-bearing issue is model selection. Tables IV–X report MFCN under PCA, ResNet-18, ViT, and DINOv2 for every task, and the abstract's range picks the best extractor per task: Task 3 uses ViT, Task 5 uses DINOv2, and the same model does worse on other tasks. That is an upper envelope, not a single model. With 416 patients and classes as small as n=18, per-fold macro-AUROC variance is going to be large, yet there are no confidence intervals, no significance tests, and no external validation. Hyperparameters were chosen on training folds without nested cross-validation. Code and data are not released, so the numbers cannot be checked. The labels are final clinical diagnoses from one center, and undiagnosed patients were excluded; that is reasonable for a first study but it means the 'early diagnosis' claim rests on label validity you cannot inspect. The zero-padding mask issue is minor: the algorithm does use the mask for alignment, though the attention is not explicitly masked against padded positions, which is worth clarifying but unlikely to change the main story.\n\nThe stress-test concern holds up. The most honest summary is that the paper demonstrates a promising family of models, not a single deployable model with the stated performance.\n\nWho this is for: researchers building FUO or PET/CT decision-support tools, and anyone benchmarking multimodal fusion on small medical cohorts. It deserves a serious referee, but with major revision: report one model configuration chosen by nested CV or per-fold selection, give fold-level AUROC with intervals, add a simple late-fusion baseline, and release code and data or clearly state restrictions.","headline":"A useful new FUO dataset and a broad benchmark, but the headline AUROC range is a per-task best over four feature extractors, not a fixed, reproducible model.","tokens_in":21244,"tokens_out":2495,"would_cite":false,"duration_ms":29037,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"On 416 FUO cases, a tri-modal self-attention network reaches macro-AUROC up to 0.9291, beating all single-modality baselines.","keywords":["fever of unknown origin","18F-FDG PET/CT","multimodal fusion","self-attention","pretrained models","DINOv2","clinical decision support","macro-AUROC"],"falsifier":"Re-run the same MFCN on an external cohort that includes patients who remained undiagnosed after workup, and compare performance separately for diagnosed and undiagnosed patients; if accuracy on diagnosed patients drops when undiagnosed cases are added to the test set, or if the model confidently assigns undiagnosed cases to specific etiologies, the claim that it captures early FUO diagnosis is falsified. A simpler check: have two physicians independently re-adjudicate the 416 final diagnoses and measure label agreement; if agreement is low, the AUROC ceiling itself is questionable.","tokens_in":20224,"feed_emoji":"🩺","tokens_out":8559,"duration_ms":82094,"temperature":0.7,"pith_summary":"MedMimic is a diagnostic pipeline that treats fever of unknown origin as a multimodal classification problem. The paper claims that pretrained vision encoders can turn whole PET/CT scans into compact, semantically meaningful slice features, and that a learnable self-attention network can fuse those features with routine clinical and laboratory data. On 416 patients, the full Clinical+CT+PET configuration reached macro-AUROC (a class-balanced measure of discrimination accuracy) values between 0.8654 and 0.9291 across seven diagnostic tasks, beating conventional machine learning and single-modality deep learning. If the claim holds, a model like this could help clinicians narrow the hundreds of possible FUO causes earlier and reduce the cost and delay of diagnosis.","feed_headline":"Fusing PET, CT, and labs lifts mystery-fever diagnosis to 0.93 AUROC","feed_subtitle":"On 416 FUO cases, the tri-modal network hit macro-AUROC 0.865–0.929 and beat every single-source baseline.","key_machinery":"The load-bearing object is the MFCN (multimodal fusion classification network), built from two modules. The first is a learnable self-attention recalibration layer: CT and PET slice features extracted by pretrained encoders are zero-padded to a common slice count, concatenated with expanded clinical features, and passed through three 1×1 convolutions to form query, key, and value tensors; the resulting attention map weights which spatial and modal information matters for each patient before global average pooling. The second is ResFusion, a six-block residual classifier with batch normalization, dropout, and skip connections that maps the pooled fusion vector to a softmax distribution over etiologic classes. The self-attention is the mechanism that lets imaging and clinical modalities interact dynamically, and the pretrained encoders are what make the high-dimensional volumes tractable on a 416-patient dataset.","core_discovery":"The paper's central claim is that the bottleneck in computer-assisted FUO diagnosis is not the scarcity of deep networks but the mismatch between high-dimensional imaging data and low-dimensional clinical data. MedMimic resolves this by extracting per-slice features with pretrained DINOv2, ViT, and ResNet-18 encoders, stacking them into patient-level tensors with zero-padding, and feeding a fused tensor—imaging plus clinical features—through a learnable self-attention recalibration layer followed by a residual classification network. Across seven tasks (benign vs malignant, immune vs non-immune, infectious vs non-infectious, and finer etiologic splits), this multimodal fusion classification network achieved macro-AUROC scores of 0.8654 to 0.9291, and the tri-modality input consistently outperformed both dual-modality and single-modality configurations. The paper also demonstrates through ablations that the attention layer, dropout, residual connections, and larger hidden dimensions each contribute to the result, and that DINOv2 is the strongest feature extractor.","pith_inferences":["Because the seven tasks are re-groupings of the same final diagnoses, their high AUROC scores may partly reflect shared label structure; a task built on independently adjudicated labels would be a stricter test.","Excluding undiagnosed patients removes exactly the cases that make FUO clinically difficult, so real deployment would face a distribution shift toward harder, less certain patients.","Comparing MedMimic against a nuclear medicine physician reading the same scans would show whether the fusion adds information beyond what trained eyes already extract.","The zero-padding of slices to a common maximum length lets the attention layer see masked positions; a slice-count-aware pooling mechanism would test whether this padding biases the learned weights."],"forward_implications":["On every one of the seven tasks, the Clinical + CT + PET configuration beat the two-modality and single-modality configurations, so dropping a modality costs diagnostic accuracy.","DINOv2 features produced the best result in 59.5% of the evaluated cases, indicating that self-supervised pretraining is the most useful encoder choice for this small medical cohort.","The same encoder-plus-attention pipeline can be lifted to other diagnostic domains with small datasets, since it avoids training deep image encoders from scratch.","The model's output is a probability distribution over etiologic categories, which could be used to prioritize expensive or invasive follow-up tests rather than to replace the physician."],"supporting_citations":[{"why":"Supplies the self-attention mechanism on which the learnable fusion layer is built.","marker":"[23]"},{"why":"Supplies the Vision Transformer encoder used to extract global patch-level features from CT and PET slices.","marker":"[25]"},{"why":"Supplies DINOv2, the self-supervised encoder that produced the best features in the majority of evaluated cases.","marker":"[26]"},{"why":"Supports the use of DINOv2 by showing that self-supervised pretrained encoders transfer well to medical imaging.","marker":"[27]"},{"why":"Supplies ResNet-18, the convolutional encoder used to capture localized features.","marker":"[34]"},{"why":"Supplies PCA, the dimensionality-reduction baseline against which the pretrained encoders are compared.","marker":"[35]"}],"fun_headline_variants":["Tri-modal fusion hits 0.93 AUROC for mystery fevers","MedMimic fuses PET, CT, labs to beat single-source FUO diagnosis","Attention fusion of imaging and labs tops 0.92 AUROC in FUO","Multimodal fusion of PET/CT and clinical data sharpens FUO diagnosis"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the seven diagnostic labels, regrouped from final clinical diagnoses at a single center, are true and complete enough that predicting them measures early diagnostic skill; patients without a definitive diagnosis were excluded, so if those labels are noisy or the excluded cases differ systematically, the reported accuracy overstates real-world performance.","fun_headline_variants_meta":{"raw":{"variants":["Tri-modal fusion hits 0.93 AUROC for mystery fevers","MedMimic fuses PET, CT, labs to beat single-source FUO diagnosis","Attention fusion of imaging and labs tops 0.92 AUROC in FUO","Multimodal fusion of PET/CT and clinical data sharpens FUO diagnosis"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000592,"raw_usage":{"total_tokens":2776,"prompt_tokens":948,"completion_tokens":1828,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":1741}},"tokens_in":564,"tokens_out":1828,"duration_ms":14756,"temperature":1.0,"reasoning_tokens":1741,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T21:26:23.145962+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same MFCN on an external cohort that includes patients who remained undiagnosed after workup, and compare performance separately for diagnosed and undiagnosed patients; if accuracy on diagnosed patients drops when undiagnosed cases are added to the test set, or if the model confidently assigns undiagnosed cases to specific etiologies, the claim that it captures early FUO diagnosis is falsified. A simpler check: have two physicians independently re-adjudicate the 416 final diagnoses and measure label agreement; if agreement is low, the AUROC ceiling itself is questionable.","supporting_citations":[{"cited_title":"Exploring scalable medicalperezgarcia2025raddino image encoders beyond text supervision,","cited_arxiv_id":null,"evidence_quote":"Supports the use of DINOv2 by showing that self-supervised pretrained encoders transfer well to medical imaging."}],"review_version":1}