{"id":"47d09e35-d414-4700-a7d9-5e37f5d4bf68","arxiv_id":"2502.03772","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A hybrid CNN-transformer with mixture-of-experts (HSQformer) yields 95.38% AUC for HCC screening on a multi-center ultrasound test set, beating several baselines and matching senior radiologists.","lead":"This paper introduces HSQformer, a hybrid deep-learning model combining a CNN, a transformer, and a mixture-of-experts module, and evaluates it on ultrasound images for detecting hepatocellular carcinoma (HCC). It reports AUC up to 95.38% on a multi-center test set and performance comparable to senior radiologists, though the tuning and test-set independence deserve scrutiny.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline multi-center AUC is not an independent external validation because the test pool includes the training institution's data without documented patient-level deduplication, and hyperparameters were selected on that same test set.","rationale":"The reader's weakest assumption correctly identifies the same load-bearing issue: the multi-center test set is not a clean external validation because it shares an institution with the training set and was used for hyperparameter selection. The paper's own text in Sections 4.1 and 4.3 supports this concern: the training set is from the First Affiliated Hospital of Sun Yat-sen University, and the multi-center UE-MICAP pool includes that same hospital, while ablations for query number, dimension, stage ratios, and MoE settings are explicitly run on the multi-center test set. No patient-level deduplication is documented. This is not an internal inconsistency in the architecture—the model design and equations are plausible—but it undermines the external validity of the headline AUC and the claimed superiority over baselines. The paper has real strengths: a large retrospective dataset, open-source code, and a reasonable hybrid architecture that ablates well. However, the central generalization claim cannot be accepted at face value without addressing this validation leak. A concrete fix would be to rerun with pre-specified hyperparameters and a truly external, deduplicated test set. Since the reader already recommended conditional acceptance with these caveats, our stress-test does not move the verdict; it reinforces it. We note that the 'first systematic exploration' claim is also overstated given prior HCC ultrasound AI work, but that is secondary to the validation issue.","tokens_in":22067,"tokens_out":2686,"duration_ms":24492,"concrete_test":"Obtain or recompute image-level hashes for all training and test images; verify that no image appears in both training and any test set, and that no two images from the same patient span training and test sets. Then freeze hyperparameters to a configuration chosen without any use of the multi-center test set (e.g., via nested cross-validation on the training cohort only) and re-evaluate HSQformer-B and ConvNext on a truly external multi-center cohort that excludes the First Affiliated Hospital of Sun Yat-sen University from both training and test. Report the resulting AUC difference with DeLong confidence intervals or bootstrap tests; if the difference is not statistically significant or the overlap is non-empty, the headline claim should be downgraded.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that HSQformer-B achieves 95.38% AUC in multi-center testing and thereby outperforms state-of-the-art models—requires the multi-center test set to be a genuinely external, held-out cohort. That condition is not met. Section 4.1 states the UE-MICAP multi-center dataset includes the First, Third, Sixth, and Seventh Affiliated Hospitals of Sun Yat-sen University, while Table 1 records the training set as originating from the First Affiliated Hospital of Sun Yat-sen University. The paper does not document any patient-level or image-level deduplication between training and test cohorts, so a patient scanned at the same institution could appear in both. Moreover, Section 4.3 explicitly states that 'all ablation study results are obtained on the multi-center test set,' and those ablations select the stage ratio, query number, query dimension, number of experts, and Top-K strategy (Figures 5–8, Table 4). Thus the reported 95.38% AUC is not a pre-specified evaluation on a truly independent test set but an estimate influenced by hyperparameter selection on that same data. This selection bias can inflate performance and makes the claimed superiority over ConvNext (94.45% AUC) fragile; the gap may not replicate on a genuinely external cohort.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes HSQformer, a hybrid CNN–ViT architecture with hierarchical sparse querying and Mixture-of-Experts, for ultrasound-based screening of hepatocellular carcinoma. It reports experiments on three retrospective cohorts (single-center, multi-center, high-risk), compares against twelve SOTA models and six radiologists, and releases code. The headline result is HSQformer-B achieving 95.38% AUC on the multi-center test set, outperforming the best baseline (ConvNeXt, 94.45% AUC) and matching senior radiologists while exceeding junior radiologists.","tokens_in":22309,"tokens_out":6419,"duration_ms":55103,"significance":"If the reported multi-center performance were obtained from a genuinely held-out, pre-specified evaluation, the study would be a solid contribution to AI-assisted HCC screening: it uses a sizable retrospective dataset, patient-level cross-validation, three clinical scenarios, a human-machine comparison, and open-source code. The proposed architecture is modular and the ablation coverage is broad. However, the current evaluation protocol contains two load-bearing gaps — possible patient overlap between training and multi-center test sets and model selection directly on that same test set — that make the headline AUC and the claimed superiority over SOTA models unverified. The significance is therefore conditional on correcting the validation.","major_comments":[{"comment":"The multi-center test set is described as the UE-MICAP dataset, which per Section 4.1 includes the First Affiliated Hospital of Sun Yat-sen University — the same institution that provided the entire training set (Table 1). The paper does not document any patient-level or image-level de-duplication between the training cohort and the multi-center test cohort. Consequently, the multi-center test cannot be considered a genuinely external validation set, and a patient scanned at the First Affiliated Hospital could appear in both training and test splits. The same overlap applies to the high-risk patient test, whose data source is also the First Affiliated Hospital. Please either demonstrate explicit patient-level separation (e.g., by reporting the deduplication protocol and the number of excluded patients) or re-compute the results on a test set that excludes all patients from the training institution.","section":"Section 4.1 and Table 1"},{"comment":"Ablation results are explicitly obtained on the multi-center test set ('all ablation study results are obtained on the multi-center test set,' Section 4.3). Figures 5–8 and the accompanying text use that test set to select the stage ratio (2:2:6:2), query number (200), query dimension (384), number of experts (4), and Top-K strategy (Top-2) for HSQformer-B. The reported 95.38% AUC is therefore a post-selection estimate, not a pre-specified evaluation on an independent test set, and the comparison against baselines that did not receive the same test-set tuning is biased in favor of HSQformer. Please provide a nested or separate held-out evaluation (e.g., a validation split for hyperparameter selection and a final test set used only once), or at minimum report the performance of the final configuration on a truly untouched external cohort.","section":"Section 4.3 and Table 4"},{"comment":"The model's reported AUC is much higher on the multi-center test (95.38±0.33) than on the single-center test from a different institution (83.83±0.96). If the multi-center set were a clean external and more heterogeneous cohort, one would generally expect equal or lower performance than on the training-institution distribution, not an 11.5-point improvement. This discrepancy is consistent with the overlap and test-set tuning issues above and should be discussed explicitly. Reporting per-institution AUCs within the multi-center test (especially separating the First Affiliated Hospital from the other sites) would help determine whether the high multi-center number is driven by leakage or by genuine cross-site generalization.","section":"Section 4.2 and Table 2"},{"comment":"The abstract and Section 4.2.2 state that HSQformer 'significantly surpasses' junior radiologists and outperforms SOTA models, but no statistical significance tests are reported for the pairwise comparisons. In Table 2, the multi-center AUC gap between HSQformer-B (95.38±0.33) and ConvNeXt (94.45±0.53) is smaller than the fold-to-fold variability, and in Table 3 the senior/junior averages are based on only three radiologists each. Please provide paired tests (e.g., DeLong test for AUC or a permutation test across folds) and, for the radiologist comparison, the number of readings and a suitable effect-size measure, before claiming statistical significance.","section":"Section 4.2.2 and Tables 2 and 3"}],"minor_comments":[{"comment":"The legend spells the model name 'HSQfomer' in multiple places; this should be corrected to 'HSQformer'.","section":"Figure 9"},{"comment":"The bullet points contain numeric inconsistencies with Table 3. For example, junior radiologists' average Precision is 85.47±2.17 in Table 3, but the text reports 88.62% as the junior value and 87.62% vs. 88.62% for the HSQformer-B comparison. Please reconcile the text with the table.","section":"Section 4.2.2 and Table 3"},{"comment":"Several ablation numbers are read from plots; please include the exact values in a table so that the reported trends are reproducible.","section":"Section 4.3.1 and Figure 6"},{"comment":"The claim of being the 'first systematic exploration of AI-assisted ultrasound screening for HCC' would benefit from a more detailed comparison with prior deep-learning HCC ultrasound studies in Related Work.","section":"Section 1 and Related Work"}],"recommendation":"major_revision","confidential_remarks":"The main risk is that the reported multi-center AUC is a self-fulfilling evaluation given the overlap and test-set tuning. If the authors can provide a clean external validation with patient-level deduplication and pre-specified hyperparameters, the paper could be a useful clinical AI contribution. I would not recommend acceptance without that evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know: this is a serious, unusually thorough retrospective study, but the headline '95.38% AUC in multi-center testing' is not an independent external validation. The multi-center set includes data from the same hospital that supplied the training cohort, and Section 4.3 says all ablation results—query count, dimension, stage ratios, MoE experts, top-k—were obtained on that same multi-center test set. That selection bias can inflate the reported gap over ConvNeXt (94.45% AUC); the 0.93-point difference may not replicate on a truly external cohort.\n\nWhat is genuinely new: HSQformer stacks hierarchical learned-query cross-attention (Q-former/Perceiver style) with MoE routing on top of dual CNN and ViT feature extractors, and applies that combination to HCC ultrasound screening. The pieces are known, but the specific architecture and application are not in the cited literature. The evaluation is substantial: 11,149 cases, 19,464 images, three clinical scenarios, and 5-fold patient-level cross-validation. The code and checkpoints are open-sourced, which I value. The human-machine comparison is another real strength: six radiologists read the high-risk test set, and HSQformer-B matches seniors and clearly beats juniors. Those results are meaningful even if the external-validation story weakens.\n\nThe soft spots are real but localized. The biggest is the validation protocol: the final model configuration was chosen using the same multi-center test set, so the reported AUC is a selected estimate, not a pre-specified measurement. Table 1 lists the First Affiliated Hospital of Sun Yat-sen University as both the training source and part of the multi-center test, with no documented patient-level deduplication. That makes 'multi-center external validation' partially internal. I am not suggesting anything improper—the single-center test and CV results show real signal—but the superiority claim over strong baselines is fragile. Minor issues: the 'first systematic exploration' claim is asserted without a review of prior HCC ultrasound AI work; there is no significance testing between models; the radiologist comparison uses only the high-risk set and a small number of readers.\n\nWho this is for: anyone working on AI-assisted ultrasound or clinical validation of medical image models. It deserves a serious referee. The right next step is not rejection but a revision that documents patient-level separation across all cohorts, moves hyperparameter selection to nested CV or a pre-specified protocol, adds statistical tests, and softens the 'first' claim. If they do that, this becomes a useful benchmark paper.","headline":"A serious, unusually thorough clinical AI study whose headline multi-center AUC is undercut because the same test set was used for hyperparameter selection and patient-level deduplication is not documented.","tokens_in":22871,"tokens_out":2841,"would_cite":false,"duration_ms":26703,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a hierarchical sparse-query transformer for ultrasound HCC screening reaches 95.38% multi-center AUC and matches senior radiologists while beating junior ones.","keywords":["hepatocellular carcinoma","ultrasound screening","vision transformer","mixture of experts","computer-aided diagnosis","hierarchical sparse query","CNN-ViT hybrid","medical image classification"],"falsifier":"Run the same 5-fold protocol with a test set drawn exclusively from hospitals that contributed no training images, with hyperparameters frozen before any test evaluation; if HSQformer-B then no longer beats ConvNext or SwinTransformer on AUC, the paper's central claim of superior generalization fails.","tokens_in":21868,"feed_emoji":"🩺","tokens_out":4246,"duration_ms":39600,"temperature":0.7,"pith_summary":"The paper proposes HSQformer, a hybrid architecture that combines convolutional local features with vision-transformer global context through a hierarchical set of learned query tokens and sparse mixture-of-experts routing. It claims this model outperforms state-of-the-art CNN and ViT baselines on ultrasound screening for early hepatocellular carcinoma, reaching 95.38% AUC in a multi-center test and 88.32% AUC on a high-risk hepatitis cohort. The authors also report that HSQformer matches the accuracy of senior radiologists and clearly outperforms junior radiologists, which matters because conventional ultrasound sensitivity depends heavily on reader experience. If these results hold, the model offers a route to standardizing HCC screening and reducing missed early cancers.","feed_headline":"Ultrasound AI matches senior radiologists for liver cancer screening","feed_subtitle":"HSQformer hits 95.38% AUC across eight hospitals and beats junior radiologists on high-risk cases.","key_machinery":"The carrying mechanism is the Cross-Self-attention Mixed experts (CSM) block: cross-attention lets learned query tokens pull information from CNN and ViT feature maps, self-attention refines the pooled tokens, and a top-k Mixture-of-Experts layer routes each token to a small subset of specialist MLPs. CSM blocks are stacked in four stages that query feature maps at strides 4, 8, 16, and 32, so low-level texture and high-level semantic information are both retained. The learned query count and dimension act as a compression bottleneck that reduces redundancy in the long token sequences produced by the early stages.","core_discovery":"On the paper's own terms, the discovery is that CNN and ViT features can be integrated hierarchically, without changing the backbone structures, by projecting multi-scale feature maps into a shared latent space and letting a small set of learned query tokens extract diagnosis-relevant information at each stage. The resulting HSQformer-B reaches 95.38% AUC (88.29% accuracy, 90.38% recall) on eight-hospital multi-center data, outperforming all compared CNNs and ViTs, and 94.09% recall on high-risk patients, where it exceeds every radiologist in the comparison and matches senior-level diagnostic quality. The paper reads this as evidence that AI-assisted ultrasound screening can reach clinical-grade accuracy while being computationally sparse.","pith_inferences":["Because the headline hyperparameters were tuned on the same multi-center test set that produced the 95.38% AUC, a fair external evaluation on institutions never seen during development would be needed to know how much of the gain is architectural rather than selection.","The reported robustness to stage-ratio changes suggests the core design is insensitive to depth allocation; a natural test is freezing the architecture and tuning only query count and dimension on a new ultrasound dataset.","The attention heatmaps suggest the model focuses on perilesional tissue; adding radiologist lesion bounding boxes as supervision could test whether explicit localization improves both accuracy and clinical trust."],"forward_implications":["A deployed HSQformer could give non-specialist ultrasound operators senior-level diagnostic support, potentially raising early HCC detection in settings without expert radiologists.","The high recall (94.09%) on high-risk hepatitis patients directly targets the screening failure mode that matters most: missed early cancers in surveillance populations.","Sparse MoE routing means only a subset of experts runs at inference, so the accuracy gain does not require a proportional increase in per-image computation.","The four-stage query design should transfer to other lesion-classification tasks where both texture detail and global context matter, since it needs no changes to the backbone.","Matching senior radiologists on a multi-center set is evidence, if replicated prospectively, that AI-assisted screening could standardize HCC surveillance across institutions."],"supporting_citations":[{"why":"Serves as the CNN feature extractor whose local multi-scale features are fused into the latent space.","marker":"[5]"},{"why":"Serves as the ViT feature extractor providing global hierarchical features that the query tokens attend to.","marker":"[6]"},{"why":"Supplies the fine-tuning protocol and pre-training configuration used for fair comparison across models.","marker":"[26]"},{"why":"Provides the learned-query mechanism that HSQformer adapts for latent-space querying.","marker":"[38]"},{"why":"Supplies the multi-gate mixture-of-experts idea behind the sparse expert routing.","marker":"[39]"},{"why":"Cited as an inspiration for the learnable querying and reconstruction-style representation learning in HSQformer.","marker":"[44]"},{"why":"Used as a state-of-the-art clinical deep-learning baseline from thyroid nodule diagnosis that HSQformer must beat.","marker":"[46]"}],"fun_headline_variants":["AI ultrasound model rivals senior radiologists for liver cancer","HSQformer matches senior radiologists in liver ultrasound","Sparse transformer AI hits 95% AUC for liver cancer","Hybrid AI model excels at early liver cancer screening"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the multi-center test set is genuinely external to training, yet it includes data from the same hospital that supplied the training cohort and the paper does not document patient-level deduplication between the two.","fun_headline_variants_meta":{"raw":{"variants":["AI ultrasound model rivals senior radiologists for liver cancer","HSQformer matches senior radiologists in liver ultrasound","Sparse transformer AI hits 95% AUC for liver cancer","Hybrid AI model excels at early liver cancer screening"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00062,"raw_usage":{"total_tokens":2869,"prompt_tokens":934,"completion_tokens":1935,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":550,"completion_tokens_details":{"reasoning_tokens":1870}},"tokens_in":550,"tokens_out":1935,"duration_ms":13107,"temperature":1.0,"reasoning_tokens":1870,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T00:47:09.532558+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same 5-fold protocol with a test set drawn exclusively from hospitals that contributed no training images, with hyperparameters frozen before any test evaluation; if HSQformer-B then no longer beats ConvNext or SwinTransformer on AUC, the paper's central claim of superior generalization fails.","supporting_citations":[{"cited_title":"A convnet for the 2020s,","cited_arxiv_id":null,"evidence_quote":"Serves as the CNN feature extractor whose local multi-scale features are fused into the latent space."},{"cited_title":"Swin transformer: Hierarchical vision transformer using shifted windows,","cited_arxiv_id":null,"evidence_quote":"Serves as the ViT feature extractor providing global hierarchical features that the query tokens attend to."},{"cited_title":"Cswin transformer: A general vi- sion transformer backbone with cross-shaped windows,","cited_arxiv_id":null,"evidence_quote":"Supplies the fine-tuning protocol and pre-training configuration used for fair comparison across models."},{"cited_title":"Learned queries for efficient local attention,","cited_arxiv_id":null,"evidence_quote":"Provides the learned-query mechanism that HSQformer adapts for latent-space querying."},{"cited_title":"Modeling task relationships in multi-task learning with multi-gate mixture-of-experts,","cited_arxiv_id":null,"evidence_quote":"Supplies the multi-gate mixture-of-experts idea behind the sparse expert routing."},{"cited_title":"Masked autoencoders are scalable vision learners,","cited_arxiv_id":null,"evidence_quote":"Cited as an inspiration for the learnable querying and reconstruction-style representation learning in HSQformer."},{"cited_title":"Deep learning-based ar- tificial intelligence model to assist thyroid nodule diagno- sis and management: a multicentre diagnostic study,","cited_arxiv_id":null,"evidence_quote":"Used as a state-of-the-art clinical deep-learning baseline from thyroid nodule diagnosis that HSQformer must beat."}],"review_version":1}