{"id":"ceff7b11-521a-4b11-be9b-35b01989126b","arxiv_id":"2502.08774","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Test-time adaptation with a normative atlas prior improves fetal subcortical segmentation in 3D ultrasound, though the main quantitative evaluation is partly circular.","lead":"Test time adaptation lets a deep learning model adjust to new 3D ultrasound images and improves its segmentation of fetal brain structures when scanners or gestational weeks change. Adding a healthy brain atlas as a guide appears to help, but the strongest quantitative evidence for this comes from labels that were made with the same atlas.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"EntropyKL is evaluated against atlas-propagated labels from the same atlas that supplies its prior; the reported Dice gains may be inflated by circularity.","rationale":"The paper is a clear empirical study of test-time adaptation for fetal brain ultrasound segmentation, and the general finding that TTA helps under domain shift is plausible: even if the labels are atlas-propagated, the improvement of TENT over the base model is a meaningful demonstration. However, the paper's central novelty is EntropyKL, and its quantitative support is directly compromised by the shared origin of the prior and the reference labels. The simulated and gestational-age experiments use atlas-propagated annotations from an atlas that also defines the KL prior, so EntropyKL is effectively rewarded for matching the label-generation process. The unseen-scanner evaluation is qualitative and selected, so it cannot break the circularity. This is not an accusation of misconduct; it is a standard independence problem in evaluation design. The proposed test, using manual annotations independent of the atlas, is feasible because the source model paper [6] reports manual annotations and the authors could apply the same TTA protocol to those data. If the manual-label evaluation reproduces EntropyKL's advantage, the paper's claim would be substantially strengthened. Without it, the specific EntropyKL claim remains conditional, matching the reader's verdict.","tokens_in":6824,"tokens_out":3940,"duration_ms":42134,"concrete_test":"Obtain independent manual annotations for a subset of the test volumes used in the simulated-shift and gestational-age experiments (or for held-out volumes from [6] with the same preprocessing), and recompute Dice for source, TENT, and EntropyKL. If EntropyKL's margin over TENT does not persist on manual labels, the reported improvement is an artifact of atlas-derived labels.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The decisive quantitative evidence for the central novelty (EntropyKL) is circular. EntropyKL's loss (Eq. 3) adds a KL term to TENT that penalizes deviation of predicted class proportions from the normative atlas [11]. The simulated-shift and gestational-age experiments in Section III evaluate Dice against 'propagated annotations from an atlas [6],[11]'. If those reference labels are produced by propagating the same atlas segmentation to each volume, then the ground-truth class ratios are the very ratios EntropyKL is regularized toward, while the TENT baseline is not. This gives EntropyKL an information advantage unrelated to anatomical correctness and can inflate its measured Dice improvement. The unseen-scanner results are only qualitative and are shown for the highest-entropy slices, so they do not independently establish the advantage. The claim that EntropyKL gives the best segmentations therefore rests on an evaluation in which the prior and the reference standard share the same origin.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates test-time adaptation (TTA) for subcortical segmentation of the fetal brain in 3D ultrasound. Using a pretrained 3D UNet as the source model, the authors evaluate TENT, a proposed layer-selection method (LayerInspect), and a proposed atlas-prior method (EntropyKL) under simulated domain shifts, gestational-age variation, and unseen scanner data. They report that TTA methods improve over the source model, with EntropyKL often performing best, and they release their code.","tokens_in":6988,"tokens_out":5989,"duration_ms":53468,"significance":"If the results hold, the paper would demonstrate a practical approach to adapting fetal-brain ultrasound segmentation models to new scans or scanners without manual labels, which is valuable given the acquisition variability in freehand ultrasound. The code release and the use of a normative atlas as a prior are constructive contributions. However, the central quantitative evidence for EntropyKL is compromised by a likely circularity between the atlas used as the prior and the atlas-propagated labels used for evaluation, and the unseen-scanner evidence is qualitative and selected. The paper's broader claim that TTA improves over the source model is plausible and supported by paired tests, but the specific novelty (EntropyKL) needs stronger, independent validation.","major_comments":[{"comment":"The quantitative evaluation of EntropyKL is confounded by the shared origin of the atlas prior and the reference labels. The KL term in Eq. (3) penalizes deviations of the predicted class proportions from the atlas [11] ratios, while the Dice scores for the simulated-shift and gestational-age experiments are computed against 'propagated annotations from an atlas [6], [11]' (Section III). If those reference labels are produced by warping the same atlas segmentation to each volume, then the ground-truth class proportions are the very proportions EntropyKL is regularized toward; TENT and LayerInspect receive no such information. This can inflate EntropyKL's measured Dice improvement independently of anatomical correctness. Please clarify the provenance of the propagated annotations; if they derive from the same atlas, either evaluate on manual annotations, or demonstrate that the prior remains beneficial when the reference class ratios are deliberately varied, or restrict the claim of EntropyKL's advantage to the qualitative unseen-scanner results.","section":"Section III (Simulated Domain Shifts and Gestational Age) and Eq. (3)"},{"comment":"The claim that EntropyKL gives the best segmentations on unseen scanners is not quantitatively supported. The comparison is qualitative, shows only one slice per scanner, and the slices are selected as the highest-entropy predictions of the base model, which may preferentially display the effect of entropy minimization. Manual segmentations or a blinded reader study would be required to substantiate the advantage of EntropyKL over the other TTA methods in this setting. Without this, the abstract's statement that EntropyKL demonstrates improvements across real domain shifts rests on the simulated experiments, which are subject to the circularity concern above.","section":"Section IV (Unseen Datasets) and Fig. 5"}],"minor_comments":[{"comment":"The phrase 'propagated annotations from an atlas [6], [11]' is ambiguous; specify which atlas (e.g., [11]) and whether the same atlas is used in the EntropyKL prior.","section":"Section III, Simulated Domain Shifts"},{"comment":"The text states that 'TTA methods significantly increases the performance' but reports no test statistic or p-value for this figure; please add error bars and significance tests for the gestational-age comparisons.","section":"Section IV, Gestational Age"},{"comment":"The caption says the right-hand panel shows 'The same plane from the US atlas [11] at different ages'; indicate whether these slices are the source of the propagated annotations used as ground truth.","section":"Fig. 4 caption"},{"comment":"The phrase 'normative altas' is a typo for 'normative atlas'.","section":"Section V, Discussion"},{"comment":"Define N, l, n, and the notation used for the Taylor approximation; also clarify that the activations are L2-normalized before the difference is computed.","section":"Eq. (2)"},{"comment":"The hyperparameter choices (λ=1, m=1, number of backward passes) are selected based on sensitivity experiments in Fig. 3; state whether these selections were made on a validation set separate from the test data.","section":"Section III, Implementation"}],"recommendation":"major_revision","confidential_remarks":"The circularity concern is the main barrier to acceptance. If the authors can provide independent ground truth (e.g., manual annotations for a subset) or clearly restrict their claims, the paper could be suitable. The code release is a positive aspect. The paper fits the journal's scope but the central novelty needs stronger evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid empirical study of test-time adaptation for subcortical fetal brain segmentation in 3D ultrasound. The general result — that TENT-style adaptation improves Dice over the frozen source model on simulated shifts and gestational-age drift — is plausible and backed by paired tests. The paper's own contribution, EntropyKL, is another matter: its advantage over TENT is measured against labels that were propagated from the same atlas that supplies its KL prior. That makes the reported improvement close to circular.\n\nWhat's new here is the specific application and the combination of off-the-shelf pieces: TENT, a class-ratio KL term borrowed from source-free DA, and a layer-selection rule from pruning. None of those are new individually, but putting them on fetal ultrasound is reasonable and the benchmark across simulated, gestational-age, and cross-vendor shifts is useful. The code is released, which is a plus.\n\nThe main soft spot is exactly what the reader flagged. Section III uses 'propagated annotations from an atlas [6],[11]' for both the simulated-shift and gestational-age experiments, while Eq. 3 pulls predictions toward the class ratios of atlas [11]. If the ground-truth volumes are produced by propagating the atlas segmentation, then the reference class proportions are the same proportions EntropyKL is regularized toward. TENT is not given that information, so EntropyKL wins partly by construction. The unseen-scanner results don't fix this: they're qualitative, shown for the highest-entropy slices, and manually selected. The LayerInspect method is less affected by this specific circularity, but its contribution is minor and its sensitivity analysis is thin.\n\nI don't think the general TTA conclusion is in doubt — the improvement over the source model is consistent and makes sense. But the central claim that EntropyKL gives the best segmentations is not supported by the quantitative evidence as presented. The fix is straightforward: evaluate EntropyKL on a held-out set with independent manual annotations, and ideally compare against TENT on those same labels. Also worth reporting how many backward passes and lambda were chosen without peeking at test performance; right now several hyperparameters look test-tuned.\n\nBottom line: this deserves a serious referee, but with the expectation of major revision. The application is clinically relevant and the benchmark is valuable, yet the headline method needs a cleaner evaluation before the numbers can be trusted. I'd bring it to reading group as a case study in evaluation design as much as TTA.","headline":"Useful TTA benchmark for fetal US segmentation, but the EntropyKL claim is undermined by an evaluation that shares the same atlas as its prior.","tokens_in":7543,"tokens_out":2432,"would_cite":false,"duration_ms":22565,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Test-time adaptation lets a pretrained fetal-brain segmentation model adapt to new ultrasound scans without manual labels, and adding a normative-atlas volume prior makes the adaptation more effective.","keywords":["Test Time Adaptation","Ultrasound","Segmentation","Fetal brain","Subcortical structures","Entropy minimization","Normative atlas","Domain shift"],"falsifier":"Evaluate EntropyKL on the unseen scanner volumes using manual annotations, or on the simulated-shift volumes using manual annotations instead of atlas-propagated labels; if the Dice advantage over TENT shrinks or disappears, the atlas prior is not contributing independent anatomical information.","tokens_in":6638,"feed_emoji":"🧠","tokens_out":8039,"duration_ms":65184,"temperature":0.7,"pith_summary":"The paper asks whether a segmentation model trained on one set of 3D fetal-brain ultrasound scans can be adapted at test time to new scans that look different, without any manual labels for the new data. It shows that test-time entropy minimization improves Dice scores over the unadapted model under simulated rotations, scaling, smoothing, and gamma changes, and also across gestational weeks and unseen scanner vendors. The paper's proposed EntropyKL method adds a KL-divergence term that penalizes deviation of predicted class proportions from a normative fetal-brain atlas, and reports that this prior yields the most consistent segmentations, especially on unseen scanners. If correct, this would let automated subcortical segmentation models be deployed to new acquisition setups without retraining or re-annotation.","feed_headline":"Test-time adaptation boosts fetal-brain segmentation on unseen scans","feed_subtitle":"Adapting each new scan on the fly with an atlas prior keeps segmentations accurate without manual labels.","key_machinery":"The central objects are the entropy-minimization objective of TENT, which updates only batch-normalization parameters to reduce prediction entropy on target data, and the proposed EntropyKL extension, which appends a KL divergence between the predicted class proportion $\\hat{\\tau}_t$ and the atlas proportion $\\tau_t$ from the normative fetal-brain atlas, with weight $\\lambda$. A second variant, LayerInspect, selects the $m$ layers whose activation magnitudes differ most between source and target using a Taylor approximation, and updates only those layers. The atlas prior is the load-bearing addition: it supplies an expected volume ratio for the four subcortical classes and prevents adaptation from collapsing to trivial solutions, while still allowing the model to learn locations.","core_discovery":"The central claim is that test-time adaptation can overcome domain shift in 3D fetal ultrasound segmentation, and that the proposed EntropyKL objective — TENT's entropy loss plus $\\lambda \\, \\mathrm{KL}(\\hat{\\tau}_t \\| \\tau_t)$, where $\\hat{\\tau}_t$ is the average predicted class probability and $\\tau_t$ is the class proportion from a normative atlas — adapts a frozen pretrained 3D U-Net better than TENT or LayerInspect alone. The authors report that all TTA methods significantly outperform the source model on simulated domain shifts, that TTA improves performance at gestational ages far from the training distribution, and that EntropyKL gives the most anatomically plausible segmentations on unseen scanner data, where no manual labels exist.","pith_inferences":["If the same atlas is doing double duty as both prior and evaluation label, then the reported Dice gains for EntropyKL over TENT may partly reflect label leakage; an independent manual-annotation evaluation would settle how much of the gain is real.","The authors note the atlas encodes healthy anatomy; a direct extension would test EntropyKL on fetuses with structural anomalies, where the volume prior will compete with the entropy signal and may mask pathology.","The single-volume adaptation result implies a practical deployment recipe: adapt each incoming freehand scan before inference, a protocol that could be validated prospectively without changing the acquisition workflow."],"forward_implications":["TTA with entropy minimization significantly improves Dice over the unadapted source model for simulated ultrasound domain shifts such as rotations, scaling, smoothing, and gamma correction.","Adapting per single volume rather than per batch yields better results, matching the observation that each freehand ultrasound scan effectively forms its own domain.","EntropyKL, with the atlas class-ratio prior, gives the most consistent segmentations among the TTA methods, including on data from two unseen scanner vendors where only qualitative evaluation is possible.","For gestational-age shifts, TTA improves Dice by about 0.1 on the earliest weeks where the source model performs worst, and never reduces performance.","Histogram matching, a standard preprocessing fix, fails on morphometric shifts such as gestational-age differences, whereas TTA handles them."],"supporting_citations":[{"why":"Supplies the pretrained 3D U-Net source model that TTA adapts and defines the four subcortical classes.","marker":"[6]"},{"why":"Provides the TENT entropy-minimization baseline and the batch-normalization-only update rule.","marker":"[16]"},{"why":"Supplies the normative atlas used as the class-ratio prior in EntropyKL and as the source of propagated annotations for evaluation.","marker":"[11]"},{"why":"Introduces the class-ratio prior idea that EntropyKL adapts into the TTA loss.","marker":"[1]"},{"why":"Motivates LayerInspect by pruning-based selection of layers with the largest activation differences.","marker":"[4]"},{"why":"Provides the Taylor approximation used to score and compare layer activations in LayerInspect.","marker":"[10]"},{"why":"Supplies the training and evaluation volumes with gestational-age labels.","marker":"[12]"},{"why":"Provides the unseen scanner datasets used for qualitative cross-scanner evaluation.","marker":"[18]"}],"fun_headline_variants":["Atlas-prior TTA keeps fetal brain segmentations accurate on unseen scans","On-the-fly adaptation with atlas prior boosts fetal brain US segmentation","Test-time adaptation improves fetal brain subcortical segmentation in 3D US","Atlas-informed adaptation keeps fetal brain scans sharp without labels","TTA with an atlas prior makes fetal brain segmentation robust to domain shift"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation of EntropyKL depends on the assumption that the atlas-propagated annotations used as ground truth in the simulated and gestational-age experiments are valid and independent of the same atlas that provides the EntropyKL prior; if they are not independent, the reported Dice improvements for EntropyKL could be inflated.","fun_headline_variants_meta":{"raw":{"variants":["Atlas-prior TTA keeps fetal brain segmentations accurate on unseen scans","On-the-fly adaptation with atlas prior boosts fetal brain US segmentation","Test-time adaptation improves fetal brain subcortical segmentation in 3D US","Atlas-informed adaptation keeps fetal brain scans sharp without labels","TTA with an atlas prior makes fetal brain segmentation robust to domain shift"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000975,"raw_usage":{"total_tokens":4110,"prompt_tokens":877,"completion_tokens":3233,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":493,"completion_tokens_details":{"reasoning_tokens":3139}},"tokens_in":493,"tokens_out":3233,"duration_ms":20832,"temperature":1.0,"reasoning_tokens":3139,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T23:42:56.616157+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate EntropyKL on the unseen scanner volumes using manual annotations, or on the simulated-shift volumes using manual annotations instead of atlas-propagated labels; if the Dice advantage over TENT shrinks or disappears, the atlas prior is not contributing independent anatomical information.","supporting_citations":[{"cited_title":"NeuroImage 254, 119117 (2022)","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained 3D U-Net source model that TTA adapts and defines the four subcortical classes."},{"cited_title":"In: International Conference on Learning Representations (2021)","cited_arxiv_id":null,"evidence_quote":"Provides the TENT entropy-minimization baseline and the batch-normalization-only update rule."},{"cited_title":"Nature 623 (10 2023)","cited_arxiv_id":null,"evidence_quote":"Supplies the normative atlas used as the class-ratio prior in EntropyKL and as the source of propagated annotations for evaluation."},{"cited_title":"Medical Image Analysis 82, 102617 (2022)","cited_arxiv_id":null,"evidence_quote":"Introduces the class-ratio prior idea that EntropyKL adapts into the TTA loss."},{"cited_title":"Medical Image Analysis 81, 102583 (2022)","cited_arxiv_id":null,"evidence_quote":"Motivates LayerInspect by pruning-based selection of layers with the largest activation differences."},{"cited_title":"In: 5th In- ternational Conference on Learning Representations, ICLR 2017 (2017)","cited_arxiv_id":null,"evidence_quote":"Provides the Taylor approximation used to score and compare layer activations in LayerInspect."},{"cited_title":"American journal of obstetrics and gynecology 218(2), S630–S640 (2018)","cited_arxiv_id":null,"evidence_quote":"Supplies the training and evaluation volumes with gestational-age labels."},{"cited_title":"Medical Image Analysis 94, 103147 (2024)","cited_arxiv_id":null,"evidence_quote":"Provides the unseen scanner datasets used for qualitative cross-scanner evaluation."}],"review_version":1}