{"id":"ffdbee0c-ed15-466c-aadd-b1bddf4fa918","arxiv_id":"2608.10903","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"An OOD-aware variational head on a frozen U-Net gives uncertainty maps that align with segmentation error better than deep ensembles or PHiSeg under adult-to-pediatric shift.","lead":"VIDS-Seg adds an out-of-distribution-aware uncertainty layer to cardiac ultrasound segmentation, training on adult videos and testing on children without pediatric labels. It reports uncertainty maps that track segmentation errors better than ensemble or PHiSeg baselines, especially for infants, and more stable ejection-fraction estimates downstream.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (8) conditions the adaptive prior on target labels y*_hw, but Algorithm 1 has no access to them; the reported uncertainty advantage may depend on a label-conditioned posterior that is unavailable at deployment.","rationale":"I read the paper as claiming that an adult-trained segmentation model plus the VIDS-Seg head can produce reliable spatial uncertainty on pediatric data without pediatric labels. For that to be true, Stage-2 training must optimize a variational distribution that is computable at test time from embeddings alone. Eq. (8) breaks this: the energy defining the adaptive prior uses y* on the test branch, while Algorithm 1 has no y*. The original VIDS definition in Eq. (2) integrates over y in the test term, so this is not a notational shortcut; it is an essential difference. The issue is not a mild calibration concern or a disagreement with current consensus: if the implementation matches the text, the trained q approximates a posterior that conditions on unobserved labels, and there is no reason to expect the same q to produce OOD-widened variances when evaluated without those labels. That makes the headline result uninterpretable. If the implementation deviates from the text, the method is under-specified and cannot be independently reproduced, which still blocks acceptance under a reject/revise standard. I agree with the reader that the synthetic-subenvironment design is also a genuine transfer risk, but it is secondary: even if adult subsamples perfectly mimic infant shift, the current specification does not show how test-time h_gamma can realize the label-conditioned energy. The temperature-scaling result, if replicated under a corrected energy, would be a meaningful contribution, and the EF/AUROC analysis is a useful demonstration of downstream value; those parts do not compensate for the missing definition of the core mechanism. The requested ablation directly separates the two readings of Eq. (8) and would settle whether the concern blocks the empirical claims as reported.","tokens_in":13999,"tokens_out":6259,"duration_ms":65450,"concrete_test":"Run the Stage-2 training exactly as described, but replace the second term in Eq. (8) with the label-marginalized energy: (1/HW) * sum over h,w of sum over c in {0,1} of p(c | x*, theta) * log p(c | x*, theta), keeping all other hyperparameters and the same adult training set. Then re-evaluate the infant subgroup in Table 1. If the raw and temperature-scaled NCC values remain near 0.62 and 0.61, the reported advantage does not depend on oracle target labels and the concern is resolved; if they fall toward the Ensemble values (0.39 and 0.54), the central claim relies on a label-conditioned energy that is unavailable at deployment.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that a lightweight head trained on adult data alone can flag pediatric OOD cases—depends on the inference network h_gamma outputting a variational posterior q_phi(theta; x*) that is wider for genuinely shifted test images. As written, that posterior approximates p(theta|x1:N,x*) defined by the energy in Eq. (8), whose second term is (1/HW) * sum over h,w of log p(y*_hw | x*, theta). This requires the target segmentation y* for the test covariate. Algorithm 1 supplies neither y* nor any label-dependent feature to h_gamma; it receives only embeddings. In the original VIDS energy, Eq. (2), the corresponding term is an integral over y, not an observed label. Replacing that integral by a log-likelihood evaluated at y* changes the object of inference from a prior conditioned on x* to a posterior conditioned on (x*, y*). If the code or training run actually used y*, the q learned in Stage 2 has been trained to reproduce a label-conditioned posterior while being evaluated at test time without labels; the reported NCC/EF/AUROC advantages may then reflect a train/test mismatch rather than a mechanism that transfers. If the code instead marginalizes over labels or uses a surrogate, that detail is absent from the specification. This is load-bearing because every reported downstream benefit is mediated by the variance that h_gamma outputs at test time. The temperature-scaling comparison and the EF/AUROC findings cannot rescue the central claim if the uncertainty signal is generated by a procedure that is undefined at deployment.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes VIDS-Seg, an extension of the VIDS framework to dense image segmentation. A frozen U-Net embedding is combined with a stochastic 1x1 prediction head whose parameters are drawn from a variational posterior produced by an inference network conditioned on training-set and test-set embeddings. The method is trained on adult EchoNet-Dynamic data using synthetic environments formed by bootstrap subsampling and evaluated zero-shot on pediatric EchoNet-Pediatric. The paper reports that VIDS-Seg matches baseline segmentation accuracy across age groups, achieves substantially higher pixel-wise NCC between predictive entropy and segmentation error for both non-infant and infant subgroups, that this advantage persists after temperature scaling, and that it translates into lower EF MAE and higher AUROC for cardiac malfunction detection in infants.","tokens_in":14316,"tokens_out":10896,"duration_ms":107744,"significance":"If the method and results are correct, the paper makes a useful contribution: it provides a practical way to make OOD-aware uncertainty quantification tractable for dense segmentation, and it gives empirical evidence that uncertainty quality can be improved without pediatric labels. The comparison against deep ensembles and PHiSeg, the use of a public adult/pediatric dataset pair, the Wilcoxon signed-rank tests, and the temperature-scaling analysis are all strengths. However, the method section contains a load-bearing inconsistency between the energy function and the inference algorithm, together with an ambiguity in the test-side embedding, and these issues must be resolved before the empirical claims can be accepted.","major_comments":[{"comment":"The adaptive prior in Eq. (8) contains the term (1/HW) * sum_{h,w} log p(y*_{hw} | x*, theta), which requires the ground-truth segmentation of the test image. Algorithm 1, however, provides only x* to the inference network h_gamma (steps 1-3) and never receives y*. With this objective, the variational distribution q_phi is trained to approximate p(theta | x1:N, x*, y*), a label-conditioned posterior, while at test time it is evaluated from x* alone. Since y* is not part of the input to h_gamma, the target posterior is not a function of the network's input, and the trained network cannot consistently reproduce it. This directly affects the variance output by h_gamma, which is the signal behind every reported NCC/EF/AUROC result. Please either replace the y* term in Eq. (8) with a label-free marginalization, for example a Monte Carlo average over the predictive distribution as in the original VIDS energy of Eq. (2), and re-run the experiments, or explicitly describe how labels are used during training and justify why the label-conditioned posterior is computable at test time.","section":"Section 3.2, Eq. (8) vs Algorithm 1"},{"comment":"The text states that the inference network input has dimension 4D, formed from the context summary in R^{2D} and the test embedding g_xi(x*). But g_xi(x*) is defined as a dense map in R^{D x H x W}, and Algorithm 1 step 2 does not spatially reduce it. A fully-connected h_gamma cannot take a spatial map as input, and the stated dimensions do not add up (2D + D is 3D, not 4D). Please specify the exact test-side pooling operation and the precise vector passed to h_gamma; this is needed to reproduce Algorithm 1.","section":"Section 3.2, spatial aggregation and Algorithm 1"},{"comment":"The infant subgroup is identified as OOD after observing that all models degrade most on that group, and the uncertainty and downstream analyses are then performed only on this selected subset. This is a selection on the outcome: the subgroup used to evaluate OOD detection is chosen using the same test data on which uncertainty quality is measured. To support the claim that VIDS-Seg can flag silent failures for underrepresented subgroups at deployment, the OOD subgroup should either be prespecified, or the selection should be made on a separate development set, or a sensitivity analysis across all age strata should be provided.","section":"Section 4.2.1-4.2.2"}],"minor_comments":[{"comment":"The metric is introduced as the NCC 'between the per-pixel entropy map and a binary error map', but the error map is then defined as the pixel-wise cross entropy between mean logits and the ground-truth mask. Please clarify which error map was actually used for the reported numbers.","section":"Section 4.2.2"},{"comment":"The ejection fraction is described as computed with Simpson's biplane method, but only a single apical four-chamber view is used. Please clarify whether a single-plane modification was applied, since the standard biplane method requires two orthogonal views.","section":"Section 4.1.2 and Eq. (11)"},{"comment":"Segmentation accuracy is compared across age groups only descriptively; please add pairwise statistical tests or confidence intervals for DSC and HD95 to support the claim that VIDS-Seg 'matches' the baselines.","section":"Figure 2"},{"comment":"The text says temperature scaling is applied to 'all three baselines', but Table 1 also includes temperature-scaled VIDS-Seg; please rephrase to make clear that VIDS-Seg is included.","section":"Section 4.2.2 / Table 1"},{"comment":"Algorithm 1 (step 10) says the entropy map is computed from the per-sample maps {p^(s)}, while the text defines H_p from the mean probability sbar_p; please align the formula with the algorithm.","section":"Algorithm 1 and Section 4.2.2"}],"recommendation":"major_revision","confidential_remarks":"The main issue is not the empirical comparison but the specification of the method: if the authors can clarify or correct the energy function and the test-side embedding, the paper may be salvageable. The post hoc selection of the OOD subgroup is an additional concern. I would want to see precise pseudocode or code before accepting the reported results."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about arXiv:2608.10903. The first is that it does something genuinely useful: it makes VIDS tractable for dense segmentation by putting the variational distribution only over a 1×1 prediction head on top of a frozen U-Net, and it evaluates that on a serious adult-trained → pediatric zero-shot setup with age stratification. The second is that the method as written has a load-bearing hole. Equation (8) defines the energy with log p(y*_hw | x*, θ) — the test label is in the energy — but Algorithm 1 never receives y*; it only sees x* and the context set. In the original VIDS the corresponding term is an integral over y, not a label. So either the code trained on a label-conditioned posterior and the test-time forward is mismatched, or the equation is wrong. Either way, the paper doesn't specify what the inference network actually optimizes, and every reported benefit (NCC, EF, AUROC) is downstream of that unspecified mechanism. That alone justifies a reject in current form.\n\nWhere credit is due: the prediction-head decomposition is a clean and practical answer to the intractability of full-network VI, and the empirical design is careful — adult-only training, zero-shot pediatric evaluation, five age strata, a deep-ensemble and PHiSeg baseline, temperature scaling to check that the advantage isn't just poor scaling, and paired Wilcoxon tests on the NCC. The reported effect sizes are large (rank-biserial ~0.8), and the advantage survives calibration. If the mechanism can be fixed and re-validated, this would be a genuinely useful safety layer.\n\nThe remaining soft spots are secondary. The '4D input' description doesn't match the components (2D context summary + D test embedding = 3D), and the test embedding is never said to be pooled. The infant subgroup is chosen post hoc from the same performance data used to claim silent-failure detection, which is okay for evaluating uncertainty quality on a hard subgroup but weaker as a detection claim. The EF and AUROC results come without confidence intervals. No code is shipped.\n\nMy recommendation: treat this as a major-revision candidate, not a desk reject. The empirical backbone is strong enough that a corrected specification — either the proper integral over y or an explicit label-free surrogate, plus the embedding details — deserves a serious referee round. The reader's REJECT at moderate confidence is the right call for the current version.","headline":"A promising VIDS-to-segmentation extension whose energy function, as written, requires test labels the algorithm never sees—the empirical study is solid enough to warrant major revision rather than a desk reject.","tokens_in":14832,"tokens_out":6410,"would_cite":false,"duration_ms":57659,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An adult-trained cardiac segmentation model can detect its own silent failures on infant echocardiograms by attaching a lightweight variational head that raises uncertainty wherever the distribution shifts.","keywords":["uncertainty quantification","out-of-distribution detection","pediatric echocardiography","cardiac segmentation","variational inference","distribution shift","deep ensembles","silent failure detection"],"falsifier":"Evaluate VIDS-Seg on a held-out pediatric cohort from a different center or on a different shift type, such as adult patients with severe cardiomyopathy, and compute the NCC gap against the deep ensemble; if the advantage disappears or inverts when the shift is not mimicked by adult bootstrap subsamples, the synthetic-environment premise fails. A simpler check: measure whether the variational posterior's sigma, which is supposed to widen for novel inputs, actually correlates with the infant-versus-adult distance in the embedding space across individual cases.","tokens_in":13768,"feed_emoji":"🫀","tokens_out":5490,"duration_ms":52727,"temperature":0.7,"pith_summary":"This paper asks whether an adult-trained cardiac ultrasound segmentation model can be made to flag when it is silently failing on children, without retraining or pediatric labels. It proposes VIDS-Seg, which attaches a small stochastic prediction head to a frozen U-Net and uses an adaptive prior that widens the posterior when a test image looks novel relative to the training set. Tested on left ventricular segmentation, trained on adults and evaluated zero-shot on children, VIDS-Seg matches the segmentation accuracy of deep ensembles and PHiSeg while producing entropy maps that are far better aligned with actual per-pixel errors, especially for infants. Because infants are the subgroup where all models' accuracy drops, the improved uncertainty localization is a practical safety layer: it tells the clinician which cases deserve review, and it survives post-hoc calibration.","feed_headline":"Uncertainty head flags silent failures in pediatric echo","feed_subtitle":"VIDS-Seg matches adult-trained accuracy while localizing uncertainty at real errors, even after calibration.","key_machinery":"The central object is an adaptive prior over segmentation-head parameters, learned by variational inference. VIDS-Seg keeps the U-Net encoder-decoder frozen and treats the final 1x1 convolution (weight and bias) as the only stochastic parameters, so the posterior over $\\theta$ has just $d_\\theta = 2D+2$ dimensions for binary segmentation. An inference network $h_\\gamma$ receives a spatial-pooled embedding of the test image concatenated with the mean and standard deviation of pooled embeddings of a context set of training images, and outputs the variational mean and log-variance. The prior is defined through an energy function that averages pixel log-likelihoods over training and test images, and the inference network is trained over bootstrap-subsampled synthetic environments with a cross-environment ELBO plus a variance penalty, which is the mechanism intended to make uncertainty grow under distribution shift. This machinery does the work of producing entropy maps that localize where errors occur.","core_discovery":"The paper claims that uncertainty quantification under covariate shift can be made tractable for dense image segmentation by applying amortized variational inference only to a lightweight 1x1 prediction head on top of a frozen segmentation network. The resulting method, VIDS-Seg, conditions its posterior on a summary of the adult training context and the test image, so predictive entropy widens when the test image is atypical. Across age strata in a pediatric echocardiography dataset, VIDS-Seg matches the Dice and Hausdorff performance of a 10-member deep ensemble and PHiSeg, while achieving substantially higher normalized cross-correlation between predicted entropy and per-pixel error on both non-infant and infant groups. The claim extends beyond quality metrics: more localized uncertainty yields lower and lower-variance ejection-fraction error for infants and a higher AUROC (0.94) for flagging cardiac malfunction than the baselines.","pith_inferences":["If the mechanism generalizes, the same bootstrap-subsample training could flag other silent-failure subgroups, such as unusual adult anatomies or different ultrasound vendors and acoustic windows, provided the embedding space captures those shifts; the paper only demonstrates the pediatric echocardiography case.","A testable extension would be to check whether VIDS-Seg's NCC advantage persists under non-anatomical shifts, such as image corruption or label noise, where the synthetic environments may or may not mimic the shift.","The paper evaluates pixel-level entropy, but clinical use needs a scalar decision rule; aggregating entropy into a per-study alert and testing its operating characteristic against expert review would be the missing step toward deployment.","Because the inference network conditions on pooled embeddings, the method implicitly assumes that global appearance statistics summarize distributional novelty; shifts that live in fine spatial structure rather than global features could be missed."],"forward_implications":["A frozen adult-trained segmentation model can gain an OOD-aware uncertainty signal by adding VIDS-Seg's head, with no retraining of the backbone and no pediatric labels.","On pediatric echocardiography, VIDS-Seg's uncertainty-error alignment beats both a 10-member deep ensemble and PHiSeg, and the ranking survives post-hoc temperature scaling fitted on adult data.","Infants are the only age stratum where all tested models drop substantially (DSC around 0.84-0.85), so they are the subgroup most in need of uncertainty-based guarding.","The better-localized uncertainty carries downstream: VIDS-Seg gives lower and less variable ejection-fraction MAE for infants and the highest AUROC for detecting EF below 50%.","Post-hoc calibration improves all methods' NCC but does not close the gap, indicating the advantage is spatial localization, not just confidence scaling."],"supporting_citations":[{"why":"Supplies the VIDS adaptive-prior framework and cross-environment objective that VIDS-Seg extends to segmentation.","marker":"[15]"},{"why":"Provides the adult echocardiography training cohort used as the in-distribution source.","marker":"[22]"},{"why":"Provides the pediatric echocardiography dataset used for zero-shot evaluation and age-stratified OOD analysis.","marker":"[23]"},{"why":"PHiSeg is the hierarchical probabilistic segmentation baseline whose uncertainty quality is compared.","marker":"[16]"},{"why":"U-Net is the backbone architecture whose frozen features VIDS-Seg builds its prediction head on.","marker":"[26]"},{"why":"Supplies temperature scaling, the post-hoc calibration method used to test whether the uncertainty gap is merely a scaling artifact.","marker":"[28]"},{"why":"Provides the risk-extrapolation variance penalty used in the cross-environment training objective of Eq. (5).","marker":"[20]"}],"fun_headline_variants":["OOD-aware uncertainty flags silent errors in pediatric echo","VIDS-Seg: uncertainty head matches segmentation errors in kids","Amortized VI head localizes errors in pediatric ultrasound","Zero-shot pediatric echo: uncertainty pinpoints real mistakes","Adult-trained model, pediatric scans: uncertainty knows when it fails"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"VIDS-Seg assumes that bootstrap subsamples of the adult training set reproduce the kind of distribution shift that makes infant echocardiograms out-of-distribution, so that a prior trained on those synthetic shifts widens for real infants; this premise is never checked against pediatric data during training.","fun_headline_variants_meta":{"raw":{"variants":["OOD-aware uncertainty flags silent errors in pediatric echo","VIDS-Seg: uncertainty head matches segmentation errors in kids","Amortized VI head localizes errors in pediatric ultrasound","Zero-shot pediatric echo: uncertainty pinpoints real mistakes","Adult-trained model, pediatric scans: uncertainty knows when it fails"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0002,"raw_usage":{"total_tokens":1396,"prompt_tokens":984,"completion_tokens":412,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":600,"completion_tokens_details":{"reasoning_tokens":331}},"tokens_in":600,"tokens_out":412,"duration_ms":4118,"temperature":1.0,"reasoning_tokens":331,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:40:23.004404+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate VIDS-Seg on a held-out pediatric cohort from a different center or on a different shift type, such as adult patients with severe cardiomyopathy, and compute the NCC gap against the deep ensemble; if the advantage disappears or inverts when the shift is not mimicked by adult bootstrap subsamples, the synthetic-environment premise fails. A simpler check: measure whether the variational posterior's sigma, which is supposed to widen for novel inputs, actually correlates with the infant-versus-adult distance in the embedding space across individual cases.","supporting_citations":[{"cited_title":"Slavutsky, D","cited_arxiv_id":null,"evidence_quote":"Supplies the VIDS adaptive-prior framework and cross-environment objective that VIDS-Seg extends to segmentation."},{"cited_title":"Ouyang, B","cited_arxiv_id":null,"evidence_quote":"Provides the adult echocardiography training cohort used as the in-distribution source."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the pediatric echocardiography dataset used for zero-shot evaluation and age-stratified OOD analysis."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"PHiSeg is the hierarchical probabilistic segmentation baseline whose uncertainty quality is compared."},{"cited_title":"Ronneberger, P","cited_arxiv_id":null,"evidence_quote":"U-Net is the backbone architecture whose frozen features VIDS-Seg builds its prediction head on."},{"cited_title":"Krueger, E","cited_arxiv_id":null,"evidence_quote":"Provides the risk-extrapolation variance penalty used in the cross-environment training objective of Eq. (5)."}],"review_version":1}