{"id":"83fd73d4-e6f1-438c-95fb-7b55d8285a58","arxiv_id":"2607.14338","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Scaling region-based segmentation loss gradients by per-voxel prediction error improves calibration metrics on 2D/3D medical segmentation datasets with little or no loss in Dice score.","lead":"A segmentation-training trick replaces the usual probability-to-logit gradient with an error-scaled version of the region-based loss gradient, producing better-calibrated medical image segmentations while keeping Dice accuracy. The paper also argues the desired gradients cannot come from any scalar loss, so it ships a custom backward pass ('gradient surgery') instead.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Calibration gains may be an artifact of selecting top-5/best-of-10 runs; full-run statistics are needed to support the 'across all cases in all datasets' claim.","rationale":"The reader's weakest assumption focuses on the simplified equilibrium theory in Section 3.1. That is a real weakness in the explanatory story, but the strongest claim is empirical: surgery improves calibration while preserving Dice. The most load-bearing condition for that claim is that the reported improvements are robust rather than artifacts of selecting the best runs. The paper itself notes that 3D uses best-of-10 single runs, and 2D uses only the top 5 of 25; these selection choices can systematically favor a method with different gradient dynamics, especially when no code is released for independent verification. A straightforward re-analysis over all runs with paired significance tests would settle whether the empirical claim holds. Because the reader already assigned CONDITIONAL and this concern reinforces that condition rather than overturning it, I keep the verdict unchanged. I mark partial agreement because the reader did mention the best-of-10 issue in the rationale, but selected the theory simplification as the weakest assumption rather than the evaluation protocol.","tokens_in":19183,"tokens_out":16593,"duration_ms":162243,"concrete_test":"Recompute Tables 1 and 2 using all 25 (2D) and all 10 (3D) random-search runs, reporting mean, median, and 95% CI for NLL, ECE, MCE, and Brier, plus a paired Wilcoxon signed-rank test between each baseline loss and its surgery counterpart. If surgery is not significantly better on every dataset/baseline, the 'across all cases' claim is unsupported as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central empirical claim (Section 4.1) is that gradient surgery improves calibration metrics over each baseline 'across all cases in all datasets.' The evidence for this claim currently rests on selective run reporting: 2D tables report the average of the 5 best runs out of 25 (selected by validation Dice), and 3D tables report a single best run out of 10. With high training variance, top-k selection can make a method look better than a baseline even when the full-run distributions are equivalent, and it removes the variance information needed to assess significance. The 3D tables contain no confidence intervals, so the reader cannot tell whether the reported improvement is robust or driven by one favorable seed. The theoretical equilibrium argument in Section 3.1 is explicitly simplified and does not independently establish the empirical result: even if the mechanism is correct, the measured gains must be demonstrated on unselected runs. This is the load-bearing gap between the reported numbers and the headline claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses miscalibration of segmentation models trained with region-based losses (Dice, Tversky, Combo). The authors analyze the gradient of the Dice loss w.r.t. logits and argue that the sigmoid derivative p(1-p) suppresses gradients for confident errors, causing overconfident predictions. They propose a 'gradient surgery' that replaces the sigmoid derivative in the backward pass with an error-scaled factor (proportional to |y-p|), while retaining the region-imbalance-aware global term of the Dice loss. They show (Appendix A) that no scalar loss can produce the desired partial derivatives, so they define a non-conservative vector field and use it directly as the optimization objective. Experiments on 2D (INbreast, FIVES) and 3D (BraTS-METS, KiTS) datasets report improved calibration (NLL, ECE, MCE, Brier) relative to baseline region-based losses, with DSC largely preserved. The central claim is that the surgery 'improves calibration metrics compared to the respective baseline losses alone across all cases in all datasets' (Section 4.1).","tokens_in":19416,"tokens_out":8694,"duration_ms":80338,"significance":"If the empirical results hold across full run distributions, the proposed method is a simple and attractive plug-in for training calibrated segmentation models with region-based losses, addressing a clinically relevant problem. The paper has several strengths: it explicitly considers gradient geometry, provides an implementation listing, includes active-region calibration checks (Appendix H), an ablation of the exponential parameter n, logit-magnitude analysis, and a supplementary transformer experiment. However, the evidence is weakened by selective run reporting (top-5-of-25 in 2D, best-of-10 in 3D) and by a theoretical analysis that rests on an unproved equilibrium assumption. The paper's practical contribution could be significant for medical imaging, but the validity of the headline claim cannot be assessed from the current reported numbers.","major_comments":[{"comment":"The central claim 'across all cases in all datasets' is not supported by the reported statistics. Table 1 reports the average of the 5 best runs out of 25, selected by validation Dice, and Table 2 reports a single best run out of 10. Top-k selection biases the comparison in favor of the intervention and removes variance information needed for significance. Please report full-run statistics (e.g., mean/median with confidence intervals, or per-run scatter plots) for all 25/10 runs, or show pairwise comparisons in which the surgery improves over the baseline for a preponderance of runs. Without this, the claim is not established.","section":"Section 4.1 / Tables 1-2"},{"comment":"The non-existence proof contains an algebraic sign error. For y_i=y_k=0, the derivative of the global term with respect to p_k should be +2(2I+ε)/(P+Y+ε)^3, not the negative value printed (the derivative of N_i/D is [2(y_i−y_k) − 2N_i/(P+Y+ε)]/(P+Y+ε)^2). While the equality condition p_k=p_i still implies non-conservation after correcting the sign, the printed proof is incorrect. In addition, the proof applies to Eq. (9), not to the actual implemented field of Eq. (10) with the exponential decline terms; this gap should be addressed.","section":"Appendix A"},{"comment":"The 'indistinguishable voxels' equilibrium argument is asserted, not derived. The existence of sets of voxels with identical latent representations and coupled probabilities, and the reduction of optimization to a gradient-balance equilibrium, are assumptions. Since the paper claims a theoretical diagnosis of Dice overconfidence, this argument needs to be made rigorous (e.g., as a formal statement with conditions) or explicitly presented as a heuristic supported by empirical evidence, such as an analysis of representation similarity in trained networks.","section":"Section 3.1"},{"comment":"The claim of 'maintaining high prediction accuracy' is not uniformly supported. On KiTS, Dice - Surgery reduces DSC from 76.62 to 74.01, a drop of 2.61 points; this is not negligible in a medical segmentation context. Please quantify the accuracy/calibration trade-off and discuss the clinical acceptability of such a decrease. The current text characterizes the impact as 'negligible' for KiTS, which appears inconsistent with the reported numbers.","section":"Section 4.1 / Table 2"}],"minor_comments":[{"comment":"The caption refers to 'TunableGradSym', a name that appears nowhere else in the manuscript; likely a leftover from an earlier version.","section":"Table 3 caption"},{"comment":"The implementation uses a scaling constant 0.25 that is absent from Eq. (10). Please state the exact vector field used in experiments and clarify whether the 0.25 is a hyperparameter or a normalization factor.","section":"Eq. (10) and Listing 1"},{"comment":"Typo 'R ⋉' should likely be 'R^N' or 'R'. Also, the proof in Appendix A is referenced before the appendix is introduced.","section":"Section 3.2"},{"comment":"The caption is difficult to parse: the two added voxels have labels y1=1 and y2=2 (typo for 0?), and the description of the remaining 100 voxels could be clearer. The axes are in probability space while the text says the field is w.r.t. logits; please clarify.","section":"Figure 4"},{"comment":"The subplot label 'Validation NLL' is duplicated, and the caption contains 'training.objective'. Also, the reference for Brier (1950) is cited as 'Brier Glenn et al.', which should be corrected to 'Brier'.","section":"Appendix C / Figure 5"},{"comment":"The asterisks in Table 1 are computed on the selected 5 best runs. Even as a secondary analysis, the selection invalidates the significance test; please compute significance on all runs or describe the selection-correction procedure.","section":"Statistical significance"}],"recommendation":"major_revision","confidential_remarks":"The idea is simple and potentially useful, and the empirical direction is sound. However, the selective run reporting is a serious reproducibility concern for a venue like MIDL; it must be addressed with full-run statistics. The non-existence proof in Appendix A has a correctable error, and the theoretical equilibrium argument is underspecified. I am not recommending rejection because the fix is within scope, but the authors need to provide the full-run evidence and correct the proof before the headline claims can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Know this before you read it: the central idea is a real departure from prior work, and the empirical gains are big and consistent across four datasets. But the headline claim -- \"improves calibration across all cases in all datasets\" -- is currently supported only by selected runs: 2D tables use the 5 best of 25 runs, 3D tables the single best of 10 with no intervals. That is the paper's main soft spot, not its ideas.\n\nWhat is actually new: instead of proposing another scalar loss, the authors define a vector field directly on the logits and implement it by overwriting the softmax backward pass, scaling the gradient by the prediction error. That is a genuinely different mechanism from DSC++, SVLS, NACL, Combo, or margin-based label smoothing, and the non-conservative angle is a nice touch. The diagnosis is clean: Dice's gradient is a global overlap term times p(1-p), which vanishes for confident errors and puts the gradient equilibrium at p in {0,1}. The implementation is simple, the ablation on the exponential n shows robustness over a wide range, and the active-region, lesion-size, and transformer appendices are honest extra checks. The empirical core, as printed, holds up: surgery improves NLL, ECE, MCE, and Brier over plain Dice, Tversky, and Combo baselines in every table, with Dice roughly preserved or better.\n\nThe run-selection issue is real and it is the load-bearing gap between the numbers and the claim. With high training variance, top-k selection inflates apparent gains. To be fair to the stress-test note: given the size of the effects (4-6x reductions in NLL/ECE in several cases, consistent across four datasets and every calibration metric), I doubt selection alone creates the effect. But the 3D tables have no variance information at all, and \"across all cases\" cannot be evaluated without full-run statistics. The authors should report the distribution over seeds, or at least means with intervals for all 10 runs.\n\nThe theory is heuristic and the paper mostly labels it as such. The \"indistinguishable voxels\" equilibrium argument is a simplification, and the claim that CE reaches equilibrium exactly at p=r is asserted rather than derived. The Appendix A non-existence proof contains an algebraic slip in the mixed partials; the conclusion is likely right, but the printed derivation is not the one that proves it. Both are fixable and neither breaks the empirical result. Minor items: no temperature-scaling baseline in the main tables, the 0.25 constant is ad hoc, and no code is released.\n\nThe circularity concern does not bother me much: the field is deliberately constructed to scale gradients with error, so observing better calibration is the intended design being tested on held-out data, not a hidden confound.\n\nWho this is for: anyone working on segmentation calibration, training-time uncertainty, or custom backward passes. It deserves a serious referee -- conditional on a revision that reports unselected-run statistics and cleans up the proof -- and I would want to see the revision.","headline":"Genuinely new gradient-surgery method for calibrating Dice-type segmentation losses with large, consistent empirical gains; the main weakness is that the headline claim rests on top-k and best-of-N run reporting rather than full-run statistics.","tokens_in":19919,"tokens_out":7393,"would_cite":true,"duration_ms":65240,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that replacing the sigmoid derivative with an error-scaled vector field during backpropagation makes region-based segmentation losses yield well-calibrated probabilities while preserving Dice overlap.","keywords":["segmentation","calibration","Dice loss","region-based loss","gradient surgery","vector field","overconfidence","medical imaging"],"falsifier":"Build a synthetic segmentation task with known latent groups: each group has a fixed foreground ratio r_k, and voxels within a group are truly indistinguishable to the network. Train with Dice and with the surgery, then measure the equilibrium probabilities per group. If Dice does not converge to p near 0 or 1 regardless of r_k, or if the surgery does not converge to p near r_k, the paper's mechanism is falsified.","tokens_in":19075,"feed_emoji":"🩺","tokens_out":6318,"duration_ms":58161,"temperature":0.7,"pith_summary":"Segmentation models trained with region-based losses such as Dice are notoriously overconfident, which is a problem in medical imaging where probability maps guide decisions. The paper traces this to the gradient of Dice with respect to logits: confident predictions contribute almost nothing, and the only equilibrium between foreground and background gradients is at probabilities 0 and 1. The authors propose a \"gradient surgery\" that replaces the sigmoid derivative in the backward pass with a vector field whose magnitude scales linearly with prediction error |y-p|. This makes the equilibrium sit at the true foreground ratio of indistinguishable voxel groups, analogous to cross-entropy, while retaining the region-size weighting that makes Dice work on imbalanced data. Empirically, the surgery improves calibration metrics on every tested 2D and 3D medical segmentation task without sacrificing Dice score.","feed_headline":"Error-scaled gradients fix Dice loss overconfidence","feed_subtitle":"Replacing the sigmoid derivative with an error-scaled vector field cuts calibration error by 4-6x on medical benchmarks.","key_machinery":"The central object is the gradient vector field \"surgery\": an override of the backward pass through the sigmoid/softmax activation that replaces the standard derivative p(1-p) with a factor proportional to |y-p|, optionally multiplied by a sharp decline near 0 and 1 controlled by an exponent n. The paper proves no scalar loss function can produce these partial derivatives because mixed second partials are asymmetric, making the field non-conservative; stability is argued from small curl and from the field always pointing toward the ground truth. This surgery is what converts the Dice-style region weighting into a calibrated, error-scaled training signal.","core_discovery":"Region-based losses converge to miscalibrated solutions because their logit gradients vanish for confident predictions and only balance foreground versus background gradients at p=0 and p=1, forcing all probabilities to extremes. We define a logit-level vector field that scales gradient magnitude linearly with |y-p|, prove that no scalar loss can realize these partial derivatives (the field is non-conservative), and show in experiments that using this vector field as the backward pass for Dice, Tversky, and Combo losses improves calibration metrics across 2D and 3D segmentation tasks while maintaining Dice overlap.","pith_inferences":["If the equilibrium diagnosis is correct, the same vector-field surgery should extend to multi-class soft Dice and other region-based losses (e.g. Unified Focal) by generalizing the one-vs-rest error scaling; the paper only demonstrates binary segmentation, so that is a testable next step.","The non-existence of a scalar loss means any fully calibrated region-based objective must be non-conservative; this implies optimizer choice and learning-rate schedules could matter more than for standard losses, and formal stability bounds would be a natural follow-up.","The sharp-decline term behaves like a soft label-smoothing analog for region losses; an explicit comparison against spatially varying label smoothing on boundary voxels would isolate whether the gain comes from the error scaling or the decline.","Because the surgery reduces logit magnitudes, it may interact with post-hoc temperature scaling; a testable combination is whether temperature scaling adds anything after surgery, and whether the resulting model is more robust to domain shift than post-hoc-only calibration."],"forward_implications":["Applied to Dice, Tversky, and Combo losses, the surgery improves NLL, ECE, MCE, and Brier on all four tested medical segmentation benchmarks, often cutting calibration error by factors of 4 to 6.","Dice overlap is preserved or improved: on the 2D mammography mass dataset it rose from about 65% (Dice) to 74% (Dice + surgery), while on the other datasets it stayed within about a point.","Combining the surgery with a CE+Dice Combo loss gives the best or second-best calibration scores on most benchmarks, making it a drop-in replacement for plain region losses.","The intervention sharply reduces logit magnitudes—on the retinal vessel dataset, average foreground logits fell from roughly 26 to 3.7 with Dice—consistent with removing pathological overconfidence.","The method also improves calibration when used with a transformer-based segmentation backbone, suggesting it is not tied to one architecture."],"fun_headline_variants":["Gradient surgery cuts segmentation miscalibration","Non-scalar gradients restore calibrated segmentations","Vector field surgery calibrates segmentation models","Error-scaled logit gradients fix Dice calibration","Surgery on gradients fixes Dice overconfidence"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The root-cause story assumes that at convergence the model has pairs of pixels that look identical to the network but have different true labels, so their opposing gradients balance; if real training does not settle into that equilibrium picture, the diagnosis of Dice overconfidence does not follow even though the proposed fix may still work.","fun_headline_variants_meta":{"raw":{"variants":["Gradient surgery cuts segmentation miscalibration","Non-scalar gradients restore calibrated segmentations","Vector field surgery calibrates segmentation models","Error-scaled logit gradients fix Dice calibration","Surgery on gradients fixes Dice overconfidence"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000485,"raw_usage":{"total_tokens":2189,"prompt_tokens":665,"completion_tokens":1524,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":409,"completion_tokens_details":{"reasoning_tokens":1466}},"tokens_in":409,"tokens_out":1524,"duration_ms":12006,"temperature":1.0,"reasoning_tokens":1466,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T02:23:09.010894+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a synthetic segmentation task with known latent groups: each group has a fixed foreground ratio r_k, and voxels within a group are truly indistinguishable to the network. Train with Dice and with the surgery, then measure the equilibrium probabilities per group. If Dice does not converge to p near 0 or 1 regardless of r_k, or if the surgery does not converge to p near r_k, the paper's mechanism is falsified.","supporting_citations":[],"review_version":1}