{"id":"945961e7-3f2d-4ff3-8298-cee63707bd01","arxiv_id":"2505.07691","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A dynamic, class-wise, feedback-driven pseudo-label thresholding strategy (ENCORE) improves semi-supervised medical image segmentation, especially when very little labeled data is available.","lead":"ENCORE replaces fixed confidence thresholds for filtering pseudo-labels in semi-supervised semantic segmentation with a feedback-driven, class-aware adaptive thresholding mechanism. The approach is tested across five medical imaging datasets and three network architectures, integrating with UniMatch, Switch, and AD-MT frameworks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The threshold selector uses an in-sample Dice score on a tiny labeled batch; the paper offers no evidence this proxy predicts generalization, so the headline gains may not be attributable to the feedback mechanism.","rationale":"I read the paper as an engineering contribution: replace fixed confidence thresholds in pseudo-supervision with a class-aware, feedback-driven mechanism. The strongest evidence is Tables 1-5, but the mechanism's only feedback signal is Eq. (4). The reader's weakest assumption is exactly this proxy risk, and I agree; I want to make the failure mode concrete. In data-scarce SSL, the labeled mini-batch is both the training signal for the student and the evaluation set for threshold selection, so the selection is not a clean validation signal. The paper does not guard against this: no significance testing, no sensitivity analysis for (alpha1, alpha2, n), no comparison with random threshold selection, and no ablation across more than one fold for the main components. The gain is also largest where the proxy is smallest (ACDC 1/140, LA 1/40). A single controlled experiment replacing Eq. (4) with random selection would settle whether the feedback loop itself is doing the work. If random selection performs as well, the method's contribution reduces to the CAC initialization plus the added compute. I would keep the reader's conditional verdict: the idea is plausible and the experiments are suggestive, but the central mechanism is not yet evidenced. I am not rejecting the paper; the missing control is feasible and should be required for acceptance.","tokens_in":15800,"tokens_out":7602,"duration_ms":80901,"concrete_test":"On Cataract-1K at the 1/26 split (61 labeled images), run three variants with identical compute and update rule: (A) ENCORE as published; (B) replace Eq. (4) with random selection among tau1, tau2, tau3; (C) replace Eq. (4) with selection based on Dice computed on a held-out validation split of the same 61 labeled images. If B matches A within one standard deviation, the feedback signal is not responsible for the gain. If C differs materially from A, the in-sample labeled-batch proxy is overfitting. Report mean Dice and std over the same four folds, plus the fraction of mini-batches in which each controller is selected for A and C.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. (4) chooses the threshold controller that maximizes Dice on the labeled mini-batch after one assessor training step, and Eq. (5) moves the controllers when the same choice repeats. The central claim that ENCORE improves pseudo-supervision in data-scarce regimes depends on this Dice value being a reliable proxy for how a threshold will affect student generalization. That condition is least secure in precisely the regimes where the paper claims the largest gains. The assessor is initialized from the current student, which has already been trained on the same labeled mini-batch, so the selection signal is in-sample rather than out-of-sample. In the ACDC 1/140 setting the labeled set is a single volume, and in Cataract-1K 1/26 it is 61 images; per-class Dice on such batches is high-variance and can be high even when the filtering strategy is poor. The paper does not report the distribution of selected thresholds, any correlation between selected thresholds and held-out performance, or a control that selects among the same three controllers randomly. Table 1 also shows the standard deviation dropping from 3.8-8.4 for UniMatch to at most 2.4 for UniMatch+ENCORE, which is unexplained and suggests the four-fold results may not be stable enough to support the headline. Without a check that Eq. (4)'s signal is predictive, the reported gains could be an artifact of threshold overfitting to the labeled training batch rather than evidence that feedback-driven thresholding generalizes.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes ENCORE, a feedback-driven mechanism for selecting confidence thresholds in semi-supervised semantic segmentation. ENCORE first computes class-wise reliability scores from the labeled set (CAC) and then uses three assessor networks, each applying a different threshold controller, to dynamically select the threshold that maximizes Dice on the labeled batch (ACT). The authors integrate ENCORE into UniMatch, Switch, and AD-MT, and report consistent improvements across five medical imaging datasets, especially in low-data regimes. The central claim is that ENCORE substantially improves existing pseudo-supervision methods in data-scarce conditions.","tokens_in":16084,"tokens_out":5302,"duration_ms":50229,"significance":"If the empirical results hold, the paper addresses a practical and understudied problem: how to set pseudo-label confidence thresholds without large validation sets. The method is simple to describe, integrates with several existing frameworks, and the reported gains in low-data regimes (e.g., Cataract-1K 1/26 and ACDC 1/140) are large. The ablation study isolating CAC and ACT is useful and supports the design. The main weakness is that the threshold-selection signal, Eq. (4), is an in-sample Dice score on a very small labeled batch, and the paper provides no analysis of whether this signal reliably predicts generalization; this is especially concerning in the extreme low-data settings where the method claims its largest gains. No code or checkpoints are provided, and at least one reported metric (Table 5, Rel. Avg.) is internally inconsistent. The core idea is non-circular, however, because threshold selection is evaluated on labeled data rather than on the unlabeled pseudo-labels being filtered.","major_comments":[{"comment":"The threshold selection signal is the Dice score on the labeled mini-batch after a single assessor training step, with assessors copied from a student that has already been trained on that same labeled batch. This is an in-sample, potentially high-variance criterion, yet the paper does not analyze its reliability. In the ACDC 1/140 setting the labeled set is a single volume, and in Cataract-1K 1/26 it is 61 images, precisely the regimes where the paper claims the largest gains. Please provide evidence that the selected threshold controller predicts held-out performance: for example, the distribution of selected controllers over training, the correlation between Eq. (4)'s Dice and validation Dice, or an ablation that selects among the three controllers randomly rather than by Eq. (4). Without such evidence, the reported gains could be an artifact of overfitting the threshold to the labeled batch.","section":"§3.2, Eq. (4)"},{"comment":"The extreme low-data results (ACDC 1/70 and 1/140; LA 1/26 and 1/40) are reported as single point estimates with no standard deviations, number of runs, or seeds. These are precisely the settings where the proxy used in Eq. (4) is least reliable and where the claimed improvements are largest. The paper states that the method \"significantly improves\" performance, but without variance estimates the statistical support for that claim in these regimes is not established. Please report multiple seeds or otherwise quantify the variability of these numbers.","section":"Tables 4 and 5 (ACDC, LA)"},{"comment":"The reported relative average Dice of 8.36 for AD-MT + ENCORE2 in the LA dataset is inconsistent with the per-split values in the same row: averaging the per-split relative improvements over the supervised baseline would yield a value an order of magnitude larger. This suggests either a typo or an inconsistent computation of the relative average metric. Please correct the value and ensure the same formula is used consistently across Tables 4 and 5.","section":"Table 5, Rel. Avg. for AD-MT + ENCORE2"},{"comment":"The update rule depends on the parameters n, α1, and α2, which are fixed to n=5 and (α1,α2)=(0.98,1.02) across all datasets and settings. The text claims the method eliminates manual threshold tuning, but these meta-parameters still require manual choice, and no sensitivity analysis is reported. Please report experiments varying n and α1/α2, or at least provide a justification for the fixed values.","section":"§3.2, Eq. (5) and Section 4 implementation details"}],"minor_comments":[{"comment":"The text states that the labeled splits correspond to [1/2, 1/4, 1/8, 1/16], but Table 1 reports a 1/26 split with 61 images; please clarify how the number of training cases relates to the labeled fraction and reconcile the discrepancy.","section":"Section 4, Cataract-1K description"},{"comment":"Eq. (2) computes the class-wise reliability indicator using \"a network trained solely on labeled images,\" and Algorithm 1 begins with \"Train an assessor network on labeled images\" lines 1-2, but the relationship between this initial assessor and the three assessor networks used in ACT is not clarified; please define the initialization and whether the same weights are reused.","section":"Eq. (2) and Algorithm 1"},{"comment":"The notation Dice(·, ·, τ_k) in Eq. (4) is not formally defined; please specify the metric arguments (prediction, ground truth, threshold) and how the threshold enters the Dice computation, since Dice is normally threshold-independent.","section":"Eq. (4)"},{"comment":"The sentence \"while excluding it from our configurations\" is confusing in the ACDC dataset paragraph; please rephrase to clarify that the paper follows the settings of [52] or describe the actual split procedure.","section":"Section 4, ACDC description"},{"comment":"The kernel density estimate plots are difficult to interpret without axis labels and units; the qualitative claim that ENCORE \"bridges the distribution gap\" should be supported by a quantitative metric, such as a distributional distance between labeled and unlabeled feature distributions.","section":"Section 5, Figure 5"},{"comment":"The paper states that code and training splits will be released upon acceptance, but no code, checkpoints, or detailed per-fold splits are provided in the supplementary; please include the implementation details needed to reproduce the four-fold results, including any random-seed handling.","section":"General reproducibility"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is interesting and the experimental scope is broad, but the load-bearing assumption about the reliability of the in-sample Dice proxy is not tested, and the extreme low-data results lack variance information. The internally inconsistent relative-average metric in Table 5 adds to the concern. I would ask for the additional analyses described in the major comments before considering this for publication. The 'ENCORE2' suffix in Table 5 is unexplained and should be clarified in revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: this is a solid engineering contribution that will be useful to people in semi-supervised medical segmentation, and it deserves serious peer review. The novelty is real—class-aware calibration plus a feedback-driven ensemble of thresholds—and the experiments are extensive. The main soft spot is exactly what the stress-test note flags: the threshold selection signal in Eq. (4) is an in-sample Dice on a tiny labeled batch, and the paper doesn't show that it's predictive. That doesn't sink the paper, but it does mean the headline claim is under-supported.\n\nCAC is a nice idea: instead of a global confidence cutoff, it sets per-class thresholds from the model's average true-positive confidence on the labeled set. ACT then maintains three threshold multipliers, copies the current student into three assessors, and selects the threshold that yields the best Dice on the labeled batch after one update step. That's a clean way to avoid manual threshold tuning. The experiments are a real strength: five datasets, three architectures, many label fractions, and the gains in extreme low-data regimes are large—ACDC with one labeled volume jumps from 78.41 to 84.88 Dice for UniMatch, and the pattern holds across Switch and AD-MT. The ablation study shows both modules contribute.\n\nNow the soft spots. With one labeled volume, per-class Dice is extremely noisy, and the assessor starts from a student that has already seen those samples, so selection is in-sample. The paper doesn't report how often each threshold is chosen, doesn't include a random-selection control, and doesn't compare against a simple grid search of thresholds on the labeled set. The unexplained variance reduction in Table 1—std dropping from 3-8 to under 2.4—makes me wonder whether the across-fold results are stable enough for the headline. Also missing: code and splits (promised at acceptance), significance testing, sensitivity to alpha1/alpha2/n, and the ablation runs on a single fold. Minor: Table 5 has \"ENCORE2\" and a relative average of 8.36 that is likely a typo.\n\nThe central finding—ENCORE improves several strong baselines across multiple datasets—is probably true as reported. What is not proven is the mechanism: that the Dice feedback drives the gain, rather than just the extra threshold diversity or the .98/1.02 step. That's a fixable gap.\n\nSend it to review. The reviewers should ask for code, the random-selection control, and a short analysis of the selection signal's reliability. If the improvements survive that control, this becomes a solid, citable paper.","headline":"ENCORE is a useful adaptive thresholding scheme for low-data medical segmentation, but the paper doesn't yet prove the feedback signal does the work—send it to review with a request for a random-selection control.","tokens_in":16638,"tokens_out":3641,"would_cite":true,"duration_ms":34773,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ENCORE replaces static confidence thresholds with class-aware, feedback-driven cutoffs for pseudo-labeling and improves semi-supervised semantic segmentation, particularly when labeled data are scarce.","keywords":["semi-supervised learning","semantic segmentation","pseudo-labeling","confidence thresholding","adaptive thresholds","class-aware calibration","medical image segmentation","feedback-driven selection"],"falsifier":"Measure, across training iterations, the rank correlation between the Dice score on the labeled batch used in Eq. (4) and the Dice score on a held-out validation set; if this correlation is not positive in the low-data splits, the selected thresholds are not tracking generalization and the reported gains would not be expected to transfer.","tokens_in":15567,"feed_emoji":"🩺","tokens_out":6925,"duration_ms":62047,"temperature":0.7,"pith_summary":"Semi-supervised semantic segmentation often trains on pseudo-labels filtered by a fixed confidence threshold, but choosing that threshold needs a large labeled validation set and is especially unreliable when labeled data are scarce. The paper proposes ENCORE, a two-part strategy that sets class-wise thresholds from the model's true-positive confidence on labeled data and then adapts those thresholds during training by testing three candidate thresholds on copies of the student and keeping the one that scores best on the labeled batch. The claim is that this feedback loop keeps informative pseudo-labels while discarding noisy ones, without manual threshold tuning. If correct, it would remove a major hyperparameter from pseudo-supervision frameworks and give the largest gains exactly where labeled data are hardest to obtain.","feed_headline":"Feedback-driven thresholds lift low-data semi-supervised segmentation","feed_subtitle":"Adding the method to existing pseudo-labeling frameworks sharpens Dice scores most when labeled images are scarce.","key_machinery":"The central object is ENCORE, an ensemble of three class-wise threshold controllers coupled to the student model. Its first component, Class-Aware Confidence Calibration (CAC), computes for each class the average confidence of true-positive predictions on the labeled set and uses this vector R to initialize the thresholds, so classes with inherently lower confidence are not over-filtered. Its second component, Adaptive Confidence Thresholding (ACT), creates three assessor networks by copying the current student weights, trains each on the pseudo-labeled batch with a different threshold controller, and selects the controller maximizing Dice on the labeled batch (Eq. 4); controller values are updated only after the same controller is selected n consecutive times (Eq. 5). The assessor networks are the mechanism that carries the argument: they turn threshold selection into an online, metric-driven search rather than a fixed hyperparameter.","core_discovery":"The central discovery is that pseudo-label filtering does not need a pre-set confidence threshold: a threshold controller can be learned online from the student's own response. ENCORE first computes a class-wise reliability indicator R from the labeled set, the average confidence of true-positive predictions per class, and uses it to initialize three threshold controllers. At each training step it copies the student into three assessor networks, trains each assessor on the pseudo-labeled batch under a different controller, and selects the controller with the highest Dice on the labeled batch; after repeated selections it shifts the whole trio of thresholds upward or downward. The paper reports that this mechanism improves Dice scores over fixed-threshold baselines across five medical segmentation datasets, three network architectures, and several pseudo-supervision frameworks, with the largest relative gains in the lowest labeled-data fractions.","pith_inferences":["Editorial: The feedback loop is a form of online hyperparameter search over three threshold levels; testing more controllers or a continuous update rule might yield further gains, but the paper only evaluates three.","Editorial: In the most extreme low-data splits the labeled batch is very small, so the Dice signal in Eq. (4) could overfit; a stress test would be to swap the labeled mini-batch between iterations and check whether selected thresholds remain stable.","Editorial: Because the evaluator is a generic performance metric, the same feedback idea could be applied to other pseudo-label filters (entropy, teacher disagreement) or to other dense prediction tasks, though the paper does not test these."],"forward_implications":["Static confidence thresholds and manual threshold tuning become unnecessary in pseudo-supervision pipelines; thresholds are initialized per class and adjusted during training.","In data-scarce settings the gains are largest: for example, on Cataract-1K with 1/26 labeled data, UniMatch plus ENCORE reaches 83.62 Dice versus 78.07 for UniMatch alone, and with one labeled ACDC volume UniMatch plus ENCORE beats supervised training with ten labeled volumes.","The improvement transfers across frameworks (UniMatch, Switch, AD-MT) and architectures (DeepLabV3+, UNet, VNet), so the benefit comes from the thresholding strategy rather than from a particular teacher-student design.","Adaptive thresholding also narrows fold-to-fold variance on Cataract-1K compared with static-threshold baselines."],"supporting_citations":[{"why":"UniMatch baseline, the main framework that ENCORE is added to; its fixed-threshold results are the direct comparison for the reported gains.","marker":"[48]"},{"why":"Switch baseline, a second framework used to show ENCORE transfers; provides temporary-teacher pseudo-labels that ENCORE filters.","marker":"[30]"},{"why":"AD-MT baseline, a third framework used in ablations and low-data experiments, with its official training settings and comparisons.","marker":"[52]"},{"why":"Cataract-1K dataset, the primary multi-class benchmark where ENCORE shows the largest gains and variance reduction.","marker":"[14]"},{"why":"ACDC dataset, providing the one- and two-volume extreme low-data splits where ENCORE's gains are most pronounced.","marker":"[3]"},{"why":"LA dataset, the volumetric left-atrium benchmark used to show ENCORE improves structural retrieval in 3D segmentation.","marker":"[46]"},{"why":"DeepLabV3+, one of the architectures used for Cataract-1K, Prostate MRI, and EndoVis experiments.","marker":"[5]"},{"why":"Prostate MRI dataset, a multi-site benchmark showing gains in an extreme low-data regime.","marker":"[27]"}],"fun_headline_variants":["No manual thresholds: feedback-driven filtering sharpens semi-supervised segmentation","Adaptive thresholds from model feedback boost low-data segmentation","ENCORE: learn thresholds on the fly for semi-supervised segmentation","Dynamic pseudo-label thresholds from student self-feedback improve segmentation","No preset confidence: feedback loop sets pseudo-label thresholds"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The feedback loop assumes that the Dice score computed on the small labeled mini-batch is a faithful proxy for how well the model will generalize, and in the lowest-data settings that proxy is based on very few labeled samples.","fun_headline_variants_meta":{"raw":{"variants":["No manual thresholds: feedback-driven filtering sharpens semi-supervised segmentation","Adaptive thresholds from model feedback boost low-data segmentation","ENCORE: learn thresholds on the fly for semi-supervised segmentation","Dynamic pseudo-label thresholds from student self-feedback improve segmentation","No preset confidence: feedback loop sets pseudo-label thresholds"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000607,"raw_usage":{"total_tokens":2820,"prompt_tokens":930,"completion_tokens":1890,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":546,"completion_tokens_details":{"reasoning_tokens":1807}},"tokens_in":546,"tokens_out":1890,"duration_ms":12516,"temperature":1.0,"reasoning_tokens":1807,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:08:58.048681+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure, across training iterations, the rank correlation between the Dice score on the labeled batch used in Eq. (4) and the Dice score on a held-out validation set; if this correlation is not positive in the low-data splits, the selected thresholds are not tracking generalization and the reported gains would not be expected to transfer.","supporting_citations":[{"cited_title":"Ms- net: Multi-site network for improving prostate segmentation with heterogeneous mri data","cited_arxiv_id":null,"evidence_quote":"Prostate MRI dataset, a multi-site benchmark showing gains in an extreme low-data regime."},{"cited_title":"Revisiting weak-to-strong consistency in semi-supervised semantic segmentation","cited_arxiv_id":null,"evidence_quote":"UniMatch baseline, the main framework that ENCORE is added to; its fixed-threshold results are the direct comparison for the reported gains."},{"cited_title":"Switching temporary teachers for semi-supervised semantic segmentation","cited_arxiv_id":null,"evidence_quote":"Switch baseline, a second framework used to show ENCORE transfers; provides temporary-teacher pseudo-labels that ENCORE filters."},{"cited_title":"Alternate diverse teaching for semi-supervised medical image segmentation","cited_arxiv_id":null,"evidence_quote":"AD-MT baseline, a third framework used in ablations and low-data experiments, with its official training settings and comparisons."},{"cited_title":"Deep learning techniques for automatic mri cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE transactions on medical imaging, 37 (11):2514–2525, 2018","cited_arxiv_id":null,"evidence_quote":"ACDC dataset, providing the one- and two-volume extreme low-data splits where ENCORE's gains are most pronounced."},{"cited_title":"A global benchmark of algo- rithms for segmenting the left atrium from late gadolinium- enhanced cardiac magnetic resonance imaging","cited_arxiv_id":null,"evidence_quote":"LA dataset, the volumetric left-atrium benchmark used to show ENCORE improves structural retrieval in 3D segmentation."},{"cited_title":"Encoder-decoder with atrous separable convolution for semantic image segmentation","cited_arxiv_id":null,"evidence_quote":"DeepLabV3+, one of the architectures used for Cataract-1K, Prostate MRI, and EndoVis experiments."}],"review_version":1}