{"id":"2a4e0fac-55bd-4187-8b5c-83567effff91","arxiv_id":"2411.15763","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Group-aware contrastive embeddings improve Coreset-based slice selection for 3D medical segmentation at low annotation budgets.","lead":"This paper presents a slice-based active learning method for 3D medical segmentation that learns a contrastive metric and feeds it into Coreset to pick which slices to annotate. It reports Dice improvements over existing active learning baselines, especially with scribble annotations and small annotation budgets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported superiority may reflect test-set tuning: Appendix A selects loss weights on the evaluation metric, and main/appendix ACDC numbers disagree.","rationale":"The reader's weakest_assumption focuses on the learned metric's alignment with task difficulty, but the paper's own Appendix A exposes a more direct threat to the central claim: the loss weights used in the main ACDC experiments were selected by trying multiple combinations and reporting the best, with no indication of a validation set separate from the test set. This makes the reported 'Ours' numbers the result of a small hyperparameter search over the evaluation metric, which is known to inflate performance and invalidate comparisons when baselines are not given the same tuning. The discrepancy between Table 1 and Appendix Table 5 for ACDC weak supervision (52.3 vs 55.6 at 2%, and similar gaps at 3–5%) is additional direct evidence that the reported numbers are not stable enough to support the strong claim. This concern is concrete, textually grounded, and testable. It does not require assuming bad faith; it is a standard methodological issue that can be resolved by pre-registering weights on a validation split and reporting all configurations tried. Because the reader's verdict was already CONDITIONAL, this concern does not change the verdict but sharpens the condition: the authors must demonstrate the result with a pre-specified configuration and reconciled tables.","tokens_in":20616,"tokens_out":4337,"duration_ms":38944,"concrete_test":"Fix a single loss-weight configuration using only a held-out validation split of the ACDC training data (e.g., 15% of volumes), before any test-set evaluation, and then run the full weak-supervision AL protocol at 2–5% budgets with that configuration. Report the resulting DICE scores together with the scores for all weight combinations tried, and reconcile Table 1 vs Appendix Table 5 by publishing the exact configuration and seeds for each row. If the pre-registered configuration does not beat Coreset and Random at 2–5% by a margin exceeding the bootstrap standard errors, the central superiority claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim—that the method surpasses existing AL techniques at low annotation budgets—rests on ACDC numbers that are selected and unstable. Appendix A explicitly states that for the ablation study on ACDC, the authors tried multiple combinations of group-contrastive loss weights and 'reported the best results,' then 'utilized the best loss/weight combination for our ACDC experiments.' No separate validation split is described for this selection. Thus the 'Ours' row in Table 1 is the maximum over a small hyperparameter grid, while baselines are reported at their default settings; this is a form of test-set selection that can inflate the reported gains. The problem is compounded by an unexplained internal inconsistency: for ACDC weak supervision, Table 1 reports Ours = 52.3/59.8/73.3/76.1 at 2–5%, whereas Appendix Table 5 reports 55.6/61.4/73.7/77.5 for the same setting. Both tables are presented as results of the same method, and no explanation is given. If the lower set is correct, the reported advantage is smaller than claimed; if the higher set is correct, the main text underreports and the selection is even more favorable. Either way, the paper does not currently establish that the proposed metric, rather than the choice of loss weights, is responsible for the observed outperformance. The Lipschitz assumption in Appendix B is a secondary concern; the empirical evidence itself is not stable enough to support the headline claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Group-based Contrastive Learning (GCL) combined with the Coreset algorithm for slice-based active learning in 3D medical segmentation. The contrastive encoder is trained with a sum of NT-Xent and group-contrastive losses defined over patient, volume, and adjacent-slice groupings; the learned embedding distance is then used by K-Center Greedy Coreset to select slices for annotation. The authors evaluate on ACDC, CHAOS, MS-CMR, and DAVIS under both weak (scribble) and full supervision, and report results from scratch and with pretrained encoders. They claim superior performance over existing active learning methods at low annotation budgets, and additionally compare weak-supervised slice-based AL with fully-supervised volume-based AL in terms of annotation time.","tokens_in":20915,"tokens_out":2867,"duration_ms":26243,"significance":"If the empirical claims held, the paper would make a useful contribution: combining metric learning with Coreset for slice-based AL in medical segmentation is a sensible direction, and the use of inherent data groupings (patient/volume/slice) is a reasonable inductive bias. The paper ships source code, reports bootstrap standard errors in the appendix, and includes a t-SNE visualization and an ablation over loss combinations. The main weakness is that the central empirical claim is currently not established because the loss weights were apparently selected on the evaluation set, and the reported ACDC numbers are internally inconsistent between the main text and the appendix. The theoretical guarantee in Appendix B rests on unverified Lipschitz and zero-error assumptions, so the contribution's practical value depends on the empirical evidence, which needs substantial clarification.","major_comments":[{"comment":"The selection of contrastive loss weights is performed on the evaluation set rather than on a held-out validation split. Appendix A states that in the ablation study on ACDC the authors tried different combinations of weights and \"reported the best results,\" then \"utilized the best loss/weight combination for our ACDC experiments.\" No validation split is described anywhere in Section 4.2. Because the baseline methods are reported at their default settings, the \"Ours\" row in Table 1 is effectively the maximum over a hyperparameter grid on the test set. This is a form of test-set fitting and can inflate the reported margins, undermining the abstract's claim that the approach \"surpasses existing active learning techniques.\" The authors should either describe a proper validation procedure or re-run the comparison with weights selected on a validation split.","section":"Appendix A and Section 4.6"},{"comment":"The ACDC weakly-supervised results are inconsistent between the two tables. Table 1 reports Ours = 52.3/59.8/73.3/76.1 at 2–5%, while Table 5 in Appendix D reports Ours = 55.6/61.4/73.7/77.5 for the same setting. Both tables are presented as results of the same method and no explanation is given for the discrepancy. If the lower values are correct, the claimed advantage over Coreset is smaller than stated (e.g., 52.3 vs 45.2 at 2% instead of 55.6 vs 45.2); if the higher values are correct, the main text underreports the method's performance. Either way, the reader cannot determine which result is authoritative, and this undermines the central quantitative claim in Section 4.3.","section":"Table 1 versus Appendix D Table 5"},{"comment":"The comparison is incomplete for several baselines the paper claims to evaluate. Section 4.2 lists VAAL, TypiClust, and CoreGCN as comparison methods, but VAAL and TypiClust are absent from the MS-CMR and CHAOS panels of Table 1, and all three are absent from the DAVIS results in Table 2. The summary in Section 4.5 says \"our method achieves the best performance on 21 out of 27 comparison points\" without accounting for these missing entries. The authors should clarify whether these baselines were run on those datasets and, if so, report the results; otherwise, the claim of a comprehensive comparison is not supported.","section":"Tables 1 and 2 and Section 4.5"},{"comment":"The abstract and Section 4.3 claim the method \"surpasses existing active learning techniques\" on weak and full annotations, but Table 2 shows that on DAVIS at 30% and 40% annotation, Random sampling achieves higher DICE (47.4 and 48.5) than Ours (45.5 and 46.6). Similarly, in Table 3 the pretrained results on CHAOS (Ours 95.2 vs Coreset 95.1) and DAVIS (Ours 75.1 vs Stochastic Batches 75.1) are effectively ties. The claim of universal superiority is therefore too strong as stated and should be qualified to the low-budget regimes where the advantage actually appears.","section":"Table 2 and Abstract"},{"comment":"The theoretical bound in Theorem 1 assumes the loss function L and the segmentation function eta_c are Lipschitz continuous with respect to the learned metric d_phi, and it assumes both zero training error and zero generalization error. These assumptions are not verified experimentally or by construction, and the paper provides no evidence that the self-supervised contrastive embedding aligns with segmentation-task difficulty. As a result, the Coreset guarantee in Equation (2) is not established for the actual learned d_phi. This does not invalidate the empirical approach, but it should be stated more cautiously, and the authors should at least discuss whether the Lipschitz constants can be bounded in practice.","section":"Appendix B and Section 3.2"}],"minor_comments":[{"comment":"The DAVIS dataset description says the 2016 train set was used and the 2016 val set was split into val and test, but it does not specify how many objects/videos appear in each split beyond the totals; please state the exact number of training and test videos used in the DAVIS experiments.","section":"Section 4.1 and Table 8"},{"comment":"Figure 2's annotation-time comparison relies on the assumption that scribble annotation is 15x faster than full mask annotation, but no sensitivity analysis is given for this multiplier; a brief discussion of how the comparison changes with a different multiplier would strengthen the claim.","section":"Section 4.4 and Figure 2"},{"comment":"The ablation table does not report standard errors, making it hard to judge whether the differences between loss combinations (e.g., 65.4 vs 64.1) are significant; please add error bars or state the number of seeds used for each ablation row.","section":"Section 4.6 and Table 4"},{"comment":"The paper claims to be \"the first work to integrate deep metric learning with Coreset during active learning for 3D medical segmentation,\" but prior work on unsupervised Coreset selection with contrastive learning (e.g., [48] and [49]) exists; please soften the novelty claim or clearly distinguish the contribution from those methods.","section":"Section 1"},{"comment":"The related work section cites [42] for Random Sampling, but [42] is a paper on robust active learning, not a canonical random sampling reference; please provide an appropriate citation for uniform random selection.","section":"Section 2"},{"comment":"The bootstrap standard errors in the appendix are reported as two standard deviations, but the main text never defines this convention; please define the error bars when they are first introduced.","section":"Appendix D and Tables 5-11"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant problem and the general idea is worth pursuing, but the current manuscript does not support its headline claim because the hyperparameter selection procedure and the inconsistent ACDC numbers leave the reported gains in doubt. I would encourage the editor to request a revision where the authors either provide a proper validation-based selection protocol or present results without test-set tuning, and where the discrepancy between Table 1 and Table 5 is resolved. The theoretical appendix should also be reframed as a heuristic justification rather than a formal guarantee, given the unverified assumptions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid, useful AL paper with one significant empirical weakness. The combination of group-aware contrastive metric learning with Coreset for slice-based active learning in 3D medical segmentation is genuinely new relative to the cited work, and the weak-supervision slice-based evaluation is a valuable addition to the literature. The code is released, and the experimental scope (four datasets, weak and full supervision, multiple baselines) is more than most AL papers attempt.\n\nThe method itself is sensible: learn a SimCLR-style encoder with patient/volume/slice group contrastive losses, use the resulting L2 metric as the diversity measure in K-Center Greedy Coreset, and select 2D slices rather than entire volumes. The ablation shows the group losses help, and the qualitative t-SNE supports the intended mechanism. On CHAOS, MS-CMR, and DAVIS the reported gains are fairly consistent across budgets, which suggests the approach has real value beyond the one dataset.\n\nNow the soft spots, in order of importance. First, Appendix A says the authors tried several loss-weight combinations on ACDC and \"reported the best results,\" then used that combination for the ACDC experiments. No validation split is described. That is test-set selection: the Ours row in Table 1 is a max over a small hyperparameter grid, while the baselines are reported at their default settings. This inflates the headline margin (52.3 vs 45.2 at 2% weak) and needs to be fixed with a proper validation protocol before the claim can be trusted. Second, the main text Table 1 and Appendix Table 5 report different numbers for exactly the same ACDC weak-supervision setting: 52.3/59.8/73.3/76.1 in one place and 55.6/61.4/73.7/77.5 in the other, with no explanation. A referee will not let that pass. Third, the comparison is incomplete on two datasets: VAAL and TypiClust are missing from MS-CMR and CHAOS, and the pretrained table includes only four methods. That weakens the \"21 out of 27\" summary. Finally, the Appendix B bound is a standard Coreset-style argument, but it assumes Lipschitzness of both loss and label function with respect to the learned metric and zero training/generalization error. Those are strong assumptions; I'd treat the theory as illustrative, not load-bearing.\n\nVerdict: the core idea is plausible and the experiments are extensive, but as written the paper does not establish that the metric, rather than the chosen loss weights, is responsible for the reported gains. It deserves a serious referee, because the approach is worth testing under a cleaner protocol, but it needs major revision. I would not cite it as-is; I would revisit once the inconsistency and selection issue are resolved.","headline":"Useful idea and thorough experiments, but the headline ACDC results are selected on the test set and the main/appendix numbers disagree, so the paper needs a clean revision before the claims hold.","tokens_in":21448,"tokens_out":2954,"would_cite":false,"duration_ms":25095,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a contrastively learned, group-aware metric makes slice-based active learning for 3D medical segmentation substantially more efficient at annotation budgets of 2-5%.","keywords":["active learning","3D medical segmentation","coreset","contrastive learning","metric learning","weak supervision","slice-based annotation","deep learning"],"falsifier":"On the ACDC dataset, compare the 2% weak-supervision DICE of the full method (52.3 in the main table) against the same pipeline with the metric replaced by raw-pixel Euclidean distance and by an embedding trained on randomly shuffled group labels. If either replacement retains the same gap over vanilla Coreset (45.2), the group-aware metric itself is not the cause of the gain; additionally, measuring the ratio $|L(\\hat y_1,y_1)-L(\\hat y_2,y_2)|/d_\\phi(x_1,x_2)$ on selected pairs would test the proportionality assumption directly.","tokens_in":20391,"feed_emoji":"🩺","tokens_out":6960,"duration_ms":56421,"temperature":0.7,"pith_summary":"Deep learning segmentation of 3D medical volumes is annotation-hungry, and active learning is meant to cut that cost. This paper argues that the standard Coreset strategy, which picks diverse slices by Euclidean distance, picks the wrong slices because that distance ignores medically meaningful structure. The authors replace it with a learned metric $d_\\phi(x_1,x_2)=\\ell_2(g_\\phi(x_1),g_\\phi(x_2))$ trained by a Group-based Contrastive Learning loss that pulls slices from the same patient, volume, or adjacent-slice group together. They report that this group-aware Coreset beats existing active learning methods on both weak and full annotations at low budgets, for example 52.3 versus 45.2 DICE at 2% weak supervision on ACDC. If correct, the method would let clinicians train usable 3D segmentation models from a tiny fraction of labeled slices.","feed_headline":"Learned group metric lifts 3D segmentation with 2-5% labels","feed_subtitle":"Contrastive groupings of patients, volumes, and slices make slice-based active learning beat vanilla Coreset.","key_machinery":"The load-bearing object is the learned metric $d_\\phi(x_1,x_2)=\\ell_2(g_\\phi(x_1),g_\\phi(x_2))$, produced by a SimCLR-style encoder trained with Group-based Contrastive Learning. The group loss modifies NT-Xent so that slices from the same patient, volume, or adjacent-slice group are treated as positives while non-group slices from the same patient are excluded from the denominator, allowing several group losses to be summed without cancelling. This metric replaces the Euclidean distance in the Coreset objective $\\arg\\min_{\\Delta s} \\max_{x_1\\in D}\\min_{x_2\\in s\\cup\\Delta s} d_\\phi(x_1,x_2)$, solved by K-Center Greedy, a 2-approximation algorithm. The paper adapts the Coreset radius bound, assuming the loss and label function are Lipschitz in $d_\\phi$ with zero training and generalization error, to justify the choice.","core_discovery":"On the paper's own terms, the central discovery is that injecting inherent data groups into metric learning fixes the main weakness of Coreset-based active learning for slice-based 3D segmentation. The paper proposes the loss $L_{\\text{contrastive}} = L_{\\text{NT-Xent}} + \\lambda_1 L_{\\text{patient}} + \\lambda_2 L_{\\text{volume}} + \\lambda_3 L_{\\text{slice}}$, where each group loss is an NT-Xent-style contrastive term over slices sharing that grouping, with a batch sampler that guarantees group mates appear in each batch. The learned encoder $g_\\phi$ defines the Coreset distance, and K-Center Greedy then selects slices so that every unlabeled slice is close to some selected slice in that metric. Across ACDC, MS-CMR, CHAOS, and DAVIS, the paper reports that this approach achieves the highest or near-highest DICE at low annotation budgets under both weak and full supervision, with the best ablation combining patient, volume, and NT-Xent losses.","pith_inferences":[],"forward_implications":["At annotation budgets of 2-5%, group-aware Coreset gives the largest reported gains; for example, ACDC weak supervision jumps from 45.2 DICE with vanilla Coreset to 52.3 DICE at 2% annotation.","Combining active learning with weak supervision works: the method is competitive or best in both weak and full annotation settings, suggesting scribble-based active learning is a viable cost cut.","Slice-based selection with the learned metric outperforms volume-based random sampling for equal annotation time, per the paper's Figure 2.","The grouping idea transfers to video: treating videos as volumes and frames as slices yields strong results on DAVIS, so the method is not confined to medical images.","With pre-trained segmentation backbones, the method still improves mean DICE over baselines on ACDC, CHAOS, and DAVIS, though the gains shrink.","Inference: the same group-contrastive encoder could be trained once and reused across active learning rounds without retraining, which would cut the reported 24-hour ACDC experiment cost; the paper does not test this.","Inference: patient, volume, and slice groupings are proxies for covariate shift, so at a deployment site with different anatomy distributions the metric's diversity may miss task-relevant slices; a testable extension would build groups from clustering the embedding itself.","Inference: the ablation ranking, with volume group best and adjacent-slice group worst, suggests the diversity signal matters more than local redundancy; one could test this by weighting group losses by measured within-group variance rather than tuning weights."],"supporting_citations":[{"why":"Supplies the Coreset formulation and the radius-based bound that the paper adapts when replacing the Euclidean metric with the learned $d_\\phi$.","marker":"[37]"},{"why":"NT-Xent loss is the base contrastive objective that the proposed group-based contrastive loss modifies.","marker":"[43]"},{"why":"Provides the SimCLR training framework used to learn the encoder $g_\\phi$ that defines the metric.","marker":"[74]"},{"why":"Source of the K-Center Greedy algorithm used to solve the Coreset optimization and of the Random sampling baseline.","marker":"[42]"},{"why":"The DMPLS weakly supervised segmentation model used to evaluate annotation efficiency with scribble labels.","marker":"[18]"},{"why":"Provides the scribble annotations and the 15x annotation-speed assumption used in the annotation-time comparison.","marker":"[17]"}],"fun_headline_variants":["Group-aware metric improves 3D slice active learning","Patient, volume, slice groups lift 3D active learning","Contrastive grouping fixes coreset for 3D segmentation","Slice-level active learning gains from inherent data groups","Grouped metric learning cuts 3D segmentation labeling cost"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that Euclidean distance in the learned embedding is a faithful proxy for how much a slice will improve the segmentation model, and the formal bound additionally assumes that small changes in the learned distance guarantee small changes in the loss and label function, with the model perfectly fitting the training data.","fun_headline_variants_meta":{"raw":{"variants":["Group-aware metric improves 3D slice active learning","Patient, volume, slice groups lift 3D active learning","Contrastive grouping fixes coreset for 3D segmentation","Slice-level active learning gains from inherent data groups","Grouped metric learning cuts 3D segmentation labeling cost"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000216,"raw_usage":{"total_tokens":1474,"prompt_tokens":1032,"completion_tokens":442,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":648,"completion_tokens_details":{"reasoning_tokens":362}},"tokens_in":648,"tokens_out":442,"duration_ms":4236,"temperature":1.0,"reasoning_tokens":362,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:55:21.673024+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On the ACDC dataset, compare the 2% weak-supervision DICE of the full method (52.3 in the main table) against the same pipeline with the metric replaced by raw-pixel Euclidean distance and by an embedding trained on randomly shuffled group labels. If either replacement retains the same gap over vanilla Coreset (45.2), the group-aware metric itself is not the cause of the gain; additionally, measuring the ratio $|L(\\hat y_1,y_1)-L(\\hat y_2,y_2)|/d_\\phi(x_1,x_2)$ on selected pairs would test the proportionality assumption directly.","supporting_citations":[{"cited_title":"Towards robust and reproducible active learning using neural networks","cited_arxiv_id":null,"evidence_quote":"Source of the K-Center Greedy algorithm used to solve the Coreset optimization and of the Random sampling baseline."},{"cited_title":"Scribble-supervised medical image segmentation via dual-branch network and dynamically mixed pseudo labels supervision","cited_arxiv_id":null,"evidence_quote":"The DMPLS weakly supervised segmentation model used to evaluate annotation efficiency with scribble labels."},{"cited_title":"Learning to segment from scrib- bles using multi-scale adversarial attention gates","cited_arxiv_id":null,"evidence_quote":"Provides the scribble annotations and the 15x annotation-speed assumption used in the annotation-time comparison."}],"review_version":1}