{"id":"6346cd33-a235-40a7-ac98-e2d14444d503","arxiv_id":"2608.11076","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"FEEDS uses DINOv2 embeddings of PET maximum-intensity projections to select farthest unlabeled cases for annotation, matching fully-labeled training performance on AutoPET-III, DeepPSMA, and Dartmouth data with 70% less labeling.","lead":"This paper presents a data-selection method that uses a computer vision foundation model to pick the most diverse and informative PET/CT scans for a radiologist to label, then trains a segmentation model once on the selected plus a small initial set. The authors report that this one-step strategy matches the performance of training on 100% of labels while needing only 30% of the annotation effort, across three test datasets and two tracers.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim depends on an unvalidated proxy: Eq. 1's DINOv2-MIP distance to the fixed labeled set is never shown to track segmentation informativeness, and one-shot farthest selection can select a redundant tail rather than a diverse, informative set.","rationale":"The most load-bearing condition for the central claim is not the statistical test details but the causal mechanism: DINOv2-MIP distance must rank unlabeled scans by how much they improve the segmentation model. The authors' own limitation section concedes the 2D MIP loses spatial information and DINOv2 is not PET-specific, but the paper does not quantify whether the residual embedding signal correlates with segmentation informativeness. This matters because the method is specifically a selection rule; if the rule is equivalent to random or to selecting a cluster of outliers, the headline result is not attributable to FEEDS. The random baselines already show that 30% labels put Dice within roughly 0.02-0.03 of the 100% model, so non-inferiority on Dice with a 0.05 margin is not a discriminating test of FEEDS. The only robust differentiator is FP volume, and the current design does not rule out the simpler explanation that farthest cases are disproportionately lesion-free or protocol outliers that suppress false positives without improving detection. A closest-case control and an oracle-error-selection control would settle this. This is the same load-bearing assumption the reader identified, so the conditional verdict is appropriate; the condition should be empirical validation of the selection proxy, not just corrected statistics.","tokens_in":14683,"tokens_out":10183,"duration_ms":97339,"concrete_test":"On AutoPET-III, train two extra 10+20% nnU-Net baselines using the same fixed 10% labeled set: (1) closest 20% by Eq. 1 (smallest d_j) and (2) oracle-selected 20% with largest per-case Dice error of the 10%-only model on the unlabeled pool. Compare Dice, FPVol, and FNVol on the held-out test set, and report the disease/tracer composition of the FEEDS-selected set. If closest-case training matches FEEDS, the distance ranking is not the driver. If oracle selection beats FEEDS by more than 0.02 Dice, the chosen proxy is not the right informativeness measure. If FEEDS-selected cases are mostly lesion-free, the FP benefit is explained by negative-case oversampling rather than diversity.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"FEEDS's central promise is that cases with largest d_j in Eq. 1 are the most informative to annotate. The paper never tests this link. No analysis correlates d_j with any measure of segmentation error or expected improvement of the 10%-model, and no control uses the closest cases. Because Eq. 1 computes distance only to the fixed 10% labeled set and not to previously selected cases, the top-20% by distance can be a mutually redundant tail of the embedding distribution (e.g., one scanner/protocol cluster or a set of lesion-free scans) rather than a diverse coverage of error-prone lesion patterns. Consistent with this, FEEDS's only significant advantage over random is lower false-positive volume; its Dice is equal to random on AutoPET (0.613 vs 0.613) and worse on DeepPSMA (0.645 vs 0.653). The FP reduction could therefore come from oversampling negative or atypical cases, not from the diversity mechanism the method is named for. Section 5 concedes the MIP projection loses spatial information, but no experiment quantifies whether the embedding ranking preserves the information needed for 3D lesion segmentation. Until the proxy is validated, the causal claim that farthest-first DINOv2 selection produces the reported gains is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FEEDS, a one-shot label-efficient data sampling strategy for whole-body PET/CT lesion segmentation. FEEDS computes DINOv2 embeddings of 2D maximum-intensity projections of PET volumes, measures the minimum cosine distance from each unlabeled scan to a fixed 10% labeled AutoPET-III training pool (Eq. 1), and selects the farthest 20% of scans per tracer for expert annotation. These 30% of labeled data are then used to train an nnU-Net, which is evaluated on a held-out AutoPET-III test set, the Deep-PSMA dataset, and an internal DHMC dataset with voxel-, lesion-, and anatomical-region-level metrics. The paper claims that FEEDS outperforms random sampling and matches fully-labeled (100%) training performance with 70% less annotation burden.","tokens_in":14920,"tokens_out":4021,"duration_ms":39036,"significance":"If the claims are fully supported, FEEDS would be practically valuable because it requires no iterative active-learning or semi-supervised training, uses a frozen general-purpose foundation model, and is evaluated on multiple external datasets with clinically oriented metrics. The inclusion of lesion-level and anatomical-region-level analysis, the comparison against DPP-based sampling and pseudo-label SSL, and public code release are strengths. However, the evidence in the manuscript supports a more limited claim: consistent and statistically significant improvement over random sampling is shown for false-positive volume, whereas Dice improvements are absent or reversed on two of three datasets, and the causal role of the proposed selection rule is not directly validated.","major_comments":[{"comment":"The central claim that the largest dj identifies the most informative unlabeled cases for segmentation is not validated. No experiment correlates dj with any per-case measure of segmentation error or expected improvement of the 10%-labeled model, no control experiment selects the closest cases instead of the farthest, and the distance is computed only to the fixed labeled set, not to previously selected cases, so the top 20% can be a mutually redundant tail of the embedding distribution rather than a diverse informative set. Please add a closest-case control and/or an analysis showing that selected cases cover lesion patterns that cause errors in the 10% model.","section":"§3.1.2, Eq. (1)"},{"comment":"The statistical results do not support the broad claim that FEEDS outperforms random sampling. On AutoPET, Dice is identical (0.613 vs 0.613); on DeepPSMA, FEEDS Dice is lower than random (0.645 vs 0.653); only DH shows a Dice improvement, with p=0.050. False-positive volume is the only metric with consistently significant improvement across all datasets, while false-negative volume differences are not significant. The paper should reframe its central claim to a false-positive-volume reduction with non-inferiority in Dice and false-negative volume, and discuss why the diversity mechanism specifically reduces false positives.","section":"§4.4, Table 6"},{"comment":"The non-inferiority margins stated in the text (0.05 Dice, 5 cc) disagree with those reported in Table 6 (0.02 Dice, 2 mL). This is not a cosmetic discrepancy: DeepPSMA FNVol differs by 4.35 cc between FEEDS and random, which is within 5 cc but outside 2 mL, and Table 6 marks the NI test as failed (×). Also, the text says non-inferiority passes in the pooled analysis for all three metrics, but Table 6 shows a failure for DeepPSMA FNVol. Please reconcile the margins, report both analyses, and justify the chosen margin.","section":"§4.4, Table 6"},{"comment":"The reported sample sizes are inconsistent. Section 2.1 states that the AutoPET-III test set has 321 scans, but Table 6 reports n=207 for AutoPET Dice and FNVol. If the Dice/FN analysis is limited to diseased cases only, that filtering is not described in the statistical analysis section, and the pooled n=430 (207+200+23) should be explained. These decisions affect the paired Wilcoxon tests and the strength of the non-inferiority claims.","section":"§2.1, §4.4, Table 6"},{"comment":"The authors concede that the 2D MIP projection can lose spatial information and that DINOv2 is a natural-image model, but no experiment quantifies whether the embedding ranking preserves the information needed for 3D lesion segmentation. Given that the only significant advantage over random sampling is lower false-positive volume, it remains plausible that the selection preferentially oversamples negative or atypical scans rather than error-prone lesion patterns. A focused analysis of what the selected cases actually contain—lesion counts, sizes, tracers, scanners—would help establish the mechanism claimed in the paper.","section":"§5, Limitations"}],"minor_comments":[{"comment":"The text states that the DINOv2 feature representation is f ∈ R^768, but Figure 2 refers to a 784-dimensional feature space; these numbers should be reconciled.","section":"§3.1.1, Figure 2"},{"comment":"The narrative says FEEDS performs consistently better than random sampling, but at the 10+10% budget FEEDS Dice equals the random mean (0.637 vs 0.637), and at 10+20% FEEDS FPVol is higher than the random mean (18.98 vs 18.33); the phrasing should be softened to reflect the actual pattern.","section":"§4.1, Table 2"},{"comment":"The abstract and conclusion state that FEEDS outperforms random sampling on all evaluated metrics, but Table 6 shows no significant Dice or FNVol differences on AutoPET and DeepPSMA; the abstract and conclusion should be revised to match the evidence.","section":"§4.4, Table 6"},{"comment":"On the DHMC cohort, the model trained on only 10% fixed labeled data achieves the highest Dice and lowest FPVol, with FEEDS second. This counterintuitive result is not discussed beyond the 100% model; the authors should explain why the 10% model outperforms both FEEDS and random on this small cohort.","section":"§4.3, Table 5"},{"comment":"The statistical analysis says a two-sided Wilcoxon signed-rank test with FDR-BH correction is used for FEEDS versus random comparisons, but the exact pairing structure and the handling of ties in Dice values (where many per-case scores may be equal) are not described; a sentence clarifying the test procedure would strengthen reproducibility.","section":"§4.4"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nFEEDS is a practical one-shot annotation selection strategy: compute DINOv2 embeddings of MIPs, select the unlabeled scans farthest from a small fixed labeled set, add them, train nnU-Net once. The genuinely new bit is applying this to pan-cancer, multi-tracer whole-body PET/CT and evaluating at voxel, lesion, and region levels across three datasets. That evaluation breadth is real credit; most papers in this space stop at Dice on one test set.\n\nThe paper also deserves credit for comparing against DPP, random sampling with five repeats, and pseudo-label SSL, and for reporting per-case statistics with non-inferiority margins. The core operational finding—that 30% labels can land close to 100% performance—is plausible and supported, at least for non-inferiority.\n\nNow the soft spots. First, 'outperforms random sampling' is not true on Dice: on AutoPET test it is exactly equal (0.613 vs 0.613) and on DeepPSMA it is lower (0.645 vs 0.653). The consistent win is false-positive volume, which is clinically meaningful but a different claim. Second, Eq. 1 measures distance only to the fixed labeled set, not to previously selected cases, so the top 20% can be a redundant cluster. The paper never validates that the distance ordering tracks segmentation informativeness; the stress-test concern is fair. A quick control using closest cases or a correlation with per-case error would settle this. Third, there are internal inconsistencies: Section 4.4 claims DH improved in all three metrics, but Table 6 shows FN p=0.948; the non-inferiority margin is stated as 0.05 Dice/5cc in text but 0.02/2mL in Table 6. These need fixing. Minor: FEEDS itself has no repeated-run variance; random sampling gets five seeds, FEEDS gets one.\n\nThe paper is overclaimed but not broken. The mechanism is plausible, the evaluation is broad, and the flaws are fixable reporting issues plus one missing validation. I'd send it to review with a request to temper the abstract, add a farthest-vs-closest control, and reconcile the stats.","headline":"A practical label-efficient sampling strategy for PET/CT that mostly delivers on false-positive reduction and non-inferiority, but its 'outperforms random' claim overreaches on Dice and the selection criterion is not validated as a diversity proxy.","tokens_in":15508,"tokens_out":3136,"would_cite":true,"duration_ms":29329,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single farthest-first pass over unlabeled PET/CT scans, using DINOv2 embeddings, produces a lesion segmentation model that matches full-label training with 70% fewer annotations.","keywords":["PET/CT lesion segmentation","label-efficient training","farthest-first sampling","DINOv2 embeddings","whole-body imaging","active learning","semi-supervised learning","nnU-Net"],"falsifier":"Compare FEEDS against random sampling on a cohort where each scan's true lesion count and segmentation difficulty are known; if the farthest-first additions do not systematically lower false-negative volume relative to random additions across repeated random seeds, then embedding distance is not selecting the informative lesions.","tokens_in":14460,"feed_emoji":"🩻","tokens_out":10345,"duration_ms":81182,"temperature":0.7,"pith_summary":"FEEDS is a one-step method for deciding which unlabeled PET/CT scans to annotate when building a whole-body lesion segmentation model. It claims that selecting the 20% of scans farthest from a fixed 10% labeled pool, using DINOv2 embeddings of maximum-intensity projections, closes most of the gap between limited labels and full labeling. On the AutoPET-III test set and two unseen datasets, the resulting model is statistically non-inferior to a model trained on 100% of the labels while using 70% fewer annotations. The gains are most consistent in false-negative volume and lesion-level sensitivity, and FEEDS also beats random sampling, DPP sampling, and pseudolabel semi-supervision at the same label budget. If the claim holds, it gives clinical groups a practical way to prioritize annotation effort in large unannotated PET/CT archives.","feed_headline":"Farthest-first sampling needs 30% of labels to match full training","feed_subtitle":"DINOv2 embeddings pick the most informative PET/CT scans to label, cutting annotation effort by 70%.","key_machinery":"The load-bearing object is a tracer-stratified farthest-first sampling rule in DINOv2 feature space. DINOv2 is a self-supervised vision model pretrained on 142 million natural images; here it encodes z-score-normalized maximum-intensity projections of each 3D PET scan into 768-dimensional embeddings. For each unlabeled scan $j$ of a tracer type, the rule computes $d_j = 1 - \\max_{i \\in L_t} \\frac{z_j \\cdot z_i}{\\|z_j\\| \\|z_i\\|}$, the minimum cosine distance to the labeled set $L_t$ of the same tracer, and selects the scans with the largest $d_j$. This greedy procedure fills gaps in the training distribution by adding cases least represented by the existing labels, and it preserves the FDG:PSMA ratio because selection is done separately per tracer. The selected cases are annotated and combined with the fixed 10% labeled pool to train a single nnU-Net model, with no iterative retraining or pretraining loop.","core_discovery":"FEEDS' central discovery is that diversity in a self-supervised embedding space, measured by the minimum cosine distance to a small labeled set, is a usable proxy for training informativeness in whole-body PET/CT lesion segmentation. For each tracer type, the method embeds every scan's maximum-intensity projection with DINOv2, computes $d_j$ as the distance to the labeled set, and annotates the scans with the largest $d_j$. Trained with 30% of the AutoPET-III labels (the fixed 10% pool plus the farthest 20%), the nnU-Net model matches the fully supervised model under the paper's non-inferiority margin, with lower false-positive volume and comparable or better lesion-level positive predictive value. Raw Dice on the AutoPET-III test set is slightly lower for FEEDS than for full training (0.61 vs. 0.64), but the difference falls within the margin the paper defines, and the effect generalizes to held-out AutoPET-III, Deep-PSMA, and an internal clinical dataset across FDG and PSMA tracers and multiple cancer types.","pith_inferences":["A reader could test whether the same farthest-first rule improves other dense prediction tasks, such as organ or tumor segmentation in CT or MRI, whenever a small labeled pool and a large unlabeled archive exist.","Because the paper uses only PET maximum-intensity projections, replacing them with 3D PET or combined PET+CT embeddings is a natural extension; the paper itself notes the 2D projection can lose spatial information, so such a test would reveal how much of the gain depends on that projection.","The same ranking could also be used to build a prioritized annotation queue for a clinical archive, letting radiologists annotate only the most informative scans; the paper frames this as a deployment benefit but does not measure the time saved.","A fair reading of 'matching 100%' is statistical non-inferiority within pre-specified margins rather than equal performance, since the raw Dice on the AutoPET-III test set is 0.61 for FEEDS versus 0.64 for full training."],"forward_implications":["A model trained with FEEDS-selected 30% labels is non-inferior in Dice to a model trained with 100% labels on the AutoPET-III test set, and it reduces false-positive volume relative to random sampling.","FEEDS generalizes to unseen Deep-PSMA and internal clinical scans across FDG and PSMA tracers and multiple cancer types, with lesion-level sensitivity near or above random sampling.","At the same 30% label budget, FEEDS yields lower false-negative volume than random sampling on the AutoPET-III validation and test sets, meaning fewer whole lesions are missed.","FEEDS outperforms one-iteration pseudolabel-based semi-supervised learning, which produced noisy labels and worse Dice than training on the same amount of strongly labeled data.","The selection step runs once before any training, so the annotation queue can be produced with a single pass of feature extraction, unlike active learning or semi-supervised loops."],"supporting_citations":[{"why":"Supplies the DINOv2 encoder whose embeddings define the distance metric in the farthest-first selection rule.","marker":"[23]"},{"why":"Provides the nnU-Net segmentation model trained on the FEEDS-selected labeled set and used for all comparisons.","marker":"[16]"},{"why":"Provides the PET/CT preprocessing recipe used before nnU-Net training.","marker":"[26]"},{"why":"Supplies the AutoPET-III training, validation, and test scans that are the primary experimental material.","marker":"[15]"},{"why":"Supplies the Deep-PSMA unseen test set used to test cross-dataset generalization.","marker":"[21]"},{"why":"Supplies the whole-body FDG-PET/CT dataset with manually annotated tumor lesions that underlies part of AutoPET-III.","marker":"[10]"},{"why":"Supplies the whole-body PSMA-PET/CT dataset with manual annotations used for the PSMA portion of the experiments.","marker":"[17]"}],"fun_headline_variants":["Farthest-first scan selection slashes PET/CT labeling workload by 70%","Diversity-based AI sampling needs only 30% of labels for full performance","DINOv2 picks diverse lesions, matching full training with a third of labels","70% less annotation effort, same PET/CT segmentation accuracy","Foundation model embeddings cut PET/CT annotation burden by 70%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that how different an unlabeled scan looks from the already-labeled scans, as measured by distance in a self-supervised image embedding, is the same as how much that scan will teach the segmentation model; if that link breaks, farthest-first selection adds outliers instead of informative lesions.","fun_headline_variants_meta":{"raw":{"variants":["Farthest-first scan selection slashes PET/CT labeling workload by 70%","Diversity-based AI sampling needs only 30% of labels for full performance","DINOv2 picks diverse lesions, matching full training with a third of labels","70% less annotation effort, same PET/CT segmentation accuracy","Foundation model embeddings cut PET/CT annotation burden by 70%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000315,"raw_usage":{"total_tokens":1863,"prompt_tokens":1097,"completion_tokens":766,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":713,"completion_tokens_details":{"reasoning_tokens":667}},"tokens_in":713,"tokens_out":766,"duration_ms":6745,"temperature":1.0,"reasoning_tokens":667,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:47:52.475463+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare FEEDS against random sampling on a cohort where each scan's true lesion count and segmentation difficulty are known; if the farthest-first additions do not systematically lower false-negative volume relative to random additions across repeated random seeds, then embedding distance is not selecting the informative lesions.","supporting_citations":[{"cited_title":"nnu-net: a self- configuring method for deep learning-based biomedical image segmentation.Nature methods, 18(2):203–211, 2021","cited_arxiv_id":null,"evidence_quote":"Provides the nnU-Net segmentation model trained on the FEEDS-selected labeled set and used for all comparisons."},{"cited_title":"Au- tomated lesion segmentation in whole-body pet/ct - multitracer multicenter generalization","cited_arxiv_id":null,"evidence_quote":"Supplies the AutoPET-III training, validation, and test scans that are the primary experimental material."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Deep-PSMA unseen test set used to test cross-dataset generalization."},{"cited_title":"A whole-body fdg-pet/ct dataset with manually annotated tumor lesions.Scientific Data, 9(1):601, 2022","cited_arxiv_id":null,"evidence_quote":"Supplies the whole-body FDG-PET/CT dataset with manually annotated tumor lesions that underlies part of AutoPET-III."},{"cited_title":"Jeblick et al","cited_arxiv_id":null,"evidence_quote":"Supplies the whole-body PSMA-PET/CT dataset with manual annotations used for the PSMA portion of the experiments."}],"review_version":1}