{"id":"2465d958-08a3-4b4c-9133-41d0609b314f","arxiv_id":"2607.22371","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A PPO-based agent selects few-shot support sets by maximizing validation Dice, outperforming random, TBP, and DVRL on cross-institutional pelvic MRI segmentation.","lead":"The paper trains a reinforcement-learning agent to pick which few medical images should be used to adapt a segmentation model, choosing the whole support set rather than ranking images one by one. On a cross-institutional pelvic MRI benchmark it reports higher Dice scores than random selection and two existing selection methods.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation protocol may fine-tune the RL policy on target-domain validation labels (Sec 3.1 vs Sec 2.3), making the comparison against random/TBP/DVRL unfair and potentially explaining all reported gains.","rationale":"I read the paper as proposing a PPO policy that, given an unlabeled candidate pool, predicts a K-sparse support set and is trained on rewards from post-adaptation validation Dice. The central claim is empirical: selected sets beat random/TBP/DVRL. For this to hold, the policy must not be given target-domain labels beyond the K support examples. The manuscript is internally ambiguous on exactly this point: Sec 2.3 explicitly forbids reward computation at inference, yet Sec 3.1 includes 8 target-domain validation volumes 'for reinforcement learning reward computation' in every evaluation experiment. If those volumes update the policy, the comparison is not a few-shot comparison and the reported gains could be due to extra labels. This is more load-bearing than the unspecified architecture, because it would invalidate even the K=4 result that is the paper's strongest evidence. A second issue—repeated independent K=4 selections for K=8/16 (Sec 3.3)—would only weaken the K=8/16 interpretation, not the core K=4 claim, so I did not choose it as the primary attack. The reader's weakest assumption partially overlaps (small validation reward signal), but the sharper formulation is the label-budget contradiction. One concrete check from the released code can settle it; therefore the paper should remain conditional pending that check.","tokens_in":7083,"tokens_out":11019,"duration_ms":121733,"concrete_test":"Use the released code to check whether any PPO parameter update consumes rewards computed from the 8 target-domain validation volumes during the evaluation experiments. Then run a clean evaluation in true inference mode: freeze the policy trained only on source domains, with no target-domain reward computation or fine-tuning, and compare test Dice against random, TBP, and DVRL at K=4, P=32. If the advantage over random disappears or drops below significance, the reported results are explained by target-domain label leakage; if the advantage persists, the concern is resolved.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central empirical claim is that a PPO policy selects support sets that improve few-shot Dice on a held-out target domain. That claim requires that the policy be evaluated without access to target-domain labels beyond the K support examples. The manuscript appears to violate this requirement. Sec 2.3 states: 'No policy optimisation or reward computation is performed during inference.' Yet Sec 3.1 states that for each evaluation experiment 'a further 8 examples were used as the validation set for reinforcement learning reward computation.' If those 8 target volumes are used to train or fine-tune the policy before selecting the support set, the proposed method has access to V=8 labelled target-domain images that random selection and TBP do not; even DVRL's validation-set usage may not match this protocol. The reported 8.1-ppt gain over random at K=4 could then be caused by supervised exposure to the target domain rather than by learning set-level complementarity. The text never specifies whether RL training is done on source domains only or per target domain, nor how the 8-image validation set is used. This is the single most load-bearing concern because it threatens all results, including the strongest K=4 claim, and is directly testable from the released code.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a reinforcement-learning (RL) framework for support-set selection in few-shot medical image segmentation. Given a pool of P unlabelled images from a target domain, a policy π(s;φ) predicts a binary selection mask a with exactly K ones; the selected images are labelled, used to adapt a Reptile (or MAML) meta-trained UNet, and the reward is the Dice score on a validation set. Experiments on a cross-institutional male pelvic MRI dataset (NB and OI structures from held-out institutions 3 and 4) compare the method against random selection, TBP, and DVRL for K = 4, 8, 16. The paper reports consistent and statistically significant improvements, with combined Dice at K = 4 of 59.4% versus 51.2% (random), plus ablations on candidate pool size and the meta-learning backbone. The central claim is that jointly optimised, set-level support-set selection outperforms individual-sample scoring and random selection.","tokens_in":7362,"tokens_out":3301,"duration_ms":39213,"significance":"If the claim holds, the contribution is valuable: it addresses a real and under-explored problem, and the set-level formulation is a principled departure from per-sample active-learning and data-valuation methods. The paper also provides a public code repository, which is welcome. However, the current manuscript does not provide enough implementation detail to reproduce the method, and the evaluation protocol as written creates a potentially serious fairness problem: Section 3.1 says that a further 8 labelled examples are used as a validation set for 'reinforcement learning reward computation', but Section 2.3 says no reward computation is performed during inference. If the policy is trained or fine-tuned on target-domain validation labels, the comparison against random/TBP/DVRL is not apples-to-apples and the reported gains could be explained by access to extra target-domain supervision. Because this ambiguity affects every reported result, the significance of the work cannot be assessed from the manuscript as currently written.","major_comments":[{"comment":"The evaluation protocol is internally inconsistent. Sec. 2.3 states 'No policy optimisation or reward computation is performed during inference', but Sec. 3.1 states that for each evaluation experiment 'a further 8 examples were used as the validation set for reinforcement learning reward computation'. If those 8 target-domain labels are used to train or fine-tune the policy before support-set selection, then the proposed method has access to V = 8 labelled target-domain images that random selection and TBP do not. This would invalidate the comparison in Table 1. The paper must specify clearly whether RL training is performed only on source domains, or per target domain, and exactly how the 8 validation examples are used. If per-target training or fine-tuning is performed, the experiments need to be redone under a protocol that gives comparators the same information.","section":"Sec. 3.1 vs Sec. 2.3"},{"comment":"The policy is described only as π(s;φ), with no architecture, no input encoding, no action distribution, and no mechanism to enforce the constraint ∑ a_j = K. Since the central claim is that a learned policy captures set-level complementarity, the representation of the candidate pool and the sampling procedure for a K-sparse binary mask are load-bearing. The paper should specify the network architecture, how variable-size pools are handled, how the K-of-P constraint is enforced (e.g., Gumbel-top-K, REINFORCE with a combinatorial action space, or a deterministic mask from a scoring network), and the PPO hyper-parameters used. Without these details the method is not reproducible.","section":"Sec. 2.2, Eq. (4)"},{"comment":"The sentence 'in actuality K remained fixed at K = 4 and where K > 4, selection was repeated multiple times with randomly sampled candidate pools' is ambiguous and potentially inconsistent with Table 1, which reports results for K = 8 and K = 16. If the policy always selects 4 images and the support set for K > 4 consists of multiple such selections, the adaptation procedure in Eq. (2) with K examples needs to be described precisely. If instead the policy is trained at each K, that should be stated. As written, a reader cannot determine what was actually evaluated for K = 8 and K = 16.","section":"Sec. 3.3"},{"comment":"The statistical reporting is too terse. The paper reports 'all p-values < 0.01' for comparisons against three baselines across three K values and two structures, but does not state the number of test volumes per comparison, whether the t-test is one-sided or two-sided, or whether any multiple-comparison correction was applied. Given that 27 comparisons are made (3 baselines × 3 K values × 3 rows including combined), uncorrected p < 0.01 is not by itself convincing. The paper should report effect sizes, confidence intervals, and the correction procedure.","section":"Sec. 4, Table 1"}],"minor_comments":[{"comment":"The dataset description says 'up to 16 labelled examples were used for support-set adaptation and a further 8 examples were used as the validation set for reinforcement learning reward computation'. The phrase 'up to' is vague; specify exactly how many validation examples are used and whether the same 8 are used for all experiments.","section":"Sec. 3.1"},{"comment":"The meta-training objective writes an expectation over source domains, but the Reptile procedure is not described. Please state explicitly how tasks/domains are sampled and how Reptile's inner-loop adaptation is performed, since this is relevant to the few-shot adaptation mechanism that the RL policy is optimizing.","section":"Sec. 2.1, Eq. (1)"},{"comment":"The ablation for P reports p = 0.09 for P = 16 vs P = 32, but the main text says 'statistical significance was not observed'. The same level of detail should be provided for all comparisons, not only for the ablations.","section":"Sec. 4"},{"comment":"The qualitative example is interesting, but claims about 'broader range of image appearances' and 'oversegmentation' are not quantified. A figure with a single query and no error maps is not strong evidence for complementarity; consider adding quantitative artefact-specific Dice or error heatmaps.","section":"Fig. 2"},{"comment":"The UNet architecture is described as receiving a 'grayscale image (single channel)', while the dataset is 3D with dimensions 256×256×32. Please clarify whether the 3D volumes are processed slice-wise or as 3D inputs, and whether the 3×3×3 convolutions operate on 2D or 3D grids.","section":"Sec. 3.2"}],"recommendation":"major_revision","confidential_remarks":"The central issue is the protocol ambiguity in Sec. 3.1 versus Sec. 2.3. If the code shows that the RL policy is trained only on source domains and the 8 validation examples are used solely as a fixed reward signal during training on source domains, the paper could be acceptable after major revision. If the policy is trained or fine-tuned per target domain using those 8 labels, the comparison is unfair and the paper should be rejected or substantially reworked. I recommend requiring the authors to disclose the exact training/evaluation protocol and to make the policy architecture and hyper-parameters available in the manuscript, not only in the repository."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this for the question it raises about evaluation hygiene, not for the numbers as reported. The genuinely new thing is framing support-set selection for few-shot segmentation as a set-level RL problem rather than scoring samples independently. That is a fair gap, and the experiments show consistent gains over random, TBP, and DVRL across K=4, 8, 16 on a cross-institutional MRI set, with ablations on pool size and MAML/Reptile. If the protocol is clean, an ~8 Dice point gain at K=4 is practically meaningful.\n\nThe problem is that the paper never makes the protocol clean. Section 2.3 says no reward computation during inference, but Section 3.1 says that for each evaluation experiment “a further 8 examples were used as the validation set for reinforcement learning reward computation.” That sentence is hard to read as anything other than the RL policy being trained or fine-tuned on the target domain using 8 labelled validation volumes. If that is happening, the comparison to random and TBP is not apples-to-apples: the proposed method gets supervised exposure to the target distribution that the baselines do not. The stress-test note is on target.\n\nAlso missing: the policy architecture (π(s;ϕ) is never given a form), how a K-of-P binary mask is sampled from a variable-size pool, and the PPO hyperparameters. The K>4 protocol is ambiguous too — “K remained fixed at K=4” but results are reported for K=8 and 16; the text suggests repeated random pools combined, but that needs spelling out.\n\nTo be fair, circularity in the narrow sense is not the issue: the final test set appears disjoint, and the reward is validation Dice, not test Dice. The issue is fairness, not circularity. The paper is not fatally flawed in design; it is incomplete and possibly leaking access to target-domain labels.\n\nIf the authors clarify the training/inference split and release code, this could be a solid paper. As written, I would not trust the headline numbers. My recommendation: send it to peer review, but with a clear note that the protocol ambiguity is load-bearing and the authors must specify exactly when the RL policy is trained and whether target validation labels are used at any point during evaluation. A reproducibility check on the code would settle it.","headline":"Plausible and potentially useful, but the evaluation protocol as written may leak target-domain labels into the RL policy; that has to be fixed before the central claim can be trusted.","tokens_in":7870,"tokens_out":2029,"would_cite":false,"duration_ms":21014,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Reinforcement learning can be used to select the few images that best adapt a segmentation model, and on cross-institutional pelvic MRI this beats random selection, TBP, and DVRL at K=4, 8, and 16.","keywords":["few-shot learning","support-set selection","reinforcement learning","medical image segmentation","active learning","meta-learning","pelvic MRI","data selection"],"falsifier":"Run the same protocol with different random 8-image validation splits (or with 4 or 16 validation images) and check whether the Dice advantage over random selection persists. If the advantage disappears or reverses, the policy is exploiting a specific validation set rather than learning a generalizable notion of support-set complementarity.","tokens_in":6973,"feed_emoji":"🎯","tokens_out":4726,"duration_ms":50004,"temperature":0.7,"pith_summary":"This paper asks a practical question: when a few-shot segmentation model can be adapted with only a handful of labelled images, which handful should you pick? The authors propose training a reinforcement-learning agent that looks at a pool of unlabelled candidate images and directly outputs a K-image support set, with the reward being the segmentation Dice the adapted model achieves on a small validation set. In experiments on a cross-institutional pelvic MRI dataset, the agent's selections outperform random selection and two sample-scoring baselines at every support-set size tested, with the largest gain at K=4. The central insight is that support-set quality is a property of the whole set, not a sum of individual sample utilities, so the selection policy is optimised jointly rather than by ranking samples independently.","feed_headline":"RL-trained selection lifts few-shot MRI Dice by 8 points","feed_subtitle":"Agent chooses support sets jointly, beating random, TBP, and DVRL across K=4–16.","key_machinery":"The key mechanism is the RL formulation of support-set selection: the state is the set of P unlabelled candidate images, the action is a binary mask a∈{0,1}^P with exactly K ones, and the reward is the post-adaptation validation Dice. Policy parameters are optimised with PPO to maximise expected reward. This enables the agent to evaluate the support set as a whole, rather than scoring each image independently, and to learn selection strategies that balance coverage, informativeness, and complementarity without hand-crafted heuristics.","core_discovery":"The paper's central claim is that support-set selection for few-shot medical image segmentation can be formulated as a combinatorial decision problem and learned end-to-end with reinforcement learning. A policy network maps a candidate image pool to a binary selection mask that picks exactly K images; those images are then labelled and used to adapt a fixed meta-learned segmentation model. The reward is the mean Dice similarity coefficient on a small validation set after adaptation. On a cross-institutional pelvic MRI dataset at K=4, this yields 59.4% combined Dice versus 51.2% for random selection, 53.8% for task-based prioritisation, and 54.8% for DVRL, all with p<0.01. The advantage holds","pith_inferences":["A natural extension would be to replace the fixed labelled validation set with a learned reward model, removing the extra annotation cost; the paper does not explore this direction.","If the trained policy transfers to new institutions or anatomies without retraining, it could be a plug-in component in clinical few-shot segmentation workflows, but the current evaluation is limited to two held-out structures from two institutions.","The largest relative gains occur in the lowest-data regime (K=4), suggesting the approach is most valuable when annotation budgets are extremely tight; testing at K=1 or K=2 would probe how far the benefit extends.","It remains an open question how the policy behaves when the candidate pool contains a mix of very similar and very unusual images; a controlled experiment varying pool diversity would clarify whether the agent is learning a general complementarity heuristic or overfitting to the training distribution."],"forward_implications":["At K=4, the proposed method improves combined Dice by 8.1 percentage points over random selection and 4.5 over DVRL, with all reported improvements statistically significant at p<0.01.","The method maintains its advantage at K=8 and K=16, suggesting it is useful across a range of annotation budgets.","Larger candidate pools (P=8, 16, 32) lead to monotonically better performance, confirming that more options help the policy find better support sets.","The benefit transfers across meta-learning backbones: using MAML instead of Reptile gives statistically similar performance (p=0.06), so the selection strategy is not tied to one adaptation algorithm.","Qualitative inspection shows the policy selecting an image with a distortion artefact and a broader intensity spread, supporting the claim that it captures set-level complementarity rather than just individual sample quality."],"fun_headline_variants":["RL-selected support sets boost few-shot MRI Dice by 8 points","Joint support-set selection via RL beats random and prior methods","At K=4, RL support-set selection lifts Dice to 59.4% vs 51.2% random","Complementary support sets via RL outperform individual scoring in few-shot MRI","Active few-shot segmentation improved by RL-based support-set selection"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The method depends on a fixed 8-image validation set from the target domain providing a stable enough reward signal to train the RL policy; if that reward signal is noisy or unrepresentative, the learned selection strategy may not transfer to the true test distribution.","fun_headline_variants_meta":{"raw":{"variants":["RL-selected support sets boost few-shot MRI Dice by 8 points","Joint support-set selection via RL beats random and prior methods","At K=4, RL support-set selection lifts Dice to 59.4% vs 51.2% random","Complementary support sets via RL outperform individual scoring in few-shot MRI","Active few-shot segmentation improved by RL-based support-set selection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000322,"raw_usage":{"total_tokens":1622,"prompt_tokens":695,"completion_tokens":927,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":439,"completion_tokens_details":{"reasoning_tokens":828}},"tokens_in":439,"tokens_out":927,"duration_ms":8822,"temperature":1.0,"reasoning_tokens":828,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T04:56:45.584414+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same protocol with different random 8-image validation splits (or with 4 or 16 validation images) and check whether the Dice advantage over random selection persists. If the advantage disappears or reverses, the policy is exploiting a specific validation set rather than learning a generalizable notion of support-set complementarity.","supporting_citations":[],"review_version":1}