Pith. sign in

REVIEW 4 major objections 5 minor 27 references

Reinforcement learning can be used to select the few images that best adapt a segmentation model, and on cross-institutional pelvic MRI this beats random selection, TBP, and DVRL at K=4, 8, and 16.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 04:56 UTC pith:B2MQUERD

load-bearing objection Plausible and potentially useful, but the evaluation protocol as written may leak target-domain labels into the RL policy; that has to be fixed before the central claim can be trusted. the 4 major comments →

arxiv 2607.22371 v1 pith:B2MQUERD submitted 2026-07-24 cs.CV

Active few-shot segmentation by reinforcing data selection

Chenlan Zhao , Benny Wong , Timothy F. Lundberg , Ahmed M. Elsayed , Abdallah Aljarkas , Hamad A. Aljamaan , Lynn Karam , Qianye Yang
show 3 more authors
Yipeng Hu Claire C. Villette Shaheer U. Saeed
This is my paper
classification cs.CV
keywords few-shot learningsupport-set selectionreinforcement learningmedical image segmentationactive learningmeta-learningpelvic MRIdata selection
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks a practical question: when a few-shot segmentation model can be adapted with only a handful of labelled images, which handful should you pick? The authors propose training a reinforcement-learning agent that looks at a pool of unlabelled candidate images and directly outputs a K-image support set, with the reward being the segmentation Dice the adapted model achieves on a small validation set. In experiments on a cross-institutional pelvic MRI dataset, the agent's selections outperform random selection and two sample-scoring baselines at every support-set size tested, with the largest gain at K=4. The central insight is that support-set quality is a property of the whole set, not a sum of individual sample utilities, so the selection policy is optimised jointly rather than by ranking samples independently.

Core claim

The paper's central claim is that support-set selection for few-shot medical image segmentation can be formulated as a combinatorial decision problem and learned end-to-end with reinforcement learning. A policy network maps a candidate image pool to a binary selection mask that picks exactly K images; those images are then labelled and used to adapt a fixed meta-learned segmentation model. The reward is the mean Dice similarity coefficient on a small validation set after adaptation. On a cross-institutional pelvic MRI dataset at K=4, this yields 59.4% combined Dice versus 51.2% for random selection, 53.8% for task-based prioritisation, and 54.8% for DVRL, all with p<0.01. The advantage holds

What carries the argument

The key mechanism is the RL formulation of support-set selection: the state is the set of P unlabelled candidate images, the action is a binary mask a∈{0,1}^P with exactly K ones, and the reward is the post-adaptation validation Dice. Policy parameters are optimised with PPO to maximise expected reward. This enables the agent to evaluate the support set as a whole, rather than scoring each image independently, and to learn selection strategies that balance coverage, informativeness, and complementarity without hand-crafted heuristics.

Load-bearing premise

The method depends on a fixed 8-image validation set from the target domain providing a stable enough reward signal to train the RL policy; if that reward signal is noisy or unrepresentative, the learned selection strategy may not transfer to the true test distribution.

What would settle it

Run the same protocol with different random 8-image validation splits (or with 4 or 16 validation images) and check whether the Dice advantage over random selection persists. If the advantage disappears or reverses, the policy is exploiting a specific validation set rather than learning a generalizable notion of support-set complementarity.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • At K=4, the proposed method improves combined Dice by 8.1 percentage points over random selection and 4.5 over DVRL, with all reported improvements statistically significant at p<0.01.
  • The method maintains its advantage at K=8 and K=16, suggesting it is useful across a range of annotation budgets.
  • Larger candidate pools (P=8, 16, 32) lead to monotonically better performance, confirming that more options help the policy find better support sets.
  • The benefit transfers across meta-learning backbones: using MAML instead of Reptile gives statistically similar performance (p=0.06), so the selection strategy is not tied to one adaptation algorithm.
  • Qualitative inspection shows the policy selecting an image with a distortion artefact and a broader intensity spread, supporting the claim that it captures set-level complementarity rather than just individual sample quality.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension would be to replace the fixed labelled validation set with a learned reward model, removing the extra annotation cost; the paper does not explore this direction.
  • If the trained policy transfers to new institutions or anatomies without retraining, it could be a plug-in component in clinical few-shot segmentation workflows, but the current evaluation is limited to two held-out structures from two institutions.
  • The largest relative gains occur in the lowest-data regime (K=4), suggesting the approach is most valuable when annotation budgets are extremely tight; testing at K=1 or K=2 would probe how far the benefit extends.
  • It remains an open question how the policy behaves when the candidate pool contains a mix of very similar and very unusual images; a controlled experiment varying pool diversity would clarify whether the agent is learning a general complementarity heuristic or overfitting to the training distribution.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a reinforcement-learning (RL) framework for support-set selection in few-shot medical image segmentation. Given a pool of P unlabelled images from a target domain, a policy π(s;φ) predicts a binary selection mask a with exactly K ones; the selected images are labelled, used to adapt a Reptile (or MAML) meta-trained UNet, and the reward is the Dice score on a validation set. Experiments on a cross-institutional male pelvic MRI dataset (NB and OI structures from held-out institutions 3 and 4) compare the method against random selection, TBP, and DVRL for K = 4, 8, 16. The paper reports consistent and statistically significant improvements, with combined Dice at K = 4 of 59.4% versus 51.2% (random), plus ablations on candidate pool size and the meta-learning backbone. The central claim is that jointly optimised, set-level support-set selection outperforms individual-sample scoring and random selection.

Significance. If the claim holds, the contribution is valuable: it addresses a real and under-explored problem, and the set-level formulation is a principled departure from per-sample active-learning and data-valuation methods. The paper also provides a public code repository, which is welcome. However, the current manuscript does not provide enough implementation detail to reproduce the method, and the evaluation protocol as written creates a potentially serious fairness problem: Section 3.1 says that a further 8 labelled examples are used as a validation set for 'reinforcement learning reward computation', but Section 2.3 says no reward computation is performed during inference. If the policy is trained or fine-tuned on target-domain validation labels, the comparison against random/TBP/DVRL is not apples-to-apples and the reported gains could be explained by access to extra target-domain supervision. Because this ambiguity affects every reported result, the significance of the work cannot be assessed from the manuscript as currently written.

major comments (4)
  1. [Sec. 3.1 vs Sec. 2.3] The evaluation protocol is internally inconsistent. Sec. 2.3 states 'No policy optimisation or reward computation is performed during inference', but Sec. 3.1 states that for each evaluation experiment 'a further 8 examples were used as the validation set for reinforcement learning reward computation'. If those 8 target-domain labels are used to train or fine-tune the policy before support-set selection, then the proposed method has access to V = 8 labelled target-domain images that random selection and TBP do not. This would invalidate the comparison in Table 1. The paper must specify clearly whether RL training is performed only on source domains, or per target domain, and exactly how the 8 validation examples are used. If per-target training or fine-tuning is performed, the experiments need to be redone under a protocol that gives comparators the same information.
  2. [Sec. 2.2, Eq. (4)] The policy is described only as π(s;φ), with no architecture, no input encoding, no action distribution, and no mechanism to enforce the constraint ∑ a_j = K. Since the central claim is that a learned policy captures set-level complementarity, the representation of the candidate pool and the sampling procedure for a K-sparse binary mask are load-bearing. The paper should specify the network architecture, how variable-size pools are handled, how the K-of-P constraint is enforced (e.g., Gumbel-top-K, REINFORCE with a combinatorial action space, or a deterministic mask from a scoring network), and the PPO hyper-parameters used. Without these details the method is not reproducible.
  3. [Sec. 3.3] The sentence 'in actuality K remained fixed at K = 4 and where K > 4, selection was repeated multiple times with randomly sampled candidate pools' is ambiguous and potentially inconsistent with Table 1, which reports results for K = 8 and K = 16. If the policy always selects 4 images and the support set for K > 4 consists of multiple such selections, the adaptation procedure in Eq. (2) with K examples needs to be described precisely. If instead the policy is trained at each K, that should be stated. As written, a reader cannot determine what was actually evaluated for K = 8 and K = 16.
  4. [Sec. 4, Table 1] The statistical reporting is too terse. The paper reports 'all p-values < 0.01' for comparisons against three baselines across three K values and two structures, but does not state the number of test volumes per comparison, whether the t-test is one-sided or two-sided, or whether any multiple-comparison correction was applied. Given that 27 comparisons are made (3 baselines × 3 K values × 3 rows including combined), uncorrected p < 0.01 is not by itself convincing. The paper should report effect sizes, confidence intervals, and the correction procedure.
minor comments (5)
  1. [Sec. 3.1] The dataset description says 'up to 16 labelled examples were used for support-set adaptation and a further 8 examples were used as the validation set for reinforcement learning reward computation'. The phrase 'up to' is vague; specify exactly how many validation examples are used and whether the same 8 are used for all experiments.
  2. [Sec. 2.1, Eq. (1)] The meta-training objective writes an expectation over source domains, but the Reptile procedure is not described. Please state explicitly how tasks/domains are sampled and how Reptile's inner-loop adaptation is performed, since this is relevant to the few-shot adaptation mechanism that the RL policy is optimizing.
  3. [Sec. 4] The ablation for P reports p = 0.09 for P = 16 vs P = 32, but the main text says 'statistical significance was not observed'. The same level of detail should be provided for all comparisons, not only for the ablations.
  4. [Fig. 2] The qualitative example is interesting, but claims about 'broader range of image appearances' and 'oversegmentation' are not quantified. A figure with a single query and no error maps is not strong evidence for complementarity; consider adding quantitative artefact-specific Dice or error heatmaps.
  5. [Sec. 3.2] The UNet architecture is described as receiving a 'grayscale image (single channel)', while the dataset is 3D with dimensions 256×256×32. Please clarify whether the 3D volumes are processed slice-wise or as 3D inputs, and whether the 3×3×3 convolutions operate on 2D or 3D grids.

Circularity Check

0 steps flagged

No significant circularity: the RL reward is validation Dice and the reported results are held-out test Dice; self-citations are not load-bearing.

full rationale

The paper's derivation chain is: meta-train a segmentation model (Eq. 1), adapt it on a selected support set (Eq. 2), compute a reward as the adapted model's mean Dice on a validation set (Eq. 3), and optimize the selection policy against that reward (Eq. 4). The reported evaluation uses a held-out test set that is described as separate from the support and validation examples ('Remaining volumes were reserved for evaluation'). The reward is therefore not the same quantity as the headline test Dice, and the test result is not forced by the training objective by construction. The comparison methods and the proposed method all use the same few-shot adaptation procedure, so the proposed method's advantage is an empirical claim. The self-citations (e.g., Refs. [5], [20], [27]) are used as prior baselines or background literature, not as a uniqueness theorem or as the load-bearing justification for the central claim. The main caveat is a protocol ambiguity: Sec. 3.1 says 'a further 8 examples were used as the validation set for reinforcement learning reward computation', which could be read as using target-domain validation labels during evaluation, while Sec. 2.3 says 'No policy optimisation or reward computation is performed during inference.' If the policy were trained or fine-tuned on target-domain validation labels, the comparison to random selection would be unfair, but that would be a correctness/fairness concern rather than a circularity of the derivation, because the reward still is not identical to the test metric and no equation reduces the prediction to the fitted inputs. No specific circular step can be exhibited.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

The central claim rests on four key assumptions: the reliability of the meta-learning reward, the sufficiency of a tiny validation set for RL training, the existence of an unspecified policy network, and the representativeness of the evaluation domains. None is independently validated in the paper. The method introduces no new physical or conceptual entities.

axioms (4)
  • domain assumption Reptile/MAML provide a valid few-shot adaptation mechanism, and validation Dice after a few gradient steps is a stable reward signal.
    The entire method relies on the assumption that the meta-learned model adapts predictably from a support set, so the validation Dice used as reward reflects true generalization. This is standard for meta-learning but is not justified with error bars across seeds. (Sec 2.1)
  • domain assumption The 8-example validation set from the target domain is sufficient to train or reward an RL policy that selects good support sets.
    The paper does not analyze the sample complexity of PPO with only 8 reward samples per episode. This is a heavy assumption, especially if the policy is trained at test time. (Sec 3.1)
  • ad hoc to paper A policy network can encode a set of P images and output a K-sparse binary mask for any P and K.
    No architecture, state encoding, or action sampling procedure is given. The existence of such a network is presupposed by the method description. (Sec 2.2)
  • domain assumption The chosen evaluation structures (NB, OI) and institutions (3, 4) are representative of realistic few-shot segmentation scenarios.
    Results on two structures from two institutions may not generalize to other anatomies or acquisition conditions. The paper does not justify why these are representative. (Sec 3.1)

pith-pipeline@v1.3.0-alltime-deepseek · 6770 in / 12646 out tokens · 140611 ms · 2026-08-01T04:56:45.584414+00:00 · methodology

0 comments
read the original abstract

Few-shot learning enables medical image segmentation models to adapt to new tasks using only a small number of labelled examples. However, adaptation performance depends strongly on which examples are selected for the support set. Effective support sets should capture relevant variation within the target domain and be informative for adaptation, with constituent samples providing complementary information. Despite this, existing active data selection approaches largely prioritise samples individually and do not explicitly account for interactions between examples. In this work, we propose a reinforcement learning framework for support-set selection in few-shot medical image segmentation, enabling support sets to be optimised jointly rather than through independent sample scoring. Given a pool of unlabelled candidate images, an agent directly predicts a support set that maximises downstream segmentation performance. Experiments on a cross-institutional pelvic MRI dataset demonstrate improvements over random selection and current state-of-the-art methods. Our findings highlight the importance of support-set complementarity for effective adaptation and demonstrate the potential of reinforcement learning for optimising adaptation sets.

Figures

Figures reproduced from arXiv: 2607.22371 by Abdallah Aljarkas, Ahmed M. Elsayed, Benny Wong, Chenlan Zhao, Claire C. Villette, Hamad A. Aljamaan, Lynn Karam, Qianye Yang, Shaheer U. Saeed, Timothy F. Lundberg, Yipeng Hu.

Figure 1
Figure 1. Figure 1: Overall framework of the proposed method. An overview of the proposed framework is shown in [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Qualitative results from our proposed method, showing selected support sets. Qualitative analysis: Qualitative results are presented in [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

27 extracted references · 5 linked inside Pith

  1. [1]

    U-net: Convolu- tional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. “U-net: Convolu- tional networks for biomedical image segmentation”. In:International Con- ference on Medical image computing and computer-assisted intervention. Springer. 2015, pp. 234–241

    2015

  2. [2]

    Medical image segmentation: A comprehensive review of deep learning-based methods

    Yuxiao Gao et al. “Medical image segmentation: A comprehensive review of deep learning-based methods”. In:Tomography11.5 (2025), p. 52

    2025

  3. [3]

    A review of artificial intelligence in prostate cancerdetectiononimaging

    Indrani Bhattacharya et al. “A review of artificial intelligence in prostate cancerdetectiononimaging”.In:Therapeutic advances in urology14(2022), p. 17562872221128791

    2022

  4. [4]

    Medical image segmentation with limited su- pervision: a review of deep network models

    Jialin Peng and Ye Wang. “Medical image segmentation with limited su- pervision: a review of deep network models”. In:Ieee Access9 (2021), pp. 36827–36851

    2021

  5. [5]

    Active learning using adaptable task-based priori- tisation

    Shaheer U Saeed et al. “Active learning using adaptable task-based priori- tisation”. In:Medical Image Analysis95 (2024), p. 103181

    2024

  6. [6]

    Issegmentationuncertaintyuseful?

    SteffenCzolbeetal.“Issegmentationuncertaintyuseful?” In:International conference on information processing in medical imaging. Springer. 2021, pp. 715–726

    2021

  7. [7]

    Development and evaluation of intraoperative ultrasound segmentation with negative image frames and multiple observer labels

    Liam F Chalcroft et al. “Development and evaluation of intraoperative ultrasound segmentation with negative image frames and multiple observer labels”. In:International Workshop on Advances in Simplifying Medical Ultrasound. Springer. 2021, pp. 25–34

    2021

  8. [8]

    Embracing imperfect datasets: A review of deep learningsolutionsformedicalimagesegmentation

    Nima Tajbakhsh et al. “Embracing imperfect datasets: A review of deep learningsolutionsformedicalimagesegmentation”.In:Medical image anal- ysis63 (2020), p. 101693

    2020

  9. [9]

    A systematic review of few-shot learn- ing in medical imaging

    Eva Pachetti and Sara Colantonio. “A systematic review of few-shot learn- ing in medical imaging”. In:Artificial intelligence in medicine156 (2024), p. 102949

    2024

  10. [10]

    Reasoning in machine vision by learning fast and slow thinking

    Shaheer U Saeed et al. “Reasoning in machine vision by learning fast and slow thinking”. In:Nature Communications(2026)

    2026

  11. [11]

    Model-agnostic meta- learning for fast adaptation of deep networks

    Chelsea Finn, Pieter Abbeel, and Sergey Levine. “Model-agnostic meta- learning for fast adaptation of deep networks”. In:International conference on machine learning. PMLR. 2017, pp. 1126–1135

    2017

  12. [12]

    On first-order meta- learning algorithms

    Alex Nichol, Joshua Achiam, and John Schulman. “On first-order meta- learning algorithms”. In:arXiv preprint arXiv:1803.02999(2018)

    Pith/arXiv arXiv 2018

  13. [13]

    Meta-learning with implicit gradients in a few- shot setting for medical image segmentation

    Rabindra Khadka et al. “Meta-learning with implicit gradients in a few- shot setting for medical image segmentation”. In:Computers in Biology and Medicine143 (2022), p. 105227

    2022

  14. [14]

    Few-shot learning for medical image segmentation using 3d u-net and model-agnostic meta-learning (maml)

    Aqilah M Alsaleh et al. “Few-shot learning for medical image segmentation using 3d u-net and model-agnostic meta-learning (maml)”. In:Diagnostics 14.12 (2024), p. 1213

    2024

  15. [15]

    Prototypical few-shot segmentation for cross-institution male pelvic structures with spatial registration

    Yiwen Li et al. “Prototypical few-shot segmentation for cross-institution male pelvic structures with spatial registration”. In:Medical Image Anal- ysis90 (2023), p. 102935. Active few-shot segmentation by reinforcing data selection 11

    2023

  16. [16]

    Few-shot medical image segmentation with high-fidelity prototypes

    Song Tang et al. “Few-shot medical image segmentation with high-fidelity prototypes”. In:Medical Image Analysis100 (2025), p. 103412

    2025

  17. [17]

    Variability and reproducibility in deep learning for medical image segmentation

    Félix Renard et al. “Variability and reproducibility in deep learning for medical image segmentation”. In:Scientific Reports10.1 (2020), p. 13724

    2020

  18. [18]

    A comprehensive survey on deep active learning in medical image analysis

    Haoran Wang et al. “A comprehensive survey on deep active learning in medical image analysis”. In:Medical Image Analysis95 (2024), p. 103201

    2024

  19. [19]

    A survey on deep active learning: Recent advances and new frontiers

    Dongyuan Li et al. “A survey on deep active learning: Recent advances and new frontiers”. In:IEEE Transactions on Neural Networks and Learning Systems36.4 (2024), pp. 5879–5899

    2024

  20. [20]

    Image quality assessment for machine learning tasks using meta-reinforcement learning

    Shaheer U Saeed et al. “Image quality assessment for machine learning tasks using meta-reinforcement learning”. In:Medical Image Analysis78 (2022), p. 102427

    2022

  21. [21]

    Learning to learn for few-shot continual active learning

    Stella Ho et al. “Learning to learn for few-shot continual active learning”. In:arXiv preprint arXiv:2311.03732(2023)

    Pith/arXiv arXiv 2023

  22. [22]

    Data valuation using re- inforcement learning

    Jinsung Yoon, Sercan Arik, and Tomas Pfister. “Data valuation using re- inforcement learning”. In:International Conference on Machine Learning. PMLR. 2020, pp. 10842–10851

    2020

  23. [23]

    Labeled TrustSet Guided: Batch Active Learning with Reinforcement Learning

    Guofeng Cui et al. “Labeled TrustSet Guided: Batch Active Learning with Reinforcement Learning”. In:arXiv preprint arXiv:2604.12303(2026)

    Pith/arXiv arXiv 2026

  24. [24]

    Batchbald: Effi- cient and diverse batch acquisition for deep bayesian active learning

    Andreas Kirsch, Joost Van Amersfoort, and Yarin Gal. “Batchbald: Effi- cient and diverse batch acquisition for deep bayesian active learning”. In: Advances in neural information processing systems32 (2019)

    2019

  25. [25]

    Solving continual combinatorial selection via deep reinforcement learning

    Hyungseok Song et al. “Solving continual combinatorial selection via deep reinforcement learning”. In:arXiv preprint arXiv:1909.03638(2019)

    Pith/arXiv arXiv 1909

  26. [26]

    Proximal policy optimization algorithms

    John Schulman et al. “Proximal policy optimization algorithms”. In:arXiv preprint arXiv:1707.06347(2017)

    Pith/arXiv arXiv 2017

  27. [27]

    Adaptable image quality assessment using meta- reinforcement learning of task amenability

    Shaheer U Saeed et al. “Adaptable image quality assessment using meta- reinforcement learning of task amenability”. In:International Workshop on Advances in Simplifying Medical Ultrasound. Springer. 2021, pp. 191–201

    2021