REVIEW 4 major objections 5 minor 1 cited by
Adaptive Pseudo Label Selection for Individual Unlabeled Data by Positive and Unlabeled Learning
T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read Adding per-image PU-selected negative pseudo-labels improves semi-supervised medical image segmentation.
desk verdict The per-image PU selection idea is a reasonable incremental twist, but the evidence is thin: the public gains over plain pseudo-labeling are within noise, the class-prior estimate in Eq. (2) looks wrong, and the promising photoacoustic results need error bars. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The per-image PU scorer. For each unlabeled image $I_i$, a fully connected classifier $f(\cdot; \theta^{(i)}_{pu})$ is trained on the feature vectors of foreground pixels from the labeled images ($X_p$) against the image's ambiguous pixels ($X_u^{(i)}$), using the fixed pre-trained feature extractor $g'$ and the non-negative PU risk $\mathcal{L}_{pu} = \pi_p \hat{R}^+_p(f) + \max\{0, \hat{R}^-_u(f) - \pi_p \hat{R}^-_p(f)\}$ with empirical risk terms over positives and unlabeled pixels. Pixels whose PU score (the likelihood of being foreground) falls in the bottom $\alpha\%$ of that image become new negative pseudo-labels. The mechanism's job is to replace one global confidence threshold with
What would settle it
Hold out images whose background noise comes from an artifact class absent from the labeled set, then compute the ranking quality of the per-image PU score against the real masks (e.g., AUC of the PU score versus foreground/background within each image); if the AUC falls toward chance or the added negative labels lower Dice relative to the no-PU baseline as novel noise is introduced, the per-image ranking assumption fails.
Extended reading notes
Core claim
The paper's central claim is that the failure of confidence-based pseudo-labeling on medical images is a metric problem, not merely a threshold problem: a single confidence metric mis-ranks pixels in images whose background-noise distribution differs from the labeled set. Its remedy is a per-image PU scorer. For each unlabeled image, foreground feature vectors extracted from the supervised images serve as positives, the image's ambiguous pixels (those not already pseudo-labeled by confidence thresholds) serve as unlabeled, and a fully connected classifier trained with the non-negative PU loss ranks the pixels; the bottom α% (α = 20%) are added as negative (background) pseudo-labels, and the
Load-bearing premise
The method assumes that the foreground feature vectors taken from the few labeled images represent the foreground in every unlabeled image, so that a per-image PU score built on those positives genuinely ranks each image's background below its foreground.
Editorial extensions
If this is right
- Per-image adaptive scoring beats a single global confidence threshold: average Dice rises from 0.786 (threshold pseudo-labeling) to 0.799, and from 0.776 for the supervised baseline, across two retinal datasets and two backbones.
- The added negative labels can rescue cases where confidence pseudo-labeling hurts: on the FR-UNet backbone, threshold pseudo-labeling lowered average Dice while the full method improved it.
- Batch-mode PU — one scorer trained on all unlabeled images together — performs worse (0.759 average Dice) than per-image PU (0.799), indicating the per-image fit is the effective ingredient.
- The gains appear with very few labeled images (two per fold) and transfer from 2D retinal vessel segmentation to 3D photoacoustic vessel tracing, where average coverage improves from 0.712 to 0.763 (U-Net) and 0.876 to 0.917 (FR-UNet).
- Training on the added background pseudo-labels makes weak vessels visible that the baseline misses, as shown in the qualitative vessel-segmentation results.
Reading between the lines
- If per-image adaptation is the true mechanism, the claim predicts larger gains when background noise is more heterogeneous across images (multi-site or multi-scanner collections) and smaller gains on homogeneous image sets — a testable condition the paper's experiments do not vary.
- The same per-image PU ranking could be mirrored for the foreground, adding the top-scoring ambiguous pixels as positive pseudo-labels; the paper's own future-work note plans to refine foreground selection, so this is a direct extension rather than a paper claim.
- A natural extension would replace the pixel-count positive-ratio estimate $\pi_p = n_p/(n_p + n_u)$ with a density-based prior estimate, which could stabilise the per-image ranking when an image's ambiguous region is dominated by background.
- Because the PU scorer sits on a feature extractor fixed after pre-training, iterating the select-and-retrain loop could let the improved features feed back into sharper per-image rankings.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a semi-supervised medical image segmentation method that selects negative pseudo-labels adaptively for each unlabeled image via Positive and Unlabeled (PU) learning. After pre-training on labeled data, confident foreground/background pseudo-labels are obtained by thresholding; the remaining ambiguous pixels are treated as unlabeled data for a per-image nnPU classifier trained on positive feature vectors from labeled foregrounds. The bottom α% of PU scores are added as negative pseudo-labels, and the network is retrained on labeled plus pseudo-labeled pixels. Experiments on DRIVE, CHASE DB, and 3D photoacoustic images with U-Net and FR-UNet report average Dice or vessel-tracing coverage improvements over supervised baselines and conventional pseudo-labeling.
Significance. If the mechanism is valid, the idea of per-image adaptive pseudo-label selection via PU learning is original and relevant to label-efficient medical image segmentation, where background noise varies across images. The paper addresses a practically important setting and evaluates on two public datasets and one clinical dataset with two backbones. However, the current manuscript has a load-bearing methodological flaw in the PU class-prior estimator and the experimental evidence lacks statistical support. The central claim is therefore not yet established, though the underlying idea is promising.
major comments (4)
- [§2.3, Eq. (2)] The class-prior estimate π_p = np/(np+nu) is not a consistent estimator of the positive proportion in the unlabeled set. In the nnPU risk (Kiryo et al.), π_p denotes P(y=1) in the unlabeled sample. Here np is the number of positive feature vectors extracted from labeled foreground regions and nu is the number of unlabeled (ambiguous) pixels in the current image. This ratio conflates the size of the labeled positive set with the foreground frequency in the unlabeled mixture; it depends on the number of labeled images, the thresholds thp/thn that define the unlabeled set, and the image size. With a misspecified π_p, the non-negative risk estimator is biased and the PU-score is not a calibrated positive likelihood. The paper provides no diagnostic showing that the bottom α% of PU scores are actually background. Since this step is the entire basis for the adaptive per-image metric, the centr
- [§3, Tables 1 and 2] The experiments report no error bars, no multiple-seed variation, and no significance tests. Several claimed improvements are within run-to-run noise: DRIVE U-Net Ours 0.796 vs Pseudo 0.795; CHASE U-Net Ours 0.810 vs Pseudo 0.809. The statement that 'our method was the best in any dataset and backbone' is not statistically justified. Please provide standard deviations across at least five seeds and paired significance tests (e.g., paired bootstrap or Wilcoxon signed-rank) for the main comparisons.
- [§3.1, §3.2] Sensitivity to hyperparameters is not analyzed. The method depends on thp, thn, and α, and the paper sets these differently for retinal images (0.8/0.1/20%) and photoacoustic images (100/256, 2/256, 20%) without a principled justification. It is unknown whether the reported improvements are tied to these specific choices or whether they generalize. Report results over a range of α and thresholds, or justify the selections using a validation set.
- [§3.1, Table 1] The batch variant 'Ours (batch)' collapses on both public datasets with U-Net (DRIVE 0.720, CHASE 0.737) compared with Baseline (0.787, 0.730). The paper attributes this to background noise variation but provides no diagnostic. More importantly, no analysis is given of the quality or quantity of the added negative pseudo-labels Npu (e.g., precision of selected negatives, number of pixels added, effect on the training gradient). Without such an analysis, the improvement of Ours over Pseudo could be due to extra regularization or label balance rather than to the PU-based selection mechanism.
minor comments (5)
- [§3.1 / Table 1] The method name is inconsistent: the text lists 'MNS [13]' but Table 1 says 'MMS'. Please unify.
- [§2.1, Eq. (1)] The expression for the supervised loss is not the usual cross-entropy; as written it lacks a minus sign and the label-dependent form. Please correct to L = -Σ A(j) log g(I(j);θ) (or the equivalent).
- [§2.3, Eq. (2)] Notation is confusing: X(i)_p is used in the definition of np but X_p was defined earlier without a superscript; it is also unclear whether np and nu are per-image or global totals. Clarify the indexing.
- [Related work / references] When estimating the PU class prior, the paper should cite standard mixture-proportion estimation literature (e.g., du Plessis et al., Ramaswamy et al.) and explain why the simple ratio np/(np+nu) is chosen.
- [§3.1] The paper states that 'an analysis of the effectiveness of the number of unlabeled images is needed in future work.' This is an acknowledged limitation; it should be reflected in the strength of the claims in the abstract and conclusion.
Circularity Check
No significant circularity: the PU-selected pseudo-labels are produced by a per-image classifier trained on labeled positives and unlabeled pixels, and the reported Dice improvements are evaluated on held-out test sets.
full rationale
The derivation chain is self-contained with respect to the paper's claims. The method pre-trains a network on supervised data (Eq. 1), obtains confidence-based pseudo-labels, then trains a per-image PU classifier on labeled foreground features and that image's remaining unlabeled pixels using the nnPU loss (Eq. 2), and finally selects the lowest-α% PU scores as additional negative pseudo-labels for re-training (Eq. 3). The evaluation metric (Dice or vessel-tracing coverage) is computed on held-out test images and is never used to fit a parameter or to define the pseudo-label selection rule, so the reported improvements are not forced by construction. The class-prior estimate π_p = np/(np+nu) may be statistically questionable, but that is a correctness/robustness concern about the PU score, not a circular reduction: it does not make the test-set result equivalent to an input. The same-group citations ([2], [3], [5], [6], [7]) are contextual related work or application precedents; the load-bearing PU formulation is cited to external works [4], [8]. No uniqueness theorem, ansatz, or fitted-input-called-prediction pattern is present. The paper's own noted limitations (e.g., lack of analysis of unlabeled-image count and foreground pseudo-label refinement) are acknowledged open issues, not circular steps. Therefore no circularity is found.
Assumptions & free parameters
free parameters (3)
- thp =
0.8 (DRIVE/CHASE), 100/256 (PA)
- thn =
0.1 (DRIVE/CHASE), 2/256 (PA)
- alpha =
20%
assumptions (3)
- ad hoc to paper The class prior of positives in each unlabeled image is correctly estimated by pi_p = np/(np+nu), where np is the number of positive feature vectors from the labeled set and nu is the number of unlabeled pixels in the current image.
- domain assumption Positive features from labeled images are representative of the foreground distribution in every unlabeled image, so a single positive pool can seed each per-image PU classifier.
- domain assumption The low PU-score quantile (lowest alpha%) corresponds to true background pixels.
Cite this review
Pith. "Pith review of Adaptive Pseudo Label Selection for Individual Unlabeled Data by Positive and Unlabeled Learning." pith.science (2026). https://pith.science/paper/KRI5VMRZ
@misc{pith2026250807548,
author = {Pith},
title = {Pith review of: Adaptive Pseudo Label Selection for Individual Unlabeled Data by Positive and Unlabeled Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/KRI5VMRZ}},
note = {Machine review of arXiv:2508.07548}
}
read the original abstract
This paper proposes a novel pseudo-labeling method for medical image segmentation that can perform learning on ``individual images'' to select effective pseudo-labels. We introduce Positive and Unlabeled Learning (PU learning), which uses only positive and unlabeled data for binary classification problems, to obtain the appropriate metric for discriminating foreground and background regions on each unlabeled image. Our PU learning makes us easy to select pseudo-labels for various background regions. The experimental results show the effectiveness of our method.
Figures
Forward citations
Cited by 1 Pith paper
-
Quantum-state block texture and its quantification
Quantum-state texture is generalized to a block version with several proposed measures; the closed-form relative-entropy measure is false as stated.
Reference graph
Works this paper leans on
-
[1]
Physics in Medicine & Biology 58(13), R97 (2013)
Bauer, S., et al.: A survey of mri-based medical image analysis for brain tumor studies. Physics in Medicine & Biology 58(13), R97 (2013)
work page 2013
-
[2]
In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVMI)
Bise, R., et al.: 3d structure modeling of dense capillaries by multi-objects tracking. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVMI). pp. 29–37 (2016)
work page 2016
-
[3]
Bise, R.e.a.: Vascular registration in photoacoustic imaging by low-rank alignment via fore- ground, background and complement decomposition. In: MICCAI. pp. 326–334 (2016)
work page 2016
-
[4]
Elkan, C., et al.: Learning classifiers from only positive and unlabeled data. In: Proceed- ings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. pp. 213–220 (2008)
work page 2008
-
[5]
In: International Conference on Medical Image Computing and Computer- Assisted Intervention
Fujii, K., et al.: Cell detection from imperfect annotation by pseudo label selection using p-classification. In: International Conference on Medical Image Computing and Computer- Assisted Intervention. pp. 425–434. Springer (2021)
work page 2021
-
[6]
Kikkawa, R., Sekiguchi, H., Tsuge, I., Saito, S., Bise, R.: Semi-supervised learning with structured knowledge for body hair detection in photoacoustic image. In: ISBI (2019)
work page 2019
-
[7]
Kikkawa, R., et al.: Unsupervised body hair detection by positive-unlabeled learning in pho- toacoustic image. In: 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). pp. 3349–3352 (2021)
work page 2021
-
[8]
In: Guyon, I., Luxburg, U.V ., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R
Kiryo, R., et al.: Positive-unlabeled learning with non-negative risk estimator. In: Guyon, I., Luxburg, U.V ., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 30. Curran Associates, Inc. (2017)
work page 2017
Show all 23 references
-
[9]
In: Workshop on challenges in representation learning, International Conference on Machine Learning
Lee, D.H., et al.: Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on challenges in representation learning, International Conference on Machine Learning. vol. 3, p. 896 (2013)
2013
-
[10]
Physics in Medicine & Biology 54(19), R59 (2009)
Li, C., et al.: Photoacoustic tomography and sensing in biomedicine. Physics in Medicine & Biology 54(19), R59 (2009)
2009
-
[11]
IEEE Journal of Biomedical and Health Informatics 26(9), 4623–4634 (2022)
Liu, W., et al.: Full-resolution network and dual-threshold iteration for retinal vessel and coronary angiograph segmentation. IEEE Journal of Biomedical and Health Informatics 26(9), 4623–4634 (2022)
2022
-
[12]
Nature medicine 26(6), 900–908 (2020)
Liu, Y ., et al.: A deep learning system for differential diagnosis of skin diseases. Nature medicine 26(6), 900–908 (2020)
2020
-
[13]
IEEE Transactions on Medical Imaging (2023)
Lou, A., et al.: Min-max similarity: A contrastive semi-supervised deep learning network for surgical tools segmentation. IEEE Transactions on Medical Imaging (2023)
2023
-
[14]
In: International Conference on Medical Image Computing and Computer- Assisted Intervention
Nishimura, K., et al.: Weakly supervised cell instance segmentation by propagating from de- tection response. In: International Conference on Medical Image Computing and Computer- Assisted Intervention. pp. 649–657. Springer (2019)
2019
-
[15]
arXiv preprint arXiv:2006.05278 (2020)
Ouali, Y ., et al.: An overview of deep semi-supervised learning. arXiv preprint arXiv:2006.05278 (2020)
2006 arXiv
-
[16]
Investigative Ophthalmology & Visual Science 50(5), 2004–2010 (2009)
Owen, C.G., et al.: Measuring Retinal Vessel Tortuosity in 10-Year-Old Children: Validation of the Computer-Assisted Image Analysis of the Retina (CAIAR) Program. Investigative Ophthalmology & Visual Science 50(5), 2004–2010 (2009)
2004
-
[17]
Yamane et al
PapersWithCode: Retinal vessel segmentation on drive leaderboard (2023), https://paperswithcode.com/sota/retinal-vessel-segmentation-on-drive 10 T. Yamane et al
2023
-
[18]
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelli- gence and Lecture Notes in Bioinformatics) 9351, 234–241 (2015)
Ronneberger, O., et al.: U-Net: Convolutional networks for biomedical image segmentation. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelli- gence and Lecture Notes in Bioinformatics) 9351, 234–241 (2015)
2015
-
[19]
Medical image analysis13(5), 701–714 (2009)
Schaap, M., et al.: Standardized evaluation methodology and reference database for evaluat- ing coronary artery centerline extraction algorithms. Medical image analysis13(5), 701–714 (2009)
2009
-
[20]
IEEE Trans- actions on Medical Imaging 23(4), 501–509 (2004)
Staal, J., et al.: Ridge-Based Vessel Segmentation in Color Images of the Retina. IEEE Trans- actions on Medical Imaging 23(4), 501–509 (2004)
2004
-
[21]
Machine learning 109(2), 373–440 (2020)
Van Engelen, J.E., Hoos, H.H.: A survey on semi-supervised learning. Machine learning 109(2), 373–440 (2020)
2020
-
[22]
Annals of the BMV A2013(7), 1–22 (2013)
Zhao, F., Xie, X.: An overview of interactive medical image segmentation. Annals of the BMV A2013(7), 1–22 (2013)
2013
-
[23]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Zhao, Z., et al.: Positive-unlabeled learning for cell detection in histopathology images with incomplete annotations. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 509–518. Springer (2021)
2021
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.