REVIEW 4 major objections 5 minor 26 references
Pathology-Guided Virtual Staining Metric for Evaluation and Training
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read PaPIS, a full-reference metric built on nuclei-segmentation features and Retinex decomposition, claims to capture pathology-relevant similarity better than SSIM, LPIPS, and DISTS, and to improve virtual staining when used as a training…
desk verdict Sensible metric construction, but the pathology-alignment claim is unvalidated and the training evaluation is partly circular; worth reviewing with major revisions. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the feature extractor: a pre-trained nuclei-segmentation encoder whose deep feature maps concentrate on nuclear regions rather than natural-image textures, in contrast to ImageNet-trained backbones used by LPIPS and DISTS. On top of that, Multi-Scale Retinex decomposes each feature map into reflectance and illumination components, and PaPIS is defined as a weighted sum of a high-frequency reflectance similarity term (comparing means and standard deviations of reflectance features) and a low-frequency illumination MSE across layers and channels, with a hyperparameter λ balancing the two. The same distance is converted into a training loss, Lpapis = 1 − PaPIS, and added to the CycleGAN objective.
What would settle it
A reader study in which board-certified pathologists rank the same virtual-staining output pairs would settle the claim: if PaPIS rankings disagree with expert rankings on pairs where SSIM, LPIPS, or DISTS agree with pathologists, the central claim fails. A cheaper check would be to perturb images so that nuclear boundaries are slightly shifted and measure whether PaPIS changes in the expected direction, while the natural-image metrics stay flat.
Extended reading notes
Core claim
On its own terms, the paper's central claim is that a perceptual distance computed on cell-morphology features, with Retinex illumination/reflectance separation, captures pathological fidelity that SSIM, LPIPS, and DISTS miss, and that using this distance as a training loss improves virtual H&E generation. The evidence is categorical and qualitative: scatter-plot comparisons label image pairs as PaPIS-dominant or traditionally-dominant, and heatmaps show PaPIS scores dropping in regions with staining artifacts or degraded micro-anatomy even when SSIM stays high. The authors state that PaPIS more accurately aligns with pathology-relevant visual cues and distinguishes subtle cellular structures that conventional metrics tend to overlook.
Load-bearing premise
The load-bearing premise is that distances computed in the feature space of the nuclei-segmentation network reflect what pathologists care about, and the paper never tests this against expert ratings.
Editorial extensions
If this is right
- PaPIS can flag virtual staining artifacts and morphologically degraded regions that SSIM scores as high-quality, as demonstrated in the patch-wise heatmap analysis.
- Adding PaPIS as a perceptual loss to a CycleGAN-based virtual staining model improves preservation of cell position, shape, and size relative to the standard CycleGAN.
- Because PaPIS operates on cell-level morphology rather than pixel statistics, it is claimed to be modality-agnostic and applicable to autofluorescence, quantitative phase, and hyperspectral virtual staining.
- The modular design allows the segmentation encoder to be replaced or fine-tuned for different stain types and tissue domains, extending PaPIS beyond H&E.
- PaPIS provides a quantitative, automated alternative to subjective expert review for routine quality control in virtual staining pipelines.
Reading between the lines
- A direct, cheaper validation than a full reader study would be to correlate PaPIS scores with downstream nuclei-segmentation accuracy on the same image pairs; the paper does not report this correlation.
- If the feature-space assumption holds, PaPIS could be used as a screening filter that flags low-scoring patches for targeted pathologist review, reducing the workload of manual quality control.
- The same Retinex-plus-morphology-feature recipe could generalize to other stain targets by retraining the encoder on IHC or Masson's trichrome segmentation, a testable extension the authors leave implicit.
- PaPIS's sensitivity to nuclear organization suggests it may also predict performance of cell-counting or tumor-grading algorithms, though the paper does not test such downstream clinical tasks.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces PaPIS (Pathology-Aware Perceptual Image Similarity), a full-reference image quality assessment (FR-IQA) metric for virtual staining in histopathology. The metric uses a pre-trained cell-morphology segmentation encoder to extract multi-layer features, applies multi-scale Retinex decomposition to separate illumination and reflectance components, and computes a perceptual distance as a weighted combination of a low-frequency MSE term and a high-frequency SSIM-like term. The authors also integrate PaPIS as an auxiliary loss in a CycleGAN-based virtual staining model (Eq. 8). Experiments compare PaPIS with SSIM, LPIPS, and DISTS on PARS-to-H&E image pairs via scatter plots and selected examples, and present qualitative evaluations of the PaPIS-guided model, including patch-wise similarity heatmaps.
Significance. The paper addresses a genuine need: standard perceptual metrics such as SSIM, LPIPS, and DISTS are designed for natural images and may not reflect what pathologists care about in virtual staining output. The proposed construction is coherent, and the use of a nuclei-segmentation encoder is a plausible domain-specific choice. The metric is explicitly defined in Section 3.3, which allows independent implementation. The figures provide qualitative evidence that PaPIS can differ from existing metrics in ways interpretable in terms of cellular structure. However, the central empirical claim—that PaPIS aligns more closely with pathology-relevant visual cues—is not supported by any external ground truth. Section 5 explicitly acknowledges the absence of subjective validation by expert pathologists. The paper also uses PaPIS as a training loss and then evaluates the resulting model with PaPIS-related reasoning, which is circular. If validated with expert ratings or a downstream task, PaPIS could be a valuable contribution; as it stands, the evidence is insufficient to support the abstract's claims.
major comments (4)
- [Section 3.1 and Section 5] The load-bearing premise of the paper is that distances computed in the feature space of a nuclei-segmentation encoder (Ignatov et al. [24]) reflect pathologically meaningful differences in virtual staining quality. This premise is asserted in Section 3.1, where the authors state that pathologists focus on attributes that cannot be captured by models trained on natural images, but it is never tested against expert ratings or an independent diagnostic task. Section 5 explicitly concedes 'the absence of subjective validation by expert pathologists.' Consequently, the abstract's claim that PaPIS 'more accurately aligns with pathology-relevant visual cues' is not supported by the evidence; the demonstrated differences from SSIM, LPIPS, and DISTS could simply reflect a reweighting of texture and structure statistics. A correlation study with pathologist ratings or a downstream histopathology task is necessary to validate this claim.
- [Section 4.3 and Eq. (8)] The evaluation of the PaPIS-guided CycleGAN is circular. The model is trained with L_papis = 1 − PaPIS (Eq. 8), and then the reported improvements in 'histological fidelity' (Section 4.3) and the regional sensitivity analysis (Section 4.3.1) rely on PaPIS-based reasoning or PaPIS-derived heatmaps. Since the training objective directly minimizes PaPIS distance, any improvement in PaPIS-related behavior is partly guaranteed by optimization. Independent evaluation—for example, segmentation performance, expert assessment, or conventional IQA metrics with statistical testing—is required to support the claim that PaPIS guidance improves histological fidelity.
- [Section 4.2] The comparative experiments use hand-selected scatter points (AH, AL, PD, TD) and qualitative image examples. These illustrate that PaPIS can disagree with existing metrics, but they do not establish which metric is more accurate with respect to pathology. There are no correlation statistics, no error bars, and no statistical tests. The four categories are defined post hoc from the same data, which creates a risk of confirmation bias. To support the claim that PaPIS 'captures relevant pathological features overlooked by conventional metrics,' the authors should report quantitative agreement with a ground truth, such as expert rankings or an objective histopathology task.
- [Section 3.3, Eq. (6)] The weights α_ij and β_ij are described as 'randomized' with sum 1, making PaPIS stochastic. The paper does not report the random seed, the number of random draws, or the variance of PaPIS scores across draws. This undermines reproducibility and leaves open the possibility that the scatter-plot patterns in Section 4.2 depend on a particular random realization. The authors should either fix the weights with a sensitivity analysis or provide variance estimates for the reported scores.
minor comments (5)
- [Section 3.3, Eq. (6)] The notation µ(i) with a tilde and the channel indices j are not fully defined; the authors should specify that j indexes channels within layer i and that the tilde denotes channel-wise min-max normalized features.
- [Section 4.1.1] The WSI dimension '18, 202 × 48, 800' is formatted inconsistently; the spaces around the multiplication sign should be standardized across all dimensions (e.g., 18,202×48,800).
- [Sections 4.1.2 and 4.3] The loss weight λ2 is reported as 1.0 in Section 4.1.2 but as 2.0 in Section 4.3 for the PaPIS-guided model; please clarify whether these refer to different configurations or whether one is a typographical error.
- [Section 3.4.1] The heading 'P ARS images Acquisition' contains a spacing typo; it should read 'PARS image acquisition.'
- [Figures 5, 7, and 8] The scatter plots would benefit from axis labels, units, a legend for the AH/AL/PD/TD categories, and a statement of the number of image pairs shown.
Circularity Check
No significant circularity: PaPIS is a constructed metric over a pretrained cell-morphology encoder; the acknowledged lack of expert validation is an empirical gap, not a definitional reduction.
full rationale
I walked the derivation chain from Eq. (1) through Eq. (7). PaPIS is explicitly defined as a Retinex-decomposed feature distance computed from a pretrained nuclei-segmentation encoder (Ignatov et al. [24]) plus an MSE term on illumination maps; no equation defines pathology-relevant quality in terms of PaPIS, nor defines the encoder features in terms of PaPIS. The premise that this feature space reflects what pathologists care about is asserted in Section 3.1 and is not tested against expert ratings; the paper concedes in Section 5 'the absence of subjective validation by expert pathologists.' That is an unsupported empirical assumption, which per the analysis rules belongs to correctness risk rather than circularity. The training experiment in Section 4.3 minimizes L_papis = 1 - PaPIS (Eq. 8), but the reported support is qualitative image comparison and visual observation, not a quantitative claim that PaPIS scores themselves improved; thus the improved-fidelity conclusion is not forced by the objective alone. Self-citations [11, 26] support data acquisition and registration and are not load-bearing for the metric's validity. The randomized alpha/beta weights and unspecified lambda are reproducibility concerns, not circular steps.
Assumptions & free parameters
free parameters (5)
- lambda (PaPIS balance weight) =
Not specified
- alpha_ij and beta_ij (PaPIS high-frequency weights) =
Randomized, sum to 1
- sigma_i (Retinex Gaussian scales) =
Not specified
- c1 and c2 (stability constants) =
Not specified
- Training loss weights lambda1 to lambda4 =
lambda1=2.0, lambda2=1.0 (2.0 in guided model), lambda3=1.0, lambda4=1.0
assumptions (4)
- domain assumption Retinex product model I = R * L applies to deep feature maps of a segmentation encoder.
- domain assumption Distances in EfficientNet-B7 nuclei-segmentation feature space correspond to pathologically relevant quality differences.
- ad hoc to paper Randomly chosen alpha and beta weights with sum 1 yield a meaningful perceptual distance.
- domain assumption The cell-morphology encoder transfers to PARS-to-H&E virtual staining images.
Cite this review
Pith. "Pith review of Pathology-Guided Virtual Staining Metric for Evaluation and Training." pith.science (2026). https://pith.science/paper/VSOFKDNU
@misc{pith2026250712624,
author = {Pith},
title = {Pith review of: Pathology-Guided Virtual Staining Metric for Evaluation and Training},
year = {2026},
howpublished = {\url{https://pith.science/paper/VSOFKDNU}},
note = {Machine review of arXiv:2507.12624}
}
read the original abstract
Virtual staining has emerged as a powerful alternative to traditional histopathological staining techniques, enabling rapid, reagent-free image transformations. However, existing evaluation methods predominantly rely on full-reference image quality assessment (FR-IQA) metrics such as structural similarity, which are originally designed for natural images and often fail to capture pathology-relevant features. Expert pathology reviews have also been used, but they are inherently subjective and time-consuming. In this study, we introduce PaPIS (Pathology-Aware Perceptual Image Similarity), a novel FR-IQA metric specifically tailored for virtual staining evaluation. PaPIS leverages deep learning-based features trained on cell morphology segmentation and incorporates Retinex-inspired feature decomposition to better reflect histological perceptual quality. Comparative experiments demonstrate that PaPIS more accurately aligns with pathology-relevant visual cues and distinguishes subtle cellular structures that traditional and existing perceptual metrics tend to overlook. Furthermore, integrating PaPIS as a guiding loss function in a virtual staining model leads to improved histological fidelity. This work highlights the critical need for pathology-aware evaluation frameworks to advance the development and clinical readiness of virtual staining technologies.
Reference graph
Works this paper leans on
-
[24]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp
Ignatov, A., Yates, J., Boeva, V.: Histopathological image classification with cell morphology aware deep neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6913–6925 (2024)
work page 2024
-
[1]
Light: Science & Applications 12(1), 57 (2023)
Bai, B., Yang, X., Li, Y., Zhang, Y., Pillar, N., Ozcan, A.: Deep learning-enabled virtual histological staining of biological samples. Light: Science & Applications 12(1), 57 (2023)
work page 2023
-
[2]
Trends in Biotechnology (2024)
Latonen, L., Koivukoski, S., Khan, U., Ruusuvuori, P.: Virtual staining for histology by deep learning. Trends in Biotechnology (2024)
work page 2024
-
[3]
A study of why we need to reassess full reference image quality assessment with medical images
Breger, A., Biguri, A., Landman, M.S., Selby, I., Amberg, N., Brunner, E., Gr¨ ohl, J., Hatamikia, S., Karner, C., Ning, L., et al.: A study of why we need to reassess full reference image quality assessment with medical images. arXiv preprint arXiv:2405.19097 (2024)
work page Pith review arXiv 2024
-
[4]
In: 2015 IEEE International Conference on Image Processing (ICIP), pp
Pambrun, J.-F., Noumeir, R.: Limitations of the ssim quality metric in the con- text of diagnostic imaging. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 2960–2963 (2015). IEEE 17
work page 2015
-
[5]
In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp
Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 586–595 (2018)
2018
-
[6]
IEEE transactions on pattern analysis and machine intelligence 44(5), 2567–2581 (2020)
Ding, K., Ma, K., Wang, S., Simoncelli, E.P.: Image quality assessment: Unify- ing structure and texture similarity. IEEE transactions on pattern analysis and machine intelligence 44(5), 2567–2581 (2020)
work page 2020
-
[7]
A study on the adequacy of common IQA measures for medical images
Breger, A., Karner, C., Selby, I., Gr¨ ohl, J., Dittmer, S., Lilley, E., Babar, J., Beckford, J., Else, T.R., Sadler, T.J., et al.: A study on the adequacy of common iqa measures for medical images. arXiv preprint arXiv:2405.19224 (2024)
work page Pith review arXiv 2024
Show all 26 references
-
[8]
arXiv preprint arXiv:2403.04229 (2024)
Ecclestone, B.R., Simmons, J.A.T., Tweel, J.E., Kaur, C., Hajiahmadi, A., Reza, P.H.: Photon absorption remote sensing (pars): A comprehensive approach to label-free absorption microscopy across biological scales. arXiv preprint arXiv:2403.04229 (2024)
2024 arXiv
-
[9]
arXiv preprint arXiv:1803.11293 (2018)
Rivenson, Y., Wang, H., Wei, Z., Zhang, Y., Gunaydin, H., Ozcan, A.: Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue. arXiv preprint arXiv:1803.11293 (2018)
2018 arXiv
-
[10]
arXiv preprint arXiv:2501.03592 (2025)
Wang, J., Xiong, B., Zhou, Y., Cao, X., Ma, Z.: A value mapping virtual staining framework for large-scale histological imaging. arXiv preprint arXiv:2501.03592 (2025)
2025 arXiv
-
[11]
arXiv preprint arXiv:2306.08583 (2023)
Tweel, J.E., Ecclestone, B.R., Boktor, M., Simmons, J.A.T., Fieguth, P., Reza, P.H.: Virtual histology with photon absorption remote sensing using a cycle- consistent generative adversarial network with weakly registered pairs. arXiv preprint arXiv:2306.08583 (2023)
2023 arXiv
-
[12]
Scientific Reports 14(1), 2009 (2024)
Boktor, M., Tweel, J.E., Ecclestone, B.R., Ye, J.A., Fieguth, P., Haji Reza, P.: Multi-channel feature extraction for virtual histological staining of photon absorption remote sensing images. Scientific Reports 14(1), 2009 (2024)
2024
-
[13]
In: ACM SIGGRAPH 2022 Conference Proceedings, pp
Saharia, C., Chan, W., Chang, H., Lee, C., Ho, J., Salimans, T., Fleet, D., Norouzi, M.: Palette: Image-to-image diffusion models. In: ACM SIGGRAPH 2022 Conference Proceedings, pp. 1–10 (2022)
2022
-
[14]
In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp
Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with condi- tional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)
2017
-
[15]
In: Proceedings of the IEEE International Conference on Computer Vision, pp
Zhu, J.-Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image transla- tion using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017) 18
2017
-
[16]
IEEE transactions on image processing 13(4), 600–612 (2004)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assess- ment: from error visibility to structural similarity. IEEE transactions on image processing 13(4), 600–612 (2004)
2004
-
[17]
In: The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, vol
Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multiscale structural similarity for image quality assessment. In: The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, vol. 2, pp. 1398–1402 (2003). Ieee
2003
-
[18]
Tian, Y., Chen, B., Wang, S., Kwong, S.: Towards thousands to one reference: Can we trust the reference image for quality assessment? IEEE Transactions on Multimedia 26, 3278–3290 (2023)
2023
-
[19]
IEEE Transactions on Multimedia 26, 2219–2234 (2023)
Xian, W., Zhou, M., Fang, B., Xiang, T., Jia, W., Chen, B.: Perceptual quality analysis in deep domains using structure separation and high-order moments. IEEE Transactions on Multimedia 26, 2219–2234 (2023)
2023
-
[20]
Journal of Digital Imaging 36(6), 2623–2634 (2023)
Ohashi, K., Nagatani, Y., Yoshigoe, M., Iwai, K., Tsuchiya, K., Hino, A., Kida, Y., Yamazaki, A., Ishida, T.: Applicability evaluation of full-reference image quality assessment methods for computed tomography images. Journal of Digital Imaging 36(6), 2623–2634 (2023)
2023
-
[21]
Journal of Imaging 8(8), 224 (2022)
Varga, D.: Full-reference image quality assessment based on an optimal linear combination of quality measures selected by simulated annealing. Journal of Imaging 8(8), 224 (2022)
2022
-
[22]
In: 2024 IEEE International Conference on Contemporary Computing and Communica- tions (InC4), vol
Sujana, D.S., Augustine, D.P., Grace, D.S.R.: Full reference image quality assess- ment (fr-iqa) of pre-processed structural magnetic resonance images. In: 2024 IEEE International Conference on Contemporary Computing and Communica- tions (InC4), vol. 1, pp. 1–5 (2024). IEEE
2024
-
[23]
Multimedia Tools and Applications, 1–34 (2024)
Rodrigues, R., L´ evˆ eque, L., Guti´ errez, J., Jebbari, H., Outtas, M., Zhang, L., Chetouani, A., Al-Juboori, S., Martini, M.G., Pinheiro, A.M.: Objective quality assessment of medical images and videos: Review and challenges. Multimedia Tools and Applications, 1–34 (2024)
2024
-
[25]
IEEE Transactions on Image processing 6(7), 965–976 (1997)
Jobson, D.J., Rahman, Z.-u., Woodell, G.A.: A multiscale retinex for bridg- ing the gap between color images and the human observation of scenes. IEEE Transactions on Image processing 6(7), 965–976 (1997)
1997
-
[26]
IEEE Transactions on Biomedical Engineering 71(6), 1901–1912 (2024) 19
Tweel, J.E., Ecclestone, B.R., Boktor, M., Dinakaran, D., Mackey, J.R., Reza, P.H.: Automated whole slide imaging for label-free histology using photon absorp- tion remote sensing microscopy. IEEE Transactions on Biomedical Engineering 71(6), 1901–1912 (2024) 19
2024
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.