{"id":"80f929dc-a795-490b-bb80-2a17a47f9952","arxiv_id":"2411.11613","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Converting OCT and RCM scans into H&E-like virtual histology lets off-the-shelf computational pathology models segment and classify skin tissue more accurately, with no model retraining.","lead":"This paper shows that converting noninvasive optical images (OCT and RCM) into H&E-like virtual histology before feeding them to existing computational pathology AI models improves accuracy without retraining those models. The approach, called FoundationShift, could let clinicians apply powerful pathology AI to live tissue scans for real-time analysis.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The RCM cell-segmentation result is entangled with its own ground truth: Hover-Net on virtual H&E generated the 4,503-cell annotation and is also the method being scored, so the reported superiority over CellProfiler is not independently established.","rationale":"The reader's weakest_assumption identifies the same load-bearing issue: the RCM cell-segmentation ground truth is initialized by the very model being evaluated, creating a circularity that directly threatens the quantitative RCM result. My read of the manuscript confirms this. The OCT tissue-segmentation experiment appears credible: expert ground truth was created independently (with inter-expert Dice 0.85), and domain transfer plus MedSAM shows large, statistically significant improvements. However, the main claim is not limited to OCT; it explicitly names Hover-Net and RCM. The paper's own acknowledgment of the ground-truth bias, plus the small 50-cell control, does not fully dispel the concern. The UNI BCC classification is also a single-mosaic demo with a post hoc cosine threshold, but the circular RCM ground truth is more central because it is presented as a quantitative benchmark against CellProfiler. The paper already received a CONDITIONAL verdict with this concern; my stress-test does not move that verdict. If the independent ground-truth test succeeds, the condition is met; if it fails, the RCM cell-segmentation claim should be removed.","tokens_in":12211,"tokens_out":3352,"duration_ms":35258,"concrete_test":"Regenerate the RCM cell-segmentation ground truth independently: have one or more blinded annotators manually segment all cells in the four RCM images (or a stratified random sample of at least 500 cells) from the original RCM images, without viewing Hover-Net, CellProfiler, or virtual H&E outputs. Then compute DQ, SQ, and PQ for Hover-Net on virtual H&E versus CellProfiler on RCM using only this independent ground truth. If Hover-Net plus domain transfer still outperforms CellProfiler by similar margins, the circularity concern is resolved; if the margin collapses or reverses, the RCM cell-segmentation claim should be removed or substantially weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that FoundationShift 'significantly enhances the accuracy of... Hover-Net' for RCM cell segmentation rests on a circular evaluation. In Methods (RCM Cell Segmentation and Statistical Analysis), the ground truth was created by first running Hover-Net on the virtual H&E images and then manually adjusting the results, exactly the model and input modality that constitute 'our method.' The paper acknowledges this ('We acknowledge that generating ground truth in this method may skew results in favor of Hover-Net') and reports a 50-cell control using a CellProfiler-derived ground truth with Dice 0.9 between the two ground truths. That control is helpful but not sufficient: 50 cells is about 1% of the 4,503 cells, the control ground truth also involves manual adjustment and is tied to the same annotation workflow, and agreement between ground-truth masks does not directly bound the DQ/SQ/PQ differences that drive the headline comparison. If the manual 'adjustment' preserves Hover-Net's systematic errors (e.g., oversegmentation or boundary placement biases), the reported superiority over CellProfiler and the zero-shot cell-segmentation claim are not supported by current evidence. This is load-bearing because RCM is one of only two modalities used to support the paper's main claim, and the RCM cell-segmentation experiment is the only quantitative cell-level result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FoundationShift, a method for applying off-the-shelf computational pathology (CPath) models to noninvasive optical images by first translating OCT and RCM images into H&E-like 'virtual histology' images using existing domain-transfer models, without retraining or fine-tuning the CPath models. The authors evaluate FoundationShift on several tasks: epidermis segmentation in OCT using SAM, MedSAM, and SAM-Med2D; cell segmentation in RCM using Hover-Net compared with CellProfiler; image retrieval using PLIP; BCC nodule classification using UNI; and image captioning using GPT-4o. The central claim is that domain transfer to H&E-like images significantly improves the accuracy of these models on noninvasive optical images, thereby enabling zero-shot use of CPath foundation models on OCT and RCM data.","tokens_in":12529,"tokens_out":4868,"duration_ms":48320,"significance":"The OCT segmentation results are the strongest part of the paper: they use 95 paired OCT/H&E images, expert ground truth with inter-observer agreement analysis, paired Wilcoxon tests, and consistent improvements across three segmentation models, with data and code links provided. If the RCM cell-segmentation result were independently validated, FoundationShift would be a practical and potentially impactful way to repurpose the large ecosystem of H&E-trained CPath models for noninvasive optical imaging, which is a clinically relevant direction. The paper is also commendable for evaluating multiple model families (generic segmentation, medical segmentation, cell segmentation, vision-language models, large language models) and for explicitly discussing limitations of the domain-transfer step. However, the RCM cell-segmentation evaluation is entangled with its own ground truth, and several secondary claims (UNI classification, ChatGPT analysis) rest on small or subjective evaluations. The overall significance is high if the RCM circularity can be resolved, but current evidence is mixed.","major_comments":[{"comment":"The RCM cell-segmentation ground truth was created by first running Hover-Net on the virtually stained H&E images and then manually adjusting the results, as stated in this section. Because the evaluated method is exactly Hover-Net applied to the same virtual H&E input, this creates a circular evaluation: the model's own outputs seed the reference standard. The authors acknowledge that this 'may skew results in favor of Hover-Net,' but the 50-cell control using a CellProfiler-derived ground truth is insufficient: 50 cells is roughly 1% of the 4,503-cell evaluation set, the control still involves manual adjustment within the same workflow, and the reported Dice score of 0.9 between the two ground truths does not directly bound the DQ/SQ/PQ differences used for the headline comparison. This circularity is load-bearing because RCM is one of only two modalities supporting the paper's main claim, and the RCM experiment is the only quantitative cell-level result. The authors should provide an independently annotated ground truth, for example manual expert annotation on raw RCM images or on virtual H&E with an annotation protocol that does not involve Hover-Net, and re-run the comparison before claiming that FoundationShift 'significantly enhances the accuracy of ... Hover-Net' for RCM cell segmentation.","section":"Methods, RCM Cell Segmentation and Statistical Analysis; Figure 2c"},{"comment":"The bounding box prompt used for all OCT segmentation models is derived from expert annotations, with a height factor adjustment of 1.15, as described in this section. Since the same expert annotations define the ground truth used for Dice evaluation, the segmentation models receive prompt information that is already aligned with the reference standard. This is a form of ground-truth leakage that inflates absolute Dice scores and makes the reported accuracies unrepresentative of a true zero-shot deployment, where users would need to supply or generate prompts without access to expert ground truth. The relative improvement from domain transfer is likely still informative because the same prompts are used in both conditions, but the paper's claim that FoundationShift 'significantly improves model accuracy' for off-the-shelf models needs to be qualified by this protocol. Please state clearly that the evaluation assumes expert-provided bounding boxes, and, if feasible, include a comparison with automatically generated prompts (e.g., from a simple detector or a fixed grid) to assess performance in a more realistic setting.","section":"Methods, OCT Tissue Segmentation and Statistical Analysis"},{"comment":"The UNI classification experiment is conducted on a single RCM mosaic, and the reported F1 scores depend on a cosine-similarity color mapping that the authors state was manually selected to maximize visibility of BCC nodules and minimize false negatives. This hand-tuning of the visualization threshold, combined with the single-image evaluation, means that the reported F1 values (ResNet-50: 0.46, UNI: 0.35, FoundationShift+UNI: 0.85) are not a reliable quantitative demonstration that FoundationShift 'significantly enhances' UNI. The claim should be supported by a pre-specified threshold, multiple test images, and ideally a statistical test across images. Without this, the UNI result remains anecdotal and should be presented as such rather than as a quantitative comparison.","section":"FoundationShift Applied to Visual Language Foundation Model Classification of RCM (Supplementary Figure S7)"}],"minor_comments":[{"comment":"There is a typo in 'promt' (should be 'prompt') and the model name 'GPT-o4' appears in the text, which should be 'GPT-4o' to be consistent with the rest of the paper. Please also correct 'utilzing' and other small spelling errors throughout the manuscript.","section":"FoundationShift Applied to Large Language Model Analysis of OCT"},{"comment":"The paper states that all data needed to evaluate the conclusions are in the paper and/or Supplementary Materials, but the RCM data and annotation spreadsheets are provided via external Google Drive links. Please make the data availability statement more explicit by describing what files are included, and consider depositing the code and processed data in a persistent repository to improve reproducibility.","section":"Data availability"},{"comment":"The KL divergence is computed on VGG16 embeddings, but the manuscript does not specify which layer or pooling is used, nor whether the values are symmetric or averaged across images. Providing these details would make the analysis reproducible and would clarify the interpretation of the reported divergence values.","section":"Theoretical Discussion (Supplementary Figure S2)"},{"comment":"The paper reports multiple paired Wilcoxon tests across three segmentation models but does not mention any multiple-testing correction. The p-values are extremely small, so the conclusions are unlikely to change, but a brief note about this would strengthen the statistical reporting.","section":"OCT Tissue Segmentation and Statistical Analysis"}],"recommendation":"major_revision","confidential_remarks":"The OCT portion of the paper is well executed and likely publishable after clarification of the prompt-leakage issue. The RCM cell-segmentation circularity is the main barrier to accepting the paper's broad claims, and the UNI classification result needs substantial strengthening. The paper would benefit from an independent RCM ground-truth annotation and a more rigorous evaluation of the secondary claims before publication. The self-citation to the authors' prior OCT2Hist paper is appropriate given the methodological continuity, but the dependence on Li et al. for RCM data and virtual staining should be described with sufficient detail for reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper is better than the RCM circularity problem makes it look at first glance. The OCT evidence is genuinely solid: 95 paired images, expert ground-truth segmentation, Wilcoxon tests, and consistent gains across SAM, MedSAM, and SAM-Med2D. Domain transfer lifts MedSAM's Dice from 0.67 to 0.80, and the effect is statistically significant. That part is credible and useful, and the data links are a plus. The PLIP recall@5 jump (19% to 93%) and the GPT-4o tissue-identification result (0% to 84%) are also striking, though the GPT-4o analysis is informal and should be treated as anecdotal. The paper's core idea—that virtual H&E preprocessing lets off-the-shelf CPath models work on OCT without retraining—is well supported by the OCT segmentation experiments.\n\nThe soft spot is real and exactly where the stress-test puts it. The RCM cell-segmentation ground truth was created by running Hover-Net on the virtual H&E images and then manually adjusting the output. That is the same model and input modality being scored. The paper acknowledges this, and the 50-cell control is a good-faith attempt, but it is only about 1% of the 4,503 cells, and agreement between two manually adjusted masks (Dice 0.9) does not rule out shared systematic errors like boundary placement or oversegmentation. So the claim that domain transfer + Hover-Net beats CellProfiler on RCM cells is not independently established. This matters because RCM is one of only two modalities and the only quantitative cell-level result. The UNI BCC classification is a single-mosaic demo with a post hoc threshold, so it should be described as qualitative, not as an F1-based benchmark.\n\nThe framing also oversells a bit: 'any model' really means the handful tested, and 'without retraining' applies to the CPath models, not to the domain-transfer network, which requires training (though on unlabeled data). Those are scoping issues, not fatal flaws.\n\nWho this is for: people working on noninvasive optical imaging, virtual staining, or clinical translation of CPath models. They will get a clear, reproducible demonstration on OCT and a cautionary example of how evaluation can get entangled with the tested model. It deserves a serious referee. I would send it to peer review, but with a required revision: regenerate the RCM ground truth independently (e.g., full manual annotation or a truly independent automated pipeline), add confidence intervals for DQ/SQ/PQ, and rescope the claims to the tested models and modalities. The OCT story alone is worth publishing; the RCM story needs the fix.","headline":"The OCT half is a solid, citable result; the RCM cell-segmentation claim is weakened by a circular ground truth and needs an independent re-annotation before the paper's strongest claims stand.","tokens_in":13078,"tokens_out":1586,"would_cite":true,"duration_ms":16970,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Converting OCT and RCM scans into H&E-like images lets off-the-shelf pathology AI analyze them accurately without retraining.","keywords":["computational pathology","domain transfer","virtual staining","optical coherence tomography","reflectance confocal microscopy","zero-shot segmentation","foundation models","noninvasive imaging"],"falsifier":"Obtain an independently annotated RCM cell segmentation ground truth, for example manual tracing by a dermatopathologist on raw RCM or a much larger CellProfiler-derived set with manual correction, then re-run the comparison on the same images or a larger cohort. If domain transfer plus Hover-Net no longer outperforms CellProfiler, or the gap narrows substantially, the central RCM cell segmentation claim fails. For the broader claim, apply FoundationShift to a noninvasive modality whose virtual staining model is known to be less accurate and check whether accuracy gains disappear as KL divergence to H&E increases.","tokens_in":1903,"feed_emoji":"🔬","tokens_out":2197,"duration_ms":60523,"temperature":0.7,"pith_summary":"The paper proposes FoundationShift, a preprocessing step that converts noninvasive optical images (OCT and RCM) into H&E-like \"virtual histology\" images before feeding them to off-the-shelf computational pathology models. The authors claim that this domain transfer, performed without any retraining or fine-tuning, substantially improves accuracy across tissue segmentation, cell segmentation, image retrieval, and image description. If true, it would let the large existing ecosystem of pathology AI models be applied to OCT and RCM data, bypassing the need for modality-specific annotated datasets exceeding 100,000 images. The paper reports that virtual H&E images lie much closer to the H&E training distribution, as measured by KL divergence, which the authors credit for the accuracy boost.","feed_headline":"Pathology AI reads OCT and RCM scans after a virtual H&E conversion","feed_subtitle":"Domain-shifted H&E-like images boost Dice, recall@5, and F1 scores for SAM, Hover-Net, PLIP, and UNI.","key_machinery":"The central object is a domain transfer network (also called virtual staining), such as OCT2Hist for OCT and the Li et al. model for RCM, which maps an optical image into an H&E-like image. These networks are trained on co-registered optical and H&E image pairs, typically using a few thousand images and micrometer-level registration accuracy. The H&E-like image is then passed to an off-the-shelf computational pathology model (SAM, MedSAM, SAM-Med2D, Hover-Net, PLIP, UNI, or GPT-4o) in zero-shot mode. The Kullback-Leibler (KL) divergence between virtual H&E and real H&E embeddings, computed with VGG16, is the explanatory metric: reducing the distance to the training domain is what the paper credits for the accuracy gains.","core_discovery":"FoundationShift's central claim is that converting optical images (OCT and RCM) to H&E-like images before using off-the-shelf computational pathology models significantly improves model accuracy. The paper demonstrates this across multiple models: SAM, MedSAM, and SAM-Med2D for epidermis segmentation in OCT (Dice scores rise from 0.68, 0.67, and 0.38 to 0.73, 0.80, and 0.74, respectively); Hover-Net for RCM cell segmentation, where raw Hover-Net segments zero cells but domain-transferred Hover-Net segments 4,503 cells and outperforms CellProfiler; PLIP for OCT image retrieval (recall@5 rises from 19% to 93%); GPT-4o for tissue identification (from 0% correct to over 80%); and UNI for BCC nodule classification in RCM (F1 score rises from 0.35 to 0.85). The authors also show that virtual H&E images reduce KL divergence to the H&E domain more than a comparable CT-to-MR domain transfer does.","pith_inferences":["If FoundationShift generalizes to other virtual staining models (photoacoustic, Raman, two-photon), a single pathology foundation model could serve many noninvasive modalities without retraining, potentially changing deployment of optical imaging AI in dermatology and beyond.","The reported KL-divergence reduction predicts that models whose training domain is closest to H&E will benefit most; a testable extension is to vary domain-transfer fidelity and measure accuracy versus KL distance across modalities.","The RCM cell segmentation ground truth was seeded by Hover-Net on virtual H&E, so the reported margin over CellProfiler may partly reflect this construction; an independent manual ground truth or a substantially larger CellProfiler-based control would settle the unbiasedness question."],"forward_implications":["Domain transfer plus MedSAM yields a statistically significant Dice improvement on OCT epidermis segmentation ($p < 2 \\times 10^{-15}$), with the largest gains for models that were trained on pathology images.","FoundationShift produces the first zero-shot cell segmentation pipeline for RCM images, since Hover-Net alone segments no cells on raw RCM but works on virtual H&E.","PLIP, a visual-language model with no OCT training, reaches 93% recall@5 on OCT after domain transfer, comparable to the 82% achieved on real H&E images.","GPT-4o identifies the tissue region in over 80% of virtual H&E images versus 0% on raw OCT, making LLM-based description of noninvasive scans feasible.","Because OCT and RCM are 3D modalities, FoundationShift extends 2D pathology foundation models to 3D volumes, demonstrated by a 3D epidermis and BCC nodule segmentation."],"supporting_citations":[{"why":"Supplies the OCT2Hist domain transfer model and the co-registration method (fluorescent gel and photobleached barcode) that aligns OCT with H&E.","marker":"[14]"},{"why":"Supplies the RCM virtual H&E domain transfer model and the four RCM images used for cell segmentation and BCC classification.","marker":"[15]"},{"why":"Segment Anything, the generic segmentation model evaluated with and without domain transfer on OCT.","marker":"[23]"},{"why":"MedSAM, the medical-image-tuned SAM variant used as a domain-specific segmentation baseline.","marker":"[24]"},{"why":"SAM-Med2D, the pathology-heavy SAM variant that showed the largest Dice gain after domain transfer.","marker":"[25]"},{"why":"Hover-Net, the cell segmentation model applied to raw RCM and virtual H&E, and the source of the detection/segmentation/panoptic quality metrics.","marker":"[26]"},{"why":"CellProfiler, the tunable cell segmentation baseline that FoundationShift's Hover-Net pipeline is compared against.","marker":"[27]"},{"why":"PLIP, the visual-language pathology model used for OCT image retrieval after domain transfer.","marker":"[28]"},{"why":"UNI, the self-supervised CPath foundation model used for BCC classification in RCM after domain transfer.","marker":"[30]"},{"why":"Provides the Hover-Net-based ground truth generation method used for RCM cell segmentation.","marker":"[38]"}],"fun_headline_variants":["Pathology AI analyzes OCT and RCM via virtual H&E, no retraining","Virtual H&E conversion lets pretrained pathology AI handle optical scans","FoundationShift: recycle pathology AI for OCT/RCM without retraining","Fake H&E images unlock pathology models for noninvasive optical imaging"],"cache_read_input_tokens":15104,"weakest_assumption_plain":"The RCM cell segmentation ground truth was created by running Hover-Net on the virtual H&E images and manually adjusting the results; if that ground truth is biased toward Hover-Net, the reported superiority over CellProfiler would not be supported.","fun_headline_variants_meta":{"raw":{"variants":["Pathology AI analyzes OCT and RCM via virtual H&E, no retraining","Virtual H&E conversion lets pretrained pathology AI handle optical scans","FoundationShift: recycle pathology AI for OCT/RCM without retraining","Fake H&E images unlock pathology models for noninvasive optical imaging"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000709,"raw_usage":{"total_tokens":3198,"prompt_tokens":955,"completion_tokens":2243,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":571,"completion_tokens_details":{"reasoning_tokens":2166}},"tokens_in":571,"tokens_out":2243,"duration_ms":16228,"temperature":1.0,"reasoning_tokens":2166,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:19:41.714919+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Obtain an independently annotated RCM cell segmentation ground truth, for example manual tracing by a dermatopathologist on raw RCM or a much larger CellProfiler-derived set with manual correction, then re-run the comparison on the same images or a larger cohort. If domain transfer plus Hover-Net no longer outperforms CellProfiler, or the gap narrows substantially, the central RCM cell segmentation claim fails. For the broader claim, apply FoundationShift to a noninvasive modality whose virtual staining model is known to be less accurate and check whether accuracy gains disappear as KL divergence to H&E increases.","supporting_citations":[{"cited_title":"Noninvasive virtual biopsy using micro-registered optical coherence tomography (oct) in human subjects","cited_arxiv_id":null,"evidence_quote":"Supplies the OCT2Hist domain transfer model and the co-registration method (fluorescent gel and photobleached barcode) that aligns OCT with H&E."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the RCM virtual H&E domain transfer model and the four RCM images used for cell segmentation and BCC classification."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Segment Anything, the generic segmentation model evaluated with and without domain transfer on OCT."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CellProfiler, the tunable cell segmentation baseline that FoundationShift's Hover-Net pipeline is compared against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"UNI, the self-supervised CPath foundation model used for BCC classification in RCM after domain transfer."}],"review_version":1}