{"id":"9b62e3e9-19f6-4276-8970-59ba44e1649c","arxiv_id":"2608.03079","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"CorePath, a breast-specialized pathology model, improves zero-shot breast biopsy classification across centers and, with conformal risk control, releases fewer hallucinated reports.","lead":"This paper fine-tunes a general pathology AI model on breast biopsy images and reports, then adds a safety layer that only releases confident AI-written diagnoses. The system improves breast cancer subtyping and lowers made-up disease descriptions, but the strongest safety claim is asserted rather than measured.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'zero non-breast hallucinations' headline for CorePath-CRG rests on a design assertion, not the same Qwen3-based measurement used for PRISM and CorePath; Section 2.6 explicitly excludes CorePath-CRG from the hallucination comparison.","rationale":"The paper's most defensible contribution is the classification core: CorePath improves over PRISM on zero-shot breast CNB tasks across multiple private centers and on public benchmarks with independent labels, so the classifier result is not the primary weakness. The load-bearing concern is the report-safety headline. The abstract and results text claim a strict 0% non-breast hallucination rate for CorePath-CRG, but Section 2.6 explicitly removes CorePath-CRG from the Qwen3-based hallucination evaluation used for PRISM and CorePath, and Figure 3a omits it. The only stated basis for the 0% figure is constrained decoding and guardrails, which is a design argument, not a measured outcome. The LTT calibration described in Supplementary Methods A.5 additionally controls a Judge-Score-defined risk on the calibration set, not the Qwen3-defined hallucination rate, and the external-center application violates the exchangeability assumption the guarantee requires. The reader's stated weakest assumption focuses on LLM-extracted labels, which is a real secondary concern: if those labels are systematically wrong, both training and evaluation could be affected. However, the public benchmark results partially buffer the classification claim because those labels are externally defined. The hallucination protocol gap is more directly fatal to the abstract's safety claim, and it is fixable with a straightforward evaluation. I therefore keep the reader's CONDITIONAL verdict rather than moving to accept or reject; the classifier results remain credible, but the zero-hallucination claim should not be reported as measured until the same detector is run on released CorePath-CRG outputs.","tokens_in":33327,"tokens_out":4517,"duration_ms":41255,"concrete_test":"","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.6 states that hallucination evaluation is performed only on PRISM and CorePath using Qwen3-30B-A3B-Instruct-2507, and that CorePath-CRG is excluded from the primary comparison to maintain a fair evaluation under identical unconstrained generation conditions. Section 3.3 nevertheless asserts that CorePath-CRG 'eliminates the residual errors entirely via constrained decoding, achieving a strict 0% hallucination rate among released outputs,' and the abstract repeats this as a headline result. No alternative measurement, such as running the same Qwen3 detector on released CorePath-CRG outputs or a pathologist-based hallucination audit, is reported. The LTT guarantee in Supplementary Methods A.5 controls the rate of Judge-Score-defined clinical risk on the calibration distribution; it does not by itself imply zero hallucinations, and applying the SPH-2-calibrated threshold to SWH, WCH-2, WTH, and SJH relies on exchangeability that the Discussion admits may be violated by distribution shift. Therefore the strongest safety claim, strict 0% hallucination among released outputs, is currently an assertion about system design rather than a measured result.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript introduces CorePath, a breast-specific multimodal pathology foundation model obtained by Perceiver-only fine-tuning of PRISM on 7901 paired breast core-needle-biopsy whole-slide images and diagnostic reports from two centers. The authors evaluate zero-shot classification across six private cohorts and two public benchmarks (BCNB, BRACS), reporting that CorePath consistently outperforms PRISM and TITAN on cancer detection, invasion assessment, and histological subtyping (weighted AUCs 0.9526-0.9735 on five-class subtyping). They further develop CorePath-CRG, a conformalized and Learn-Then-Test-gated report-generation framework with three-tier selective release (trusted synthesis, subtype-only fallback, and deferral), and claim zero non-breast hallucinations among released outputs. The paper includes pathologist validation of an LLM-based Evaluation Score (ICC(A,1)=0.843 on 144 reports) and reports improved Evaluation Scores and text-similarity metrics for CorePath-CRG.","tokens_in":33574,"tokens_out":8853,"duration_ms":78003,"significance":"If confirmed, the classification results are a strong demonstration that domain-specialized fine-tuning of a histopathology foundation model improves zero-shot generalization across institutions and public datasets; the use of bootstrap confidence intervals and multiple independent test cohorts is a notable strength. The report-generation risk-control architecture is a timely contribution, and the paper appropriately separates diagnosis from output authorization. However, the headline safety claim of a 'strict 0% hallucination rate' is not empirically supported as written, and the ground-truth labels depend on unvalidated LLM extraction. The statistical proof of the LTT guarantee also needs strengthening.","major_comments":[{"comment":"The claimed 'strict 0% hallucination rate' for CorePath-CRG is not measured: Section 2.6 states that CorePath-CRG is excluded from the hallucination evaluation, and Figure 3a reports only PRISM and CorePath. The LTT guarantee in Supplementary A.5 controls the rate of Judge-Score-defined clinical risk on the SPH-2 calibration set, not the hallucination rate on the four external centers. Please measure the hallucination rate of released CorePath-CRG reports using the same Qwen3-based detector or a pathologist audit, report the observed count with a confidence interval, and revise the abstract and Section 3.3 accordingly.","section":"2.6, 3.3, Abstract"},{"comment":"The ground-truth labels for both fine-tuning and evaluation are derived from LLM extraction (DeepSeek-R1-Distill-Qwen-32B) and translation (OpenBioLLM-70B), with manual review only 'when needed'. The manuscript does not report the proportion of reports manually reviewed, inter-method agreement, or an error analysis. Since every reported AUC depends on these labels, a systematic extraction error would directly bias the classification results. Please provide a statistical validation of the label-extraction pipeline (e.g., a random sample audited by a pathologist, with agreement rates and a breakdown of discrepancies) or otherwise justify the reliability of the labels.","section":"2.2"},{"comment":"The proof sketch for the distribution-free risk-control guarantee is incomplete: the p-values p_k defined in Equation (11) are not shown to be super-uniform or monotone in k, and the claim that the nested structure of the acceptance sets A_k suffices to control FWER for the maximum-rejecting rule is not justified. The guarantee in Corollary A.4 is load-bearing for the safety claims of CorePath-CRG, so please provide a rigorous derivation, or state precisely which theorem in [17] is being invoked and verify that its monotonicity conditions hold for this choice of acceptance sets and risk indicators.","section":"A.5.4, Theorem A.3"},{"comment":"The LLM-based Evaluation Score used for the main report-quality comparison is computed by the DeepSeek API, the same model family used for the Self-Confidence Score and Judge Score inside CorePath-CRG. Although the Evaluation Score was validated against a pathologist on 144 reports (Table S18, ICC(A,1)=0.843), the validation subset is small and the possibility remains that the evaluator is biased toward outputs from the DeepSeek-based synthesis pipeline. Please report per-center agreement statistics for the pathologist validation, or compare models with an independent human-annotated subset across all centers, to rule out scoring bias as an explanation for the reported improvements.","section":"2.6, A.3, A.4"}],"minor_comments":[{"comment":"Reference [16] contains a typo: 'Vladimir V ovk' should read 'Vladimir Vovk'.","section":"References"},{"comment":"Figure 3(a) shows hallucination rates only for PRISM and CorePath; the caption should state that CorePath-CRG is omitted and explain why.","section":"Figure 3"},{"comment":"Section 2.2 states that translations were manually reviewed 'when needed'; please specify the criteria and the number of reports audited.","section":"2.2"},{"comment":"The abstract's 'zero non-breast hallucinations among released outputs' should be revised to reflect the actual measurement once provided; as written it overstates the evidence.","section":"Abstract"},{"comment":"Table S20 reports retained-sample metrics; consider adding a column with the N per center to the main text to make the selective-release behavior transparent.","section":"Table S20"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the journal's scope and the classification core is likely solid. The main risks are the unmeasured hallucination claim and the LLM-derived labels. I recommend major revision rather than rejection because these issues appear addressable with additional analyses. Please ensure the authors provide the requested measurements."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"CorePath is a breast-specialized fine-tune of PRISM that consistently beats the base model and TITAN on zero-shot breast CNB classification across six private centers and two public benchmarks. The classification work looks genuinely solid: independent cohorts, bootstrap CIs, and internally consistent gains, especially in histological subtyping. The report-generation story is more uneven. The improvement from PRISM's 30% non-breast hallucination rate down to 2.8% for CorePath is measured with Qwen3, but the claim that CorePath-CRG achieves a strict 0% hallucination rate is not measured with the same protocol. Section 2.6 explicitly excludes CorePath-CRG from the hallucination comparison to maintain \"fair\" unconstrained conditions, then Section 3.3 and the abstract assert 0% as a design outcome. LTT controls the Judge-Score-defined risk on the calibration set; it does not imply zero hallucinations, and applying the SPH-2-calibrated threshold to SWH, WCH-2, WTH, and SJH rests on exchangeability assumptions the Discussion admits may be violated. They need to run the same Qwen3 detector on released CorePath-CRG outputs, or provide a pathologist audit, before making that headline claim.\n\nThe second soft spot is the label pipeline: diagnostic labels and English translations were produced by DeepSeek-R1-Distill-Qwen-32B and OpenBioLLM-70B, with manual review only \"when needed.\" If those extractions are systematically wrong, both the training signal and the ground-truth labels for the classification AUCs are corrupted. A small manual audit of a random sample would settle this. The LLM-as-judge evaluation is partly self-referential (DeepSeek used for Self-Confidence, Judge, and Evaluation Scores), but the pathologist validation of the Evaluation Score on 144 reports shows Spearman 0.911 and ICC 0.843, which is decent evidence the metric has some external anchor.\n\nThe LTT theory is standard and correctly applied, but the authors oversell what it buys: it controls a specific risk definition, not all hallucinations. The core classifier results hold up, and the idea of combining domain specialization with conformal gating for open-ended report generation is worth pursuing. This paper deserves serious peer review, but it needs revision: measure the hallucination rate of released CorePath-CRG outputs honestly, validate a sample of labels, and soften the zero-hallucination wording to what the data actually support.","headline":"Solid classification results with a credible breast-specialized fine-tune of PRISM, but the headline 'zero hallucinations' claim for CorePath-CRG is asserted rather than measured and needs an honest fix before the paper is accepted.","tokens_in":34158,"tokens_out":1970,"would_cite":true,"duration_ms":18986,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A breast-specialized fine-tune of a pathology foundation model outperforms the base model on zero-shot breast core needle biopsy diagnosis and, with conformal risk control, releases zero non-breast hallucinations.","keywords":["breast core needle biopsy","pathology foundation model","zero-shot classification","whole-slide imaging","conformal prediction","risk-controlled report generation","hallucination reduction","multimodal fine-tuning"],"falsifier":"Have a pathologist who was not involved in the study re-derive the five-class subtype labels directly from the slides of one entire private cohort, then rerun CorePath's zero-shot prompts and compare weighted AUCs; a meaningful drop below the reported 0.9526-0.9735 band would show that label extraction, not morphology, drove part of the result. Alternatively, on a deliberately shifted cohort, count how many auto-released CorePath-CRG reports a panel judges clinically unacceptable; if substantially more than the declared 10% risk bound fail, the exchangeability assumption behind the guarantee did not hold.","tokens_in":1926,"feed_emoji":"🔬","tokens_out":2725,"duration_ms":71528,"temperature":0.7,"pith_summary":"The paper argues that a pathology foundation model specialized to breast core needle biopsy (CNB) can beat a general-purpose foundation model at zero-shot diagnosis and do so safely enough for clinical report generation. The authors fine-tune PRISM on 7,901 paired whole-slide images and diagnostic reports from two centers, then test it without task-specific retraining on six private cohorts and two public benchmarks. Across cancer detection, invasion assessment, and five-class histological subtyping, the specialized model consistently achieves higher weighted AUCs than the base model, with subtyping AUCs between 0.9526 and 0.9735. For report generation, the specialization cuts severe non-breast hallucinations from 30.1% to 2.8%, and a conformal risk-control extension eliminates them entirely among released reports.","feed_headline":"Breast-specialized AI tops general model in biopsy diagnosis","feed_subtitle":"Specialized fine-tuning lifts zero-shot biopsy accuracy; risk-controlled release blocks hallucinated reports.","key_machinery":"The central object is CorePath-CRG, a three-tier selective report generation pipeline built on the fine-tuned CorePath model. CorePath itself is produced by Perceiver-only fine-tuning: small adapter modules are inserted into the feed-forward sublayers of the Perceiver slide encoder, which aggregates patch-level features into a whole-slide representation, while the language encoder and all cross-modal components stay frozen. CorePath-CRG adds conformal calibration of diagnostic probabilities, a Self-Confidence Score that rates candidate reports without ground truth, a Judge Score used only during calibration, and a Learn-Then-Test procedure that selects a release threshold with a finite-sample guarantee on the risk of auto-releasing an unacceptable report. At inference, cases above the threshold go to trusted synthesis, cases below it with a confident conformal subtype fall back to subtype-only output, and cases with neither are deferred as Unknown for pathologist review.","core_discovery":"The authors report that a breast-specialized pathology foundation model, obtained by fine-tuning only the Perceiver-based slide encoder of PRISM on paired CNB whole-slide images and diagnostic reports, consistently outperforms the unmodified PRISM model in zero-shot breast CNB classification across six private centers and two public benchmarks. The gains appear at every diagnostic level, from cancer detection to invasion assessment to five-class histological subtyping, with weighted subtyping AUCs of 0.9526-0.9735 on private cohorts and 0.7780, 0.8178, and 0.8252 on the BCNB and BRACS public tasks. The same model reduces severe non-breast hallucinations in generated reports from 30.1% to 2.8%, and the CorePath-CRG extension, which applies conformal subtype gating and Learn-Then-Test threshold calibration to a three-tier selective release policy, achieves zero non-breast hallucinations among released reports while preserving high-quality outputs when confidence is sufficient.","pith_inferences":["Editorial inference: The same recipe, fine-tuning only the visual encoder of a multimodal foundation model on paired WSI-report data and then gating releases by conformal risk control, could transfer to other organ systems where core needle biopsy diagnosis is difficult, such as prostate or thyroid, although the paired data requirement would need to be met anew.","Editorial inference: The risk-control layer is largely independent of the underlying generative model, so Learn-Then-Test gating could be attached to any pathology report generator as a plug-in safety mechanism, not just to CorePath.","Editorial inference: Because the reported labels and English translations were produced by language models and only partly reviewed by humans, the cleanest external check is to re-label a full cohort by pathologists directly from slides and rerun the zero-shot evaluation; if the extracted labels were systematically biased, the reported accuracy numbers would shift."],"forward_implications":["A breast-specialized pathology foundation model can be used for zero-shot hierarchical diagnosis, meaning new diagnostic label spaces can be defined by text prompts without training task-specific classifiers.","Fine-tuning only the visual encoder on paired WSI-report data preserves the language capabilities of the base model while anchoring its output to the breast pathology domain, cutting non-breast hallucinations by an order of magnitude.","Adding conformal gating and Learn-Then-Test risk control to report generation makes it possible to guarantee, with user-chosen confidence, that auto-released reports stay below a target risk level, while uncertain cases are explicitly deferred to pathologists.","The three-tier release policy separates diagnostic inference from output authorization, so a model can refuse to narrate a case it is unsure about while still returning a confident subtype label.","LLM-based Evaluation Scores validated against pathologist scores (Spearman rho 0.911, ICC(A,1) 0.843) can serve as a scalable proxy for clinical report quality in this setting."],"supporting_citations":[{"why":"Supplies the PRISM base model with its pretrained multimodal architecture and patch-level features that CorePath fine-tunes.","marker":"[11]"},{"why":"Provides the TITAN comparison pipeline and its patch-level feature extraction approach used as a baseline in the zero-shot evaluations.","marker":"[14]"},{"why":"Provides the conformal prediction and Learn-Then-Test theory that underlies the risk-controlled report release in CorePath-CRG.","marker":"[16, 17]"},{"why":"The LLM used to extract structured diagnostic information from Chinese reports, which defines the labels for fine-tuning and evaluation.","marker":"[22]"},{"why":"The biomedical language model used to translate Chinese diagnostic text into English for WSI-report alignment supervision.","marker":"[23]"},{"why":"Defines the Perceiver architecture that is the slide encoder targeted by CorePath's parameter-efficient fine-tuning.","marker":"[24]"},{"why":"Supplies the BCNB public benchmark used for external evaluation of invasive carcinoma subtype categorization.","marker":"[30]"},{"why":"Supplies the BRACS public benchmark used for lesion stratification and fine-grained breast lesion classification.","marker":"[31]"}],"fun_headline_variants":["Breast-specialized AI outperforms general model in needle biopsy","Risk-controlled AI slashes breast biopsy report hallucinations","Zero-shot breast biopsy AI with risk control beats general model","Specialized fine-tuning lifts biopsy accuracy, blocks bad reports","Breast-specific AI tops general model in biopsy diagnosis and safety"],"cache_read_input_tokens":36224,"weakest_assumption_plain":"The whole pipeline, including training labels, evaluation ground truth, and translated report text, depends on what language models extracted from the Chinese pathology reports, and only a fraction of that extracted text was manually checked; if those extractions are systematically wrong, the reported accuracy is measuring mistakes as ground truth.","fun_headline_variants_meta":{"raw":{"variants":["Breast-specialized AI outperforms general model in needle biopsy","Risk-controlled AI slashes breast biopsy report hallucinations","Zero-shot breast biopsy AI with risk control beats general model","Specialized fine-tuning lifts biopsy accuracy, blocks bad reports","Breast-specific AI tops general model in biopsy diagnosis and safety"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000263,"raw_usage":{"total_tokens":1665,"prompt_tokens":1078,"completion_tokens":587,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":694,"completion_tokens_details":{"reasoning_tokens":505}},"tokens_in":694,"tokens_out":587,"duration_ms":6263,"temperature":1.0,"reasoning_tokens":505,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:52:48.596313+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have a pathologist who was not involved in the study re-derive the five-class subtype labels directly from the slides of one entire private cohort, then rerun CorePath's zero-shot prompts and compare weighted AUCs; a meaningful drop below the reported 0.9526-0.9735 band would show that label extraction, not morphology, drove part of the result. Alternatively, on a deliberately shifted cohort, count how many auto-released CorePath-CRG reports a panel judges clinically unacceptable; if substantially more than the declared 10% risk bound fail, the exchangeability assumption behind the guarantee did not hold.","supporting_citations":[],"review_version":1}