{"id":"b657003f-0eee-414b-a02c-efdfcda22d83","arxiv_id":"2411.11458","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A prostate-specific self-supervised foundation model outperforms ImageNet-pretrained models on prostate cancer detection and improves mortality prediction when combined with clinical nomograms.","lead":"HistoEncoder is a neural network trained on 48 million prostate tissue images that learns to group similar tissue patterns together. It can classify prostate cancer and predict survival with little fine-tuning data, which may help hospitals with limited resources build clinical AI tools.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The prostate-s vs. natural-s comparison is confounded: §2.3.1 builds HelsinkiProstate by thresholding a trained prostate-cancer classifier, so label information enters pre-training and may explain the KNN and few-shot advantages.","rationale":"The reader's weakest_assumption identifies the same mechanism I consider most load-bearing: the selection of pre-training tiles via a trained cancer classifier injects label-dependent signal into the self-supervised pre-training distribution. This concern directly targets the strongest claim because the 'without fine-tuning' KNN comparison and the '1000 times less data' comparisons all rely on the feature geometry learned during pre-training. If the cancer-enriched curation is what creates the cancer-vs-benign axis in feature space, then the comparison to natural-s is not a fair test of tissue-specific SSL, and the causal attribution in the abstract and Section 3.1 is overstated. I do not see a more fundamental internal inconsistency: the methods are clearly described, code and weights are released, and the external PESO, Karolinska, and Radboud benchmarks are a genuine strength. The Helsinki30/Helsinki60 in-distribution reporting and the survival-analysis feature selection are real but secondary issues; they do not by themselves explain the external KNN gains. The proposed concrete test would settle the concern directly by removing the classifier filter while holding architecture, SSL objective, and data budget fixed. Since the reader already conditioned acceptance on this kind of concern, I keep the verdict unchanged rather than moving it.","tokens_in":18543,"tokens_out":4110,"duration_ms":45596,"concrete_test":"Train a matched XCiT-S12 with the same DINO recipe and the same 48M-tile budget, but sample tiles uniformly at random from the same HelsinkiProstate slides without applying the cancer-score threshold from §2.3.1. Repeat the exact KNN and few-shot fine-tuning protocol on PESO, Karolinska, and Radboud. If the no-filter model's KNN AUROC falls from the reported ~0.82/0.90 on Karolinska/Radboud toward the natural-s values (~0.66/0.78), the label-biased tile selection is the load-bearing cause of the advantage. If the no-filter model retains most of the gap, the domain-specific SSL claim survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that domain-specific self-supervised pre-training, not supervised curation, is what makes HistoEncoder features outperform ImageNet-DINO features. Section 2.3.1 describes constructing HelsinkiProstate by running the cancer classifier from Pohjonen et al. [13] on all tiles and keeping all 16M tiles with predicted cancer score >0.2 plus 32M random tiles with score ≤0.2. That classifier was trained on pathologist-annotated prostate cancer data, so its predictions encode expert labels. The resulting pre-training distribution is therefore a 1:2 cancer-enriched, label-dependent sample rather than an uncurated sample of prostate histology. DINO will organize features around the dominant visual axis separating the two curated groups, and the subsequent KNN classifier, which operates directly on feature geometry, is particularly sensitive to such an axis. This undermines the attribution in Section 3.1 that 'without any annotated information on tissue types being available during training' the model learns useful histology: the annotation signal is present indirectly in the tile-selection filter. The external PESO, Karolinska, and Radboud results do not rule this out, because the same label-filtered encoder is used in all of them. The comparison to natural-s is thus not a clean test of domain-specific SSL; it is a test of SSL on a cancer-enriched, classifier-selected corpus versus SSL on unselected ImageNet. If the label-filter is the true driver, the headline 'outperforms even without fine-tuning or with 1000 times less data' would need to be re-stated as 'outperforms when pre-training data are curated by a supervised cancer detector.'","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces HistoEncoder, a prostate pathology foundation model built by pre-training XCiT-S12 and XCiT-M24 backbones with DINO on 48 million tissue tile images derived from Helsinki University Hospital slides. The central claims are that these prostate-pretrained encoders outperform natural-image DINO counterparts in prostate cancer classification, even with a zero-training KNN classifier or with orders of magnitude less fine-tuning data; that learned feature clusters enable automatic annotation of large-scale datasets; and that patient-level cluster fractions improve prostate-cancer-specific mortality prediction over Gleason grade, CAPRA-S, and MSKCC-S nomograms. The evaluation includes external datasets (PESO, Karolinska, Radboud) and internal Helsinki datasets, with code made publicly available.","tokens_in":18856,"tokens_out":3168,"duration_ms":32181,"significance":"If the central attribution claim is correct, the paper demonstrates a concrete and reproducible path to data-efficient, domain-specific foundation models in digital pathology, with potentially high clinical value for resource-limited groups. The public code release, the use of external datasets for classification evaluation, the five-fold repeated experiments, and the candid statement about the lack of external validation for survival analysis are strengths. However, the headline claims currently rest on a comparison that is confounded by the label-dependent construction of the pre-training dataset and on benchmarks that overlap with pre-training, so the significance is not yet established.","major_comments":[{"comment":"The construction of HelsinkiProstate is label-dependent, which undermines the claim that HistoEncoder learns useful histology 'without any annotated information' and confounds the prostate-s versus natural-s comparison. Section 2.3.1 states that all 16 million tiles with predicted cancer score ŷ > 0.2 from the Pohjonen et al. classifier are kept, together with 32 million randomly sampled tiles with ŷ ≤ 0.2. That classifier was trained on pathologist-annotated prostate cancer data, so its predictions encode expert label information, and the resulting pre-training corpus is a 1:2 cancer-enriched, classifier-selected sample rather than an uncurated sample of prostate tissue. DINO organizes features around the dominant visual axis separating these two curated groups, and the KNN classifier used in Section 3.1 is particularly sensitive to such an axis. The external PESO, Karolinska, and Radboud results do not rule out this explanation, because the same label-filtered encoder is used in all of them. To support the attribution, the authors should provide an ablation that pre-trains on an unselected or score-stratified but non-thresholded subset of the same tile pool, or otherwise demonstrate that the label-filter is not the driver of the reported advantages.","section":"§2.3.1, §3.1"},{"comment":"Helsinki30 and Helsinki60 are reported as evaluation benchmarks even though Section 2.3.2 and Table 1 explicitly state that these datasets are part of HelsinkiProstate, the pre-training set. The high AUROC scores on these datasets therefore reflect test-on-train overlap and are not evidence of generalization. They should be removed from the main comparisons or reported only as memorization checks, with the external datasets carrying the generalization claims.","section":"Table 1, §2.3.2, Fig. 2, Fig. S6"},{"comment":"The survival analysis selects six out of 32 clusters based on a 'parameter importance analysis' on the full HelsinkiTMA cohort before performing 1,000 random stratified splits. This selection uses outcome information from the same cohort that is later split into training and test sets, yielding optimistically biased concordance, time-dependent AUC, and net benefit estimates. The paper correctly notes the lack of an external validation cohort, but it does not address the within-cohort selection circularity. The authors should either pre-specify the cluster selection protocol, use nested cross-validation in which cluster selection is performed inside each training fold, or restrict the reported results to an external cohort.","section":"§2.4.1, §3.3"}],"minor_comments":[{"comment":"The abstract says '48 million prostate tissue tile images' while Table 1 reports 898.4 million tiles for HelsinkiProstate before the classifier-based selection; please clarify that 48 million is the number of tiles retained after the score-thresholding and random sampling step.","section":"Abstract, Table 1"},{"comment":"The text reports that 45.6% of all tile images, 53.3% of cancerous epithelium, and 42.4% of benign epithelium and stroma fall in high-purity clusters, but the corresponding labels in Figure 4a appear to be ordered differently and one value is printed as 53.6% in the figure. Please reconcile the text and figure numbers.","section":"§3.2, Fig. 4"},{"comment":"Since Helsinki30 and Helsinki60 come from the same Pohjonen et al. work [13] that supplied the cancer classifier used for pre-training selection, the relationship between the classifier, these datasets, and HelsinkiProstate should be stated explicitly to avoid confusion.","section":"§2.3.2"},{"comment":"The heading 'Predicing prostate cancer-specific mortality' contains a typo; it should read 'Predicting'.","section":"§3.3"},{"comment":"The Discussion states that previous foundation models for histopathology were 'typically pre-trained with natural images' and cites [57-59], but those references are for ResNet, Inception, and Swin architectures rather than foundation-model training pipelines; please cite the actual ImageNet-pretrained models used in pathology transfer-learning studies.","section":"§4, References [57-59]"},{"comment":"The fine-tuning section does not specify some details needed for reproducibility, such as the choice of optimizer, learning rate schedule, number of epochs, and whether KNN features are L2-normalized before distance computation; please add these details or point to the code configuration.","section":"§2.4"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read: this is a useful, clearly written prostate pathology foundation model with released code and weights, but the headline comparison against ImageNet pre-training is not as clean as it looks. The pre-training dataset is built by running a supervised prostate cancer classifier on all tiles and keeping the 16M tiles it scores above 0.2 plus 32M random below-threshold tiles. That makes the SSL pre-training distribution cancer-enriched and label-dependent. So the claim that the model learns without any annotated information on tissue types is technically true about the DINO objective, but the tile selection itself injects expert labels. The external PESO, Karolinska, and Radboud results are real, but they all use the same label-filtered encoder, so they can't rule out this confound.\n\nWhat's good: the model release itself, the 48M-tile prostate-specific pre-training, and the cluster-purity annotation workflow are tangible contributions that could save pathologist time. The data-efficiency experiments are nicely designed, and the external datasets are appropriate. The KNN result is striking, and it would be interesting to see if it survives a corrected comparison.\n\nSoft spots beyond the confound: Helsinki30 and Helsinki60 are part of HelsinkiProstate, so reporting them as evaluation benchmarks is misleading; the relative prostate-vs-natural comparison is still informative, but the absolute numbers should be labeled in-distribution. The survival analysis selects six clusters based on outcome from the same cohort before the random splits, which is mild circularity, and the authors acknowledge there is no external validation for the survival part. They also do not benchmark against existing tissue-agnostic pathology foundation models (UNI, Virchow, etc.); they only mention it as future work.\n\nBottom line: worth a serious referee, but the central attribution needs fixing. The cleanest fix is an ablation with unselected or randomly selected prostate tiles; without that, the 'domain-specific SSL vs ImageNet SSL' claim should be softened to 'SSL on cancer-enriched, classifier-curated tiles vs SSL on ImageNet.'","headline":"Useful prostate pathology encoder with released code and weights, but the headline ImageNet comparison is confounded by cancer-classifier-based pre-training tile selection.","tokens_in":19425,"tokens_out":2048,"would_cite":false,"duration_ms":20750,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pre-training a Transformer on 48 million prostate-tissue tiles yields image features that outperform natural-image pre-training on prostate cancer tasks, even without fine-tuning.","keywords":["digital pathology","foundation model","self-supervised learning","prostate cancer","DINO","XCiT","transfer learning","histomics"],"falsifier":"Take the same 48M-tile pool, drop the $\\hat{y} > 0.2$ filter, randomly sample 48M tiles from all tissue, and re-run the DINO/XCiT pre-training with identical hyperparameters; then compare KNN AUROC on Karolinska and Radboud. If the unfiltered model's advantage over the ImageNet baseline largely disappears, the tile-selection filter, not histology-specific learning, drove the reported result.","tokens_in":18352,"feed_emoji":"🧫","tokens_out":8371,"duration_ms":70165,"temperature":0.7,"pith_summary":"This paper claims that a self-supervised model pre-trained on prostate tissue images captures histology far better than the same model pre-trained on natural images, and that this advantage is large enough to matter in practice. The authors train two XCiT vision transformers with the DINO self-supervised objective on 48 million tiles from prostate biopsies and prostatectomy sections, then compare them against identical models pre-trained on ImageNet. Across several prostate cancer classification datasets, the prostate-pretrained models achieve higher AUROC scores with far fewer fine-tuned parameters and far less training data; even a parameter-free KNN classifier on their features beats a fully fine-tuned ImageNet model. The same features also cluster into interpretable histological patterns, enabling automatic annotation of large slide datasets and adding prognostic information to Gleason, CAPRA-S, and MSKCC-S nomograms for prostate-cancer-specific death.","feed_headline":"Prostate pre-training beats ImageNet for cancer AI","feed_subtitle":"48M prostate tiles yield better pathology features than ImageNet, even with no fine-tuning.","key_machinery":"HistoEncoder is an XCiT (cross-covariance image transformer), a vision-transformer backbone with linear complexity in token count, trained with DINO, a self-supervised self-distillation objective, on 48 million 640x640 tissue tiles from 1,307 patients in the HelsinkiProstate dataset. The pre-training set is balanced by a cancer classifier from the authors' earlier work: all 16 million tiles with predicted cancer score $\\hat{y} > 0.2$ are kept, alongside 32 million randomly sampled tiles with $\\hat{y} \\le 0.2$. Downstream, the encoder is used in three ways: as a frozen feature extractor for KNN classification, as a partially fine-tuned backbone for classifiers, and as a source of tile-level features that are clustered into per-patient histological pattern fractions for survival models. The feature clustering is the load-bearing operation for the annotation and histomics use cases.","core_discovery":"The paper's central claim is that self-supervised pre-training on prostate tissue yields a feature extractor that beats natural-image pre-training for prostate cancer tasks, decisively and data-efficiently. The strongest demonstration is that a KNN classifier fit to prostate-s features, with no fine-tuning and no learned classifier, outperforms a fully fine-tuned natural-s model on every evaluation dataset. Under limited fine-tuning of the last transformer blocks, prostate-s reaches higher AUROC than natural-s with orders of magnitude fewer parameters and with as little as one-thousandth of the training tiles. The feature space is also semantically coherent: benign versus cancerous epithelium and different Gleason grades form separated clusters, and these clusters can be used to annotate more than two million PANDA tiles at over 90 percent purity after inspecting only 256 clusters. Finally, the paper shows that patient-level cluster fractions add clinically meaningful prognostic signal to Gleason grade, CAPRA-S, and MSKCC-S nomograms for prostate-cancer-specific death, forming the basis of two real workflows.","pith_inferences":["A cleaner test of the domain-specific pre-training claim would pre-train on randomly sampled tiles without the cancer-score filter; if the advantage shrinks, part of the foundation model value actually comes from the curation step rather than from self-supervised learning of histology.","If the domain-specific advantage holds up, the same DINO/XCiT recipe likely transfers to other organ sites, making organ-level pathology foundation models a general pattern rather than a prostate-specific result.","The survival analysis is single-centre with no external validation, a limitation the paper itself acknowledges, so the six selected clusters should be treated as hypothesis-generating until replicated in an independent cohort.","Comparing HistoEncoder not only to ImageNet but also to pan-cancer histopathology models would clarify whether the benefit is prostate-specific or merely any histology beats natural images."],"forward_implications":["Prostate cancer classification models can be built with a few hundred annotated tiles, matching or exceeding ImageNet-initialized models trained on the full PANDA datasets.","Large unlabelled slide collections can be annotated automatically: clustering HistoEncoder features labels more than two million PANDA tiles with over 90 percent purity after reviewing only 256 cluster centres.","Adding histology cluster fractions to Gleason grade, CAPRA-S, or MSKCC-S improves concordance, time-dependent AUC, and net benefit for prostate-cancer-specific death prediction.","Fine-tuning HistoEncoder for clinical tasks is feasible on a single laptop, lowering the computational barrier for building clinical software tools.","The authors state that evaluation against tissue-agnostic pathology foundation models is a necessary next step, which will isolate the value of organ-specific pre-training."],"supporting_citations":[{"why":"Supplies the DINO self-distillation objective used for self-supervised pre-training of the XCiT models.","marker":"[31]"},{"why":"Defines the XCiT backbone whose linear token complexity makes high-resolution tile pre-training efficient.","marker":"[42]"},{"why":"Provides the prostate cancer classifier used to select and balance the 48M-tile pre-training set (HelsinkiProstate).","marker":"[13]"},{"why":"Supplies the Karolinska and Radboud biopsy datasets from the PANDA challenge, the main cross-centre evaluation sets.","marker":"[39]"},{"why":"Supplies the PESO region-of-interest dataset used for fine-tuning and evaluation of cancer classification.","marker":"[40]"},{"why":"Provides the ImageNet dataset used to pre-train the natural-s and natural-m DINO baselines that prostate models are compared against.","marker":"[43]"},{"why":"Preprocessing tool that removes non-tissue tiles from slides before pre-training and evaluation.","marker":"[44]"},{"why":"Defines the CAPRA-S nomogram that is augmented with HistoEncoder cluster fractions in the survival models.","marker":"[46]"},{"why":"Defines the MSKCC-S 15-year nomogram used as a clinical baseline and augmented with cluster fractions.","marker":"[47]"}],"fun_headline_variants":["48M prostate tiles beat ImageNet for cancer AI","Prostate features outperform ImageNet with no fine-tuning","HistoEncoder: 1000x less data still tops ImageNet","Prostate pre-training: better AI from a thousandth of data","Prostate-specific pre-training beats generic ImageNet"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported advantage over ImageNet rests on the assumption that the cancer-score-based selection of pre-training tiles does not leak cancer label information into the learned features; if the selection filter is itself a weak cancer classifier, the comparison is not purely about domain-specific pre-training.","fun_headline_variants_meta":{"raw":{"variants":["48M prostate tiles beat ImageNet for cancer AI","Prostate features outperform ImageNet with no fine-tuning","HistoEncoder: 1000x less data still tops ImageNet","Prostate pre-training: better AI from a thousandth of data","Prostate-specific pre-training beats generic ImageNet"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000747,"raw_usage":{"total_tokens":3316,"prompt_tokens":922,"completion_tokens":2394,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":538,"completion_tokens_details":{"reasoning_tokens":2312}},"tokens_in":538,"tokens_out":2394,"duration_ms":32290,"temperature":1.0,"reasoning_tokens":2312,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:30:10.246368+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the same 48M-tile pool, drop the $\\hat{y} > 0.2$ filter, randomly sample 48M tiles from all tissue, and re-run the DINO/XCiT pre-training with identical hyperparameters; then compare KNN AUROC on Karolinska and Radboud. If the unfiltered model's advantage over the ImageNet baseline largely disappears, the tile-selection filter, not histology-specific learning, drove the reported result.","supporting_citations":[{"cited_title":"Emerging properties in self-supervised vision transformers, 2021","cited_arxiv_id":null,"evidence_quote":"Supplies the DINO self-distillation objective used for self-supervised pre-training of the XCiT models."},{"cited_title":"Xcit: Cross-covariance image transformers, 2021","cited_arxiv_id":null,"evidence_quote":"Defines the XCiT backbone whose linear token complexity makes high-resolution tile pre-training efficient."},{"cited_title":"Artificial intelligence for diagnosis and gleason grading of prostate cancer: the panda challenge","cited_arxiv_id":null,"evidence_quote":"Supplies the Karolinska and Radboud biopsy datasets from the PANDA challenge, the main cross-centre evaluation sets."},{"cited_title":"Peso: Prostate epithelium segmentation Preprint – HistoEncoder: a digital pathology foundation model for prostate cancer 10 on h&e-stained prostatectomy whole slide images, 2018","cited_arxiv_id":null,"evidence_quote":"Supplies the PESO region-of-interest dataset used for fine-tuning and evaluation of cancer classification."},{"cited_title":"Histoprep: Prepro- cessing large medical images for machine learning made easy! https: //github.com/jopo666/HistoPrep, 2022","cited_arxiv_id":null,"evidence_quote":"Preprocessing tool that removes non-tissue tiles from slides before pre-training and evaluation."},{"cited_title":"The capra-s score: a straightforward tool for improved prediction of outcomes after radical prostatectomy.Cancer, 117(22):5039–5046, 2011","cited_arxiv_id":null,"evidence_quote":"Defines the CAPRA-S nomogram that is augmented with HistoEncoder cluster fractions in the survival models."},{"cited_title":"https://www.mskcc.org/nomograms/prostate/ post_op/coefficients","cited_arxiv_id":null,"evidence_quote":"Defines the MSKCC-S 15-year nomogram used as a clinical baseline and augmented with cluster fractions."}],"review_version":1}