{"id":"dc56334e-ee68-4cb0-9d07-ab7a34131246","arxiv_id":"2501.14948","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"HECLIP imputes spatial gene expression from H&E histology images using an image-centric contrastive loss and nearest-neighbor retrieval of reference profiles.","lead":"HECLIP is a deep learning method that predicts gene activity in tissue by comparing H&E-stained microscope images to stored samples and averaging the gene expression of the most similar ones. This could make spatial gene expression profiling much cheaper if the predictions are reliable.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Same-block evaluation inflates retrieval-based imputation; no independent-sample test supports the superiority claim.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing concern: the held-out slice is from the same tissue block as the reference slices, so the benchmark measures within-block retrieval rather than generalization to independent samples. My read agrees with this assessment and does not move the conditional verdict. The paper's central claim is plausible only if the image-centric contrastive loss induces a histology-to-expression mapping that transfers across blocks or individuals, but no such experiment is reported. The concrete leave-one-block-out test would settle this: if HECLIP's advantage disappears or it fails to beat a trivial mean-expression baseline, then the reported 'consistent outperformance' and 'accurate prediction solely from images' are unsupported. Until that test is run, the conditional verdict remains appropriate. I considered other potential concerns, such as unreported hyperparameters and the use of reference expression labels during inference, but the same-block protocol is the most direct threat to the central claim because it undermines the external validity of every reported metric.","tokens_in":11194,"tokens_out":5379,"duration_ms":55992,"concrete_test":"Run a leave-one-block-out evaluation on spatialLIBD: train on all slices from two subjects and test on the remaining subject's slice, splitting by subject rather than by sample continuity; similarly test across donors in GSE240429/GSE245620 if donor identity is available. Report SSIM, RMSE, and Hit@1 for HECLIP, BLEEP, CLIP, and a reference-mean baseline. If HECLIP's margin over baselines collapses or it fails to beat the mean-expression baseline on held-out blocks, the reported superiority is an artifact of the same-block retrieval protocol.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that HECLIP 'consistently outperforms other methods across all datasets' and 'enables accurate prediction of gene expression profiles solely from histological images' rests on an evaluation protocol where the query slice and the reference slices are consecutive sections or adjacent replicates from the same tissue block (§2.1, Table 1). Because adjacent Visium sections share tissue architecture and gene-expression autocorrelation, a retrieval-based model can achieve high matching accuracy by exploiting near-duplicate morphology rather than by learning a generalizable histology-to-expression mapping. The inference stage (§2.5) averages gene-expression profiles of top-K reference patches, and with all reference patches coming from the same block, the task becomes one of locating the same spatial region in a nearly identical section. The paper includes no leave-one-block-out or cross-subject evaluation, so the reported advantage over BLEEP, CLIP, HisToGene, and ST-Net may not transfer to independent tissue samples. The near-zero absolute SSIM medians (0.007–0.029) further undermine the 'accurate prediction' component of the claim, even within the same-block protocol. This is load-bearing because both the relative superiority and the general predictive claim are benchmarked only in this shared-tissue regime.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces HECLIP, a contrastive-learning framework for imputing spatial transcriptomics from H&E histology images. The method uses a ResNet-50 image encoder and a linear spot encoder trained with an image-centric variant of the CLIP loss; at inference, each query patch retrieves the top-K most similar reference patches in the learned image-embedding space and averages their gene-expression profiles. The authors evaluate on three public Visium datasets (liver GSE240429 and GSE245620, and spatialLIBD split into two sub-datasets) against BLEEP, CLIP, HisToGene, and ST-Net, reporting RMSE, SSIM, and Hit@T. They claim that HECLIP consistently outperforms the baselines across all datasets and scenarios, and that it enables accurate prediction of gene expression profiles solely from histology images.","tokens_in":11444,"tokens_out":5527,"duration_ms":49181,"significance":"If the claims were established for independent tissue samples, HECLIP would be a useful and simple contribution: the asymmetric loss is easy to implement, the inference procedure is transparent, the code is publicly available, and the paper benchmarks several public datasets with multiple baselines. However, the evaluation protocol and the very small absolute effect sizes mean that the current evidence does not support the central claims of consistent superiority and of accurate prediction solely from histology. With a more rigorous evaluation, including cross-block or cross-subject tests and trivial baselines, the work could be valuable to the computational pathology community.","major_comments":[{"comment":"The evaluation is performed entirely within the same tissue block or donor. GSE240429 and GSE245620 consist of consecutive thick sections of the same liver block; spatialLIBD is split into adjacent replicates from the same subjects, and Table 1 shows that one slice is used as query while the remaining slices from the same block form the reference. Because inference (Section 2.5) averages the expression profiles of top-K reference patches, a retrieval model can achieve high scores by matching nearly identical morphology between adjacent sections rather than by learning a generalizable histology-to-expression mapping. The paper does not include leave-one-block-out or cross-subject experiments, so the claimed advantage over BLEEP, CLIP, HisToGene, and ST-Net is not established for independent samples. Please add an evaluation where query and reference come from different tissue blocks or subjects, or explicitly demonstrate that within-block performance transfers to cross-block performance.","section":"Section 2.1, Table 1, Section 2.5"},{"comment":"Reported median SSIM values are 0.007-0.029, and the largest ablation gain in Table 3 is about 0.002 in median SSIM. Such near-zero SSIM values are hard to reconcile with the abstract's claim of 'accurate gene expression profiles.' No trivial baselines are reported, such as predicting the mean expression vector, random retrieval from the reference set, or nearest-neighbor retrieval in raw pixel space with no learned embedding. Without these, it is unclear whether the learned embedding contributes anything beyond the retrieval setup. Please report these baselines and include per-gene R^2 or correlation, and report confidence intervals or significance tests for all comparative claims.","section":"Section 3.1, Table 3, Eq. (5)"},{"comment":"Because predictions are averages of K reference expression profiles, they are convex combinations of training spots; the paper does not analyze sensitivity to K or to the composition of the reference set. This matters for the claim that HECLIP 'infers' expression from images, since the method may largely be performing memorized nearest-neighbor lookup within the same block. Please report retrieval accuracy and Hit@T as a function of K, and compare against an untrained pixel-space retrieval baseline to isolate the contribution of the contrastive loss.","section":"Section 2.5 and inference procedure"}],"minor_comments":[{"comment":"For GSE240429 HEG, HECLIP and BLEEP have identical Hit@T values (1, 1, 0.9978); the text should say 'at least tied for the best' rather than 'consistently achieved the highest accuracy.'","section":"Table 2 and Section 3.1"},{"comment":"The list of patch vertices repeats '(x + 128, y - 128)' and contains five entries instead of four; please correct the square coordinates.","section":"Section 2.1"},{"comment":"As written, h2 = Dropout(W2 h1 + b2) and hp = LayerNorm(h1 + h2), so the projection head is not described consistently with a standard residual MLP; please clarify the intended architecture.","section":"Equations (1)-(3)"},{"comment":"The image-centric loss uses soft targets derived from image-image similarity; please clarify how the temperature is chosen and why the cross-entropy direction is as written, since temperature is a free parameter not reported in the main text.","section":"Algorithm 1 and Section 2.4"},{"comment":"The claims about HENMT1, METTL11B, PPIAL4A, and EIPR1 would be stronger with quantitative evidence that HECLIP predicts these genes more accurately than the baselines; currently the section only lists known biology.","section":"Section 3.3"},{"comment":"Please state the exact K, temperature, embedding dimension d_o, learning rate, and number of epochs used for the main results; these are only referenced to 'Supplementary Table 2' in Section 3.4.","section":"Main text, hyperparameters"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a plausible but incremental CLIP variant for predicting spatial expression from H&E, and the reported edge over baselines mostly reflects an easy same-block retrieval setup, not a generalizable method.\n\nThe genuinely new bit is the asymmetric image-centric contrastive loss: instead of the symmetric CLIP objective, they drop the spot-to-image direction and use image-image similarities as soft targets. That is a sensible modification for a unidirectional prediction task, and the retrieval-only inference is a natural fit. The code is available, the datasets are public, and the paper runs ablations for the loss and data augmentation. Those are real virtues.\n\nBut the evaluation does not support the strength of the claims. Test and reference slices are consecutive sections or adjacent replicates from the same donor (Section 2.1, Table 1). Retrieval from near-duplicate morphology can look great without learning anything about histology-to-expression mapping. There is no leave-one-block-out or cross-subject test, so I do not think we know whether HECLIP transfers to independent samples. The absolute SSIM medians are 0.007–0.029, which are close to zero, so “accurate prediction” is a stretch even within the protocol. And there is no trivial baseline—predicting the mean expression or a random patch from the same block—so we cannot tell how much of the performance is just spatial autocorrelation. The differences between HECLIP and the original CLIP loss are small in the ablation table, and several hyperparameters (K, temperature, embedding dimension) are not reported.\n\nTo be fair, the paper is not circular in the mathematical sense: the loss is a legitimate asymmetric contrastive objective and the held-out slice is a real test. The flaw is external validity, not internal contradiction. The Hit@T numbers are more believable, though they too are within the same-block setting.\n\nWho is this for? Someone working on histology-based expression imputation will want to know about HECLIP as a baseline variant, but the superiority claim should be treated with caution until an independent-sample evaluation appears. I would send it to peer review, with the strong request that the authors add a cross-sample evaluation and a trivial baseline; without those, the headline claim is not convincing.","headline":"A modest CLIP variant for H&E-to-expression imputation whose same-block retrieval evaluation inflates the reported gains.","tokens_in":11942,"tokens_out":2050,"would_cite":false,"duration_ms":17645,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"HECLIP claims that an image-centric contrastive loss lets it impute gene-expression profiles directly from H&E-stained histology images, outperforming CLIP, BLEEP, HisToGene, and ST-Net on all tested datasets.","keywords":["contrastive learning","gene expression imputation","histology image","spatial transcriptomics","image-centric loss","H&E staining","multimodal retrieval","top-K retrieval"],"falsifier":"Run the same benchmarking with query and reference slices taken from different donors or different tissue blocks; if HECLIP's reported advantage over CLIP and BLEEP in SSIM, RMSE, and Hit@T disappears or reverses, then the shared-tissue context of consecutive sections is carrying the result rather than the image-centric loss.","tokens_in":11031,"feed_emoji":"🧬","tokens_out":5652,"duration_ms":66920,"temperature":0.7,"pith_summary":"HECLIP is a deep-learning framework that imputes spatial gene-expression profiles from H&E-stained histology images, without running spatial transcriptomics. The paper's central claim is that an image-centric contrastive loss—built from image-to-image similarity instead of symmetric image-to-gene similarity—makes the image encoder retrieve the correct reference spots more accurately than existing methods. Across liver and brain datasets, HECLIP reports lower RMSE, higher SSIM, and higher Hit@T than CLIP, BLEEP, HisToGene, and ST-Net under both highly-variable and highly-expressed gene selections. The motivation is practical: if the claim holds, routine H&E slides can provide molecular readouts at a fraction of the cost of sequencing.","feed_headline":"HECLIP reads gene expression straight from tissue images","feed_subtitle":"An image-focused contrastive loss beats CLIP, BLEEP, HisToGene and ST-Net on liver and brain datasets.","key_machinery":"The central object is the image-centric contrastive loss (Algorithm 1). Standard CLIP loss balances image-to-spot and spot-to-image directions; HECLIP instead computes logits from spot-to-image dot products but derives soft targets from image-to-image similarities, then applies cross-entropy to the transposed logits against those targets while omitting the spot-based loss. This forces the image encoder to organize embedding space according to visual similarity that predicts molecular profiles. Inference is a top-K retrieval: fixed image-encoder embeddings of reference patches are ranked by dot-product similarity against each query patch, and the average of the top-K reference gene-expression profiles is the imputed output.","core_discovery":"HECLIP is a deep-learning framework that predicts spatial gene expression from H&E-stained histology patches. Its central claim is that a unidirectional, image-centric contrastive loss—one that builds soft targets from image-to-image similarity and excludes the spot-to-image loss term—produces image embeddings that retrieve the correct reference spots more accurately than the symmetric contrastive loss used by CLIP or the bi-modal embeddings of BLEEP. At inference, HECLIP computes dot-product similarity between query and reference image embeddings, selects the top-K most similar reference spots, and averages their gene-expression profiles to impute the query's transcriptome. The paper reports that across GSE240429, GSE245620, and the two spatialLIBD subsets, HECLIP achieves lower RMSE, higher SSIM, and higher Hit@T values than CLIP, BLEEP, HisToGene, and ST-Net in both highly-variable and highly-expressed gene settings, and that ablation studies trace the gain to the image-centric loss and to paired data augmentation.","pith_inferences":["The top-K averaging mechanism means the prediction is bounded by the quality of the reference set: a query patch with no morphologically similar reference will be assigned the average of its nearest neighbors, which the paper does not quantify.","The reported gains may not transfer to cross-donor or cross-batch settings, since every dataset pairs the query with consecutive or adjacent slices from the same donor; a multi-donor benchmark would be the natural next test.","The same image-centric loss could be applied to other one-directional biomedical mappings, such as radiology images to genomic features or histology to DNA methylation, where symmetric contrastive alignments are a poor fit.","The paper does not report the value of K or a sensitivity analysis for it; retrieval accuracy likely depends on K, and tuning it per dataset could change the rankings."],"forward_implications":["Gene-expression profiles could be imputed from routine H&E slides without spatial transcriptomics assays, lowering cost and turnaround in clinical workflows.","The image-centric loss is a general recipe for one-way multimodal mapping: any task that predicts a molecular or phenotypic readout from images can drop the reverse loss and use same-modality similarities as soft targets.","Better Hit@1 and Hit@2 accuracy on top-expressed genes would help prioritize candidate biomarkers and therapeutic targets directly from histology.","Because inference is retrieval-plus-averaging, the model's output stays interpretable: each imputed spot can be traced back to the reference spots that contributed to it."],"supporting_citations":[{"why":"Supplies the CLIP contrastive-learning baseline and the symmetric image-text alignment that HECLIP modifies.","marker":"[11]"},{"why":"Provides ST-Net, a deep-learning baseline that predicts local gene expression directly from H&E images.","marker":"[12]"},{"why":"Provides BLEEP, a bi-modal contrastive baseline that maps image and gene-expression data into a shared embedding space.","marker":"[13]"},{"why":"Provides HisToGene, a Vision Transformer baseline for gene-expression prediction from histology images.","marker":"[14]"},{"why":"Supplies the GSE240429 and GSE245620 liver datasets used for benchmarking.","marker":"[17]"},{"why":"Supplies the spatialLIBD dorsolateral prefrontal cortex datasets used for benchmarking.","marker":"[18]"},{"why":"Defines the SSIM and RMSE evaluation metrics used to compare all methods.","marker":"[23]"},{"why":"Inspires the Hit@T metric for measuring top-gene retrieval accuracy in the evaluation.","marker":"[24]"}],"fun_headline_variants":["HECLIP maps gene expression from tissue stains","Image-only contrastive loss beats CLIP for expression imputation","Histology alone predicts spatial transcriptomics with HECLIP","HECLIP: spot-on gene prediction from H&E slides","No sequencing needed: HECLIP imputes genes from images"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation protocol assumes that a held-out slice from the same tissue block, with the remaining consecutive or adjacent slices as the reference set, reflects real-world generalization to new tissue samples.","fun_headline_variants_meta":{"raw":{"variants":["HECLIP maps gene expression from tissue stains","Image-only contrastive loss beats CLIP for expression imputation","Histology alone predicts spatial transcriptomics with HECLIP","HECLIP: spot-on gene prediction from H&E slides","No sequencing needed: HECLIP imputes genes from images"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000271,"raw_usage":{"total_tokens":1673,"prompt_tokens":1034,"completion_tokens":639,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":650,"completion_tokens_details":{"reasoning_tokens":552}},"tokens_in":650,"tokens_out":639,"duration_ms":6185,"temperature":1.0,"reasoning_tokens":552,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:45:46.963704+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same benchmarking with query and reference slices taken from different donors or different tissue blocks; if HECLIP's reported advantage over CLIP and BLEEP in SSIM, RMSE, and Hit@T disappears or reverses, then the shared-tissue context of consecutive sections is carrying the result rather than the image-centric loss.","supporting_citations":[{"cited_title":"Integrating spatial gene expression and breast tumour morphology via deep learning","cited_arxiv_id":null,"evidence_quote":"Provides ST-Net, a deep-learning baseline that predicts local gene expression directly from H&E images."},{"cited_title":"Spatially resolved gene expression prediction from histol- ogy images via bi-modal contrastive learning","cited_arxiv_id":null,"evidence_quote":"Provides BLEEP, a bi-modal contrastive baseline that maps image and gene-expression data into a shared embedding space."},{"cited_title":"Leveraging information in spatial transcriptomics to predict super-resolution gene expression from histology images in tumors","cited_arxiv_id":null,"evidence_quote":"Provides HisToGene, a Vision Transformer baseline for gene-expression prediction from histology images."},{"cited_title":"Single-cell, single-nucleus, and spatial transcriptomics characterization of the immunological landscape in the healthy and psc human liver","cited_arxiv_id":null,"evidence_quote":"Supplies the GSE240429 and GSE245620 liver datasets used for benchmarking."},{"cited_title":"Transcriptome-scale spatial gene expression in the human dorsolateral prefrontal cortex","cited_arxiv_id":null,"evidence_quote":"Supplies the spatialLIBD dorsolateral prefrontal cortex datasets used for benchmarking."},{"cited_title":"Benchmarking spatial and single- cell transcriptomics integration methods for transcript distribution prediction and cell type deconvolution","cited_arxiv_id":null,"evidence_quote":"Defines the SSIM and RMSE evaluation metrics used to compare all methods."},{"cited_title":"Gudn: A novel guide network with label reinforcement strategy for extreme multi-label text classification","cited_arxiv_id":null,"evidence_quote":"Inspires the Hit@T metric for measuring top-gene retrieval accuracy in the evaluation."}],"review_version":1}