REVIEW 4 major objections 5 minor 16 references
GeneQuery: A General QA-based Framework for Spatial Gene Expression Predictions from Histology Images
T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read GeneQuery recasts gene-expression prediction as a question-answering task, letting one model predict known and unseen genes from H&E slides.
desk verdict Useful attempt at text-conditioned zero-shot gene prediction; evaluation needs patient-level splits and a no-query control before the headline claims hold. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the gene random variable $g$, which turns each prediction into a conditional distribution $P(y \mid x, g)$ rather than a per-gene regressor $P(y_j \mid x)$. This is operationalized by a gene encoder (clinical BERT) that converts gene metadata into a query vector and a fusion module that cross-attends image and gene features; because the same image encoder and regressor handle all genes, the only thing that changes between genes is the query. The two architecture variants choose which modality becomes the sequence dimension: spot-aware GeneQuery sequences spots and adds the queried gene's embedding to every spot, while gene-aware GeneQuery sequences genes and adds the spot embedding to every gene.
What would settle it
Hold out entire patients rather than random slides when splitting HER2+ (8 patients, 36 slides) and check whether unseen-gene Pearson correlation survives; or shuffle the gene metadata among genes at test time. If predicted expression maps barely change under shuffled text, the model is not using the text query and the claimed generalization mechanism is absent.
Extended reading notes
Core claim
The central claim is that introducing a gene random variable and feeding gene metadata as a query allows a single model to estimate a conditional distribution $P(y \mid x, g)$ and thereby generalize across both spots and genes. Concretely, GeneQuery encodes each 224-by-224 spot with ResNet50, encodes gene metadata with clinical BERT, projects both to a common dimension, fuses them with a simple addition followed by two transformer blocks, and regresses expression values with mean-squared error. Two variants are proposed: spot-aware GeneQuery treats all spot images of a slide as a sequence and adds the queried gene's feature to each spot, while gene-aware GeneQuery treats the gene list as a sequence and adds the spot feature to each gene. The paper reports that this method beats STNet, HistoGene, and BLEEP on known-gene Pearson correlation and gives competitive predictions for genes held out from training, along with stronger transfer between datasets of the same tissue type.
Load-bearing premise
The argument stands on the premise that a short text description of a gene carries enough information about that gene's spatial expression pattern that, when fused with image features, it can determine where and how strongly the gene is expressed; if the text is just a name or a description with no morphology-expression link, unseen-gene generalization would have no mechanism and the model would only memorize gene identities seen in training.
Editorial extensions
If this is right
- One trained GeneQuery model can produce expression predictions for any gene in a library, so adding a new gene to the panel no longer requires training a separate per-gene head.
- Unseen-gene performance improves as the number of seen genes grows, suggesting the model learns transferable gene-morphology associations from the training gene set.
- Richer metadata, such as GPT-4-generated descriptions, improves predictions, implying that gene text quality is a controllable input to accuracy.
- The learned spot representations separate tissue structures such as invasive cancer, connective tissue, and adipose tissue, so the same model may double as a tissue-segmentation tool.
- Within-tissue transfer (HER2+ to HBD and back) exceeds cross-tissue transfer, indicating the query mechanism generalizes most reliably when tissue context is similar.
Reading between the lines
- A direct test of the mechanism would replace each gene's real description with another gene's description at evaluation; if predicted expression maps barely change, the model is not using text and unseen-gene gains must come from shared image features.
- The same query-conditioned design could accept other text modalities, including pathways, protein domains, disease annotations, or free-form questions about a region, because anything that correlates with expression can enter through the gene encoder.
- If the text-query mechanism holds, the architecture points toward foundation-model-style scaling: keep one image encoder and one gene encoder and treat the gene library as a prompt, replacing task-specific regression heads in other spatial-omics prediction problems.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes GeneQuery, a framework that reformulates prediction of spatial gene expression from H&E whole-slide images as a question-answering task: a spot image is the context, gene metadata is the query, and the model regresses the queried gene's expression. Two fusion architectures are presented, spot-aware GeneQuery and gene-aware GeneQuery, and the model is trained with an MSE loss over all spots and genes. The central empirical claims are (i) that this shared-weight formulation outperforms STNet, HistoGene, and BLEEP on known genes, and (ii) that it can predict genes never seen during training. Experiments are reported on GSE240429 (liver), HER2+ breast, and HBD breast datasets, together with transfer-learning results and an ablation using GPT-4-generated gene descriptions.
Significance. If the claims hold, the paper makes a useful contribution: replacing per-gene output heads with a model that conditions on gene identity would be a practical advance for spatial transcriptomics and would open up zero-shot prediction for new genes from histology alone. The paper's strengths are that it evaluates on multiple datasets, holds out entire genes for the unseen-gene experiments, releases code, and includes transfer experiments that are concrete and falsifiable. However, the current evidence is not yet sufficient: the evaluation lacks patient-level split control, an ablation showing that the gene query rather than a generic image trend drives unseen-gene predictions, and error-bounded comparisons for several headline claims.
major comments (4)
- [§4.1–4.2, Table 2] The cross-validation splits are by whole-slide image, not by patient. The HER2+ set is described as collected from 8 patients with 36 WSIs, so a random 5-fold WSI split will almost certainly place slides from the same patient in both training and test. Same-patient slides share patient-specific morphology and expression, which can inflate the reported PCCs through memorization rather than generalizable prediction. Please use patient-stratified folds (all slides from one patient in the same fold) and report patient-level summary statistics. The manuscript should also state the patient/slide mapping for HBD, since the same concern applies if it contains repeated patients.
- [§3.2, Eqs. (3)–(7); §4.4, Table 3] The unseen-gene result is the paper's most distinctive claim, but Table 3 contains no gene-agnostic control. The load-bearing premise, introduced in Eqs. (3)–(7), is that the gene text embedding, when fused with image features, determines a gene-specific spatial expression pattern for genes never seen during training. To test this, the authors should evaluate the same architecture with a constant or empty gene embedding, or with randomly permuted gene-metadata pairs. If such a control reaches similar unseen-gene PCC, the predictions are driven by a generic spatial-expression trend shared by HEG/HVG genes rather than by the gene query. Table 3 itself contains a warning sign: on GSE at 20% seen genes, unseen-gene HEG PCC (0.137) is larger than seen-gene HEG PCC (0.117), which is difficult to explain if the query is informative.
- [§4.3, Table 2] Several headline claims of consistent superiority are not supported by the reported uncertainty. For example, on HER2+ HVG, GeneQuery_gene (0.318±0.025) and BLEEP (0.322±0.100) have strongly overlapping standard deviations, and on HBD ALL, GeneQuery_gene (0.061±0.007) is numerically below STNet (0.073±0.011). Please report paired significance tests across folds or across genes for every comparison, and revise the blanket statement in §4.3 that GeneQuery achieves the best performance across all datasets and settings.
- [§4.6, Table 5] The GPT-4 enhancement results are reported on a single validation fold with no standard deviations or significance tests, so the conclusion that GPT-4 metadata improves GeneQuery by 3.6% and 1.8% is not reliable. In the HER2+ rows of Table 5, spot-aware GeneQuery with GPT-4 is actually lower on all three metrics (0.342→0.336 for HEG, 0.347→0.341 for HVG, and 0.183→0.164 for ALL). Please run the ablation across the same multiple folds as the main results and report paired differences with uncertainty.
minor comments (5)
- [§4.4, Table 3] The table header contains the typo 'Unseeen'; it should read 'Unseen'.
- [§5] The section heading 'Conlusion' should be 'Conclusion'.
- [§4.5, Table 4] Several entries in Table 4 appear fused in the text, for example '0.1170.071' and '0.0900.041'; the column separation should be fixed so each mean and standard deviation is readable.
- [§3.1, Eqs. (1)–(2)] Equation (1) writes P(y_j|x_i) for a real-valued quantity, and Eq. (2) presents an MLE-style objective that is then used as an MSE loss without an explicit likelihood or noise model; the formulation should be cleaned up or the claims about 'estimating the gene distribution' should be softened.
- [References, [14]] The clinical BERT encoder is cited to a reinforcement-learning diabetes trial; the reference appears to be incorrect and should be replaced with the actual clinical BERT publication.
Circularity Check
No significant circularity: held-out unseen-gene evaluation is externally constructed, and the gene-query framing does no derivational work.
full rationale
GeneQuery's Eq. (2) is an ordinary conditional regression objective: max E P(y|x,g) = min Σ L(y_ij, f_φ(x_i, g_j)), and Eq. (8) is MSE on all spot-gene pairs. The 'gene random variable' g is just the gene index/conditioning variable; no quantity is defined in terms of the target expression values and then rediscovered as a prediction. The unseen-gene experiments in Table 3 hold out entire genes from training; the gene query is the clinical BERT description or GPT-4 text generated from the gene name only ('The brief definition of gene GENE_NAME is'), so expression targets are not used to construct the query. This is a genuine external evaluation, not a fitted input renamed as a prediction. The comparisons to STNet, HistoGene, and BLEEP are independent baselines, and the paper's references to prior work are third-party rather than load-bearing self-citations. The QA framing and 'random variable' language are conceptual reframings of conditional regression, but the paper does not derive a mathematical prediction from them; the empirical claims stand or fall on held-out correlation. Limitations (low absolute PCC, one-fold GPT-4 validation, transfer performance below STNet/BLEEP) affect evidence strength, not circularity. No circular step can be exhibited by reduction, so score 0.
Assumptions & free parameters
free parameters (6)
- highly variable gene count =
1000 per slide; 3467 union (GSE), 785 (HER2+), 723 (HBD)
- minimum spot expression filter =
1000 spots
- transformer depth L =
2
- fusion dimension =
256
- epochs and batch size =
100 epochs, batch 100
- input patch size =
224x224 pixels
assumptions (5)
- domain assumption H&E histology images contain learnable signal about spatial gene expression
- domain assumption Gene metadata text is a sufficient conditioning signal for unseen-gene prediction
- domain assumption Pretrained ResNet50 and clinical BERT features transfer to this task
- domain assumption WSI-level random cross-validation provides unbiased performance estimates
- standard math MSE loss is a valid MLE objective for the conditional model
Cite this review
Pith. "Pith review of GeneQuery: A General QA-based Framework for Spatial Gene Expression Predictions from Histology Images." pith.science (2026). https://pith.science/paper/VOK3RZZZ
@misc{pith2026241118391,
author = {Pith},
title = {Pith review of: GeneQuery: A General QA-based Framework for Spatial Gene Expression Predictions from Histology Images},
year = {2026},
howpublished = {\url{https://pith.science/paper/VOK3RZZZ}},
note = {Machine review of arXiv:2411.18391}
}
read the original abstract
Gene expression profiling provides profound insights into molecular mechanisms, but its time-consuming and costly nature often presents significant challenges. In contrast, whole-slide hematoxylin and eosin (H&E) stained histological images are readily accessible and allow for detailed examinations of tissue structure and composition at the microscopic level. Recent advancements have utilized these histological images to predict spatially resolved gene expression profiles. However, state-of-the-art works treat gene expression prediction as a multi-output regression problem, where each gene is learned independently with its own weights, failing to capture the shared dependencies and co-expression patterns between genes. Besides, existing works can only predict gene expression values for genes seen during training, limiting their ability to generalize to new, unseen genes. To address the above limitations, this paper presents GeneQuery, which aims to solve this gene expression prediction task in a question-answering (QA) manner for better generality and flexibility. Specifically, GeneQuery takes gene-related texts as queries and whole-slide images as contexts and then predicts the queried gene expression values. With such a transformation, GeneQuery can implicitly estimate the gene distribution by introducing the gene random variable. Besides, the proposed GeneQuery consists of two architecture implementations, i.e., spot-aware GeneQuery for capturing patterns between images and gene-aware GeneQuery for capturing patterns between genes. Comprehensive experiments on spatial transcriptomics datasets show that the proposed GeneQuery outperforms existing state-of-the-art methods on known and unseen genes. More results also demonstrate that GeneQuery can potentially analyze the tissue structure.
Figures
Reference graph
Works this paper leans on
-
[1]
Visualization and analysis of gene expression in tissue sections by spatial transcriptomics
Patrik L Ståhl, Fredrik Salmén, Sanja Vickovic, Anna Lundmark, José Fernández Navarro, Jens Magnusson, Stefania Giacomello, Michaela Asp, Jakub O Westholm, Mikael Huss, et al. Visualization and analysis of gene expression in tissue sections by spatial transcriptomics. Science, 353(6294):78–82, 2016
2016
-
[2]
Spatially resolved, highly multiplexed rna profiling in single cells
Kok Hao Chen, Alistair N Boettiger, Jeffrey R Moffitt, Siyuan Wang, and Xiaowei Zhuang. Spatially resolved, highly multiplexed rna profiling in single cells. Science, 348(6233):aaa6090, 2015
2015
-
[3]
Transcriptome-scale super-resolved imaging in tissues by rna seqfish+
Chee-Huat Linus Eng, Michael Lawson, Qian Zhu, Ruben Dries, Noushin Koulena, Yodai Takei, Jina Yun, Christopher Cronin, Christoph Karp, Guo-Cheng Yuan, et al. Transcriptome-scale super-resolved imaging in tissues by rna seqfish+. Nature, 568(7751):235–239, 2019
2019
-
[4]
Three-dimensional intact-tissue sequencing of single-cell transcriptional states
Xiao Wang, William E Allen, Matthew A Wright, Emily L Sylwestrak, Nikolay Samusik, Sam Vesuna, Kathryn Evans, Cindy Liu, Charu Ramakrishnan, Jia Liu, et al. Three-dimensional intact-tissue sequencing of single-cell transcriptional states. Science, 361(6400):eaat5691, 2018
2018
-
[5]
Spatial organization of the somatosensory cortex revealed by osmfish
Simone Codeluppi, Lars E Borm, Amit Zeisel, Gioele La Manno, Josina A van Lunteren, Camilla I Svensson, and Sten Linnarsson. Spatial organization of the somatosensory cortex revealed by osmfish. Nature Methods, 15(11):932–935, 2018
work page 2018
-
[6]
Expansion sequencing: Spatially precise in situ transcriptomics in intact biological systems
Shahar Alon, Daniel R Goodwin, Anubhav Sinha, Asmamaw T Wassie, Fei Chen, Evan R Daugharthy, Yosuke Bando, Atsushi Kajita, Andrew G Xue, Karl Marrett, et al. Expansion sequencing: Spatially precise in situ transcriptomics in intact biological systems. Science, 371(6528):eaax2656, 2021
work page 2021
-
[7]
Integrating spatial gene expression and breast tumour morphology via deep learning
Bryan He, Ludvig Bergenstråhle, Linnea Stenbeck, Abubakar Abid, Alma Andersson, Åke Borg, Jonas Maaskola, Joakim Lundeberg, and James Zou. Integrating spatial gene expression and breast tumour morphology via deep learning. Nature Biomedical Engineering, 4(8):827–834, 2020
work page 2020
-
[8]
Leveraging information in spatial transcriptomics to predict super-resolution gene expression from histology images in tumors
Minxing Pang, Kenong Su, and Mingyao Li. Leveraging information in spatial transcriptomics to predict super-resolution gene expression from histology images in tumors. BioRxiv, pages 2021–11, 2021
2021
Show all 16 references
-
[9]
Ronald Xie, Kuan Pang, Sai Chung, Catia Perciani, Sonya MacParland, Bo Wang, and Gary D. Bader. Spatially resolved gene expression prediction from histology images via bi-modal contrastive learning. In Proceedings of the Advances in Neural Information Processing Systems 36: An...
2023
-
[10]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV , USA, June 27-30, 2016, pages 770–778. IEEE Computer Society, 2016. 10 Running Tit...
2016
-
[11]
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...
2021
-
[12]
A deep learning model to predict rna-seq expression of tumours from whole slide images
Benoît Schmauch, Alberto Romagnoni, Elodie Pronier, Charlie Saillard, Pascale Maillé, Julien Calderaro, Aurélie Kamoun, Meriem Sefta, Sylvain Toldo, Mikhail Zaslavskiy, et al. A deep learning model to predict rna-seq expression of tumours from whole slide images. Nature Commun...
2020
-
[13]
Detecting gene–gene interactions that underlie human diseases
Heather J Cordell. Detecting gene–gene interactions that underlie human diseases. Nature Reviews Genetics, 10(6):392–404, 2009
2009
-
[14]
Optimized glycemic control of type 2 diabetes with reinforcement learning: a proof-of-concept trial
Guangyu Wang, Xiaohong Liu, Zhen Ying, Guoxing Yang, Zhiwei Chen, Zhiwen Liu, Min Zhang, Hongmei Yan, Yuxing Lu, Yuanxu Gao, et al. Optimized glycemic control of type 2 diabetes with reinforcement learning: a proof-of-concept trial. Nature Medicine, 29(10):2633–2642, 2023
2023
-
[15]
Uniform manifold approximation and projection for dimension reduction
McInnes Leland, Healy John, and Melville James. Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426, 2018
2018 arXiv
-
[16]
From louvain to leiden: guaranteeing well-connected communities
Vincent A Traag, Ludo Waltman, and Nees Jan Van Eck. From louvain to leiden: guaranteeing well-connected communities. Scientific Reports, 9(1):5233, 2019. 11
2019
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.