Evaluating Integrative Strategies for Incorporating Phenotypic Features in Spatial Transcriptomics

Ahmad Kamal Hamid; Beth A Cimini; Esteban Miglietta; Levin M Moser; Nodar Gogoberidze

REVIEW 3 major objections 5 minor 2 references

Evaluating Integrative Strategies for Incorporating Phenotypic Features in Spatial Transcriptomics

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Under constrained real-world conditions, minimally tuned VAEs extract biological signal from stain images, and multiplex Leiden clustering with these latent spaces consistently improves cluster homogeneity.

desk verdict Useful proof-of-concept on VAE-based morphology integration in spatial transcriptomics, but a weight-labeling inconsistency in the multiplex clustering undermines the headline claim until clarified. read the letter →

arxiv 2507.22212 v1 pith:JYEQX4UE submitted 2025-07-29 q-bio.QM

Levin M Moser , Ahmad Kamal Hamid , Esteban Miglietta , Nodar Gogoberidze , Beth A Cimini This is my paper · ORCID

classification q-bio.QM

keywords spatialtranscriptomicsmulti-modalintegrationvariationalautoencodercelltypedeconvolutionmorphologicalfeaturesMERFISHLeidenclusteringlatentspace

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Spatial transcriptomics measures gene expression in tissue while preserving spatial context, but no standard method exists for adding the paired nuclear and membrane stain images that routine workflows already capture. This paper asks whether a minimally tuned variational autoencoder (VAE) can turn such images into usable cell representations under realistic constraints: one small dataset, heavily imbalanced cell types, and bounding-box crops instead of precise cell boundaries. Using a public MERFISH mouse ileum sample with 5,192 cells, 241 genes, and 19 annotated cell types, the authors report that VAE-derived latent spaces capture meaningful biological variation, with a stain-only latent space even predicting a small set of genes. The central result is that multiplex Leiden clustering, which combines transcript counts with these latent spaces, consistently improves cluster homogeneity compared with transcript counts alone, and classical image-profiling features fail to match the learned representations. If this holds, even simple deep learning models can make practical use of routine stains to sharpen cell-type identification in spatial transcriptomics.

What carries the argument

The load-bearing mechanism is a convolutional variational autoencoder whose inputs are multi-plane binary spot maps (each gene as a plane, with spots expanded into local patches) and/or paired nuclear and membrane stain crops, mapped through a single encoder to a shared 50-dimensional latent space. Reconstruction is trained with a Dice-plus-Focal loss with dynamic gene weighting to counter extreme sparsity, and the KL term is omitted so the latent space prioritizes reconstruction fidelity over regularization. The integration step uses multiplex Leiden clustering, which builds layered k-nearest-neighbor graphs with a shared vertex set and distinct edge sets for each modality and optimizes a modality weight; that is the machinery that produces the reported homogeneity gains.

What would settle it

Re-run the identical VAE training and multiplex Leiden clustering on a second independently annotated spatial transcriptomics sample (or on this sample re-annotated with a different marker list). If the homogeneity gain from TC+LS2 over TC-only does not reproduce, or if the gain persists when cell-type labels are randomly shuffled, the central claim is falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a standard convolutional VAE with a 50-dimensional latent space, trained without systematic hyperparameter optimization and evaluated through sampling rather than mean embeddings, can extract biologically informative features from cell crops in imaging-based spatial transcriptomics. Transcript counts remain the strongest feature space, but the latent spaces recover specific cell types—smooth muscle cells, mid-villus enterocytes, stromal cells, and CD8+ T cells—and a latent space trained only on nuclear and membrane stains can predict a handful of genes, including Acta2 and Neat1. The decisive finding is integrative: multiplex Leiden clustering of transcript counts with the VAE latent spaces (LS2–LS5) raises the homogeneity score of the resulting clusters even when the transcript input is reduced to small gene subsets, with optimized morphology weights of roughly 3–11%. In contrast, classical image-profiling features underperform across all evaluations, indicating that learned representations, not hand-crafted ones, carry the usable morphological signal in this constrained setting.

Load-bearing premise

The evaluation treats the published 19-cell-type labels of this single mouse ileum sample as ground truth for every metric; if those labels are systematically wrong, the claim that the latent spaces carry biological signal and improve cluster purity does not follow.

Editorial extensions

If this is right

If the central claim holds, imaging-based spatial transcriptomics can be augmented with routine nuclear and membrane stains without requiring precise cell segmentation, since bounding-box crops sufficed for the reported gains.
Learned morphological embeddings can replace hand-crafted image features as the default complement to transcript counts, since the classical features collapsed when multiplexed with transcript data.
A stain-only latent representation can serve as a weak gene-expression predictor for a few morphologically or positionally anchored transcripts, which may help flag candidate genes for targeted follow-up in samples without a full transcript panel.
Small modality weights (around 3–11%) are enough for morphology to improve cluster purity, suggesting the integration benefit is real but modest and should not be expected to dominate the transcriptional signal.
The approach transfers to reduced gene panels (PCA-based, ground-truth-based, or random subsets), so it could be relevant to targeted or cost-limited spatial assays.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An extension the paper leaves implicit is applying the same VAE-plus-multiplex-Leiden recipe to multi-sample or disease-versus-health comparisons, where the homogeneity gain should be tested for whether it helps detect disease-associated cell states rather than just the dominant healthy enterocyte population.
Because the homogeneity improvement tracks the abundant mid-villus enterocyte population, the method may preferentially aid major cell types; rare-type recovery might require class-balanced training or a different clustering objective.
The near-zero optimized morphology weights hint that the latent morphology signal acts as a consistent prior or regularizer rather than a competing data source, so weight selection could potentially be fixed heuristically instead of grid-searched in future applications.
A cheap falsifiable extension is to shuffle the ground-truth labels within the same dataset and verify that the multiplex homogeneity gain disappears; if it persists under shuffled labels, the claimed signal would be an artifact of clustering bias rather than biology.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The genuinely new piece here is the head-to-head comparison of minimally tuned VAE latent spaces against CellProfiler features for integrating nuclear/membrane stains with MERFISH transcript counts under realistic constraints: 5k cells, 19 imbalanced cell types, and bounding-box crops instead of precise segmentation. The main positive result—stain-only LS2 improves multiplex cluster homogeneity and predicts a handful of genes—is backed by several independent analyses (PERMANOVA, random forest, ridge regression, clustering), and the authors are candid about trade-offs like lower AMI and worse label recovery. Code is public. That is a solid proof-of-concept.

The soft spots are real but mostly minor. One sample, transductive VAE training with no repeated runs or error bars, and ground-truth labels taken from the original MERFISH study without independent validation. These limit confidence but do not sink the paper; the authors acknowledge them.

The load-bearing issue is the weight ambiguity in the multiplex clustering. In the gene-subset section the text reports optimized weights of 3%, 11%, and 5% and interprets them as 'small contributions from morphological features.' The Figure 5 caption (and Figure 4) labels the same quantity as the 'TC modality weight.' If the caption is correct, the TC layer carries only 3–11% of the weight and LS2 carries 89–97%, meaning the clustering is LS2-dominated, not TC augmented with a dash of morphology. That turns the headline 'augmenting subsets of TC with LS2' into 'replacing most of the degraded TC signal with LS2.' The authors need to clarify which weight they optimized and how it enters the multiplex graph. If the small-morphology-weight reading is intended, the captions are wrong; if the captions are right, the text and abstract overstate the case. Either way, it must be fixed before the paper is usable. I do not think this is fatal—there is independent evidence that LS2 carries biological signal—but it is exactly the kind of inconsistency that should be caught in review.

Who is this for? Anyone working on integrating image morphology with spatial transcriptomics, especially in the small-data regime. It will not settle the open question, but it is a careful, reproducible data point. I would send it to review: the code and multi-metric evaluation deserve a serious referee, and the weight issue is fixable. I would want to see the corrected version before relying on the headline claim.

Referee Report

3 major / 5 minor

Summary. This manuscript benchmarks strategies for integrating phenotypic imaging features with spatial transcriptomics using a public MERFISH mouse ileum dataset (5,192 cells, 241 genes, 19 annotated cell types). The authors train five VAEs (transcript-only, stain-only, and three multimodal combinations) with a binary multi-plane spot representation, extract latent spaces (LSs), and evaluate them via PERMANOVA/PERMDISP, random forest classification, unsupervised Leiden clustering, multiplex Leiden clustering with transcript counts (TC), and ridge regression for predicting gene expression from a stain-only LS. They report that TC generally outperforms all feature spaces, that LS2 captures some cell-type and gene-expression signal, that CellProfiler features underperform, and that multiplex clustering of TC with LSs yields higher homogeneity scores than TC alone, including when TC is restricted to gene subsets. The paper concludes that minimally tuned VAEs can extract biologically meaningful morphological signals under real-world constraints.

Significance. If the findings hold, this is a useful, reproducible benchmark (code and data are made available) for a practical question: can routine nuclear/membrane stains improve cell-type clustering in imaging-based spatial transcriptomics? The study's strengths include the use of multiple independent evaluation modalities, the explicit treatment of dispersion effects via PERMDISP, and the honest discussion of class-imbalance effects on the homogeneity metric. The main claims are plausible and appropriately restrained in the Discussion, though the abstract and lay summary are more assertive than the evidence warrants given the unresolved weight interpretation and the absence of repeated-run variability.

major comments (3)

[Results, 'Joint clustering of transcript counts and latent spaces enhanced cluster homogeneity'; Figure 5 caption] The manuscript contains a direct contradiction about what the optimized multiplex-clustering weights represent. The Results text reports optimized weights of 3%, 11%, and 5% for the GT-based, PCA-based, and random gene subsets and interprets them as 'small contributions from morphological features,' i.e., the weight of LS2 in the multiplex graph. The Figure 5 caption (and the Figure 4 caption) labels the same displayed quantity as the 'TC modality weight.' These cannot both be correct. If the captions are correct, then TC carries 3–11% of the total graph weight and LS2 carries 89–97%, so the clustering is effectively LS2-dominated with a minor TC perturbation; the homogeneity improvement over TC-only would not support the abstract's claim that 'augmenting only subsets of TC with the stain-derived LS2' produces gains. If the text is correct, the captions must be changed and the actual TC weights should be reported. Because the central integrative claim depends on this quantity, the authors must clarify the definition and report both weights.
[Results, 'Unsupervised Leiden clustering...' and Figures 4–5] The central claim of 'consistent gains' in homogeneity is supported only by point estimates from single clustering runs. The VAE is trained once on the full dataset (transductively), and each Leiden clustering and metric is reported without any repeated initialization, bootstrap interval, or alternative graph-construction settings. Since the homogeneity improvements in Figures 4 and 5 are modest in magnitude and the number of clusters changes from 16 to 6–8, the authors should provide variability across runs (e.g., different random seeds for the VAE sampling and Leiden initialization) or otherwise demonstrate that the gain is robust rather than a single realization. This is needed specifically because the Discussion attributes the homogeneity boost to improved recovery of the dominant mid-villus enterocyte population.
[Methods, 'Models' (transductive training) and Results, multiplex clustering with LS3–5] For the multiplex clustering with LS3–5, the latent spaces were trained on the same transcript counts that are subsequently clustered together with TC. Any homogeneity gain in these combinations could partly reflect duplicated information rather than complementary signal. The paper does not address this circularity. The LS2 result is independent and remains the cleanest evidence, but the abstract generalizes the integration claim beyond LS2. Please state this limitation explicitly and, if possible, provide a control where the VAE is trained on a held-out subset of cells or on shuffled TC labels to show that the multiplex gain is not an artifact of information duplication.

minor comments (5)

[Table 1 and Methods, 'Models'] The text states that filtering genes with fewer than 100 transcripts results in 196 genes for VAE3–5, but Table 1 lists 182 genes for all three multimodal models. Please correct the inconsistency.
[Equation 2] The Focal loss formula uses α_t, γ, and p_t without defining these symbols in the text or in the equation; please add definitions.
[Figure 5 caption] The phrase 'TC modality weight' in the caption conflicts with the main text's description of the same quantity as the morphological-feature weight; this is already raised as a major point, but the caption should be corrected in any case.
[Methods, 'Statistical analysis and modeling'] In the description of PERMANOVA, the combination of scikit-learn, scikit-bio, and SciPy is mentioned; please clarify which package implements the permutation test and how the distance matrix is standardized.
[Discussion, paragraph on multiplex clustering] The statement that the approach 'uncovered structure that was not captured by TC alone' refers to a stromal-cell cluster in TC+LS2, but the preceding sentences note a decline in overall label recovery; consider rephrasing to avoid overstatement.

Circularity Check

2 steps flagged · score 2.0 of 10

Minor circular content from TC-trained latent spaces and GT-based gene subsetting; the stain-only LS2 result is independent.

fitted input called prediction [Discussion, 'Building on these findings' paragraph; Results, 'Joint clustering of transcript counts and latent spaces enhanced cluster homogeneity']
"we also jointly clustered TC with LSs from VAE1 and VAE3–5, as the TC component of their input retained local spatial context that could provide complementary information."

VAE1 is trained exclusively on the TC matrix, and VAE3–5 are trained on TC plus stain images, so their latent spaces are fitted functions of the same transcript counts that form the TC layer of the multiplex graph. Reporting 'combining TC with LSs' as evidence for cross-modal integration therefore conflates genuine morphological augmentation (LS2) with re-weighting a transformed copy of the TC signal. The homogeneity gain from TC+LS1 or TC+LS3–5 can be produced by the duplicated TC information, so it is not a clean demonstration that imaging features add new biological signal. The paper's independent stain-only LS2 result remains valid, making this a partial rather than global circularity.
fitted input called prediction [Methods, 'Dataset and pre-processing'; Results, gene-subset paragraph]
"The GT-based approach involved the identification of the top gene per true label based on differential expression analysis with the Wilcoxon rank sum test, yielding 19 genes."

This gene subset is selected by ranking genes for separation of the ground-truth labels, and the same ground-truth labels are later used to compute homogeneity, adjusted mutual information, and Jaccard scores for the clustering. The GT-based arm therefore evaluates clustering on a feature set that was explicitly engineered to discriminate those labels, so its high homogeneity is partly by construction. The conclusion does not depend on this arm alone because the PCA-based and random subsets show the same trend, so the circularity is minor.

full rationale

The central derivation is not circular: the VAE embeddings are benchmarked against an external ground-truth partition, and LS2 is trained only on stain images, so the claim that morphological signal improves cluster homogeneity has independent support. The abstract's key wording ('a trend that also held when augmenting only subsets of TC with the stain-derived LS2') specifically rests on LS2. Two minor circular elements exist: multiplex clustering with LS1/LS3–5 reuses TC-trained latent spaces, and the GT-based gene subset is selected using the same labels used for evaluation. Neither is load-bearing because the PCA/random subsets and the stain-only LS2 reproduce the trend. I also note a serious internal inconsistency that is not a circularity: the text interprets the optimized 3%/11%/5% weights as 'small contributions from morphological features,' while Figure 5 and Figure S5 captions call the displayed quantity the 'TC modality weight.' If the captions are correct, the multiplex clustering is LS2-dominated and the 'augmenting TC with LS2' framing is inverted. This should be resolved as a correctness/interpretation issue, but it does not make the derivation equivalent to its inputs.

Assumptions & free parameters 9 free parameters · 6 assumptions · 0 invented entities

The paper's contributions are empirical and depend on many hand-set hyperparameters (latent size, crop size, epochs, loss weights, clustering resolution, modality weights), on the accuracy of the external GT labels, and on a single-sample dataset; no new physical or biological entities are introduced.

free parameters (9)

Latent dimension = 50
Bottleneck size for all VAEs, chosen by hand; controls capacity of the learned representation used in all downstream analyses.
KL divergence weight beta = 0
KL term omitted from loss; without it the encoder posterior is unregularized, so sampling the latent distribution is not a standard VAE posterior sample.
Loss weighting (Dice : Focal) = 0.8 : 0.4
Empirically chosen to balance sparse multi-plane spot reconstruction; influences which transcript patterns the LS retains.
Dynamic gene weighting scheme = inverse expression, rescaled per cell/batch
Introduced ad hoc to counteract bias toward highly expressed genes in reconstruction; affects TC-derived LSs.
Crop sizes per model = 100, 150, 200, 300 px by VAE
Empirically selected; Table 3 shows crop size strongly affects stain-only LS homogeneity, so this choice is load-bearing for LS2 claims.
Training epochs per model = 15, 20, 30, 500 by VAE
Empirically selected; Table 2 shows epoch count materially changes TC LS homogeneity, so the stopping point affects results.
Multiplex modality weight for TC = e.g., 3%, 11%, 5% for gene subsets
Optimized per multiplex Leiden run by grid search on silhouette score; small weights are reported as evidence that small morphological contributions help.
Leiden k and resolution gamma = grid-searched per dataset/feature space
Tuned via silhouette score with a lower cluster bound of 5; these choices shape all reported clustering metrics.
Ridge regression alpha = 1
L2 penalty for LS2-to-gene prediction; chosen a priori rather than tuned, but affects the R2 values cited for gene predictability.

assumptions (6)

domain assumption Published CellAnnotatoR GT labels are accurate enough to serve as ground truth.
All evaluations use these labels as benchmarks; any systematic label error, especially for rare cell types, would weaken claims about biological signal. Location: Methods/Dataset.
domain assumption A single MERFISH section from one mouse ileum represents the integration problem.
No biological replicates; authors call it a proof-of-concept; results may be tissue- or sample-specific. Location: Methods/Dataset and Discussion.
domain assumption Bounding-box crops based on convex hull centroids preserve sufficient morphology.
Bounding boxes replace exact cell borders; Table 3 shows crop size changes LS quality, so the crop construction is load-bearing. Location: Methods/Dataset.
ad hoc to paper Transductive training on all cells is acceptable for feature extraction.
No held-out set; authors argue overfitting is controlled by the bottleneck; this limits generalization claims about LS quality. Location: Methods/Models.
ad hoc to paper Sampling from the latent posterior with beta=0 yields valid embeddings.
With KL omitted, the posterior is not forced toward a prior; the paper still samples latent variables for all LSs, which may inject unregularized noise. Location: Methods/Models and Figure 1.
ad hoc to paper The dynamic gene weighting scheme recovers rare-gene signal without distorting the learned LS.
Inverse-expression weights rescaled per batch are a bespoke design choice; their effect on downstream clustering is not independently validated. Location: Methods/Models.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Evaluating Integrative Strategies for Incorporating Phenotypic Features in Spatial Transcriptomics." pith.science (2026). https://pith.science/paper/JYEQX4UE

@misc{pith2026250722212,
  author       = {Pith},
  title        = {Pith review of: Evaluating Integrative Strategies for Incorporating Phenotypic Features in Spatial Transcriptomics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JYEQX4UE}},
  note         = {Machine review of arXiv:2507.22212}
}

read the original abstract

Spatial transcriptomics (ST) technologies not only offer an unprecedented opportunity to interrogate intact biological samples in a spatially informed manner, but also set the stage for integration with other imaging-based modalities. However, how to best exploit spatial context and integrate ST with imaging-based modalities remains an open question. To address this, particularly under real-world experimental constraints such as limited dataset size, class imbalance, and bounding-box-based segmentation, we used a publicly available murine ileum MERFISH dataset to evaluate whether a minimally tuned variational autoencoder (VAE) could extract informative low-dimensional representations from cell crops of spot counts, nuclear stain, membrane stain, or a combination thereof. We assessed the resulting embeddings through PERMANOVA, cross-validated classification, and unsupervised Leiden clustering, and compared them to classical image-based feature vectors extracted via CellProfiler. While transcript counts (TC) generally outperformed other feature spaces, the VAE-derived latent spaces (LSs) captured meaningful biological variation and enabled improved label recovery for specific cell types. LS2, in particular, trained solely on morphological input, also exhibited moderate predictive power for a handful of genes in a ridge regression model. Notably, combining TC with LSs through multiplex clustering led to consistent gains in cluster homogeneity, a trend that also held when augmenting only subsets of TC with the stain-derived LS2. In contrast, CellProfiler-derived features underperformed relative to LSs, highlighting the advantage of learned representations over hand-crafted features. Collectively, these findings demonstrate that even under constrained conditions, VAEs can extract biologically meaningful signals from imaging data and constitute a promising strategy for multi-modal integration.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

2 extracted references · 1 canonical work pages

[1]

Du, Y., Ding, X., & Ye, Y. (2024). The spatial multi-omics revolution in cancer therapy: Precision redefined. Cell reports. Medicine , 5 , 101740. 2. van Olst, L., Simonton, B., Edwards, A. J., Forsyth, A. V., Boles, J., Jamshidi, P., Watson, T., Shepard, N., Krainc, T., Argue, B. M., Zhang, Z., Kuruvilla, J., Camp, L., Li, M., Xu, H., Norman, J. L., Caha...

work page 2024
[27]

R., Swain-Bowden, M

Stirling, D. R., Swain-Bowden, M. J., Lucas, A. M., Carpenter, A. E., Cimini, B. A., & Goodman, A. (2021). CellProfiler 4: improvements in speed, utility and usability. BMC bioinformatics , 22 , 433. 28. Bray, M.-A., Vokes, M. S., & Carpenter, A. E. (2015). Using CellProfiler for Automatic Identification and Measurement of Biological Objects in Images. Cu...

arXiv 2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Du, Y., Ding, X., & Ye, Y. (2024). The spatial multi-omics revolution in cancer therapy: Precision redefined. Cell reports. Medicine , 5 , 101740. 2. van Olst, L., Simonton, B., Edwards, A. J., Forsyth, A. V., Boles, J., Jamshidi, P., Watson, T., Shepard, N., Krainc, T., Argue, B. M., Zhang, Z., Kuruvilla, J., Camp, L., Li, M., Xu, H., Norman, J. L., Caha...

work page 2024

[2] [27]

R., Swain-Bowden, M

Stirling, D. R., Swain-Bowden, M. J., Lucas, A. M., Carpenter, A. E., Cimini, B. A., & Goodman, A. (2021). CellProfiler 4: improvements in speed, utility and usability. BMC bioinformatics , 22 , 433. 28. Bray, M.-A., Vokes, M. S., & Carpenter, A. E. (2015). Using CellProfiler for Automatic Identification and Measurement of Biological Objects in Images. Cu...

arXiv 2021