What Makes a Representation Good for Single-Cell Perturbation Prediction?
Pith reviewed 2026-05-20 08:03 UTC · model grok-4.3
The pith
PerturbedVAE separates sparse perturbation signals from dominant invariant gene expression to recover causal representations for accurate single-cell response prediction.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
PerturbedVAE explicitly separates perturbation-specific information from dominant invariant structure and recovers causal representations to effectively utilize such information for prediction, achieving state-of-the-art performance on a widely used benchmark with significant gains on out-of-distribution combinatorial predictions.
What carries the argument
PerturbedVAE, a variational autoencoder that disentangles invariant cellular structure from sparse perturbation effects while recovering causal factors, guided by an identifiability analysis that specifies when the separation is possible.
If this is right
- Predictions become more accurate for unseen combinations of genetic perturbations.
- Recovered representations reveal interpretable perturbation-response programs.
- Causal structure in the latent space supports more reliable downstream use of the learned representations.
- The framework generalizes across multiple evaluation settings on standard single-cell benchmarks.
Where Pith is reading between the lines
- Similar separation of dominant background from sparse signals could apply to other high-dimensional biological datasets where most variation is unrelated to the intervention of interest.
- The identifiability conditions might be tested directly by measuring how well recovered factors align with known causal perturbation pathways in new datasets.
- If the separation holds, it suggests a route to more parameter-efficient models that do not need to encode the full invariant background for every prediction task.
Load-bearing premise
The identifiability analysis correctly characterizes the conditions under which sparse perturbation effects can be reliably recovered from the data.
What would settle it
A controlled experiment in which representations learned without the explicit separation match or exceed PerturbedVAE performance on out-of-distribution combinatorial perturbation predictions would falsify the central claim.
Figures
read the original abstract
Single-cell perturbation modeling is fundamental for understanding and predicting cellular responses to genetic perturbations. However, existing approaches, from causal representation learning to foundation models, often struggle with an overlooked challenge: gene expression is dominated by perturbation-invariant information, while perturbation-specific signals are intrinsically sparse. As a result, learned representations either entangle invariant and perturbation-specific information, leading to spurious and non-generalizable predictors, or suppress perturbation-specific signals altogether, rendering them ineffective for prediction. To address this, we propose PerturbedVAE, a general framework designed to resolve this signal imbalance. The framework explicitly separates perturbation-specific information from dominant invariant structure and recovers causal representations to effectively utilize such information for prediction. We further provide an identifiability analysis that characterizes the conditions under which sparse perturbation effects can be reliably recovered, thereby clarifying how the framework can be concretely specified under such conditions. Empirically, PerturbedVAE achieves state-of-the-art performance on a widely used benchmark across multiple evaluation settings, yielding significant gains on out-of-distribution combinatorial predictions and uncovering interpretable perturbation-response programs.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces PerturbedVAE, a VAE-based framework for single-cell perturbation prediction that explicitly separates sparse perturbation-specific signals from dominant invariant gene expression structure. It provides an identifiability analysis characterizing conditions for recovering causal representations under sparsity assumptions and reports state-of-the-art empirical performance on a standard benchmark, with particular gains on out-of-distribution combinatorial perturbation predictions.
Significance. If the identifiability results and empirical claims hold under realistic data conditions, the work could advance causal representation learning for perturbation modeling by directly addressing signal imbalance, potentially improving generalization in single-cell biology applications. The combination of theoretical analysis and benchmark results on combinatorial OOD tasks represents a meaningful contribution if the separation mechanism is shown to be load-bearing.
major comments (2)
- [§3] §3 (Identifiability Analysis): The analysis relies on assumptions of independent sparse perturbations and a specific mixing model to guarantee recovery of causal representations. However, the manuscript does not verify whether the high-dimensional single-cell benchmark datasets satisfy these conditions (e.g., via correlation analysis or simulation under batch effects), leaving open the possibility that recovered latents remain entangled rather than causally separated.
- [§5.2, Table 2] §5.2, Table 2 (OOD combinatorial results): The reported gains over baselines are presented as evidence for effective use of perturbation-specific information, but the paper lacks an ablation removing the explicit separation component while keeping other regularization choices fixed. Without this, it is unclear whether the improvements stem from causal recovery or from general VAE regularization benefits.
minor comments (3)
- [Abstract] The abstract claims 'significant gains' on OOD predictions but does not include even approximate quantitative values (e.g., percentage improvement or delta in a metric); adding a brief quantification would improve readability.
- [§2.2] Notation in §2.2 for the latent decomposition (invariant vs. perturbation-specific) could be made more explicit by introducing distinct symbols for each component rather than relying on context.
- [Figure 3] Figure 3 caption does not specify the exact number of runs or error bars used for the performance curves, which is needed to assess variability of the SOTA claims.
Simulated Author's Rebuttal
We thank the referee for their constructive feedback on our manuscript. We address the major comments point by point below, providing clarifications and outlining planned revisions to strengthen the paper.
read point-by-point responses
-
Referee: [§3] §3 (Identifiability Analysis): The analysis relies on assumptions of independent sparse perturbations and a specific mixing model to guarantee recovery of causal representations. However, the manuscript does not verify whether the high-dimensional single-cell benchmark datasets satisfy these conditions (e.g., via correlation analysis or simulation under batch effects), leaving open the possibility that recovered latents remain entangled rather than causally separated.
Authors: We agree that explicitly verifying the assumptions on the benchmark datasets would enhance the robustness of the identifiability claims. The analysis assumes independent sparse perturbations and a linear mixing model, which are motivated by the biological sparsity of perturbation effects. In the revised manuscript, we will include additional analyses, such as computing correlations between different perturbations in the datasets and performing simulations that incorporate batch effects, to assess how well the conditions hold in practice. revision: yes
-
Referee: [§5.2, Table 2] §5.2, Table 2 (OOD combinatorial results): The reported gains over baselines are presented as evidence for effective use of perturbation-specific information, but the paper lacks an ablation removing the explicit separation component while keeping other regularization choices fixed. Without this, it is unclear whether the improvements stem from causal recovery or from general VAE regularization benefits.
Authors: We acknowledge the value of an ablation that isolates the contribution of the explicit separation mechanism. While the current manuscript compares against various baselines and includes some regularization ablations, it does not specifically remove only the separation component. We will add this ablation study in the revised version to demonstrate that the performance gains on OOD combinatorial predictions are attributable to the perturbation-specific separation rather than general VAE benefits. revision: yes
Circularity Check
No significant circularity; derivation is self-contained
full rationale
The paper's central derivation introduces PerturbedVAE as a framework that explicitly separates perturbation-specific signals from invariant structure, then supplies its own identifiability analysis characterizing recovery conditions for sparse effects. No load-bearing step reduces by construction to a fitted parameter renamed as prediction, nor to a prior self-citation whose content is unverified; the analysis is presented as part of the current work rather than imported from overlapping prior authorship. The empirical SOTA claims rest on benchmark evaluation rather than tautological re-expression of inputs. This is the normal case of an independent modeling contribution.
Axiom & Free-Parameter Ledger
free parameters (1)
- VAE hyperparameters such as latent dimension and regularization strength
axioms (2)
- domain assumption Gene expression is dominated by perturbation-invariant information while perturbation-specific signals are intrinsically sparse.
- domain assumption Sparse perturbation effects are recoverable under conditions characterized by the identifiability analysis.
invented entities (1)
-
PerturbedVAE
no independent evidence
Lean theorems connected to this paper
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
We propose PerturbedVAE... explicitly separates perturbation-specific information from dominant invariant structure... identifiability analysis that characterizes the conditions under which sparse perturbation effects can be reliably recovered
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
Exploring genetic interaction manifolds constructed from rich single-cell phenotypes , author=. Science , volume=. 2019 , publisher=
work page 2019
-
[2]
Genome-scale CRISPR-mediated control of gene repression and activation , author=. Cell , volume=. 2014 , publisher=
work page 2014
-
[3]
Understanding intermediate layers using linear classifier probes
Understanding intermediate layers using linear classifier probes , author=. arXiv preprint arXiv:1610.01644 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[4]
Language models represent space and time,
Language models represent space and time , author=. arXiv preprint arXiv:2310.02207 , year=
-
[5]
Advances in Neural Information Processing Systems , volume=
Inference-time intervention: Eliciting truthful answers from a language model , author=. Advances in Neural Information Processing Systems , volume=
-
[6]
A visual--language foundation model for pathology image analysis using medical twitter , author=. Nature medicine , volume=. 2023 , publisher=
work page 2023
-
[7]
Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines , author=. Nature Methods , volume=. 2025 , publisher=
work page 2025
-
[8]
Random forests , author=. Machine learning , volume=. 2001 , publisher=
work page 2001
-
[9]
IEEE transactions on information theory , volume=
Nearest neighbor pattern classification , author=. IEEE transactions on information theory , volume=. 1967 , publisher=
work page 1967
-
[10]
Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=
Regularization and variable selection via the elastic net , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 2005 , publisher=
work page 2005
-
[11]
Benchmarking foundation cell models for post-perturbation RNA-seq prediction , author=. BMC genomics , volume=. 2025 , publisher=
work page 2025
-
[12]
Transfer learning enables predictions in network biology , author=. Nature , volume=. 2023 , publisher=
work page 2023
-
[13]
Nature Machine Intelligence , volume=
scBERT as a large-scale pretrained deep language model for cell type annotation of single-cell RNA-seq data , author=. Nature Machine Intelligence , volume=. 2022 , publisher=
work page 2022
-
[14]
Large-scale foundation model on single-cell transcriptomics , author=. Nature methods , volume=. 2024 , publisher=
work page 2024
-
[15]
International conference on machine learning , pages=
A theoretical analysis of contrastive unsupervised representation learning , author=. International conference on machine learning , pages=. 2019 , organization=
work page 2019
-
[16]
International conference on machine learning , pages=
Understanding contrastive representation learning through alignment and uniformity on the hypersphere , author=. International conference on machine learning , pages=. 2020 , organization=
work page 2020
-
[17]
Causal representation learning from multiple distributions: A general setting
Causal representation learning from multiple distributions: A general setting , author=. arXiv preprint arXiv:2402.05052 , year=
-
[18]
scGPT: toward building a foundation model for single-cell multi-omics using generative AI , author=. Nature methods , volume=. 2024 , publisher=
work page 2024
-
[19]
Advances in Neural Information Processing Systems , volume=
Identifiability guarantees for causal disentanglement from soft interventions , author=. Advances in Neural Information Processing Systems , volume=
-
[20]
arXiv preprint arXiv:2206.06169 , year=
Causal representation learning for instantaneous and temporal effects in interactive systems , author=. arXiv preprint arXiv:2206.06169 , year=
-
[21]
International Conference on Machine Learning , pages=
Citris: Causal identifiability from temporal intervened sequences , author=. International Conference on Machine Learning , pages=. 2022 , organization=
work page 2022
-
[22]
Advances in Neural Information Processing Systems , volume=
Weakly supervised causal representation learning , author=. Advances in Neural Information Processing Systems , volume=
-
[23]
Journal of machine learning research , volume=
Visualizing data using t-SNE , author=. Journal of machine learning research , volume=
-
[24]
Advances in neural information processing systems , volume=
Self-supervised learning with data augmentations provably isolates content from style , author=. Advances in neural information processing systems , volume=
-
[25]
Nonlinear independent component analysis: Existence and uniqueness results , author=. Neural networks , volume=. 1999 , publisher=
work page 1999
-
[26]
Natural Image Statistics: A Probabilistic Approach to Early Computational Vision , pages=
Independent component analysis , author=. Natural Image Statistics: A Probabilistic Approach to Early Computational Vision , pages=. 2001 , publisher=
work page 2001
-
[27]
Journal of Machine Learning Research , volume=
Identifying weight-variant latent causal models , author=. Journal of Machine Learning Research , volume=
-
[28]
The Fourteenth International Conference on Learning Representations , year=
I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data? , author=. The Fourteenth International Conference on Learning Representations , year=
-
[29]
Yuhang Liu and Zhen Zhang and Dong Gong and Erdun Gao and Biwei Huang and Mingming Gong and Anton van den Hengel and Kun Zhang and Javen Qinfeng Shi , booktitle=. Beyond
-
[30]
Transactions on Machine Learning Research , issn=
Latent Covariate Shift: Unlocking Partial Identifiability for Multi-Source Domain Adaptation , author=. Transactions on Machine Learning Research , issn=
-
[31]
arXiv preprint arXiv:2403.15711 , year=
Towards Identifiable Latent Additive Noise Models , author=. arXiv preprint arXiv:2403.15711 , year=
-
[32]
Advances in neural information processing systems , volume=
Unsupervised feature extraction by time-contrastive learning and nonlinear ica , author=. Advances in neural information processing systems , volume=
-
[33]
Predicting single-cell cellular responses to perturbations using cycle consistency learning , author=. Bioinformatics , volume=. 2024 , publisher=
work page 2024
-
[34]
International conference on machine learning , pages=
Linear causal disentanglement via interventions , author=. International conference on machine learning , pages=. 2023 , organization=
work page 2023
-
[35]
Artificial intelligence and statistics , pages=
Nonlinear ICA of temporally dependent stationary sources , author=. Artificial intelligence and statistics , pages=. 2017 , organization=
work page 2017
-
[36]
The Twelfth International Conference on Learning Representations , year=
Identifiable Latent Polynomial Causal Models through the Lens of Change , author=. The Twelfth International Conference on Learning Representations , year=
-
[37]
arXiv preprint arXiv:2001.04872 , year=
Disentanglement by nonlinear ica with general incompressible-flow networks (gin) , author=. arXiv preprint arXiv:2001.04872 , year=
-
[38]
Probabilistic and causal inference: The works of Judea Pearl , pages=
Causality for machine learning , author=. Probabilistic and causal inference: The works of Judea Pearl , pages=
-
[39]
International conference on artificial intelligence and statistics , pages=
Variational autoencoders and nonlinear ica: A unifying framework , author=. International conference on artificial intelligence and statistics , pages=. 2020 , organization=
work page 2020
- [40]
-
[41]
International conference on machine learning , pages=
Stochastic backpropagation and approximate inference in deep generative models , author=. International conference on machine learning , pages=. 2014 , organization=
work page 2014
-
[42]
Advances in neural information processing systems , volume=
Dags with no tears: Continuous optimization for structure learning , author=. Advances in neural information processing systems , volume=
-
[43]
Advances in Neural Information Processing Systems , volume=
Learning identifiable factorized causal representations of cellular responses , author=. Advances in Neural Information Processing Systems , volume=
-
[44]
Advances in Neural Information Processing Systems , volume=
Nonparametric identifiability of causal representations from unknown interventions , author=. Advances in Neural Information Processing Systems , volume=
-
[45]
scGen predicts single-cell perturbation responses , author=. Nature methods , volume=. 2019 , publisher=
work page 2019
-
[46]
Molecular systems biology , volume=
Predicting cellular responses to complex perturbations in high-throughput screens , author=. Molecular systems biology , volume=
-
[47]
Nature Biotechnology , volume=
Predicting transcriptional outcomes of novel multigene perturbations , author=. Nature Biotechnology , volume=. 2024 , publisher=
work page 2024
-
[48]
Advances in Neural Information Processing Systems , volume=
Modelling cellular perturbations with the sparse additive mechanism shift variational autoencoder , author=. Advances in Neural Information Processing Systems , volume=
-
[49]
arXiv preprint arXiv:2506.12439 , year=
Interpretable Causal Representation Learning for Biological Data in the Pathway Space , author=. arXiv preprint arXiv:2506.12439 , year=
-
[50]
Multi-ContrastiveVAE disentangles perturbation effects in single cell images from optical pooled screens , author=. bioRxiv , pages=. 2023 , publisher=
work page 2023
-
[51]
International conference on machine learning , pages=
Interventional causal representation learning , author=. International conference on machine learning , pages=. 2023 , organization=
work page 2023
-
[52]
Advances in Neural Information Processing Systems , volume=
Predicting cellular responses to novel drug perturbations at a single-cell resolution , author=. Advances in Neural Information Processing Systems , volume=
-
[53]
Gpt-4 technical report , author=. arXiv preprint arXiv:2303.08774 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[54]
IEEE transactions on pattern analysis and machine intelligence , volume=
Representation learning: A review and new perspectives , author=. IEEE transactions on pattern analysis and machine intelligence , volume=. 2013 , publisher=
work page 2013
-
[55]
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context , author=. arXiv preprint arXiv:2403.05530 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[56]
Journal of Machine Learning Research , volume=
Underspecification presents challenges for credibility in modern machine learning , author=. Journal of Machine Learning Research , volume=
-
[57]
Accounts of chemical research , volume=
The chemical space project , author=. Accounts of chemical research , volume=. 2015 , publisher=
work page 2015
-
[58]
Mapping information-rich genotype-phenotype landscapes with genome-scale Perturb-seq , author=. Cell , volume=. 2022 , publisher=
work page 2022
-
[59]
International conference on learning representations , year=
beta-vae: Learning basic visual concepts with a constrained variational framework , author=. International conference on learning representations , year=
-
[60]
Learning in graphical models , pages=
An introduction to variational methods for graphical models , author=. Learning in graphical models , pages=. 1998 , publisher=
work page 1998
-
[61]
Journal of the American statistical Association , volume=
Variational inference: A review for statisticians , author=. Journal of the American statistical Association , volume=. 2017 , publisher=
work page 2017
-
[62]
Pattern recognition and machine learning , author=. 2006 , publisher=
work page 2006
-
[63]
arXiv preprint arXiv:2104.08835 , year=
Crossfit: A few-shot learning challenge for cross-task generalization in nlp , author=. arXiv preprint arXiv:2104.08835 , year=
-
[64]
Journal of Machine Learning Research , volume=
Score-based causal representation learning: Linear and general transformations , author=. Journal of Machine Learning Research , volume=
-
[65]
NeurIPS 2022 Workshop on Causality for Real-world Impact , year=
Learning Causal Representations of Single Cells via Sparse Mechanism Shift Modeling , author=. NeurIPS 2022 Workshop on Causality for Real-world Impact , year=
work page 2022
-
[66]
Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear
Sebastien Lachapelle and Pau Rodriguez and Yash Sharma and Katie E Everett and R. Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear. First Conference on Causal Learning and Reasoning , year=
-
[67]
Nature Machine Intelligence , volume=
Shortcut learning in deep neural networks , author=. Nature Machine Intelligence , volume=. 2020 , publisher=
work page 2020
-
[68]
Proceedings of the IEEE , volume=
Toward causal representation learning , author=. Proceedings of the IEEE , volume=. 2021 , publisher=
work page 2021
- [69]
-
[70]
Dissecting cell identity via network inference and in silico gene perturbation , author=. Nature , volume=. 2023 , publisher=
work page 2023
-
[71]
Annual Review of Statistics and Its Application , volume=
Causal inference in the social sciences , author=. Annual Review of Statistics and Its Application , volume=. 2024 , publisher=
work page 2024
-
[72]
The Econometrics Journal , volume=
Causal inference and data fusion in econometrics , author=. The Econometrics Journal , volume=. 2025 , publisher=
work page 2025
-
[73]
Toward a foundation model of causal cell and tissue biology with a Perturbation Cell and Tissue Atlas , author=. Cell , volume=. 2024 , publisher=
work page 2024
-
[74]
How to build the virtual cell with artificial intelligence: Priorities and opportunities , author=. Cell , volume=. 2024 , publisher=
work page 2024
-
[75]
Nature biotechnology , volume=
Combinatorial single-cell CRISPR screens by direct guide RNA capture and targeted sequencing , author=. Nature biotechnology , volume=. 2020 , publisher=
work page 2020
-
[76]
Conference on Causal Learning and Reasoning , pages=
Learning causal representations of single cells via sparse mechanism shift modeling , author=. Conference on Causal Learning and Reasoning , pages=. 2023 , organization=
work page 2023
-
[77]
Causal machine learning for single-cell genomics , author=. Nature Genetics , pages=. 2025 , publisher=
work page 2025
-
[78]
arXiv preprint arXiv:2410.03380 , year=
Identifying biological perturbation targets through causal differential networks , author=. arXiv preprint arXiv:2410.03380 , year=
-
[79]
Learning Genetic Perturbation Effects with Variational Causal Inference , author=. bioRxiv , pages=. 2025 , publisher=
work page 2025
-
[80]
European Conference on Computer Vision (ECCV) , pages =
CLAP: Isolating Content from Style through Contrastive Learning with Augmented Prompts , author =. European Conference on Computer Vision (ECCV) , pages =
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.