REVIEW 4 major objections 7 minor 1 cited by
No Foundations without Foundations -- Why semi-mechanistic models are essential for regulatory biology
T0 review · 4 major / 7 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read Foundation models of regulatory biology will stay out of reach unless they encode the order of perturbation, media, and time; this paper builds a semi-mechanistic framework to do so.
desk verdict The compositional framework is a genuinely useful new synthesis for perturb-seq modelling; the empirical pilot is too under-specified and potentially leaky to carry the weight of the central claim, so treat the paper as a principled proposal, not a validated method. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the experiment-as-function $F=W\circ M\circ P$, with each operation defined as a map on cell-state random variables: $P(X,p_\gamma)=X_{p_\gamma}$ applies the CRISPR edit, $M(X,m)=X_m$ applies a media condition, and $W(X,t)=X_t$ waits. The argument advances by asking which paths in the four-branch experimental diagram can be treated as identity, deriving the loss in Eq. (6) from the necessary assumption $F(X,\cdot,\cdot,t)=X$ and the additional regularizer in Eq. (10) from the assumption that some perturbed populations are dynamical attractors. The proof of principle instantiates $F$ as a neural ODE, $F(x,p_\gamma,t)=x_0+\int_0^t G(x,p_\gamma,s)\,ds$, and the error analysis models batch effects as zero-mean additive noise on top of the measurement function $V$, leading to the recommendation to replicate baseline controls across batches.
What would settle it
Take any perturb-seq dataset used to train a perturbation predictor and compare the transcriptomes of unedited cells harvested at day 0 with those of unedited cells in baseline media at the end time $t$, using the same within-day-split baseline the paper uses for its iPSC analysis. If the end-time distribution separates from the day-0 distribution by more than technical noise, the necessary assumption $F(X,\cdot,\cdot,t)=X$ fails and the paper's recommended loss is not applicable to that experiment; conversely, a dataset that passes this check and contains known steady states should reproduce the reported MSE gap between Eq. (6) and Eq. (10).
Extended reading notes
Core claim
The central claim is that the genotype-to-phenotype mapping measured in a perturb-seq screen is the composed function $F(X,p_\gamma,m,t)=W(M(P(X,p_\gamma),m),t)$, where $P$ applies a CRISPR perturbation, $M$ switches media conditions, and $W$ waits $t$ units of time, in that order. Writing the experiment this way makes the non-commutativity of biological operations explicit and brings hidden assumptions to the surface: the standard supervised loss (Eq. 6) is only valid if unedited cells do not respond to baseline media, $F(X,\cdot,\cdot,t)=X$; otherwise paired measurements are counterfactual and require an inverse $F^{-1}$. The paper then derives a “steady-state” regularizer (Eq. 10): wherever biology dictates that a perturbed population is an attractor, the loss demands $F(X^{p_\gamma}_{m,t},\cdot,\cdot,s)=X^{p_\gamma}_{m,t}$, and the proof of principle shows that this term lowers test MSE and reduces seed-to-seed variance for a neural ODE on an iPSC dataset. Variational autoencoders and structural causal models are shown to be special cases of the same composition, and the batch-error analysis yields a concrete experimental prescription: include unperturbed baseline cells in every sequencing batch.
Load-bearing premise
The framework's loss functions collapse if unedited cells in baseline media are not statistically stationary over the experiment's time window, because the paper's central pathway $F(X,\cdot,\cdot,t)=X$ is then false and every paired prediction becomes a counterfactual requiring an invertible map.
Editorial extensions
If this is right
- Established architectures—VAE-style perturbation autoencoders and structural causal models of perturbation screens—are recovered as special cases of the unified $F$-composition, so their design choices can be compared by the assumptions they make about $P$, $M$, and $W$.
- Datasets must be checked for the necessary assumption before being used to train or benchmark perturbation models; if unedited cells drift in baseline media, Eq. (6) is a counterfactual loss, not a supervised one.
- Enforcing biologically known steady states acts as a regularizer that improves prediction for other, non-steady-state perturbations, which is a concrete, testable benefit of injecting mechanism.
- Batch design can be optimised from the error analysis: placing unperturbed baseline cells in every batch reduces total prediction error because the $\eta_b$ and $\eta_{b'}$ terms enter with opposite signs.
Reading between the lines
- Editorial inference: if the necessary-assumption check becomes standard practice, several published perturb-seq benchmarks may need re-scoring, because models trained on drifting baseline cells could be fitting counterfactual structure rather than biology.
- Editorial inference: the steady-state regularizer should transfer to RNA-velocity and cell-differentiation models—any dataset with time series and known terminal states is a candidate for the same term, not just CRISPR screens.
- Editorial inference: a direct quantitative prediction is that the performance gap between Eq. (6) and Eq. (10) should grow with the number of perturbed populations that are at steady state and with the duration of the experiment relative to relaxation time; both are testable by re-running the NODE experiment on datasets with longer time courses.
- Editorial inference: the framework suggests a natural benchmark protocol—per-dataset verification of $F(X,\cdot,\cdot,t)=X$ and known attractors—before comparing foundation models; without it, leaderboard positions conflate architecture quality with baseline drift.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper argues that foundation models for regulatory biology need to be grounded in mechanistic models of how perturbation experiments are conducted. The authors formalize an in vitro perturb-seq experiment as a composition of operators P (genetic perturbation), M (media change), and W (waiting), yielding the function F in Eq. (4). Under a 'necessary assumption' that unedited cells in baseline media do not change over time (Eq. 5), they derive a supervised loss (Eq. 6); under an optional steady-state assumption they add a loss term (Eq. 10). They then connect this framework to variational autoencoders (Section 5.1) and causal models (Section 5.2), and analyze batch effects in Appendix B, recommending that unperturbed controls be included in every batch. The empirical proof-of-principle in Section 6 trains a neural ODE on pseudobulked iPSC data from Ishikawa et al. (2023), reporting that enforcing steady states improves test MSE for 11 held-out non-steady perturbations.
Significance. If the framework's utility is borne out by further validation, it would provide a principled way to design loss functions and experimental batches for perturbation modeling, and would clarify the hidden assumptions in existing methods. The formal derivation of the loss structure from the experimental protocol is a conceptual contribution, and the demonstration of links between the framework and VAEs/causal models is interesting. However, the paper's central empirical claim — that the steady-state loss improves predictive performance over standard training — is currently supported only by a single dataset with a post hoc selected steady-state set and no non-mechanistic regularizer control. The strength of the claim in the abstract therefore exceeds what the evidence establishes.
major comments (4)
- [Section 6 / Appendix D] The steady-state subset S is selected from the same days-2–5 time course that is later used for training and testing. Appendix D shows that the classification of the 14 steady-state perturbations is based on absolute LFC between sequential days, including the day-4-to-day-5 comparison; Section 6 then evaluates on the 11 'non-steady state' perturbations at t=5. If day-5 measurements contribute to deciding which perturbations are steady (and therefore which are non-steady and held out), the improved test MSE in Fig. 5 could reflect target leakage rather than a benefit of the mechanistic prior. The paper does not state whether the steady-state labeling uses only training days or an independent replicate. Please clarify the exact selection procedure and, if leakage is possible, re-run the experiment with S fixed a priori from a separate experiment or from a random split that does not use test-day information.
- [Section 6 / Figure 5] The comparison between the 'original loss' and the 'modified loss' does not control for the presence of an additional loss term. Adding the steady-state loss (Eq. 10) introduces extra supervision on the training set; a generic regularizer of comparable strength — e.g., L2 weight decay, dropout, or an output-smoothing term — could plausibly produce the same faster convergence and lower test MSE on a 100-point pseudobulk dataset. Without ablations that match the number, type, and strength of extra loss terms, the improvement cannot be attributed specifically to the mechanistic steady-state prior. Please include such controls.
- [Section 2.3.1 / Eq. (5)] The 'necessary assumption' that unedited cells do not respond to baseline media is explicit, but it is not validated on the Ishikawa et al. (2023) dataset used in Section 6. If this assumption fails, the paired supervised loss in Eq. (6) is not a proper objective for the observed data, and the counterfactual alternative in Eq. (7) requires an inverse F^{-1} whose existence is itself assumed. The paper's experimental recommendations (Section 2.3.1) call for verifying this assumption, yet the proof-of-principle does not report the stability of non-targeting controls across days. Please report this stability check or otherwise justify the assumption for the dataset analyzed.
- [Section 6] The neural ODE model and training protocol are not described in sufficient detail. The paper does not specify the network architecture, the number of parameters, the optimizer, the learning rate schedule, the weight of the steady-state loss, or the hyperparameter selection procedure. Without these details and without code, the proof-of-principle is not reproducible, and it is unclear how sensitive the improvement in Fig. 5 is to the chosen regularization strength. Please provide full experimental details or release code.
minor comments (7)
- [Section 2.2 / Eq. (4)] The notation X^{pγ}_{m,t} is used before the sub/superscript convention is explained; define it immediately before Eq. (4).
- [Section 3.1 / Eq. (11)] The sentence ending 'respectfully' should be 'respectively,' and the spaces N0^{n_W n_G} and N0^{n_G} should be introduced before use.
- [Section 3.1 / after Eq. (13)] The phrase 'to the extent that V has an inverse' is vague for a count-valued measurement function; clarify what is meant by V^{-1} in this setting.
- [Section 5.1 / Eq. (18)] The latent function L in the VAE derivation conflicts with the loss L used elsewhere; use a different symbol (e.g., h).
- [Section 6] The statement 'pseudobulk single-cell data over (pγ, t) pairs giving 100 unique data points' should state the number of perturbations and time points explicitly so the reader can verify the count.
- [Appendix B / Eq. (23)] The symbol V is used both for the measurement function (Eq. 11) and for the valid set; rename one of them.
- [Throughout] The phrase 'non-differentiating cellular models' is potentially confusing; consider using 'cells that do not differentiate over the experimental window.'
Circularity Check
Formal derivation is self-contained, but the proof-of-principle evaluation shows target leakage: the steady-state set S is selected from the same days-2–5 time course that defines the held-out t=5 test labels.
-
fitted input called prediction
[Section 6, Proof of principle; Appendix D, Analysis of iPSC time series dataset]
"In Appendix D, we confirm this from transcriptomic signatures and identify 14 perturbations (including the non-targeting control) that appear to converge on a steady state. ... We train the model using the remaining data via the original loss function given by equation (6), or a loss function with steady states enforced using a modification analogous to equation (10)."
The steady-state subset S used in the modified loss (Eq. 10) is identified in Appendix D by comparing mean LFC between days 2–3, 3–4, and 4–5 against a within-day baseline. The day 4–5 comparison uses precisely the t=5 measurements that are later held out as the test targets for the 11 non-steady-state perturbations. Thus the choice of which perturbations receive the Eq. (10) autoencoding regularizer is informed by the evaluation outcome, and the reported test MSE improvement in Figure 5 may restate the day-5 information used to select S rather than independently demonstrate the value of the mechanistic prior. No ablation is reported to control for the amount or type of extra regularization, so the pilot is confounded as evidence for the central empirical claim.
full rationale
The core derivation is a formal abstraction and is not equivalent to its inputs: F is defined by composition of perturbation, media, and waiting operators (Eq. 4); the necessary assumption (Eq. 5) is stated explicitly and flagged for validation; the supervised loss (Eq. 6) and steady-state loss (Eq. 10) are constructed from stated assumptions rather than fitted to any specific dataset. The VAE and causal-model connections are presented as demonstrations that popular architectures become special cases under explicit assumptions, not as predictions derived from the framework. Self-citations (Gaudelet et al. 2024, Peidli et al. 2024, Kovačević et al. 2024, Taylor-King et al. 2024) are used for background, benchmarks, and terminology, but no load-bearing argument reduces to a self-cited uniqueness theorem or ansatz. The only concerning step is the pilot evaluation: Appendix D selects the steady-state set S from the same days-2–5 time course that supplies the held-out t=5 test labels, and the paper does not state that this classification used only training days or an independent replicate. This is an evaluative circularity/leakage that weakens the proof-of-principle, but it does not make the mathematical framework circular by construction, so the score is 4 rather than 6 or higher.
Assumptions & free parameters
free parameters (4)
- NODE neural network parameters =
unknown (not reported)
- Steady-state subset S =
14 perturbations including non-targeting control
- Genes retained after filtering =
120 genes
- Regularization strength for steady-state loss =
unknown
assumptions (5)
- domain assumption Cell state is representable by a random variable X in an abstract space, and the experiment is a well-defined function F = W∘M∘P.
- domain assumption Unedited cells in baseline media do not respond over time, X_t = X.
- domain assumption The measurement map V has an inverse on the support of the data, X = supp(V(X)).
- ad hoc to paper Selected perturbations are at steady state within the observation window.
- standard math Batch errors are small enough for a first-order Taylor expansion.
Cite this review
Pith. "Pith review of No Foundations without Foundations -- Why semi-mechanistic models are essential for regulatory biology." pith.science (2026). https://pith.science/paper/ODH7AK5I
@misc{pith2026250119178,
author = {Pith},
title = {Pith review of: No Foundations without Foundations -- Why semi-mechanistic models are essential for regulatory biology},
year = {2026},
howpublished = {\url{https://pith.science/paper/ODH7AK5I}},
note = {Machine review of arXiv:2501.19178}
}
read the original abstract
Despite substantial efforts, deep learning has not yet delivered a transformative impact on elucidating regulatory biology, particularly in the realm of predicting gene expression profiles. Here, we argue that genuine "foundation models" of regulatory biology will remain out of reach unless guided by frameworks that integrate mechanistic insight with principled experimental design. We present one such ground-up, semi-mechanistic framework that unifies perturbation-based experimental designs across both in vitro and in vivo CRISPR screens, accounting for differentiating and non-differentiating cellular systems. By revealing previously unrecognised assumptions in published machine learning methods, our approach clarifies links with popular techniques such as variational autoencoders and structural causal models. In practice, this framework suggests a modified loss function that we demonstrate can improve predictive performance, and further suggests an error analysis that informs batching strategies. Ultimately, since cellular regulation emerges from innumerable interactions amongst largely uncharted molecular components, we contend that systems-level understanding cannot be achieved through structural biology alone. Instead, we argue that real progress will require a first-principles perspective on how experiments capture biological phenomena, how data are generated, and how these processes can be reflected in more faithful modelling architectures.
Figures
Forward citations
Cited by 1 Pith paper
-
When three experiments are better than two: Avoiding intractable correlated aleatoric uncertainty by leveraging a novel bias--variance tradeoff
An active-learning method using a bias-variance decomposition and a 'cobias-covariance' matrix with eigendecomposition-based batch selection outperforms BALD and least confidence on synthetic noise benchmarks.
Reference graph
Works this paper leans on
-
[1]
• For (nB > 1), as the ηb and ηb′ terms have opposite signs, the total error can be reduced by incorporating unperturbed cells in the baseline media into every batch. • For (nB > 1), the first term in the square brackets suggests that some batch effects are irreducible, however modelling FL via convex Lipschitz functions would be desirable if possible, be...
work page 2023
-
[3]
N., Ackerman, J., Alix, E., Ando, D
Chandrasekaran, S. N., Ackerman, J., Alix, E., Ando, D. M., Arevalo, J., Bennion, M., Boisseau, N., Borowa, A., Boyd, J. D., Brino, L., et al. Jump cell painting dataset: morpho- logical impact of 136,000 chemical and genetic perturba- tions. BioRxiv, pp. 2023–03,
work page 2023
-
[4]
M., Cudini, J., Kapourani, C.-A., Uhler, C., and Edwards, L
Gaudelet, T., Del Vecchio, A., Carrami, E. M., Cudini, J., Kapourani, C.-A., Uhler, C., and Edwards, L. Season combinatorial intervention predictions with salt & peper. arXiv preprint arXiv:2404.16907,
-
[5]
Predicting single-cell per- turbation responses for unseen drugs
Hetzel, L., B ¨ohm, S., Kilbertus, N., G ¨unnemann, S., Lot- follahi, M., and Theis, F. Predicting single-cell per- turbation responses for unseen drugs. arXiv preprint arXiv:2204.13545,
-
[8]
On neural differential equations
Kidger, P. On neural differential equations. arXiv preprint arXiv:2202.02435,
-
[10]
Causal modeling with stationary diffusions
Lorch, L., Krause, A., and Sch¨olkopf, B. Causal modeling with stationary diffusions. In International Conference on Artificial Intelligence and Statistics, pp. 1927–1935. PMLR,
work page 1927
-
[11]
Magliacane, S., Claassen, T., and Mooij, J. M. Joint causal inference on observational and experimental datasets. arXiv preprint arXiv:1611.10351,
-
[13]
Powell, J. E., Xue, A., Yazar, S., Alquicira, J., Cuomo, A., Senabouth, A., Gordon, G., Kathail, P., Ye, J., and Hewitt, A. Genetic variants associated with cell-type-specific intra-individual gene expression variability reveal new mechanisms of genome regulation. bioRxiv, pp. 2024–05,
work page 2024
Show all 19 references
-
[16]
M., Nassar, M., Osi´nski, B., Eksi, R., Zhang, K., and Graepel, T
Wu, Y ., Wershof, E., Schmon, S. M., Nassar, M., Osi´nski, B., Eksi, R., Zhang, K., and Graepel, T. Perturbench: Bench- marking machine learning models for cellular perturba- tion analysis. arXiv preprint arXiv:2408.10609,
-
[17]
14 No Foundations without Foundations — Why semi-mechanistic models are essential for regulatory biology A. Optional assumption II: Genetic perturbations do not induce responses in baseline media The steady state assumption detailed in Section 2.3.2 may be too extreme for many...
2021
-
[19]
This is repeated five times to obtain the final baseline
The baseline LFC is obtained by randomly splitting cells with the non-targeting guide for each day into two groups and computing the LFC between the two splits. This is repeated five times to obtain the final baseline. 18 No Foundations without Foundations — Why semi-mechanist...
2023
-
[2005]
Simulation-based benchmarking for causal structure learning in gene perturbation experiments
Kovaˇcevi´c, L., Newsham, I., Mukherjee, S., and Whittaker, J. Simulation-based benchmarking for causal structure learning in gene perturbation experiments. arXiv preprint arXiv:2407.06015,
-
[2016]
Learning identifiable factor- ized causal representations of cellular responses
Mao, H., Lopez, R., Liu, K., Huetter, J.-C., Richmond, D., Benos, P., and Qiu, L. Learning identifiable factor- ized causal representations of cellular responses. arXiv preprint arXiv:2410.22472,
-
[2017]
Perteval-scfm: Benchmark- ing single-cell foundation models for perturbation effect prediction
Wenteler, A., Occhetta, M., Branson, N., Huebner, M., Curean, V ., Dee, W., Connell, W., Hawkins-Hooker, A., Chung, P., Ektefaie, Y ., et al. Perteval-scfm: Benchmark- ing single-cell foundation models for perturbation effect prediction. bioRxiv, pp. 2024–10,
2024
-
[2019]
P., Edwards, L., and Gaudelet, T
Scherer, P., Pouplin, A., Del Vecchio, A., M S, S., Bolton, O., Soman, J., Taylor-King, J. P., Edwards, L., and Gaudelet, T. Pyrelational: a python library for ac- tive learning research and development. arXiv preprint arXiv:2205.11117,
-
[2020]
R., Dunn, S.-J., Bornschein, J., Chiappa, S., Rey, M., Lespiau, J.-B., Cassirer, A., Wang, J., Weber, T., Barrett, D., et al
Ke, N. R., Dunn, S.-J., Bornschein, J., Chiappa, S., Rey, M., Lespiau, J.-B., Cassirer, A., Wang, J., Weber, T., Barrett, D., et al. Discogen: Learning to discover gene regulatory networks. arXiv preprint arXiv:2304.05823,
-
[2021]
B., Mesbahi, Y
Bendidi, I., Whitfield, S., Kenyon-Dean, K., Yedder, H. B., Mesbahi, Y . E., Noutahi, E., and Denton, A. K. Bench- marking transcriptomics foundation models for perturba- tion analysis: one pca still rules them all. arXiv preprint arXiv:2410.13956,
-
[2023]
Systematic reconstruction of molecular pathway signatures using scalable single-cell perturbation screens
Jiang, L., Dalgarno, C., Papalexi, E., Mascio, I., Wessels, H.-H., Yun, H., Iremadze, N., Lithwick-Yanai, G., Lipson, D., and Satija, R. Systematic reconstruction of molecular pathway signatures using scalable single-cell perturbation screens. bioRxiv, pp. 2024–01,
2024
-
[2024]
Deep learning-based predictions of gene perturbation effects do not yet outperform simple linear methods
Ahlmann-Eltze, C., Huber, W., and Anders, S. Deep learning-based predictions of gene perturbation effects do not yet outperform simple linear methods. BioRxiv, pp. 2024–09,
2024
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.