{"id":"8f742daa-5605-44cc-9f30-4787ebccbd0f","arxiv_id":"2501.19178","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A semi-mechanistic model of CRISPR perturbation screens, built from editing, media, and waiting operations, improves gene-expression prediction when trained with an extra steady-state constraint.","lead":"A new framework describes gene-editing experiments as ordered steps, edit, change growth medium, wait, and uses that structure to make machine learning models of cell behavior better. The paper shows that adding a simple steady-state rule to the training loss can improve predictions of how cells respond to unseen gene edits.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proof-of-principle in §6 is confounded: the steady-state set S is selected from the same time course used for evaluation, and no non-mechanistic regularizer control is reported.","rationale":"The reader's weakest-assumption analysis targets Eq. (5), the necessary assumption that unedited cells do not respond to baseline media. That is a clearly stated but unvalidated assumption, and I agree it matters. However, my concern is more directly load-bearing for the paper's quantitative claim: even if Eq. (5) holds in the Ishikawa dataset, the Section 6 comparison does not establish that the mechanistic steady-state prior causes the improvement, because the steady-state subset is selected from the evaluation time course and no non-mechanistic regularization control is reported. The paper's formal contribution—composing P, M, W and exposing hidden assumptions in VAEs and causal models—is coherent and useful, and the experimental recommendations are sensible. The proof-of-principle is the only quantitative evidence, and its design leaves two unresolved confounds: post hoc selection of S from the evaluation time course, and a missing control for generic regularizers. The appropriate verdict remains CONDITIONAL, as the reader concluded; I do not propose REJECT because the central argument is largely conceptual and the concerns are in the supporting pilot, not in the internal consistency of the formalism.","tokens_in":21513,"tokens_out":7602,"duration_ms":75381,"concrete_test":"Re-run the §6 experiment with S fixed by a pre-registered rule using only days 2–4 (e.g., mean absolute LFC between consecutive training days below a threshold), evaluate on day 5 for the 11 non-steady perturbations, and include a matched control where the same number of additional loss terms are non-mechanistic (e.g., L2 weight decay on the NODE parameters or self-consistency on randomly chosen non-steady states). If the modified loss no longer beats the unmodified loss by a significant margin, or is matched by generic regularization, the claimed benefit of the mechanistic steady-state prior is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The empirical case for the central claim rests on §6, where the steady-state loss (Eq. 10) reportedly improves NODE test MSE on the Ishikawa et al. (2023) iPSC data. That comparison does not yet isolate the mechanistic prior as the cause of improvement. First, the steady-state subset S ('14 perturbations ... that appear to converge on a steady state') is identified in Appendix D from the same days-2–5 time course later used for training and testing; the paper does not state whether the steady/non-steady classification uses only training days or an independent replicate. If day-5 measurements contribute to selecting S and to labeling the 11 'non-steady state' test perturbations, then the mechanistic prior is constructed with access to test outcomes, so the gap in Fig. 5 could reflect target leakage rather than the value of the prior. Second, no ablation controls for the number or type of extra loss terms: adding L2 weight decay, dropout, or a generic output-invariance/smoothing term of comparable strength may produce the same faster convergence and lower MSE. Without these controls, the demonstration conflates 'mechanistic prior' with 'additional regularization on a small (100-point) pseudobulk dataset.' The paper explicitly says its assumptions need validation; the pilot does not show this validation was performed out-of-sample.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that foundation models for regulatory biology need to be grounded in mechanistic models of how perturbation experiments are conducted. The authors formalize an in vitro perturb-seq experiment as a composition of operators P (genetic perturbation), M (media change), and W (waiting), yielding the function F in Eq. (4). Under a 'necessary assumption' that unedited cells in baseline media do not change over time (Eq. 5), they derive a supervised loss (Eq. 6); under an optional steady-state assumption they add a loss term (Eq. 10). They then connect this framework to variational autoencoders (Section 5.1) and causal models (Section 5.2), and analyze batch effects in Appendix B, recommending that unperturbed controls be included in every batch. The empirical proof-of-principle in Section 6 trains a neural ODE on pseudobulked iPSC data from Ishikawa et al. (2023), reporting that enforcing steady states improves test MSE for 11 held-out non-steady perturbations.","tokens_in":21783,"tokens_out":5617,"duration_ms":46518,"significance":"If the framework's utility is borne out by further validation, it would provide a principled way to design loss functions and experimental batches for perturbation modeling, and would clarify the hidden assumptions in existing methods. The formal derivation of the loss structure from the experimental protocol is a conceptual contribution, and the demonstration of links between the framework and VAEs/causal models is interesting. However, the paper's central empirical claim — that the steady-state loss improves predictive performance over standard training — is currently supported only by a single dataset with a post hoc selected steady-state set and no non-mechanistic regularizer control. The strength of the claim in the abstract therefore exceeds what the evidence establishes.","major_comments":[{"comment":"The steady-state subset S is selected from the same days-2–5 time course that is later used for training and testing. Appendix D shows that the classification of the 14 steady-state perturbations is based on absolute LFC between sequential days, including the day-4-to-day-5 comparison; Section 6 then evaluates on the 11 'non-steady state' perturbations at t=5. If day-5 measurements contribute to deciding which perturbations are steady (and therefore which are non-steady and held out), the improved test MSE in Fig. 5 could reflect target leakage rather than a benefit of the mechanistic prior. The paper does not state whether the steady-state labeling uses only training days or an independent replicate. Please clarify the exact selection procedure and, if leakage is possible, re-run the experiment with S fixed a priori from a separate experiment or from a random split that does not use test-day information.","section":"Section 6 / Appendix D"},{"comment":"The comparison between the 'original loss' and the 'modified loss' does not control for the presence of an additional loss term. Adding the steady-state loss (Eq. 10) introduces extra supervision on the training set; a generic regularizer of comparable strength — e.g., L2 weight decay, dropout, or an output-smoothing term — could plausibly produce the same faster convergence and lower test MSE on a 100-point pseudobulk dataset. Without ablations that match the number, type, and strength of extra loss terms, the improvement cannot be attributed specifically to the mechanistic steady-state prior. Please include such controls.","section":"Section 6 / Figure 5"},{"comment":"The 'necessary assumption' that unedited cells do not respond to baseline media is explicit, but it is not validated on the Ishikawa et al. (2023) dataset used in Section 6. If this assumption fails, the paired supervised loss in Eq. (6) is not a proper objective for the observed data, and the counterfactual alternative in Eq. (7) requires an inverse F^{-1} whose existence is itself assumed. The paper's experimental recommendations (Section 2.3.1) call for verifying this assumption, yet the proof-of-principle does not report the stability of non-targeting controls across days. Please report this stability check or otherwise justify the assumption for the dataset analyzed.","section":"Section 2.3.1 / Eq. (5)"},{"comment":"The neural ODE model and training protocol are not described in sufficient detail. The paper does not specify the network architecture, the number of parameters, the optimizer, the learning rate schedule, the weight of the steady-state loss, or the hyperparameter selection procedure. Without these details and without code, the proof-of-principle is not reproducible, and it is unclear how sensitive the improvement in Fig. 5 is to the chosen regularization strength. Please provide full experimental details or release code.","section":"Section 6"}],"minor_comments":[{"comment":"The notation X^{pγ}_{m,t} is used before the sub/superscript convention is explained; define it immediately before Eq. (4).","section":"Section 2.2 / Eq. (4)"},{"comment":"The sentence ending 'respectfully' should be 'respectively,' and the spaces N0^{n_W n_G} and N0^{n_G} should be introduced before use.","section":"Section 3.1 / Eq. (11)"},{"comment":"The phrase 'to the extent that V has an inverse' is vague for a count-valued measurement function; clarify what is meant by V^{-1} in this setting.","section":"Section 3.1 / after Eq. (13)"},{"comment":"The latent function L in the VAE derivation conflicts with the loss L used elsewhere; use a different symbol (e.g., h).","section":"Section 5.1 / Eq. (18)"},{"comment":"The statement 'pseudobulk single-cell data over (pγ, t) pairs giving 100 unique data points' should state the number of perturbations and time points explicitly so the reader can verify the count.","section":"Section 6"},{"comment":"The symbol V is used both for the measurement function (Eq. 11) and for the valid set; rename one of them.","section":"Appendix B / Eq. (23)"},{"comment":"The phrase 'non-differentiating cellular models' is potentially confusing; consider using 'cells that do not differentiate over the experimental window.'","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper reads more like a perspective proposing a framework with a preliminary proof-of-principle than a fully validated empirical study. The formal contributions are interesting, but the empirical section would need to rule out leakage and add non-mechanistic controls before the central claim can be accepted. If the authors cannot provide those, the empirical claim should be softened. The absence of code or detailed architecture also makes the empirical result hard to assess."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the formalism is the contribution, and it is a real one. The paper gives the regulatory-biology ML community a clean compositional language for what happens in a perturb-seq experiment — edit, change media, wait — with the non-commutativity and the hidden assumptions made explicit. The derivation from equation (4) through the loss functions is coherent, and the observation that VAE-style models and structural causal models fall out as special cases is genuinely instructive. I learned something from how they trace the paired-data assumption back to equation (5). The paper is also honest about its necessary and optional assumptions and translates them into concrete experimental recommendations, which is rarer than it should be.\n\nThe soft spots are all on the empirical side. The proof-of-principle in Section 6 is a pilot, not evidence. One pseudobulk dataset, no architecture details, no code, no hyperparameter reporting, and no ablation that rules out the explanation that any extra regularization term would produce the same faster convergence and lower test MSE. The stress-test concern about the steady-state subset lands: S is selected from the same days 2–5 time course that supplies the test labels, and the paper does not state that the steady/non-steady classification was done using training days only. That makes the reported gap in Figure 5 consistent with target leakage rather than with the value of the mechanistic prior. The authors explicitly flag their assumptions as needing validation, but the pilot does not show that validation being done out-of-sample.\n\nThe math and the citation pattern look solid. The compositional abstraction is a new synthesis, not a repackaging of existing results; the derivation of VAEs and causal models as special cases is their own argument and it checks out. The paper engages fairly with the recent benchmarks showing simple linear methods often beat deep models. I do not see a circularity problem in the formal core.\n\nWho should read this: anyone building or benchmarking ML models for perturbation screens, and anyone designing perturb-seq experiments. They will get a useful conceptual frame, a checklist of assumptions to verify, and a batching error analysis that suggests practical experimental design. It is not a validated method paper. It deserves a serious referee, not a desk reject, because the framework is likely to be built on even if the pilot is reworked or removed. For peer review: request code, architecture details, a pre-registered or training-only steady-state selection, and an ablation with a generic regularizer of comparable strength.","headline":"The compositional framework is a genuinely useful new synthesis for perturb-seq modelling; the empirical pilot is too under-specified and potentially leaky to carry the weight of the central claim, so treat the paper as a principled proposal, not a validated method.","tokens_in":22315,"tokens_out":2856,"would_cite":true,"duration_ms":27655,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Foundation models of regulatory biology will stay out of reach unless they encode the order of perturbation, media, and time; this paper builds a semi-mechanistic framework to do so.","keywords":["regulatory biology","foundation models","CRISPR screens","perturb-seq","semi-mechanistic models","gene expression prediction","neural ODEs","batch effects"],"falsifier":"Take any perturb-seq dataset used to train a perturbation predictor and compare the transcriptomes of unedited cells harvested at day 0 with those of unedited cells in baseline media at the end time $t$, using the same within-day-split baseline the paper uses for its iPSC analysis. If the end-time distribution separates from the day-0 distribution by more than technical noise, the necessary assumption $F(X,\\cdot,\\cdot,t)=X$ fails and the paper's recommended loss is not applicable to that experiment; conversely, a dataset that passes this check and contains known steady states should reproduce the reported MSE gap between Eq. (6) and Eq. (10).","tokens_in":21335,"feed_emoji":"🧬","tokens_out":8274,"duration_ms":73218,"temperature":0.7,"pith_summary":"This paper argues that the stalled progress in predicting gene-expression responses to perturbations is not mainly a data problem: it is a modelling problem. The authors propose that any reliable “foundation model” of regulatory biology must first encode how a CRISPR experiment actually works, as a sequence of non-commuting operations — edit the gene, change the media, wait. From that sequence they derive loss functions and expose an assumption, often unstated, that unedited cells do not respond to baseline media; when it fails, standard supervised training becomes counterfactual prediction. On an iPSC time-course they show that adding a steady-state term to the loss improves a neural-ODE model’s predictions of unseen perturbations and stabilises training. If the argument is correct, future perturbation models should be evaluated by whether they respect experimental structure, not just by architecture or data volume.","feed_headline":"A steady-state loss sharpens CRISPR prediction models","feed_subtitle":"Paper derives a semi-mechanistic map of edit-media-time and shows it improves neural ODE forecasts of unseen perturbations.","key_machinery":"The load-bearing object is the experiment-as-function $F=W\\circ M\\circ P$, with each operation defined as a map on cell-state random variables: $P(X,p_\\gamma)=X_{p_\\gamma}$ applies the CRISPR edit, $M(X,m)=X_m$ applies a media condition, and $W(X,t)=X_t$ waits. The argument advances by asking which paths in the four-branch experimental diagram can be treated as identity, deriving the loss in Eq. (6) from the necessary assumption $F(X,\\cdot,\\cdot,t)=X$ and the additional regularizer in Eq. (10) from the assumption that some perturbed populations are dynamical attractors. The proof of principle instantiates $F$ as a neural ODE, $F(x,p_\\gamma,t)=x_0+\\int_0^t G(x,p_\\gamma,s)\\,ds$, and the error analysis models batch effects as zero-mean additive noise on top of the measurement function $V$, leading to the recommendation to replicate baseline controls across batches.","core_discovery":"The central claim is that the genotype-to-phenotype mapping measured in a perturb-seq screen is the composed function $F(X,p_\\gamma,m,t)=W(M(P(X,p_\\gamma),m),t)$, where $P$ applies a CRISPR perturbation, $M$ switches media conditions, and $W$ waits $t$ units of time, in that order. Writing the experiment this way makes the non-commutativity of biological operations explicit and brings hidden assumptions to the surface: the standard supervised loss (Eq. 6) is only valid if unedited cells do not respond to baseline media, $F(X,\\cdot,\\cdot,t)=X$; otherwise paired measurements are counterfactual and require an inverse $F^{-1}$. The paper then derives a “steady-state” regularizer (Eq. 10): wherever biology dictates that a perturbed population is an attractor, the loss demands $F(X^{p_\\gamma}_{m,t},\\cdot,\\cdot,s)=X^{p_\\gamma}_{m,t}$, and the proof of principle shows that this term lowers test MSE and reduces seed-to-seed variance for a neural ODE on an iPSC dataset. Variational autoencoders and structural causal models are shown to be special cases of the same composition, and the batch-error analysis yields a concrete experimental prescription: include unperturbed baseline cells in every sequencing batch.","pith_inferences":["Editorial inference: if the necessary-assumption check becomes standard practice, several published perturb-seq benchmarks may need re-scoring, because models trained on drifting baseline cells could be fitting counterfactual structure rather than biology.","Editorial inference: the steady-state regularizer should transfer to RNA-velocity and cell-differentiation models—any dataset with time series and known terminal states is a candidate for the same term, not just CRISPR screens.","Editorial inference: a direct quantitative prediction is that the performance gap between Eq. (6) and Eq. (10) should grow with the number of perturbed populations that are at steady state and with the duration of the experiment relative to relaxation time; both are testable by re-running the NODE experiment on datasets with longer time courses.","Editorial inference: the framework suggests a natural benchmark protocol—per-dataset verification of $F(X,\\cdot,\\cdot,t)=X$ and known attractors—before comparing foundation models; without it, leaderboard positions conflate architecture quality with baseline drift."],"forward_implications":["Established architectures—VAE-style perturbation autoencoders and structural causal models of perturbation screens—are recovered as special cases of the unified $F$-composition, so their design choices can be compared by the assumptions they make about $P$, $M$, and $W$.","Datasets must be checked for the necessary assumption before being used to train or benchmark perturbation models; if unedited cells drift in baseline media, Eq. (6) is a counterfactual loss, not a supervised one.","Enforcing biologically known steady states acts as a regularizer that improves prediction for other, non-steady-state perturbations, which is a concrete, testable benefit of injecting mechanism.","Batch design can be optimised from the error analysis: placing unperturbed baseline cells in every batch reduces total prediction error because the $\\eta_b$ and $\\eta_{b'}$ terms enter with opposite signs."],"supporting_citations":[{"why":"The key ML method that, in the paper's view, implicitly assumes unedited cells do not respond to baseline media; this is the target of the necessary-assumption critique.","marker":"Roohani et al., 2022"},{"why":"Supplies the iPSC pooled CRISPR time-series dataset (days 2–5) used in the proof-of-principle NODE experiment.","marker":"Ishikawa et al., 2023"},{"why":"Provides the neural ODE architecture used to represent $F$ in the proof of principle.","marker":"Chen et al., 2018a"},{"why":"Introduces perturb-seq, the pooled CRISPR screen with single-cell RNA readout that the mathematical model abstracts.","marker":"Dixit et al., 2016"},{"why":"Documented evidence that simple linear methods still outperform deep learning on perturbation prediction, the motivation for the semi-mechanistic argument.","marker":"Ahlmann-Eltze et al., 2024"},{"why":"Another benchmarking study showing deep models do not yet beat simple baselines, reinforcing the paper's motivation.","marker":"Wu et al., 2024"},{"why":"Provides the in vivo differentiating haematopoiesis example used to motivate the framework for differentiating systems and pseudotime labelling.","marker":"Lara-Astiaso et al., 2023"},{"why":"Harmonized single-cell perturbation datasets used to support the statement that most perturb-seq datasets are non-differentiating and to define the baseline for benchmarking.","marker":"Peidli et al., 2024"}],"fun_headline_variants":["Steady-state loss sharpens CRISPR forecast accuracy","Semi-mechanistic priors improve CRISPR gene predictions","Why mechanistic models are key for CRISPR predictive power","New regularizer boosts neural ODE gene expression forecasts","Steady-state term cuts error in CRISPR perturbation models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework's loss functions collapse if unedited cells in baseline media are not statistically stationary over the experiment's time window, because the paper's central pathway $F(X,\\cdot,\\cdot,t)=X$ is then false and every paired prediction becomes a counterfactual requiring an invertible map.","fun_headline_variants_meta":{"raw":{"variants":["Steady-state loss sharpens CRISPR forecast accuracy","Semi-mechanistic priors improve CRISPR gene predictions","Why mechanistic models are key for CRISPR predictive power","New regularizer boosts neural ODE gene expression forecasts","Steady-state term cuts error in CRISPR perturbation models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000167,"raw_usage":{"total_tokens":1298,"prompt_tokens":1025,"completion_tokens":273,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":641,"completion_tokens_details":{"reasoning_tokens":198}},"tokens_in":641,"tokens_out":273,"duration_ms":3338,"temperature":1.0,"reasoning_tokens":198,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T21:01:17.649827+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take any perturb-seq dataset used to train a perturbation predictor and compare the transcriptomes of unedited cells harvested at day 0 with those of unedited cells in baseline media at the end time $t$, using the same within-day-split baseline the paper uses for its iPSC analysis. If the end-time distribution separates from the day-0 distribution by more than technical noise, the necessary assumption $F(X,\\cdot,\\cdot,t)=X$ fails and the paper's recommended loss is not applicable to that experiment; conversely, a dataset that passes this check and contains known steady states should reproduce the reported MSE gap between Eq. (6) and Eq. (10).","supporting_citations":[{"cited_title":"Deep learning-based predictions of gene perturbation effects do not yet outperform simple linear methods","cited_arxiv_id":null,"evidence_quote":"Documented evidence that simple linear methods still outperform deep learning on perturbation prediction, the motivation for the semi-mechanistic argument."}],"review_version":1}