Pith. sign in

REVIEW 3 major objections 4 minor 10 references

A multi-modal neural network for learning cis and trans regulation of stress response in yeast

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A neural network that reads promoter DNA and regulator expression predicts yeast stress-response gene expression across many conditions, beating motif-based ensemble methods and approximating a real transcription-factor knockout.

desk verdict The performance claim is inflated by a leaky data split, but the in-silico knockout validation is a genuinely useful part. read the letter →

arxiv 1908.09426 v1 pith:ZY6PQEHB submitted 2019-08-26 q-bio.GN

Boxiang Liu , Nadine Hussami , Avanti Shrikumar , Tyler Shimko , Salil Bhate , Scott Longwell , Stephen Montgomery , Anshul Kundaje This is my paper · ORCID
classification q-bio.GN
keywords geneexpressionpredictioncis-regulatorysequencetransfactormulti-modalneuralnetworkyeaststressresponseSaccharomycescerevisiaein-silicoknockouttranscriptionmotif
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a single multi-modal neural network can predict the stress-response expression of any yeast gene in any stress condition from two inputs: the raw promoter DNA sequence of the gene and the expression levels of 472 transcription factors and signaling molecules in that condition. In the authors' tests on a dataset of 6,100 genes across 173 stress conditions, the classification version reaches 79.5% accuracy, compared with 62.9% for the best motif-based ensemble baseline, and the regression version achieves a Pearson correlation of 0.845 between predicted and measured expression changes. The paper's deeper claim is that cis and trans regulatory information can be learned jointly from raw data, without predefined transcription-factor motif compendia, and that the learned model can simulate regulatory perturbations. This matters because a predictive model of gene regulation would let researchers anticipate how sequence variants or transcription-factor knockouts change expression across cellular states.

What carries the argument

The load-bearing mechanism is the multi-modal neural network architecture: a convolutional cis module takes 1 kb of promoter DNA as a one-hot encoded four-channel sequence and learns motif-like filters de novo; a dense trans module takes the expression of 472 known regulators and compresses it; an integration module concatenates the two representations through dense layers before a linear neuron (regression) or softmax neuron (classification). Reverse-complement parameter sharing lets each filter recognize a motif and its reverse complement. The integration step is what enables cis-trans interaction: the sequence features learned by the convolutional layers can be modulated by the condition-specific regulator state, which is what allows one gene to be predicted under many different stresses.

What would settle it

Retrain the identical architecture on the identical data but split by condition instead of by pair: hold out all pairs belonging to, say, 17 of the 173 stress conditions and test only on those held-out conditions. If accuracy on held-out conditions remains near 79.5% and regression correlation near 0.845, the generalization claim holds; if accuracy falls toward the 62.9% motif baseline, the model is largely memorizing per-condition expression means. The mirror-image test holds out entire genes rather than conditions.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a convolutional module reading raw promoter sequence, combined with a dense module reading trans-factor expression, learns regulatory programs that outperform existing models built on known motifs. The network's filters recover several known yeast stress-response motifs, and gradient-based importance scoring ranks known regulators such as MSN2/4 and TPK1 at the top. The paper also demonstrates an in-silico knockout of MSN2/4, in which the motif is masked and the trans-factor input is reduced, and shows that the predicted expression changes correlate with a real MSN2/4 knockout microarray experiment at Spearman 0.486. The authors interpret these results as evidence that joint cis-trans modeling from raw inputs can predict expression and perturbation response, laying a foundation for similar models in more complex organisms.

Load-bearing premise

The load-bearing premise is that a random 80-10-10 split of gene-condition pairs measures how well the model predicts genes and stress states it has not seen, even though every test gene and test condition already appeared in training; if that is not true, the reported accuracy and correlation do not support the paper's claim of predicting any gene in any condition.

Editorial extensions

If this is right

  • The reported results imply that predefined transcription-factor motif compendia are not required to build competitive regulatory models; raw DNA sequence plus regulator expression can substitute for engineered PWM features.
  • In-silico knockout becomes a viable screening tool: before running a wet-lab transcription-factor deletion, the model can rank which genes will change and in which direction.
  • The architecture is transferable to other organisms and other molecular readouts, provided the training data cover enough genes and conditions to learn cis and trans effects.
  • The gradient-based regulator importance scores provide a data-driven, genome-wide shortlist of candidate trans regulators for follow-up experiments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The random 80-10-10 split on gene-condition pairs allows every test gene and test condition to appear in training, so the headline numbers measure interpolation between memorized per-gene and per-condition means; a held-out-condition or held-out-gene split would be a stricter test of the 'any gene, any condition' claim.
  • Because a real knockout measures both direct and indirect effects while the in-silico knockout perturbs the motif and the trans-factor input directly, the reported Spearman 0.486 may understate accuracy on direct targets; comparing predictions to in-vivo TF binding maps, as the authors say they plan to do, would refine this.
  • The same model could be used for in-silico variant effect scoring: single-nucleotide promoter mutations could be evaluated for their predicted expression effect in every stress condition, an application the paper does not report but the architecture permits.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a multi-modal neural network that predicts yeast stress-response gene expression from two inputs: the raw 1 kb promoter sequence of a gene (cis module, a convolutional network) and the expression levels of 472 transcription factors and signaling molecules in a condition (trans module, a dense network). The two modalities are concatenated and integrated by dense layers, with a linear output for regression or a softmax output for three-class classification. The model is trained on the Gasch compendium of 6100 genes under 173 stress conditions, using an 80-10-10 split of (gene, condition) pairs. The authors report 79.5% classification accuracy versus 62.9% for BDTree and 60.9% for GeneClass, a test-set Pearson correlation of 0.845 for regression, recovery of known cis motifs by convolutional filter PWM analysis, and gradient-based ranking of trans regulators that includes known stress-response factors. Finally, they perform an in-silico MSN2/4 knockout by mutating its motif and reducing its expression 32-fold, and report a Spearman correlation of 0.486 between predicted and experimentally measured expression changes.

Significance. If the evaluation supported the claims, this would be a useful contribution: it demonstrates that a convolutional network on raw promoter sequence can be combined with trans-factor expression in a single differentiable model, that the learned cis filters recover known yeast motifs without prior motif annotation, and that the model can be probed by in-silico perturbation to generate falsifiable predictions about knockout effects. The framework is simple and reproducible in principle, and the external knockout comparison is an unusually direct validation step. However, the headline performance numbers are weakened substantially by the data-splitting protocol, which allows per-gene and per-condition memorization, and the in-silico knockout relies on an ad hoc perturbation whose numerical value is not justified. The central claim of predicting expression for any gene in any cellular state therefore requires a re-evaluation under held-out-gene and held-out-condition splits.

major comments (3)
  1. [Section 2.4, Table 1, Figure 1a] The random 80-10-10 split of (gene, condition) pairs does not test the central claim of predicting expression for any gene in any cellular state. Under this split, every test gene appears in training pairs with the same promoter sequence, and every test condition appears in training pairs with the same 472-dimensional trans vector, so the model can memorize gene-level and condition-level mean expression and combine them to score test pairs. The reported 79.5% accuracy and Pearson 0.845 therefore measure interpolation over seen genes and seen conditions, not out-of-sample generalization to unseen genes or unseen cellular states. The authors should report results under held-out-gene, held-out-condition, and held-out-combination splits, or justify why the central claim does not require such generalization.
  2. [Section 3.4, Figure 2b] The in-silico MSN2/4 knockout is implemented by replacing every instance of the motif AGGGG with NNNNN and reducing MSN2/4 expression by 32-fold, but the manuscript gives no justification for either the specific motif representation or the 32-fold reduction. The reported Spearman correlation of 0.486 is based on a single TF and a single hand-set perturbation magnitude, so it does not provide strong evidence that the model predicts perturbation effects generally. The authors should justify the 32-fold value from known biology, perform a sensitivity analysis over perturbation magnitudes and motif-alternative choices, and ideally report results for additional TFs.
  3. [Section 3.1, Table 1] The comparison with GeneClass and BDTree reports point estimates only, with no error bars, confidence intervals, or significance tests. Because the split is random, the apparent 16.6% improvement over BDTree could be affected by split variability and by the leakage described in the first comment. Repeated stratified splits or a paired test over multiple random seeds are needed before claiming that the deep model substantially outperforms the baselines.
minor comments (4)
  1. [Figure 1] The label 'grounth truth' is a typo and should read 'ground truth'.
  2. [Section 2.4] The discretization thresholds of +/-0.5 fold change are stated but their influence on the classification results is not discussed; a short sensitivity analysis or a reference to previous use of these thresholds would help.
  3. [Section 2.2] The trans module input is described as a vector in 'stress condition C', but uppercase C is inconsistent with the lowercase c used elsewhere; please standardize notation.
  4. [Section 3.2] The TomTom comparison against YEASTRACT is mentioned but the number of significantly matched known motifs and the chosen significance threshold are not reported, making it hard to assess how many of the learned filters correspond to known cis-regulatory elements.

Circularity Check

1 steps flagged · score 6.0 of 10

Target-expression leakage for TF genes makes a subset of the paper's predictions reduce to the input by construction; the main evaluation split is a generalization limitation rather than circularity.

  1. self definitional [Section 2 (problem formulation), Section 2.2 (trans regulator expression module), Section 2.4 (gene expression data)]
    "the goal is learn a model Eg,c = F(Sg, Tc) that can predict gene expression (Eg,c) of any gene (g) in any stress condition (c) ... The input to this module is a vector of expression levels (Tc) of 472 known transcription factors and signal molecules (kinases, phosphatases, receptors) in stress condition C."

    For any target gene g that is also one of the 472 trans factors, Eg,c is a component of the input vector Tc. The optimal model can simply copy that input dimension to the output, so those predictions are not derived from learned cis/trans regulatory rules. The dataset of 6100 genes therefore contains a subset of approximately 7-8% target genes whose predicted expression is present in the input by construction. This can partially inflate the reported classification accuracy (79.5%, Table 1) and regression correlation (0.845, Figure 1a). The paper does not exclude trans-factor genes from the target set or mask their expression in Tc.

full rationale

The paper is otherwise a standard supervised learning study: a convolutional cis module and dense trans module are trained to minimize a loss on the Gasch compendium, and the learned filters/importances are interpreted post hoc. The in-silico MSN2/4 knockout uses an externally defined motif replacement and an arbitrary 32-fold expression reduction; the text does not indicate that the 32-fold value or the target-gene list was tuned to the validation knockout microarray, so no circularity can be substantiated there on the present evidence. The 80-10-10 split of (gene, condition) pairs means every test gene and condition appears elsewhere in training, so the reported numbers measure interpolation over seen genes/conditions rather than generalization to unseen ones; this is a validity limitation, not a circular reduction, because the test labels are not by construction equal to the inputs. The only concrete circular structure is the trans-module input/target overlap: for the 472 trans factors, the predicted gene expression is literally one of the input features, so those predictions reduce to copying. This makes the paper partially circular, but the majority of target genes are not trans factors and their predictions retain independent content.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The model's performance depends on a handful of hand-selected quantities: the classification thresholds, the network hyperparameters, and especially the 32-fold expression reduction used in the in-silico MSN2/4 knockout, whose value is not justified and could have been chosen in light of the real knockout data. The modeling relies on domain assumptions that 1 kb of promoter sequence captures relevant cis regulation, that mRNA levels of 472 regulators are a faithful proxy for their activity, and that a motif mutation plus expression reduction approximates a true knockout. These assumptions are standard in the field but are not validated inside the paper.

free parameters (3)
  • MSN2/4 expression reduction in in-silico knockout = 32-fold
    The paper reduces MSN2/4 expression by 32-fold without justifying this value. If it was chosen by looking at the real knockout data, the Spearman 0.486 correlation is partly fitted.
  • Expression class discretization thresholds = -0.5 and +0.5 log2 fold-change
    Continuous expression values are binned into three classes using thresholds of -0.5 and +0.5. These hand-selected thresholds affect the classification accuracy reported in Table 1.
  • Network hyperparameters = 50 filters of size 9, 512 dense units, pool size 4, learning rate 0.01, momentum 0.5
    Architecture and training hyperparameters are hand-chosen standard choices. The reported performance depends on them, though they are not fitted to the target result in a direct way.
assumptions (4)
  • domain assumption A 1 kb upstream promoter sequence contains sufficient cis-regulatory information to predict stress-response expression changes
    The cis module uses only the 1 kb promoter; distal enhancers, chromatin state, and other regulatory layers are ignored, so predictive performance is bounded by this representation.
  • domain assumption mRNA expression levels of 472 trans factors are a valid proxy for their regulatory activity in a condition
    Many transcription factors are regulated post-translationally (localization, phosphorylation, degradation), so transcript levels may not capture activity. This is a known limitation in regulatory modeling.
  • standard math The Gasch et al. 2000 microarray compendium provides accurate, batch-consistent expression measurements for 6100 genes across 173 conditions
    The model is trained and evaluated entirely on this dataset. Measurement noise and batch effects are not modeled, and any systematic errors propagate into the reported performance.
  • ad hoc to paper Replacing the MSN2/4 motif AGGGG with NNNNN and reducing its expression by 32-fold simulates a knockout
    This perturbation is proposed in Section 3.4 as an in-silico knockout, but a real knockout abolishes the protein, while the model perturbation only weakens one motif and lowers the transcript level. The mapping is approximate and not validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A multi-modal neural network for learning cis and trans regulation of stress response in yeast." pith.science (2026). https://pith.science/paper/ZY6PQEHB

@misc{pith2026190809426,
  author       = {Pith},
  title        = {Pith review of: A multi-modal neural network for learning cis and trans regulation of stress response in yeast},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZY6PQEHB}},
  note         = {Machine review of arXiv:1908.09426}
}
read the original abstract

Deciphering gene regulatory networks is a central problem in computational biology. Here, we explore the use of multi-modal neural networks to learn predictive models of gene expression that include cis and trans regulatory components. We learn models of stress response in the budding yeast Saccharomyces cerevisiae. Our models achieve high performance and substantially outperform other state-of-the-art methods such as boosting algorithms that use pre-defined cis-regulatory features. Our model learns several cis and trans regulators including well-known master stress response regulators. We use our models to perform in-silico TF knock-out experiments and demonstrate that in-silico predictions of target gene changes correlate with the results of the corresponding TF knockout microarray experiment.

Figures

Figures reproduced from arXiv: 1908.09426 by the authors.

Figure 1
Figure 1. (a) Predicted vs grounth truth (b-e) examples of recovered motifs [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. (a) The expression of MSN2/4 known target gene experience larger change under stress [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

10 extracted references · 10 canonical work pages

  1. [1]

    Regulatory element detection using correla- tion with expression

    Harmen J Bussemaker, Hao Li, and Eric D Siggia. Regulatory element detection using correla- tion with expression. Nature Genetics, 27(2):167–174, February 2001

    work page 2001

  2. [2]

    Regression trees for regulatory element identification

    Tu Minh Phuong, Doheon Lee, and Kwang Hyung Lee. Regression trees for regulatory element identification. Bioinformatics, 20(5):750–757, March 2004

    work page 2004

  3. [3]

    Towards reconstruction of gene networks from expression data by supervised learning

    Lev A Soinov, Maria A Krestyaninova, and Alvis Brazma. Towards reconstruction of gene networks from expression data by supervised learning. Genome biology, 4(1):R6, January 2003

    work page 2003

  4. [4]

    Module networks: identifying regulatory modules and their condition-specific regulators from gene expression data

    Eran Segal, Michael Shapira, Aviv Regev, Dana Pe’er, David Botstein, Daphne Koller, and Nir Friedman. Module networks: identifying regulatory modules and their condition-specific regulators from gene expression data. Nature Genetics, 34(2):166–176, June 2003

    work page 2003

  5. [5]

    Predicting genetic regulatory response using classification

    Manuel Middendorf, Anshul Kundaje, Chris Wiggins, Yoav Freund, and Christina Leslie. Predicting genetic regulatory response using classification. Bioinformatics, 20(suppl 1):i232– i240, August 2004

    work page 2004

  6. [6]

    A bi-dimensional regression tree approach to the modeling of gene expression regulation

    Jianhua Ruan and Weixiong Zhang. A bi-dimensional regression tree approach to the modeling of gene expression regulation. Bioinformatics, 22(3):332–340, February 2006

    work page 2006

  7. [7]

    Basset: learning the regulatory code of the accessible genome with deep convolutional neural networks

    David R Kelley, Jasper Snoek, and John L Rinn. Basset: learning the regulatory code of the accessible genome with deep convolutional neural networks. Genome research, 26(7):990–999, July 2016

    work page 2016

  8. [8]

    Genomic expression programs in the response of yeast cells to environmental changes

    A P Gasch, P T Spellman, C M Kao, O Carmel-Harel, M B Eisen, G Storz, D Botstein, and P O Brown. Genomic expression programs in the response of yeast cells to environmental changes. Molecular Biology of the Cell , 11(12):4241–4257, December 2000

    work page 2000

Show all 10 references
  1. [9]

    Reverse-complement parameter sharing improves deep learning models for genomics

    Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Reverse-complement parameter sharing improves deep learning models for genomics. bioRxiv, 2017

    2017

  2. [10]

    Fine-tuning of the Msn2/4- mediated yeast stress responses as revealed by systematic deletion of Msn2/4 partners

    A Sadeh, N Movshovich, M V olokh, L Gheber, and A Aharoni. Fine-tuning of the Msn2/4- mediated yeast stress responses as revealed by systematic deletion of Msn2/4 partners. - PubMed - NCBI. Molecular Biology of the Cell , 22(17):3127–3138, August 2011. 5

    2011

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.