REVIEW 3 major objections 4 minor 1 cited by
Sparsity is All You Need: Rethinking Biological Pathway-Informed Approaches in Deep Learning
T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read This paper claims that replacing biological pathway priors with random priors that preserve network sparsity leaves predictive performance unchanged in deep learning models, so the benefits attributed to pathway knowledge are mostly a…
desk verdict A genuinely useful systematic benchmark, but the randomization null does not isolate biological content from topology; the paper deserves peer review after reworking its claims and adding structural checks. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the null model generated by the randomization procedure. Starting from the binary gene–pathway association matrix $A$, the method permutes the 1-entries while preserving the total number of connections $C$ and each pathway's column sum $d_j$ (Equations 4 and 5), then uses the shuffled matrix in place of the real pathway prior in the model's architecture or input representation. This isolates the sparsity induced by pathways from the specific biological identities of the connections. A second knob varies sparsity level directly, pruning 60–99% of connections, to test whether biological levels are optimal. The comparison metric is whether the randomized or pruned networks match or beat the biologically constrained ones.
What would settle it
Train a single pathway-informed model on a dataset where the true pathway graph is known to be functionally important, and compare it with an ensemble of random graphs matched not only on total edges and per-pathway degree but also on the full feature-degree sequence and largest-component structure; if the true graph consistently and significantly outperforms every matched randomization across repeated seeds on held-out data, the claim that biology adds nothing beyond sparsity is falsified.
Extended reading notes
Core claim
The central claim is that pathway-informed neural networks owe their reported predictive success to the sparse connectivity that pathway annotations impose, not to the biological content of those annotations. The evidence is a controlled comparison: for each of 15 state-of-the-art models with available code, the authors randomize the gene–pathway matrix while preserving the total number of connections and the number of features assigned to each pathway, leaving the network sparsity structure intact. Randomized models perform statistically indistinguishably from pathway-informed models on every task, and three models (MPVNN, DeepKEGG, PathDNN) do significantly better when the biological labels are shuffled. Interpreting feature importance with SHAP and related methods shows randomized models recover the same disease-associated genes as the original models. The paper also proposes a general benchmarking workflow whereby any new pathway-informed architecture must beat its randomized counterpart before claiming a biological advantage.
Load-bearing premise
The comparison assumes that shuffling pathway labels removes biological meaning while leaving every other performance-relevant property unchanged; in practice the randomization preserves only total edge count and per-pathway degree, so changes in gene degree distribution, hubs, and connectivity could be doing some of the work attributed to sparsity.
Editorial extensions
If this is right
- Reported gains from pathway integration in existing deep learning models cannot be attributed to biological relevance unless the model beats a degree-preserving randomized prior.
- Sparsity should be treated as a tunable hyperparameter rather than a fixed property dictated by pathway annotations.
- Interpretability claims based on pathway-informed models need re-examination, because randomized models recover similar biomarkers.
- The proposed randomization benchmark can be applied across omics tasks and data modalities, including MLP-based, CNN-based, GNN-based, and transformer-based architectures.
- Future pathway-informed designs should test richer priors, such as protein–protein interaction networks or condition-specific pathway information, to see whether they outperform sparsity alone.
Reading between the lines
- An implication the authors leave implicit is that a stronger null model would randomize edges while exactly preserving the feature-degree sequence and connected-component structure, separating hub topology from raw sparsity; the current randomization preserves total edges and per-pathway degree but not the full graph statistics.
- Because pathway databases are static and incomplete, the result may not generalize to condition-specific or high-quality interaction networks; repeating the comparison with PPI-derived or single-cell-specific priors is a natural testable extension.
- A practical consequence beyond the paper is that model developers could save effort and avoid dependence on annotation databases by using random sparse masks with tuned sparsity, at least until a biological prior demonstrates an advantage in the same benchmark.
- The benchmark could become a standard control for publications proposing biology-informed architectures, analogous to a permutation test in statistics.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. Through a literature review of pathway-informed deep learning models, the authors identify 20 models with public code and compare 15 of them against randomized counterparts that permute pathway-feature associations while preserving, according to the stated formalization, the total number of connections and per-pathway degrees. Across tasks and metrics, they observe no significant performance advantage for pathway-informed models, with randomized versions significantly better in 3 of 15 cases. Additional experiments on five models examine fixed-seed robustness, sparsity levels, feature ablation, and biomarker interpretability. The paper proposes a benchmark workflow for testing whether pathway priors add value beyond sparsity.
Significance. If the core comparison were established with a null that fully controls all non-biological structural properties, the conclusion would be important: performance gains attributed to pathway knowledge in current architectures may be primarily a sparsity artifact, and the proposed randomized-baseline benchmark would be a useful community resource. The paper has concrete strengths: it evaluates 15 existing models with their original implementations, reports a 30-seed robustness check for five models, provides public code for the randomization procedure, and its central comparison is a direct empirical test rather than a circular derivation. The main caveats concern the null model's structural fidelity and multiple-testing interpretation, both of which are addressable in revision.
major comments (3)
- [Additional Information, Eqs. (4)-(5); Results, Pathway-Informed vs. Randomized Models] The randomization null preserves only the total number of connections C and the per-pathway degrees d_j; it does not preserve the per-gene degree sequence, hub structure, largest connected component, diameter, or assortativity, which are quantities the paper itself tabulates for the biological graphs in Tables A4 and A6. The Results statement that randomization preserves 'network sparsity and structural integrity' is therefore stronger than the formal null. In an MLP pathway layer, per-pathway fan-in is fixed but gene fan-out changes, altering gradient flow and effective capacity; in the GNN and CNN variants, the randomized graphs and images differ in row statistics and neighborhood structure. The randomized network is thus a different sparse graph rather than the same architecture with biological labels permuted, so the observed equivalence does not isolate 'biological relevance' as the discarded factor. The 30-seed trial checks seed stability, not whether the null preserves the relevant structure. Please use a row-and-column degree-preserving null (e.g., a curveball or swap null) for the MLP case and an analogous structural null for the GNN and CNN inputs, or show that the randomized graphs match the Tables A4 and A6 statistics and that residual differences do not affect generalization.
- [Results, Pathway-Informed vs. Randomized Models; Table 2; Figure 3] The 15 model comparisons are not adjusted for multiple testing. At the nominal p < 0.05 level, roughly 0.75 false positives are expected among 15 independent tests, so the '3 out of 15' randomized wins could arise under the global null and should not be stated as evidence that randomized models outperformed pathway-informed ones without a correction. Please report adjusted p-values (e.g., Benjamini-Hochberg) or a permutation-based family-wise error control, and state explicitly how the abstract's 3-of-15 claim is affected by the adjustment.
- [Results, Comparison of Biological Information Extracted by Pathway-Informed Models and Randomized Counterparts; Figure…] The interpretability comparisons lack chance baselines. The DeepKEGG top-100 overlap (21 vs. 20 tumor-related features) should be compared with the expected overlap for a random top-100 feature set; the PINNet SHAP test shows that AD-related genes contribute within each model, not that the pathway-informed model's ranking is more concordant with known biology than a random model; and the PASNet and BINN Spearman correlations (0.4 and 0.56) have no null distribution. Without these baselines, the conclusion that pathway information provides 'no clear advantage in interpretability' is not established.
minor comments (4)
- [Abstract] The phrase 'the benefits of pathway integration does not arise' should agree in number ('do not arise').
- [Background & Summary] The claim of covering 'all relevant pathway-based neural network models' is too strong given that GCN-MAE, GCS-Net, PathDeep, ReGeNNe, and PGLCN could not be included in the comparison; please qualify the comprehensiveness claim.
- [Table 2] Specify how multiple configurations (OP, PP, OP+PP) for P-NET, BINN, DeepKEGG, GraphPath, and Pathformer are counted in the '15 models' total and in the '3 of 15' significant results.
- [Methods, Optimal Sparsity Level] The test-set comparison of sparsity levels is acknowledged as post hoc, but the subsequent significance statements should be labeled exploratory because selecting the best sparsity level on the test set can capitalize on chance.
Circularity Check
No circularity found: the study is an empirical benchmark against an explicit random null model, with no fitting loop or self-citation chain.
full rationale
The paper's central claim is an empirical comparison: biologically informed models versus randomized counterparts across multiple datasets and metrics. The randomization procedure formalized in Additional Information Equations 1-5 removes the biological labels while preserving total connection count and per-pathway degree, which is a control condition rather than an input from which the conclusion is derived. No parameter is fitted to the outcome and then renamed as a prediction; hyperparameters are either taken from the original model repositories or optimized on training splits and evaluated on held-out test splits. The statistical tests (Wilcoxon and Kolmogorov-Smirnov) are applied externally to the observed distributions. The possible limitation that the null model preserves only column degrees and not row degrees, hub structure, or connectivity patterns is a methodological validity concern about what the randomization isolates, not a circularity: the comparison still tests the stated hypothesis, even if the interpretation is debatable. The authors' own randomization code implements the null model rather than supplying the conclusion, and no load-bearing argument reduces to a self-citation. Consequently, the derivation chain is self-contained and the appropriate circularity score is 0.
Assumptions & free parameters
assumptions (5)
- domain assumption Randomized pathway information, generated by permuting gene-pathway associations while preserving per-pathway degrees, removes biological relevance while preserving all performance-relevant structure.
- domain assumption The original public repositories and their hyperparameters are adequate implementations of each model for a fair baseline.
- domain assumption Performance on each original dataset and task is a meaningful measure of the value of pathway information.
- domain assumption Known disease-gene databases such as AlzGene and GeDiPNet are valid ground truth for biomarker interpretation.
- standard math Uniform shuffling of ones with fixed column sums samples the intended null distribution.
Cite this review
Pith. "Pith review of Sparsity is All You Need: Rethinking Biological Pathway-Informed Approaches in Deep Learning." pith.science (2026). https://pith.science/paper/EAGIQR5D
@misc{pith2026250504300,
author = {Pith},
title = {Pith review of: Sparsity is All You Need: Rethinking Biological Pathway-Informed Approaches in Deep Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/EAGIQR5D}},
note = {Machine review of arXiv:2505.04300}
}
read the original abstract
Biologically-informed neural networks typically leverage pathway annotations to enhance performance in biomedical applications. We hypothesized that the benefits of pathway integration does not arise from its biological relevance, but rather from the sparsity it introduces. We conducted a comprehensive analysis of all relevant pathway-based neural network models for predictive tasks, critically evaluating each study's contributions. From this review, we curated a subset of methods for which the source code was publicly available. The comparison of the biologically informed state-of-the-art deep learning models and their randomized counterparts showed that models based on randomized information performed equally well as biologically informed ones across different metrics and datasets. Notably, in 3 out of the 15 analyzed models, the randomized versions even outperformed their biologically informed counterparts. Moreover, pathway-informed models did not show any clear advantage in interpretability, as randomized models were still able to identify relevant disease biomarkers despite lacking explicit pathway information. Our findings suggest that pathway annotations may be too noisy or inadequately explored by current methods. Therefore, we propose a methodology that can be applied to different domains and can serve as a robust benchmark for systematically comparing novel pathway-informed models against their randomized counterparts. This approach enables researchers to rigorously determine whether observed performance improvements can be attributed to biological insights.
Figures
Figures from the paper (3 more)
Forward citations
Cited by 1 Pith paper
-
bioMoR: Biology-Guided Mixture-of-Recursions for Effective Genomic Learning
A biology-aware Mixture-of-Recursions model, which routes gene and pathway tokens to different recursion depths using a graph of biological interactions, reports consistent accuracy and efficiency gains over biology-a...
Reference graph
Works this paper leans on
-
[1]
Michael Ashburner, Catherine A. Ball, Judith A. Blake, David Botstein, Heather Butler, J. Michael Cherry, Allan P. Davis, Kara Dolinski, Selina S. Dwight, Janan T. Eppig, Midori A. Harris, David P. Hill, Lori Issel-Tarver, Andrew Kasarskis, Suzanna Lewis, John C. Matese, Joel E. Richardson, Martin Ringwald, Gerald M. Rubin, and Gavin 10 Sherlock. Gene ont...
work page 2000
-
[2]
Structured sparsity-inducing norms through submodular functions
Francis Bach. Structured sparsity-inducing norms through submodular functions.arXiv preprint, arXiv:1008.4220, 2010
work page Pith review arXiv 2010
-
[3]
L. Bertram, M. McQueen, K. Mullin, D. Blacker, and R. E. Tanzi. Systematic meta- analyses of alzheimer disease genetic association studies: the alzgene database.Nature Genetics, 39:17–23, 2007
work page 2007
-
[4]
L. Deng, Y. Cai, W. Zhang, W. Yang, B. Gao, and H. Liu. Pathway-guided deep neural network toward interpretable and predictive modeling of drug sensitivity.Journal of Chemical Information and Modeling, 2020
work page 2020
-
[5]
Toy models of superposition.arXiv preprint arXiv:2209.10652, 2022
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superposition.arXiv preprint arXiv:2209.10652, 2022
arXiv 2022
-
[6]
H.A.Elmarakeby, J.Hwang, R.Arafeh, J.Crowdis, D.andAlDubayanS.H.andSalariK. Gang, S.and Liu, S. Kregel, C. Richter, T. E. Arnoff, J. Park, W. C. Hahn, and E. M. Van Allen. Biologically informed deep neural network for prostate cancer discovery. Nature, 2021
work page 2021
-
[7]
T. Gaudelet, N. Malod-Dognin, J. Sànchez-Valle, V. Pancaldi, A. Valencia, and N. Pržulj. Unveiling new disease, pathway, and gene associations via multi-scale neural network. PLOS One, 2020
work page 2020
-
[8]
J.Hao, Y.Kim, T.K.Kim, andM.Kang. Pasnet: pathway-associatedsparsedeepneural network for prognosis prediction from high-throughput data. BMC Bioinformatics, 2018
work page 2018
Show all 46 references
-
[9]
J. Hao, Y. Kim, T. Mallavarapu, J. H. Oh, and M. Kang. Cox-pasnet: An artificial neural network for predicting prognosis in cancer patients based on pathway-associated sparse deep neural networks. InIEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages ...
2018
-
[10]
J. Hao, M. Masum, J. H. Oh, and M. Kang. Gene- and pathway-based deep neu- ral network for multi-omics data integration to predict cancer survival outcomes. In Bioinformatics Research and Applications, pages 113–124, Cham, 2019. Springer Inter- national Publishing
2019
-
[11]
Hartman, A.M
E. Hartman, A.M. Scott, C. Karlsson, and et al. Interpreting biologically informed neural networks for enhanced proteomic biomarker discovery and pathway analysis. Nature Communications, 14, 2023
2023
-
[12]
Statistical Learning with Sparsity: The Lasso and Generalizations
Trevor Hastie, Robert Tibshirani, and Martin Wainwright. Statistical Learning with Sparsity: The Lasso and Generalizations. Chapman & Hall/CRC, 2015. 11
2015
-
[13]
Hoefler, D
T. Hoefler, D. Alistarh, T. Ben-Nun, N. Dryden, and A. Peste. Sparsity in deep learning: pruning and growth for efficient inference and training in neural networks.Journal of Machine Learning Research, 2021
2021
-
[14]
J. Hu, W. Yu, Y. Dai, C. Liu, Y. Wang, and Q. Wu. A deep neural network for gastric cancer prognosis prediction based on biological information pathways.Journal of Oncology, 2022
2022
-
[15]
Jiang, C
L. Jiang, C. Xu, Y. Bai, A. Liu, Y. Gong, Y. Wang, and H. Deng. Autosurv: inter- pretable deep learning framework for cancer survival analysis incorporating clinical and multi-omics data. npj Precision Oncology, 2024
2024
-
[16]
Kegg for taxonomy-based analysis of pathways and genomes.Nu- cleic Acids Research, 51(D1):D587–D592, 10 2023
Minoru Kanehisa, Miho Furumichi, Yoko Sato, Masayuki Kawashima, and Mari Ishiguro-Watanabe. Kegg for taxonomy-based analysis of pathways and genomes.Nu- cleic Acids Research, 51(D1):D587–D592, 10 2023
2023
-
[17]
Kim and H
H. Kim and H. Lee. Pinnet: a deep neural network with pathway prior knowledge for alzheimer’s disease. Frontiers in Aging Neuroscience, 2023
2023
-
[18]
Kundu, M
I. Kundu, M. Sharma, R. S. Barai, K. Pokar, and S. Idicula-Thomas. Gedipnet: Online resource of curated gene-disease associations for polypharmacological targets discovery. Genes & Diseases, 2023
2023
-
[19]
W. Lan, H. Liao, Q. Chen, L. Zhu, Y. Pan, and Y. P. Chen. Deepkegg: a multi-omics data integration framework with biological insights for cancer recurrence prediction and biomarker discovery.Briefings in Bioinformatics, 2024
2024
-
[20]
S. Lee, S. Lim, T. Lee, I. Sung, and S. Kim. Cancer subtype classification and modeling by pathway attention and propagation.Bioinformatics, 2020
2020
-
[21]
Liang, H
B. Liang, H. Gong, L. Lu, and J. Xu. Risk stratification and pathway analysis based on graph neural network and interpretable algorithm.BMC Bioinformatics, 2022
2022
-
[22]
Arthur Liberzon, Aravind Subramanian, Reid Pinchback, Helga Thorvaldsdóttir, Pablo Tamayo, and Jill P. Mesirov. Molecular signatures database (msigdb) 3.0.Bioinformat- ics, 27(12):1739–1740, 2011
2011
-
[23]
C. Liu, A. H. Wan, H. Liang, L. Sun, J. Li, R. Yang, Q. Li, R. Wu, K. Hu, Y. Yang, S. Cai, G. Wan, and W. He. Biological informed graph neural network for tumor mutation burden prediction and immunotherapy-related pathway analysis in gastric cancer. Computational and Structura...
2023
-
[24]
X. Liu, Y. Tao, Z. Cai, P. Bao, H. Ma, K. Li, M. Li, Y. Zhu, and Z. H. Lu. Pathformer: a biological pathway informed transformer for disease diagnosis and prognosis using multi-omics data. Bioinformatics, 2024
2024
-
[25]
A unified approach to interpreting model predictions
Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R.Garnett, editors,Advances in Neural Information Processing Systems 30, pages4765–
-
[26]
Ma and J
T. Ma and J. Wang. Graphpath: a graph attention model for molecular stratification with interpretability based on the pathway–pathway interaction network.Bioinformat- ics, 2024
2024
-
[27]
The reactome pathway knowledge- base 2024
Marija Milacic, Deidre Beavers, Patrick Conley, Chuqiao Gong, Marc Gillespie, Jo- hannes Griss, Robin Haw, Bijay Jassal, Lisa Matthews, Bruce May, Robert Petryszak, Eliot Ragueneau, Karen Rothfels, Cristoffer Sevilla, Veronica Shamovsky, Ralf Stephan, Krishna Tiwari, Thawfeek ...
2024
-
[28]
Biocarta.Biotech Software & Internet Report, 2004
Darryl Nishimura. Biocarta.Biotech Software & Internet Report, 2004
2004
-
[29]
J. H. Oh, W. Choi, E. Ko, M. Kang, A. Tannenbaum, and J. O. Deasy. Pathcnn: interpretable convolutional neural networks for survival prediction and pathway analysis applied to glioblastoma.Bioinformatics, 2021
2021
-
[30]
S. Park, E. Huang, and T. Ahn. Classification and functional analysis between cancer and normal tissues using explainable pathway deep learning through rna-sequencing gene expression. International Journal of Molecular Sciences, 2021
2021
-
[31]
T. Poggio. How deep sparse networks avoid the curse of dimensionality: Efficiently computable functions are compositionally sparse. Technical Report CBMM Memo 118, Center for Brains, Minds and Machines (CBMM), 2022. https://hdl.handle.net/ 1721.1/145776
2022
-
[32]
Compositional sparsity of learnable functions.Bulletin of the American Mathematical Society, 2024
Tomaso Poggio and Maia Fraser. Compositional sparsity of learnable functions.Bulletin of the American Mathematical Society, 2024
2024
-
[33]
G. G. Roy, N. Geard, K. Verspoor, and S. He. Mpvnn: Mutated pathway visible neural network architecture for interpretable prediction of cancer-specific survival risk. Bioinformatics, 2022
2022
-
[34]
Group sparse regularization for deep neural networks.Neurocomputing, 241:81–89, 2017
Simone Scardapane, Danilo Comminiello, Amir Hussain, and Aurelio Uncini. Group sparse regularization for deep neural networks.Neurocomputing, 241:81–89, 2017
2017
-
[35]
Schaefer, Karthik Anthony, Stephen Krupa, Jonathan Buchoff, Michael Day, Tom Hannay, and Kenneth H
Carl F. Schaefer, Karthik Anthony, Stephen Krupa, Jonathan Buchoff, Michael Day, Tom Hannay, and Kenneth H. Buetow. Pid: the pathway interaction database.Nucleic Acids Research, 37:D674–D679, 2009
2009
-
[36]
Sharma and W
D. Sharma and W. Xu. Regenne: genetic pathway-based deep neural network using canonical correlation regularizer for disease prediction.Bioinformatics, 2023
2023
-
[37]
Learning structured sparsity in deep neural networks
Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. Learning structured sparsity in deep neural networks. InProceedings of the 30th International Conference on Neural Information Processing Systems, volume 30, pages 2082–2090, 2016. 13
2016
-
[38]
Combined group and exclusive sparsity for deep neural networks
Jaehong Yoon and Sung Ju Hwang. Combined group and exclusive sparsity for deep neural networks. In Doina Precup and Yee Whye Teh, editors,Proceedings of the 34th International Conference on Machine Learning, volume 70 ofProceedings of Machine Learning Research, pages 3958–3966...
2017
-
[39]
Group variable selection via group sparse neural network
Xin Zhang and Junlong Zhao. Group variable selection via group sparse neural network. Computational Statistics & Data Analysis, 192:107911, 2024. Author Information Authors and Affiliations Computational Biomedicine Unit, Department of Medical Sciences, University of Torino, T...
2024
-
[41]
Preservation of the total number of connections: NX i=1 MX j=1 a′ ij = C, (4) where A′ = [a′ ij] is the resulting matrix after randomization
-
[42]
Preservation of the number of features per pathway: NX i=1 a′ ij = dj, ∀j∈{ 1, . . . , M}. (5)
-
[43]
25 Randomization Method The randomization operation can be performed through a uniform permutation of the con- nections while maintaining the above constraints
Uniform sampling of connections:The reassignment of connections is performed uniformly among all possible configurations satisfying the above constraints, ensuring that no structural bias or prior is introduced. 25 Randomization Method The randomization operation can be perfor...
-
[44]
Extract a list of all existing1’s in matrixA along with their respective indices(i, j)
-
[45]
Shuffle this list uniformly
-
[46]
Variable
Redistribute the 1’s in the matrixA′ while ensuring that each columnj maintains the same number of connectionsdj as in the original matrix. This ensures that the matrix sparsity remains unchanged after the randomization process. Randomization Trials In this trial, the randomiz...
2018
-
[4774]
Curran Associates, Inc., 2017. 12
2017
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.