Pith. sign in

REVIEW 3 major objections 5 minor 67 references

PertReason shows that perturbation-prediction models often get the right answer for the wrong biological reason, and offers a benchmark and a trained model that make this failure measurable and fixable.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 14:20 UTC pith:TECX5WTE

load-bearing objection A genuinely useful benchmark for reasoning-centric perturbation evaluation, but the reference mechanisms are outcome-conditioned and the unchanged-path construction confounds 'unchanged' with basal boundary states, so the faithful-reasoning claims need reframing and a stratified control. the 3 major comments →

arxiv 2607.18777 v1 pith:TECX5WTE submitted 2026-07-21 cs.LG q-bio.MN

PertReason: A Knowledge-Grounded Benchmark and Framework for Cell-State-Conditioned Mechanistic Reasoning of Perturbation Effects

Dongkwan Kim , Yiming Gao , Yining Yang , Yang Shen This is my paper
classification cs.LG q-bio.MN
keywords perturbation predictionmechanistic reasoningknowledge graphcell statebenchmarklarge language modelreinforcement learningsingle-cell transcriptomics
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that evaluating perturbation-prediction models on final expression outcomes alone is misleading, because models can reach correct answers through biologically wrong reasoning. To make the distinction measurable, it builds PertReasonQA, a benchmark in which each perturbation-versus-effect-gene question comes with a reference mechanistic pathway drawn from knowledge graphs and conditioned on the cell's basal expression state. Tested on current models, the benchmark exposes failure modes that outcome-centric benchmarks miss: correct answers justified by flawed logic, ignored cellular context, and directionally inconsistent mechanisms. The paper then introduces PertReasonLM, a language model trained with supervised fine-tuning and reinforcement learning to align its predicted outcome with a context-specific causal chain; it reports 0.736 balanced accuracy, 0.976 edge recall, and a drop in 'right answer, wrong reason' rate from 2.1% to 0.8%. If the benchmark's mechanistic labels reflect true causal processes, the framework gives the field a way to diagnose and correct unfaithful reasoning in data-rich biological systems.

Core claim

The paper's central claim is that reasoning faithfulness can and should be separated from predictive accuracy in perturbation modeling. It operationalizes this with a benchmark that conditions knowledge-graph pathways on cell-specific basal states, then filters paths so their cumulative sign propagation matches the observed direction of expression change; unchanged responses are explained by floor and ceiling effects on basally silent or saturated target genes. Evaluating existing models on this benchmark reveals large fractions of correct answers paired with incorrect mechanisms, as well as context-blind predictions that ignore the cell state. As a reference probe, the paper trains PertReas

What carries the argument

The load-bearing object is the cell-state-conditioned, outcome-filtered knowledge-graph path. A global gene-regulatory graph is reweighted by sigmoid gates based on cell-specific basal expression thresholds; for up or down genes, paths are kept only if their cumulative activating/inhibitory sign propagation matches the observed change, and for unchanged genes, paths are kept when the incoming signal lands on a basally silent or saturated target. These paths become the ground-truth 'reason' paired with each question. The same paths serve as supervision for PertReasonLM and as the reference for edge-recall, path-connectivity, and functional-similarity metrics, so the entire framework's validit

Load-bearing premise

The knowledge-graph paths that the benchmark treats as ground-truth mechanisms really are the causal mechanisms at work in the cells under study.

What would settle it

Take a sample of the benchmark's reference paths and test the proposed causal edges experimentally in the same cell contexts—for instance, perturb each intermediate gene and check whether the downstream gene changes in the predicted direction. If the curated paths agree with direct measurements no better than chance once the final expression change is known, then the benchmark's reasoning metrics are measuring fidelity to a constructed story rather than to biological mechanisms.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Outcome accuracy alone is insufficient: perturbation models should be scored on whether their reasoning aligns with a reference mechanism.
  • Cell-state conditioning makes pathways context-specific, so models cannot satisfy the benchmark through generic memorization of a static graph.
  • Fine-tuning with mechanistic supervision and reinforcement learning improves both prediction and reasoning, including generalization to new cells and perturbations.
  • The four-level answer-reason taxonomy provides a diagnostic that identifies which failure mode a model exhibits rather than just whether its final answer is right.
  • The released benchmark and trained model provide a public resource for measuring progress toward faithful virtual-cell models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the reference paths are filtered using the observed expression outcome itself, the benchmark's 'mechanism' labels are consistency stories rather than independent causal evidence; direct experimental checks of intermediate regulatory edges could reveal how much of the reported edge recall is fidelity to a post-hoc artifact.
  • The same answer-reason taxonomy could be applied to other outcome-prediction domains with causal structure, such as drug response, toxicity, or clinical risk, to detect models that are right for the wrong reason.
  • A model could in principle learn to game the path-filtering heuristics; testing on adversarial cases where sign-propagation is ambiguous but the correct mechanism is known would show whether the model's success reflects mechanism understanding or pattern matching.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents PertReasonQA, a benchmark that pairs single-cell perturbation outcome labels (up/down/unchanged) with cell-state-conditioned regulatory pathway explanations derived from knowledge graphs, and PertReasonLM, a Qwen3-based model trained via SFT and GRPO to produce outcome-aligned mechanistic rationales. The central claims are that (i) standard outcome-centric evaluation misses systematic failures in mechanistic reasoning, (ii) PertReasonQA exposes these failures through new metrics and an answer–reason taxonomy, and (iii) PertReasonLM, especially after GRPO, substantially closes the gap between outcome accuracy and mechanistic faithfulness, with reported balanced accuracy of 0.736, edge recall of 0.976, and GO similarity of 0.938.

Significance. If the constructed reasoning references were independently valid, the resource would be valuable: it combines multiple perturbation sources, uses functional splits for OOD generalization, and introduces a four-level taxonomy separating answer correctness from reason correctness. The dynamic cell-state conditioning is a useful design direction, and the authors are transparent about several limitations, including incompleteness of knowledge graphs and the synthetic nature of the references. The main contribution, however, is conditional on the references being trustworthy mechanistic ground truth. Because the reference paths are outcome-conditioned at construction and then reused for training, reward, and evaluation, the headline 'faithful reasoning' results may reflect learning the benchmark's own filtering artifacts rather than independent biological mechanism. The reported numbers should therefore be read as upper bounds on a diagnostic consistency benchmark, not as evidence of biological faithfulness.

major comments (3)
  1. [§3.3, §A.2.4, §B.2, §3.4] The reference paths are filtered to match the observed DE labels: Algorithm 1 retains only paths whose cumulative sign matches 'up'/'down', and Algorithm 2 accepts only unchanged paths whose target is basally saturated or silent. These same paths then serve as (i) SFT targets (§B.1), (ii) the triplet reward r_trp in GRPO (§B.2), and (iii) the evaluation references for edge recall (§3.4). This closed loop means that a model that learns the filtering rules can report high 'edge recall' and 'GO similarity' without recovering independent causal mechanisms. The claim that PertReasonQA exposes 'failure modes largely invisible to standard benchmarks' needs a validity check that does not use these references. Please validate on a random subset with expert annotation or against an independent curated pathway database, and report edge recall / GO similarity on that independent reference separately
  2. [§A.2.4, Algorithm 2] Algorithm 2 ties the validity of unchanged reference paths to the target gene's basal boundary state: a path is accepted only if an activating signal reaches a basally saturated gene or an inhibitory signal reaches a basally silent gene. This creates a direct confound between the 'unchanged' label and basal expression extremes. A model can achieve high balanced accuracy on unchanged samples by thresholding basal expression alone, and the case studies in Appendices G–I show exactly this behavior (the model repeatedly invokes 'low basal buffering' as the reason for unchanged). The central reasoning claim is therefore not yet established. Please add a stratified analysis by target basal state (silent / intermediate / saturated) and report whether PertReasonLM's gains over baselines persist on the intermediate-basal unchanged subset. If most unchanged references are boundary cases, the bench
  3. [§3.3, §A.2.5, §5.4] The path-to-text references are generated by Qwen3-4B and filtered by keyword matching. PertReasonLM uses Qwen3-4B as its backbone and is supervised on these same texts. Part of the reported improvement over other LLMs may therefore come from stylistic and template alignment with the generator rather than from biological reasoning. The authors acknowledge this limitation in §5.4, but the benchmark's ability to separate 'correct predictions' from 'correct reasons' depends on references being linguistically neutral as well as biologically valid. Please include a sensitivity analysis using references written independently (e.g., by a different model or expert curation) or at least report the taxonomy and edge-recall changes when the reference text style is perturbed.
minor comments (5)
  1. [§1] Typo: 'alack of mechanistic reasoning' should be 'a lack of mechanistic reasoning'.
  2. [References] Reference [11] contains garbled author text ('Maximillian Fornitz V ording') and should be cleaned.
  3. [Tables 2–6] Scores are macro-averaged over the eight test folds without per-fold variance or confidence intervals. Given the large performance gaps claimed, adding standard errors or per-fold box plots would strengthen the robustness of the comparisons.
  4. [§3.4] The criteria for 'reason correct' — edge recall >= 0.5 plus path connectivity — are reasonable but somewhat arbitrary. Please justify the 0.5 threshold or report sensitivity of the taxonomy to this threshold.
  5. [Table 1] The reasoning-sample counts are split across many columns; a total row or a clearer breakdown would help readers assess the scale of the dataset at a glance.

Circularity Check

2 steps flagged

Mechanistic-faithfulness metrics are closed-loop: reference paths are filtered to match outcome labels, then used as SFT/GRPO targets and as Edge Recall references.

specific steps
  1. fitted input called prediction [§3.3 (Reference Paths via Evidence Augmentation and Filtering), §4.1/B.2 (SFT/GRPO), §3.4 (Edge Recall)]
    "To ensure the quality of reference reasoning, we filter the extracted paths to retain only those that logically justify the observed transcriptomic outcomes. ... For differentially expressed genes, we select only those paths where the cumulative sign propagation matches the observed direction of change (Algorithm 1). ... Edge Recall: We compare the predicted signed triplets against the curated reference paths and measure the fraction of reference regulatory edges recovered by the model."

    The reference 'reason' is admitted only if it entails the DE label. PERTREASONLM is SFT-trained on these label-conditioned paths and GRPO-rewarded (0.8*r_ans + 0.2*r_trp) for reproducing their triplets; Edge Recall then scores outputs against the same curated references. The headline 'faithful reasoning' numbers (0.976 edge recall, shift into Ans✓Reason✓) therefore measure how well the model re-derives the benchmark's own label-conditioned synthetic paths, not whether the reasons are biologically true. The §5.4 caveat that references are 'curated mechanistic proxies rather than exhaustive biological ground truth' confirms the reference is internal to the construction.

  2. self definitional [§A.2.4 Algorithm 2 (Sign-Consistency Filter for Unchanged Targets); cf. §G.1 NK-cell case study]
    "if (b== 1∧1∈Z L)∨(b== 0∧0∈Z L) then return True ;// Ceiling or floor effect observed"

    By construction, an unchanged reference path exists only when the effect gene is basally saturated (b=1) or basally silent (b=0); intermediate-basal unchanged genes have no reference path. The basal-state discretization is also placed in the prompt, so a model can predict 'unchanged' from a boundary-state rule and then generate the corresponding floor/ceiling rationale. The paper's own case study shows PERTREASONLM-GRPO doing exactly this: 'LZIC exhibits low basal expression ... the downstream effect ... is absorbed within the existing expression floor.' Thus 'unchanged' reasoning is definitionally tied to the very basal-state feature the paper claims the model is learning to condition on.

full rationale

The circularity is located on the mechanistic-reasoning axis, not on the outcome axis. The DE labels come from real perturbation data, so the reported balanced accuracy (0.736) is an honest, non-circular result. But the benchmark's 'correct reasons' are generated by filtering KG paths with Algorithms 1-2 so that they entail the observed label; unchanged references are additionally restricted to basal boundary states. PERTREASONLM is trained (SFT) and reward-optimized (GRPO r_trp) to reproduce those same paths, and Edge Recall/Ans✓Reason✓ are computed against them. Consequently, the headline statements that the model achieves 0.976 edge recall and that training shifts predictions into Ans✓Reason✓ are statements about recovering the benchmark's own outcome-conditioned construction. The §5.4 admission that reference pathways are 'curated mechanistic proxies rather than exhaustive biological ground truth' narrows but does not remove the loop, because no external benchmark or held-out mechanistic ground truth is used to validate the reasons. The boundary-state confound in Algorithm 2 is a concrete instance: unchanged grounds are definitionally available only for basally silent/saturated effect genes, so the model can score by recognizing the low/high-basal shortcut. I found no load-bearing self-citation or imported uniqueness theorem.

Axiom & Free-Parameter Ledger

12 free parameters · 8 axioms · 0 invented entities

The benchmark introduces no new physical or biological entities; its 'reference pathways' are synthetic evaluation constructs built from KGs, DE labels, and LLM text synthesis. The central claim therefore rests on a chain of fitted thresholds and hyperparameters, plus the domain assumption that boolean sign propagation on knowledge graphs approximates true cellular mechanisms.

free parameters (12)
  • cell-specific activity thresholds tau_low, tau_high = fitted per cell context via KS statistics on training DE labels; OOD predicted by regressor
    Discretize basal expression into low/medium/high and gate KG edges; changing them changes reference paths and prompt construction.
  • threshold prediction regressor = fit from quantile summaries to (tau_low, tau_high)
    Used to set thresholds for OOD cells; fitted on training contexts, affecting all OOD evaluation.
  • sigmoid gating steepness beta = 10
    Controls sensitivity of basal-activity gating in the conditioned edge weight w(c)_uv.
  • source floor delta_src = 0.05
    Minimum traversability for source genes; asymmetric gating that shapes which paths are extracted.
  • target floor delta_tgt = 0.5
    Minimum traversability for target genes; allows induction of silent genes and shapes reference paths.
  • PPR damping alpha = 0.85
    Diffusion parameter in the perturbation-centered flow used for path cost.
  • max hop depth D_max = 4 (3 for chemical after prepending drug-target edge)
    Caps path length and directly constrains which reference mechanisms are included.
  • path budget K and penalty lambda = K=4, lambda=2.0
    Diversification of reference paths; changes which alternative mechanisms are retained.
  • ambiguity budget B0 = not specified
    Algorithm 1 rejects paths with more ambiguous edges than B0; affects reference path inclusion and downstream metrics.
  • functional split KMeans clusters k = 30
    Determines ID/OOD perturbation split; different k changes the difficulty and composition of OOD test folds.
  • GRPO reward weights = 0.8*r_ans + 0.2*r_trp
    Sets optimization trade-off; the triplet reward directly trains the model against reference paths.
  • reason-correct threshold = edge recall >= 0.5
    Defines mechanistic correctness in the taxonomy and in the GRPO objective; no independent justification is provided.
axioms (8)
  • standard math Personalized PageRank fixed point converges and Dijkstra shortest paths on the weighted graph are valid path-finding tools.
    Used in A.2.3 for perturbation-centered flow and path extraction; assumed without proof.
  • standard math KS test, Wilcoxon rank-sum test, and Benjamini-Hochberg correction are statistically valid for threshold calibration and DE labeling.
    Used in A.2.1 and §3.2; standard statistical machinery.
  • domain assumption OmniPath and INDRA CoGEx knowledge graphs and their regulatory sign assignments are accurate enough to serve as ground-truth mechanism references.
    The entire benchmark rests on the fidelity of these KGs in §3.3 / A.2.4; the paper later notes they are incomplete and biased.
  • domain assumption Boolean sign propagation along a path (activating/inhibiting cascades) correctly determines whether a perturbation changes downstream gene expression.
    Algorithm 1 filters reference paths by cumulative polarity matching observed labels; this monotonic boolean model is a strong simplification of cellular signaling.
  • domain assumption Unchanged expression can be explained by ceiling or floor effects when an incoming signal reaches a saturated or silent target gene.
    Algorithm 2 defines 'unchanged' reference paths entirely through this physiological boundary assumption.
  • domain assumption Discretizing basal expression into low/medium/high via tau_low and tau_high captures functionally active and inactive gene states across cell contexts.
    The dynamic cell-state conditioning in A.2.1 / A.2.2 depends on this discretization being biologically meaningful.
  • domain assumption Pseudo-bulk aggregation and DE labeling of the four public perturbation datasets produce trustworthy outcome ground truth.
    All labels in §3.2 come from pseudobulk Wilcoxon tests; batch effects and aggregation choices are assumed not to distort the reference paths.
  • domain assumption Qwen3-4B generated path-to-text narratives preserve the mechanism, and keyword-based filtering removes only low-quality samples.
    Textual fidelity of the benchmark in A.2.5 depends on the generator and the filtering scheme; the authors acknowledge stylistic bias in §5.4.

pith-pipeline@v1.3.0-alltime-deepseek · 30062 in / 14400 out tokens · 139671 ms · 2026-08-01T14:20:16.258853+00:00 · methodology

0 comments
read the original abstract

Evaluating machine learning in scientific domains requires separating correct predictions from correct reasons under realistic distribution shifts. We introduce PertReason, a knowledge-grounded benchmark and framework suite for cell-state--conditioned reasoning about perturbation effects. At its core, PertReasonQA is a benchmark that tests whether models can generate mechanistically faithful explanations while remaining robust to complex shifts, such as new cells and unseen perturbations. PertReasonQA combines single-cell genetic and chemical perturbation data across multiple cellular contexts with knowledge graphs, and dynamically conditions pathways on cell-specific basal states to avoid generic memorization. Evaluations on state-of-the-art models reveal systematic gaps between predictive accuracy and mechanistic reasoning. Specifically, these models exhibit failure modes largely invisible to standard benchmarks, such as deriving correct answers through flawed logic, ignoring cellular context, and generating directionally inconsistent mechanisms. As a reference probe of the benchmark, we present PertReasonLM, a large language model trained to align outcome predictions with context-specific mechanistic reasoning. Our model targets the identified failure modes by grounding rationales in context-specific pathways and tightening agreement between outcomes and mechanisms. Together, we provide a diagnostic framework for exposing and mitigating failures in faithful reasoning in data-rich scientific systems.

Figures

Figures reproduced from arXiv: 2607.18777 by Dongkwan Kim, Yang Shen, Yiming Gao, Yining Yang.

Figure 1
Figure 1. Figure 1: Overview of our proposed solutions in perturbation modeling. Current methods (left) [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: To rigorously evaluate mechanistic reasoning, the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Answer–reason taxonomy distribution for Qwen3-4B base, PERTREASONLM-SFT, and PERTREASONLM-GRPO in the scenarios with the no￾path context and with the cell-conditioned-path context. The four colors indicate joint answer/reason correct￾ness, where reason correctness requires triplet exis￾tence, path connectivity, and edge recall ≥ 0.5 [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

67 extracted references · 2 canonical work pages

  1. [1]

    Predicting cellular responses to perturbation across diverse contexts with state.BioRxiv, 2025

    Abhinav K Adduri, Dhruv Gautam, Beatrice Bevilacqua, Alishba Imran, Rohan Shah, Mohsen Naghipour- far, Noam Teyssier, Rajesh Ilango, Sanjay Nagaraj, Mingze Dong, et al. Predicting cellular responses to perturbation across diverse contexts with state.BioRxiv, 2025

    2025

  2. [2]

    Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines.Nature Methods, 22(8):1657–1661, 2025

    Constantin Ahlmann-Eltze, Wolfgang Huber, and Simon Anders. Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines.Nature Methods, 22(8):1657–1661, 2025. doi: 10.1038/s41592-025-02772-6. URLhttps://doi.org/10.1038/s41592-025-02772-6

    doi:10.1038/s41592-025-02772-6 2025

  3. [3]

    Bachman, Benjamin M

    John A. Bachman, Benjamin M. Gyori, and Peter K. Sorger. Automated assembly of molecular mechanisms at scale from text mining and curated databases.Molecular Systems Biology, 19(5):MSB202211325, 2023. doi: 10.15252/msb.202211325. URLhttps://doi.org/10.15252/msb.202211325

    doi:10.15252/msb.202211325 2023

  4. [4]

    On the adaptive control of the false discovery rate in multiple testing with independent statistics.Journal of educational and Behavioral Statistics, 25(1):60–83, 2000

    Yoav Benjamini and Yosef Hochberg. On the adaptive control of the false discovery rate in multiple testing with independent statistics.Journal of educational and Behavioral Statistics, 25(1):60–83, 2000

    2000

  5. [5]

    Building a knowledge graph to enable precision medicine.Scientific data, 10(1):67, 2023

    Payal Chandak, Kexin Huang, and Marinka Zitnik. Building a knowledge graph to enable precision medicine.Scientific data, 10(1):67, 2023

    2023

  6. [6]

    Scaling instruction-finetuned language models

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1–53, 2024

    2024

  7. [7]

    scgpt: toward building a foundation model for single-cell multi-omics using generative ai.Nature methods, 21(8): 1470–1480, 2024

    Haotian Cui, Chloe Wang, Hassaan Maan, Kuan Pang, Fengning Luo, Nan Duan, and Bo Wang. scgpt: toward building a foundation model for single-cell multi-omics using generative ai.Nature methods, 21(8): 1470–1480, 2024

    2024

  8. [8]

    Interpretation, extrapolation and perturbation of single cells.Nature Reviews Genetics, pages 1–22, 2026

    Daniel Dimitrov, Stefan Schrod, Martin Rohbeck, and Oliver Stegle. Interpretation, extrapolation and perturbation of single cells.Nature Reviews Genetics, pages 1–22, 2026

    2026

  9. [9]

    Gene2vec: distributed representation of genes based on co-expression.BMC genomics, 20(Suppl 1):82, 2019

    Jingcheng Du, Peilin Jia, Yulin Dai, Cui Tao, Zhongming Zhao, and Degui Zhi. Gene2vec: distributed representation of genes based on co-expression.BMC genomics, 20(Suppl 1):82, 2019

    2019

  10. [10]

    Yiming Gao, Zhen Wang, Jefferson Chen, Mark Antkowiak, Mengzhou Hu, JungHo Kong, Dexter Pratt, Jieyuan Liu, Enze Ma, Zhiting Hu, and Eric P. Xing. scpilot: Large language model reasoning toward automated single-cell analysis and discovery. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URLhttps://openreview.net/forum?...

    2025

  11. [11]

    scvae: variational auto-encoders for single-cell gene expression data

    Christopher Heje Grønbech, Maximillian Fornitz V ording, Pascal N Timshel, Casper Kaae Sønderby, Tune H Pers, and Ole Winther. scvae: variational auto-encoders for single-cell gene expression data. Bioinformatics, 36(16):4415–4422, 2020

    2020

  12. [12]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Pith/arXiv arXiv 2025

  13. [13]

    Predicting cellular responses to novel drug perturbations at a single-cell resolution

    Leon Hetzel, Simon Böhm, Niki Kilbertus, Stephan Günnemann, Mohammad Lotfollahi, and Fabian J Theis. Predicting cellular responses to novel drug perturbations at a single-cell resolution. InNeurIPS 2022, 2022

    2022

  14. [14]

    Progressive multi-agent reasoning for biological perturbation prediction, 2026

    Hyomin Kim, Sang-Yeon Hwang, Jaechang Lim, Yinhua Piao, Yunhak Oh, Woo Youn Kim, Chanyoung Park, Sungsoo Ahn, and Junhyeok Jeon. Progressive multi-agent reasoning for biological perturbation prediction, 2026. URLhttps://arxiv.org/abs/2602.07408

    Pith/arXiv arXiv 2026

  15. [15]

    Biomistral: A collection of open-source pretrained large language models for medical domains,

    Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pierre-Antoine Gourraud, Mickael Rouvier, and Richard Dufour. Biomistral: A collection of open-source pretrained large language models for medical domains,

  16. [16]

    Cell2sentence: Teaching large language models the language of biology

    Daniel Levine, Syed A Rizvi, Sacha Lévy, Nazreen Pallikkavaliyaveetil, David Zhang, Xingyu Chen, Sina Ghadermarzi, Ruiming Wu, Zihe Zheng, Ivan Vrkic, et al. Cell2sentence: Teaching large language models the language of biology. InInternational Conference on Machine Learning, pages 27299–27325. PMLR, 2024

    2024

  17. [17]

    Retrieval- augmented generation for knowledge-intensive nlp tasks, 2021

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval- augmented generation for knowledge-intensive nlp tasks, 2021. URL https://arxiv.org/abs/2005 .11401. 11

    2021

  18. [18]

    The molecular signatures database hallmark gene set collection.Cell systems, 1(6):417–425, 2015

    Arthur Liberzon, Chet Birger, Helga Thorvaldsdóttir, Mahmoud Ghandi, Jill P Mesirov, and Pablo Tamayo. The molecular signatures database hallmark gene set collection.Cell systems, 1(6):417–425, 2015

    2015

  19. [19]

    Deep generative modeling for single-cell transcriptomics.Nature methods, 15(12):1053–1058, 2018

    Romain Lopez, Jeffrey Regier, Michael B Cole, Michael I Jordan, and Nir Yosef. Deep generative modeling for single-cell transcriptomics.Nature methods, 15(12):1053–1058, 2018

    2018

  20. [20]

    Predicting cellular responses to complex perturbations in high-throughput screens.Molecular Systems Biology, page e11517, 2023

    Mohammad Lotfollahi, Anna Klimovskaia Susmelj, Carlo De Donno, Leon Hetzel, Yuge Ji, Ignacio L Ibarra, Sanjay R Srivatsan, Mohsen Naghipourfar, Riza M Daza, Beth Martin, et al. Predicting cellular responses to complex perturbations in high-throughput screens.Molecular Systems Biology, page e11517, 2023

    2023

  21. [21]

    Biogpt: generative pre-trained transformer for biomedical text generation and mining.Briefings in Bioinformatics, 23(6), September 2022

    Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie-Yan Liu. Biogpt: generative pre-trained transformer for biomedical text generation and mining.Briefings in Bioinformatics, 23(6), September 2022. ISSN 1477-4054. doi: 10.1093/bib/bbac409. URL http://dx.doi.org/10.10 93/bib/bbac409

    doi:10.1093/bib/bbac409 2022

  22. [22]

    Teaching small language models to reason

    Lucie Charlotte Magister, Jonathan Mallinson, Jakub Adamek, Eric Malmi, and Aliaksei Severyn. Teaching small language models to reason. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1773–1781, 2023

    2023

  23. [23]

    Orca 2: Teaching small language models how to reason.arXiv preprint arXiv:2311.11045, 2023

    Arindam Mitra, Luciano Del Corro, Shweti Mahajan, Andres Codas, Clarisse Simoes, Sahaj Agarwal, Xuxi Chen, Anastasia Razdaibiedina, Erik Jones, Kriti Aggarwal, et al. Orca 2: Teaching small language models how to reason.arXiv preprint arXiv:2311.11045, 2023

    Pith/arXiv arXiv 2023

  24. [24]

    Replogle, Angela N

    Ajay Nadig, Joseph M. Replogle, Angela N. Pogson, Steven A McCarroll, Jonathan S. Weissman, Elise B. Robinson, and Luke J. O’Connor. Transcriptome-wide characterization of genetic perturbations.bioRxiv,

  25. [25]

    Identifying compound-protein interactions with knowledge graph embedding of perturbation transcriptomics.Cell genomics, 4(10), 2024

    Shengkun Ni, Xiangtai Kong, Yingying Zhang, Zhengyang Chen, Zhaokun Wang, Zunyun Fu, Ruifeng Huo, Xiaochu Tong, Ning Qu, Xiaolong Wu, et al. Identifying compound-protein interactions with knowledge graph embedding of perturbation transcriptomics.Cell genomics, 4(10), 2024

    2024

  26. [26]

    URL https://www.biorxiv.org/content/early/2024/0 7/03/2024.07.03.601903

    doi: 10.1101/2024.07.03.601903. URL https://www.biorxiv.org/content/early/2024/0 7/03/2024.07.03.601903

    work page doi:10.1101/2024.07.03.601903 2024

  27. [27]

    Obtaining genetics insights from deep learning via explainable artificial intelligence.Nature Reviews Genetics, 24(2):125–137, 2023

    Gherman Novakovsky, Nick Dexter, Maxwell W Libbrecht, Wyeth W Wasserman, and Sara Mostafavi. Obtaining genetics insights from deep learning via explainable artificial intelligence.Nature Reviews Genetics, 24(2):125–137, 2023

    2023

  28. [28]

    Capabilities of gpt-4 on medical challenge problems, 2023

    Harsha Nori, Nicholas King, Scott Mayer McKinney, Dean Carignan, and Eric Horvitz. Capabilities of gpt-4 on medical challenge problems, 2023. URLhttps://arxiv.org/abs/2303.13375

    Pith/arXiv arXiv 2023

  29. [29]

    scperturb: harmonized single-cell perturbation data.Nature Methods, 21(3):531–540, 2024

    Stefan Peidli, Tessa D Green, Ciyue Shen, Torsten Gross, Joseph Min, Samuele Garda, Bo Yuan, Linus J Schumacher, Jake P Taylor-King, Debora S Marks, et al. scperturb: harmonized single-cell perturbation data.Nature Methods, 21(3):531–540, 2024

    2024

  30. [30]

    Paull, Daniel E

    Evan O. Paull, Daniel E. Carlin, Mario Niepel, Peter K. Sorger, David Haussler, and Joshua M. Stuart. Discovering causal pathways linking genomic events to transcriptional states using tied diffusion through interacting events (tiedie).Bioinformatics, 29(21):2757–2764, 08 2013. ISSN 1367-4803. doi: 10.1093/bi oinformatics/btt471. URLhttps://doi.org/10.109...

    doi:10.1093/bi 2013

  31. [31]

    Tracking biological hallucinations in single-cell perturbation predictions using scarchon, a comprehensive benchmarking platform.bioRxiv, 2025

    Jean Radig, Robin Droit, Daria Doncevic, Albert Li, Duc Thien Bui, Luis Herfurth, Thaddeus Kühn, and Carl Herrmann. Tracking biological hallucinations in single-cell perturbation predictions using scarchon, a comprehensive benchmarking platform.bioRxiv, 2025. doi: 10.1101/2025.06.23.661046. URL https://www.biorxiv.org/content/early/2025/06/27/2025.06.23.661046

    work page doi:10.1101/2025.06.23.661046 2025

  32. [32]

    Prada-Medina, Rory Donovan-Maiye, and Kaspar Märtens

    Lawrence Phillips, Marc Boubnovski Martell, Aditya Misra, Josefa Lia Stoisser, Cesar A. Prada-Medina, Rory Donovan-Maiye, and Kaspar Märtens. Synthpert: Enhancing llm biological reasoning via synthetic reasoning traces for cellular perturbation prediction, 2025. URL https://arxiv.org/abs/2509.253 46

    2025

  33. [33]

    Replogle, Reuben A

    Joseph M. Replogle, Reuben A. Saunders, Angela N. Pogson, Jeffrey A. Hussmann, Alexander Lenail, Alina Guna, Lauren Mascibroda, Eric J. Wagner, Karen Adelman, Gila Lithwick-Yanai, Nika Iremadze, Florian Oberstrass, Doron Lipson, Jessica L. Bonnar, Marco Jost, Thomas M. Norman, and Jonathan S. Weissman. Mapping information-rich genotype-phenotype landscape...

    doi:10.1016/j.cell.2022.05.013 2022

  34. [34]

    Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters

    Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. InProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’20, page 3505–3506, New York, NY , USA, 2020. Association for Computing Machinery. ...

    arXiv 2020

  35. [35]

    Universal cell embeddings: A foundation model for cell biology.BioRxiv, 2023

    Yanay Rosen, Yusuf Roohani, Ayush Agarwal, Leon Samotorˇcan, Tabula Sapiens Consortium, Stephen R Quake, and Jure Leskovec. Universal cell embeddings: A foundation model for cell biology.BioRxiv, 2023

    2023

  36. [36]

    Predicting transcriptional outcomes of novel multigene perturbations with gears.Nature Biotechnology, 42(6):927–935, 2024

    Yusuf Roohani, Kexin Huang, and Jure Leskovec. Predicting transcriptional outcomes of novel multigene perturbations with gears.Nature Biotechnology, 42(6):927–935, 2024

    2024

  37. [37]

    Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, Perry Payne, Martin Seneviratne, Paul Gamble, Chris Kelly, Nathaneal Scharli, Aakanksha Chowdhery, Philip Mansfield, Blaise Aguera y Arcas, Dale Webster, Greg S. Corrado, Yossi Matias, Katherine Chou, Juraj G...

    Pith/arXiv arXiv 2022

  38. [38]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URLhttps://arxiv.org/abs/2402.03300

    Pith/arXiv arXiv 2024

  39. [39]

    Srivatsan, José L

    Sanjay R. Srivatsan, José L. McFaline-Figueroa, Vijay Ramani, Lauren Saunders, Junyue Cao, Jonathan Packer, Hannah A. Pliner, Dana L. Jackson, Riza M. Daza, Lena Christiansen, Fan Zhang, Frank Steemers, Jay Shendure, and Cole Trapnell. Massively multiplex chemical transcriptomics at single-cell resolution. Science, 367(6473):45–51, 2020. doi: 10.1126/scie...

    doi:10.1126/science.aax6234 2020

  40. [40]

    Pfohl, Heather Cole-Lewis, Darlene Neal, Qazi Mamunur Rashid, Mike Schaekermann, Amy Wang, Dev Dash, Jonathan H

    Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen R. Pfohl, Heather Cole-Lewis, Darlene Neal, Qazi Mamunur Rashid, Mike Schaekermann, Amy Wang, Dev Dash, Jonathan H. Chen, Nigam H. Shah, Sami Lachgar, Philip Andrew Mansfield, Sushant Prakash, Bradley Green, Ewa Dominowska, Blaise Agüera y Arcas,...

    2025

  41. [41]

    Mas-Rosario, Rico Meinl, Jalil Nourisa, Jared Tumiel, Tin M

    Artur Szał ata, Andrew Benz, Robrecht Cannoodt, Mauricio Cortes, Jason Fong, Sunil Kuppasani, Richard Lieberman, Tianyu Liu, Javier A. Mas-Rosario, Rico Meinl, Jalil Nourisa, Jared Tumiel, Tin M. Tunjic, Mengbo Wang, Noah Weber, Hongyu Zhao, Benedict Anchang, Fabian J. Theis, Malte D. Luecken, and Daniel B. Burkhardt. A benchmark for prediction of transcr...

    2024

  42. [42]

    KGARevion: An AI agent for knowledge-intensive biomedical QA

    Xiaorui Su, Yibo Wang, Shanghua Gao, Xiaolong Liu, Valentina Giunchiglia, Djork-Arné Clevert, and Marinka Zitnik. KGARevion: An AI agent for knowledge-intensive biomedical QA. InThe Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum ?id=tnB94WQGrn

    2025

  43. [43]

    Theodoris, Ling Xiao, Anant Chopra, Mark D

    Christina V . Theodoris, Ling Xiao, Anant Chopra, Mark D. Chaffin, Zeina R. Al Sayed, Matthew C. Hill, Helene Mantineo, Elizabeth M. Brydon, Zexian Zeng, X. Shirley Liu, and Patrick T. Ellinor. Transfer learning enables predictions in network biology.Nature, 618:616–624, May 2023

    2023

  44. [44]

    Inductive relation prediction by subgraph reasoning

    Komal Teru, Etienne Denis, and Will Hamilton. Inductive relation prediction by subgraph reasoning. In International conference on machine learning, pages 9448–9457. PMLR, 2020

    2020

  45. [45]

    TRL: Transformers Reinforcement Learning,

    Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. TRL: Transformers Reinforcement Learning,

  46. [46]

    Omnipath: guidelines and gateway for literature-curated signaling pathway resources.Nature methods, 13(12):966–967, 2016

    Dénes Türei, Tamás Korcsmáros, and Julio Saez-Rodriguez. Omnipath: guidelines and gateway for literature-curated signaling pathway resources.Nature methods, 13(12):966–967, 2016

    2016

  47. [47]

    Benchmarking algorithms for generalizable single-cell perturbation response prediction.Nature Methods, pages 1–14, 2025

    Zhiting Wei, Yiheng Wang, Yicheng Gao, Shuguang Wang, Ping Li, Duanmiao Si, Yuli Gao, Siqi Wu, Danlu Li, Kejing Dong, et al. Benchmarking algorithms for generalizable single-cell perturbation response prediction.Nature Methods, pages 1–14, 2025

    2025

  48. [48]

    Txpert: Leveraging biochemical re- lationships for out-of-distribution transcriptomic perturbation prediction.arXiv preprint arXiv:2505.14919, 2025

    Frederik Wenkel, Wilson Tu, Cassandra Masschelein, Hamed Shirzad, Cian Eastwood, Shawn T Whitfield, Ihab Bendidi, Craig Russell, Liam Hodgson, Yassir El Mesbahi, et al. Txpert: Leveraging biochemical re- lationships for out-of-distribution transcriptomic perturbation prediction.arXiv preprint arXiv:2505.14919, 2025

    Pith/arXiv arXiv 2025

  49. [49]

    Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  50. [50]

    Individual comparisons by ranking methods.Biometrics bulletin, 1(6):80–83, 1945

    Frank Wilcoxon. Individual comparisons by ranking methods.Biometrics bulletin, 1(6):80–83, 1945

    1945

  51. [51]

    Contextualizing biological perturbation experiments through language

    Menghua Wu, Russell Littman, Jacob Levine, Lin Qiu, Tommaso Biancalani, David Richmond, and Jan-Christian Huetter. Contextualizing biological perturbation experiments through language. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  52. [52]

    Perteval-scFM: Benchmarking single-cell foundation models for perturbation effect prediction

    Aaron Wenteler, Martina Occhetta, Nikhil Branson, Victor Curean, Magdalena Huebner, William Dee, William Connell, Siu Pui Chung, Alex Hawkins-Hooker, Yasha Ektefaie, César Miguel Valdez Córdova, and Amaya Gallagher-Syed. Perteval-scFM: Benchmarking single-cell foundation models for perturbation effect prediction. InForty-second International Conference on...

    2025

  53. [53]

    Nature language model: Deciphering the language of nature for scientific discovery, 2025

    Yingce Xia, Peiran Jin, Shufang Xie, Liang He, Chuan Cao, Renqian Luo, Guoqing Liu, Yue Wang, Zequn Liu, Yuan-Jyue Chen, Zekun Guo, Yeqi Bai, Pan Deng, Yaosen Min, Ziheng Lu, Hongxia Hao, Han Yang, Jielan Li, Chang Liu, Jia Zhang, Jianwei Zhu, Ran Bi, Kehan Wu, Wei Zhang, Kaiyuan Gao, Qizhi Pei, Qian Wang, Xixian Liu, Yanting Li, Houtian Zhu, Yeqing Lu, M...

    Pith/arXiv arXiv 2025

  54. [54]

    Qwen3 technical report, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, ...

    Pith/arXiv arXiv 2025

  55. [55]

    Perturbench: Benchmarking machine learning models for cellular perturbation analysis

    Yan Wu, Esther Wershof, Sebastian M Schmon, Marcel Nassar, Bła˙zej Osi´nski, Ridvan Eksi, Zichao Yan, Rory Stark, Kun Zhang, and Thore Graepel. Perturbench: Benchmarking machine learning models for cellular perturbation analysis. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2025. URL https://...

    2025

  56. [56]

    Making large language models perform better in knowledge graph completion

    Yichi Zhang, Zhuo Chen, Lingbing Guo, Yajing Xu, Wen Zhang, and Huajun Chen. Making large language models perform better in knowledge graph completion. InProceedings of the 32nd ACM international conference on multimedia, pages 233–242, 2024

    2024

  57. [57]

    Biomaze: Benchmarking and enhancing large language models for biological pathway reasoning, 2025

    Haiteng Zhao, Chang Ma, Fangzhi Xu, Lingpeng Kong, and Zhi-Hong Deng. Biomaze: Benchmarking and enhancing large language models for biological pathway reasoning, 2025. URL https://arxiv.or g/abs/2502.16660

    Pith/arXiv arXiv 2025

  58. [58]

    scbert as a large-scale pretrained deep language model for cell type annotation of single-cell rna-seq data.Nature machine intelligence, 4(10):852–866, 2022

    Fan Yang, Wenchuan Wang, Fang Wang, Yuan Fang, Duyu Tang, Junzhou Huang, Hui Lu, and Jianhua Yao. scbert as a large-scale pretrained deep language model for cell type annotation of single-cell rna-seq data.Nature machine intelligence, 4(10):852–866, 2022

    2022

  59. [61]

    cannot be explained

    Marinka Zitnik, Monica Agrawal, and Jure Leskovec. Modeling polypharmacy side effects with graph convolutional networks.Bioinformatics, 34(13):i457–i466, 06 2018. ISSN 1367-4803. doi: 10.1093/bioi nformatics/bty294. URLhttps://doi.org/10.1093/bioinformatics/bty294. 14 A Data Details A.1 Splits To evaluate generalizability, we partition the dataset into fo...

    doi:10.1093/bioi 2018

  60. [62]

    Reducing a drug to its top-2 targets ignores significant off-target effects and downstream modulation that define the drug’s actual transcriptomic footprint

    Loss of polypharmacology.Small molecules rarely act with the precision of genetic edits. Reducing a drug to its top-2 targets ignores significant off-target effects and downstream modulation that define the drug’s actual transcriptomic footprint

  61. [63]

    up", "down

    Input misalignment.Gene-space models treat multi-gene inputs as discrete genetic in- terventions, failing to capture continuous binding affinities, target competition, and other synergistic chemical-biological mechanisms. Treating chemical perturbations as gene-set proxies produced high-variance predictions that are better interpreted as artifacts of the ...

  62. [66]

    Alogliptin

    <triplet>A JSON list of causal edges from your reasoning, e.g., [["Alogliptin", "inhibits", "DPP4"], [" DPP4", "activates", "TGFB1"], ["TGFB1", "regulates", "ARHGEF19"]]. Entities must come from this set: ["Alogliptin", "TGFB1", "DPP4", "TP53", "ARHGEF19"].</triplet> Model output. <thinking>Alogliptin inhibits DPP4, which reduces activation of TGFB1, lead...

  63. [67]

    <thinking>Your step-by-step biological reasoning here.</thinking>

  64. [68]

    <answer>up</answer> OR <answer>down</answer> OR <answer>unchanged</answer>

  65. [69]

    Alogliptin

    <triplet>A JSON list of causal edges from your reasoning, e.g., [["Alogliptin", "inhibits", "DPP4"], [" DPP4", "activates", "TGFB1"], ["TGFB1", "regulates", "ARHGEF19"]]. Entities must come from this set: ["Alogliptin", "TGFB1", "DPP4", "TP53", "ARHGEF19"].</triplet> Model output. <thinking>Alogliptin inhibits DPP4, which normally activates TGFB1; however...

  66. [2020]

    URLhttps://github.com/huggingface/trl. 13

  67. [2024]

    URLhttps://arxiv.org/abs/2402.10373

    Pith/arXiv arXiv