Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

OwkinZero: Accelerating Biological Discovery with AI

T0 review · 5 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single phase of RL on curated, verifiable biological questions turns 8–32B open LLMs into specialists that beat far larger commercial models and improve on unseen tasks.

desk verdict A valuable benchmark and credible in-domain RLVR gains, but the cross-task generalization claim is contradicted by the paper's own figures; a major revision is needed, not a desk reject. read the letter →

arxiv 2508.16315 v2 pith:6XFJFIBY submitted 2025-08-22 cs.LG

Nathan Bigaud , Vincent Cabeli , Meltem Gürel , Arthur Pignet , John Klein , Gilles Wainrib , Eric Durand This is my paper · ORCID
classification cs.LG
keywords biologicalreasoningreinforcementlearningfromverifiablerewards(RLVR)largelanguagemodelsdrugdiscoverybenchmarkcross-taskgeneralizationgeneexpressionanalysisdruggabilityGRPO
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a single phase of Reinforcement Learning from Verifiable Rewards (RLVR) can convert moderately-sized open-source language models (8–32B parameters) into state-of-the-art biological reasoners. To do this, the authors build eight benchmark datasets totaling over 300,000 question-answer pairs drawn from proprietary and newly published sources (spatial transcriptomics, perturbation assays, structural biology), each question designed so the answer is machine-verifiable. After RL post-training, the OwkinZero models outperform much larger commercial systems on these benchmarks, and—a key finding—specialist models trained on a single task beat their base models on previously unseen tasks. Mixture-trained models broaden this cross-task gain, but at the cost of less faithful reasoning traces. If correct, the work implies that expert data curation plus targeted RL, rather than raw model scale, is what unlocks specialized scientific reasoning.

What carries the argument

The load-bearing mechanism is RLVR (Reinforcement Learning from Verifiable Rewards): answers are graded by an exact, deterministic check rather than by a learned reward model, so the policy can be optimized purely for correctness. The training objective is BNPO, a GRPO variant whose key design choice is batch-level token normalization—each token in every response contributes equally to the gradient, counteracting the length bias that would otherwise discourage long reasoning traces—and a zero KL penalty. The verifiable rewards come from eight benchmark datasets whose labels are produced by deterministic pipelines (differential-expression tests, ssGSEA signature scores, Fpocket druggability s

What would settle it

Train the identical RL recipe on a version of each dataset whose answer labels are randomly permuted while preserving the question templates, score ranges, and option structures; if the models still reach high accuracy (for example, above 90% on SpDE), the reward signal is available without biology, which would falsify the interpretation that RLVR unlocks biological reasoning. A second, complementary test: evaluate the specialist models on a genuinely new task family whose answers no computational-score threshold can predict (e.g., predicting clinical-trial outcomes or patient survival from th

Watch

Extended reading notes

Core claim

The paper's central claim is that reinforcement learning against verifiable rewards, applied to curated biology questions, unlocks generalizable biological reasoning that the base models lack. Trained for a single epoch with a GRPO-style objective called BNPO—which normalizes gradients by total tokens in the batch and drops the KL penalty—OwkinZero-8B reaches 99.7% accuracy on spatial differential expression, 99.2% on drug-seq perturbation effects, and 89.4% on signature-activity comparison, against roughly 45–71% for base models and commercial baselines. The most distinctive finding is the generalization evidence: a model fine-tuned only on drug-perturbation questions improves on spatial-ex

Load-bearing premise

The entire argument rests on the benchmark questions actually requiring biological reasoning: the answers are generated by thresholding computational scores (for example, SpDE samples positives from the top 1% of an in-house spatial score with distractors capped at 0.5), so a model could score well by learning score statistics or template patterns instead of biology. If that is what the RL reward optimizes, the claimed generalization to real biological discovery would not fol

Editorial extensions

If this is right

  • Specialist models trained on a single task outperform their base models on previously unseen tasks, so RLVR transfers reasoning skills instead of only memorizing one answer distribution.
  • Moderately sized open models (8–32B) post-trained this way beat larger closed commercial LLMs on the eight benchmarks, so in specialized scientific domains, curated data plus targeted RL can substitute for raw scale.
  • Mixture-trained OwkinZero models exceed every single-task specialist on the specialists' own domains, implying multi-task RL shares useful inductive biases across biological tasks.
  • Mixture training also degrades reasoning–answer consistency, so deploying RL-trained models in biomedical settings will need multi-stage training (SFT plus a second RL pass) or explicit faithfulness checks.
  • The eight datasets with entity-disjoint splits form a reusable benchmark for measuring biological reasoning in LLMs, since answers are machine-verifiable rather than judged.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because several datasets label answers by thresholding computational scores (SpDE uses the top/bottom 1% of an in-house score; TCGASA ranks signatures by ssGSEA), a model could in principle raise accuracy by learning score-statistics or template cues rather than biology. A shuffled-label control would reveal how much of the RL gain is genuinely biological.
  • Editorial inference: the recipe is portable to any experimental science where ground truth can be computed from data—perturbation screens, binding assays, clinical endpoints—so the same RLVR-plus-curated-Q&A pipeline could be pointed at other sparse experimental domains.
  • Editorial inference: transfer direction looks asymmetric (the drug-perturbation specialist helps spatial-expression and structural tasks, but the structural specialist does not help back), which hints that transfer tracks shared reasoning style rather than task overlap; the paper does not test this.
  • Editorial inference: the SpDE results rest on the stability of the in-house spatial contrast scores; an independent external spatial-transcriptomics cohort would test whether the 99% accuracy reflects durable biology or cohort-specific score artifacts.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces OwkinZero, a suite of eight multiple-choice biology QA datasets (~300k items) covering spatial and tumour-vs-healthy expression, signature activity, drug perturbation, target druggability, and structural druggability, and uses RLVR (GRPO/BNPO) to post-train Qwen3-8B and Qwen3-32B models. The authors report that 8-32B specialists beat larger commercial models on in-domain tasks, that single-task specialists generalize to unseen tasks, that mixture-trained OwkinZero models further amplify cross-task gains, and that RLVR improves in-domain accuracy while reasoning faithfulness varies. The paper also evaluates reasoning quality and faithfulness using LLM-as-a-judge.

Significance. If the benchmark and numbers are sound, this is a valuable contribution: a large, mostly entity-disjoint verifiable-QA resource for an underexplored area, and a clear demonstration that RLVR can produce large in-domain accuracy gains (e.g., OwkinZero-8B reaches 99.7 on SpDE and 99.2 on DSeqDE versus 68.6 and 51.0 for o3-Mini). The cross-task generalization claim is the main scientific novelty and is currently unsupported by the paper's own data. The benchmark itself, if released with the disclosed curation code, would be useful to the community.

major comments (5)
  1. [Abstract; §3.1; Fig. 2b; Discussion] The headline claim that 'specialist models trained on a single task consistently outperform their base models on previously unseen tasks' is contradicted by the paper's own Fig. 2b. For OwkinDrugSeq, five of the six off-diagonal deltas are negative (only DPP is positive); OwkinTherapeuticTarget has negative deltas on all six other tasks. The Discussion itself concedes that 'most fine-tuned models were prone to catastrophic forgetting.' The generalization claim must be removed or substantially reframed, and the surviving credible claim is in-domain RLVR accuracy.
  2. [Fig. 2a vs Table 2] The absolute accuracies for the same model/dataset pairs disagree between Fig. 2a and Table 2. For example, OwkinDrugSeq on SpDE is 69.9 in Fig. 2a but 90.84 in Table 2; on DPP it is 60.1 vs 46.75. OwkinSpatial on DSeqDE is 90.8 in Fig. 2a but 69.92 in Table 2. Since Fig. 2 is the paper's central evidence for cross-domain generalization, this internal inconsistency must be resolved or explicitly explained before the generalization analysis can be used.
  3. [§3.1; Table 2] The claim that 'for all the tasks at hand, the training on the mixture dataset allows OwkinZero to reach better performances than the specialist models fine-tuned on their respective tasks' is false as stated. In Table 2, OwkinZero-8B scores 76.0 on TTP, below OwkinTherapeuticTarget's 79.17, and OwkinZero-8B on DPP (79.67) is above, but the universal quantifier is not supported. The abstract's related claim that mixture models 'achieve even broader cross-task improvements' should be reworded to reflect the actual pattern of partial gains and losses.
  4. [S1.7 (TTP split)] The TTP train/test split is not subject-disjoint: targets and even full questions can appear in both train and test, and the supplement reports exact duplicate questions within each split and conflicting correct answers for the same question text. This makes TTP unsuitable as a test of generalization and can inflate the reported in-domain gain for OwkinTherapeuticTarget. The paper should either re-split TTP with entity-disjoint constraints, report results on a leakage-free subset, or explicitly exclude TTP from the cross-task generalization analysis.
  5. [S1.1 (SpDE curation)] The SpDE ground truth is generated by sampling positives from extreme tails of an in-house spatial score (e.g., s >= Q0.99) and distractors from a bounded low-score region (e.g., s <= 0.5). This construction may make the task solvable by learning score statistics or distributional priors rather than by biological reasoning, especially because the same construction is used at test time. I am not claiming the dataset is invalid, but the paper should provide a diagnostic: re-generate items with distractors drawn from the same tail or from rank-matched scores and report whether the RLVR gains persist. Without this, the claim that RLVR unlocks 'generalizable biological reasoning' is not fully established.
minor comments (5)
  1. [§3.2 / Fig. 4] The reasoning-quality and faithfulness analysis uses LLM-as-a-judge with no human validation and no inter-judge agreement. The authors call the evidence 'anecdotal,' which is appropriate, but the paper should state this limitation prominently and avoid drawing strong conclusions from Fig. 4.
  2. [§4, Eq. (1)] The BNPO objective in §4 has unbalanced parentheses in the displayed equation. Please fix and make the relationship to DAPO more precise, e.g., specify whether the equivalence holds when N==G as group size and batch size or under another condition.
  3. [Table 2 / Fig. 3] All reported accuracies are point estimates without variance, confidence intervals, or repeated evaluations. Given the small test sizes for some datasets (e.g., SpDE test n=81; TTP test n=276; SD test n=376), adding error bars or at least bootstrap intervals would substantially strengthen the comparison.
  4. [S1.2] Typo: 'followind' should be 'following'. Also, the rephrasing process is described as using an LLM helper 'which did not have access to the questions’ answers'; it would help to state the model used and whether the rephrasing was manually verified.
  5. [Figure 3 / Figure 4] Figure 3 and Figure 4 are visually dense and the legend in Figure 3 is placed as a caption line rather than a clear legend. Consider separating the base-model and Finetuned-model curves and adding a marker key for Figure 4.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's central claims are empirical held-out evaluations, not derivations from fitted inputs or self-citations.

full rationale

The paper's main results—RLVR-trained specialists outperforming base models and mixture-trained models improving in-domain accuracy—are supported by direct evaluations on held-out test splits. The datasets are constructed by applying computational thresholds (e.g., SpDE extreme-tail sampling, DPP |NES| ranking), but the models are not given these scores at inference, and train/test splits are disjoint in biological entities (genes, indications, drugs, cell lines, pathway modules). Thus test accuracy is not forced by construction: the model must generalize from entity-disjoint training data. The cross-task generalization claim is also an empirical comparison of fine-tuned versus base models on tasks not used in training, not a quantity derived from a fitted parameter. The reasoning-quality and faithfulness analyses use LLM-as-judge, which the authors explicitly label 'anecdotal' and limited; this is a methodological caveat, not a circular derivation. There is no load-bearing self-citation, no imported uniqueness theorem, and no ansatz smuggled in via citation. The apparent numerical disagreements between Figure 2 and Table 2 are internal-consistency or reporting issues, not circularity. Under the stated criteria, the derivation chain is self-contained and the circularity score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper's ground truths are all outputs of computational pipelines (ssGSEA, DEA, Fpocket, thresholded in-house scores) that are treated as correct without external validation. The headline generalization claim additionally rests on hand-chosen curation thresholds and a mixture-sampling budget. No new physical or biological entities are postulated, so the invented-entities ledger is empty.

free parameters (5)
  • SpDE extreme-tail sampling thresholds and distractor bounds = Q0.99/Q0.01 for positives; distractor s <= 0.5 or s >= -0.5
    Hand-chosen thresholds define positives vs distractors in the Spatial Differential Expression dataset (S1.1). Models can exploit these score statistics rather than biological reasoning.
  • TvHE differential expression thresholds = FDR < 0.05, |log2FC| > 1
    Hand-chosen inclusion criteria for tumour-up and normal-up question generation (S1.2).
  • DPP ground-truth selection rule = FDR < 0.05, max absolute NES pathway
    The most perturbed pathway is defined as the retained pathway with largest absolute normalized enrichment score (S1.6), a modelling choice for the correct answer.
  • Mixture training sampling budget = about 5,000 samples per question type
    Ad hoc balancing choice for the multi-task OwkinZero training set (Section 4.1), not swept or justified by experiments.
  • RL hyperparameters = G=10, N=10, beta=0, one epoch
    Group size, batch size, disabled KL penalty, and single epoch were chosen from practice/defaults and are not varied, so the sensitivity of the central result to them is unknown.
assumptions (5)
  • domain assumption ssGSEA enrichment scores are a valid measure of pathway or signature activity
    Used as ground truth for all TCGASA questions (S1.4.1) and DPP questions (S1.6), following Barbie et al. 2009.
  • domain assumption Fpocket druggability scores are a valid measure of binding-site druggability
    The SD dataset labels the pocket with the higher Fpocket score as the correct answer (S1.8).
  • domain assumption Differential expression calls from scanpy rank_genes_groups with FDR thresholds are true perturbation effects
    Used for TvHE (S1.2) and DSeqDE (S1.5) ground-truth labels; TCGA and Ginkgo assay outputs are treated as biology.
  • domain assumption LLM-as-judge preference and consistency scores reflect reasoning quality and faithfulness
    Reasoning quality and faithfulness are measured by a judge LLM (S2), with the authors themselves flagging this as anecdotal (Section 3.2).
  • domain assumption Answering automatically generated multiple-choice questions is a valid proxy for biological reasoning
    The whole RLVR setup optimizes exact-answer accuracy on template-generated QA pairs, assuming this transfers to real biological problem-solving.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OwkinZero: Accelerating Biological Discovery with AI." pith.science (2026). https://pith.science/paper/6XFJFIBY

@misc{pith2026250816315,
  author       = {Pith},
  title        = {Pith review of: OwkinZero: Accelerating Biological Discovery with AI},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6XFJFIBY}},
  note         = {Machine review of arXiv:2508.16315}
}
read the original abstract

While large language models (LLMs) are rapidly advancing scientific research, they continue to struggle with core biological reasoning tasks essential for translational and biomedical discovery. To address this limitation, we created and curated eight comprehensive benchmark datasets comprising over 300,000 verifiable question-and-answer pairs, each targeting critical challenges in drug discovery including target druggability, modality suitability, and drug perturbation effects. Using this resource, we developed the OwkinZero models by post-training open-source LLMs through a Reinforcement Learning from Verifiable Rewards strategy. Our results demonstrate that specialized 8-32B OwkinZero models substantially outperform larger, state-of-the-art commercial LLMs on these biological benchmarks. Remarkably, we uncover evidence of a key aspect of generalization: specialist models trained on a single task consistently outperform their base models on previously unseen tasks. This generalization effect is further amplified in our comprehensive OwkinZero models, which were trained on a mixture of datasets and achieve even broader cross-task improvements. This study represents a significant step toward addressing the biological reasoning blind spot in current LLMs, demonstrating that targeted reinforcement learning on carefully curated data can unlock generalizable performance in specialized models, thereby accelerating AI-driven biological discovery.

Figures

Figures reproduced from arXiv: 2508.16315 by the authors.

Figure 1
Figure 1. Performance of base, fine-tuned specialist, and OwkinZero models on the biological datasets. [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Confusion matrices for cross-domain generalisation of specialist models. Left: absolute accuracy. Right: [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Qwen3-8B Specialist (fine-tuned) OwkinZero 8B Qwen3-32B OwkinZero 32B Performance breakdown by dataset for all models. Each subfigure highlights the performance of a specialist 8B model fine-tuned on a single dataset, versus OwkinZero 8B model and the base model. The training dataset is starred, except for the subfigure on the bottom right which shows the difference between the 8B and 32B models, for the base models… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Each point represents a fine-tuned model evaluated against its corresponding base model. The accuracy [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation

    cs.LG 2026-02 conditional novelty 4.0 of 10

    A taxonomy-driven survey arguing that reward design is the central mechanism shaping reliable LLM reasoning, with maps of reward paradigms, reward-hacking failure modes, and benchmark pitfalls.

Reference graph

Works this paper leans on

30 extracted references · 21 canonical work pages · cited by 1 Pith paper

  1. [1]

    Upregulated in tumour islets versus stroma

    work page

  2. [2]

    arXiv:2503.24290 [cs]

    URL http://arxiv.org/abs/2503.24290. arXiv:2503.24290 [cs]. Kexin Huang, Tianfan Fu, Wenhao Gao, Yue Zhao, Yusuf Roohani, Jure Leskovec, Connor W. Coley, Cao Xiao, Jimeng Sun, and Marinka Zitnik. Thera- peutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development, August 2021. URL http://arxiv.org/abs/2102. 09548. Kexin Hua...

    arXiv 2021

  3. [3]

    Hugging Face

    URL https://www.biorxiv.org/content/ 10.1101/2025.05.30.656746v1. Hugging Face. Open R1: A fully open reproduction of DeepSeek-R1, January 2025. URLhttps://github. com/huggingface/open-r1. Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Ji- awei Han. Search-r1: Training LLMs to reason and leverage search engines...

    work page doi:10.1101/2025.05.30.656746v1 2025

  4. [4]

    Yang Jin, Yuanli Zuo, Gang Li, Wenrong Liu, Yitong Pan, Ting Fan, Xin Fu, Xiaojun Yao, and Yong Peng

    URL http://arxiv.org/abs/2503.09516. Yang Jin, Yuanli Zuo, Gang Li, Wenrong Liu, Yitong Pan, Ting Fan, Xin Fu, Xiaojun Yao, and Yong Peng. Advances in spatial transcriptomics and its applications in cancer research.Molecular Cancer, 23(1):129, June 2024. ISSN 1476-4598. doi: 10. 1186/s12943-024-02040-9. URL https://doi.org/ 10.1186/s12943-024-02040-9. Jon...

    arXiv 2024

  5. [5]

    arXiv:2506.17238 [cs]

    URL http://arxiv.org/abs/2506.17238. arXiv:2506.17238 [cs]. Emmanuel Noutahi, Jason Hartford, Prudencio Tossou, Shawn Whitfield, Alisandra K. Denton, Cas Wognum, Kristina Ulicna, Michael Craig, Jonathan Hsu, Michael Cuccarese, Emmanuel Bengio, Do- minique Beaini, Christopher Gibson, Daniel Co- hen, and Berton Earnshaw. Virtual Cells: Pre- dict, Explain, D...

    arXiv 2025

  6. [7]

    Reactome Consortium

    URL http://arxiv.org/abs/2504.13958. Reactome Consortium. Reactome pathways relation file, 2025. URLhttps://reactome.org/download/ current/ReactomePathwaysRelation.txt. Ac- cessed: 2025-08-20. Andrew Sellergren, Sahar Kazemzadeh, Tiam Jaroensri, Atilla Kiraly, Madeleine Traverse, Timo Kohlberger, Shawn Xu, Fayaz Jamil, Cían Hughes, Charles Lau, Justin Che...

    arXiv 2025

  7. [8]

    arXiv:2402.03300 [cs]

    URL http://arxiv.org/abs/2402.03300. arXiv:2402.03300 [cs]. Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters,

    arXiv

  8. [9]

    Kyle Swanson, Yiqun T

    URL https://arxiv.org/abs/2408.03314. Kyle Swanson, Yiqun T. Chen, Aaron Jaech, and James Zou. RL-Finetuning of OpenAI o1-mini to En- hance Biomedical Reasoning.bioRxiv, pages 2025–05,

    arXiv 2025

Show all 30 references
  1. [10]

    Pub- lisher: Cold Spring Harbor Laboratory

    URL https://www.biorxiv.org/content/ 10.1101/2025.05.19.654988.abstract. Pub- lisher: Cold Spring Harbor Laboratory. Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. Language models don’t always say what they think: Unfaithful ex- planations in chain-of-thought pr...

    2025 doi

  2. [11]

    Hanchen Wang, Tianfan Fu, Yuanqi Du, Wenhao Gao, Kexin Huang, Ziming Liu, Payal Chandak, Shengchao Liu, Peter Van Katwyk, Andreea Deac, Anima Anandkumar, Karianne Bergen, Carla P

    URL http://arxiv.org/abs/2504.06196. Hanchen Wang, Tianfan Fu, Yuanqi Du, Wenhao Gao, Kexin Huang, Ziming Liu, Payal Chandak, Shengchao Liu, Peter Van Katwyk, Andreea Deac, Anima Anandkumar, Karianne Bergen, Carla P. Gomes, Shirley Ho, Pushmeet Kohli, Joan Lasenby, Jure Leskov...

    2023 arXiv

  3. [13]

    Bladder urothelial carcinoma

    Downregulated in tumour islets versus stroma Q&A Schema Which gene is {upregulated/downregulated} in tumour islets versus stroma in {INDICATION}? A) {HGNC_A} (ensembl {ENSG_A}) B) {HGNC_B} (ensembl {ENSG_B}) Answer: {A/B} Q&A Example Q: Which gene is downregulated in tumour is...

    2018

  4. [14]

    Target mapping:associate compounds to their annotated inhibitory target(s) using curated compound metadata; compounds without a clear inhibitory mechanism or with missing target annotations are excluded

  5. [15]

    control contrasts to obtain sets of differentially expressed genes (DEGs) per target

    Differential expression (DEA):compute treated vs. control contrasts to obtain sets of differentially expressed genes (DEGs) per target

  6. [16]

    Filtering to inhibitors:retain only compounds acting as inhibitors (including ATP-competitive, allosteric, covalent inhibitors, antagonists, degraders, etc.) to ensure a consistentloss-of-function interpretation

  7. [17]

    Balancing: to control class imbalance, we downsample non-DEGs when forming negatives so that, for each target, the number of negative gene items matches the positives

  8. [18]

    Would a drug inhibiting the activity of a target induce a deregulation of a gene in given cancer cells?

    Question generation:produce natural-language items with randomized A/B answer assignment; metadata store the target (and gene/pathway when relevant). Pairwise items sample one deregulated feature and one non-deregulated feature from the same candidate universe. Reactome varian...

    2025

  9. [19]

    Load plate-level expression matrices intoAnnData objects using scanpy

  10. [20]

    control) using rank_genes_groups

    For each (drug, cell line, concentration) context, rank genes by differential expression (treated vs. control) using rank_genes_groups

  11. [21]

    Keep gene sets with robust enrichment results (FDR < 0.05)

    Run ssGSEA with Reactome gene sets to compute enrichment scores per context. Keep gene sets with robust enrichment results (FDR < 0.05)

  12. [22]

    Identify the pathway with the largest absolute NES whilst recording its direction of deregulation (upregulated / downregulated)

  13. [23]

    Map these pathway calls into natural-language multiple-choice Q&A format, with two answer options (A or B). Q&A Types The dataset contains a single question type: • most_perturbed_pathway: Identify the Reactome pathway most significantly affected by a given drug treatment in a...

  14. [24]

    No shared compoundsbetween train and test sets

  15. [25]

    No shared cell linesbetween train and test sets

  16. [26]

    parent–child

    No shared or highly similar Reactome gene sets: (a) Build the Reactome ontology. We used the official Reactome pathways relations (Reactome Consortium, 2025) to construct a directed graph, where: • Each node corresponds to a gene set (with a stable Reactome ID). • Each edge en...

    2025

  17. [27]

    S c i e n t i f i c accuracy

  18. [28]

    Logical co he ren ce and depth of r ea son in g

  19. [29]

    R el ev anc e and c o m p l e t e n e s s of e x p l a n a t i o n

  20. [30]

    rating

    Clarity and p re ci si on of language Then provide your final rating : - If Response 1 is better : +1 - If Response 2 is better : -1 You MUST respond by first j u s t i f y i n g your rating , then a JSON object in this exact format : < json > {{" rating ": < rating >}} </ jso...

  21. [2024]

    The Cancer Genome Atlas Program

    URL https://arxiv.org/abs/2402.13950. The Cancer Genome Atlas Program. The Can- cer Genome Atlas Program (TCGA) - NCI, May 2022. URL https://www.cancer.gov/ccg/ research/genome-sequencing/tcga. Archive Lo- cation: nciglobal,ncienterprise. Cheng Qian, Emre Can Acikgoz, Qi He, H...

    2022 arXiv

  22. [2025]

    URL http://arxiv.org/abs/2502.18864. Rajiv G. Govindaraj and Michał Brylinski. Compara- tive assessment of strategies to identify similar ligand- binding pockets in proteins.BMC Bioinformatics, 19 (1):91, 2018. doi: 10.1186/s12859-018-2109-2. Pub- lished 09 March 2018. Michael...

    2018 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.