Pith. sign in

REVIEW 5 major objections 7 minor 32 references

RAGPPI: RAG Benchmark for Protein-Protein Interactions in Drug Discovery

T0 review · 5 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RAGPPI introduces the first benchmark for evaluating RAG systems on the biological impacts of protein-protein interactions.

desk verdict A useful 500-pair gold set for RAG PPI QA, but the silver half is built by an unvalidated rule and the totals don't add up. read the letter →

arxiv 2505.23823 v2 pith:CFQEYN6A submitted 2025-05-28 cs.CL

Youngseung Jeon , Ziwen Li , Thomas Li , JiaSyuan Chang , Morteza Ziyadi , Xiang 'Anthony' Chen This is my paper
classification cs.CL
keywords retrieval-augmentedgenerationprotein-proteininteractionsbenchmarkdatasettargetidentificationdrugdiscoveryatomic-factevaluationexpertannotation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RAGPPI aims to give drug-discovery researchers a way to test whether retrieval-augmented generation (RAG) systems can correctly state what happens when two proteins interact. The paper claims to deliver the first benchmark built for this, with 4,420 question-answer pairs about the biological, functional, or physical effects of protein-protein interactions (PPIs) that matter for choosing drug targets. Five hundred pairs were reviewed and corrected by domain experts as a gold set, and 3,720 more were labeled automatically by an ensemble evaluator that mimics expert judgment, reported as 93.71% accurate on 70 held-out samples. If the benchmark works as described, RAG systems for target identification can be compared by factual correctness rather than by loose semantic similarity, and new benchmark pairs can be produced without paying for expert annotation every time.

What carries the argument

The load-bearing mechanism is the ensemble auto-evaluation LLM built on atomic-fact features $F_1$ and $F_2$. Atomic facts are decomposed from answers and abstracts following the FactScore approach; each answer fact gets its maximum cosine similarity to any abstract fact, $F_1$ is the per-answer average of those maxima, and $F_2$ counts answer facts falling below a threshold (mean minus two standard deviations, 0.61). Distributions of these features separate the expert-labeled Correct group (mean $F_1$ 0.83, mean $F_2$ 0.22) from the Incorrect group (mean $F_1$ 0.72, mean $F_2$ 1.37), and three GPT-4o sub-models, two specialized to those features and one to global ground-truth alignment, are combined by majority voting. The same features and M1/M2 agreement rule (Algorithm 2) decide which of the 5,000 generated answers become silver labels, so the auto-evaluator is the instrument that scales 500 gold pairs to the full 4,420-pair claim.

What would settle it

Randomly sample about 200 silver-standard pairs, have drug-discovery experts label them blind with the paper's Correct/Incorrect rubric, and compare with the ensemble labels; if the agreement falls well below 93.71%, the silver set and the 4,420-pair benchmark count collapse.

Watch

Extended reading notes

Core claim

The paper's central claim is that RAGPPI is the first benchmark specifically designed to evaluate retrieval-augmented generation for target identification in drug discovery, where the task is to state the downstream biological, functional, or physical effects of a protein-protein interaction. The benchmark is a factual QA set of 4,420 pairs: 500 expert-validated gold pairs and 3,720 silver pairs accepted by an ensemble auto-evaluation LLM. The auto-evaluator combines three GPT-4o sub-models that score an answer through average atomic-fact similarity to the source abstract ($F_1$) and the count of low-similarity atomic facts ($F_2$), plus a global semantic match to ground truth; its majority vote reaches 93.71% accuracy against expert labels on 70 balanced held-out answers.

Load-bearing premise

The auto-evaluator's 93.71% accuracy was measured on 70 GPT-4o answers, and the 3,720 silver pairs assume that same accuracy holds for all answer styles and all PPI types with no human check.

Editorial extensions

If this is right

  • RAG systems for target identification can now be ranked on factual correctness against a common expert-anchored benchmark, making results like the Table 4 model comparison reproducible.
  • The ensemble evaluator can replace expert labeling for future benchmark expansion, since it was designed to reproduce the gold-label distributions without further human annotation.
  • The reported results that plain LLMs outperform the tested RAG systems identify suboptimal retrieval as the bottleneck to fix in this domain.
  • Fact-level features expose hallucination risk that semantic-similarity metrics such as BERTScore miss, so evaluation of biomedical RAG answers should move toward atomic-fact alignment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the 93.71% auto-evaluator accuracy generalizes beyond the 70 GPT-4o answers it was tested on, the same pipeline could label the remaining sampled PPIs and thousands of other single-PPI abstracts without new expert cost, scaling well beyond 4,420 pairs.
  • The $F_1$/$F_2$ feature pair may function as a general hallucination detector for biomedical RAG output, not just for protein-protein interactions, because it measures whether answer claims are anchored in the retrieved source.
  • The paper's own Table 1 sums the gold and silver sets to 4,220, not the 4,420 stated in the abstract; the exact total should be reconciled before the headline count is used in comparisons.
  • Because the benchmark stratifies by literature frequency, low-frequency PPIs can be isolated to study how retrieval systems behave on novel, underexplored target candidates.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. RAGPPI is a proposed benchmark for evaluating RAG systems on the biological, functional, or physical impacts of protein-protein interactions (PPIs) for target identification. The authors report constructing 4,420 QA pairs: 500 expert-validated gold pairs and 3,720 silver pairs labeled by an ensemble auto-evaluator built from GPT-4o sub-models that use atomic-fact similarity features (F1, F2). They claim this is the first RAG benchmark for PPI target identification and that the auto-evaluator reaches 93.71% accuracy, allowing silver labels without expert involvement. The paper also reports an evaluation of six LLM/RAG systems on 372 sampled PPIs. The 500 gold pairs grounded in expert annotation are a useful independent resource, but the headline counts and accuracy figures contain internal contradictions, and the silver-labeling rule is not the rule that was validated.

Significance. If the claims were fully supported, RAGPPI would fill a real gap: a domain-specific QA benchmark for PPI target identification, with expert-derived design criteria, a public dataset and code, and an automatic evaluator that could scale annotation. The gold-standard construction is the strongest part: the 500 pairs come from independent expert judgments (three experts in the preliminary study and ten in the main study), which grounds the benchmark outside the LLM itself. The paper's own limitation section (§5) appropriately acknowledges the small sample and limited expert perspectives, but those caveats do not address the internal inconsistencies below. Because the silver data and the Table 4 rankings are produced by the same auto-evaluator, the central generalization claims need additional validation before the resource can be used reliably.

major comments (5)
  1. [Abstract; §1; §3.3; Tables 1 and 8] The benchmark is sized as 4,420 QA pairs in the abstract, §1, and §3.3, but Table 1 and Table 8 both total 4,220 (500 + 3,720); please reconcile the count and update the dataset and text accordingly.
  2. [§3.2.2; §3.3; Algorithm 2; Table 3] The silver-standard rule in Algorithm 2 and §3.3 returns GT only when M1 and M2 both say 'accurate', while the 93.71% (or 95.71%) accuracy in §1 and Table 3 is for majority voting over M1, M2, and M3; since Table 3 does not validate the M1∧M2 rule, the reliability of the 3,720 silver labels is not established. Please report the held-out accuracy of the actual rule or change the rule to the validated ensemble.
  3. [§1; §3.2; Table 3] The claimed accuracy of 93.71% in §1 is inconsistent with Table 3, whose 4.29% total error rate on 70 samples implies 95.71% accuracy; please correct the number and report raw counts and a confidence interval.
  4. [§3.2.1; §3.2.2] The F2 statistics for Correct answers are given as mean 0.22 (SD = 0.48) in §3.2.1 but as 0.22 ± 0.08 in the M2 specification in §3.2.2; since M2 is prompted with this distribution, the discrepancy affects reproducibility and should be corrected.
  5. [§3.2; §4; Tables 3 and 4] The auto-evaluator is calibrated on 70 GPT-4o answers with incorrect answers oversampled to 50%, but it is then applied to 3,720 silver pairs and to six systems in Table 4 with no transfer evidence; if Table 4's ground truth is the silver labels generated by the same M1/M2 rule, the rankings are not independent of the evaluator. Please provide human agreement on a sample of silver labels and on each system's outputs, and state whether Table 4 uses gold or silver labels.
minor comments (7)
  1. [§3.3] Section 3.3 says the pseudo-algorithm for F1 and F2 is shown in Algorithm 2, but Algorithm 2 is the M1/M2 decision rule; it should refer to Algorithm 1.
  2. [Figure 1] Figure 1's caption contains 'BigGrid' and 'Goal-standard' typos that should be 'BioGRID' and 'Gold-standard'.
  3. [§A.1] Section A.1 spells the model as 'GTP-4o'; use 'GPT-4o'.
  4. [Table 5] Table 5 has 'deepsseek-v3' and 'BER T' formatting errors; fix the typos.
  5. [§3.2.1; Figure 2] The text alternates between 'Incorrect' and 'Inaccurate' (e.g., §3.2.1 and Figure 2); choose one term and use it consistently.
  6. [Table 4] Table 4's heading 'Accuracy of atomic facts' is misleading; rename the column group to 'Accuracy' since the M1/M2/M3/Ensemble columns report accuracy percentages.
  7. [§5] The limitation section (§5) appropriately acknowledges the small gold set and limited expert perspectives, but it does not address the count and validation inconsistencies described above.

Circularity Check

3 steps flagged · score 6.0 of 10

Silver-standard labels are definitionally the auto-evaluator's own M1-and-M2 verdict, while the claimed 93.71% accuracy is measured on a different M1/M2/M3 majority rule using 70 samples drawn from the same 500 gold answers that fitted the evaluator; 84% of the benchmark's pairs and the Table 4 rankings therefore rest on the evaluator agreeing with itself.

  1. self definitional [Section 3.3 and Algorithm 2 (labeling rule); Section 3.2.2 and Table 3 (validated rule)]
    "By using M1 and M2 of our ensemble auto-eval LLM, we labeled an AI-generated answer as GT if both M1 and M2 evaluated it as Accurate. Only those answers were included in the final silver-standard dataset. ... Algorithm 2: if L1 = accurate and L2 = accurate then return "GT""

    The silver ground truth for all 3,720 pairs is defined, with no human check, as the auto-evaluator's own M1 and M2 agreement: GT is true exactly when M1 and M2 say accurate. The only external validation of the evaluator (Table 3; claimed as 93.71% in Section 1) measures the majority vote over M1, M2, and M3 on 70 gold-derived samples. That validated rule is different from Algorithm 2's M1-and-M2 rule that actually produced the silver labels, so the published accuracy does not cover the labeling rule used. The silver half of the benchmark is therefore its own justification by construction.

  2. fitted input called prediction [Section 3.2.1, Section 3.2.2, Table 3 (70-sample evaluation set)]
    "Model-1 (M1): GPT-4o evaluates the answer based on the average cosine similarity between its atomic facts and the corresponding abstract by comparing it with the distribution of GT (Correct: 0.83± 0.04; Incorrect: 0.72± 0.04). ... To ensure a balanced class distribution in the evaluation, we randomly selected 35 samples from the Correct class and 35 samples from the Incorrect class, excluding those used as examples for the models. In total, 70 samples were used for evaluation."

    The M1 and M2 priors embedded in the prompts (F1 means and SDs 0.83±0.04 vs 0.72±0.04; F2 0.22±0.08 vs 1.37±1.27) and the F2 threshold T = mean(S) - 2*std(S) = 0.61 are computed over all 500 gold-labeled answers in Section 3.2.1. The 70-sample evaluation in Table 3 is then drawn from those same Correct and Incorrect classes, so the supposedly held-out samples lie inside the distribution used to fit the evaluator's parameters. The reported accuracy is in part the evaluator re-detecting statistics already contained in its own prompts, and the 3,720 silver labels inherit this fitted, self-referential calibration rather than independent validation.

1 more flagged steps
  1. other [Section 4, Experiment setup, and Table 4]
    "Once the results are generated, the question-answer pairs, P(Qppi, Asys1, Asys2,....,Asys6), are passed to our auto-eval LLM. ... Table 4 shows performance ... in terms of ... accuracy ... M1 M2 M3 Ensemble."

    The same auto-evaluation LLM that defines the silver-standard ground truth (Algorithm 2) is the instrument that scores every system in Table 4. If any of the 372 sampled PPIs come from the 3,720 silver pairs (the paper does not state that the sample is restricted to the 500 gold pairs), a system's accuracy is the evaluator agreeing with labels that the evaluator itself generated, so the metric measures the labeler's internal consistency rather than agreement with independent ground truth. The loop between label generator and scorer is closed on the silver subset; the 500 expert-validated gold pairs remain the only external anchor, which is why this is partial rather than total circularity.

full rationale

The 500 gold-standard pairs are expert-verified human judgments, and the auto-evaluator has genuine external anchoring in Table 3's 70-sample comparison against expert labels; those facts prevent this from being a fully circular paper and justify a score of 6 rather than 8-10. However, the central scaling claim - that 3,720 silver pairs can be used 'without relying on further expert involvement' - reduces by construction to the auto-evaluator's own M1-and-M2 verdict: silver GT is defined as the evaluator's agreement with itself (Algorithm 2), with no human check on any of the 3,720 pairs. The validation offered for this rule is mismatched: Section 1's 'achieving an accuracy of 93.71%' corresponds to no row of Table 3 (the ensemble row shows 4.29% total error, i.e., 95.71% accuracy), and Table 3 validates the M1/M2/M3 majority vote, not the M1-and-M2 rule that produced the silver labels. In addition, the priors and the 0.61 threshold are fitted to all 500 gold answers, and the 70 validation samples are drawn from that same 500, so even the reported accuracy is partly self-referential. Finally, Table 4 ranks systems with the same evaluator that defines half the benchmark's ground truth, closing the loop on silver data. Separate internal inconsistencies (4,420 claimed pairs vs 4,220 in Tables 1 and 8; 93.71% vs 95.71% in Table 3) are correctness issues, not circularity, but they further weaken the headline quantitative claims. The paper's own Limitation section honestly acknowledges the small gold sample and limited expert perspectives, which is favorable evidence of proportionality. The GraPPI self-citation is motivational and evaluative rather than load-bearing, so it does not raise the score. Overall: the independent gold core is real, but 84% of the benchmark and the Table 4 rankings rest on a self-referential evaluator whose validated rule differs from the rule actually deployed.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The benchmark depends on expert judgments, BioGRID data, and a GPT-4o-based evaluator. The evaluator's threshold and prompt priors are fitted to the 500 gold labels, and the silver set is defined by the evaluator itself, so the ledger is dominated by domain assumptions and fitted parameters rather than invented entities.

free parameters (3)
  • F2 low-similarity outlier threshold T = 0.61 (reported threshold)
    Set as mean minus two standard deviations of atomic-fact similarities computed from the gold-standard answers; used to define F2 in Algorithm 1. The two-SD choice is ad hoc and directly shapes silver labeling.
  • Evaluator prompt reference distributions for M1 and M2 = Correct F1: 0.83 +/- 0.04; Incorrect F1: 0.72 +/- 0.04; Correct F2: 0.22 +/- 0.08; Incorrect F2: 1.37 +/- 1.27
    Fitted from 500 expert-labeled answers and inserted into GPT-4o prompts so sub-models reproduce expert thresholds; the paper reports Correct F2 SD as 0.08 in one place and 0.48 in another, so the value is internally inconsistent.
  • Literature frequency stratification boundaries = Low = frequency 1; High = frequency >= mean; Medium otherwise
    Chosen to balance the dataset; the mean frequency is not reported, and different cutoffs would change the PPI sample and benchmark difficulty.
assumptions (5)
  • domain assumption BioGRID PPI records and their curated paper abstracts are accurate and sufficient sources for PPI biological impacts.
    The benchmark uses BioGRID as the sole source of PPI truth and abstracts as reference evidence; no validation against other PPI databases is provided.
  • domain assumption Ten expert labels, one label per PPI, are reliable ground truth without inter-annotator agreement checks.
    Each of ten experts labeled 50 answers, and labels were merged into Correct and Incorrect; no agreement metric or adjudication process is reported.
  • domain assumption GPT-4o atomic-fact decomposition and text-embedding-3-small cosine similarities capture factual correctness of answers.
    F1 and F2 are computed with these models, so the evaluator inherits their errors; no comparison to human similarity judgments is given.
  • ad hoc to paper Five Expert-Perfect examples are sufficient few-shot demonstrations for generating and evaluating all PPI answers.
    The five examples come from the ten highest-frequency PPIs and are reused in prompts for all generation tasks; coverage of diverse PPI types and frequency levels is not demonstrated.
  • ad hoc to paper Auto-evaluator calibration on GPT-4o answers transfers to answers from MedLlama, GeneGPT, and GraPPI.
    The 70-sample validation includes only GPT-4o responses; no holdout validation on other generators is reported before the evaluator is used in Table 4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RAGPPI: RAG Benchmark for Protein-Protein Interactions in Drug Discovery." pith.science (2026). https://pith.science/paper/CFQEYN6A

@misc{pith2026250523823,
  author       = {Pith},
  title        = {Pith review of: RAGPPI: RAG Benchmark for Protein-Protein Interactions in Drug Discovery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CFQEYN6A}},
  note         = {Machine review of arXiv:2505.23823}
}
read the original abstract

Retrieving the biological impacts of protein-protein interactions (PPIs) is essential for target identification (Target ID) in drug development. Given the vast number of proteins involved, this process remains time-consuming and challenging. Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) frameworks have supported Target ID; however, no benchmark currently exists for identifying the biological impacts of PPIs. To bridge this gap, we introduce the RAG Benchmark for PPIs (RAGPPI), a factual question-answer benchmark of 4,420 question-answer pairs that focus on the potential biological impacts of PPIs. Through interviews with experts, we identified criteria for a benchmark dataset, such as a type of QA and source. We built a gold-standard dataset (500 QA pairs) through expert-driven data annotation. We developed an ensemble auto-evaluation LLM that incorporates expert labeling characteristics, average fact-abstract similarity (F1), and low-similarity fact counts (F2), enabling the construction of a silver-standard dataset (3,720 QA pairs). We are committed to maintaining RAGPPI as a resource to support the research community in advancing RAG systems for drug discovery QA solutions.

Figures

Figures reproduced from arXiv: 2505.23823 by the authors.

Figure 1
Figure 1. The process of building Goal-standard dataset: A) Preliminary user study: we [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Distributions of semantic features (F1 and F2) of Correct (green) and Incorrect (pink) answers based on user study labeling: (A) Distribution of the mean cosine similarity between atomic facts of an answer and its corresponding abstract (F1); (B) Distribution of the number of atomic facts considered as lower outliers (F2). Correct answers show higher similarity and fewer outliers compared to Incorrect answers. Inacc… view at source ↗
Figure 3
Figure 3. The illustration of multi-turn question answering. [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The example of the questionnaire used in our main user study, where experts [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

32 extracted references · 22 canonical work pages

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Gemini: Google’s large language models.https://gemini.google.com/,

    Google AI. Gemini: Google’s large language models.https://gemini.google.com/,

    work page

  3. [3]

    Gemini 2.0 flash | generative ai on vertex ai, February 2025

    Google Cloud. Gemini 2.0 flash | generative ai on vertex ai, February 2025. Accessed: 2025-05-11

    work page 2025

  4. [4]

    Deepseek-v3

    DeepSeek. Deepseek-v3. https://deepseek.com/blog/deepseek-v3.html, 2024. Ac- cessed: 2025-05-13

    work page 2024

  5. [5]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186. Association ...

    work page 2019

  6. [6]

    Drug discovery: a historical perspective.science, 287(5460):1960–1964, 2000

    Jurgen Drews. Drug discovery: a historical perspective.science, 287(5460):1960–1964, 2000

    work page 1960

  7. [7]

    Accelerating therapeutics for opportunities in medicine: a paradigm shift in drug discovery.Frontiers in pharmacology, 11:770, 2020

    Izumi V Hinkson, Benjamin Madej, and Eric A Stahlberg. Accelerating therapeutics for opportunities in medicine: a paradigm shift in drug discovery.Frontiers in pharmacology, 11:770, 2020

    work page 2020

  8. [8]

    Survey of hallucination in natural language generation

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1–38, 2023

    2023

Show all 32 references
  1. [9]

    Genegpt: Augmenting large language models with domain tools for improved access to biomedical information

    Qiao Jin, Yifan Yang, Qingyu Chen, and Zhiyong Lu. Genegpt: Augmenting large language models with domain tools for improved access to biomedical information. Bioinformatics, 40(2):btae075, 2024

    2024

  2. [10]

    Generalization through memorization: Nearest neighbor language models.arXiv preprint arXiv:1911.00172, 2019

    Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Generalization through memorization: Nearest neighbor language models.arXiv preprint arXiv:1911.00172, 2019

    1911 arXiv

  3. [11]

    Jsl-medllama-3-8b-v2.0, 2024

    John Snow Labs. Jsl-medllama-3-8b-v2.0, 2024. Accessed: 2025-05-11

    2024

  4. [12]

    Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in Neural Information Processing Systems, 33:9459–9474, 2020

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in Neural Information Processing Systems,...

    2020

  5. [13]

    Understand the dynamic world: An end-to-end knowledge informed framework for open domain entity state tracking

    Mingchen Li and Lifu Huang. Understand the dynamic world: An end-to-end knowledge informed framework for open domain entity state tracking. InProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 842–851, 2023

    2023

  6. [14]

    Grappi: A retrieve-divide- solve graphrag framework for large-scale protein-protein interaction exploration.arXiv preprint arXiv:2501.16382, 2025

    Ziwen Li, Xiang’Anthony’ Chen, and Youngseung Jeon. Grappi: A retrieve-divide- solve graphrag framework for large-scale protein-protein interaction exploration.arXiv preprint arXiv:2501.16382, 2025

    2025 arXiv

  7. [15]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. InText summarization branches out, pages 74–81, 2004

    2004

  8. [16]

    What is retrieval-augmented generation? IBM Research Blog, 22, 2023

    Kim Martineau, AI Explainable, and AI Generative. What is retrieval-augmented generation? IBM Research Blog, 22, 2023. 10

    2023

  9. [17]

    Factscore: Fine-grained atomic evaluation of factual precision in long form text generation.arXiv preprint arXiv:2305.14251, 2023

    Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. Factscore: Fine-grained atomic evaluation of factual precision in long form text generation.arXiv preprint arXiv:2305.14251, 2023

    2023 arXiv

  10. [18]

    Chatgpt: Optimizing language models for dialogue.https://openai.com/ chatgpt, 2023

    OpenAI. Chatgpt: Optimizing language models for dialogue.https://openai.com/ chatgpt, 2023. Accessed: 2024-08-31

    2023

  11. [19]

    Introducing gpt-4.1 in the api, April 2025

    OpenAI. Introducing gpt-4.1 in the api, April 2025. Accessed: 2025-05-11

    2025

  12. [20]

    The biogrid database: A comprehensive biomedical resource of curated protein, genetic, and chemical interactions.Protein Science, 30(1):187–200, 2021

    Rose Oughtred, Jennifer Rust, Christie Chang, Bobby-Joe Breitkreutz, Chris Stark, Andrew Willems, Lorrie Boucher, Genie Leung, Nadine Kolas, Frederick Zhang, et al. The biogrid database: A comprehensive biomedical resource of curated protein, genetic, and chemical interactions...

    2021

  13. [21]

    Target identification approaches in drug discovery

    Azhar Rasul, Ammara Riaz, Iqra Sarfraz, Samreen Gul Khan, Ghulam Hussain, Rabia Zara, Ayesha Sadiqa, Gul Bushra, Saba Riaz, Muhammad Javid Iqbal, et al. Target identification approaches in drug discovery. InDrug Target Selection and Validation, pages 41–59. Springer, 2022

    2022

  14. [22]

    Druggen: Advancing drug discovery with large language models and reinforcement learning feedback, 2024

    Mahsa Sheikholeslami, Navid Mazrouei, Yousof Gheisari, Afshin Fasihi, Matin Irajpour, and Ali Motahharynia. Druggen: Advancing drug discovery with large language models and reinforcement learning feedback, 2024

    2024

  15. [23]

    Proteoform: a single term describing protein complexity

    Lloyd M Smith and Neil L Kelleher. Proteoform: a single term describing protein complexity. Nature methods, 10(3):186–187, 2013

    2013

  16. [24]

    Damian Szklarczyk, Annika L Gable, Katerina C Nastou, David Lyon, Rebecca Kirsch, Sampo Pyysalo, Nadezhda T Doncheva, Marc Legeay, Tao Fang, Peer Bork, et al. The string database in 2021: customizable protein–protein networks, and functional charac- terization of user-uploaded...

    2021

  17. [25]

    Damian Szklarczyk, Rebecca Kirsch, Mikaela Koutrouli, Katerina Nastou, Farrokh Mehryary, Radja Hachilif, Annika L Gable, Tao Fang, Nadezhda T Doncheva, Sampo Pyysalo, et al. The string database in 2023: protein–protein association networks and functional enrichment analyses fo...

    2023

  18. [26]

    Llama: Open and efficient foundation language models.https: //github.com/facebookresearch/llama, 2023

    Hugo Touvron et al. Llama: Open and efficient foundation language models.https: //github.com/facebookresearch/llama, 2023. Accessed: 2024-08-31

    2023

  19. [27]

    Estimated research and de- velopment investment needed to bring a new medicine to market, 2009-2018.Jama, 323(9):844–853, 2020

    Olivier J Wouters, Martin McKee, and Jeroen Luyten. Estimated research and de- velopment investment needed to bring a new medicine to market, 2009-2018.Jama, 323(9):844–853, 2020

    2009

  20. [28]

    Grok-2-1212

    xAI. Grok-2-1212. https://x.ai/blog/grok-2, 2024. Accessed: 2025-05-13

    2024

  21. [29]

    The knowledge alignment problem: Bridging human and external knowledge for large language models

    Shuo Zhang, Liangming Pan, Junzhou Zhao, and William Yang Wang. The knowledge alignment problem: Bridging human and external knowledge for large language models. arXiv preprint arXiv:2305.13669, 2023

    2023 arXiv

  22. [30]

    Bertscore: Evaluating text generation with bert.arXiv preprint arXiv:1904.09675, 2019

    TianyiZhang, VarshaKishore, FelixWu, KilianQWeinberger, andYoavArtzi. Bertscore: Evaluating text generation with bert.arXiv preprint arXiv:1904.09675, 2019

    1904 arXiv

  23. [31]

    Which proteins are involved in the protein-protein interactions (PPIs) described in the abstract?

    Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. Siren’s song in the ai ocean: a survey on hallucination in large language models.arXiv preprint arXiv:2309.01219, 2023. 11 A Appendix A.1 Preliminary user ...

    2023 arXiv

  24. [2024]

    Accessed: 2024-08-31

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.