{"id":"b9499882-929c-466e-ad35-e08bd890d15f","arxiv_id":"2507.19011","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A pairwise knowledge selector and a marginalization-based integration strategy let a language model generate drug-drug interaction event descriptions that outperform MolTC and MolT5 on BLEU, ROUGE, and METEOR across two datasets.","lead":"This paper presents a two-stage AI system that predicts which biological functions of each drug matter in a drug pair and then uses that knowledge to generate readable text describing the drug-drug interaction event. It reports better text-generation scores than prior language-model baselines, especially when test drugs are new to the model.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Inductive-superiority claim rests on unquantified margins: on MecDDI Cold Start/Scaffold, PKAG-DDI trails MolT5 on METEOR/ROUGE-L, and no variance or significance tests are reported.","rationale":"The reader's verdict is CONDITIONAL and centers on BM25 labels as the weakest assumption. I agree that BM25-derived gold labels are a genuine risk, and the authors' own Appendix B notes the trade-off; however, that concern targets the first stage and is mitigated by the fact that only 19% of MecDDI pairs have multiple functions and the method's final generation numbers still beat baselines despite PKS errors. The more load-bearing place is the empirical basis of the headline: the inductive claim is supported by small, mixed, and unquantified differences on MecDDI. Since the paper reports only means of 3 runs and no error bars, the observed METEOR and ROUGE-L deficits on the inductive MecDDI rows are not addressed. The proposed check is direct and would settle it. I recommend keeping the reader's CONDITIONAL verdict: the architecture and ablations are credible, but the inductive claim should be explicitly conditioned on a variance and significance analysis. This does not change the existing verdict.","tokens_in":17048,"tokens_out":11823,"duration_ms":123364,"concrete_test":"Re-run the Table 1 experiments on both datasets with at least 10 fixed seeds and report per-seed scores, standard deviations, and paired bootstrap confidence intervals for PKAG-DDI versus MolT5 (and MolTC) separately for each split and metric. If the PKAG-DDI advantage on MecDDI Cold Start or Scaffold BLEU-4 or ROUGE-L includes zero in a 95% CI, or if the METEOR or ROUGE-L deficit persists, then the 'especially in inductive scenarios' claim should be restricted to DDInter2.0 and the paper's conclusion qualified accordingly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that PKAG-DDI outperforms existing methods for DDIE text generation, 'especially in challenging inductive scenarios.' The evidence for that claim is Table 1, but exactly on the inductive rows the advantage is small and inconsistent. On MecDDI Cold Start, PKAG-DDI's ROUGE-L is 61.87 versus MolT5's 61.95, and its METEOR lead over MolT5 is 0.05 points. On MecDDI Scaffold, PKAG-DDI trails MolT5 on METEOR (49.53 vs 50.29) and ROUGE-L (45.78 vs 46.45), while the BLEU-2 and BLEU-4 leads over MolT5 are only 0.52 and 0.92 points. All results are means of three runs, with no standard deviations, confidence intervals, or significance tests. The inductive scenarios are precisely where the paper claims the method is most useful, so this is not a peripheral reporting gap: without variance estimates the reader cannot determine whether the observed gains are reproducible or within run-to-run noise. The BM25-label concern raised by the reader is real but secondary; PKS errors do propagate (Table 4 gives top-2 accuracy 53.98 on Cold Start and 25.80 on Scaffold), yet the headline claim stands or falls on whether the final generation metrics are statistically reliable.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PKAG-DDI, a two-stage generative model for drug-drug interaction event (DDIE) text generation. Stage one is a pairwise knowledge selector (PKS) that takes the SMILES of two drugs, builds graph and fingerprint representations, exchanges information between the drugs via bidirectional cross-attention with a weight-reuse strategy, and predicts top-K biological functions for each drug. Stage two is a knowledge-augmented language model built on Galactica 1.3B with the MolTC graph-to-text adapter; it marginalizes over the K×K selected biological-function pairs and generates the DDIE text. Experiments on MecDDI and DDInter2.0 under random, cold-start, and scaffold splits compare against generation-based baselines (MolT5, MolTC) and classification baselines, and include ablations of the selector, the integration strategy, and the input modalities. The central claim is that PKAG-DDI outperforms existing methods for DDIE text generation, especially in inductive scenarios.","tokens_in":17336,"tokens_out":6614,"duration_ms":68200,"significance":"If the reported results hold, the paper makes a useful contribution: it introduces pairwise biological-function selection as a knowledge-augmentation mechanism for DDIE text generation, and it provides a concrete architecture with a public code and data release. The evaluation is broad—two datasets, three splits, generation and classification metrics, a gold-function upper bound, and several ablations—and the paper is candid about the limitation that the fixed knowledge set does not support zero-shot generalization to novel biological functions. The marginalization-based integration strategy is a reasonable response to the noise problem in retrieval-augmented generation. The main reservations concern statistical support for the headline inductive-scenario claim and the construction of the PKS training labels, both of which are addressable in revision.","major_comments":[{"comment":"The central claim that PKAG-DDI outperforms existing methods 'especially in challenging inductive scenarios' is not supported with statistical precision. All results are means of three runs, but no standard deviations, confidence intervals, or significance tests are reported anywhere in the paper. On the inductive rows where the claim is strongest, the differences are small and inconsistent: on MecDDI Cold Start, the METEOR lead over MolT5 is 64.34 versus 64.29 and ROUGE-L is 61.87 versus 61.95; on MecDDI Scaffold, PKAG-DDI trails MolT5 on METEOR (49.53 versus 50.29) and ROUGE-L (45.78 versus 46.45). Because the inductive scenarios are named in the abstract as the method's main strength, the manuscript should either add variance estimates and significance tests for all runs or explicitly temper the inductive-superiority claim.","section":"Section 4.1, Table 1"},{"comment":"The gold biological-function labels for PKS training are selected by BM25 similarity between candidate biological functions and the target DDIE text. This means the first-stage supervision is derived from the same text that the second stage is trained to generate, creating a training-time leakage: the selector is rewarded for predicting functions that lexically overlap the target, and the PKS accuracies in Table 4 may therefore overstate how well the selector recovers biologically relevant functions without access to the target. At inference the target text is unavailable, so the end-to-end gains could shrink if the selector were trained on labels that do not depend on the target. Please add a robustness check, for example training PKS with alternative labels (all inherent functions, or a human-curated relevant function) and re-measuring end-to-end generation, or at least reporting agreement between BM25-chosen labels and independent annotations. This point does not invalidate the method, but it is load-bearing for the interpretation of the PKS and generation results.","section":"Section 3.2 (Training) and Appendix B"}],"minor_comments":[{"comment":"In the model-architecture paragraph, the same symbol is used twice for the molecular token embeddings: 'Ta∈ RQ×dt and Ta∈ RQ×dt' should read 'Ta and Tb'.","section":"Section 3.3"},{"comment":"The caption says 'The abbreviations are BLEU-2, BLEU-4, and ROUGE-L' but the table also reports METEOR; please include METEOR in the list of abbreviations.","section":"Table 1 caption"},{"comment":"The sentence 'The results shown in Figure 4 demonstrate that PKS outperforms the PKR w/ BoW and PKR w/ BERT' appears to refer to Table 4, not Figure 4; the cross-reference should be corrected.","section":"Appendix D.1"},{"comment":"The paper recommends K=2 and states that larger K introduces more noise, but no sensitivity analysis for K is reported; a small K ∈ {1,2,3,4} ablation would make the choice of K and the noise claim concrete.","section":"Appendix C.4"},{"comment":"The efficiency comparison is reported only on MecDDI; since the inductive-superiority claim is a central theme, reporting PKS efficiency and accuracy on DDInter2.0 under the same splits would make the comparison more complete.","section":"Section 4.3 and Table 4"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read. The genuinely new bit is using the drug pair's biological functions as conditional context for DDIE text generation, and the two-stage design—cross-attention selector with weight reuse, then marginalization over top-K biological-function pairs—is a coherent combination I haven't seen in the cited work. The experiments are broad: two datasets, three splits, generation and classification metrics, ablations, and an oracle upper bound. The oracle experiment is the most useful result: with gold biological functions, PKAG-DDI* hits ~99% on MecDDI, which strongly supports the premise that biological functions carry the information the generator needs. The released code and data make this reproducible.\n\nThe soft spot the stress-test note identifies is real. The abstract says the method wins 'especially in challenging inductive scenarios,' but on MecDDI the inductive margins are tiny or negative: PKAG-DDI trails MolT5 on METEOR and ROUGE-L in the scaffold split, and the cold-start lead over MolT5 is 0.05 METEOR points. The clear inductive wins are on DDInter2.0 (e.g., +6.8 BLEU-2 cold start), so the claim should be dataset-dependent. Since all numbers are means of three runs with no standard deviations or significance tests, the reader genuinely cannot tell whether the MecDDI differences are noise. That is a reporting gap, not a fraudulent result. I would not call the inductive claim load-bearing enough to reject; but it needs to be hedged or backed by variance estimates.\n\nThe BM25-label issue is real but secondary. Training the selector on labels chosen by lexical similarity to the target DDIE text does create a mild circular signal, and the authors admit in Appendix B that they chose single-label classification to reduce error propagation. At inference the target text isn't used, so the generation comparison is not circular. Still, PKS top-2 accuracy is 25.8% on the MecDDI scaffold split, which makes the small inductive gains even harder to interpret.\n\nMinor: leaving ZeroDDI out of the classification baselines is odd, since they cite it; including it would strengthen the comparison. The 'first to use biological functions' claim is fine if read narrowly.\n\nWho gets value: people working on generation-based drug interaction prediction or molecule-to-text retrieval-augmented generation. It deserves a serious referee. I'd recommend conditional acceptance with a required pass on variance reporting and a toned-down inductive claim.","headline":"Worth reading and worth refereeing, but the 'especially in inductive scenarios' claim is thinner than the abstract suggests once you look at the MecDDI rows.","tokens_in":17850,"tokens_out":2372,"would_cite":true,"duration_ms":24722,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pairwise drug roles improve AI-written drug-interaction texts.","keywords":["drug-drug interaction event","text generation","biological function","pairwise knowledge selection","retrieval-augmented generation","language model","molecular graph","inductive scenario"],"falsifier":"Retrain the pairwise knowledge selector with randomly chosen biological functions as gold labels, keeping the generator fixed; if PKAG-DDI still beats the baselines by the same margin, biological-function selection is not the source of the improvements. Alternatively, feed the generator deliberately mismatched function pairs and test whether the generated texts degrade in a blinded human evaluation.","tokens_in":16825,"feed_emoji":"💊","tokens_out":7243,"duration_ms":68494,"temperature":0.7,"pith_summary":"The paper claims that drug-drug interaction event (DDIE) text is generated more accurately when the language model knows, for each of the two drugs, which biological function of that drug is actually engaged in the interaction. To supply that knowledge without manual annotation, the proposed model PKAG-DDI first predicts the most relevant biological function of each drug conditioned on the other drug, then feeds the selected function pairs into the generator by marginalizing over all plausible pairings. On two professional databases, the generated texts score higher on BLEU, ROUGE, and METEOR than generation-based baselines, and the advantage grows in inductive settings where test drugs were unseen during training. An upper-bound version that receives the gold biological functions achieves near-perfect generation scores, indicating that the selected functions carry most of the useful signal.","feed_headline":"Knowing each drug's role improves AI-written drug-interaction texts","feed_subtitle":"PKAG-DDI pairs each drug with its relevant biological function to generate precise interaction-event text.","key_machinery":"The load-bearing mechanism is the pairwise knowledge selector (PKS) together with the pairwise knowledge integration strategy. PKS builds drug representations from fingerprints and molecular-graph node prototypes, uses cross-attention to let each drug condition the other's representation in a single shared computation, and predicts each drug's top-K biological functions. The integration strategy constructs all K×K pairs, assigns each pair the product of the two renormalized top-K probabilities, and generates the DDIE text token by token by marginalizing over these pairs, turning knowledge selection into a probabilistic part of generation rather than a rigid prompt addition.","core_discovery":"The central discovery is that biological function is a pairwise, interaction-specific property: the relevant role of one drug can only be decided in the presence of the other. PKAG-DDI represents both drugs as molecular graphs and injects each drug's node prototypes into the other via cross-attention, producing representations from which a classifier selects the top-K biological functions for each direction. The generator does not simply concatenate these functions into the prompt; it treats them as latent variables and computes each token's probability as a weighted sum over all K×K function pairs, with weights given by the joint selector distribution. The paper argues this avoids the noise of dumping every function into the prompt and prevents the mismatches that arise from ranking-based pairing. The full model outperforms two generation baselines on both the MecDDI and DDInter2.0 datasets under random, cold-start, and scaffold splits, and it does so without sacrificing classification accuracy relative to dedicated classifiers.","pith_inferences":["If the biological-function selector were raised to near-oracle accuracy, generation scores would likely approach the gold-function upper bound; the gap between the full model and that bound measures how much error comes from knowledge selection rather than language generation.","The use of BM25 to pick gold labels ties the selector to lexical overlap between function names and event text; a semantically or causally grounded label choice might improve the inductive scenarios, where the selector currently weakens most.","A testable extension would swap the MecDDI knowledge source for another structured drug-knowledge base while keeping the same architecture; persistent gains would show the method exploits pairwise context in general, not the specific vocabulary of one database."],"forward_implications":["DDIE prediction can be formulated as open-ended text generation instead of label classification, giving clinicians detailed mechanism descriptions without needing a label-to-text dictionary.","The near-perfect scores of the gold-function upper bound imply that once the correct pairwise biological functions are known, the remaining text-generation task is almost solved on the datasets studied.","Because the selected biological functions are visible before the text is written, the model offers an interpretable intermediate step showing why an interaction is expected.","The probabilistic marginalization over knowledge pairs is a general recipe for relational text generation where two entities jointly determine an outcome."],"supporting_citations":[{"why":"Supplies the pretrained molecular-graph encoder, graph-to-sequence adapter, and Galactica fine-tuning setup that PKAG-DDI builds on and compares against as a baseline.","marker":"Fang et al., 2024"},{"why":"Provides the MecDDI database of drug pairs with biological functions, which is the knowledge source for the selector and one of the two evaluation datasets.","marker":"Hu et al., 2023"},{"why":"Provides the DDInter2.0 database with detailed DDIE descriptions, the second evaluation dataset that tests long-form generation.","marker":"Tian et al., 2024"},{"why":"Defines the BM25 ranking used to select the single gold biological function for each drug pair from the corresponding DDIE text when training PKS.","marker":"Robertson et al., 2009"},{"why":"Contributes the latent-variable marginalization formulation that the pairwise knowledge integration strategy adapts to K×K biological-function pairs.","marker":"Guu et al., 2020"},{"why":"Establishes retrieval-augmented generation and the token-level marginalization over retrieved knowledge that the generator's probability decomposition follows.","marker":"Lewis et al., 2020"}],"fun_headline_variants":["Pairwise drug roles sharpen AI-written interaction text","Bidirectional biology boosts drug-event text generation","PKAG-DDI: Pairing functions for better DDI text","Knowing both drugs' parts yields precise event text","Selective pairwise knowledge enhances DDI text generation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the BM25-selected gold biological functions used to train the selector are correct enough; if these labels are noisy, selection errors propagate into the generated interaction text.","fun_headline_variants_meta":{"raw":{"variants":["Pairwise drug roles sharpen AI-written interaction text","Bidirectional biology boosts drug-event text generation","PKAG-DDI: Pairing functions for better DDI text","Knowing both drugs' parts yields precise event text","Selective pairwise knowledge enhances DDI text generation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000645,"raw_usage":{"total_tokens":2965,"prompt_tokens":944,"completion_tokens":2021,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":560,"completion_tokens_details":{"reasoning_tokens":1947}},"tokens_in":560,"tokens_out":2021,"duration_ms":16735,"temperature":1.0,"reasoning_tokens":1947,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:02:28.291757+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the pairwise knowledge selector with randomly chosen biological functions as gold labels, keeping the generator fixed; if PKAG-DDI still beats the baselines by the same margin, biological-function selection is not the source of the improvements. Alternatively, feed the generator deliberately mismatched function pairs and test whether the generated texts degrade in a blinded human evaluation.","supporting_citations":[],"review_version":2}