{"id":"d75cdeef-d30a-47a3-a8ef-aaae2b352d30","arxiv_id":"2508.10425","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"HiRef embeds medical codes in hyperbolic space using their ontology and sparsifies EHR co-occurrence graphs, reporting improved medication recommendation accuracy and strong performance on simulated unseen-code cases.","lead":"HiRef is a medication recommendation model that combines medical ontology hierarchies, embedded in hyperbolic space, with a carefully pruned graph of code co-occurrences from electronic health records. The authors claim this makes recommendations accurate even when important diagnosis or procedure codes were never seen during training, a key weakness of prior models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unseen-code evaluation may leak target medication through the co-occurrence prior if the graph is not recomputed after masking.","rationale":"The reader's weakest assumption concerned the informativeness of noisy co-occurrence priors for graph sparsification. That is a legitimate modeling concern, but the more immediate and load-bearing issue is whether the unseen-code evaluation actually isolates the ontology pathway at all. The paper constructs a global co-occurrence graph from EHR data before any masking, and Section 4.3 does not say that this graph is rebuilt after removing the selected entities. If it is not rebuilt, the masked entities retain edges to target medications, and the graph encoder can use those static prior weights at test time. That would directly explain the unusually high tF1 for R03A (0.9768) without any zero-shot generalization. The concern is concrete and testable: recompute the graph from masked data and rerun. If the result survives, the claim is strengthened; if not, the headline robustness result is an artifact. This is more central than the reader's noise-based concern, though both relate to the same prior. The in-distribution improvements are modest and not the main issue; the paper deserves a conditional path forward, contingent on this test and on releasing the masking specification and code.","tokens_in":16368,"tokens_out":5751,"duration_ms":69296,"concrete_test":"Rerun the Section 4.3 unseen protocol with the co-occurrence graph A recomputed from the masked training data only (excluded codes contributing no visits/edges), and retrain the L0 masks on that graph. If HiRef's target tF1 stays near 0.98 for R03A and above baselines for N03A/C01C, the zero-shot mechanism is confirmed. If performance drops to baseline levels, the reported result is due to prior-graph leakage. Additionally, report the number of masked entities and the identities of retained edges involving them.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central zero-shot claim (Section 4.3, Table 3) requires that the model cannot access co-occurrence information for the masked entities. The paper never states whether the co-occurrence prior A in Eq. (9) is recomputed on the masked training set. As written, A is a global graph constructed from the full EHR. If the same A is used in the unseen experiment, every masked entity still has outgoing edges with weights p_ij computed from the very visits that were supposed to be removed. The entity-level graph encoder (Eqs. (10)-(12)) adds η log(p_ij) to the attention logit, so at inference a masked source code can route activation to the target medication through the static prior even if the source was never seen during training. This would make the reported tF1=0.9768 an artifact of graph leakage rather than hyperbolic ontology transfer. The protocol also does not report how many codes are masked, whether their nodes/edges are removed from the graph, or how the L0 gates for unseen nodes are initialized or thresholded. Without this information, the central robustness claim is not verifiable.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HiRef, a medication recommendation model that combines two entity-level representation pathways: hyperbolic embeddings of medical ontology hierarchies, and a co-occurrence graph refined by prior-informed hard-concrete gates with L0 sparsity. The two entity representations are fused by a per-entity convex gate, then passed to type-specific GRUs for longitudinal patient encoding; medications are predicted with a multi-label head. The authors evaluate on MIMIC-III and MIMIC-IV against ten baselines, reporting the best or near-best results in the in-distribution setting, and design an 'unseen' experiment in which diagnosis/procedure codes correlated with a target medication are masked during training. The paper also includes ablations, graph-structure analyses, and embedding visualizations.","tokens_in":16712,"tokens_out":7773,"duration_ms":88484,"significance":"If fully supported, HiRef would be a useful contribution: it makes a concrete architectural proposal for combining curated ontology structure with data-driven co-occurrence, evaluates on two standard benchmarks with error bars, and provides several sanity checks (sparsity analysis with a Mann-Whitney test, t-SNE visualizations, case studies). The main value is the idea of prior-guided sparsification of the EHR co-occurrence graph and hyperbolic ontology transfer for rare or unseen codes. However, the central robustness claim currently rests on an under-specified and potentially leaking evaluation protocol. Until the protocol is clarified and the results re-verified under a clean holdout, the magnitude of the claimed improvement over baselines in the unseen setting is uncertain.","major_comments":[{"comment":"The unseen-code protocol is under-specified. A is defined in Eq. (9) as a global co-occurrence matrix over the full EHR, and Eqs. (10) and (17) inject η log p_ij and γ log p_ij into attention logits and gate biases. Section 4.3 states only that a subset of entities strongly correlated with a target medication is removed from training; it does not say whether A, p_ij, and the graph are recomputed on the masked training set, how unseen nodes are initialized, or how gates are thresholded at inference. If the original A is retained, masked source codes still have edges to the target through the static prior, so the reported tF1 values (R03A 0.9768) could be an artifact of graph leakage rather than of hyperbolic ontology transfer. Please specify the masking procedure and confirm whether A is recomputed.","section":"§4.3, Eqs. (9)-(11), (17)"},{"comment":"The zero-shot claim is stronger than the experiment supports. §4.3 masks source entities correlated with a target medication (e.g., diagnoses leading to N03A), but the target medication remains in the label set and is present in training. This is a missing-input robustness test, not a test of predicting a medication code never seen during training. The abstract, Introduction, and contribution bullets should either be reworded to describe this accurately or supplemented with a true holdout of unseen medication codes.","section":"§4.3, Table 3; Abstract"},{"comment":"Eq. (6) is described as a margin-regularized objective, but it contains no margin or negative term: it minimizes unweighted hyperbolic distances over child-ancestor pairs. Alone, its global minimizer collapses all nodes to a point. Downstream losses may prevent collapse, but since ancestor transfer is the core mechanism for unseen-code generalization, the paper should add negative sampling/margin or analyze why distance-only minimization preserves hierarchy under the full objective.","section":"§3.1, Eq. (6)"},{"comment":"The unseen evaluation covers only three targets, and R03A has tPrecision=1.0 and tF1=0.9768±0.0255. Such numbers are unstable when the number of positive test visits is small. Please report how many source codes are masked, how many test visits contain each target, and per-target confidence intervals. With n=3, 'consistently the highest' should be qualified.","section":"Table 3, Appendix Table 6"}],"minor_comments":[{"comment":"The text refers to 'Table 4' and 'Table 3.2/4.3'; the table/figure numbering is inconsistent throughout.","section":"§4.2, Table 2"},{"comment":"Notation is inconsistent: L_BCE vs L_bce, and Eq. (14) uses λ without a subscript. Please unify loss notation.","section":"Eq. (24)"},{"comment":"The patient-level module is said to be in Figure 1A, but the caption lists it as Figure 1B.","section":"§3.3"},{"comment":"Typo: 'oncology structure' should be 'ontology structure'.","section":"Introduction"},{"comment":"The DDI tables are not referenced in the main text; either integrate them into the experimental discussion or explicitly state they are supplementary.","section":"Appendix A.4"},{"comment":"No code release is mentioned. Given the sensitivity of the unseen-code protocol, releasing code would materially improve reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The central issue is protocol transparency. If the authors can confirm that A is recomputed after masking and that the target is truly held out in the intended sense, the results would be credible. As written, the ambiguity makes the unseen-code claim unverifiable and the paper unsuitable for acceptance in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Sam,\n\nQuick take on arXiv:2508.10425. The architecture is a reasonable mashup: hyperbolic ontology embeddings for transfer, SGAT-style graph sparsification, and a learned gate to fuse them. The in-distribution numbers on MIMIC-III/IV are competitive, though the gains over Carmen and LAMRec are small. The graph analysis (pruned edges are high out-degree, etc.) is a genuine attempt to show interpretability. There is real work here.\n\nBut the main robustness claim—that HiRef handles unseen codes—is not supported as written. The unseen-code protocol in Sec 4.3 masks a few source entities from training, but the paper never says whether the co-occurrence prior A (Eq. 9) is recomputed on the masked set. A is described as a global graph over all visits. If A still contains edges from the masked sources to the target medication, then at inference the model can route attention to that target through the η log(p_ij) term in Eq. 10 and the γ log(p_ij) term in Eq. 17. The R03A tF1 of 0.9768 with precision 1.0 smells like exactly this kind of leakage. The stress-test hit this, and I think it lands.\n\nThere are other problems. The unseen evaluation is only three medication classes, chosen because they are 'clearly associated' with distinct domains—cherry-picked. No code release. And some internal inconsistencies are worrying: Eq. (6) minimizes raw distance over all child-ancestor pairs, which would collapse the embedding space unless there are negative samples that aren't shown; the appendix 'exponential map' (Eq. 25) is not the standard Poincaré exponential map; and the conclusion claims the model lacks 'explicit temporal modeling of visit sequences' while the method uses three GRUs. That last one isn't fatal but makes you wonder how carefully the paper was proofread.\n\nIf the graph-leakage question resolves cleanly—i.e., A is recomputed after masking and the model still does well—then the zero-shot transfer story is interesting and worth publishing. As it stands, the central claim is unverifiable. I'd send this to peer review, but with a strong request to disclose the masking details, broaden the unseen evaluation, release code, and fix the stated technical errors. It's not desk-reject material; it's a promising draft with a load-bearing omission.","headline":"The architecture is plausible, but the central unseen-code claim is unverifiable because the paper never says whether the co-occurrence graph is rebuilt after masking.","tokens_in":17148,"tokens_out":5489,"would_cite":false,"duration_ms":52834,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Coupling hyperbolic ontology embeddings with a sparsified co-occurrence graph lets a medication recommender keep accuracy when medical codes were never seen in training.","keywords":["Medication Recommendation","Medical Ontology","Network Refinement","Hyperbolic Embedding","Electronic Health Records","Zero-shot Generalization","Graph Sparsification","MIMIC"],"falsifier":"Construct an unseen-code test in which the target medication's entire ancestor path in the ATC tree is absent from the ontology, while its co-occurrence partners are also masked from training. If HiRef still recommends it at high tF1, the ancestor-driven mechanism is not what is doing the work; if accuracy collapses when the ancestor path is missing, the zero-shot claim is confirmed as hierarchy-dependent.","tokens_in":16265,"feed_emoji":"💊","tokens_out":12157,"duration_ms":107924,"temperature":0.7,"pith_summary":"The paper tries to establish that a medication recommender can remain accurate even when a diagnosis, procedure, or drug code is absent from the training data, by merging two complementary structures: the parent-child hierarchy of curated medical ontologies and a cleaned version of EHR co-occurrence statistics. The authors argue that purely data-driven models fail in such unseen-code situations because they lean on observed co-occurrence counts, whereas hyperbolic embeddings of ontology trees let a code inherit representation from its ancestors and siblings. They further claim that refining the EHR co-occurrence graph with a prior-guided sparsity penalty keeps clinically meaningful associations and discards noisy, high-degree ones. If correct, the result matters for clinical deployment because real records are incomplete and coding practices shift, so a recommender that handles rare or new codes without retraining is more dependable. On MIMIC-III and MIMIC-IV, the paper reports top in-distribution scores and the highest target-medication F1 under simulated unseen-code settings.","feed_headline":"Sparse graph plus ontology keeps drug picks right for unseen codes","feed_subtitle":"Paper reports top scores on MIMIC-IV and highest target-drug F1 when strong predictors are masked out.","key_machinery":"Two mechanisms carry the argument. (1) A hyperbolic ontology encoder: each ontology tree is embedded in the Poincaré ball and each node's representation is the Möbius sum of all its ancestors, $\\tilde{e}^B_{*,i} = e^B_{*,a_1} \\oplus \\cdots \\oplus e^B_{*,a_\\ell} \\oplus e^B_{*,i}$, making shared ancestry the channel for zero-shot transfer. (2) A prior-guided sparse co-occurrence encoder: edge priors $p_{ij} = |\\mathrm{occ}(i) \\cap \\mathrm{occ}(j)| / |\\mathrm{occ}(i)|$ enter learnable hard-concrete gates through $\\log \\bar{\\kappa}_{ij} = \\log \\kappa_{ij} + \\gamma \\log p_{ij}$, with an L0 penalty on active edges, and a masked softmax renormalizes attention over surviving neighbors. A per-entity","core_discovery":"HiRef's central claim is that the failure of existing medication recommenders on rare or unseen codes comes from over-reliance on observed co-occurrence, and that this can be repaired by combining two evidence sources. Ontology codes (ICD-9 diagnoses and procedures, ATC level-3 medications) are embedded in the Poincaré ball and each leaf representation is built by Möbius-adding its ancestor vectors, so an unseen code inherits a position near its relatives. In parallel, a directed visit-level co-occurrence graph is initialized with conditional probabilities $a_{ij} = |\\mathrm{occ}(i) \\cap \\mathrm{occ}(j)|/|\\mathrm{occ}(i)|$, then pruned by a learned hard-concrete mask with L0 sparsity pressur","pith_inferences":["The paper does not test ICD-10: because the authors restrict evaluation to ICD-9 codes, the zero-shot claim has not yet been shown to survive a coding-system transition; applying HiRef to mapped ICD-10 data would be a direct test.","A stress test for the graph refinement would inject synthetic documentation gaps into the co-occurrence statistics and ask whether the learned mask removes the artifacts; the paper's own framing implies the model should, but this is not run.","The paper's listed limitations (no demographics, labs, notes, or visit timing) mean the patient-level encoder is deliberately simple; combining the entity-level machinery with those signals is a natural extension, though not one the paper demonstrates."],"forward_implications":["Under the paper's unseen-code simulation, HiRef beats all tested baselines on target-medication F1 for R03A, N03A, and C01C, indicating the ontology pathway carries useful signal when co-occurrence is masked.","The ablation shows the co-occurrence encoder is the most important single module; removing it causes the largest drop, so data-driven associations remain necessary even when hierarchical semantics help.","The learned graph is interpretable: strong-prior edges such as those among gastrointestinal drug classes survive, some weak-prior edges are deliberately retained because they aid prediction, and high-degree hub nodes are pruned.","The convex fusion gate is entity-specific: procedure codes receive an average beta of 0.89 toward the ontology pathway, consistent with their sparse co-occurrence and strong hierarchy."],"supporting_citations":[{"why":"Supplies Poincaré-ball hyperbolic embeddings used to encode ontology trees.","marker":"[Nickel and Kiela, 2017]"},{"why":"Supplies the ancestor-aggregation idea, adapted here as Möbius addition along the hierarchy.","marker":"[Choi et al., 2017]"},{"why":"Supplies the L0-style stochastic mask and hard-concrete reparameterization used for graph sparsification.","marker":"[Ye and Ji, 2021]"},{"why":"MIMIC-III, one of the two EHR benchmarks the claims are tested on.","marker":"[Johnson et al., 2016]"},{"why":"MIMIC-IV, the larger EHR benchmark where HiRef reports the highest in-distribution scores.","marker":"[Johnson et al., 2023]"},{"why":"ATC classification that provides the medication ontology hierarchy.","marker":"[Organization et al., 2021]"},{"why":"Documents ICD-9/ICD-10 differences and supports the paper's decision to evaluate on ICD-9 codes only.","marker":"[Hirsch et al., 2016]"},{"why":"SafeDrug; supplies the data preprocessing pipeline and a key baseline.","marker":"[Yang et al., 2021a]"},{"why":"Carmen; contributes the critique that raw DDI rates misalign with real prescriptions, shaping the safety evaluation.","marker":"[Chen et al., 2023]"}],"fun_headline_variants":["Hyperbolic ontology + pruned graphs keep drug picks robust for unseen codes","Medication recommender that handles rare codes via ontology and graph pruning","Unseen-code medication picks from hierarchical ontology and refined co-occurrence","Robust drug choices for rare diseases via hyperbolic embedding and sparse graphs","Boosting medication recommendations for missing codes with ontology and sparse graph"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise is that the co-occurrence statistics used to set edge priors come from the same noisy, incomplete records the method is meant to survive, so if those statistics are corrupted by documentation gaps or coding changes, the sparsification may preserve the corrupt edges rather than remove them.","fun_headline_variants_meta":{"raw":{"variants":["Hyperbolic ontology + pruned graphs keep drug picks robust for unseen codes","Medication recommender that handles rare codes via ontology and graph pruning","Unseen-code medication picks from hierarchical ontology and refined co-occurrence","Robust drug choices for rare diseases via hyperbolic embedding and sparse graphs","Boosting medication recommendations for missing codes with ontology and sparse graph"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000171,"raw_usage":{"total_tokens":1137,"prompt_tokens":801,"completion_tokens":336,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":545,"completion_tokens_details":{"reasoning_tokens":245}},"tokens_in":545,"tokens_out":336,"duration_ms":3936,"temperature":1.0,"reasoning_tokens":245,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T20:27:05.732762+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct an unseen-code test in which the target medication's entire ancestor path in the ATC tree is absent from the ontology, while its co-occurrence partners are also masked from training. If HiRef still recommends it at high tF1, the ancestor-driven mechanism is not what is doing the work; if accuracy collapses when the ancestor path is missing, the zero-shot claim is confirmed as hierarchy-dependent.","supporting_citations":[{"cited_title":"Poincar \\'e embeddings for learning hierarchical representations","cited_arxiv_id":null,"evidence_quote":"Supplies Poincaré-ball hyperbolic embeddings used to encode ontology trees."},{"cited_title":"Gram: graph-based attention model for healthcare representation learning","cited_arxiv_id":null,"evidence_quote":"Supplies the ancestor-aggregation idea, adapted here as Möbius addition along the hierarchy."},{"cited_title":"Mimic-iii, a freely accessible critical care database","cited_arxiv_id":null,"evidence_quote":"MIMIC-III, one of the two EHR benchmarks the claims are tested on."},{"cited_title":"Mimic-iv, a freely accessible electronic health record dataset","cited_arxiv_id":null,"evidence_quote":"MIMIC-IV, the larger EHR benchmark where HiRef reports the highest in-distribution scores."},{"cited_title":"Anatomical therapeutic chemical (atc) classification, 2021","cited_arxiv_id":null,"evidence_quote":"ATC classification that provides the medication ontology hierarchy."},{"cited_title":"Icd-10: history and context","cited_arxiv_id":null,"evidence_quote":"Documents ICD-9/ICD-10 differences and supports the paper's decision to evaluate on ICD-9 codes only."}],"review_version":1}