REVIEW 4 major objections 6 minor 29 references
HiRef: Leveraging Hierarchical Ontology and Network Refinement for Robust Medication Recommendation
T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read Coupling hyperbolic ontology embeddings with a sparsified co-occurrence graph lets a medication recommender keep accuracy when medical codes were never seen in training.
desk verdict The architecture is plausible, but the central unseen-code claim is unverifiable because the paper never says whether the co-occurrence graph is rebuilt after masking. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Two mechanisms carry the argument. (1) A hyperbolic ontology encoder: each ontology tree is embedded in the Poincaré ball and each node's representation is the Möbius sum of all its ancestors, $\tilde{e}^B_{*,i} = e^B_{*,a_1} \oplus \cdots \oplus e^B_{*,a_\ell} \oplus e^B_{*,i}$, making shared ancestry the channel for zero-shot transfer. (2) A prior-guided sparse co-occurrence encoder: edge priors $p_{ij} = |\mathrm{occ}(i) \cap \mathrm{occ}(j)| / |\mathrm{occ}(i)|$ enter learnable hard-concrete gates through $\log \bar{\kappa}_{ij} = \log \kappa_{ij} + \gamma \log p_{ij}$, with an L0 penalty on active edges, and a masked softmax renormalizes attention over surviving neighbors. A per-entity
What would settle it
Construct an unseen-code test in which the target medication's entire ancestor path in the ATC tree is absent from the ontology, while its co-occurrence partners are also masked from training. If HiRef still recommends it at high tF1, the ancestor-driven mechanism is not what is doing the work; if accuracy collapses when the ancestor path is missing, the zero-shot claim is confirmed as hierarchy-dependent.
Extended reading notes
Core claim
HiRef's central claim is that the failure of existing medication recommenders on rare or unseen codes comes from over-reliance on observed co-occurrence, and that this can be repaired by combining two evidence sources. Ontology codes (ICD-9 diagnoses and procedures, ATC level-3 medications) are embedded in the Poincaré ball and each leaf representation is built by Möbius-adding its ancestor vectors, so an unseen code inherits a position near its relatives. In parallel, a directed visit-level co-occurrence graph is initialized with conditional probabilities $a_{ij} = |\mathrm{occ}(i) \cap \mathrm{occ}(j)|/|\mathrm{occ}(i)|$, then pruned by a learned hard-concrete mask with L0 sparsity pressur
Load-bearing premise
The load-bearing premise is that the co-occurrence statistics used to set edge priors come from the same noisy, incomplete records the method is meant to survive, so if those statistics are corrupted by documentation gaps or coding changes, the sparsification may preserve the corrupt edges rather than remove them.
Editorial extensions
If this is right
- Under the paper's unseen-code simulation, HiRef beats all tested baselines on target-medication F1 for R03A, N03A, and C01C, indicating the ontology pathway carries useful signal when co-occurrence is masked.
- The ablation shows the co-occurrence encoder is the most important single module; removing it causes the largest drop, so data-driven associations remain necessary even when hierarchical semantics help.
- The learned graph is interpretable: strong-prior edges such as those among gastrointestinal drug classes survive, some weak-prior edges are deliberately retained because they aid prediction, and high-degree hub nodes are pruned.
- The convex fusion gate is entity-specific: procedure codes receive an average beta of 0.89 toward the ontology pathway, consistent with their sparse co-occurrence and strong hierarchy.
Reading between the lines
- The paper does not test ICD-10: because the authors restrict evaluation to ICD-9 codes, the zero-shot claim has not yet been shown to survive a coding-system transition; applying HiRef to mapped ICD-10 data would be a direct test.
- A stress test for the graph refinement would inject synthetic documentation gaps into the co-occurrence statistics and ask whether the learned mask removes the artifacts; the paper's own framing implies the model should, but this is not run.
- The paper's listed limitations (no demographics, labs, notes, or visit timing) mean the patient-level encoder is deliberately simple; combining the entity-level machinery with those signals is a natural extension, though not one the paper demonstrates.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes HiRef, a medication recommendation model that combines two entity-level representation pathways: hyperbolic embeddings of medical ontology hierarchies, and a co-occurrence graph refined by prior-informed hard-concrete gates with L0 sparsity. The two entity representations are fused by a per-entity convex gate, then passed to type-specific GRUs for longitudinal patient encoding; medications are predicted with a multi-label head. The authors evaluate on MIMIC-III and MIMIC-IV against ten baselines, reporting the best or near-best results in the in-distribution setting, and design an 'unseen' experiment in which diagnosis/procedure codes correlated with a target medication are masked during training. The paper also includes ablations, graph-structure analyses, and embedding visualizations.
Significance. If fully supported, HiRef would be a useful contribution: it makes a concrete architectural proposal for combining curated ontology structure with data-driven co-occurrence, evaluates on two standard benchmarks with error bars, and provides several sanity checks (sparsity analysis with a Mann-Whitney test, t-SNE visualizations, case studies). The main value is the idea of prior-guided sparsification of the EHR co-occurrence graph and hyperbolic ontology transfer for rare or unseen codes. However, the central robustness claim currently rests on an under-specified and potentially leaking evaluation protocol. Until the protocol is clarified and the results re-verified under a clean holdout, the magnitude of the claimed improvement over baselines in the unseen setting is uncertain.
major comments (4)
- [§4.3, Eqs. (9)-(11), (17)] The unseen-code protocol is under-specified. A is defined in Eq. (9) as a global co-occurrence matrix over the full EHR, and Eqs. (10) and (17) inject η log p_ij and γ log p_ij into attention logits and gate biases. Section 4.3 states only that a subset of entities strongly correlated with a target medication is removed from training; it does not say whether A, p_ij, and the graph are recomputed on the masked training set, how unseen nodes are initialized, or how gates are thresholded at inference. If the original A is retained, masked source codes still have edges to the target through the static prior, so the reported tF1 values (R03A 0.9768) could be an artifact of graph leakage rather than of hyperbolic ontology transfer. Please specify the masking procedure and confirm whether A is recomputed.
- [§4.3, Table 3; Abstract] The zero-shot claim is stronger than the experiment supports. §4.3 masks source entities correlated with a target medication (e.g., diagnoses leading to N03A), but the target medication remains in the label set and is present in training. This is a missing-input robustness test, not a test of predicting a medication code never seen during training. The abstract, Introduction, and contribution bullets should either be reworded to describe this accurately or supplemented with a true holdout of unseen medication codes.
- [§3.1, Eq. (6)] Eq. (6) is described as a margin-regularized objective, but it contains no margin or negative term: it minimizes unweighted hyperbolic distances over child-ancestor pairs. Alone, its global minimizer collapses all nodes to a point. Downstream losses may prevent collapse, but since ancestor transfer is the core mechanism for unseen-code generalization, the paper should add negative sampling/margin or analyze why distance-only minimization preserves hierarchy under the full objective.
- [Table 3, Appendix Table 6] The unseen evaluation covers only three targets, and R03A has tPrecision=1.0 and tF1=0.9768±0.0255. Such numbers are unstable when the number of positive test visits is small. Please report how many source codes are masked, how many test visits contain each target, and per-target confidence intervals. With n=3, 'consistently the highest' should be qualified.
minor comments (6)
- [§4.2, Table 2] The text refers to 'Table 4' and 'Table 3.2/4.3'; the table/figure numbering is inconsistent throughout.
- [Eq. (24)] Notation is inconsistent: L_BCE vs L_bce, and Eq. (14) uses λ without a subscript. Please unify loss notation.
- [§3.3] The patient-level module is said to be in Figure 1A, but the caption lists it as Figure 1B.
- [Introduction] Typo: 'oncology structure' should be 'ontology structure'.
- [Appendix A.4] The DDI tables are not referenced in the main text; either integrate them into the experimental discussion or explicitly state they are supplementary.
- [General] No code release is mentioned. Given the sensitivity of the unseen-code protocol, releasing code would materially improve reproducibility.
Circularity Check
Unseen-code evaluation leaks target medication through the global co-occurrence prior unless A is recomputed after masking.
-
fitted input called prediction
[Section 3.2 (Eq. 9); Section 4.3 (Table 3)]
"To fully leverage co-occurrence signals, we first construct a dense, global, directed co-occurrence graph by connecting all medical entities that co-occur within a single visit. ... aij = |occ(i) ∩ occ(j)| / |occ(i)| ∈ [0, 1], ... We remove a subset of medical entities strongly correlated with a target medication from the training data, then see if the model can still recommend the right medications. The criteria of selecting strongly correlated target entities are: 1) co-occurrence rate from source to target > 0.5, 2) co-occurrence rate from target to source above 0.01."
As written, the co-occurrence graph A in Eq. (9) is global and built from all visits, and Section 4.3 only removes entities from the training data; the paper never states that A is recomputed on the masked set. Thus for a masked source code, the edge weight p_{source→target} still counts the very visits the protocol claims to delete. Eq. (10) injects η log(p_ij) into the masked-softmax attention logit, so at inference the target medication's embedding can be retrieved from the static prior even though the source was 'unseen.' The target-selection criterion (co-occurrence rate > 0.5) makes the leak direct: the target is by construction a strong prior neighbor of the source. The reported tF1 (e.g., R03A 0.9768) is then a readout of the co-occurrence prior, not a test of ontology transfer. Th
full rationale
The in-distribution results (Table 2) are a standard supervised benchmark: HiRef is trained and evaluated on disjoint visits, and no reported number is a relabeled training quantity. The ontology encoder is trained with a structural hyperbolic-distance loss, not with the target medication label, so there is no definitional circularity there. The only problematic step is the unseen-code protocol. Section 3.2 constructs the co-occurrence prior A from all visits and does not say it is recomputed after Section 4.3 masks entities from the training data. Since the masked source node's edge weights remain in A and Eq. (10) adds η log(p_ij) to the attention logit, the target medication is reachable at inference through the very co-occurrence signal the experiment claims to remove. The target-selection criterion (co-occurrence rate > 0.5) makes this leak especially direct: the target is by construction a strong prior neighbor of the source. Therefore the headline zero-shot tF1 values are not an independent confirmation of ontology transfer; they are at least partially forced by the input graph. No other circularity pattern (self-citation, uniqueness import, ansatz-by-citation, renaming) is present. Score 6 reflects that the central unseen-code claim partially reduces to its input; if the authors can confirm A was rebuilt on masked data, the score would drop to 0-1.
Assumptions & free parameters
free parameters (8)
- lambda_hyp =
0.01
- lambda_sparse =
0.01
- lambda_margin =
0.04
- eta (prior logit weight)
- gamma (prior gate bias)
- tau (attention temperature)
- beta (hard-concrete temperature)
- embedding dimension d
assumptions (6)
- domain assumption Medical ontologies (ICD-9, ATC) are rooted trees where each code has a single parent, so hyperbolic space represents them with low distortion.
- domain assumption ATC Level 3 is a sufficient medication label granularity for the recommendation task.
- domain assumption The co-occurrence prior pij (Eq. 9) computed from the same EHRs is informative for distinguishing true associations from spurious ones.
- ad hoc to paper L0 sparse regularization with hard-concrete gates preserves clinically essential edges while suppressing spurious ones.
- ad hoc to paper Mobius aggregation of ancestors (Eq. 7) yields an informative representation for codes unseen in training.
- ad hoc to paper Minimizing all child-ancestor hyperbolic distances in Eq. (6) does not collapse the embedding space.
Cite this review
Pith. "Pith review of HiRef: Leveraging Hierarchical Ontology and Network Refinement for Robust Medication Recommendation." pith.science (2026). https://pith.science/paper/5ATRVMTX
@misc{pith2026250810425,
author = {Pith},
title = {Pith review of: HiRef: Leveraging Hierarchical Ontology and Network Refinement for Robust Medication Recommendation},
year = {2026},
howpublished = {\url{https://pith.science/paper/5ATRVMTX}},
note = {Machine review of arXiv:2508.10425}
}
read the original abstract
Medication recommendation is a crucial task for assisting physicians in making timely decisions from longitudinal patient medical records. However, real-world EHR data present significant challenges due to the presence of rarely observed medical entities and incomplete records that may not fully capture the clinical ground truth. While data-driven models trained on longitudinal Electronic Health Records often achieve strong empirical performance, they struggle to generalize under missing or novel conditions, largely due to their reliance on observed co-occurrence patterns. To address these issues, we propose Hierarchical Ontology and Network Refinement for Robust Medication Recommendation (HiRef), a unified framework that combines two complementary structures: (i) the hierarchical semantics encoded in curated medical ontologies, and (ii) refined co-occurrence patterns derived from real-world EHRs. We embed ontology entities in hyperbolic space, which naturally captures tree-like relationships and enables knowledge transfer through shared ancestors, thereby improving generalizability to unseen codes. To further improve robustness, we introduce a prior-guided sparse regularization scheme that refines the EHR co-occurrence graph by suppressing spurious edges while preserving clinically meaningful associations. Our model achieves strong performance on EHR benchmarks (MIMIC-III and MIMIC-IV) and maintains high accuracy under simulated unseen-code settings. Extensive experiments with comprehensive ablation studies demonstrate HiRef's resilience to unseen medical codes, supported by in-depth analyses of the learned sparsified graph structure and medical code embeddings.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
- [1]
-
[2]
An overview of clinical decision support systems: benefits, risks, and strategies for success
Reed T Sutton, David Pincock, Daniel C Baumgart, Daniel C Sadowski, Richard N Fedorak, and Karen I Kroeker. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ digital medicine, 3 0 (1): 0 17, 2020
work page 2020
-
[3]
Deep learning for medication recommendation: a systematic survey
Zafar Ali, Yi Huang, Irfan Ullah, Junlan Feng, Chao Deng, Nimbeshaho Thierry, Asad Khan, Asim Ullah Jan, Xiaoli Shen, Wu Rui, et al. Deep learning for medication recommendation: a systematic survey. Data Intelligence, 5 0 (2): 0 303--354, 2023
work page 2023
-
[4]
Rajat Mishra and S Shridevi. Knowledge graph driven medicine recommendation system using graph neural networks on longitudinal medical records. Scientific Reports, 14 0 (1): 0 25449, 2024
work page 2024
-
[5]
Safedrug: Dual molecular graph encoders for recommending effective and safe drug combinations
Chaoqi Yang, Cao Xiao, Fenglong Ma, Lucas Glass, and Jimeng Sun. Safedrug: Dual molecular graph encoders for recommending effective and safe drug combinations. arXiv preprint arXiv:2105.02711, 2021 a
arXiv 2021
-
[6]
Molerec: Combinatorial drug recommendation with substructure-aware molecular representation learning
Nianzu Yang, Kaipeng Zeng, Qitian Wu, and Junchi Yan. Molerec: Combinatorial drug recommendation with substructure-aware molecular representation learning. In Proceedings of the ACM web conference 2023, pages 4075--4085, 2023
work page 2023
-
[7]
Gamenet: Graph augmented memory networks for recommending medication combination
Junyuan Shang, Cao Xiao, Tengfei Ma, Hongyan Li, and Jimeng Sun. Gamenet: Graph augmented memory networks for recommending medication combination. In proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 1126--1133, 2019 a
work page 2019
-
[8]
Pre-training of graph augmented transformers for medication recommendation
Junyuan Shang, Tengfei Ma, Cao Xiao, and Jimeng Sun. Pre-training of graph augmented transformers for medication recommendation. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19 , pages 5953--5959. International Joint Conferences on Artificial Intelligence Organization, 7 2019 b . doi:10.24963/ijcai.2...
Show all 29 references
-
[9]
Hi-dr: Exploiting health status-aware attention and an ehr graph+ for effective medication recommendation
Taeri Kim, Jiho Heo, Hyunjoon Kim, and Sang-Wook Kim. Hi-dr: Exploiting health status-aware attention and an ehr graph+ for effective medication recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 11950--11958, 2025
2025
-
[10]
Large language models encode clinical knowledge
Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. Large language models encode clinical knowledge. Nature, 620 0 (7972): 0 172--180, 2023
2023
-
[11]
Generalized zero-shot text classification for icd coding
Congzheng Song, Shanghang Zhang, Najmeh Sadoughi, Pengtao Xie, and Eric Xing. Generalized zero-shot text classification for icd coding. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pages 4018--4024, 2021
2021
-
[12]
Constraints on biological mechanism from disease comorbidity using electronic medical records and database of genetic variants
Steven C Bagley, Marina Sirota, Richard Chen, Atul J Butte, and Russ B Altman. Constraints on biological mechanism from disease comorbidity using electronic medical records and database of genetic variants. PLoS computational biology, 12 0 (4): 0 e1004885, 2016
2016
-
[13]
Development of electronic health record based algorithms to identify individuals with diabetic retinopathy
Joseph H Breeyear, Sabrina L Mitchell, Cari L Nealon, Jacklyn N Hellwege, Brian Charest, Anjali Khakharia, Christopher W Halladay, Janine Yang, Gustavo A Garriga, Otis D Wilson, et al. Development of electronic health record based algorithms to identify individuals with diabet...
2024
-
[14]
Nuha A ElSayed, Rozalina G McCoy, Grazia Aleppo, Kirthikaa Balapattabi, Elizabeth A Beverly, Kathaleen Briggs Early, Dennis Bruemmer, Osagie Ebekozien, Justin B Echouffo-Tcheugui, Laya Ekhlaspour, et al. 2. diagnosis and classification of diabetes: Standards of care in diabete...
2025
-
[15]
Mimic-iii, a freely accessible critical care database
Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. Mimic-iii, a freely accessible critical care database. Scientific data, 3 0 (1): 0 1--9, 2016
2016
-
[16]
Mimic-iv, a freely accessible electronic health record dataset
Alistair EW Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, et al. Mimic-iv, a freely accessible electronic health record dataset. Scientific data, 10 0 (1): 0 1, 2023
2023
-
[17]
Icd-10: history and context
JA Hirsch, G Nicola, G McGinty, RW Liu, RM Barr, MD Chittle, and L Manchikanti. Icd-10: history and context. American Journal of Neuroradiology, 37 0 (4): 0 596--599, 2016
2016
-
[18]
Anatomical therapeutic chemical (atc) classification, 2021
World Health Organization et al. Anatomical therapeutic chemical (atc) classification, 2021
2021
-
[19]
Poincar \'e embeddings for learning hierarchical representations
Maximillian Nickel and Douwe Kiela. Poincar \'e embeddings for learning hierarchical representations. Advances in neural information processing systems, 30, 2017
2017
-
[20]
Gram: graph-based attention model for healthcare representation learning
Edward Choi, Mohammad Taha Bahadori, Le Song, Walter F Stewart, and Jimeng Sun. Gram: graph-based attention model for healthcare representation learning. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 787--795, 2017
2017
-
[21]
Sparse graph attention networks
Yang Ye and Shihao Ji. Sparse graph attention networks. IEEE Transactions on Knowledge and Data Engineering, 35 0 (1): 0 905--916, 2021
2021
-
[22]
Binary relevance efficacy for multilabel classification
Oscar Luaces, Jorge D \' ez, Jos \'e Barranquero, Juan Jos \'e del Coz, and Antonio Bahamonde. Binary relevance efficacy for multilabel classification. Progress in Artificial Intelligence, 1 0 (4): 0 303--313, 2012
2012
-
[23]
Classifier chains for multi-label classification
Jesse Read, Bernhard Pfahringer, Geoff Holmes, and Eibe Frank. Classifier chains for multi-label classification. In Joint European conference on machine learning and knowledge discovery in databases, pages 254--269. Springer, 2009
2009
-
[24]
Retain: An interpretable predictive model for healthcare using reverse time attention mechanism
Edward Choi, Mohammad Taha Bahadori, Jimeng Sun, Joshua Kulas, Andy Schuetz, and Walter Stewart. Retain: An interpretable predictive model for healthcare using reverse time attention mechanism. Advances in neural information processing systems, 29, 2016
2016
-
[25]
Leap: learning to prescribe effective and safe treatment combinations for multimorbidity
Yutao Zhang, Robert Chen, Jie Tang, Walter F Stewart, and Jimeng Sun. Leap: learning to prescribe effective and safe treatment combinations for multimorbidity. In proceedings of the 23rd ACM SIGKDD international conference on knowledge Discovery and data Mining, pages 1315--1324, 2017
2017
-
[26]
Change matters: Medication change prediction with recurrent residual networks
Chaoqi Yang, Cao Xiao, Lucas Glass, and Jimeng Sun. Change matters: Medication change prediction with recurrent residual networks. arXiv preprint arXiv:2105.01876, 2021 b
2021 arXiv
-
[27]
Context-aware safe medication recommendations with molecular graph and ddi graph embedding
Qianyu Chen, Xin Li, Kunnan Geng, and Mingzhong Wang. Context-aware safe medication recommendations with molecular graph and ddi graph embedding. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 7053--7060, 2023
2023
-
[28]
Lamrec: label-aware multi-view drug recommendation
Yunsen Tang, Ning Liu, Haitao Yuan, Yonghe Yan, Lei Liu, Weixing Tan, and Lizhen Cui. Lamrec: label-aware multi-view drug recommendation. In Proceedings of the 33rd ACM international conference on information and knowledge management, pages 2230--2239, 2024
2024
-
[29]
Data-driven prediction of drug effects and interactions
Nicholas P Tatonetti, Patrick P Ye, Roxana Daneshjou, and Russ B Altman. Data-driven prediction of drug effects and interactions. Science translational medicine, 4 0 (125): 0 125ra31--125ra31, 2012
2012
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.