REVIEW 4 major objections 5 minor 43 references
The paper argues that automated ICD coding improves when the coder also sees a structured, patient-level knowledge graph extracted from the clinical note, not just the raw text. On its own benchmarks, this graph branch raises Macro-F1 from
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
Integrating patient-level knowledge graphs into the PLM-ICD model improves ICD-9 coding Macro-F1 by up to 3.2% on MIMIC-III while adding explainability.
T0 review reviewed 2026-08-05 challenge →
load-bearing objection Solid extension of patient-level KG coding, but the headline F1 gains are not yet attributable to graph structure — needs a capacity-matched control and a proper validation split. the 4 major comments →
Structured Information Matters: Explainable ICD Coding with Patient-Level Knowledge Graphs
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
On the paper's own terms, the central discovery is that a patient-level knowledge graph is a useful patient representation for ICD-9 coding: it retains most of the note's information in a small fraction of its tokens, and injecting that graph into a transformer-based coder improves Macro-F1 over the text-only baseline by 1.36 points on the full label set and 3.20 points on the top-50 subset. The gains are not uniform across graph components. Ablations show that the 'clinical relationship' relation and the 'problem' entity type carry the most coding signal, while body-part-direction relations contribute least. The authors interpret these patterns as evidence that structured, entity-and-relati
What carries the argument
The load-bearing object is the patient-level knowledge graph: a document-level graph assembled from 14 entity types—problems, treatments, tests, drugs, dosages, frequencies, routes, forms, body parts, symptoms, and related categories—connected by five relation families: clinical, temporal, posology, bodypart-direction, and bodypart-problem. Entities and relations are extracted by off-the-shelf clinical named-entity recognition and relation extraction models, then serialized as triples. A deep graph convolutional network (DGCNN) embeds the graph, with node features concatenated across layers, and these graph embeddings are fused with text segment embeddings from a biomedical pre-trained trans
Load-bearing premise
The observed F1 gains are caused by the structured content of the knowledge graph, not by the added DGCNN encoder and extra attention head; if a text-only model with equal added capacity matched the graph model's F1, the structured-information explanation would be unsupported.
What would settle it
Train the same model with a control graph built from randomly paired entities and random relation labels, keeping graph size and model capacity fixed. If Macro-F1 stays near 11.05 on the full label set—or the gap to the text-only baseline stays near 1.36—then the specific structure of the graph is not what drives the gain. The paper's own entity and relation ablations provide a direct partial check: removing the problem entity drops Macro-F1 to 9.77, so a random-graph control would fill in what remains unexplained.
If this is right
- ICD coding accuracy improves on both the full and top-50 code sets, with the largest gains on frequent codes and on codes the text-only model misses entirely, such as anemia and personal history of tobacco use.
- Training efficiency improves: the graph branch reaches competitive Macro-F1 within the first few epochs, which is practically useful when computational resources are constrained.
- The ablation results give a prioritization signal for graph construction: clinical relationships and problem entities matter most, so future extraction pipelines can focus annotation or model effort on those components.
- The graph offers a compact structured artifact—about a quarter of the original note—that can serve simultaneously as an input feature and as a source of attention-based evidence for predictions.
- The paper's gains are demonstrated on top of its chosen baseline; the authors note that integrating structured input into more recent, stronger coding architectures could yield further improvements.
Where Pith is reading between the lines
- My inference: the same graph-construction and fusion recipe would likely transfer to other clinical coding settings, including ICD-10 and long clinical documents beyond discharge summaries, because the entity and relation types are largely vocabulary-agnostic; this transfer is not tested in the paper.
- My inference: a matched-capacity text-only control or a control with randomly wired graphs would isolate whether the benchmark improvements come from the graph's structure or simply from the extra DGCNN parameters; without such a control, the causal attribution to structure remains open.
- My inference: the entropy-based compression measure could be used as a routing signal—documents whose graphs lose more information than usual could be flagged for additional full-text processing, a practical design the paper does not explore.
- My inference: the attention highlights shown in the case studies are visually more focused than the text-only baseline, but turning this into a trustworthy clinician-facing explanation would require expert evalution rather than visual inspection alone.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes constructing patient-level knowledge graphs from MIMIC-III discharge summaries using Healthcare NLP named-entity recognition and relation-extraction models, yielding 14 entity types and 5 relationship types. These graphs are integrated into the PLM-ICD architecture by adding a DGCNN encoder whose node representations are initialized with RoBERTa-base, followed by a separate multi-head label-wise attention layer; text and graph representations are concatenated for ICD-9 code prediction. The authors report Macro-F1 improvements over PLM-ICD of +1.36 on MIMIC-III Full (9.69 to 11.05) and +3.20 on Top-50 (64.61 to 67.81), faster early-epoch training, an entropy-based claim that the graph retains about 90% of the information in 23% of the original text, and qualitative evidence of improved attention-based explanations. Ablations remove individual relationship and entity types to analyze their contribution.
Significance. The idea of using patient-level knowledge graphs to represent the input document, rather than only to enhance code representations, is a genuinely underexplored direction in automated ICD coding. If the reported gains are robust, the approach could provide a practical way to inject structured clinical information into PLM-based coders and to improve the conciseness of attention-based explanations. However, the significance is currently moderated by the absence of a capacity-matched text-only control and a structure-destroying graph control, by apparent model selection on the test set, and by the absence of variance/error reporting. These issues prevent the paper from establishing its central causal claim that the structure of the knowledge graph, rather than added model capacity or surface entity cues, drives the improvement.
major comments (4)
- [§3, Figure 2, Eqs. (5)–(8); Table 4]
- [Appendix A.4, Tables 10–11]
- [§4.2, Table 3]
- [§3, Eqs. (10)–(13); Tables 2 and 7]
minor comments (5)
- [Appendix A.1, Table 8 text]
- [Figure 2]
- [§6 Limitations]
- [Eq. (9)]
- [Figure 6]
Circularity Check
No significant circularity: F1 gains are genuine held-out predictions; the only self-citation (Nguyen et al. 2023) is non-load-bearing, and the capacity-matched-control concern is a confound, not a definitional reduction.
full rationale
The paper's central claim is that integrating patient-level knowledge graphs into PLM-ICD improves ICD coding, evaluated on MIMIC-III Full and Top-50 held-out test sets with external ICD-9 labels. The Macro-F1 improvements are measured predictions from a trained model, not quantities fitted to the test labels; no equation defines the output in terms of the KG construction itself. Equations (1)-(9) describe a standard text-plus-graph multi-label classifier with binary cross-entropy loss on held-out labels, so the coding result is not equivalent to the input by construction. The entropy-based 'information retained' metric (Table 2, Eqs. 10-13) is an internal compression statistic comparing serialized graph to original text; it is not presented as the coding result, so it does not create circularity in the coding claim. The ablations (Table 4) are empirical sensitivity analyses that retain the same model architecture, and although they do not isolate graph structure from added capacity, that is a confound in experimental attribution, not a circular derivation. The only self-citation is Nguyen et al. (2023), co-authored by Viktor Schlegel, used as a baseline (2Stage) and referenced in Limitations as a more advanced model; it is not load-bearing for the paper's central claim. The paper itself acknowledges not testing other baseline architectures. No specific circular step can be exhibited via quotation and reduction, so the score reflects only the minor non-load-bearing self-citation and otherwise the evaluation is self-contained against external benchmarks.
Axiom & Free-Parameter Ledger
free parameters (2)
- DGCNN architecture (layers and node embedding size) =
Full: 1 layer, 768; Top-50: 2 layers, 384-384
- RE model selection (5 of 14) =
CR, TE, PR, BD, BP
axioms (4)
- domain assumption Named entities and relationships extracted by the off-the-shelf Healthcare NLP models are sufficiently accurate and relevant for ICD coding.
- ad hoc to paper Token-level Shannon entropy of a serialized graph measures how much clinical information is retained.
- domain assumption Label-wise attention weights in the graph branch represent feature importance for explainability.
- domain assumption The graph and text representations provide complementary, non-redundant information.
Cite this review
Pith. "Pith review of Structured Information Matters: Explainable ICD Coding with Patient-Level Knowledge Graphs." pith.science (2026). https://pith.science/paper/XNEBNCAM
@misc{pith2026250909699,
author = {Pith},
title = {Pith review of: Structured Information Matters: Explainable ICD Coding with Patient-Level Knowledge Graphs},
year = {2026},
howpublished = {\url{https://pith.science/paper/XNEBNCAM}},
note = {Machine review of arXiv:2509.09699}
}
read the original abstract
Mapping clinical documents to standardised clinical vocabularies is an important task, as it provides structured data for information retrieval and analysis, which is essential to clinical research, hospital administration and improving patient care. However, manual coding is both difficult and time-consuming, making it impractical at scale. Automated coding can potentially alleviate this burden, improving the availability and accuracy of structured clinical data. The task is difficult to automate, as it requires mapping to high-dimensional and long-tailed target spaces, such as the International Classification of Diseases (ICD). While external knowledge sources have been readily utilised to enhance output code representation, the use of external resources for representing the input documents has been underexplored. In this work, we compute a structured representation of the input documents, making use of document-level knowledge graphs (KGs) that provide a comprehensive structured view of a patient's condition. The resulting knowledge graph efficiently represents the patient-centred input documents with 23\% of the original text while retaining 90\% of the information. We assess the effectiveness of this graph for automated ICD-9 coding by integrating it into the state-of-the-art ICD coding architecture PLM-ICD. Our experiments yield improved Macro-F1 scores by up to 3.20\% on popular benchmarks, while improving training efficiency. We attribute this improvement to different types of entities and relationships in the KG, and demonstrate the improved explainability potential of the approach over the text-only baseline.
Figures
Reference graph
Works this paper leans on
-
[1]
James Blundell. 2023. Health information and the importance of clinical coding. Anaesthesia & Intensive Care Medicine
work page 2023
-
[2]
Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao, Shengping Liu, and Weifeng Chong. 2020. Hypercore: Hyperbolic and co-graph representation for automatic icd coding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3105--3114
work page 2020
-
[3]
Finneas Catling, Georgios P Spithourakis, and Sebastian Riedel. 2018. Towards automated clinical coding. International journal of medical informatics, 120:50--61
work page 2018
-
[4]
Ilias Chalkidis, Manos Fergadiotis, Sotiris Kotitsas, Prodromos Malakasiotis, Nikolaos Aletras, and Ion Androutsopoulos. 2020. An empirical study on large-scale multi-label text classification including few and zero-shot labels. arXiv preprint arXiv:2010.01653
work page internal anchor Pith review Pith/arXiv arXiv 2020
-
[5]
Koby Crammer, Mark Dredze, Kuzman Ganchev, Partha Talukdar, and Steven Carroll. 2007. Automatic code assignment to medical text. In Biological, translational, and clinical language processing, pages 129--136
work page 2007
-
[6]
Hang Dong, V \' ctor Su \'a rez-Paniagua, William Whiteley, and Honghan Wu. 2021. Explainable automated coding of clinical notes using hierarchical label-wise attention networks and label embedding initialisation. Journal of biomedical informatics, 116:103728
work page 2021
-
[7]
Joakim Edin, Alexander Junge, Jakob D Havtorn, Lasse Borgholt, Maria Maistro, Tuukka Ruotsalo, and Lars Maal e. 2023. Automated medical coding on mimic-iii and mimic-iv: a critical review and replicability study. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2572--2582
work page 2023
-
[8]
Mat \'u s Falis, Hang Dong, Alexandra Birch, and Beatrice Alex. 2022. Horses to zebras: ontology-guided data augmentation and synthesis for icd-9 coding. In Proceedings of the 21st Workshop on Biomedical Language Processing. Association for Computational Linguistics
work page 2022
-
[9]
Mat \'u s Falis, Maciej Pajak, Aneta Lisowska, Patrick Schrempf, Lucas Deckers, Shadia Mikhael, Sotirios Tsaftaris, and Alison O’Neil. 2019. Ontological attention ensembles for capturing semantic concepts in icd code prediction from clinical text. In Proceedings of the Tenth International Workshop on Health Text Mining and Information Analysis (LOUHI 2019...
work page 2019
-
[10]
Malte Feucht, Zhiliang Wu, Sophia Althammer, and Volker Tresp. 2021. Description-based label attention classifier for explainable icd-9 classification. arXiv preprint arXiv:2109.12026
Pith/arXiv arXiv 2021
-
[11]
Chao-Wei Huang, Shang-Chi Tsai, and Yun-Nung Chen. 2022. Plm-icd: automatic icd coding with pretrained language models. arXiv preprint arXiv:2207.05289
Pith/arXiv arXiv 2022
-
[12]
Shaoxiong Ji, Matti H \"o ltt \"a , and Pekka Marttinen. 2021. Does the magic of bert apply to medical code assignment? a quantitative study. Computers in biology and medicine, 139:104998
work page 2021
-
[13]
Shaoxiong Ji, Xiaobo Li, Wei Sun, Hang Dong, Ara Taalas, Yijia Zhang, Honghan Wu, Esa Pitk \"a nen, and Pekka Marttinen. 2022. A unified review of deep learning for automated medical coding. ACM Computing Surveys
work page 2022
-
[14]
John Snow Labs . 2024. Healthcare NLP . https://www.johnsnowlabs.com/healthcare-nlp/
work page 2024
-
[15]
Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. 2016. Mimic-iii, a freely accessible critical care database. Scientific data, 3(1):1--9
work page 2016
-
[16]
Patrick Lewis, Myle Ott, Jingfei Du, and Veselin Stoyanov. 2020. Pretrained language models for biomedical and clinical tasks: understanding and extending the state-of-the-art. In Proceedings of the 3rd clinical natural language processing workshop, pages 146--157
work page 2020
-
[17]
Fei Li and Hong Yu. 2020. Icd coding from clinical text using multi-filter residual convolutional neural network. In proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8180--8187
work page 2020
-
[18]
Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel. 2015. Gated graph sequence neural networks. arXiv preprint arXiv:1511.05493
Pith/arXiv arXiv 2015
-
[19]
Yang Liu, Hua Cheng, Russell Klopfer, Matthew R Gormley, and Thomas Schaaf. 2021. Effective convolutional attention network for multi-label clinical document classification. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5941--5953
work page 2021
-
[20]
Jueqing Lu, Lan Du, Ming Liu, and Joanna Dipnall. 2020. Multi-label few/zero-shot learning with knowledge aggregated from multiple label graphs. arXiv preprint arXiv:2010.07459
work page internal anchor Pith review Pith/arXiv arXiv 2020
-
[21]
George Michalopoulos, Michal Malyska, Nicola Sahar, Alexander Wong, and Helen Chen. 2022. Icdbigbird: a contextual embedding model for icd code classification. arXiv preprint arXiv:2204.10408
work page internal anchor Pith review Pith/arXiv arXiv 2022
-
[22]
James Mullenbach, Sarah Wiegreffe, Jon Duke, Jimeng Sun, and Jacob Eisenstein. 2018. Explainable prediction of medical codes from clinical text. arXiv preprint arXiv:1802.05695
Pith/arXiv arXiv 2018
-
[23]
Anthony N Nguyen, Donna Truran, Madonna Kemp, Bevan Koopman, David Conlan, John O’Dwyer, Ming Zhang, Sarvnaz Karimi, Hamed Hassanzadeh, Michael J Lawley, et al. 2018. Computer-assisted diagnostic coding: effectiveness of an nlp-based approach using snomed ct to icd-10 mappings. In AMIA Annual Symposium Proceedings, volume 2018, page 807. American Medical ...
work page 2018
-
[24]
Thanh-Tung Nguyen, Viktor Schlegel, Abhinav Kashyap, and Stefan Winkler. 2023. A two-stage decoder for efficient icd coding. arXiv preprint arXiv:2306.00005
work page internal anchor Pith review Pith/arXiv arXiv 2023
-
[25]
World Health Organization et al. 1978. International classification of diseases:[9th] ninth revision, basic tabulation list with alphabetic index. World Health Organization
work page 1978
-
[26]
Suzanne Pereira, Aur \'e lie N \'e v \'e ol, Philippe Massari, Michel Joubert, and Stefan Darmoni. 2006. Construction of a semi-automated icd-10 coding help system to optimize medical and economic coding. In MIE, pages 845--850. Citeseer
work page 2006
-
[27]
Anthony Rios and Ramakanth Kavuluru. 2018. Few-shot and zero-shot multi-label learning for structured label spaces. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing, volume 2018, page 3132. NIH Public Access
work page 2018
-
[28]
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In The semantic web: 15th international conference, ESWC 2018, Heraklion, Crete, Greece, June 3--7, 2018, proceedings 15, pages 593--607. Springer
work page 2018
-
[29]
Congzheng Song, Shanghang Zhang, Najmeh Sadoughi, Pengtao Xie, and Eric Xing. 2021. Generalized zero-shot text classification for icd coding. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pages 4018--4024
work page 2021
-
[30]
Wei Sun, Shaoxiong Ji, Erik Cambria, and Pekka Marttinen. 2021. Multitask recalibrated aggregation network for medical code prediction. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 367--383. Springer
work page 2021
-
[31]
Fei Teng, Yiming Liu, Tianrui Li, Yi Zhang, Shuangqing Li, and Yue Zhao. 2022. A review on deep neural networks for icd coding. IEEE Transactions on Knowledge and Data Engineering, 35(5):4357--4375
work page 2022
-
[32]
Betty Van Aken, Jens-Michalis Papaioannou, Marcel G Naik, Georgios Eleftheriadis, Wolfgang Nejdl, Felix A Gers, and Alexander L \"o ser. 2022. This patient looks like that patient: Prototypical networks for interpretable diagnosis prediction from clinical text. arXiv preprint arXiv:2210.08500
work page internal anchor Pith review Pith/arXiv arXiv 2022
-
[33]
Petar Veli c kovi \'c , Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903
Pith/arXiv arXiv 2017
-
[34]
Thanh Vu, Dat Quoc Nguyen, and Anthony Nguyen. 2020. A label attention model for icd coding from clinical text. arXiv preprint arXiv:2007.06351
Pith/arXiv arXiv 2020
-
[35]
Tao Wang, Linhai Zhang, Chenchen Ye, Junxi Liu, and Deyu Zhou. 2022. A novel framework based on medical concept driven attention for explainable medical code prediction via external knowledge. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1407--1416
work page 2022
-
[36]
Xiancheng Xie, Yun Xiong, Philip S Yu, and Yangyong Zhu. 2019. Ehr coding with multi-scale feature attention and structured knowledge graph propagation. In Proceedings of the 28th ACM international conference on information and knowledge management, pages 649--658
work page 2019
-
[37]
Zhichao Yang, Shufan Wang, Bhanu Pratap Singh Rawat, Avijit Mitra, and Hong Yu. 2022. Knowledge injected prompt based fine-tuning for multi-label few-shot icd coding. In Proceedings of the conference on empirical methods in natural language processing. Conference on empirical methods in natural language processing, volume 2022, page 1767. NIH Public Access
work page 2022
-
[38]
Quan Yuan, Jun Chen, Chao Lu, and Haifeng Huang. 2021. The graph-based mutual attentive network for automatic diagnosis. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pages 3393--3399
work page 2021
-
[39]
Zheng Yuan, Chuanqi Tan, and Songfang Huang. 2022. Code synonyms do matter: Multiple synonyms matching network for automatic icd coding. arXiv preprint arXiv:2203.01515
work page internal anchor Pith review Pith/arXiv arXiv 2022
-
[40]
Muhan Zhang, Zhicheng Cui, Marion Neumann, and Yixin Chen. 2018. An end-to-end deep learning architecture for graph classification. In AAAI
work page 2018
-
[41]
Zachariah Zhang, Jingshu Liu, and Narges Razavian. 2020. Bert-xml: Large scale automated icd coding using bert pretraining. arXiv preprint arXiv:2006.03685
work page internal anchor Pith review Pith/arXiv arXiv 2020
-
[42]
online" 'onlinestring :=
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
-
[43]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
This paper was first reviewed by deepseek-v4-flash on August 5, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.