REVIEW 4 major objections 4 minor 1 cited by
A General Knowledge Injection Framework for ICD Coding
T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read GKI-ICD turns ICD descriptions, synonyms, and hierarchy into synthetic guidelines that teach coding models during training alone, improving accuracy, especially on rare codes.
desk verdict A simple, architecture-agnostic knowledge-injection framework for ICD coding that clearly helps rare codes, but the central similarity-loss mechanism is never ablated and the SOTA claim is overstated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the synthesized guideline $\hat{x}$: for each training document, the assigned codes are shuffled, and each code is expanded into its official description, a randomly chosen synonym, and the descriptions of all parent groups in the ICD hierarchy, then concatenated into a single text string. The training signal combines three losses: binary cross-entropy on the raw clinical text, binary cross-entropy on the guideline, and a cosine-similarity term $L_{\text{sim}} = 1 - \cos(E, \hat{E})$ that aligns the aggregated code-specific evidence from both inputs. The framework's novelty is that the knowledge enters only through this synthetic text and these losses, not through graph networks, synonym-attention modules, or prompt engineering.
What would settle it
Replace every code's description, synonym, and hierarchy in the synthesized guidelines with random text drawn from the same source vocabulary and retrain; if validation F1 does not drop relative to the full framework, the specific knowledge content is not what drives the gain. A complementary check is to hold out all examples of a rare code from training and see whether the rare-code F1 gain over the base model disappears, as the paper's mechanism predicts it should.
Extended reading notes
Core claim
The paper's central claim is that a training-time framework, GKI-ICD, can inject three complementary types of ICD knowledge — the official description of each assigned code, randomly sampled synonyms from a biomedical terminology source, and the code's place in the ICD hierarchy — by synthesizing a clean guideline text and training the model to do three things at once: predict codes from that guideline, predict codes from the raw clinical note, and pull the code-specific evidence representations of the two inputs toward each other with a cosine similarity loss. No new network modules are introduced, so the same recipe attaches to any existing encoder-and-label-attention coding model. The paper reports that this beats its strongest baselines across nearly all metrics on the standard full and top-50 clinical benchmarks, and that the biggest gain appears on codes with ten or fewer training examples.
Load-bearing premise
The whole approach assumes that training on clean summary sentences built from the true codes makes the model better at finding evidence in messy real clinical notes, rather than simply memorizing the summary format.
Editorial extensions
If this is right
- Any ICD coding model that already extracts code-specific evidence from text can adopt the framework without architectural changes, so knowledge injection becomes compatible with future backbones.
- Combining description, synonym, and hierarchy knowledge yields higher accuracy than any single knowledge source alone, which the ablation tables support.
- The largest measured improvement is on rare codes (ten or fewer training examples), suggesting the framework directly targets the long-tail problem that dominates ICD coding.
- Because the knowledge enters only at training time, inference speed and memory use are unchanged relative to the base model.
- The framework is a general multi-label recipe: any task with label-specific descriptions, synonyms, or hierarchies could reuse the same guideline-synthesis and alignment scheme.
Reading between the lines
- A testable extension is applying the same recipe to ICD-10 or other coded vocabularies; the paper only validates on ICD-9, so whether the gains survive the larger, more granular code space is open.
- The cosine alignment is one of many possible constraints; a contrastive or distribution-matching alternative might be more robust when the guideline and the note are semantically distant, but the paper does not compare such variants.
- The rare-code gain hints that the framework could reduce dependence on extra human-annotated inputs such as DRG or procedure codes, but the paper only shows comparable performance, not that the framework can replace those annotations.
- Because synonym selection is random, the method could be made stronger with a curriculum that prefers synonyms hard to align to the raw text; the paper does not test this.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes GKI-ICD, a training-time knowledge-injection framework for ICD coding. For each training document, it synthesizes a 'guideline' string from the ground-truth ICD codes' official descriptions, UMLS synonyms, and ICD-9 hierarchical descriptions. The ICD coding model (RoBERTa-PM encoder with cross-attention label queries, following PLM-CA) is then trained with three losses: binary cross-entropy on the raw text, binary cross-entropy on the synthetic guideline, and a cosine-similarity constraint between the code-specific evidence vectors extracted from the raw text and from the guideline. Experiments on MIMIC-III-Full and MIMIC-III-Top-50 report improvements over several baselines and particularly large gains for rare codes, and the code is publicly released. The manuscript's own Limitations section acknowledges that the method is evaluated only on ICD-9 and that it does not use the Alphabetic Index.
Significance. If the central claim holds, GKI-ICD is a useful contribution: it is architecture-agnostic, requires no extra parameters or specialized modules, leaves inference-time cost unchanged, and shows a large rare-code improvement (Table 4, F1 0.132 vs. 0.029 for codes occurring at most 10 times). The paper ships code, states the loss terms explicitly, and honestly lists its scope limitations (ICD-9 only, no Alphabetic Index). However, the current experimental evidence does not isolate the mechanism that is claimed to be central---the semantic-similarity alignment of Eq. (14)---and the reported SOTA claim is stronger than the table supports. These gaps are fixable but require additional experiments and a more careful presentation of results.
major comments (4)
- [Section 4.2, Table 2] The claim that 'our method outperforms state-of-the-arts significantly on all evaluation metrics' is not supported by the table. On MIMIC-III-Top-50, CoRelation has higher Macro F1 (0.693 vs. 0.692) and higher P@5 (0.683 vs. 0.681), and MRR has higher P@5 (0.685). On MIMIC-III-Full, AKIL and MRR have higher P@15 (0.637 and 0.623 vs. 0.624 for GKI-ICD) and AKIL has higher P@8 (0.784 vs. 0.777). Even setting aside the methods that use additional human annotations, CoRelation beats GKI-ICD on two Top-50 metrics. The paper should state precisely on which metrics and against which baselines GKI-ICD is state-of-the-art, and should not claim superiority on all metrics.
- [Section 4.3, Table 3 and Eq. (15)] The ablation study never removes the semantic-similarity loss L_sim. Every 'w/ knowledge' row in Table 3 is trained with the full objective L_raw + L_guide + lambda*L_sim, while the 'w/o knowledge' row already includes R-Drop regularization and 8,192-token truncation. Consequently, the contribution of Eq. (14)---the mechanism claimed to align raw-text evidence with guideline evidence and thereby inject knowledge---is completely confounded with L_guide and with R-Drop. The reported gains could in principle come entirely from supervising the model on clean code-description strings (L_guide) rather than from the cross-sample similarity constraint. An ablation with L_raw + L_guide but without L_sim, and ideally also a variant with L_sim alone or with lambda=0, is required to support the paper's central claim about knowledge alignment.
- [Section 4.1, Eq. (15)] The hyperparameter lambda that weights the semantic-similarity loss is never reported. Eq. (15) defines the total loss with lambda, but the implementation-details paragraph gives only the learning rate, batch size, epochs, and the R-Drop coefficient alpha. Since L_sim is the central novel term, the reader cannot tell whether lambda was tuned, what value was used, or whether the reported results are sensitive to it. A sensitivity study over lambda (e.g., 0.1, 1, 10) should be added, and the selected value must be stated.
- [Section 4.2, Table 3 note] The main comparison against PLM-CA is not apples-to-apples. The note under Table 3 says that all models, including the 'w/o knowledge' baseline, use R-Drop and truncate input to 8,192 tokens, whereas PLM-CA used 6,144 tokens and no R-Drop. The reported 4.6% Macro-AUC improvement over PLM-CA on MIMIC-III-Full therefore mixes the effect of knowledge injection with the effects of additional regularization and a longer truncation window. The authors should either re-run PLM-CA under the same training recipe, or use the 'w/o knowledge' row as the direct control in the SOTA table. Additionally, no multiple-seed results or significance tests are reported, so the word 'significantly' is not justified even where the improvements are numerically positive.
minor comments (4)
- [Section 3.3, Eqs. (17)-(18)] The dimension of the attention matrix A_i is inconsistent: Eq. (17) computes a vector of attention weights over the N input tokens, but the text and Eq. (18) describe A_i as an R^{C x N} matrix. A_i should be a vector in R^N for each code i, and the notation should be corrected.
- [Throughout] There are several typos and corrupted figure strings: 'knowldge' in the contributions list, 'fisrt' in Section 2.1, 'Prediciton' and 'Esophegeal Reflux' in Figure 2, and the nonsensical synonym strings 'sahedfoufhlsreflux' and 'shcofiaodi' in Figure 2. The figure should be regenerated and proofread.
- [Section 4.2] The phrase 'state-of-the-arts' is nonstandard; 'state-of-the-art methods' or 'state of the art' would be clearer.
- [Section 4.1] The implementation-details paragraph does not state the number of random seeds, the random seed, or how hyperparameters (including lambda and alpha) were selected. Adding these details would improve reproducibility.
Circularity Check
No circularity: knowledge injection uses external code knowledge and training-time labels; test-time inference is label-free.
full rationale
GKI-ICD's guideline synthesis (Section 3.2.1) builds x̂ from ground-truth codes plus external knowledge sources (ICD descriptions, UMLS synonyms, ICD hierarchy). This is a training-time augmentation: the model is supervised on both raw text and guideline with the same labels, and the similarity loss (Eq. 14) aligns raw-text evidence with guideline evidence. At test time, only raw text is used, so no ground-truth label is required for inference. The guideline is not a prediction of test labels; it is a transformed training input. No parameter is fitted to a subset of data and then presented as a prediction of a closely related quantity. The paper cites prior work for standard components (R-Drop, RoBERTa-PM, PLM-CA) but none of these citations are self-citations, and no load-bearing premise is justified solely by the authors' own prior work. The absence of an ablation that isolates Lsim from Lguide is an experimental-support limitation, not circularity, because the reported improvements could still come from the auxiliary supervision without the similarity constraint. The derivation chain is therefore self-contained and not circular.
Assumptions & free parameters
free parameters (2)
- lambda =
not reported
- alpha (R-Drop) =
5 for MIMIC-III-Full, 10 for MIMIC-III-Top-50
assumptions (3)
- domain assumption The ICD code descriptions, UMLS synonyms, and ICD-9 hierarchy are accurate and complete enough to serve as effective training signals.
- domain assumption The multi-task learning setup (predicting labels from both raw text and guideline, plus similarity constraint) does not hurt the raw-text classification task.
- standard math The benchmark splits and evaluation settings follow Mullenbach et al. (2018) and Edin et al. (2024), enabling fair comparison.
Cite this review
Pith. "Pith review of A General Knowledge Injection Framework for ICD Coding." pith.science (2026). https://pith.science/paper/N4EN62FL
@misc{pith2026250518708,
author = {Pith},
title = {Pith review of: A General Knowledge Injection Framework for ICD Coding},
year = {2026},
howpublished = {\url{https://pith.science/paper/N4EN62FL}},
note = {Machine review of arXiv:2505.18708}
}
read the original abstract
ICD Coding aims to assign a wide range of medical codes to a medical text document, which is a popular and challenging task in the healthcare domain. To alleviate the problems of long-tail distribution and the lack of annotations of code-specific evidence, many previous works have proposed incorporating code knowledge to improve coding performance. However, existing methods often focus on a single type of knowledge and design specialized modules that are complex and incompatible with each other, thereby limiting their scalability and effectiveness. To address this issue, we propose GKI-ICD, a novel, general knowledge injection framework that integrates three key types of knowledge, namely ICD Description, ICD Synonym, and ICD Hierarchy, without specialized design of additional modules. The comprehensive utilization of the above knowledge, which exhibits both differences and complementarity, can effectively enhance the ICD coding performance. Extensive experiments on existing popular ICD coding benchmarks demonstrate the effectiveness of GKI-ICD, which achieves the state-of-the-art performance on most evaluation metrics. Code is available at https://github.com/xuzhang0112/GKI-ICD.
Figures
Forward citations
Cited by 1 Pith paper
-
Expert-guided Clinical Text Augmentation via Query-Based Model Collaboration
Constraining LLM rewriting with a biomedical NER model improves medical entity preservation and reduces hallucinations in synthetic clinical notes, with modest downstream gains on MIMIC-III tasks.
Reference graph
Works this paper leans on
-
[1]
Olivier Bodenreider. 2004. The unified medical language system (umls): integrating biomedical terminology. Nucleic acids research, 32(suppl\_1):D267--D270
work page 2004
-
[2]
Joseph Boyle, Antanas Kascenas, Pat Lok, Maria Liakata, and Alison O'Neil. 2023. https://openreview.net/forum?id=mqnR8rGWkn Automated clinical coding using off-the-shelf large language models . In Deep Generative Models for Health Workshop NeurIPS 2023
work page 2023
-
[3]
Joakim Edin, Alexander Junge, Jakob D Havtorn, Lasse Borgholt, Maria Maistro, Tuukka Ruotsalo, and Lars Maal e. 2023. Automated medical coding on mimic-iii and mimic-iv: a critical review and replicability study. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2572--2582
2023
-
[4]
Joakim Edin, Maria Maistro, Lars Maal e, Lasse Borgholt, Jakob Drachmann Havtorn, and Tuukka Ruotsalo. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.280 An unsupervised approach to achieve supervised-level explainability in healthcare records . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 4869--4890,...
-
[5]
Xueren Ge, Abhishek Satpathy, Ronald Dean Williams, John Stankovic, and Homa Alemzadeh. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.712 DKEC : Domain knowledge enhanced multi-label classification for diagnosis prediction . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 12798--12813, Miami, Florida, U...
-
[6]
Goncalo Gomes, Isabel Coutinho, and Bruno Martins. 2024. https://aclanthology.org/2024.eacl-long.141/ Accurate and well-calibrated ICD code assignment through attention over diverse label embeddings . In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2302--2315, St...
work page 2024
-
[7]
Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2021. Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare (HEALTH), 3(1):1--23
2021
-
[8]
Chao-Wei Huang, Shang-Chi Tsai, and Yun-Nung Chen. 2022. https://doi.org/10.18653/v1/2022.clinicalnlp-1.2 PLM - ICD : Automatic ICD coding with pretrained language models . In Proceedings of the 4th Clinical Natural Language Processing Workshop, pages 10--20, Seattle, WA. Association for Computational Linguistics
Show all 29 references
-
[9]
Kexin Huang, Jaan Altosaar, and Rajesh Ranganath. 2019. Clinicalbert: Modeling clinical notes and predicting hospital readmission. arXiv preprint arXiv:1904.05342
2019 arXiv
-
[10]
Shaoxiong Ji, Xiaobo Li, Wei Sun, Hang Dong, Ara Taalas, Yijia Zhang, Honghan Wu, Esa Pitk \"a nen, and Pekka Marttinen. 2024. A unified review of deep learning for automated medical coding. ACM Computing Surveys, 56(12):1--41
2024
-
[11]
Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. 2016. Mimic-iii, a freely accessible critical care database. Scientific data, 3(1):1--9
2016
-
[12]
Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234--1240
2020
-
[13]
Patrick Lewis, Myle Ott, Jingfei Du, and Veselin Stoyanov. 2020. Pretrained language models for biomedical and clinical tasks: understanding and extending the state-of-the-art. In Proceedings of the 3rd clinical natural language processing workshop, pages 146--157
2020
-
[14]
Fei Li and Hong Yu. 2020. Icd coding from clinical text using multi-filter residual convolutional neural network. In proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8180--8187
2020
-
[15]
Gormley, and Thomas Schaaf
Yang Liu, Hua Cheng, Russell Klopfer, Matthew R. Gormley, and Thomas Schaaf. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.481 Effective convolutional attention network for multi-label clinical document classification . In Proceedings of the 2021 Conference on Empirical Me...
2021 doi
-
[16]
Junyu Luo, Xiaochen Wang, Jiaqi Wang, Aofei Chang, Yaqing Wang, and Fenglong Ma. 2024. https://aclanthology.org/2024.lrec-main.355/ C o R elation: Boosting automatic ICD coding through contextualized code relation learning . In Proceedings of the 2024 Joint International Confe...
2024
-
[17]
James Mullenbach, Sarah Wiegreffe, Jon Duke, Jimeng Sun, and Jacob Eisenstein. 2018. https://doi.org/10.18653/v1/N18-1100 Explainable prediction of medical codes from clinical text . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Co...
2018 doi
-
[18]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32
2019
-
[19]
Thanh Vu, Dat Quoc Nguyen, and Anthony Nguyen. 2021. A label attention model for icd coding from clinical text. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI'20
2021
-
[20]
Guoyin Wang, Chunyuan Li, Wenlin Wang, Yizhe Zhang, Dinghan Shen, Xinyuan Zhang, Ricardo Henao, and Lawrence Carin. 2018. https://doi.org/10.18653/v1/P18-1216 Joint embedding of words and labels for text classification . In Proceedings of the 56th Annual Meeting of the Associa...
2018 doi
-
[21]
Xindi Wang, Robert Mercer, and Frank Rudzicz. 2024 a . https://doi.org/10.18653/v1/2024.naacl-long.273 Multi-stage retrieve and re-rank model for automatic medical coding recommendation . In Proceedings of the 2024 Conference of the North American Chapter of the Association fo...
2024 doi
-
[22]
Mercer, and Frank Rudzicz
Xindi Wang, Robert E. Mercer, and Frank Rudzicz. 2024 b . https://aclanthology.org/2024.lrec-main.181/ Auxiliary knowledge-induced learning for automatic multi-label medical document classification . In Proceedings of the 2024 Joint International Conference on Computational Li...
2024
-
[23]
Lijun Wu, Juntao Li, Yue Wang, Qi Meng, Tao Qin, Wei Chen, Min Zhang, Tie-Yan Liu, et al. 2021. R-drop: Regularized dropout for neural networks. Advances in neural information processing systems, 34:10890--10905
2021
-
[24]
Xiancheng Xie, Yun Xiong, Philip S Yu, and Yangyong Zhu. 2019. Ehr coding with multi-scale feature attention and structured knowledge graph propagation. In Proceedings of the 28th ACM international conference on information and knowledge management, pages 649--658
2019
-
[25]
Zhichao Yang, Shufan Wang, Bhanu Pratap Singh Rawat, Avijit Mitra, and Hong Yu. 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.127 Knowledge injected prompt based fine-tuning for multi-label few-shot ICD coding . In Findings of the Association for Computational Linguist...
2022 doi
-
[26]
Zheng Yuan, Chuanqi Tan, and Songfang Huang. 2022. https://doi.org/10.18653/v1/2022.acl-short.91 Code synonyms do matter: Multiple synonyms matching network for automatic ICD coding . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (V...
2022 doi
-
[27]
Tong Zhou, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao, Kun Niu, Weifeng Chong, and Shengping Liu. 2021. https://doi.org/10.18653/v1/2021.acl-long.463 Automatic ICD coding via interactive shared representation networks with self-distillation mechanism . In Proceedings of the 59...
2021 doi
-
[28]
online" 'onlinestring :=
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
-
[29]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.