Pith. sign in

REVIEW 3 major objections 7 minor 47 references

Practical Design and Benchmarking of Generative AI Applications for Surgical Billing and Coding

T0 review · 3 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A locally fine-tuned small language model can generate surgical billing codes from operative reports as accurately as GPT-4o, while training on four consumer-grade 24GB GPUs.

desk verdict A useful, mostly sound empirical benchmark of fine-tuned Phi-3 models for surgical billing coding, but the undisclosed GPT-4o baseline makes the headline comparison unauditable. read the letter →

arxiv 2501.05479 v1 pith:Y2MXRA7R submitted 2025-01-07 cs.CL cs.LG

John C. Rollman (1) , Bruce Rogers (1) , Hamed Zaribafzadeh (1) , Daniel Buckland (2) , Ursula Rogers (1) , Jennifer Gagnon (1) , Ozanan Meireles (1) , Lindsay Jennings (3)
show 10 more authors
Jim Bennett (1) Jennifer Nicholson (3) Nandan Lad (4) Linda Cendales (1) Andreas Seas (4 5 6) Alessandro Martinino (6) E. Shelley Hwang (1) Allan D. Kirk (1)
This is my paper · ORCID
classification cs.CLcs.LG
keywords medicalbillingandcodingICD-10-CMCPTcodesmodifierQLoRAfine-tuningPhi-3smalllanguagemodelssurgicaloperativereportshealthcaregenerativeAI
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reports a practical recipe for building a generative-AI medical billing coder from a small open-weight language model instead of a large commercial API. The authors fine-tuned Phi-3 Mini and Phi-3 Medium on operative reports paired with the billing claims actually submitted for payment, and compared them against the base model, a retrieval-augmented version, and GPT-4o. Their central claim is that both fine-tuned models match or beat GPT-4o on exact ICD-10, CPT, and modifier code generation, with the fine-tuned Phi-3 Medium performing best while fabricating only about 1% of ICD-10 codes and 0.6% of CPT codes. The significance is practical: if true, a health system could train and run a competitive coder entirely on local, protected infrastructure with four 24GB GPUs, avoiding both the cost and the privacy exposure of sending patient notes to a hosted model.

What carries the argument

The load-bearing mechanism is QLoRA (quantized low-rank adaptation), a parameter-efficient fine-tuning method that trains small adapter matrices on a 4-bit quantized base model, applied to the Phi-3 family of small open-weight language models. The authors fine-tuned Phi-3 Mini and Phi-3 Medium on paired operative reports and billing claims, using the model's unused special tokens to segment the prompt into system, user, and assistant components so the model learns the claim format. At inference they used greedy decoding with a repetition penalty and a 512-token limit, and they merged the trained adapters back into the models. This mechanism carries the argument because it is what converts a general chat model into a structured billing-claim generator while keeping training feasible on four 24GB GPUs.

What would settle it

Re-run the same test set with GPT-4o using the exact structured prompt, special-token segmentation, greedy decoding, and output-format instruction used for the fine-tuned Phi-3 models, and compare exact-match recall and precision on ICD-10, CPT, and modifier codes. If GPT-4o then matches or exceeds the fine-tuned Phi-3 Medium, the paper's central comparison would not survive.

Watch

Extended reading notes

Core claim

The paper claims that a small open-weight language model, fine-tuned with parameter-efficient supervised learning on institution-specific operative reports and their associated billing claims, can generate ICD-10, CPT, and modifier codes from an operative report as accurately as or more accurately than GPT-4o. In the authors' benchmark, the fine-tuned Phi-3 Medium performed best, with ICD-10 recall and precision of 72% and 72%, CPT recall and precision of 77% and 79%, and modifier recall and precision of 63% and 64%; the fine-tuned Phi-3 Mini also performed better than or as well as GPT-4o. The Medium model fabricated only 1% of ICD-10 codes and 0.6% of CPT codes, which the authors contrast with roughly half of base-model ICD-10 codes being invalid. The authors' conclusion is that task-specific fine-tuning on local data, with simple open-source tooling and four 24GB GPUs, can close the gap with much larger general-purpose models on a real clinical documentation task.

Load-bearing premise

The load-bearing premise is that the GPT-4o baseline was given a fair and comparably optimized prompt and decoding setup; the paper reports the Phi-3 prompts in full but does not specify GPT-4o's prompt, model version, temperature, or generation settings.

Editorial extensions

If this is right

  • A health system with four 24GB GPUs can build a private, on-site billing-code generator without sending protected health information to an external API.
  • Fine-tuned small models reduce code fabrication to roughly 1%, which is what makes a coder-assist tool practical; the base and RAG configurations fabricated codes far more often.
  • Fine-tuning on institutional data outperforms retrieval-augmented prompting for exact code-set generation, suggesting that for highly structured outputs in-domain weights matter more than in-context examples.
  • The paper's own conclusion bounds the claim: these models are not coder replacements and did not show reduced staffing or faster claim submission, but they could serve as a lower-cost alternative to commercial computer-assisted coding software.
  • Because both fine-tuned models matched or exceeded GPT-4o, model scale is not the decisive factor for this structured coding task; task-specific adaptation is.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the result generalizes, the same local fine-tuning recipe could be applied to other structured clinical-extraction tasks, such as pulling diagnoses from pathology or radiology reports, where privacy and exact vocabulary matter.
  • The reported comparison would be strengthened by a head-to-head rerun in which GPT-4o receives the exact same structured prompt and decoding settings as the Phi-3 models; the paper does not disclose GPT-4o's prompt, so part of the measured gap could be prompt-format rather than model capability.
  • Adding the History and Physical text, which the paper identifies as the source coders use for diagnosis codes, would likely improve ICD-10 recall and precision beyond the reported levels.
  • Because the evaluation counts only exact code-set matches as correct, the reported recall and precision may understate the clinical usefulness of outputs that are semantically appropriate but differ from the submitted bill; a family-level or coder-reviewed metric would test that.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper evaluates four locally deployed Phi-3-based configurations (base Mini, Mini with RAG, fine-tuned Mini, fine-tuned Medium) against GPT-4o on the task of generating ICD-10-CM, CPT, and modifier codes from operative reports. The authors fine-tune Phi-3 Mini and Phi-3 Medium with QLoRA on institutional surgical billing data, run inference with greedy decoding and repetition penalty, and measure exact-code recall/precision, fabrication rates, and output-format fidelity. The headline claim is that the fine-tuned models perform as well as or better than GPT-4o while requiring only four 24GB consumer GPUs for training. The study uses a large real-world dataset (192,585 encounters), a held-out split in principle, bootstrap confidence intervals, and validity checks against reference code lists. The abstract reports that the fine-tuned Phi-3 Medium achieves ICD-10 recall/precision of 72%/72%, CPT recall/precision of 77%/79%, and modifier recall/precision of 63%/64%.

Significance. If the comparative results are sound, this is a practically valuable demonstration: small open models fine-tuned on institution-specific data can approach or match a much larger commercial model on a real clinical documentation task while keeping data on-premises and using modest hardware. The strengths of the paper are its scale of real patient data, the explicit local-development and privacy-preserving setup, the use of bootstrap confidence intervals, and the fabrication-rate validity checks. The main weakness is that the GPT-4o baseline is not specified, which makes the central comparative claim unauditable as reported. There is also an ambiguity about which split (validation vs. test) underlies the main results, and the abstract's ICD-10 numbers do not match Table 2. These issues are fixable but must be resolved before the conclusions can be accepted.

major comments (3)
  1. [Methods, 'Language Model Configurations' and 'Inference'] The GPT-4o baseline is not specified. The Methods state only that 'GPT-4o was used to establish a comparative benchmark representing large state of the art (SOTA) models,' with no prompt template, system message, output-format instructions, model version or date, temperature, sampling strategy, repetition penalty, or maximum token count, while the Phi-3 configurations are described in full. Because the headline conclusion is explicitly comparative ('Both fine-tuned models performed better or as well as GPT-4o'), the comparison is not auditable without this information. Please provide the exact GPT-4o prompt (ideally the same system prompt and output-format instructions used for the other configurations), the specific model version, all decoding settings, and any post-processing applied to its output, or explicitly weaken the comparative claim to what can be supported without that information.
  2. [Evaluation, 'Data Extraction and Processing' and Figure 3] It is unclear whether the main results in Table 2 and Figure 3 are computed on the held-out test set or on the validation set. The paper says the data were split 60/20/20 into training, validation, and test, and the inference section says the evaluation test set was used, but the bootstrap paragraph in Evaluation says 'The validation set is sampled with replacement (N = 39052)' for the confidence intervals shown in Figure 3. If the metrics are computed on the validation set, the held-out claim is compromised; if they are computed on the test set, the bootstrap text is incorrect. Additionally, N = 39,052 does not match 20% of 192,585 (which is 38,517). Please clarify which split produced the numbers in Table 2 and Figure 3 and correct the sample size.
  3. [Abstract and Table 2] The abstract reports ICD-10 recall and precision for the fine-tuned Phi-3 Medium as 72% and 72%, but Table 2 lists ICD-10 recall as 0.65 and precision as 0.72 for the same configuration. The CPT and modifier numbers in the abstract match Table 2, suggesting a typo, but this is a headline result and must be corrected or reconciled.
minor comments (7)
  1. [Results, 'Code Validity Analysis'] The phrase 'provided in Table X' should be replaced with the actual table number (Table 2).
  2. [Evaluation, equations] The precision and recall formulas contain unreadable special characters and should be typeset using standard mathematical notation.
  3. [Evaluation, F1 equation] The F1 formula is garbled; please replace it with a properly typeset expression and state explicitly that F1 is computed from the mean precision and mean recall rather than averaged per-case F1.
  4. [Background] The sentence 'augment and existing LLM' should read 'augment an existing LLM.'
  5. [Throughout] The model name is inconsistently capitalized as 'PHI-3' and 'Phi-3'; please use 'Phi-3' consistently.
  6. [Results, 'Data Extraction and Processing'] The size of the test set is not reported; please report the number of encounters in the training, validation, and test splits.
  7. [Evaluation, 'Format Consistency Analysis'] The 'ROUGE L Sum' metric is not standard in the cited reference; please define it or cite a source that defines it.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the evaluation is a supervised held-out benchmark; the only self-citation is non-load-bearing, and the undisclosed GPT-4o baseline is an auditability concern, not a circularity.

full rationale

The paper's central claim is that fine-tuned Phi-3 models match or beat GPT-4o on surgical billing-code generation. The supporting evidence is a supervised benchmark: models are trained on a 60% training split and evaluated on a 20% held-out test split, with recall and precision computed by comparing generated codes against the billing claims in that held-out split. No parameter is fitted to the test outcomes, and the reported metrics are empirical quantities computed from model outputs and ground-truth codes, not consequences of the training labels by construction. The RAG configuration retrieves similar examples from the training set and appends them to the prompt as in-context context; this is standard inference-time augmentation and does not incorporate the test claim's own answer into the input. The only self-citation (reference 39, Zaribafzadeh et al.) supports the general statement that the authors' prior institutional AI work serves as a model for integration under realistic constraints; it is not used to establish the accuracy numbers, to define the evaluation metric, or to rule out alternative designs, so it is not load-bearing. The undisclosed GPT-4o prompt and decoding settings are a legitimate experimental auditability concern because the headline comparison cannot be fully reproduced, but this is a missing control or fairness issue, not a circular reduction: the result is an empirical measurement that could in principle have gone either way and does not reduce to an identity or to a fitted parameter. Therefore I find no circular step and assign score 0.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The central claim is empirical and rests on a handful of modeling choices (LoRA hyperparameters, decoding settings, retrieval count, embedding model) and on four domain assumptions about ground-truth quality, information sufficiency, split independence, and GPT-4o comparability. No new entities are postulated.

free parameters (7)
  • LoRA rank = 64
    LoRA rank chosen following the QLoRA reference; controls the number of trainable parameters (about 5% of Phi-3 Mini, 2.7% of Medium).
  • LoRA alpha = 16
    LoRA scaling factor; set according to QLoRA defaults.
  • LoRA dropout = 0.1
    Dropout applied to LoRA adapters; set following QLoRA defaults.
  • Max new tokens = 512
    Generation length cap for inference; chosen to accommodate typical billing claim outputs.
  • Repetition penalty = 1.1
    Decoding hyperparameter intended to reduce repetitive output.
  • RAG retrieval count (top-k) = 2
    Number of similar operative notes retrieved as in-context examples in the RAG configuration; chosen by the authors.
  • Embedding model = all-MiniLM-L6-v2
    Sentence transformer model chosen for the RAG vector index; a design choice that affects retrieval quality.
assumptions (4)
  • domain assumption The institutional billing codes used as ground truth are accurate and complete.
    The evaluation compares model outputs to coder-assigned codes; any errors or inconsistencies in the ground-truth labels would bias the reported precision and recall.
  • domain assumption Operative reports alone contain sufficient information to predict the assigned billing codes.
    The authors note that coders normally use the full patient record (e.g., H&P), so ICD-10 codes may be partially underdetermined by the operative report alone; this is acknowledged in the Discussion.
  • domain assumption The 60/20/20 split creates independent training and test sets.
    The split is balanced by surgical encounter date but not by patient or procedure text, so the same patient or nearly identical operative notes could appear in both training and test sets.
  • ad hoc to paper The GPT-4o comparison used a prompt of comparable quality to the Phi-3 prompts.
    The GPT-4o prompt, API version, temperature, and decoding settings are not disclosed, so the central 'better or as well as GPT-4o' claim rests on an unverified comparability assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Practical Design and Benchmarking of Generative AI Applications for Surgical Billing and Coding." pith.science (2026). https://pith.science/paper/Y2MXRA7R

@misc{pith2026250105479,
  author       = {Pith},
  title        = {Pith review of: Practical Design and Benchmarking of Generative AI Applications for Surgical Billing and Coding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y2MXRA7R}},
  note         = {Machine review of arXiv:2501.05479}
}
read the original abstract

Background: Healthcare has many manual processes that can benefit from automation and augmentation with Generative Artificial Intelligence (AI), the medical billing and coding process. However, current foundational Large Language Models (LLMs) perform poorly when tasked with generating accurate International Classification of Diseases, 10th edition, Clinical Modification (ICD-10-CM) and Current Procedural Terminology (CPT) codes. Additionally, there are many security and financial challenges in the application of generative AI to healthcare. We present a strategy for developing generative AI tools in healthcare, specifically for medical billing and coding, that balances accuracy, accessibility, and patient privacy. Methods: We fine tune the PHI-3 Mini and PHI-3 Medium LLMs using institutional data and compare the results against the PHI-3 base model, a PHI-3 RAG application, and GPT-4o. We use the post operative surgical report as input and the patients billing claim the associated ICD-10, CPT, and Modifier codes as the target result. Performance is measured by accuracy of code generation, proportion of invalid codes, and the fidelity of the billing claim format. Results: Both fine-tuned models performed better or as well as GPT-4o. The Phi-3 Medium fine-tuned model showed the best performance (ICD-10 Recall and Precision: 72%, 72%; CPT Recall and Precision: 77%, 79%; Modifier Recall and Precision: 63%, 64%). The Phi-3 Medium fine-tuned model only fabricated 1% of ICD-10 codes and 0.6% of CPT codes generated. Conclusions: Our study shows that a small model that is fine-tuned on domain-specific data for specific tasks using a simple set of open-source tools and minimal technological and monetary requirements performs as well as the larger contemporary consumer models.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 30 canonical work pages

  1. [1]

    A study of generative large language model for medical research and healthcare

    Peng C, Yang X, Chen A, et al. A study of generative large language model for medical research and healthcare. npj Digit Med 2023;6(1):1–10

    work page 2023

  2. [2]

    Large language models in medicine

    Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, Ting DSW. Large language models in medicine. Nat Med 2023;29(8):1930–40

    work page 2023

  3. [3]

    Using ChatGPT to write patient clinic letters

    Ali SR, Dobbs TD, Hutchings HA, Whitaker IS. Using ChatGPT to write patient clinic letters. The Lancet Digital Health 2023;5(4):e179–81

    work page 2023

  4. [4]

    ChatGPT: the future of discharge summaries? The Lancet Digital Health 2023;5(3):e107–8

    Patel SB, Lam K. ChatGPT: the future of discharge summaries? The Lancet Digital Health 2023;5(3):e107–8

    2023

  5. [5]

    Ethics of large language models in medicine and medical research

    Li H, Moon JT, Purkayastha S, Celi LA, Trivedi H, Gichoya JW. Ethics of large language models in medicine and medical research. The Lancet Digital Health 2023;5(6):e333–5

    work page 2023

  6. [6]

    Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine

    Lee P, Bubeck S, Petro J. Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine. New England Journal of Medicine 2023;388(13):1233–9

    work page 2023

  7. [7]

    Evaluating the Feasibility of ChatGPT in Healthcare: An Analysis of Multiple Clinical and Research Scenarios

    Cascella M, Montomoli J, Bellini V, Bignami E. Evaluating the Feasibility of ChatGPT in Healthcare: An Analysis of Multiple Clinical and Research Scenarios. J Med Syst 2023;47(1):33

    work page 2023

  8. [8]

    Key challenges for delivering clinical impact with artificial intelligence

    Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Med 2019;17(1):195

    work page 2019

Show all 47 references
  1. [9]

    GPT versus Resident Physicians — A Benchmark Based on Official Board Scores

    Katz U, Cohen E, Shachar E, et al. GPT versus Resident Physicians — A Benchmark Based on Official Board Scores. NEJM AI 2024;1(5):AIdbp2300192

    2024

  2. [10]

    Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large Language Models [Internet]

    Kung TH, Cheatham M, ChatGPT, et al. Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large Language Models [Internet]. 2022 [cited 2024 Aug 12];2022.12.19.22283643. Available from: https://www.medrxiv.org/content/10.1101/2022.12.19.22283643v2

    2022 doi

  3. [11]

    Large Language Models Are Poor Medical Coders — Benchmarking of Medical Code Querying

    Soroush A, Glicksberg BS, Zimlichman E, et al. Large Language Models Are Poor Medical Coders — Benchmarking of Medical Code Querying. NEJM AI 2024;1(5):AIdbp2300040

    2024

  4. [12]

    The shaky foundations of large language models and foundation models for electronic health records

    Wornow M, Xu Y, Thapa R, et al. The shaky foundations of large language models and foundation models for electronic health records. npj Digit Med 2023;6(1):1–10

    2023

  5. [13]

    A large language model for electronic health records

    Yang X, Chen A, PourNejatian N, et al. A large language model for electronic health records. npj Digit Med 2022;5(1):1–9. 19

    2022

  6. [14]

    Health system-scale language models are all-purpose prediction engines

    Jiang LY, Liu XC, Nejatian NP, et al. Health system-scale language models are all-purpose prediction engines. Nature 2023;619(7969):357–62

    2023

  7. [15]

    Scaling Down to Scale Up: A Guide to Parameter- Efficient Fine-Tuning [Internet]

    Lialin V, Deshpande V, Rumshisky A. Scaling Down to Scale Up: A Guide to Parameter- Efficient Fine-Tuning [Internet]. arXiv.org. 2023 [cited 2024 Aug 12];Available from: https://arxiv.org/abs/2303.15647v1

    2023 arXiv

  8. [16]

    Language Models are Few-Shot Learners [Internet]

    Brown T, Mann B, Ryder N, et al. Language Models are Few-Shot Learners [Internet]. In: Advances in Neural Information Processing Systems. Curran Associates, Inc.; 2020 [cited 2024 Aug 12]. p. 1877–901.Available from: https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4...

    2020

  9. [17]

    Potential for GPT Technology to Optimize Future Clinical Decision-Making Using Retrieval-Augmented Generation

    C W, J O, C W, H O, R C, D O. Potential for GPT Technology to Optimize Future Clinical Decision-Making Using Retrieval-Augmented Generation. Annals of biomedical engineering [Internet] 2024 [cited 2024 Aug 12];52(5). Available from: https://pubmed.ncbi.nlm.nih.gov/37530906/

    2024

  10. [18]

    BioinspiredLLM: Conversational Large Language Model for the Mechanics of Biological and Bio-Inspired Materials

    Luu RK, Buehler MJ. BioinspiredLLM: Conversational Large Language Model for the Mechanics of Biological and Bio-Inspired Materials. Adv Sci (Weinh) 2023;11(10):2306724

    2023

  11. [19]

    Retrieval augmentation of large language models for lay language generation

    Guo Y, Qiu W, Leroy G, Wang S, Cohen T. Retrieval augmentation of large language models for lay language generation. Journal of Biomedical Informatics 2024;149:104580

    2024

  12. [20]

    Empirical Analysis of the Strengths and Weaknesses of PEFT Techniques for LLMs [Internet]

    Pu G, Jain A, Yin J, Kaplan R. Empirical Analysis of the Strengths and Weaknesses of PEFT Techniques for LLMs [Internet]. 2023 [cited 2024 Aug 12];Available from: http://arxiv.org/abs/2304.14999

    2023 arXiv

  13. [21]

    Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning [Internet]

    Liu H, Tam D, Muqeeth M, et al. Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning [Internet]. 2022 [cited 2024 Aug 12];Available from: http://arxiv.org/abs/2205.05638

    2022 arXiv

  14. [22]

    Lost in the Middle: How Language Models Use Long Contexts [Internet]

    Liu NF, Lin K, Hewitt J, et al. Lost in the Middle: How Language Models Use Long Contexts [Internet]. 2023 [cited 2024 Aug 12];Available from: http://arxiv.org/abs/2307.03172

    2023 arXiv

  15. [23]

    Towards Medical Billing Automation: NLP for Outpatient Clinician Note Classification [Internet]

    Crowson MG, Alsentzer E, Fiskio J, Bates DW. Towards Medical Billing Automation: NLP for Outpatient Clinician Note Classification [Internet]. 2023 [cited 2024 Aug 12];2023.07.07.23292367. Available from: https://www.medrxiv.org/content/10.1101/2023.07.07.23292367v1

    2023 doi

  16. [24]

    Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone [Internet]

    Abdin M, Jacobs SA, Awan AA, et al. Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone [Internet]. 2024 [cited 2024 Aug 12];Available from: http://arxiv.org/abs/2404.14219 20

    2024 arXiv

  17. [25]

    Introducing Phi-3: Redefining what’s possible with SLMs [Internet]

    Bilenko M. Introducing Phi-3: Redefining what’s possible with SLMs [Internet]. Microsoft Azure Blog. 2024 [cited 2024 Aug 12];Available from: https://azure.microsoft.com/en- us/blog/introducing-phi-3-redefining-whats-possible-with-slms/

    2024

  18. [26]

    Link and code: Fast indexing with graphs and compact regression codes [Internet]

    Douze M, Sablayrolles A, Jégou H. Link and code: Fast indexing with graphs and compact regression codes [Internet]. arXiv.org. 2018 [cited 2024 Sep 5];Available from: https://arxiv.org/abs/1804.09996v2

    2018 arXiv

  19. [27]

    Billion-scale similarity search with GPUs [Internet]

    Johnson J, Douze M, Jégou H. Billion-scale similarity search with GPUs [Internet]. arXiv.org. 2017 [cited 2024 Sep 5];Available from: https://arxiv.org/abs/1702.08734v1

    2017 arXiv

  20. [28]

    Sentence-BERT: Sentence Embeddings using Siamese BERT- Networks [Internet]

    Reimers N, Gurevych I. Sentence-BERT: Sentence Embeddings using Siamese BERT- Networks [Internet]. arXiv.org. 2019 [cited 2024 Sep 5];Available from: https://arxiv.org/abs/1908.10084v1

    2019 arXiv

  21. [29]

    QLoRA: Efficient Finetuning of Quantized LLMs [Internet]

    Dettmers T, Pagnoni A, Holtzman A, Zettlemoyer L. QLoRA: Efficient Finetuning of Quantized LLMs [Internet]. 2023 [cited 2024 Aug 13];Available from: http://arxiv.org/abs/2305.14314

    2023 arXiv

  22. [30]

    FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning [Internet]

    Dao T. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning [Internet]. 2023 [cited 2024 Aug 13];Available from: http://arxiv.org/abs/2307.08691

    2023 arXiv

  23. [31]

    ZeRO: Memory Optimizations Toward Training Trillion Parameter Models [Internet]

    Rajbhandari S, Rasley J, Ruwase O, He Y. ZeRO: Memory Optimizations Toward Training Trillion Parameter Models [Internet]. 2020 [cited 2024 Sep 6];Available from: http://arxiv.org/abs/1910.02054

    2020 arXiv

  24. [32]

    ZeRO-Offload: Democratizing Billion-Scale Model Training [Internet]

    Ren J, Rajbhandari S, Aminabadi RY, et al. ZeRO-Offload: Democratizing Billion-Scale Model Training [Internet]. 2021 [cited 2024 Sep 6];Available from: http://arxiv.org/abs/2101.06840

    2021 arXiv

  25. [33]

    ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning [Internet]

    Rajbhandari S, Ruwase O, Rasley J, Smith S, He Y. ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning [Internet]. 2021 [cited 2024 Sep 6];Available from: http://arxiv.org/abs/2104.07857

    2021 arXiv

  26. [34]

    A Thorough Examination of Decoding Methods in the Era of LLMs [Internet]

    Shi C, Yang H, Cai D, et al. A Thorough Examination of Decoding Methods in the Era of LLMs [Internet]. 2024 [cited 2024 Aug 13];Available from: http://arxiv.org/abs/2402.06925

    2024 arXiv

  27. [35]

    Diagnosis code assignment: models and evaluation metrics

    Perotte A, Pivovarov R, Natarajan K, Weiskopf N, Wood F, Elhadad N. Diagnosis code assignment: models and evaluation metrics. J Am Med Inform Assoc 2014;21(2):231–7

    2014

  28. [36]

    3M Inside Angle

    About accuracy...part 2 [Internet]. 3M Inside Angle. [cited 2024 Sep 6];Available from: https://insideangle.3m.com/his/blog-post/about-accuracy-part-2/

    2024

  29. [37]

    ROUGE: A Package for Automatic Evaluation of Summaries [Internet]

    Lin C-Y. ROUGE: A Package for Automatic Evaluation of Summaries [Internet]. In: Text Summarization Branches Out. Barcelona, Spain: Association for Computational Linguistics; 2004 [cited 2024 Aug 13]. p. 74–81.Available from: https://aclanthology.org/W04-1013 21

    2004

  30. [38]

    METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments [Internet]

    Banerjee S, Lavie A. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments [Internet]. In: Goldstein J, Lavie A, Lin C-Y, Voss C, editors. Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translat...

    2005

  31. [39]

    Development, Deployment, and Implementation of a Machine Learning Surgical Case Length Prediction Model and Prospective Evaluation

    Zaribafzadeh H, Webster WL, Vail CJ, et al. Development, Deployment, and Implementation of a Machine Learning Surgical Case Length Prediction Model and Prospective Evaluation. Annals of Surgery 2023;278(6):890

    2023

  32. [40]

    Automated clinical coding: what, why, and where we are? NPJ Digit Med 2022;5(1):159

    Dong H, Falis M, Whiteley W, et al. Automated clinical coding: what, why, and where we are? NPJ Digit Med 2022;5(1):159

    2022

  33. [41]

    A systematic literature review of automated clinical coding and classification systems

    Stanfill MH, Williams M, Fenton SH, Jenders RA, Hersh WR. A systematic literature review of automated clinical coding and classification systems. J Am Med Inform Assoc 2010;17(6):646–51

    2010

  34. [42]

    3M Inside Angle

    CAC Exact: A new CAC system performance metric [Internet]. 3M Inside Angle. [cited 2024 Sep 6];Available from: https://insideangle.3m.com/his/blog-post/cac-exact-a-new-cac- system-performance-metric/

    2024

  35. [43]

    Read, Attend, and Code: Pushing the Limits of Medical Codes Prediction from Clinical Notes by Machines [Internet]

    Kim B-H, Ganapathi V. Read, Attend, and Code: Pushing the Limits of Medical Codes Prediction from Clinical Notes by Machines [Internet]. arXiv.org. 2021 [cited 2024 Sep 6];Available from: https://arxiv.org/abs/2107.10650v1

    2021 arXiv

  36. [44]

    Injecting New Knowledge into Large Language Models via Supervised Fine-Tuning [Internet]

    Mecklenburg N, Lin Y, Li X, et al. Injecting New Knowledge into Large Language Models via Supervised Fine-Tuning [Internet]. 2024 [cited 2024 Aug 13];Available from: http://arxiv.org/abs/2404.00213

    2024 arXiv

  37. [45]

    Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment [Internet]

    Xu L, Xie H, Qin S-ZJ, Tao X, Wang FL. Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment [Internet]. 2023 [cited 2024 Aug 12];Available from: http://arxiv.org/abs/2312.12148

    2023 arXiv

  38. [46]

    Efficient Memory Management for Large Language Model Serving with PagedAttention [Internet]

    Kwon W, Li Z, Zhuang S, et al. Efficient Memory Management for Large Language Model Serving with PagedAttention [Internet]. 2023 [cited 2024 Sep 6];Available from: http://arxiv.org/abs/2309.06180

    2023 arXiv

  39. [47]

    Journal of AHIMA

    Are LLMs Coming for Coding? Yes, and Medical Coders Should Prepare [Internet]. Journal of AHIMA. [cited 2024 Sep 6];Available from: https://journal.ahima.org/page/are-llms- coming-for-coding-yes-and-medical-coders-should-prepare

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.