Pith. sign in

REVIEW 5 major objections 6 minor 39 references

Towards Scalable SOAP Note Generation: A Weakly Supervised Multimodal Framework

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A weakly supervised multimodal framework generates clinically structured SOAP notes from skin-lesion images and sparse text, matching the performance of GPT-4o, Claude, and DeepSeek Janus Pro on clinical relevance metrics.

desk verdict A clearly described weak-supervision pipeline for dermatology SOAP notes, but the headline evidence rests on three expert cases and unvalidated synthetic labels. read the letter →

arxiv 2506.10328 v1 pith:IM6TIT4W submitted 2025-06-12 cs.CV cs.AIcs.LG

Sadia Kamal , Tim Oates , Joy Wan This is my paper
classification cs.CVcs.AIcs.LG
keywords SOAPnotegenerationweaklysupervisedlearningmultimodalvision-languagemodelretrieval-augmenteddermatologyclinicaldocumentationmedicalimagecaptioningNLP
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SOAP notes are the standard structure for clinical documentation, but writing them by hand is time-consuming and a known driver of physician burnout. This paper tries to establish that a small vision-language model, fine-tuned on synthetic labels, can generate clinically structured SOAP notes from a skin-lesion image plus a few structured text fields (size, location, symptoms) as well as large proprietary models such as GPT-4o, Claude, and DeepSeek Janus Pro. The training labels are produced without expert annotation: GPT-3.5 turns the structured fields into a caption, a retrieval-augmented step pulls context from curated medical sources, and Vision-LLaMA 3.2 writes the weakly supervised note. The authors also introduce two new metrics, MedConceptEval and Clinical Coherence Score, meant to capture clinical semantic alignment rather than surface overlap. If the claims hold, automated documentation could scale to settings where expert-annotated data is scarce, reducing the documentation burden on clinicians.

What carries the argument

The load-bearing mechanism is the weak-supervision pre-training pipeline: a structured-to-caption transformation by GPT-3.5, a retrieval-augmented generation step that grounds the caption in passages from curated medical sources, and a Vision-LLaMA 3.2 model that synthesizes the SOAP note from the image, caption, and retrieved context. Fine-tuning uses QLoRA, a parameter-efficient low-rank adaptation, combined with supervised fine-tuning on the synthetic input–output pairs. The evaluation machinery consists of two new metrics—MedConceptEval, which measures cosine similarity between generated sections and curated clinical concept sets using ClinicalBERT embeddings, and the Clinical Coherence Score, which measures alignment between the input caption and each SOAP section.

What would settle it

Randomly select fifty generated notes, have two board-certified dermatologists independently flag any clinical statement (diagnosis, treatment, or symptom) that is not supported by the input image and structured fields, and measure inter-rater agreement and the rate of unsupported statements; a high rate of unsupported content would refute the claim that the notes are clinically grounded.

Watch

Extended reading notes

Core claim

The central claim is that a weakly supervised multimodal framework can generate clinically grounded SOAP notes from limited dermatologic inputs—a lesion image and sparse clinical text—without large-scale expert annotations. The authors build a synthetic training set by converting structured metadata into a clinical caption with GPT-3.5, retrieving relevant passages from a curated vector database, and asking a pre-trained Vision-LLaMA 3.2 model to write the note in SOAP format. The model is then fine-tuned with QLoRA and supervised fine-tuning. On three expert-annotated test cases, the resulting model scores comparably to GPT-4o, Claude, and DeepSeek Janus Pro on ROUGE, METEOR, CHRF++, and ClinicalBERT F1, and an LLM-as-a-Judge evaluation rates it 20/20 against 18–19/20 for the baselines. The paper also reports statistically significant section-level effects in its MedConceptEval metric and introduces the Clinical Coherence Score to quantify alignment between the input caption and generated sections.

Load-bearing premise

The whole framework inherits the factual quality of the synthetic training notes: if the captions or retrieved passages contain errors, the fine-tuned model will reproduce and amplify them, since there is no expert check on the training labels.

Editorial extensions

If this is right

  • Dermatology documentation could move from manual note-writing to automated draft generation from a photo and a few structured fields, reducing the documentation burden on clinicians.
  • The approach demonstrates a path to clinical note generation without large expert-annotated datasets, using retrieval-grounded synthetic labels as a substitute.
  • The new MedConceptEval and Clinical Coherence Score metrics provide evaluation tools that go beyond token overlap, capturing semantic alignment with clinical concepts and input consistency.
  • The fine-tuning procedure (QLoRA plus SFT on a single 80 GB GPU) makes the approach feasible for settings with limited compute.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable implication the authors do not state: if the synthetic labels contain systematic bias—for example, over-representation of certain diagnoses in the retrieval corpus—the fine-tuned model will inherit that bias, so the framework's clinical reliability is bounded by the retrieval sources and caption generator, not by the fine-tuning step.
  • The 20/20 LLM-as-a-Judge score may partly reflect the judge's preference for templated SOAP structure over the free-form outputs of general-purpose models; an expert human evaluation comparing factual accuracy, rather than structure, would clarify whether the parity claim extends to clinical decision-making.
  • The framework could be extended to other image-plus-metadata clinical tasks, such as radiology or ophthalmology documentation, by swapping the retrieval corpus and keeping the same three-phase pipeline.
  • A direct extension would be auditing the Plan section for safety—checking whether every treatment recommendation is grounded in the retrieved guidelines—by tracing the RAG passages used for each generated note.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes a weakly supervised multimodal framework for generating dermatology SOAP notes from lesion images and sparse clinical text. In a pre-training phase, GPT-3.5 converts structured metadata into a caption, retrieval-augmented generation retrieves relevant passages from curated medical websites, and Vision-LLaMA 3.2 produces synthetic SOAP notes that serve as pseudo-labels. A fine-tuning phase then adapts Vision-LLaMA 3.2 with QLoRA and supervised fine-tuning using these pseudo-labels. The authors introduce two evaluation metrics, MedConceptEval and Clinical Coherence Score (CCS), and report quantitative comparisons against GPT-4o, Claude, and DeepSeek Janus Pro on three expert-annotated cases, together with a qualitative LLM-as-a-Judge evaluation. The central claim is that the framework achieves performance comparable to these large proprietary models while requiring far less expert annotation, and that the resulting notes are clinically grounded.

Significance. If the central claim were fully supported, the paper would make a useful contribution: it would demonstrate that a relatively small vision-language model fine-tuned on LLM-generated, retrieval-augmented pseudo-labels can approach the quality of much larger proprietary models for structured clinical documentation, potentially reducing annotation cost in a domain where expert labels are scarce. The authors also deserve credit for attempting a direct comparison with strong baselines on clinical-text metrics, for using a public dataset (PAD-UFES-20), and for explicitly acknowledging several limitations in Appendix 10. However, the current evidence is not sufficient to establish the claimed parity or the clinical grounding of the outputs. The comparison rests on three expert-annotated cases with no confidence intervals or significance tests, the new CCS metric is partly circular, the pseudo-label training targets are unvalidated, and no train/test split is reported. These issues are load-bearing for the manuscript's main claims, so the result is presently more a proof-of-concept than a validated framework.

major comments (5)
  1. [Section 3.2, Table 2] The central quantitative comparison is based on exactly three expert-annotated cases. With n=3, differences of 0.01-0.03 in ROUGE, METEOR, or ClinicalBERT F1 cannot support the claim that the proposed method is 'comparable to' or 'outperforms' GPT-4o, Claude, or Janus Pro. No confidence intervals, bootstrap estimates, or significance tests are reported for these lexical and semantic metrics. The manuscript should either substantially expand the expert-annotated evaluation set or present statistical evidence that the observed differences are not noise.
  2. [Section 3.1.2, Table 3] The Clinical Coherence Score is partly circular as a measure of clinical quality. Because the model is fine-tuned to map the input caption to the SOAP note, cosine similarity between that same caption and the generated note sections will be high by construction, as Table 3 confirms. This does not validate that the note is clinically grounded or that it contains correct medical content. The score also is not reported for the baseline models, so it cannot be used to support the comparison with GPT-4o, Claude, and Janus Pro. The metric would be informative only if it were shown to correlate with expert judgments or if it were applied to all models as a consistency check rather than a quality measure.
  3. [Section 2.1, Eq. (1), Appendix 10] The pseudo-label training targets created in Section 2.1 are never validated for factual correctness. Equation (1) trains the model token-by-token against these synthetic SOAP notes, so any hallucinated diagnosis, omitted symptom, or unsupported treatment is learned as ground truth. The manuscript's own limitation statement (Appendix 10) concedes that output quality 'depends on the accuracy of the retrieved domain-specific knowledge' and that the model 'may hallucinate,' yet no clinician review, error-rate audit, or comparison against a verified gold standard for the synthesized dataset is reported. Without such an audit, both the 'clinically grounded' claim and the performance comparison rest on an unverified assumption about pseudo-label reliability.
  4. [Section 7.4] The paper does not report a train/test split. Fine-tuning is described as running for 500 epochs on the full dataset, and the evaluation uses three held-out expert-annotated cases, but it is unclear whether those cases overlap with the training set or whether the final model was selected on a validation set. With 500 epochs of fine-tuning, overfitting is a serious risk, and the reported metrics may not reflect generalization to new patients. The authors should specify the exact split, the validation procedure, and how the three test cases were held out.
  5. [Section 3.1.1] MedConceptEval is presented as a clinically validated evaluation framework, but the descriptor banks are constructed by an LLM and are not verified against a human expert gold standard. In addition, MedConceptEval scores are reported only for the proposed model, not for GPT-4o, Claude, or Janus Pro, so the metric cannot support the comparative claim. To be convincing, the authors should validate the descriptor banks, report inter-annotator agreement or expert review of the concept sets, and score all baselines under the same protocol.
minor comments (6)
  1. [Section 3.2.1 vs. Section 9] The paper reports a perfect 20/20 Flow-Judge score for the proposed approach, but Section 9 shows a generated note in which the diagnosis of BCC is prematurely placed in the Chief Complaint section and describes this as a structural inaccuracy. These statements are in tension and should be reconciled or the qualitative claim should be qualified.
  2. [Appendix 8.1.2] METEOR is cited as reference [12], but reference [12] is Lewis et al. on retrieval-augmented generation; the correct METEOR citation is missing and appears to have been displaced.
  3. [Section 7.1] The PAD-UFES-20 dataset is described as containing 'dermoscopic images,' but the dataset's own documentation says the images are clinical photographs collected from smartphones. This distinction matters for the dermatological scope of the claim and should be corrected.
  4. [Section 3.1.1] There is a typographical error: 'one-way ANOV A' should read 'one-way ANOVA.'
  5. [Eq. (1)] The multimodal input x is defined in the text but does not appear in the displayed loss; the equation should either include the conditioning on x or the notation should be adjusted.
  6. [Section 2.1] The reference list numbering appears off for several citations (for example, METEOR and RAG share number [12]); please renumber the bibliography and check all in-text citations after the fix.

Circularity Check

1 steps flagged · score 6.0 of 10

The Clinical Coherence Score is a by-construction fit: the fine-tuned model is trained to map the input caption to a SOAP note, and CCS then measures cosine similarity between that same caption and the generated note.

  1. fitted input called prediction [Section 3.1.2 (Clinical Coherence Score), with Section 2.2 (Fine-Tuning) and Eq. (1) in Section 7.3]
    "We introduce the Clinical Coherence Score (CCS), a metric that evaluates the semantic alignment between the caption derived from input features and the structured SOAP note sections... the cosine similarity between these embeddings is calculated to assess how well the generated note aligns with the corresponding caption. ... the lesion image and generated caption are treated as multimodal inputs, and the weakly supervised SOAP note serves as the training target."

    The caption-to-note alignment that CCS measures is exactly the mapping optimized during fine-tuning: Section 2.2 makes the caption an input and the weakly supervised SOAP note the training target, and Eq. (1) minimizes token-level cross-entropy against that target. The synthetic target note was itself generated in Section 2.1 from the same caption plus retrieved context, so a high CCS (0.88-0.93 in Table 3) reports how faithfully the model reproduced its training signal, not an independent clinical-quality result. The metric is defined as the similarity the training loss enforces, so the reported strong alignment is forced by construction and cannot support the 'clinically grounded' claim.

full rationale

One clear by-construction step was found: CCS evaluates cosine similarity between the input caption and the generated SOAP note, while the model was fine-tuned precisely to map that caption to that note; the score is therefore a fit diagnostic, not an external validation. MedConceptEval was checked for source overlap: its descriptor banks cite Mayo Clinic [15], which is not among the RAG sources ([25], [16], [2], [17]), so it is not a direct source-match tautology, although it is LLM-mediated. Table 2's comparison against expert-written SOAP notes with standard lexical and semantic metrics is independent evidence for the comparability claim. However, the 'clinically grounded' claim also rests on the unverified quality of the synthetic pseudo-labels; Appendix 10 concedes output quality 'depends on the accuracy of the retrieved domain-specific knowledge' and that the model 'may hallucinate.' Since one of the two headline metrics reduces by construction and the clinical-grounding claim partly relies on it, the paper is partially circular rather than fully self-contained.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper is empirical. No closed-form derivation or physical constants are fitted; the trainable weights of Vision-LLaMA and the QLoRA adapters are learned from data, not introduced as scientific parameters. The evaluation rests on ad hoc constructs: the descriptor banks for MedConceptEval, the ClinicalBERT embedding space, and the Flow-Judge as a judge, none of which are validated against independent clinical benchmarks. The central claim is only as good as the synthetic labels, the metadata accuracy, and the validity of these evaluation tools.

assumptions (3)
  • domain assumption Retrieved passages from the cited medical websites (NCI, ACS, NHS, etc.) are accurate and clinically current.
    The RAG pipeline in Section 2.1 concatenates retrieved context with the generated caption; if sources are incomplete or misleading, the synthetic SOAP notes inherit those flaws.
  • domain assumption The PAD-UFES-20 structured metadata (diameter, biopsy status, symptoms) is correct and complete for each lesion.
    Captions in Section 2.1 are generated from this metadata, so errors in the metadata propagate to the training labels.
  • domain assumption Cosine similarity in ClinicalBERT embedding space reflects clinical relevance.
    Both MedConceptEval and CCS rely on this assumption; the paper provides no validation of it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Scalable SOAP Note Generation: A Weakly Supervised Multimodal Framework." pith.science (2026). https://pith.science/paper/IM6TIT4W

@misc{pith2026250610328,
  author       = {Pith},
  title        = {Pith review of: Towards Scalable SOAP Note Generation: A Weakly Supervised Multimodal Framework},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IM6TIT4W}},
  note         = {Machine review of arXiv:2506.10328}
}
abstract

Skin carcinoma is the most prevalent form of cancer globally, accounting for over $8 billion in annual healthcare expenditures. In clinical settings, physicians document patient visits using detailed SOAP (Subjective, Objective, Assessment, and Plan) notes. However, manually generating these notes is labor-intensive and contributes to clinician burnout. In this work, we propose a weakly supervised multimodal framework to generate clinically structured SOAP notes from limited inputs, including lesion images and sparse clinical text. Our approach reduces reliance on manual annotations, enabling scalable, clinically grounded documentation while alleviating clinician burden and reducing the need for large annotated data. Our method achieves performance comparable to GPT-4o, Claude, and DeepSeek Janus Pro across key clinical relevance metrics. To evaluate clinical quality, we introduce two novel metrics MedConceptEval and Clinical Coherence Score (CCS) which assess semantic alignment with expert medical concepts and input features, respectively.

Figures

Figures reproduced from arXiv: 2506.10328 by the authors.

Figure 1
Figure 1. Overview of the proposed framework, consisting of pre-training, fine-tuning, and inference phases. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Structured SOAP note components. 6. Related Works Artificial intelligence has been widely applied in the med￾ical domain for tasks such as medical image captioning [6], clinical text summarization [26], discharge note gen￾eration [10], and medical question answering [28]. Recent efforts have focused on generating clinical notes, especially SOAP notes, from Electronic Health Records (EHRs) and doctor-patient conversa… view at source ↗
Figure 3
Figure 3. Comparison of Flow-Judge Feedback Across Four Language Models [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Examples (a) and (b) show structured SOAP notes generated by our proposed multimodal framework using an input lesion image [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 26 canonical work pages

  1. [1]

    Assessment of ai-generated pe- diatric rehabilitation soap-note quality

    Solomon Amenyo, Maura R Grossman, Daniel G Brown, and Brendan Wylie-Toal. Assessment of ai-generated pe- diatric rehabilitation soap-note quality. arXiv preprint arXiv:2503.15526, 2025. 1

    arXiv 2025

  2. [2]

    Donate to support the fight against cancer

    American Cancer Society. Donate to support the fight against cancer. https://donate.cancer.org/, 2024. 2

    work page 2024

  3. [3]

    Intelligent clinical documentation: Harnessing generative ai for patient-centric clinical note generation

    Anjanava Biswas and Wrick Talukdar. Intelligent clinical documentation: Harnessing generative ai for patient-centric clinical note generation. arXiv preprint arXiv:2405.18346 ,

    arXiv

  4. [4]

    Lan- guage models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Lan- guage models are few-shot learners. Advances in neural in- formation processing systems, 33:1877–1901, 2020. 2

    1901

  5. [5]

    Exploring robustness in doctor-patient conversation summarization: An analysis of out-of-domain soap notes

    Yu-Wen Chen and Julia Hirschberg. Exploring robustness in doctor-patient conversation summarization: An analysis of out-of-domain soap notes. arXiv preprint arXiv:2406.02826,

    arXiv

  6. [6]

    Caparena: Benchmarking and analyzing detailed image captioning in the llm era

    Kanzhi Cheng, Wenpo Song, Jiaxin Fan, Zheng Ma, Qiushi Sun, Fangzhi Xu, Chenyang Yan, Nuo Chen, Jianbing Zhang, and Jiajun Chen. Caparena: Benchmarking and analyzing detailed image captioning in the llm era. arXiv preprint arXiv:2503.12329, 2025. 1

    arXiv 2025

  7. [7]

    Qlora: Efficient finetuning of quantized llms

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. Advances in neural information processing systems , 36: 10088–10115, 2023. 1

    2023

  8. [8]

    pathclip: Detection of genes and gene relations from biological pathway figures through image-text contrastive learning

    Fei He, Kai Liu, Zhiyuan Yang, Yibo Chen, Richard D Ham- mer, Dong Xu, and Mihail Popescu. pathclip: Detection of genes and gene relations from biological pathway figures through image-text contrastive learning. IEEE Journal of Biomedical and Health Informatics , 2024. 1

    work page 2024

Show all 39 references
  1. [9]

    Clini- calbert: Modeling clinical notes and predicting hospital read- mission

    Kexin Huang, Jaan Altosaar, and Rajesh Ranganath. Clini- calbert: Modeling clinical notes and predicting hospital read- mission. arXiv preprint arXiv:1904.05342, 2019. 4

    1904 arXiv

  2. [10]

    Enhancing clinical efficiency through llm: Discharge note generation for cardiac patients

    HyoJe Jung, Yunha Kim, Heejung Choi, Hyeram Seo, Minkyoung Kim, JiYe Han, Gaeun Kee, Seohyun Park, Soy- oung Ko, Byeolhee Kim, et al. Enhancing clinical efficiency through llm: Discharge note generation for cardiac patients. arXiv preprint arXiv:2404.05144, 2024. 1

    2024 arXiv

  3. [11]

    Efficient fine-tuning of large language mod- els for automated medical documentation

    Hui Yi Leong, Yi Fan Gao, Ji Shuai, Yang Zhang, and Uktu Pamuksuz. Efficient fine-tuning of large language mod- els for automated medical documentation. arXiv preprint arXiv:2409.09324, 2024. 1

    2024

  4. [12]

    Retrieval-augmented generation for knowledge-intensive nlp tasks

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K¨uttler, Mike Lewis, Wen-tau Yih, Tim Rockt ¨aschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing syst...

    2020

  5. [13]

    Improving clinical note generation from complex doctor-patient conversation

    Yizhan Li, Sifan Wu, Christopher Smith, Thomas Lo, and Bang Liu. Improving clinical note generation from complex doctor-patient conversation. arXiv preprint arXiv:2408.14568, 2024. 1

    2024 arXiv

  6. [14]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out , pages 74–81, 2004. 4

    2004

  7. [15]

    Melanoma - symptoms and causes, 2023

    Mayo Clinic Staff. Melanoma - symptoms and causes, 2023. Accessed: 2025-04-30. 3

    2023

  8. [16]

    Comprehensive cancer informa- tion

    National Cancer Institute. Comprehensive cancer informa- tion. https://www.cancer.gov/, 2024. 2

    2024

  9. [17]

    Melanoma skin cancer - symptoms

    NHS. Melanoma skin cancer - symptoms. https://www. nhs.uk/conditions/melanoma- skin- cancer/ symptoms/, 2024. 2

    2024

  10. [18]

    Pad-ufes-20: A skin lesion dataset composed of patient data and clinical images collected from smartphones

    Andre GC Pacheco, Gustavo R Lima, Amanda S Salo- mao, Breno Krohling, Igor P Biral, Gabriel G de Angelo, F´abio CR Alves Jr, Jos ´e GM Esgario, Alana C Simora, Pe- dro BC Castro, et al. Pad-ufes-20: A skin lesion dataset composed of patient data and clinical images collected f...

    2020

  11. [19]

    chrf++: words helping character n-grams

    Maja Popovi ´c. chrf++: words helping character n-grams. In Proceedings of the second conference on machine transla- tion, pages 612–618, 2017. 4

    2017

  12. [20]

    Generating more faithful and consistent soap notes using attribute-specific parameters

    Sanjana Ramprasad, Elisa Ferracane, and Sai P Selvaraj. Generating more faithful and consistent soap notes using attribute-specific parameters. In Machine Learning for Healthcare Conference, pages 631–649. PMLR, 2023. 1

    2023

  13. [21]

    Rogers, Martin A

    Howard W. Rogers, Martin A. Weinstock, Steven R. Feld- man, and Brett M. Coldiron. Incidence estimate of non- melanoma skin cancer (keratinocyte carcinomas) in the us population, 2012. JAMA Dermatology, 151(10):1081–1086,

    2012

  14. [22]

    Towards an auto- mated soap note: classifying utterances from medical con- versations

    Benjamin Schloss and Sandeep Konam. Towards an auto- mated soap note: classifying utterances from medical con- versations. In Machine Learning for Healthcare Conference, pages 610–631. PMLR, 2020. 1

    2020

  15. [23]

    Large scale sequence-to-sequence models for clinical note genera- tion from patient-doctor conversations

    Gagandeep Singh, Yue Pan, Jesus Andres-Ferrer, Miguel Del-Agua, Frank Diehl, Joel Pinto, and Paul V ozila. Large scale sequence-to-sequence models for clinical note genera- tion from patient-doctor conversations. InProceedings of the 5th Clinical Natural Language Processing Wo...

    2023

  16. [24]

    Large language models encode clinical knowledge

    Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tan- wani, Heather Cole-Lewis, Stephen Pfohl, et al. Large language models encode clinical knowledge. Nature, 620 (7972):172–180, 2023. 1

    2023

  17. [25]

    Skin cancer diagnosis and treat- ment

    Skin Cancer Specialists. Skin cancer diagnosis and treat- ment. https://www.stxskincancer.com, 2024. 2

    2024

  18. [26]

    Clinical text summarization: adapting large language models can outperform human experts

    Dave Van Veen, Cara Van Uden, Louis Blankemeier, Jean- Benoit Delbrouck, Asad Aali, Christian Bluethgen, Anuj Pa- reek, Malgorzata Polacin, Eduardo Pontes Reis, Anna See- hofnerova, et al. Clinical text summarization: adapting large language models can outperform human experts...

    2023

  19. [27]

    Artificial intelligence and skin cancer

    Maria L Wei, Mikio Tada, Alexandra So, and Rodrigo Tor- res. Artificial intelligence and skin cancer. Frontiers in medicine, 11:1331895, 2024. 1

    2024

  20. [28]

    Large language model benchmarks in medical tasks

    Lawrence KQ Yan, Qian Niu, Ming Li, Yichao Zhang, Cait- lyn Heqi Yin, Cheng Fei, Benji Peng, Ziqian Bi, Pohsun Feng, Keyu Chen, et al. Large language model benchmarks in medical tasks. arXiv preprint arXiv:2410.21348, 2024. 1

    2024

  21. [29]

    Large language models in health care: Development, applications, and challenges

    Rui Yang, Ting Fang Tan, Wei Lu, Arun James Thirunavukarasu, Daniel Shu Wei Ting, and Nan Liu. Large language models in health care: Development, applications, and challenges. Health Care Science, 2(4):255–263, 2023. 1

    2023

  22. [30]

    Bertscore: Evaluating text genera- tion with bert

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Wein- berger, and Yoav Artzi. Bertscore: Evaluating text genera- tion with bert. arXiv preprint arXiv:1904.09675, 2019. 4

    1904 arXiv

  23. [31]

    Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in Neural Information Processing Systems, 36:46595–46623, 2023

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in Neural Information Processing Systems, 36:46595–46623, 2023. 3

    2023

  24. [32]

    Pre-trained multimodal large language model enhances dermatological diagnosis using skingpt-4

    Juexiao Zhou, Xiaonan He, Liyuan Sun, Jiannan Xu, Xi- uying Chen, Yuetan Chu, Longxi Zhou, Xingyu Liao, Bin Zhang, Shawn Afvari, et al. Pre-trained multimodal large language model enhances dermatological diagnosis using skingpt-4. Nature Communications, 15(1):5649, 2024. 1 Tow...

    2024

  25. [33]

    Structured SOAP note components

    SOAP Note Structure Figure 2. Structured SOAP note components

  26. [34]

    Recent efforts have focused on generating clinical notes, especially SOAP notes, from Electronic Health Records (EHRs) and doctor-patient conversations

    Related Works Artificial intelligence has been widely applied in the med- ical domain for tasks such as medical image captioning [6], clinical text summarization [26], discharge note gen- eration [10], and medical question answering [28]. Recent efforts have focused on generat...

  27. [35]

    Similar work has explored domains like pediatric rehabilitation [1] and fine-tuning LLMs for effi- cient SOAP note generation [11]

    evaluated generative models across general and SOAP- specific formats. Similar work has explored domains like pediatric rehabilitation [1] and fine-tuning LLMs for effi- cient SOAP note generation [11]. Earlier approaches to SOAP note generation, such as [22, 23], used sequenc...

  28. [36]

    Dataset We use the PAD-UFES-20 dataset [18], which consists of 2,298 dermoscopic images along with structured metadata for 1,641 skin lesions collected from 1,373 patients

    Methodology 7.1. Dataset We use the PAD-UFES-20 dataset [18], which consists of 2,298 dermoscopic images along with structured metadata for 1,641 skin lesions collected from 1,373 patients. The lesions are classified into six types: Basal Cell Carci- noma (BCC), Melanoma (MEL)...

  29. [37]

    carci- noma

    Evaluation 8.1. Quantitative Evaluation 8.1.1. ROUGE ROUGE (Recall-Oriented Understudy for Gisting Evalua- tion) measures n-gram overlap between generated and ref- erence notes. We report ROUGE-1 (unigrams), ROUGE-2 (bigrams), and ROUGE-L (longest common subsequence), capturin...

  30. [38]

    These examples highlights how the framework is able to produce clinically structured documentation aligned with Figure 3

    Example of Generated SOAP Notes Figure 4 presents two representative examples of struc- tured SOAP notes generated by our proposed method, us- ing an input lesion image and its corresponding caption. These examples highlights how the framework is able to produce clinically str...

  31. [39]

    The quality of the generated SOAP notes remains dependent on the accuracy of the retrieved domain-specific knowledge, which may introduce biases or propagate incomplete information

    Limitations and Future Work While our weakly supervised multimodal framework shows strong promise for structured SOAP notes generation, it has few limitations. The quality of the generated SOAP notes remains dependent on the accuracy of the retrieved domain-specific knowledge,...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.