Pith. sign in

REVIEW 3 major objections 5 minor 65 references

Expert-guided Clinical Text Augmentation via Query-Based Model Collaboration

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that having a lightweight clinical named-entity recognition model extract medical tokens and requiring the LLM to preserve them during rewriting yields safer augmented clinical notes and consistent gains on downstream…

desk verdict A plausible incremental method for safer LLM-based clinical augmentation, but the headline safety numbers are partly circular and the downstream gains need significance testing. read the letter →

arxiv 2509.21530 v2 pith:J5TSHM3B submitted 2025-09-25 cs.LG

Dongkyu Cho , Miao Zhang , Rumi Chunara This is my paper · ORCID
classification cs.LG
keywords clinicaltextaugmentationlargelanguagemodelsnamedentityrecognitionmodelcollaborationhallucinationpreventionpredictionMIMIC-IIIpreferenceoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Data augmentation for clinical text is unsafe when a general-purpose LLM decides on its own which parts of a note matter: paraphrases delete diagnoses, symptoms, and medication details, and add content that was never in the record. This paper proposes a query-based model collaboration in which a lightweight biomedical named-entity recognition model first extracts the medical tokens that must not change, and the LLM is then instructed to rewrite the note while preserving those tokens verbatim. Across MIMIC-III notes, the authors report that this guidance raises entity preservation and cuts hallucinated entities compared with naive paraphrasing and with CATO, a style-only augmentation method. If the result holds, small expert models can act as a practical safety layer on top of strong LLMs for high-stakes domains, improving downstream clinical predictions without retraining the LLM or paying for human annotation.

What carries the argument

The load-bearing object is the query-based model collaboration loop. A weak expert $W(\cdot)$ extracts safety-critical tokens $K_i = W(x_i)$; a prompt template passes both the original text and $K_i$ to a strong generalist $G(\cdot)$; and $G$ generates $\tilde{x}_i = G(\mathrm{prompt}(x_i, K_i))$ under an explicit instruction to alter no token in $K_i$. The paper instantiates $W$ as a DistilBERT-based biomedical NER model and $G$ as an instruction-tuned LLM (Qwen-3 or Llama-3), and evaluates quality with Preservation Rate and Hallucination Rate. The framework's extension replaces the two-model loop with a single "strong expert" trained by direct preference optimization on expert-guided versus naive rewrites.

What would settle it

Build an augmented corpus from clinical notes whose label-bearing content includes entities the NER model is known to miss, such as negation cues ("no fever"), rare drug-dose pairs, or abbreviations, and have clinicians judge whether each augmented note still supports its original label. If a substantial fraction of notes change meaning while Preservation Rate and Hallucination Rate look acceptable, the token-preservation assumption fails; the same test in the over-detection direction should show that near-zero style variation produces no downstream gain.

Watch

Extended reading notes

Core claim

The central discovery is that the token-level constraint, not the LLM's own medical knowledge, is what keeps augmented notes faithful. Given an original note $x_i$, the weak expert returns $K_i = W(x_i)$; the prompt tells the strong generalist $G$ to rewrite $x_i$ while keeping every token in $K_i$ verbatim; the output $\tilde{x}_i$ is paired with the original label $y_i$ to form the augmented dataset. On 300 MIMIC-III samples the paper measures Preservation Rate rising from 0.48-0.62 for naive or CATO augmentation to 0.66-0.79 under the collaborative method, and Hallucination Rate falling from 0.59-0.77 to 0.33-0.43. Training downstream models on these notes improves readmission accuracy from 0.552 (CATO) to 0.599 with Qwen-3, and improves mortality and length-of-stay metrics; zero/few-shot phenotyping and ICD coding retain or improve signal while baseline augmentations degrade. The authors also show that the dual-model guidance can be distilled into a single model via preference learning, though with less consistent gains across backbones.

Load-bearing premise

The load-bearing premise is that the token set $K_i = W(x_i)$ extracted by the biomedical NER model is exactly the label-relevant content of the note, so preserving those tokens verbatim leaves the label $y_i$ unchanged; if the extractor misses a label-bearing term, the LLM can quietly alter it and the augmented pair becomes mislabeled.

Editorial extensions

If this is right

  • Augmented clinical notes produced under expert-token constraints are safer training data than naive or CATO rewrites, with higher Preservation Rate and lower Hallucination Rate on the reported benchmarks.
  • Downstream clinical predictors trained on these notes improve readmission and mortality accuracy and reduce length-of-stay RMSE compared with baselines, and the gains replicate across Qwen-3 and BERT backbones.
  • Zero-, one-, and few-shot phenotyping and ICD coding retain or improve performance on augmented notes, whereas naive and CATO augmentations degrade these tasks.
  • A single LLM fine-tuned with direct preference optimization can partially reproduce the dual-model guidance, giving an alternate deployment mode when medical priors are already present.
  • Even a general-purpose entity extractor as the weak expert improves augmentation quality, indicating the framework tolerates imperfect expert signals.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that this framework is portable: any domain where a small extractor can flag invariant tokens (contract clauses, regulated financial terms) could reuse the same prompt-collaboration recipe, because the LLM never needs retraining.
  • A testable extension the authors mention only in passing is to use PR/HR thresholds as an acceptance gate, selecting only augmented notes that pass before training; that would turn the method into a data-selection pipeline rather than a pure generator.
  • A limitation not explored in the paper is contextual meaning: tokens like "no fever" or "without chest pain" carry negation that verbatim token preservation cannot protect, so a dependency-aware constraint would be needed to fully guarantee label preservation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a model-collaboration framework for clinical text augmentation: a lightweight clinical NER model (the weak expert) extracts safety-critical tokens, and a general-purpose LLM (the strong generalist) rewrites clinical notes while preserving those tokens verbatim. The authors evaluate the quality of the generated notes using preservation rate (PR) and hallucination rate (HR), and measure downstream utility by training Qwen-3 and BERT models on augmented MIMIC-III notes for readmission, mortality, and length-of-stay prediction, plus zero/few-shot phenotyping and ICD coding. They report that their method outperforms naive LLM paraphrasing and the CATO style-only baseline on both safety metrics and downstream tasks, and they additionally show that the guidance can be distilled into a single model via DPO. The core claim is that weak-expert constraints provide a practical safety layer for LLM-based augmentation in high-stakes domains.

Significance. If the safety and utility claims hold, the framework is attractive: it is lightweight, requires no additional training of the weak expert, and is compatible with small open-weight LLMs. The authors also state that prompts, acceptance decisions, and scripts will be released, which would aid reproducibility. The zero/few-shot phenotyping and ICD retrieval evaluations are a useful complement to supervised fine-tuning. However, the central safety evidence is weakened by a circular evaluation: PR and HR are measured with the same NER tool that produces the preservation constraints, and the acceptance filter is not described for baselines. Downstream gains are often within one standard error of the strongest baseline, and no significance tests are reported. These issues are fixable but currently leave the main claims insufficiently supported.

major comments (3)
  1. [§5.3, §A.3, Eq. (3)] The headline safety results in Table 1 are partly by construction. Equation (3) defines PR and HR using E(·), which A.3 states is "the same tool used to create K_i"—i.e., the biomedical NER that generates the constraints. The prompt in A.1 then instructs the LLM to preserve exactly those strings verbatim. Thus, Ours is rewarded for keeping the exact tokens it was told to keep, while baselines that legitimately rephrase (e.g., "hypertension" to "high blood pressure") are penalized as deletions. Moreover, A.3 describes an acceptance filter with thresholds τ_PR and τ_HR, but the thresholds are never specified and it is not stated whether the same filter is applied to the Naive and CATO baselines. If the filter is applied only to Ours, Table 1 reports selected outputs, not representative ones. Please provide an independent entity-level evaluation (e.g., a different NER or manual review), report the thresholds, and apply identical filtering to all methods.
  2. [§4.2, §6] Token-level preservation does not establish label preservation. The framework assumes that K_i = W(x_i) is exactly the label-relevant variable set V, but no evidence is provided that the NER is complete or accurate with respect to the clinical labels. Section 6 itself concedes: "When weak experts under-detect, medical facts change; when they over-detect, augmentation variation is limited." An LLM can keep an entity string while changing its negation, scope, or temporality (e.g., "no fever" to "fever"). Therefore the claim of "reduced hallucinations at both the token and concept levels" is not supported by PR/HR alone. Please add a direct check of label consistency on augmented notes, or an error analysis showing that semantic changes beyond token deletion are absent.
  3. [Table 2, Figure 5, Table 3] The downstream utility claim is weaker than stated. In Table 2, most differences between Ours and the best baseline are within one standard error: for BERT mortality, Ours is 0.929 ± 0.03 versus CATO 0.923 ± 0.003; for Qwen readmission, Ours is 0.599 ± 0.03 versus CATO 0.552 ± 0.04, which is less than two standard errors. No significance tests, confidence intervals, or paired comparisons are reported, so "consistent performance gains" is not statistically established. Additionally, hyperparameters in A.2 were selected by grid search on the readmission task, which may inflate that task's result; please clarify whether the same fixed protocol was used for all augmentation methods and report significance tests or effect sizes.
minor comments (5)
  1. [§4.2] There is a typo: "G receives explicit guidance on domain-critical tokens, t avoids distorting" should read "it avoids distorting."
  2. [§6] The word "rresults" appears in the sentence "medical specialization yields the best rresults"; please correct it.
  3. [Table 3] The column header "Pred." should be "Prec." for precision, to match the text and standard notation.
  4. [§5.2, Table 1] The model name is inconsistently written as "LLama" and "Llama"; please standardize to "Llama" or "LLaMA" throughout.
  5. [§A.2] The hyperparameter analysis in Tables 6–8 reports accuracy/F1 for readmission only; if these hyperparameters were used for all downstream tasks, please state that explicitly, since the mortality and length-of-stay results may have different optimal settings.

Circularity Check

1 steps flagged · score 6.0 of 10

PR/HR safety evidence is partly circular: the same NER that generates the preservation constraints also scores the outputs, so the headline safety gain is partly by construction.

  1. self definitional [Appendix A.3 (Augmentation Metrics), with Eq. (1) in Section 4.2 and the prompt in Appendix A.1]
    "where E(·) denotes the set of entities extracted by the same tool used to create K_i. We accept a candidate only when PR meets or exceeds τPR and HR is at or below τHR."

    Eq. (1) defines K_i = W(x_i), and the A.1 prompt instructs the generalist to preserve every token in K_i verbatim. PR is then computed as |E(˜x_i) ∩ E(x_i)| / |E(x_i)|, where E(·) is explicitly 'the same tool used to create K_i'. Thus the Ours condition is rewarded for re-emitting the exact constraint list it was ordered to include, while baselines that were never given this list are penalized for legitimate paraphrases (e.g., 'hypertension' to 'high blood pressure'). The acceptance filter selects candidates using this same circular score, so Table 1's PR/HR gap is largely enforced by construction. Moreover, token-level preservation does not establish concept-level safety, because a token can be kept while its negation, scope, or temporality changes.

full rationale

The circularity is concentrated in the safety metric, not the downstream evaluation. Appendix A.3 defines both PR and HR using E(·), explicitly 'the same tool used to create K_i', where K_i is the exact token list that the A.1 prompt orders the generalist to preserve verbatim. Therefore Ours is scored on whether it re-emits the constraint list; competing methods that were never given this list are penalized for any legitimate paraphrase, and the acceptance filter further selects candidates using the same circular score. This makes the headline 'preserves critical medical information / reduces hallucinations at token and concept levels' partly self-definitional. By contrast, the downstream readmission, mortality, length-of-stay, phenotyping, and ICD-coding results use held-out labels and real notes, so the utility claim has independent content; Table 2 gains are modest relative to standard errors, which is a statistical-strength concern rather than circularity. Section 6 openly admits the under/over-detection limitation, which corroborates the dependence of the claimed safety on the completeness of W. There is no load-bearing self-citation chain or imported uniqueness theorem; cited same-author work (Cho et al. 2025) is only a related-work justification. Overall score 6: the central safety evidence partially reduces by construction, while downstream tasks keep the paper from being fully circular.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The framework has no invented physical entities. Its conceptual products are the weak expert W, an existing NER model; the strong generalist G, an existing LLM; and the distilled Strong Expert W*, a DPO-tuned model. The free parameters are the acceptance thresholds and hyperparameters selected during development. The axioms are the causal-validity assumptions of the augmentation, all domain assumptions from the causal-invariance framing.

free parameters (3)
  • Augmentation acceptance thresholds tau_PR and tau_HR = not specified in paper
    Appendix A.3 states candidates are accepted only when PR >= tau_PR and HR <= tau_HR; these thresholds determine the reported quality metrics and downstream results, and baselines are not described as receiving the same filtering.
  • SFT hyperparameters (LoRA rank, learning rate, epochs) = r=16, LR=4e-5, epochs=3 (Tables 6-8)
    Selected by grid search on the readmission validation set in Appendix A.2; the downstream gains in Table 2 depend on this choice, and Table 8 reports a different readmission accuracy at 3 epochs than Table 2, indicating run-to-run variability.
  • DPO temperature and strength = not specified, following common practice
    Used in the Strong Expert distillation in Appendix A.3; values are asserted without details, and they affect the Table 5 results for the single-model variant.
assumptions (4)
  • domain assumption Label y depends only on domain variables V, which are medical entities, while style variables U do not affect y.
    Section 3, Figure 3 adopts this causal graph from Feder et al. (2023). If style correlates with y through confounders, rewriting style while preserving entities may still change the label.
  • domain assumption The biomedical NER model W identifies all and only the label-relevant tokens in V.
    Section 4.1 Equation (1) makes the entire method depend on the extracted set K_i. Under-detection or over-detection directly changes the augmentation semantics, as acknowledged in Section 6.
  • domain assumption LLM rewrites that preserve K_i verbatim are valid counterfactuals with the same label y_i.
    Section 4.2 Step C pairs each augmented note with the original label; no label-invariance verification is performed, and changes outside K_i may still affect the label.
  • domain assumption Entity extraction by the same NER tool is a valid measure of medical information preservation via PR and HR.
    Appendix A.3 defines E(.) using the same tool that creates K_i, so PR and HR measure constraint adherence rather than independent clinical correctness.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Expert-guided Clinical Text Augmentation via Query-Based Model Collaboration." pith.science (2026). https://pith.science/paper/J5TSHM3B

@misc{pith2026250921530,
  author       = {Pith},
  title        = {Pith review of: Expert-guided Clinical Text Augmentation via Query-Based Model Collaboration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J5TSHM3B}},
  note         = {Machine review of arXiv:2509.21530}
}
read the original abstract

Data augmentation is a widely used strategy to improve model robustness and generalization by enriching training datasets with synthetic examples. While large language models (LLMs) have demonstrated strong generative capabilities for this purpose, their applications in high-stakes domains like healthcare present unique challenges due to the risk of generating clinically incorrect or misleading information. In this work, we propose a novel query-based model collaboration framework that integrates expert-level domain knowledge to guide the augmentation process to preserve critical medical information. Compared to existing LLM-based and traditional augmentation methods, our generated data significantly improves preservation of critical medical information and reduces hallucinations at both the token and concept levels. Experiments on downstream clinical prediction tasks demonstrate consistent performance gains over existing augmentation methods. This lightweight collaborative framework addresses the gap between LLM augmentation potential and the safety requirements of specialized domains.

Figures

Figures reproduced from arXiv: 2509.21530 by the authors.

Figure 1
Figure 1. LLM-based Augmentation methods fail when data requires expert domain knowledge. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Synthetic clinical notes generated by different LLM-based augmentation methods are [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Clinical language model predictions (y) are in￾fluenced by both meaningful domain variables (V) and spu￾rious features (U) extracted from note data (X). Causally Driven Data Augmentation. In many safety-critical applications (e.g., clinical or legal domains), we often have ad￾ditional domain knowledge or causal assumptions that elucidate which variables (tokens, phrases) truly affect the label y (Feder et al., 2023;… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Comparison of augmentation strategies. (a) Previous methods do not provide supervision [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Zero/One/Few-shot F1 scores on Patient Phenotyping task using different inference models. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 25 canonical work pages

  1. [1]

    Automated clinical coding using off-the-shelf large language models

    Joseph S Boyle, Antanas Kascenas, Pat Lok, Maria Liakata, and Alison Q O'Neil. Automated clinical coding using off-the-shelf large language models. arXiv preprint arXiv:2310.06552, 2023

    arXiv 2023

  2. [2]

    Weak-to-strong generalization: Eliciting strong capabilities with weak supervision, 2023

    Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner, Bowen Baker, Leo Gao, Leopold Aschenbrenner, Yining Chen, Adrien Ecoffet, Manas Joglekar, Jan Leike, Ilya Sutskever, and Jeff Wu. Weak-to-strong generalization: Eliciting strong capabilities with weak supervision, 2023. URL https://arxiv.org/abs/2312.09390

    arXiv 2023

  3. [3]

    Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission

    Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp.\ 1721--1730, 2015

    work page 2015

  4. [4]

    Yaping Chai, Haoran Xie, and Joe S. Qin. Text data augmentation for large language models: A comprehensive survey of methods, challenges, and opportunities, 2025. URL https://arxiv.org/abs/2501.18845

    arXiv 2025

  5. [5]

    Don't do rag: When cache-augmented generation is all you need for knowledge tasks

    Brian J Chan, Chao-Ting Chen, Jui-Hung Cheng, and Hen-Hsen Huang. Don't do rag: When cache-augmented generation is all you need for knowledge tasks. In Companion Proceedings of the ACM on Web Conference 2025, pp.\ 893--897, 2025

    2025

  6. [6]

    Hiddencut: Simple data augmentation for natural language understanding with better generalizability

    Jiaao Chen, Dinghan Shen, Weizhu Chen, and Diyi Yang. Hiddencut: Simple data augmentation for natural language understanding with better generalizability. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp.\ 4380--...

    work page 2021

  7. [7]

    An empirical survey of data augmentation for limited data learning in nlp

    Jiaao Chen, Derek Tam, Colin Raffel, Mohit Bansal, and Diyi Yang. An empirical survey of data augmentation for limited data learning in nlp. Transactions of the Association for Computational Linguistics, 11: 0 191--211, 2023

    work page 2023

  8. [8]

    Robust neural machine translation with doubly adversarial inputs

    Yong Cheng, Lu Jiang, and Wolfgang Macherey. Robust neural machine translation with doubly adversarial inputs. In Anna Korhonen, David Traum, and Llu \'i s M \`a rquez (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp.\ 4324--4333, Florence, Italy, July 2019. Association for Computational Linguistics. doi:...

    doi:10.18653/v1/p19-1425 2019

Show all 65 references
  1. [9]

    Peer pressure: Model-to-model regularization for single source domain generalization

    Dong Kyu Cho, Inwoo Hwang, and Sanghack Lee. Peer pressure: Model-to-model regularization for single source domain generalization. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp.\ 15360--15370, 2025

    2025

  2. [10]

    Le, Sergey Levine, and Yi Ma

    Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V. Le, Sergey Levine, and Yi Ma. Sft memorizes, rl generalizes: A comparative study of foundation model post-training, 2025. URL https://arxiv.org/abs/2501.17161

    2025 arXiv

  3. [11]

    Auggpt: Leveraging chatgpt for text data augmentation

    Haixing Dai, Zhengliang Liu, Wenxiong Liao, Xiaoke Huang, Yihan Cao, Zihao Wu, Lin Zhao, Shaochen Xu, Fang Zeng, Wei Liu, et al. Auggpt: Leveraging chatgpt for text data augmentation. IEEE Transactions on Big Data, 2025

    2025

  4. [12]

    Effective hospital readmission prediction models using machine-learned features

    Sacha Davis, Jin Zhang, Ilbin Lee, Mostafa Rezaei, Russell Greiner, Finlay A McAlister, and Raj Padwal. Effective hospital readmission prediction models using machine-learned features. BMC Health Services Research, 22 0 (1): 0 1415, 2022

    2022

  5. [13]

    Rephrase and respond: Let large language models ask better questions for themselves, 2024

    Yihe Deng, Weitong Zhang, Zixiang Chen, and Quanquan Gu. Rephrase and respond: Let large language models ask better questions for themselves, 2024. URL https://arxiv.org/abs/2311.04205

    2024 arXiv

  6. [14]

    Data augmentation using llms: Data perspectives, learning paradigms and challenges

    Bosheng Ding, Chengwei Qin, Ruochen Zhao, Tianze Luo, Xinze Li, Guizhen Chen, Wenhan Xia, Junjie Hu, Luu Anh Tuan, and Shafiq Joty. Data augmentation using llms: Data perspectives, learning paradigms and challenges. In Findings of the Association for Computational Linguistics ...

    2024

  7. [15]

    Causal inference in natural language processing: Estimation, prediction, interpretation and beyond

    Amir Feder, Katherine A Keith, Emaad Manzoor, Reid Pryzant, Dhanya Sridhar, Zach Wood-Doughty, Jacob Eisenstein, Justin Grimmer, Roi Reichart, Margaret E Roberts, et al. Causal inference in natural language processing: Estimation, prediction, interpretation and beyond. Transac...

    2022

  8. [16]

    Data augmentations for improved (large) language model generalization

    Amir Feder, Yoav Wald, Claudia Shi, Suchi Saria, and David Blei. Data augmentations for improved (large) language model generalization. Advances in Neural Information Processing Systems, 36: 0 70638--70653, 2023

    2023

  9. [17]

    A survey of data augmentation approaches for nlp

    Steven Y Feng, Varun Gangal, Jason Wei, Sarath Chandar, Soroush Vosoughi, Teruko Mitamura, and Eduard Hovy. A survey of data augmentation approaches for nlp. arXiv preprint arXiv:2105.03075, 2021

    2021 arXiv

  10. [18]

    Interpretable machine learning models for hospital readmission prediction: a two-step extracted regression tree approach

    Xiaoquan Gao, Sabriya Alam, Pengyi Shi, Franklin Dexter, and Nan Kong. Interpretable machine learning models for hospital readmission prediction: a two-step extracted regression tree approach. BMC medical informatics and decision making, 23 0 (1): 0 104, 2023

    2023

  11. [19]

    Comparing deep learning and concept extraction based methods for patient phenotyping from clinical narratives

    Sebastian Gehrmann, Franck Dernoncourt, Yeran Li, Eric T Carlson, Joy T Wu, Jonathan Welt, John Foote Jr, Edward T Moseley, David W Grant, Patrick D Tyler, et al. Comparing deep learning and concept extraction based methods for patient phenotyping from clinical narratives. Plo...

    2018

  12. [20]

    How much data are augmentations worth? an investigation into scaling laws, invariance, and implicit regularization

    Jonas Geiping, Micah Goldblum, Gowthami Somepalli, Ravid Shwartz-Ziv, Tom Goldstein, and Andrew Gordon Wilson. How much data are augmentations worth? an investigation into scaling laws, invariance, and implicit regularization. arXiv preprint arXiv:2210.06441, 2022

    arXiv

  13. [21]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  14. [22]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. URL https://arxiv.org/abs/2106.09685

    2021 arXiv

  15. [23]

    Clinicalbert: Modeling clinical notes and predicting hospital readmission

    Kexin Huang, Jaan Altosaar, and Rajesh Ranganath. Clinicalbert: Modeling clinical notes and predicting hospital readmission. arXiv preprint arXiv:1904.05342, 2019

    1904 arXiv

  16. [24]

    Health system-scale language models are all-purpose prediction engines

    Lavender Yao Jiang, Xujin Chris Liu, Nima Pour Nejatian, Mustafa Nasir-Moin, Duo Wang, Anas Abidin, Kevin Eaton, Howard Antony Riina, Ilya Laufer, Paawan Punjabi, et al. Health system-scale language models are all-purpose prediction engines. Nature, 619 0 (7969): 0 357--362, 2023

    2023

  17. [25]

    Mimic-iii, a freely accessible critical care database

    Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. Mimic-iii, a freely accessible critical care database. Scientific data, 3 0 (1): 0 1--9, 2016

    2016

  18. [26]

    Risk prediction models for hospital readmission: a systematic review

    Devan Kansagara, Honora Englander, Amanda Salanitro, David Kagen, Cecelia Theobald, Michele Freeman, and Sunil Kripalani. Risk prediction models for hospital readmission: a systematic review. Jama, 306 0 (15): 0 1688--1698, 2011

    2011

  19. [27]

    Machine learning-based in-hospital mortality prediction models for patients with acute coronary syndrome

    Jun Ke, Yiwei Chen, Xiaoping Wang, Zhiyong Wu, Qiongyao Zhang, Yangpeng Lian, and Feng Chen. Machine learning-based in-hospital mortality prediction models for patients with acute coronary syndrome. The American journal of emergency medicine, 53: 0 127--134, 2022

    2022

  20. [28]

    Key challenges for delivering clinical impact with artificial intelligence

    Christopher J Kelly, Alan Karthikesalingam, Mustafa Suleyman, Greg Corrado, and Dominic King. Key challenges for delivering clinical impact with artificial intelligence. BMC medicine, 17: 0 1--9, 2019

    2019

  21. [29]

    Medical hallucinations in foundation models and their impact on healthcare

    Yubin Kim, Hyewon Jeong, Shan Chen, Shuyue Stella Li, Mingyu Lu, Kumail Alhamoud, Jimin Mun, Cristina Grau, Minseok Jung, Rodrigo Gameiro, et al. Medical hallucinations in foundation models and their impact on healthcare. arXiv preprint arXiv:2503.05777, 2025

    2025

  22. [30]

    Biobert: a pre-trained biomedical language representation model for biomedical text mining

    Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36 0 (4): 0 1234--1240, 2020

    2020

  23. [31]

    From quantity to quality: Boosting llm performance with self-guided data selection for instruction tuning, 2024 a

    Ming Li, Yong Zhang, Zhitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jianzong Wang, Tianyi Zhou, and Jing Xiao. From quantity to quality: Boosting llm performance with self-guided data selection for instruction tuning, 2024 a . URL https://arxiv.org/abs/2308.12032

    2024 arXiv

  24. [32]

    Empowering large language models for textual data augmentation

    Yichuan Li, Kaize Ding, Jianling Wang, and Kyumin Lee. Empowering large language models for textual data augmentation. arXiv preprint arXiv:2404.17642, 2024 b

    2024 arXiv

  25. [33]

    Benchmarking generation and evaluation capabilities of large language models for instruction controllable summarization

    Yixin Liu, Alexander Fabbri, Jiawen Chen, Yilun Zhao, Simeng Han, Shafiq Joty, Pengfei Liu, Dragomir Radev, Chien-Sheng Wu, and Arman Cohan. Benchmarking generation and evaluation capabilities of large language models for instruction controllable summarization. In Kevin Duh, H...

    2024 doi

  26. [34]

    Improving the robustness and accuracy of biomedical language models through adversarial training

    Milad Moradi and Matthias Samwald. Improving the robustness and accuracy of biomedical language models through adversarial training. Journal of Biomedical Informatics, 132: 0 104114, 2022

    2022

  27. [35]

    Explainable prediction of medical codes from clinical text

    James Mullenbach, Sarah Wiegreffe, Jon Duke, Jimeng Sun, and Jacob Eisenstein. Explainable prediction of medical codes from clinical text. arXiv preprint arXiv:1802.05695, 2018

    2018 arXiv

  28. [36]

    Large language models in healthcare and medical domain: A review

    Zabir Al Nazi and Wei Peng. Large language models in healthcare and medical domain: A review. In Informatics, volume 11, pp.\ 57. MDPI, 2024

    2024

  29. [37]

    On the impact of data augmentation on downstream performance in natural language processing

    Itsuki Okimura, Machel Reid, Makoto Kawano, and Yutaka Matsuo. On the impact of data augmentation on downstream performance in natural language processing. In Proceedings of the third workshop on insights from negative results in NLP, pp.\ 88--93, 2022

    2022

  30. [38]

    Tasnim Pervin, Linmi Tao, Aminul Huq, Zuoxiang He, and Li Huo

    Mst. Tasnim Pervin, Linmi Tao, Aminul Huq, Zuoxiang He, and Li Huo. Adversarial attack driven data augmentation for accurate and robust medical image segmentation, 2021. URL https://arxiv.org/abs/2105.12106

    2021 arXiv

  31. [39]

    Rephrasing natural text data with different languages and quality levels for large language model pre-training, 2024

    Michael Pieler, Marco Bellagente, Hannah Teufel, Duy Phung, Nathan Cooper, Jonathan Tow, Paulo Rocha, Reshinth Adithyan, Zaid Alyafeai, Nikhil Pinnaparaju, Maksym Zhuravinskyi, and Carlos Riquelme. Rephrasing natural text data with different languages and quality levels for la...

    2024 arXiv

  32. [40]

    Manning, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model, 2024. URL https://arxiv.org/abs/2305.18290

    2024 arXiv

  33. [41]

    Generalization in healthcare ai: Evaluation of a clinical large language model

    Salman Rahman, Lavender Yao Jiang, Saadia Gabriel, Yindalon Aphinyanaphongs, Eric Karl Oermann, and Rumi Chunara. Generalization in healthcare ai: Evaluation of a clinical large language model. arXiv preprint arXiv:2402.10965, 2024

    2024 arXiv

  34. [42]

    Large-scale application of named entity recognition to biomedicine and epidemiology

    Shaina Raza, Deepak John Reji, Femi Shajan, and Syed Raza Bashir. Large-scale application of named entity recognition to biomedicine and epidemiology. PLOS Digital Health, 1 0 (12): 0 e0000152, 2022

    2022

  35. [43]

    Chatgpt and other large language models are double-edged swords, 2023

    Yiqiu Shen, Laura Heacock, Jonathan Elias, Keith D Hentel, Beatriu Reig, George Shih, and Linda Moy. Chatgpt and other large language models are double-edged swords, 2023

    2023

  36. [44]

    A unified framework of data augmentation using large language models for text-based cross-modal retrieval

    Lijia Si, Caili Guo, Zheng Li, and Yang Yang. A unified framework of data augmentation using large language models for text-based cross-modal retrieval. Pattern Recognition, pp.\ 111755, 2025

    2025

  37. [45]

    Rag-hat: A hallucination-aware tuning pipeline for llm in retrieval-augmented generation

    Juntong Song, Xingguang Wang, Juno Zhu, Yuanhao Wu, Xuxin Cheng, Randy Zhong, and Cheng Niu. Rag-hat: A hallucination-aware tuning pipeline for llm in retrieval-augmented generation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Ind...

    2024

  38. [46]

    Llm-check: Investigating detection of hallucinations in large language models

    Gaurang Sriramanan, Siddhant Bharti, Vinu Sankar Sadasivan, Shoumik Saha, Priyatham Kattakinda, and Soheil Feizi. Llm-check: Investigating detection of hallucinations in large language models. Advances in Neural Information Processing Systems, 37: 0 34188--34216, 2024

    2024

  39. [47]

    Improving commonsense causal reasoning by adversarial training and data augmentation, 2021

    Ieva Staliūnaitė, Philip John Gorinski, and Ignacio Iacobacci. Improving commonsense causal reasoning by adversarial training and data augmentation, 2021. URL https://arxiv.org/abs/2101.04966

    2021 arXiv

  40. [48]

    A systematic review of the prediction of hospital length of stay: Towards a unified framework

    Kieran Stone, Reyer Zwiggelaar, Phil Jones, and Neil Mac Parthal \'a in. A systematic review of the prediction of hospital length of stay: Towards a unified framework. PLOS digital health, 1 0 (4): 0 e0000017, 2022

    2022

  41. [49]

    Corex: Pushing the boundaries of complex reasoning through multi-model collaboration, 2024

    Qiushi Sun, Zhangyue Yin, Xiang Li, Zhiyong Wu, Xipeng Qiu, and Lingpeng Kong. Corex: Pushing the boundaries of complex reasoning through multi-model collaboration, 2024. URL https://arxiv.org/abs/2310.00280

    2024 arXiv

  42. [50]

    A comprehensive survey of hallucination mitigation techniques in large language models

    SM Tonmoy, SM Zaman, Vinija Jain, Anku Rani, Vipula Rawte, Aman Chadha, and Amitava Das. A comprehensive survey of hallucination mitigation techniques in large language models. arXiv preprint arXiv:2401.01313, 6, 2024

    2024 arXiv

  43. [51]

    Cheap and good? simple and effective data augmentation for low resource machine reading

    Hoang Van, Vikas Yadav, and Mihai Surdeanu. Cheap and good? simple and effective data augmentation for low resource machine reading. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’21, pp.\ 2116–2120. A...

    2021

  44. [52]

    Unleashing the emergent cognitive synergy in large language models: A task-solving agent through multi-persona self-collaboration, 2024

    Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. Unleashing the emergent cognitive synergy in large language models: A task-solving agent through multi-persona self-collaboration, 2024. URL https://arxiv.org/abs/2307.05300

    2024 arXiv

  45. [53]

    Qwen3 technical report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jia...

    2025 arXiv

  46. [54]

    A large language model for electronic health records

    Xi Yang, Aokun Chen, Nima PourNejatian, Hoo Chang Shin, Kaleb E Smith, Christopher Parisien, Colin Compas, Cheryl Martin, Anthony B Costa, Mona G Flores, et al. A large language model for electronic health records. NPJ digital medicine, 5 0 (1): 0 194, 2022

    2022

  47. [55]

    Llm lies: Hallucinations are not bugs, but features as adversarial examples, 2024

    Jia-Yu Yao, Kun-Peng Ning, Zhen-Hui Liu, Mu-Nan Ning, Yu-Yang Liu, and Li Yuan. Llm lies: Hallucinations are not bugs, but features as adversarial examples, 2024. URL https://arxiv.org/abs/2310.01469

    2024 arXiv

  48. [56]

    Gpt3mix: Leveraging large-scale language models for text augmentation

    Kang Min Yoo, Dongju Park, Jaewook Kang, Sang-Woo Lee, and Woomyeong Park. Gpt3mix: Leveraging large-scale language models for text augmentation. arXiv preprint arXiv:2104.08826, 2021

    2021 arXiv

  49. [57]

    Large language model as attributed training data generator: A tale of diversity and bias

    Yue Yu, Yuchen Zhuang, Jieyu Zhang, Yu Meng, Alexander J Ratner, Ranjay Krishna, Jiaming Shen, and Chao Zhang. Large language model as attributed training data generator: A tale of diversity and bias. Advances in Neural Information Processing Systems, 36: 0 55734--55784, 2023

    2023

  50. [58]

    Kevin Zhou

    Xu Zhang, Kun Zhang, Wenxin Ma, Rongsheng Wang, Chenxu Wu, Yingtai Li, and S. Kevin Zhou. A general knowledge injection framework for icd coding, 2025. URL https://arxiv.org/abs/2505.18708

    2025 arXiv

  51. [59]

    Flipda: Effective and robust data augmentation for few-shot learning

    Jing Zhou, Yanan Zheng, Jie Tang, Jian Li, and Zhilin Yang. Flipda: Effective and robust data augmentation for few-shot learning. arXiv preprint arXiv:2108.06332, 2021

    2021 arXiv

  52. [60]

    Explore spurious correlations at the concept level in language models for text classification

    Yuhang Zhou, Paiheng Xu, Xiaoyu Liu, Bang An, Wei Ai, and Furong Huang. Explore spurious correlations at the concept level in language models for text classification. arXiv preprint arXiv:2311.08648, 2023

    2023 arXiv

  53. [61]

    Explore spurious correlations at the concept level in language models for text classification

    Yuhang Zhou, Paiheng Xu, Xiaoyu Liu, Bang An, Wei Ai, and Furong Huang. Explore spurious correlations at the concept level in language models for text classification. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Associa...

    2024 doi

  54. [62]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  55. [63]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  56. [64]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  57. [65]

    weak expert

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

    2019

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.