Pith. sign in

REVIEW 3 major objections 6 minor 39 references

Enhancing Clinical Models with Pseudo Data for De-identification

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that continuing to pretrain language models on MIMIC-III's redacted clinical notes—with the redaction masks either left in place or replaced by realistic pseudo data—produces protected-health-information…

desk verdict A useful and reproducible comparison of masked vs pseudo continued pretraining for de-identification, but the headline masked-beats-pseudo claim is confounded by hyperparameters tuned on the masked model alone. read the letter →

arxiv 2506.12674 v2 pith:7JIUI4ZN submitted 2025-06-15 cs.CL

Paul Landes , Aaron J Chaise , Tarak Nath Nandi , Ravi K Madduri This is my paper
classification cs.CL
keywords de-identificationprotectedhealthinformationclinicalNLPpseudodataredactedtextMIMIC-IIIi2b2/UTHealthcontinuedpretraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether redacted text, the standard form of privacy-protected clinical corpora, harms language models trained on it, and whether replacing redaction masks with realistic pseudo data helps. It pretrains eight encoder-only checkpoints on MIMIC-III free-text notes, keeping masks in one copy and substituting generated fake names, dates, numbers, and other entities in another. After fine-tuning on the 2014 i2b2/UTHealth de-identification benchmark, the masked and pseudo models outperform all previous baselines, with the best masked RoBERTa large model beating the strongest published baseline by 1.5 macro F1 points. The central claim is that this continued-pretraining recipe yields better protected health information (PHI) tagging, and that the relationship between redaction and pseudo data is subtler than expected: masking helps RoBERTa large most, while pseudo data consistently helps XLM-RoBERTa.

What carries the argument

The machinery is a two-phase training pipeline centered on a pseudo dataset constructed by regular-expression parsing of MIMIC-III's redaction masks. Each masked token such as [**Doctor Last Name**] is mapped to a tag, and a generator samples a replacement from gazetteer lists (US census surnames, first-name popularity tables, hospitals, universities, companies, states) or from randomized date, number, and phone generators; tokens without a generator are filled by the clinical language model GatorTron. From the same underlying notes the authors build two pretraining corpora—one keeping masks ('masked dataset') and one with substitutions ('pseudo dataset')—and continue pretraining RoBERTa base and large, and XLM-RoBERTa base and large, from their public checkpoints for six epochs. Each of the eight resulting checkpoints is then fine-tuned on the i2b2/UTHealth PHI tagging task with a linear classification head, and evaluated under the HIPAA label mapping used by prior work.

What would settle it

Retrain the pseudo RoBERTa large model with settings tuned on its own validation performance instead of the masked model's, and see whether it still trails the masked model; if it closes the gap, the masked advantage was a tuning artifact, and a complementary check is to score all eight checkpoints on a second de-identification corpus.

Watch

Extended reading notes

Core claim

The central discovery, on the paper's own terms, is that continued pretraining on MIMIC-III redacted text—whether the redaction masks are left intact or replaced by realistic pseudo data—produces de-identification models that beat existing published baselines and off-the-shelf clinical embeddings by a wide margin. The best model, Masked RoBERTa Large, reaches 98.4 macro F1 and 99.898 weighted F1, surpassing the best previous baseline by 1.5 F1 points. The surprising finding is that for RoBERTa large the masked checkpoint outperforms its pseudo counterpart, while for XLM-RoBERTa the pseudo checkpoints consistently beat the masked ones; this suggests that the benefit of pseudo data depends on the base model family. The paper attributes the XLM-RoBERTa advantage to its multilingual exposure, which helps it learn from low-entropy pseudo names and Latin-rooted medical terms, and notes that poor-quality pseudo replacements, such as bare numeric ages, can hurt downstream performance.

Load-bearing premise

The pretraining settings were tuned on the model trained with redaction masks left in, so the comparison assumes those settings suit the models trained on replacement data just as well; if the settings favor the masked models, the headline result could come from tuning rather than from the text.

Editorial extensions

If this is right

  • Redacted clinical corpora are not a liability for continued pretraining: leaving MIMIC-III's redaction masks intact produces the best de-identification model in the paper.
  • Replacing masks with realistic pseudo data gives a consistent advantage specifically for multilingual XLM-RoBERTa models, so pseudo data is useful for some base model families but not all.
  • The improvement over prior work (1.5 F1 points for the best model versus the strongest baseline) indicates that domain-specific continued pretraining still has headroom even on a near-saturated task.
  • Because the pseudo dataset and checkpoints are released, other researchers can reproduce the pipeline and test the masked-versus-pseudo comparison on other clinical tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The masked-versus-pseudo comparison is partly confounded: pretraining settings were tuned on the masked model, so an editorial but testable inference is that retuning on the pseudo model could shrink or reverse RoBERTa large's apparent masked advantage.
  • The same redaction question likely applies beyond PHI tagging: if masks saturate entity representations, named entity recognition and entity linking on clinical text may show similar effects, which the paper only conjectures.
  • The error analysis points to a concrete improvement: pseudo generators that respect surface forms (e.g., '40's' rather than '40') for AGE could remove one identified source of pseudo-model false positives.
  • A broader implication is that model families differ in how they absorb pseudo text: multilingual models may be more robust to low-entropy replacements, so future work should select pseudo-data strategies per base model rather than one-size-fits-all.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper asks whether continued pretraining of encoder-only language models on redacted clinical text is harmful for downstream protected health information (PHI) de-identification, and whether replacing redaction placeholders with realistic pseudo-data helps. The authors construct masked and pseudo versions of MIMIC-III notes, continue training RoBERTa and XLM-RoBERTa in base and large sizes on each version, then fine-tune all checkpoints on the 2014 i2b2/UTHealth corpus and compare with published baselines. They report token-level precision, recall, and F1. Their main empirical findings are that their trained checkpoints outperform previously published de-identification models, that the masked RoBERTa large model performs best overall, that pseudo-data improves XLM-R models, and that masked and pseudo models differ only slightly.

Significance. The paper contributes a reusable pseudo-data generation pipeline, a set of pretrained clinical checkpoints, and open-source code, which are valuable community resources. The evaluation on an external benchmark with published baselines is a strength. However, the central masked-versus-pseudo comparison is confounded by the hyperparameter tuning procedure, and the headline comparisons to earlier baselines mix different averaging schemes (weighted vs. micro F1). If these issues are addressed, the work would make a credible contribution to clinical NLP; as written, the 'surprising finding' and the claimed 1.5-point improvement over the best baseline are not supported by the evidence presented.

major comments (3)
  1. [Section 4.1.3] The pretraining hyperparameters (learning rate 2e-5, maximum sequence length 1024, batch sizes in Table 3, six epochs) were tuned on the masked RoBERTa checkpoint alone and then fixed for all other pretrained embedding models. This means every pseudo-data model and every XLM-R model was trained with a schedule selected to maximize masked RoBERTa performance, not with a schedule chosen for that model or dataset. The central masked-versus-pseudo comparisons in Table 7 (models 10 vs. 11, 12 vs. 13, 14 vs. 15, 16 vs. 17) and the paper's headline finding that masked RoBERTa large beats pseudo RoBERTa large are therefore potentially artifacts of hyperparameter selection. Please provide per-model or at least per-model-family hyperparameter tuning, or otherwise demonstrate that the reported conclusions are robust to the choice of pretraining schedule.
  2. [Section 6, Table 7] The claim that models 10-17 'perform significantly better than all previous baselines' is not supported as stated. For the newly trained models, the first three metric columns are weighted P/R/F1, while for the previously published baselines (models 1-5) the corresponding columns appear to be micro-averaged P/R/F1. On micro F1, model 12 ties baseline model 5 at 98.4, and several of models 10-17 have micro F1 below 98.4 (e.g., 96.88 for model 14). Consequently, the sentence 'Our highest performing model (12) surpasses the best baseline model (5) by 1.5 F1 points' is contradicted by the numbers in the table. Please report comparable metrics across all models and reconcile or correct this claim.
  3. [Section 6, Table 7] All fine-tuned results appear to be single runs, and the statistical claims are only described textually. The McNemar and ANOVA p-values for the 'remaining models' lack details about which model pairs were compared, whether the comparisons were corrected for multiple testing, and how the tests were computed on a single run. Given that the masked-vs-pseudo differences are often very small (e.g., models 12 and 13 differ by 0.015 weighted F1), the current evidence does not establish that these differences are stable. Please report multiple seeds with confidence intervals or standard deviations, and provide a more complete description of the statistical tests.
minor comments (6)
  1. [Table 1 and Table 2] There are several typos: 'pseduo' in Table 1, 'hosptial' in Table 2, 'contanct' in Table 2, and 'W ARDNAME' in Table 2 should be 'WARDNAME'.
  2. [Section 4.1.3] The text says 'the mini-batch sizes 4 in Table 3', but Table 3 lists batch sizes 20 for base models and 5 for large models. Please correct this inconsistency.
  3. [Section 6, Table 5] The fill-mask examples in Table 5 are illustrative but are not evaluated quantitatively. If possible, report a small quantitative measure (e.g., top-1 accuracy on held-out masked tokens) to support the characterization of masked versus pseudo embeddings.
  4. [Section 7 and Appendix A] The phrase 'models 13, 14 in Appendix A Figure 6' is inconsistent with the numbering in Table 7, where model 13 is pseudo RoBERTa large and model 14 is masked XLM-R base. Please clarify which figure and models are meant.
  5. [Appendix C] The stated limitation that 'other combinations of embeddings and fine-tuned tasks could yield different results' is too vague and does not address the hyperparameter-selection bias identified in Section 4.1.3. Please update the limitations accordingly.
  6. [Section 4] The pseudo-data generation uses GatorTron to fill in ambiguous masked tokens. Since GatorTron is itself a clinical model, a brief discussion of whether this introduces circularity or distributional bias into the pseudo dataset would improve the methodology section.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation; minor non-load-bearing self-citation only.

full rationale

The paper's central claims are judged on the external 2014 i2b2/UTHealth de-identification benchmark against published baselines (models 1-5) and independently fine-tuned off-the-shelf embeddings (models 6-9). The masked and pseudo checkpoints are produced by continued pretraining on MIMIC-III derivatives, and the downstream F1 scores are computed on held-out i2b2 test notes, not on the pretraining data. The pseudo-data construction uses external gazetteers and, for ambiguous tags, the GatorTron clinical model; GatorTron is not an output of this paper, and its use does not by construction determine the reported F1 deltas. The one author-affiliated citation is the DeepZensols framework (Landes et al., 2023) used for fine-tuning tooling; it is not load-bearing for the scientific conclusion. Section 4.1.3 does describe tuning pretraining hyperparameters on the masked RoBERTa checkpoint and then fixing them for all models; this is a genuine experimental confound for the masked-versus-pseudo comparisons, but it is a model-selection fairness issue, not a circularity: no reported quantity is defined as the fitted value, and the evaluation remains external. The limitation in Appendix C explicitly acknowledges the narrowness of the compared configurations. Accordingly, there is no self-definitional reduction, no fitted parameter renamed as a prediction, and no load-bearing self-citation chain.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The paper is an empirical ML study and introduces no new physical or mathematical postulates. Its central comparisons rest on domain assumptions about pseudo-data realism, benchmark validity, and the fairness of fixed hyperparameters. The most consequential free choice is the pretraining hyperparameter set tuned on the masked model and reused for all models.

free parameters (1)
  • Pretraining hyperparameter set (learning rate 2e-5, max sequence length 1024, batch size 4 or 5, six epochs) = lr=2e-5, seq_len=1024, batch=4 or 5, epochs=6
    Chosen by training four embedding models and fine-tuning each, then selecting the set with the best downstream validation performance; fixed for all masked and pseudo checkpoints (Section 4.1.3). This choice is a tuned input that the central cross-model comparisons inherit.
assumptions (3)
  • domain assumption MIMIC-III's masked tokens can be mapped to PHI categories via regular expressions, and each category has a plausible pseudo generator.
    The pseudo dataset construction in Section 4.1.2 assumes this mapping covers most masks and that generated text is realistic enough to serve as training signal; failures such as plural ages and complex descriptors are acknowledged in Section 7.
  • domain assumption The 2014 i2b2/UTHealth corpus, recategorized to HIPAA labels, is a valid proxy for general clinical de-identification performance.
    Used for all fine-tuning and evaluation in Section 5; the limitations appendix notes other task and embedding combinations could yield different results.
  • domain assumption GatorTron's fill-mask predictions are suitable substitutes for ambiguous masked tokens in the pseudo corpus.
    Used for NAME and other tags in Section 4.1.2; GatorTron is itself a clinical model and may propagate biases from its own deidentified training text.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Clinical Models with Pseudo Data for De-identification." pith.science (2026). https://pith.science/paper/7JIUI4ZN

@misc{pith2026250612674,
  author       = {Pith},
  title        = {Pith review of: Enhancing Clinical Models with Pseudo Data for De-identification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7JIUI4ZN}},
  note         = {Machine review of arXiv:2506.12674}
}
read the original abstract

Many models are pretrained on redacted text for privacy reasons. Clinical foundation models are often trained on de-identified text, which uses special syntax (masked) text in place of protected health information. Even though these models have increased in popularity, there has been little effort in understanding the effects of training them on redacted text. In this work, we pretrain several encoder-only models on a dataset that contains redacted text and a version with replaced realistic pseudo text. We then fine-tuned models for the protected health information de-identification task and show how our methods significantly outperform previous baselines. The contributions of this work include: a) our novel, and yet surprising findings with training recommendations, b) redacted text replacements used to produce the pseudo dataset, c) pretrained embeddings and fine-tuned task specific models, and d) freely available pseudo training dataset generation and model source code used in our experiments.

Figures

Figures reproduced from arXiv: 2506.12674 by the authors.

Figure 1
Figure 1. From top to bottom: a) a first name is sampled [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Lineage of the RoBERTa base model colored [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The loss of the fine-tuned de-identification [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Confusion matrices of the RoBERTa Large fine-tuned models on HIPAA re-categorized labels. Top (a): masked. Bottom (b): pseudo [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Confusion matrices of HIPAA i2b2 labels for RoBERTa base masked and pseudo models. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Confusion matrices of HIPAA i2b2 labels for RoBERTa large masked and pseudo models. [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Confusion matrices of HIPAA i2b2 labels for XLM-RoBERTa base masked and pseudo models. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Confusion matrices of HIPAA i2b2 labels for XLM-RoBERTa large masked and pseudo models. [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 17 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Monica Agrawal, Stefan Hegselmann, Hunter Lang, Yoon Kim, and David Sontag. 2022. https://aclanthology.org/2022.emnlp-main.130 Large language models are few-shot clinical information extractors . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages 1998--2022. Association for Computational Linguistics

    work page 2022

  4. [4]

    Emily Alsentzer, John Murphy, William Boag, Wei-Hung Weng, Di Jindi, Tristan Naumann, and Matthew McDermott. 2019. https://doi.org/10.18653/v1/W19-1909 Publicly Available Clinical BERT Embeddings . In Proceedings of the 2nd Clinical Natural Language Processing Workshop , pages 72--78. Association for Computational Linguistics

    doi:10.18653/v1/w19-1909 2019

  5. [5]

    Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy Lillicrap, Angeliki Lazaridou, Orhan Firat, and 24 others. 2024. https://doi.org/10.48550/arXiv....

    arXiv doi:10.48550/arxiv.2312.11805 2024

  6. [6]

    Malaikannan Sankarasubbu Ankit Pal. 2024. https://huggingface.co/aaditya/OpenBioLLM-Llama3-70B OpenBioLLMs : Advancing open-source large language models for healthcare and life sciences

    work page 2024

  7. [7]

    Yuhan Chen, Nuwa Xi, Yanrui Du, Haochun Wang, Jianyu Chen, Sendong Zhao, and Bing Qin. 2024. https://doi.org/10.1609/aaai.v38i20.30198 From Artificially Real to Real : Leveraging Pseudo Data from Large Language Models for Low-Resource Molecule Discovery . Proceedings of the AAAI Conference on Artificial Intelligence, 38(20):21958--21966

    work page doi:10.1609/aaai.v38i20.30198 2024

  8. [8]

    Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. https://doi.org/10.18653/v1/2020.acl-main.747 Unsupervised Cross-lingual Representation Learning at Scale . In Proceedings of the 58th Annual Meeting of the Association for Computat...

    doi:10.18653/v1/2020.acl-main.747 2020

Show all 39 references
  1. [9]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. https://doi.org/10.48550/arXiv...

    2025 doi arXiv

  2. [10]

    Franck Dernoncourt, Ji Young Lee, Ozlem Uzuner, and Peter Szolovits. 2017. https://doi.org/10.1093/jamia/ocw156 De-identification of patient notes with recurrent neural networks . Journal of the American Medical Informatics Association: JAMIA, 24(3):596--606

    2017 doi

  3. [11]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding . In Proceedings of the 2019 Conference of the North American Chapter of the Associati...

    2019 doi

  4. [12]

    Louis Ehwerhemuepha, Kimberly Carlson, Ryan Moog, Ben Bondurant, Cheryl Akridge, Tatiana Moreno, Gary Gasperino, and William Feaster. 2022. https://doi.org/10.1016/j.dib.2022.108120 Cerner real-world data ( CRWD ) - A de-identified multicenter electronic health records databas...

    2022

  5. [13]

    Donald F Gleason. 1992. Histologic grading of prostate cancer: A perspective. Human pathology, 23(3):273--279

    1992

  6. [14]

    Haoqiang Guo, Sendong Zhao, Haochun Wang, Yanrui Du, and Bing Qin. 2024. https://doi.org/10.1609/aaai.v38i16.29772 MolTailor : Tailoring Chemical Molecular Representation to Specific Tasks via Text Prompts . Proceedings of the AAAI Conference on Artificial Intelligence, 38(16)...

    2024 doi

  7. [15]

    Paul Hager, Friederike Jungmann, Robbie Holland, Kunal Bhagat, Inga Hubrecht, Manuel Knauer, Jakob Vielhauer, Marcus Makowski, Rickmer Braren, Georgios Kaissis, and Daniel Rueckert. 2024. https://doi.org/10.1038/s41591-024-03097-1 Evaluation and mitigation of the limitations o...

    2024 doi

  8. [16]

    Tzvika Hartman, Michael D. Howell, Jeff Dean, Shlomo Hoory, Ronit Slyper, Itay Laish, Oren Gilon, Danny Vainstein, Greg Corrado, Katherine Chou, Ming Jack Po, Jutta Williams, Scott Ellis, Gavin Bee, Avinatan Hassidim, Rony Amira, Genady Beryozkin, Idan Szpektor, and Yossi Mati...

    2020 doi

  9. [17]

    Keloth, Vincent J

    Yan Hu, Xu Zuo, Yujia Zhou, Xueqing Peng, Jimin Huang, Vipina K. Keloth, Vincent J. Zhang, Ruey-Ling Weng, Qingyu Chen, Xiaoqian Jiang, Kirk E. Roberts, and Hua Xu. 2024. https://doi.org/10.48550/arXiv.2411.10020 Information Extraction from Clinical Notes : Are We Ready to Swi...

    2024 doi arXiv

  10. [18]

    Kexin Huang, Jaan Altosaar, and Rajesh Ranganath. 2020. https://arxiv.org/abs/1904.05342v3 ClinicalBERT : Modeling Clinical Notes and Predicting Hospital Readmission . In Proceedings of the ACM Conference on Health , Inference , and Learning . ACM

    2020 arXiv

  11. [19]

    Alistair E. W. Johnson, Lucas Bulgarelli, and Tom J. Pollard. 2020. https://doi.org/10.1145/3368555.3384455 Deidentification of free-text medical records using pre-trained bidirectional transformers . In Proceedings of the ACM Conference on Health , Inference , and Learning , ...

    2020

  12. [20]

    Alistair E. W. Johnson, Tom J. Pollard, Lu Shen, Li-wei H. Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G. Mark. 2016. https://doi.org/10.1038/sdata.2016.35 MIMIC-III , a freely accessible critical care database . Scien...

    2016 doi

  13. [21]

    David Kartchner, Selvi Ramalingam, Irfan Al-Hussaini, Olivia Kronick, and Cassie Mitchell. 2023. https://aclanthology.org/2023.bionlp-1.37 Zero- Shot Information Extraction for Clinical Meta-Analysis using Large Language Models . In The 22nd Workshop on Biomedical Natural Lang...

    2023

  14. [22]

    Paul Landes, Barbara Di Eugenio, and Cornelia Caragea. 2023. https://aclanthology.org/2023.nlposs-1.16 DeepZensols : A Deep Learning Natural Language Processing Framework for Experimentation and Reproducibility . In Proceedings of the 3rd Workshop for Natural Language Processi...

    2023

  15. [23]

    Smith, Zachary Ziegler, Daniel Nadler, Peter Szolovits, Alistair Johnson, and Emily Alsentzer

    Eric Lehman, Evan Hernandez, Diwakar Mahajan, Jonas Wulff, Micah J. Smith, Zachary Ziegler, Daniel Nadler, Peter Szolovits, Alistair Johnson, and Emily Alsentzer. 2023. https://doi.org/10.48550/arXiv.2302.08091 Do We Still Need Clinical Language Models ? arXiv: 2302.08091 (Onl...

    2023 doi arXiv

  16. [24]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. https://arxiv.org/abs/1907.11692 RoBERTa : A Robustly Optimized BERT Pretraining Approach . arXiv: 1907.11692 (Only available as arXiv ...

    2019 arXiv

  17. [25]

    Zengjian Liu, Yangxin Chen, Buzhou Tang, Xiaolong Wang, Qingcai Chen, Haodi Li, Jingfeng Wang, Qiwen Deng, and Suisong Zhu. 2015. https://doi.org/10.1016/j.jbi.2015.06.009 Automatic de-identification of electronic medical records using token-level and character-level condition...

    2015 doi

  18. [26]

    Zengjian Liu, Buzhou Tang, Xiaolong Wang, and Qingcai Chen. 2017. https://doi.org/10.1016/j.jbi.2017.05.023 De-identification of clinical notes via recurrent neural network and conditional random field . Journal of Biomedical Informatics, 75S:S34--S42

    2017 doi

  19. [27]

    Quinn McNemar. 1947. https://doi.org/10.1007/BF02295996 Note on the sampling error of the difference between correlated proportions or percentages . Psychometrika, 12(2):153--157

    1947 doi

  20. [28]

    Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013 a . https://arxiv.org/abs/1301.3781 Efficient estimation of word representations in vector space . arXiv: 1301.3781 (Only available as arXiv preprint). Preprint, arXiv:1301.3781

    2013 arXiv

  21. [29]

    Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013 b . https://proceedings.neurips.cc/paper/2013/hash/9aa42b31882ec039965f3c4923ce901b-Abstract.html Distributed Representations of Words and Phrases and their Compositionality . In Advances in Neural In...

    2013

  22. [30]

    Mark Neumann, Daniel King, Iz Beltagy, and Waleed Ammar. 2019. https://doi.org/10.18653/v1/W19-5034 ScispaCy : Fast and Robust Models for Biomedical Natural Language Processing . In Proceedings of the 18th BioNLP Workshop and Shared Task , pages 319--327. Association for Compu...

    2019 doi

  23. [31]

    Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. http://www.aclweb.org/anthology/D14-1162 Glove: Global Vectors for Word Representation . In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing ( EMNLP ) , pages 1532--1543. A...

    2014

  24. [32]

    Konstantinos Sechidis, Grigorios Tsoumakas, and Ioannis Vlahavas. 2011. https://doi.org/10.1007/978-3-642-23808-6_10 On the Stratification of Multi-label Data . In Dimitrios Gunopulos, Thomas Hofmann, Donato Malerba, and Michalis Vazirgiannis, editors, Machine Learning and Kno...

    2011 doi

  25. [33]

    Amber Stubbs and "Ozlem Uzuner. 2015. https://doi.org/10.1016/j.jbi.2015.07.020 Annotating longitudinal clinical narratives for de-identification: The 2014 i2b2/ UTHealth corpus . Journal of Biomedical Informatics, 58:S20--S29

    2015 doi

  26. [34]

    Haochen Tan, Wei Shao, Han Wu, Ke Yang, and Linqi Song. 2022. https://doi.org/10.18653/v1/2022.findings-acl.22 A Sentence is Worth 128 Pseudo Tokens : A Semantic-Aware Contrastive Learning Framework for Sentence Embeddings . In Findings of the Association for Computational Lin...

    2022 doi

  27. [35]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. https://doi.org/10.48550/arXiv.2302.139...

    2023 doi arXiv

  28. [36]

    Hanyin Wang, Chufan Gao, Christopher Dantona, Bryan Hull, and Jimeng Sun. 2024. https://doi.org/10.1038/s41746-023-00989-3 DRG-LLaMA : Tuning LLaMA model to predict diagnosis-related group for hospitalized patients . npj Digital Medicine, 7(1):1--9

    2024 doi

  29. [37]

    Pfeffer, Jason Fries, and Nigam H

    Michael Wornow, Yizhe Xu, Rahul Thapa, Birju Patel, Ethan Steinberg, Scott Fleming, Michael A. Pfeffer, Jason Fries, and Nigam H. Shah. 2023. https://doi.org/10.48550/arXiv.2303.12961 The Shaky Foundations of Clinical Foundation Models : A Survey of Large Language Models and F...

    2023 doi arXiv

  30. [38]

    Smith, Christopher Parisien, Colin Compas, Cheryl Martin, Mona G

    Xi Yang, Aokun Chen, Nima PourNejatian, Hoo Chang Shin, Kaleb E. Smith, Christopher Parisien, Colin Compas, Cheryl Martin, Mona G. Flores, Ying Zhang, Tanja Magoc, Christopher A. Harle, Gloria Lipori, Duane A. Mitchell, William R. Hogan, Elizabeth A. Shenkman, Jiang Bian, and ...

    2022 doi arXiv

  31. [39]

    Yuki Zenimoto, Ryo Hasegawa, Takehito Utsuro, Masaharu Yoshioka, and Noriko Kando. 2024. https://doi.org/10.18653/v1/2024.naacl-srw.26 Coding Open-Ended Responses using Pseudo Response Generation by Large Language Models . In Proceedings of the 2024 Conference of the North Ame...

    2024 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.