A Topic Modeling Analysis of Stigma Dimensions, Social, and Related Behavioral Circumstances in Clinical Notes Among Patients with HIV

Jiang Bian; Krishna Vaddiparti; Mattia Prosperi; Robert L Cook; Yi Guo; Yiyang Liu; Yonghui Wu; Ziyi Chen

REVIEW 4 major objections 6 minor 4 references

A Topic Modeling Analysis of Stigma Dimensions, Social, and Related Behavioral Circumstances in Clinical Notes Among Patients with HIV

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that topic modeling of electronic health record clinical notes, restricted to sentences containing stigma keywords, uncovers eleven distinct dimensions of HIV stigma and related social and behavioral circumstances.

desk verdict A workmanlike descriptive topic-modeling study of HIV stigma in EHR notes that overreaches when it calls the pipeline a scalable stigma assessment without a validation step. read the letter →

arxiv 2506.09279 v2 pith:APPWC2KR submitted 2025-06-10 cs.LG cs.IR

Ziyi Chen , Yiyang Liu , Mattia Prosperi , Krishna Vaddiparti , Robert L Cook , Jiang Bian , Yi Guo , Yonghui Wu This is my paper

classification cs.LGcs.IR

keywords HIVstigmatopicmodelingelectronichealthrecordsclinicalnotesnaturallanguageprocessingLatentDirichletAllocationpeoplelivingwith

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that the stigma experienced by people living with HIV can be read directly from the clinical notes their providers already write, at a scale surveys cannot reach. The authors apply Latent Dirichlet Allocation to 2.9 million electronic health record notes from 9,140 patients, and show that restricting the input to sentences containing at least one of 91 stigma-related keywords surfaces eleven coherent themes, including mental health distress, treatment refusal and isolation, intimate partner violence, substance abuse, and food insecurity. The same keyword filtering on full notes or on whole notes containing keywords is much less informative, yielding about 20% and 45% stigma-related topics respectively, versus roughly 82% for sentence-level filtering. The paper concludes that this approach offers a scalable, time-efficient complement to questionnaires, and shows how the topics vary by age, with fear of falling prominent in older patients and substance abuse, intimate partner violence, and missed appointments declining with age.

What carries the argument

The machinery is a keyword-filtered sentence corpus fed to Latent Dirichlet Allocation. The keyword list is built by seeding from established stigma scales and then snowballing: clinicians review random notes, add new terms until saturation, and expand by embedding-based synonyms, ending with 91 terms. Sentences containing at least one keyword become the documents for LDA; topic coherence, similarity, and diversity are used to select the number of topics, and experts label the final clusters. The filtering changes what the model can see: full-note models are dominated by lab results, medications, and scheduling, while sentence-level models concentrate on stigma dimensions and social-behavioral circumstances.

What would settle it

Take a random sample of the same clinical notes, have clinicians mark every sentence that expresses stigma, then check how many marked sentences contain at least one of the 91 keywords; a large share of keyword-free stigma sentences would show the topic set is incomplete, while a large share of keyword-selected sentences that clinicians judge unrelated to stigma would show the model is diluted.

Watch

Extended reading notes

Core claim

The paper's central claim is that HIV-related stigma and its social and behavioral context are documented in electronic health record clinical notes often enough and consistently enough that unsupervised topic modeling can recover them. Working with a cohort of 9,140 people living with HIV and 2.9 million notes, the authors find that the unit of analysis matters: a sentence filtered by stigma keywords yields 9 of 11 topics directly about stigma and related circumstances, while topic models on complete notes mostly recover routine medical logistics. The resulting topics include mental health concerns and stigma, limited healthcare access, missed appointments, treatment refusal and isolation, intimate partner violence, fear of falling, substance abuse, and food insecurity. The paper reports no substantial male-female topic differences but clear age gradients in several topics. It argues this makes routine clinical documentation a viable data source for stigma assessment and intervention targeting.

Load-bearing premise

The whole analysis depends on the 91-term stigma keyword list being complete and unbiased enough that every sentence selected for modeling is a representative sample of how stigma is written in clinical notes; if common stigma phrases are missing or clinical language is over-represented, the topics and the age comparisons inherit that skew.

Editorial extensions

If this is right

Stigma screening can be run on existing electronic health record text without new patient questionnaires, removing response burden and recruitment cost.
Sentence-level keyword filtering is the decisive step: it raises the fraction of stigma-relevant topics from about 20% (all notes) and 45% (keyword-containing notes) to about 82%.
Age-stratified topic distributions can flag subgroups with distinct needs, such as older patients' fear of falling and physical health concerns.
The same pipeline can be ported to other stigmatized conditions by replacing the disease-specific seed keyword list.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the topic model is trained on sentences selected by keywords, the prevalence of a stigma topic is partly an artifact of how many keywords feed it; topic probabilities should be read as documentation intensity, not true stigma prevalence in the population.
If validated against structured outcome data in the same electronic health record (for example, viral suppression or antiretroviral adherence), topic proportions could serve as predictive features, turning documentation into a risk signal—a step the paper does not take.
The method's portability claim can be tested cheaply: applying the same sentence-filtering pipeline to hepatitis C or cancer notes should reproduce recognizable stigma topics if the mechanism is general.
The near-zero sex differences may reflect how clinicians document, not how stigma is experienced; a mismatch between note-derived and survey-derived stigma would identify documentation bias.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a solid LDA study applied to a large HIV cohort, and the three-strategy comparison is a useful contribution. But the abstract and conclusion oversell the result. The authors present topic prevalence from keyword-filtered sentences as an 'assessment' of stigma, with demographic comparisons, without ever testing whether those topics correspond to stigma as patients report it or as clinicians document it. The stress-test note is right: the keyword list includes generic terms like 'denies', 'feeling', 'pain', 'history', and Table 4 contains topics like 'Medical History and Screening' and 'Physical Symptoms' that are not obviously stigma. So construct validity is assumed, not shown.

What's genuinely good: the cohort phenotype is validated at 98.9% sensitivity and 97.6% specificity; the snowball keyword process is described transparently; comparing all notes, note-level filtering, and sentence-level filtering is a clean way to show that restrictive filtering surfaces more target topics. The code is public. The topics themselves are plausible and align with known issues in HIV care. The paper is honest about limitations, including note-volume skew, though it doesn't quantify the effect.

The soft spots are real but addressable. Subgroup comparisons are just average topic probabilities with no confidence intervals or significance tests, so age trends could be noise. More importantly, the phrase 'scalable, time-efficient assessment' implies the pipeline measures stigma. What it actually does is discover thematic clusters in documents that contain at least one of 91 keywords. That's a legitimate exploratory exercise, not a measurement instrument. Reframing it as characterization or theme discovery would fix most of the problem. A small validation—comparing topic prevalence against Berger scale responses in a subset, or having clinicians annotate a sample of sentences for stigma presence—would make the claim credible.

Who is this for? Health informatics researchers working on clinical NLP and social determinants, and HIV care researchers interested in what EHR notes capture about stigma. It doesn't break new methodological ground, but it's a reasonable demonstration. I'd send it to peer review with the expectation of major revision: tighten the claim, add a validation or at least a serious caveat, and run significance tests for subgroups. As written it's borderline, but the scaffolding is solid enough to merit serious refereeing.

Referee Report

4 major / 6 minor

Summary. This paper applies Latent Dirichlet Allocation to a large corpus of clinical notes from 9,140 people living with HIV at UF Health, using three input-filtering strategies to identify stigma-related topics. The authors construct a 91-term keyword list seeded from established HIV stigma scales and expanded via snowball sampling and clinical word embeddings (GatorTron), then fit LDA to all notes (Strategy 1), notes containing a keyword (Strategy 2), and sentences containing a keyword (Strategy 3). Strategy 3 yields 15 topics, later summarized to 11 after expert review, covering themes such as 'Mental Health Concerns and Stigma', 'Treatment Refusal and Isolation', 'Substance Abuse', and 'Food Insecurity and Resource Scarcity'. The paper reports word-frequency summaries and compares topic prevalence between sex and age subgroups. The central conclusion is that EHR notes can provide a scalable, time-efficient assessment of HIV stigma that overcomes limitations of traditional questionnaires.

Significance. If the central claim were validated, the paper would make a useful contribution by demonstrating that clinical narratives can be mined at scale to characterize stigma and related social/behavioral circumstances in PLWH, complementing survey-based methods. The paper has several genuine strengths: the cohort is large (9,140 patients, 2.9 million notes); the comparisons across three filtering strategies are informative and illustrate an important methodological point; the keyword list is externally grounded in established stigma scales; and the code is publicly available, which supports reproducibility. However, the paper's strongest claim — that the approach enables 'scalable, time-efficient assessment' of stigma — is currently a construct-validity assumption rather than a demonstrated result. The reported evidence supports exploratory theme discovery from clinical text, but not yet a validated stigma measurement instrument. The age/sex comparisons are descriptive and lack statistical rigor. With reframing or a validation substudy, the work would merit publication; in its present form, the headline conclusion overreaches the evidence.

major comments (4)

[Section 3.2, Table 4, and Section 4] The central conclusion that the pipeline 'enables scalable, time-efficient assessment' of HIV stigma is not supported by the evidence reported. Strategy 3 selects sentences containing at least one of 91 keywords and then fits LDA, but the resulting topics are never compared against expert-annotated stigma mentions or a validated stigma instrument. Table 4 itself contains topics such as 'Medical History and Screening' and 'Physical Symptoms and HIV Pain Management' that are not stigma dimensions, and the top words include generic terms like 'denies', 'feeling', 'pain', and 'history'. Expert post-hoc labeling of top words can name coherent themes, but it does not establish that topic prevalence is a valid measurement of stigma as experienced or documented. The authors should either add a validation step (e.g., annotation of a random sample of Strategy 3 sentences against a stigma reference standard, or comparison with a stigma scale in a subset of patients) or explicitly reframe the conclusion as exploratory theme discovery rather than stigma assessment.
[Section 3.4 and Figure 4] The sex and age subgroup comparisons average topic probabilities over documents, not over patients, and are reported without confidence intervals or significance tests. Because the number of notes per patient varies substantially — a skew the authors acknowledge in the Limitations — per-document averaging can reflect documentation volume rather than stigma prevalence. Consequently, the statements that 'no substantial difference' exists between males and females and that particular topics decline or increase with age are not statistically supported. The authors should aggregate at the patient level (e.g., average topic distribution per patient before comparing groups), include confidence intervals, and apply appropriate significance tests, or explicitly describe Figure 4 as raw descriptive patterns.
[Section 2.3] The number of topics K is selected using a composite score of topic coherence, diversity, and similarity computed on the same corpus that is used for model training, and the manuscript does not report the variability of the selected K across the 10 random seeds beyond averaged metric scores. Without reporting the composite-score curve or checking stability of the topics across neighboring K values, the reported 15-topic solution (Strategy 3) may be an artifact of the chosen hyperparameters. At minimum, the authors should report the score curve, the range of selected K over seeds, and some measure of topic stability across seeds.
[Section 2.2] The snowball keyword expansion is described as iterative review of 'a small batch of randomly sampled notes' until 'no more new keywords could be identified', but no details are given on the number of iterations, the size of each sampled batch, the number of reviewers, or inter-rater agreement. This makes the saturation claim unverifiable and is consequential because the Strategy 3 corpus — the one that produces the main results — is defined entirely by the completeness and bias of the 91-term keyword list. The authors should report the iterative process more concretely, including how saturation was operationalized and how the embedding-based synonym expansions were validated.

minor comments (6)

[Section 3.1] The sentence reporting corpus sizes — 'we identified 2,707,448 notes, 515,417 notes, and 779,702 sentences from 474,470 unique notes for Strategies 1, 2, and 3, respectively' — is malformed and should clearly associate each count with its strategy.
[Section 3.3] The word cloud is described as representing 'topics generated from the entire document'; this should say 'entire corpus' or 'dataset', since the word cloud aggregates over all documents, not a single document.
[Reference list] Reference [28] is corrupted: it reads 'Software Framework for Topic MoSoftware framework for topic modelling with large corporadelling with Large Corpora' and needs correction.
[Sections 3.2 and 3.4] The terms 'topic variance' and 'topic variation' are used inconsistently; the authors should choose one term and apply it throughout.
[Figure 4] The heatmaps in Figure 4 lack axis labels for the age groups and would benefit from reporting the sample size per subgroup so that the descriptive comparisons are interpretable.
[Section 2.2] The threshold and validation for the GatorTron embedding-based synonym expansion (cosine similarity cutoff, manual review process) are not reported, which limits reproducibility of the keyword list.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the topic-modeling pipeline is an exploratory operationalization, and the cited in-group tools (GatorTron, computable phenotype) are methodological rather than load-bearing justifications.

full rationale

The paper's derivation chain is descriptive rather than self-referential. Strategy 3 (Section 2.2, Figure 1A) selects sentences containing at least one of 91 keywords, and LDA (Section 2.3) then discovers topics from those sentences; the topic labels are manually assigned by experts and are not used to define the keyword list or to compute any fitted quantity that is later 'predicted'. The claim that nine of eleven topics are 'stigma, social, and behavioral circumstances' is an interpretive summary of emergent word clusters, not a quantity derived from the inputs by construction. The keyword seed list is grounded in established external stigma measures (Berger scale), and the snowball/embedding extension adds terms beyond the seed; this weakens, rather than creates, any self-definitional tie. The sex/age analyses (Section 3.4) are descriptive averages of document-topic probabilities, not fitted predictions validated against themselves. The in-group citations—GatorTron [26] for embeddings and the computable phenotype [25] for cohort identification—are methodological tools; the phenotype has external chart-review validation (98.9% sensitivity, 97.6% specificity), and GatorTron is used only for synonym expansion, not to justify the central result via a uniqueness or equivalence argument. The paper's own limitations (Section 4) acknowledge that topic modeling may underrepresent rare themes and that note-volume skew can affect topic distributions; those are validity concerns, not circularity. The conclusion that the pipeline 'enables scalable, time-efficient assessment' outruns the evidence because no external stigma validation is performed, but overclaiming is a correctness/construct-validity issue and does not make the derivation circular under the specified criteria.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The analysis rests on three fitted or curated elements: the number of topics K, LDA hyperparameters, and the 91-term keyword list. No new theoretical entities are introduced. The domain assumptions about note content and keyword representativeness are the main sources of uncertainty, acknowledged in the limitations.

free parameters (3)

Number of topics K = 25 (Strategies 1 and 2), 15 (Strategy 3)
Selected by maximizing the composite score of coherence, diversity, and negative similarity across K=5..30 on the same corpus. This is a fitted hyperparameter, not a priori fixed.
LDA hyperparameters (alpha, passes, iterations) = alpha='auto', passes=10, iterations=10
Alpha is inferred from data by Gensim's auto setting; passes and iterations are fixed choices that affect convergence and topic stability.
Stigma keyword list (91 terms) = 91 manually curated and snowball-expanded terms
The keyword set is a curated input that directly determines which sentences enter Strategy 3 and therefore shapes all downstream topics. It is not derived solely from the data.

assumptions (5)

standard math Bag-of-words representation and LDA generative assumptions (documents are mixtures of topics, exchangeability).
Invoked in Section 2.3 as the foundation of the topic modeling approach.
domain assumption Clinical notes contain genuine documentation of stigma and related social/behavioral circumstances.
Stated in Introduction and Section 2.1 as the rationale for mining EHR notes; if notes do not capture stigma, the whole analysis is moot.
domain assumption The computable phenotype algorithm accurately identifies PLWHs.
Section 2.1 relies on a previously validated algorithm with 98.9% sensitivity and 97.6% specificity, cited as [25].
domain assumption Sentences containing at least one keyword form a representative sample of stigma documentation.
Section 2.2 assumes that the keyword filter preserves stigma-related content without systematic bias, which is load-bearing for Strategy 3 topics.
domain assumption Manual expert labeling and merging of topics is a valid interpretation of LDA clusters.
Section 2.3 relies on two domain experts to label and merge topics; this subjective step underlies all reported topic names.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Topic Modeling Analysis of Stigma Dimensions, Social, and Related Behavioral Circumstances in Clinical Notes Among Patients with HIV." pith.science (2026). https://pith.science/paper/APPWC2KR

@misc{pith2026250609279,
  author       = {Pith},
  title        = {Pith review of: A Topic Modeling Analysis of Stigma Dimensions, Social, and Related Behavioral Circumstances in Clinical Notes Among Patients with HIV},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/APPWC2KR}},
  note         = {Machine review of arXiv:2506.09279}
}

read the original abstract

Objective: To characterize stigma dimensions, social, and related behavioral circumstances in people living with HIV(PLWHs) seeking care, using NLP methods applied to a large collection of EHR clinical notes from a large integrated health system in the southeast United States. Methods: We identified a cohort of PLWHs from the UF Health IDR and performed topic modeling analysis using Latent Dirichlet Allocation to uncover stigma-related dimensions and related social and behavioral contexts. Domain experts created a seed list of HIV-related stigma keywords, then applied a snowball strategy to review notes for additional terms until saturation was reached iteratively. To identify more target topics, we tested three keyword-based filtering strategies. The detected topics were evaluated using three widely used metrics and manually reviewed by specialists. In addition, we conducted word frequency analysis and topic variation analysis among subgroups to examine differences across age and sex-specific demographics. Results: We identified 9140 PLWHs at UF Health and collected 2.9 million clinical notes. Through the iterative keyword approach, we generated a list of 91 keywords associated with HIV-related stigma. Topic modeling on sentences containing at least one keyword uncovered a wide range of topic themes, such as "Mental Health Concern, Stigma", "Treatment Refusal, Isolation", and "Substance Abuse". Topic variation analysis across age subgroups revealed substantial differences. Conclusion: Extracting and understanding the HIV-related stigma and associated social and behavioral circumstances from EHR clinical notes enables scalable, time-efficient assessment and overcoming the limitations of traditional questionnaires. Findings from this research provide actionable insights to inform patient care and interventions to improve HIV-care outcomes.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

4 extracted references · 4 canonical work pages

[1]

Mental Health Concerns and Stigma

INTRODUCTION AND BACKGROUND The human immunodeficiency virus (HIV) is recognized as the causative agent of acquired immunodeficiency syndrome (AIDS), a global health burden. Based on the World Health Organization’s estimates, 91.4 million people have been infected with HIV, and approximately 44.1 million have died from AIDS or related complications since ...

work page 2024
[4]

Food Insecurity and Resource Scarcity

DISCUSSION AND CONCLUSION In this study, we applied topic modeling to analyze around 2.9 million clinical notes from PLWHs receiving care at a large academic medical center in Florida to discover topics of stigma dimensions, social, and related behavioral circumstances. Using an unsupervised topic modeling based on LDA, we experimented with three differen...

work page 2025
[12]

Validation of the HIV/AIDS Stigma Instrument - PLWA (HASI-P),

W. L. Holzemer et al., “Validation of the HIV/AIDS Stigma Instrument - PLWA (HASI-P),” AIDS Care, vol. 19, no. 8, pp. 1002–1012, Sept. 2007. [13] A. Malik, A. Antonino, M. L. Khan, and M. Nieminen, “Characterizing HIV discussions and engagement on Twitter,” Health Technol. (Berl.), vol. 11, no. 6, pp. 1237–1245, Nov. 2021. [14] R. K. Lomotey, S. Kumi, M. ...

work page 2007
[29]

Exploring the space of topic coherence measures,

M. Röder, A. Both, and A. Hinneburg, “Exploring the space of topic coherence measures,” in Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, Shanghai China, 2015, pp. 399–408. [30] “The distribution of the flora in the alpine zone. 1,” New Phytol, vol. 11, no. 2, pp. 37–50, 1912. [31] A. Bousso Dieng, J. R. Francisco, a...

work page 2015

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Mental Health Concerns and Stigma

INTRODUCTION AND BACKGROUND The human immunodeficiency virus (HIV) is recognized as the causative agent of acquired immunodeficiency syndrome (AIDS), a global health burden. Based on the World Health Organization’s estimates, 91.4 million people have been infected with HIV, and approximately 44.1 million have died from AIDS or related complications since ...

work page 2024

[2] [4]

Food Insecurity and Resource Scarcity

DISCUSSION AND CONCLUSION In this study, we applied topic modeling to analyze around 2.9 million clinical notes from PLWHs receiving care at a large academic medical center in Florida to discover topics of stigma dimensions, social, and related behavioral circumstances. Using an unsupervised topic modeling based on LDA, we experimented with three differen...

work page 2025

[3] [12]

Validation of the HIV/AIDS Stigma Instrument - PLWA (HASI-P),

W. L. Holzemer et al., “Validation of the HIV/AIDS Stigma Instrument - PLWA (HASI-P),” AIDS Care, vol. 19, no. 8, pp. 1002–1012, Sept. 2007. [13] A. Malik, A. Antonino, M. L. Khan, and M. Nieminen, “Characterizing HIV discussions and engagement on Twitter,” Health Technol. (Berl.), vol. 11, no. 6, pp. 1237–1245, Nov. 2021. [14] R. K. Lomotey, S. Kumi, M. ...

work page 2007

[4] [29]

Exploring the space of topic coherence measures,

M. Röder, A. Both, and A. Hinneburg, “Exploring the space of topic coherence measures,” in Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, Shanghai China, 2015, pp. 399–408. [30] “The distribution of the flora in the alpine zone. 1,” New Phytol, vol. 11, no. 2, pp. 37–50, 1912. [31] A. Bousso Dieng, J. R. Francisco, a...

work page 2015