Breaking the Stigma! Unobtrusively Probe Symptoms in Depression Disorder Diagnosis Dialogue

Chen Huang; Jieming Cao; Jincheng Zhang; Ruibo Deng; Wenqiang Lei; Yanan Zhang

REVIEW 3 major objections 4 minor 17 references

Breaking the Stigma! Unobtrusively Probe Symptoms in Depression Disorder Diagnosis Dialogue

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a dialogue system can reduce stigma-driven nondisclosure in depression screening by probing symptoms indirectly, and that its UPSD4 method shows measurable diagnostic gains in simulated conversations.

desk verdict A sensible strategy-based probing system for depression dialogue, but its diagnostic advantage is baked into a user simulator that is told to refuse direct questions. read the letter →

arxiv 2501.15260 v1 pith:ANBKLD4P submitted 2025-01-25 cs.CL cs.CY

Jieming Cao , Chen Huang , Yanan Zhang , Ruibo Deng , Jincheng Zhang , Wenqiang Lei This is my paper · ORCID

classification cs.CLcs.CY

keywords depressiondiagnosisdialoguestigmaunobtrusiveprobinglargelanguagemodelsempatheticresponsegenerationICD-11slotfillingusersimulation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Stigma keeps people from admitting depression symptoms, and this paper argues that automated diagnosis will stall unless dialogue systems learn to ask indirectly. To that end it introduces UPSD4, a two-module system that couples unobtrusive probing strategies with an ICD-11-based slot-filling diagnosis module. Evaluated against user simulators built from a depression-diagnosis dialogue benchmark, UPSD4 outperforms existing chatbots on dialogue quality and on diagnostic accuracy and completion, with the largest gains when the simulated user holds stigma beliefs. The takeaway is that how a screening system asks may matter as much as what it asks.

What carries the argument

The load-bearing component is the strategy set itself: nine fine-grained strategies organised under Discreet Probing, Empathy, and Flow Management, selected through a two-stage coarse-then-fine prompt because LLMs choose more reliably from small candidate sets. Each strategy translates a known survey- or psychotherapy-research technique into an instruction for response generation, and the selection loop is driven by slot states from the Conversational Diagnosis Module, so topic choice and questioning style are jointly determined. The same nine ICD-11 slots also serve as the diagnostic output structure, making the dialogue record directly interpretable.

What would settle it

Run UPSD4 against direct questioning in a controlled study with consenting adults who score high on the Depression Stigma Scale, measuring how many of the nine ICD-11 symptoms they disclose and whether a full diagnosis is reached; if disclosure is equal under both questioning styles, the claimed mechanism does not generalize.

Watch

Extended reading notes

Core claim

The paper's central claim is that unobtrusive probing is a learnable, promptable behavior, and that encoding it as explicit dialogue strategies is what makes an LLM-based depression screener work under stigma. UPSD4's Unobtrusive Probing Module selects a coarse strategy (Discreet Probing, Empathy, or Flow Management) and then a fine-grained one (e.g., Loading Question, Nominative Technique, Forgiving Question, Clarification) before generating each response. The Conversational Diagnosis Module tracks nine ICD-11 depression criteria as slots, updates them from the dialogue, chooses the next criterion to probe, and produces a diagnosis when all slots are filled. On the D4 benchmark with a with-stigma user simulator, the paper reports average improvements of 7.06% in Accuracy and 8.66% in Diagnosis Rate over the best baseline, and a shift toward gentler questioning strategies when the user shows reluctance.

Load-bearing premise

The main result depends on the assumption that the with-stigma user simulator's rule-based tendency to become secretive and refuse when asked directly behaves like real people with depression stigma.

Editorial extensions

If this is right

Prompt-level strategy sets can change whether an LLM asks a sensitive question bluntly or indirectly; the paper's ablation shows that simply instructing the model to be unobtrusive, without concrete strategies, produces weaker results.
Because every method in the comparison shares the same diagnosis module, the reported gains in accuracy and diagnosis rate are attributed to the dialogue itself, meaning questioning style directly affects screening completeness.
The system shifts from Loading Questions toward Forgiving Questions when interlocutors show reluctance, which is evidence that an LLM can adapt its probing style to perceived stigma sensitivity.
The nine ICD-11 slots used for diagnosis are reusable scaffolding for other LLM-based mental-health screeners, independent of the specific probing prompts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the same pattern holds with real patients, the strategy taxonomy could be turned into a training curriculum for human interviewers, since it turns survey-response psychology into concrete phrasings.
The method should transfer to other stigmatized health conditions, such as substance use, suicidal ideation, or HIV status, where direct questioning is known to suppress disclosure.
The measured margin is against a simulator whose secrecy is explicitly rule-prompted, so the size of the real-world benefit remains an open empirical question rather than a settled result.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the paper proposes UPSD4, a two-module system that uses psychology-derived probing strategies (discreet questioning, flow management, empathy) to elicit depression symptoms without asking blunt questions. That's a real and reasonable contribution. But the central evidence that it improves diagnosis with stigma-affected users rests on a simulator that is explicitly instructed to be secretive or refuse when it thinks the psychologist's response will cause stigma. That instruction almost guarantees the indirect approach will look better.

What's genuinely new: the strategy taxonomy is borrowed from established survey research, yes, but applying it as a prompt-selection method for LLM-based diagnosis is new in this literature. The architecture (UPM deciding what to ask and how, CDM slot-filling ICD-11 criteria) is clean and reproducible. The human evaluation is decent—five psychology-trained annotators, 240 paired dialogues, kappa 0.731—and it does show that judges prefer UPSD4's dialogue quality. That's a real signal, independent of the simulator.

The soft spots are in the evaluation loop. The with-stigma simulator prompt (Table 13) is hard-coded to refuse when it perceives stigma. Direct questions—'have you had thoughts of harming yourself?'—trivially trigger that rule, so the Dx Rate gap (59.85% vs 38.64%) is at least partly an artifact of instruction-following, not human-like disclosure. On top of that, the automatic scores come from GPT-4, the same model family that generates the responses and the simulator's reactions; it can't be called an independent judge. The human evaluators rated the dialogue, not whether a genuinely stigma-affected person would open up more to indirect probes. The limitations section acknowledges that prompts influence results, but that point isn't applied to the simulator itself.

None of this makes the paper unserious. The strategy decomposition and the generation behavior are worth having, and the authors are honest about using simulators. But the phrase 'enhancing depression diagnosis' is not yet supported for real users; it's supported for a GPT-4 simulator told to distrust direct questions.

For whom: people building mental-health chatbots and anyone working on LLM evaluation should read it. I'd send it to review, with major revision required. The authors should either run a small real-user study or change the simulator to ground refusal behavior in empirical stigma research—and use a different judge, or at least a judge that isn't the generator.

I'd bring it to a reading group; the evaluator-simulator circularity is a good teaching case.

Referee Report

3 major / 4 minor

Summary. The paper introduces UPSD4, a two-module framework for depression-diagnosis-oriented dialogue that aims to probe symptoms unobtrusively to mitigate stigma. The Unobtrusive Probing Module (UPM) selects among psychologically grounded questioning strategies (discreet probing, empathy, flow management) to guide response generation, while the Conversational Diagnosis Module (CDM) performs slot-filling against ICD-11 criteria and issues a diagnosis once all slots are filled. The method is evaluated on the D4 dataset using GPT-4-based user simulators with and without stigma beliefs, with automatic GPT-4 judgments of dialogue quality, a human evaluation by psychology-trained annotators, and diagnostic accuracy metrics. The central reported result is that UPSD4 outperforms baselines in dialogue unobtrusiveness and in diagnosis rate, particularly for the with-stigma simulator.

Significance. If the empirical claims held, UPSD4 would be a practically valuable contribution to mental-health dialogue systems: it offers a concrete, theory-grounded strategy taxonomy, a two-stage strategy selection mechanism, and a reproducible evaluation protocol (temperature 0, seed 42, fully specified prompts). The human evaluation with five psychology-trained annotators and Fleiss' kappa of 0.731 is a genuine strength, as is the careful reporting of prompts and simulator details. However, the central diagnostic claim rests on the behavior of a user simulator that is explicitly instructed to refuse questions it perceives as stigma-inducing, and on GPT-4 judging the output of a GPT-4-based system, so the significance for real-world deployment is not yet established.

major comments (3)

[Section 5.1, Appendix C.1, Table 13] The with-stigma user simulator is instructed to 'be secretive to respond when you think the psychologist’s response will cause stigma' and to 'refuse to answer the question when you think the psychologist’s response will cause stigma.' Since the simulator is GPT-4, the same model family as the UPSD4 backbone and the automatic judge, this rule likely determines the Diagnosis Rate gap: direct probes such as 'Have you had thoughts about harming yourself?' are almost trivially identified as stigma-causing, triggering refusal, while the indirect UPSD4 probes avoid the trigger. This is compounded by the Dx Rate definition in Appendix C.3, which counts a diagnosis as unsuccessful whenever any slot is refused or unfilled. The paper should provide evidence that this refusal behavior is a faithful model of real human stigma responses, for example by varying the refusal rule in the simulator, comparing simulator disclosures to human responses, or validating with human participants who actually answer the questions.
[Section 5.1, Evaluation Metrics] The automatic unobtrusiveness scores (Discreetness, Empathy, Coherence, Fluency) are produced by GPT-4, the same model family that generates the system responses and simulates the users. The G-Eval-style self-evaluation may therefore reflect GPT-4's own preferences about what counts as discreet rather than human perception. The human evaluation in Section 5.3 and Table 4 provides complementary evidence, but it only compares dialogue quality between methods; it does not measure whether real stigma-affected users would disclose more to indirect probing. Please add a human evaluation of disclosure behavior, or at least a judge-model swap and an analysis of judge bias, to support the claim that UPSD4 'enhances depression diagnosis' through better disclosure.
[Table 2, With-Stigma rows] The reported improvement of 7.06% in Accuracy and 8.66% in Diagnosis Rate for with-stigma users is computed relative to the best baseline, but the baselines (Vanilla GPT-4, Vanilla Qwen2) are not given any unobtrusive-probing instruction, so they are likely to ask direct questions. Combined with the simulator's hard-coded refusal rule, this makes the improvement partly an artifact of the evaluation setup rather than of UPSD4's questioning skills. The authors should either relax the refusal rule, measure the sensitivity of the Dx Rate to the refusal probability, or otherwise demonstrate that the advantage persists under less contrived simulator conditions.

minor comments (4)

[Section 3, Task Formalization] The notation 'Ht = {(uu_i, us_i)}' appears to be a typographical error; it should likely be 'u^u_i' and 'u^s_i' to distinguish user and system utterances.
[Figure 3] The figure contains the typo 'Dpression Mood' and the abbreviation 'USPD4' in the right panel; the rest of the paper uses 'UPSD4'.
[Table 2] The column headers 'Disc', 'Empth', 'Cohr', and 'Fluen' are not expanded in the caption; please define them or use the full metric names for accessibility.
[Section 5.1, Dataset section] The statement that D4 is 'the only currently available benchmark specifically designed for depression disorder diagnosis dialogue' should include a qualification such as 'to the best of our knowledge' or a citation, especially since the authors also cite newer related datasets.

Circularity Check

2 steps flagged · score 6.0 of 10

With-stigma simulator's programmed refusal rule largely predetermines the reported Diagnosis Rate gain; automatic unobtrusiveness scores also encode the paper's own strategy definitions.

self definitional [Appendix C.4, Table 13 (With-Stigma User Simulator Prompt); Section 5.2 Main Results, Table 2]
"You should be *secretive* to respond when you think the psychologist's response will cause *stigma*. You should *refuse* to answer the question when you think the psychologist's response will cause *stigma*. ... When it comes to user simulators with stigma beliefs, UPSD4 achieves an average improvement of 7.06% in Accuracy and 8.66% in Diagnosis Rate, compared to the best baseline."

The with-stigma simulator is explicitly instructed to refuse or be secretive whenever it judges the psychologist's response to be stigmatizing. The reported Diagnosis Rate improvement (e.g., 59.85% vs. 38.64% against Vanilla GPT-4) therefore measures whether the system avoids utterances that the simulator itself — a GPT-4 model with the same family as the method and the evaluator — classifies as stigma-triggering. That is, the outcome metric is operationally defined by the simulator's rule, not by independent evidence about how real stigma-affected users would respond. The central claim that unobtrusive probing 'enhances depression diagnosis' is thus partly built into the test harness.
self definitional [Appendix B.2, Table 10 (Discreetness fine-grained explanation); Table 9 (GPT-4 evaluation prompt)]
"The psychologist *skillfully* uses personal anecdotes and hints to guide the conversation, avoiding direct probes and making the inquirer comfortable."

The automatic unobtrusiveness metric rewards the exact behaviors that UPSD4's probing strategies are designed to produce: indirection, anecdotes, hints, and avoiding direct probes. The GPT-4 judge is given this rubric, and UPSD4 is prompted with those same strategies. High Discreetness scores therefore partially reflect the system's adherence to the authors' own prompt and rubric, rather than an independent measure of stigma reduction. The human evaluation (Table 4, Fleiss' kappa 0.731) provides some independent support for dialogue quality, but it evaluates dialogues with the same rule-based simulator and does not establish that real stigma-affected users would disclose more to indirect probing.

full rationale

The paper's headline diagnostic result is partially circular because the with-stigma user simulator — the entity that produces the disclosures used to compute Accuracy and Diagnosis Rate — is programmed with a rule that directly links perceived stigma to refusal and secrecy. UPSD4 is explicitly designed to avoid triggering that rule, so its large Dx Rate advantage over direct-question baselines is in substantial part an artifact of the simulator's instruction, not a demonstrated effect on real human disclosure. The automatic unobtrusiveness evaluation is also self-referential in a narrower sense: the Discreetness rubric is written in terms of the same indirect-questioning techniques that UPSD4 is prompted to use, so the GPT-4 judge is checking prompt alignment rather than providing an external criterion. That said, the paper includes a human evaluation with five psychology-trained annotators and good inter-annotator agreement, which independently supports the claim that UPSD4 produces more discreet, empathic, coherent, and fluent dialogue text. That human judgment does not, however, validate the central diagnostic-outcome claim for real stigma-affected users, and the Limitations section concedes that 'evaluation results can be influenced by the prompts we used.' No load-bearing self-citation chain or uniqueness-theorem import was found; the score reflects partial circularity in the main outcome metric rather than wholesale derivation of results from assumptions.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No fitted numeric parameters or invented objects are introduced. The key burdens are the domain assumptions around the simulator and the GPT-4 evaluator.

assumptions (3)

domain assumption The nine ICD-11 symptom slots designed by the authors and approved by a licensed psychologist faithfully operationalize ICD-11 diagnostic criteria.
Used as the slot set for diagnosis assessment and topic selection; if slots are invalid, the diagnosis results are invalid. See Appendix A and Section 4.2.
domain assumption GPT-4-based user simulators, especially the with-stigma simulator prompted with rules about secrecy and refusal, behave like real depression patients with stigma.
The entire evaluation rests on simulator behavior; the with-stigma prompt in Table 13 directly instructs the simulator to refuse when it perceives stigmatizing questions, which may predetermine the outcome.
domain assumption GPT-4 automatic evaluation of discreetness, empathy, coherence, and fluency measures these qualities reliably without confirming alignment with human ratings.
The paper uses GPT-4 as evaluator (Section 5.1, Table 9) and human evaluation only as win rate, not as a calibration of the automatic metric. The assumption is that the automatic scores reflect true dialogue quality.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Breaking the Stigma! Unobtrusively Probe Symptoms in Depression Disorder Diagnosis Dialogue." pith.science (2026). https://pith.science/paper/ANBKLD4P

@misc{pith2026250115260,
  author       = {Pith},
  title        = {Pith review of: Breaking the Stigma! Unobtrusively Probe Symptoms in Depression Disorder Diagnosis Dialogue},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ANBKLD4P}},
  note         = {Machine review of arXiv:2501.15260}
}

abstract

Stigma has emerged as one of the major obstacles to effectively diagnosing depression, as it prevents users from open conversations about their struggles. This requires advanced questioning skills to carefully probe the presence of specific symptoms in an unobtrusive manner. While recent efforts have been made on depression-diagnosis-oriented dialogue systems, they largely ignore this problem, ultimately hampering their practical utility. To this end, we propose a novel and effective method, UPSD$^{4}$, developing a series of strategies to promote a sense of unobtrusiveness within the dialogue system and assessing depression disorder by probing symptoms. We experimentally show that UPSD$^{4}$ demonstrates a significant improvement over current baselines, including unobtrusiveness evaluation of dialogue content and diagnostic accuracy. We believe our work contributes to developing more accessible and user-friendly tools for addressing the widespread need for depression diagnosis.

Figures

Figures reproduced from arXiv: 2501.15260 by the authors.

**Figure 2.** A taxonomy of related work in depression [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** USPD4 contains two interconnected modules: Unobtrusive Probing Module (UPM) and Conversational Diagnosis Module (CDM). The UPM, guided by our probing strategies, promotes a sense of unobtrusiveness within the dialogue system and cultivates questioning skills. Meanwhile, the CDM leverages established diagnostic criteria to evaluate potential symptoms in them. the GPT-4 with detailed instructions to select a strategy … view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: The distribution of coarse-grained strategies ( [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 14 canonical work pages

[1]

People with depression could snap out of it if they wanted

work page
[2]

Depression is a sign of personal weakness

work page
[3]

Depression is not a real medical illness

work page
[4]

People with depression are dangerous

work page
[5]

It is best to avoid people with depression so you don’t become depressed yourself

work page
[6]

People with depression are unpredictable

work page
[7]

If l had depression l would not tell anyone

work page
[8]

I would not employ someone if l knew they had been depressed

work page

Show all 17 references

[9]

Five-point Likert scale question: <SCALE_QUESTION> You must choose one of [{

I would not vote for a politician if l knew they had been depressed. Table 6: The nine questions comprising the Depression Stigma Scale. C Implementation Details All experiments were conducted on a machine with an Intel(R) Xeon(R) Gold 6348 CPU @ 2.60GHz Aspect Detail Aspect D...

2001
[15]

Flow Management

"Flow Management" when the Previous Topic and Current Topic are different
[16]

Empathetic Response

"Empathetic Response" when you decide to give comforting, feedback or guidance
[17]

Questioning Skill

"Questioning Skill" when you decide to proactively query to probe for in-depth information. The "<WHY>" should be short based on the Dialogue history. Think "<WHY>" before you set "<STRING>". Only generate the JSON structural response strictly following your psychologist perso...
[2000]

Roger Tourangeau and Ting Yan

The psychology of survey response. Roger Tourangeau and Ting Yan. 2007. Sensitive ques- tions in surveys. Psychological bulletin, 133(5):859. Ruiyi Wang, Stephanie Milani, Jamie C Chiu, Shaun M Eack, Travis Labrum, Samuel M Murphy, Nev Jones, Kate Hardy, Hong Shen, Fei Fang, e...

2007 arXiv
[2005]

i can’t talk about that

Breaking the cycle of stigmatization: manag- ing the stigma of incontinence in social interactions. Journal of Wound Ostomy & Continence Nursing , 32(1):38–52. Melissa D Grady, Jill S Levenson, George Mesias, Shel- ley Kavanagh, and Jennifer Charles. 2019. “i can’t talk about ...

2019
[2013]

Comprehensive psychiatry , 54(1):1–6

The diagnosis of depression: current and emerging methods. Comprehensive psychiatry , 54(1):1–6. Kai Sun, Seungwhan Moon, Paul Crook, Stephen Roller, Becka Silvert, Bing Liu, Zhiguang Wang, Honglei Liu, Eunjoon Cho, and Claire Cardie. 2020. Adding chit-chat to enhance task-ori...

2020 arXiv
[2021]

arXiv preprint arXiv:2109.05729

Cpt: A pre-trained unbalanced transformer for both chinese language understanding and generation. arXiv preprint arXiv:2109.05729. Jo Anne Sirey, Martha L Bruce, George S Alexopoulos, Deborah A Perlick, Patrick Raue, Steven J Friedman, and Barnett S Meyers. 2001. Perceived sti...

2001 arXiv
[2024]

arXiv preprint arXiv:2403.09717

Enhancing depression-diagnosis-oriented chat with psychological state tracking. arXiv preprint arXiv:2403.09717. Clara E Hill. 2020. Helping skills: Facilitating explo- ration, insight, and action. American Psychological Association. Chen Huang, Yang Deng, Wenqiang Lei, Jianch...

2020 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

People with depression could snap out of it if they wanted

work page

[2] [2]

Depression is a sign of personal weakness

work page

[3] [3]

Depression is not a real medical illness

work page

[4] [4]

People with depression are dangerous

work page

[5] [5]

It is best to avoid people with depression so you don’t become depressed yourself

work page

[6] [6]

People with depression are unpredictable

work page

[7] [7]

If l had depression l would not tell anyone

work page

[8] [8]

I would not employ someone if l knew they had been depressed

work page