Pith. sign in

REVIEW 3 major objections 6 minor 28 references

This paper claims that whether a health query needs an empathetic response can be predicted from the query itself, before any reply is written.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 10:29 UTC pith:2SSYULOH

load-bearing objection A solid, honestly reported benchmark for anticipatory empathy needs; the headline F1 and alignment numbers apply only to cases where the two annotators already agree, which is a real scope limit but not a fatal one. the 3 major comments →

arxiv 2601.09696 v2 pith:2SSYULOH submitted 2026-01-14 cs.CL

Empathy Applicability Modeling for General Health Queries

Shan Randhawa , Agha Ali Raza , Kentaro Toyama , Julie Hui , Mustafa Naseem This is my paper
classification cs.CL
keywords Empathy Applicability Frameworkclinical empathypatient queriesanticipatory modelinghealthcare NLPannotator agreementempathetic response generationLLM annotation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces the Empathy Applicability Framework (EAF), which sorts patient health queries into two dimensions of empathy—emotional reactions and interpretive understanding—and labels each as applicable or not, based on clinical, contextual, and linguistic cues in the query itself. The authors argue that this shifts empathy modeling from evaluating responses after the fact to anticipating needs before a response exists. To support the claim, they release a benchmark of 1,300 real patient queries labeled by human annotators and an LLM, and show that classifiers trained on these labels predict applicability with strong accuracy, outperforming heuristics and zero-shot LLM baselines. A sympathetic reader would care because it offers a concrete mechanism for making asynchronous healthcare communication more empathetic and for steering LLM-based clinical tools toward appropriate empathy.

Core claim

The central discovery is that empathy can be framed as a property of the patient's query, not just of the clinician's response. Under EAF, each query receives two binary labels: whether emotional reactions (warmth, compassion, concern) are applicable, and whether interpretations (acknowledging the patient's feelings or context) are applicable. The framework enumerates cues—severe negative emotion, inferred distress, symptom seriousness, concern for relations, expressions of feeling, distressing uncertainty, contextual hardship—and the paper demonstrates that these labels are learnable: a transformer classifier reaches macro-F1 of 0.92 for emotional reactions and 0.87 for interpretations on h

What carries the argument

The Empathy Applicability Framework (EAF) is the central object: a theory-grounded cue taxonomy that defines when each empathy dimension is Applicable or Not Applicable for a patient query. It carries the argument by turning the subjective judgment 'does this patient need empathy?' into a structured labeling decision with explicit subcategories and examples, which can be applied consistently by humans and LLMs and learned by classifiers. The framework's key move is temporal: instead of scoring empathy in a response, it analyzes the query pre-response, enabling anticipatory reasoning.

Load-bearing premise

The whole benchmark rests on two lay annotators, neither clinically trained, producing a trustworthy gold standard from moderate agreement (kappa about 0.52 for emotional reactions and 0.40 for interpretations); if their labels do not reflect what patients actually need, every downstream accuracy number inherits that error.

What would settle it

Recruit a small panel of clinicians to label the same 1,300 queries under EAF; if clinician consensus agrees with the lay-annotator consensus in fewer than, say, 70% of cases (or with kappa below 0.3), the benchmark's claim to capture patient-relevant empathy needs would be called into question.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If EAF labels are correct, AI scribes and clinical chatbots can detect empathy needs at intake, before a provider composes a reply.
  • The released 1,300-query benchmark gives researchers a shared testbed for anticipatory empathy modeling in general health queries.
  • Strong classifier performance implies that empathy applicability is not arbitrary: it has consistent linguistic and contextual signals that can be learned.
  • The documented divergence points—implicit distress, severity ambiguity, contextual hardship—argue for multi-annotator and clinician-in-the-loop calibration in any deployment.
  • EAF can complement response-generation systems by supplying an applicability signal that steers how much and what kind of empathy a generated reply should carry.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same applicability labeling could be repurposed as a routing signal, sending high-uncertainty empathy cases to human clinicians rather than automated replies.
  • Editorial inference: EAF's cue categories are likely to transfer to other asynchronous clinical channels like patient portals and email, though the paper does not test this.
  • Editorial inference: if clinician judgments diverge systematically from lay annotators, a clinician-calibrated variant could shift base rates; nothing in the paper rules this out.
  • Editorial inference: the length-applicability correlation reported in the appendix suggests that longer, context-rich queries are more likely to be flagged; this could be tested as a lightweight screening heuristic, but it is not a claim the paper makes.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces the Empathy Applicability Framework (EAF), which labels patient queries on two binary dimensions—Emotional Reactions (EA) and Interpretations (IA)—as Applicable or Not Applicable, with the aim of identifying empathy needs before a response is generated. The authors sample 9,500 queries from HealthcareMagic and iCliniq, use two lay annotators to label 1,300 queries, and use GPT-4o (five passes, majority vote) on the same 1,300 queries plus 8,000 additional queries. Human–human agreement is moderate (EA κ=0.52, IA κ=0.40), while human–GPT agreement on the human-consensus subset is substantial (EA κ=0.617, IA κ=0.652). Classifiers trained on human-consensus labels achieve EA macro-F1 0.92 and IA macro-F1 0.87, outperforming linear baselines, trivial heuristics, and zero-shot o1; models trained on GPT-only labels achieve 0.85/0.77 on the same human-consensus test set. Error analysis identifies three challenges: subjectivity in implied distress, clinical-severity ambiguity, and contextual hardship. The contributions claimed are the EAF, a 1,300-query benchmark, and an analysis of operationalization challenges.

Significance. If the empirical claims hold, the paper makes a useful contribution: it addresses an actual gap in clinical NLP by moving from post-hoc empathy scoring of responses to anticipatory labeling of patient queries, and it releases a sizeable benchmark with transparent annotation instructions, prompts, and analysis scripts. Strengths include the theory grounding in EPITOME and patient-centred care, the inclusion of classical TF–IDF baselines and a zero-shot LLM baseline, and a thoughtful treatment of annotation disagreement as interpretive signal rather than noise. However, the headline predictive and alignment numbers are restricted to the human-consensus subset and come from a single run, so the general claim that EAF identifies empathy needs across ‘general health queries’ is not yet fully supported by the evidence presented. The central idea is sound, but the validation protocol needs strengthening before the benchmark can be used as the paper advertises it.

major comments (3)
  1. [§4.5, Table 3; §4.2, Table 2] The central empirical claims—strong classifier performance and substantial human–GPT alignment—are computed only on the human-consensus subset. Table 2 shows that consensus holds for 981/1296 queries on EA and 898/1296 on IA, and the human–GPT comparison is on 820 queries where both humans agree. Section 5.2 demonstrates that the excluded cases are precisely where empathy applicability is genuinely contested (implied distress, clinical-severity ambiguity, contextual hardship). Reporting F1 and κ on this subset therefore measures only the easier cases, while the abstract and conclusion state the claims without this caveat (although §5.1 does acknowledge the exclusion). This is load-bearing because the benchmark is presented as covering general health queries. Please report full-set metrics, disagreement-aware metrics, and performance stratified by the ambiguous-case categories.
  2. [§3.2.1, §7, Table 2] The gold standard for the 1,300-query benchmark rests on two lay annotators from one country, with no clinical training, and only moderate inter-annotator agreement (EA κ=0.52, IA κ=0.40; overall 0.46). Because all classifier and alignment numbers are evaluated against these labels, the benchmark’s validity is bounded by them. The patient-perspective rationale for lay annotators is defensible, but the paper’s claim that the benchmark captures ‘what patients actually need’ is not established. The limitations section acknowledges this concern, but the interpretation of the results should reflect it more directly—for example, by labeling the gold standard as ‘lay-annotator consensus’ and by treating clinician-in-the-loop or multi-annotator validation as a prerequisite for clinical applicability claims.
  3. [§4.5, Table 3] All classifier results come from a single run, as stated in the table caption, and no confidence intervals, bootstrap estimates, or multiple seeds are reported. The McNemar tests compare the transformer against baselines, but not between model variants, and the held-out consensus test set is only 20% of an already-filtered subset. The reported EA macro-F1 0.92 and IA macro-F1 0.87 may therefore be unstable, and the claimed margin over the linear baselines is not quantified with uncertainty. Please add mean±std over multiple seeds or bootstrap CIs, and report a direct statistical comparison between the RoBERTa and linear classifiers.
minor comments (6)
  1. [§3.2.2, footnote 1] ‘More than 94% of queries received the same label on the first pass and as the majority vote’ is ambiguous; clarify whether this refers to agreement between the first pass and the final majority label.
  2. [§4.1, Table 2] The column headers ‘kappa(agree/disagree)’ are unclear; use explicit raw counts and totals (e.g., 668/820) and define what the parenthesized numbers represent.
  3. [§4.5] Report the absolute sizes of the train/validation/test splits for the Human Set; currently only percentages are given.
  4. [Appendix G, Tables 6–7] The table totals are 1,296, while the main text says 1,300; reconcile the discrepancy (e.g., four training-excluded or malformed queries).
  5. [§1] ‘Lahnala et al. attempt to solve this particular problem with with an Appraisal Framework’ contains a doubled ‘with’.
  6. [Appendix B.1] Typo: ‘contaning’ should be ‘containing’.

Circularity Check

0 steps flagged

No significant circularity: EAF validation is a standard supervised benchmark, and the consensus-subset evaluation is a disclosed scope limitation rather than a definitional reduction.

full rationale

The paper makes no formal predictive derivation; its empirical claims are standard supervised classification against human-defined labels. EAF labels are produced by annotators using a rubric derived via inductive coding (Section 3.2.1, Appendix A), and classifiers are trained on held-out splits of those labels with test sets described in Section 4.5. There is no fitted parameter renamed as a prediction. The human–GPT agreement analysis (Section 4.1) is restricted to the human-consensus subset, but the paper explicitly discloses this ('This allows us to evaluate GPT performance without confounding disagreement over error or subjectivity') and interprets the numbers as agreement on clearer cases only; this is a scope/validity limitation, not a definitional equivalence. The claim that the benchmark is 'reliable' rests on two lay annotators, acknowledged in Section 7, but that is an evidence-strength concern. No load-bearing self-citations are used; citations to prior empathy frameworks are contextual and do not substitute for the paper's own annotation and evaluation. Therefore no circular step is identifiable.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 1 invented entities

The central claim rests on assumptions about text-only inference of empathy needs, lay-annotator gold labels, binary applicability semantics, and GPT-as-annotator. No external clinical outcome or behavioral measure grounds these assumptions; the paper's own moderate inter-annotator agreement and cultural-bias findings weaken them further.

free parameters (2)
  • RoBERTa fine-tuning hyperparameters = lr=2e-5, batch=8, epochs=10
    Single setting with no reported grid search or multiple seeds; Table 3 results are a single run, so the reported margins are not shown to be robust.
  • GPT-4o majority-vote passes = 5
    Five annotation passes with majority voting; the paper reports >94% first-pass agreement but does not analyze whether 5 passes is necessary or sufficient.
axioms (5)
  • domain assumption Empathy applicability for a query can be inferred from text alone.
    The entire EAF labels patient queries without nonverbal cues, patient demographics, or clinician input; Section 1 frames this as the core modeling challenge.
  • domain assumption Two lay annotators, both from Pakistan and without clinical training, provide a valid gold standard for patient-side empathy needs.
    Section 3.2.1 justifies lay annotators as representing the patient perspective, but no evidence ties these two individuals' judgments to the broader patient population.
  • domain assumption Binary Applicable/Not Applicable labels per dimension are sufficient to capture empathy needs.
    Section 2 defines each dimension as a binary decision, collapsing graded or context-dependent need into two classes.
  • domain assumption GPT-4o majority-vote labels can serve as training labels for the 8,000-query autonomous set.
    Section 3.2.2 uses single-pass GPT labels for 8,000 queries with no human verification of that subset.
  • domain assumption The human-consensus subset (where HA1 and HA2 agree) is a representative evaluation set.
    Section 4.1 restricts human-GPT evaluation to 820 consensus queries, excluding the ambiguous cases that are the main concern of the paper.
invented entities (1)
  • Empathy Applicability Framework (EAF) dimensions no independent evidence
    purpose: Label patient queries as Applicable or Not Applicable for Emotional Reactions and Interpretations.
    The framework's validity is established only through the paper's own annotations and models; no external clinical, behavioral, or patient-outcome measure is provided.

pith-pipeline@v1.3.0-alltime-deepseek · 23887 in / 9989 out tokens · 105179 ms · 2026-08-03T10:29:48.834032+00:00 · methodology

0 comments
read the original abstract

LLMs are increasingly being integrated into clinical workflows, yet they often lack clinical empathy, an essential aspect of effective doctor-patient communication. Existing NLP frameworks focus on reactively labeling empathy in doctors' responses but offer limited support for anticipatory modeling of empathy needs, especially in general health queries. We introduce the Empathy Applicability Framework (EAF), a theory-driven approach that classifies patient queries in terms of the applicability of emotional reactions and interpretations, based on clinical, contextual, and linguistic cues. We release a benchmark of real patient queries, dual-annotated by human annotators and GPT-4o. In the subset with human consensus, we also observe substantial human-GPT alignment. To validate EAF, we train classifiers on human-labeled and GPT-only annotations to predict empathy applicability, achieving strong performance and outperforming the heuristic and zero-shot LLM baselines. Error analysis highlights persistent challenges: implicit distress, clinical-severity ambiguity, and contextual hardship, underscoring the need for multi-annotator modeling, clinician-in-the-loop calibration, and culturally diverse annotation. EAF provides a framework for identifying empathy needs before response generation, establishes a benchmark for anticipatory empathy modeling, and enables supporting empathetic communication in asynchronous healthcare.

Figures

Figures reproduced from arXiv: 2601.09696 by Agha Ali Raza, Julie Hui, Kentaro Toyama, Mustafa Naseem, Shan Randhawa.

Figure 1
Figure 1. Figure 1: UpSet plots comparing GPT and human rationales for [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Three-way divergence for every subcategory. Orange = [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Screenshot of the annotation spreadsheet provided to annotators. The header shows the instructions and [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Empathy Dimension Applicability Model Architecture [PITH_FULL_IMAGE:figures/full_fig_p018_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Dataset overview panel: base rates, EA–IA coupling, and query length distributions. Panels (a–b) report the binary label base rates for Emotional Reactions (EA) and Interpretations (IA) from Human Annotator 1 (HA1), Human Annotator 2 (HA2), and GPT. Bars are shown as stacked proportions of Applicable vs. Not Applicable. Panels (c–d) show EA×IA co-occurrence as 2×2 heatmaps for (c) Human consensus (only ite… view at source ↗
Figure 6
Figure 6. Figure 6: EA×IA subcategory co-occurrence (percent of queries, N = 1296). Humans provide a single EA and IA subcategory per query, yielding sharper pairings; GPT may assign multiple subcategories per query, producing broader co-occurrence patterns. manCons, MajorityCons). The x-axis reports the mean token length per decile (µ), and we anno￾tate the number of items per decile (n). Error bars denote confidence interva… view at source ↗
Figure 7
Figure 7. Figure 7: Applicability rates by query-length decile. Bars indicate the number of queries per decile ( [PITH_FULL_IMAGE:figures/full_fig_p026_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Run-order drift diagnostics using rolling mean differences in applicability rates and ADWIN change [PITH_FULL_IMAGE:figures/full_fig_p027_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Rationale Match vs. Miss by subcategory Stacked bars show, for each subcategory, the percentage of Match (green) vs. Miss (red) between GPT and the human-selected subcategory rationale, computed only on queries where GPT agrees with the human consensus applicability label. The N label denotes the number of times the subcategory appears in the human rationale set within this agreement subset. Panels show (a… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

28 extracted references · 4 linked inside Pith

  1. [1]

    What is the use of Tylenol?

    Purely Factual Medical Queries Descrip- tion:The patient requests specific medical informa- tion, including explanations of medical concepts, without emotional distress or underlying distress- ing uncertainty. Examples: • "What is the use of Tylenol?" • "Is it possible to outgrow a seafood allergy?"

  2. [2]

    InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 1393–1407

    Appraisal framework for clinical empathy: A novel application to breaking bad news conversa- tions. InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 1393–1407. Andrew Lee, Jonathan K Kummerfeld, Larry An, and Rada Mihalcea. 2023. Empathy identification sys- t...

    Pith/arXiv arXiv 2024

  3. [3]

    I have intermittent knee pain from working out. How would I know if I tore cartilage?

    Diagnosis Requests with Neutral Symptom Descriptions Description:The patient describes symptoms neutrally without expressing emotional distress or underlying distressing uncertainty. Here the request is about asking what the doctor thinks the issue is. Examples: • "I have intermittent knee pain from working out. How would I know if I tore cartilage?" 11 •...

  4. [4]

    If someone has XYZ symptoms, what might be the cause?

    Hypothetical Medical Queries Without Emotional Concern Description:The patient in- quires about hypothetical situations without emo- tional involvement. Examples: • "If someone has XYZ symptoms, what might be the cause?" • "What would happen if a person skipped their medication?" A.1.3 Emotional Reactions Applicable Definition:Emotional reactions are nece...

  5. [5]

    •Select a Subcategory: – If applicable, choose the subcategory that best explains why interpretations are needed in response to the patient query

    Classify Interpretations:For each patient query, follow these steps: • Determine Interpretations Applicability or Not Applicability:Decide whether inter- pretations are applicable or not applicable in response to the patient query. •Select a Subcategory: – If applicable, choose the subcategory that best explains why interpretations are needed in response ...

  6. [6]

    I’m managing diabetes with insulin. How often should I check my blood sugar levels?

    General Health Management Without Emotional Involvement Description:The patient seeks guidance on health management, follows up on prior advice, or requests basic guidance on mi- nor health issues, without expressing emotional distress or underlying distressing uncertainty. Here the guidance is on what the patient should do. Examples: • "I’m managing diab...

  7. [9]

    My father has been having severe chest pains and shortness of breath. Could it be a heart at- tack?

    Seriousness of Symptoms Definition:The patient describes symptoms that suggest a life- threatening or chronic health condition signifi- cantly impacting long-term health or quality of life. This includes diseases like cancer, heart dis- ease, mental health issues, or chronic conditions leading to disability. The symptoms suggest a life- threatening or ser...

  8. [10]

    I feel depressed and anxious like never before. I cannot sleep at night

    Severe Negative Emotion Expressed Defini- tion:The patient explicitly states intense emotions such as fear, frustration, or anger regarding their health. Examples: • "I feel depressed and anxious like never before. I cannot sleep at night." • "I am scared and plan on taking my son to the doctor. Should I be overly worried?" • "I’m terrified about my recen...

  9. [11]

    I am starting to get a little alarmed by this spot- ting after ovulation. Is this cause for concern?

    Underlying Negative Emotional State In- ferred Definition:The patient implies emotional distress that isn’t explicitly stated but can be in- ferred from their tone or descriptions, such as sub- tle signs of emotional worry, frustration, or distress about delays or uncertainties. Focus on emotional worry, not the medical concern. Examples: • "I am starting...

  10. [12]

    Hello, I am the mother of a five-year-old. He has a small lump that hasn’t gone away. Should I take him to a dermatologist?

    Concern Severity for Close Relations Defi- nition:The patient is asking on behalf of someone with whom they share a close, protective relation- ship, implying heightened emotional concern. Examples: • "Hello, I am the mother of a five-year-old. He has a small lump that hasn’t gone away. Should I take him to a dermatologist?" • "My son recently started day...

  11. [13]

    I’m really scared about these chest pains

    Expression of Feelings (Explicit or Implicit) Description: The patient expresses emotions directly or im- plies them through language or tone. This includes feelings such as fear, anxiety, frustration, sadness, or hopelessness. Examples: •Explicit Expression: –"I’m really scared about these chest pains." – "I’m frustrated because my symptoms aren’t improv...

  12. [14]

    With my father’s illness and financial stress, I’m feeling overwhelmed

    Sharing Experiences or Contextual Factors Affecting Emotional State and Well-being Description: The patient shares personal experiences, contex- tual factors, or circumstances that influence their health and emotional state. These include social, environmental, or personal situations beyond medi- cal concerns that affect their emotional state. Examples: •...

  13. [15]

    I’m not sure if this treatment is really working for me

    Expressions of Distressing Uncertainty About Health or Treatment Description: Uncertainties, confusion, or mistrust about their health status, treatment, or future are leading to emotional distress. This includes questions about prognosis, treatment effectiveness, or doubt about potential outcomes that indicate or imply underly- ing emotional distress. Th...

  14. [16]

    My symptoms have been affecting my job for months

    Symptoms Significantly Affecting Emo- tional Well-being or Daily Life Description: The patient describes symptoms that signifi- cantly impact their emotional well-being or daily functioning, and they express or imply emotional distress because of these symptoms. The key is the emotional impact of the symptoms, not just the symptoms themselves. Examples: •...

  15. [17]

    What is the use of Tylenol?

    Straightforward Medical Queries Lacking Emotion, Distressing Uncertainty, and Context Description:The patient requests specific med- ical information or explanations of medical con- cepts without expressing emotional distress, under- lying distressful uncertainty, or providing context (social, environmental, or personal situations) im- plying an emotional...

  16. [18]

    I’m managing diabetes with insulin. How often should I check my blood sugar levels?

    General Health Management Requests Without Emotion, Context, and Distressing Un- certainty Description:The patient seeks guidance on health management, follows up on prior advice, or requests basic guidance on minor health issues without expressing emotional distress, underlying distressful uncertainty, or providing contextual fac- tors (social, environme...

  17. [19]

    I have swelling in my ankle after a long walk. Should I be concerned?

    Diagnosis Requests with Neutral Symptom Descriptions Lacking Distressing Uncertainty and Context Description: The patient describes symptoms neutrally with- out expressing emotional distress or underlying distressful uncertainty. They provide necessary details without implying feelings or contextual fac- tors (social, environmental, or personal situations...

  18. [20]

    If someone has XYZ symptoms, what might be the cause?

    Hypothetical Medical Queries With No Emotions, Context, and Distressing Uncertainty Description: The patient inquires about hypothetical situa- tions or general medical information without ex- pressing or implying personal feelings or contex- tual factors (social, environmental, or personal situ- ations) that need acknowledgment. These queries are theoret...

  19. [21]

    Focus on understanding the details outlined below

    Read the Document:Access and thoroughly review the following document contaning the Framework Details: defined in Appendix A. Focus on understanding the details outlined below. 2.Understand Emotional Reactions: • Emotional Reactions Definition:Learn what emotional reactions are and their role in doctor-patient communication. • Understand when emotional re...

  20. [22]

    •Select a Subcategory: – If applicable, choose the subcategory that best explains why emotional reactions are needed in response to the patient query

    Classify Emotional Reactions:For each pa- tient query, follow these steps: • Determine Emotional Reactions Appli- cability or Not Applicability:Decide whether emotional reactions are applicable or not applicable in response to the patient query. •Select a Subcategory: – If applicable, choose the subcategory that best explains why emotional reactions are n...

  21. [24]

    Read the patient query

  22. [25]

    If they believe it fits, go through the Applicable subcate- gories one by one and tag at least the one they think fits best

    While annotating a dimension, first read theApplicabledefinition. If they believe it fits, go through the Applicable subcate- gories one by one and tag at least the one they think fits best

  23. [26]

    If the Applicable definition does not feel like it fits, they should still briefly review the Applicable subcategories to verify this

  24. [27]

    Agreed” and “Dis- agreed

    Then move to theNot Applicabledefinition and repeat the same process with the Not Applicable subcategories. B.3 Boundary Cases: Subjectivity and Lack of Medical Expertise Empathy applicability judgments are inherently subjective, and some patient queries lie at the boundary between emotional and informational in- tent. Such disagreements often reflectlegi...

    2019

  25. [28]

    Hello doctor, I am feeling ex- tremely anxious about my upcom- ing surgery. I can’t stop worry- ing about the possible complica- tions

    find, parents of children with health (drug) issues often experience significant distress and negative mental health effects. The mother may be ex- periencing worry and anxiety about her child’s well-being, even if she doesn’t explicitly express it. Emotional ReactionNot Needed "Hello doctor, I was suffering from an infection in my tonsil for the past fou...

  26. [2011]

    Quinn McNemar

    Measuring patient-centered communication in cancer care: a literature review and the development of a systematic approach.Social science & medicine, 72(7):1085–1095. Quinn McNemar. 1947. Note on the sampling error of the difference between correlated proportions or percentages.Psychometrika, 12(2):153–157. N Mokhberian, MG Marmarelis, FR Hopp, V Basile, F...

    Pith/arXiv arXiv 1947

  27. [2018]

    Sermo Team

    Being a parent to an adult child with drug problems: Negative impacts on life situation, health, and emotions.Journal of Family Issues, 39(8):2311– 2335. Sermo Team. 2025. Can physicians and patients trust AI doctor apps like ChatGPT? https://www.sermo. com/resources/ai-doctor-app/. Blog post; ac- cessed 22 July 2025. Ashish Sharma, Adam S Miner, David C ...

    Pith/arXiv arXiv 2025

  28. [2024]

    emo- tional support and case finding

    Cause-aware empathetic response genera- tion via chain-of-thought fine-tuning.arXiv preprint arXiv:2408.11599. Aida Mostafazadeh Davani, Mark Díaz, and Vinodku- mar Prabhakaran. 2022. Dealing with disagreements: Looking beyond the majority vote in subjective an- notations.Transactions of the Association for Com- putational Linguistics, 10:92–110. Joost De...

    Pith/arXiv arXiv 2022