K-Bench: a clinically calibrated benchmark for evaluating large language models in high-risk mental health conversations

Adva Segal; Angela Loulopoulou; Apoorv Jha; Aruba Hussain; Elena Volkova; Laura M. Vowels; Matthew J. Vowels; Rachel Francois-Walcott; Rehnuma Choudhury; Sarah Ingram

REVIEW

K-Bench: a clinically calibrated benchmark for evaluating large language models in high-risk mental health conversations

Laura M. Vowels, Matthew J. Vowels, Shivali Sharma, Apoorv Jha, Rehnuma Choudhury, Wasseem El Sarraj, Rachel Francois-Walcott, Aruba Hussain

show 4 more authors

Sarah Ingram, Angela Loulopoulou, Adva Segal, Elena Volkova

Computation and Language · Submitted 14 Sep 2026

Pith's verdict: A clinically serious, well-validated mental-health benchmark worth refereeing, with one number that needs to be reported properly: the judge's held-out agreement, not just the pooled calibration figure.

T0 · AI review

How this review was made: An AI referee (deepseek/deepseek-v4-flash) read the paper on September 15, 2026 and answered a fixed list of questions about it. T0 is the starting level every paper gets. The levels describe how deeply Pith has checked the paper, never the authors.

T0 AI review An AI read the paper and answered a fixed list of questions.
T1 Audit Automatic checks that the references and files the paper relies on exist and match.
T2 Compute The paper's code or calculations re-run with the same results.
T3 Formal statement The claims written out in a formal math language a computer can read.
T4 Proof Those claims proved, and the proof checked by computer.

How Pith checks papers →

· 4 major objections · 6 minor · Ask about this paper · The compiled record · Read the full review →

Challenge this review Comment (0) Read on arXiv ↗

Cite ▾

arXiv ID2609.15855 (version 3)

Pith ID7IUCFINM, Pith's permanent record of this paper and its review

CitationAdva Segal, Angela Loulopoulou, Apoorv Jha, Aruba Hussain, Elena Volkova, Laura M. Vowels, Matthew J. Vowels, Rachel Francois-Walcott, Rehnuma Choudhury, Sarah Ingram, Shivali Sharma, Wasseem El Sarraj. "K-Bench: a clinically calibrated benchmark for evaluating large language models in high-risk mental health conversations." arXiv:2609.15855 (2026). https://pith.science/paper/7IUCFINM

Cite this reviewBibTeX for Pith's review of this paper

Claim paper

Why Pith says this

Highlight any text to comment on it.

Editor's note

read the letter

Two things you should know. K-Bench is the first mental-health benchmark I've seen that combines four co-occurring risk domains with multi-turn conversations up to 20 turns, 125 model configurations, a clinician-calibrated judge, and protected test materials. Second, the validation is generally more careful than most work in this area—but the headline judge-agreement number is reported only on the full calibration set, and that is the soft spot worth pushing on.

What is genuinely new: nobody else has evaluated this many configurations (33 base models, 14 providers) on a fixed cohort of risk vignettes that include co-occurring and gradually disclosed risk. The factorial vignette expansion from lived-experience-derived source material is a real effort to control clinical complexity, not just a generic persona bank. The rubric is granular: 47 components across seven dimensions, with risk exploration separated from clinical judgement. That separation pays off—D2 is the discriminating dimension, and that is a clinically meaningful finding rather than an artifact.

The judge validation is thorough in most respects. Six clinicians produced 22,816 pair comparisons at 89.4% exact agreement; the frozen GPT-4o judge reached 94.2% on 6,751 item comparisons, with the expected pattern of lower agreement on severity and higher on clinical judgement and global dimensions. The matched prompting and reasoning comparisons are also done carefully, with model-cluster bootstrap intervals and clean baselines that exclude relabelled provider defaults.

Where I'd push back: the stress-test note is right. The Methods say prompt-tuning transcripts were separated from a holdout set, but the Results report only the pooled 'primary D1–D7 calibration set.' Since the judge prompt was iterated on a known subset, the pooled figure could be optimistic, and we cannot see how the judge generalizes to the holdout—which is exactly what it does on the live benchmark. The fix is simple: report judge–consensus agreement on the holdout subset separately. I suspect it remains strong, but the paper should not ask readers to take the pooled number on faith.

The realism validation is honest but limited: the reference corpus contained no-risk conversations, so the embedding overlap evidence supports the no-risk condition, and the authors say the high-risk generalization is an assumption. That is a limitation, not a flaw, and it is acknowledged. Bigger structural issue: all operational materials, judge prompts, and code are withheld, which is defensible for a protected benchmark but means external verification is impossible at item level. For a benchmark whose entire value is its validity, a little more disclosure—for example, holdout agreement, per-item confusion matrices, more transparency about the 'stealth/ox-alpha' entries—would strengthen trust.

Who should read this: anyone making deployment decisions about LLMs in mental-health contexts, regulators, and benchmark designers. It deserves a serious referee. My recommendation: send it to peer review, and make the holdout breakdown a required revision rather than a suggestion.

Referee report

Significance

If the judge's calibrated agreement generalises to unseen transcripts, K-Bench is a substantial step forward for safety evaluation of conversational AI in mental health: it is the largest clinician-calibrated, configuration-scale comparison to date in this domain, it covers co-occurring and gradually emerging risks with a granular rubric, and its protected test architecture is a principled answer to the contamination problem that open benchmarks face. The paper is unusually candid about its own limits: the no-risk-only realism reference, the not-applicable gating of unreached items, the 20-turn cap, and the descriptive rather than causal reading of severity and comorbidity differences are all stated explicitly. Methodological strengths include independent clinician ratings with adjudicated consensus, a deterministic and frozen judge, matched-pair comparisons with model-cluster bootstrap confidence intervals, and a versioned public leaderboard with full configuration metadata.

Major objections (4)

[Results, 'Automated-judge calibration'; Online Methods, 'Automated judge calibration and scoring'] The central validation statistic, 94.2% judge-consensus agreement on 6,751 eligible items, is reported only for the pooled D1-D7 calibration set. The Online Methods state that 'Prompt-tuning transcripts were separated from a holdout set reserved for downstream validation; the holdout data were not used during prompt iteration,' but the Results give no agreement figure (or even the number of transcripts) for that holdout, and no explanation of how the pooled figure decomposes into tuning and holdout subsets. Because the judge's production role is to score unseen transcripts in the 200-vignette leaderboard cohort, the holdout agreement is the direct estimate of the generalization on which the benchmark depends; iterating the prompt on part of the data and reporting only the pooled number leaves the possibility that the headline figure is optimistic. Please report the holdout size and its judge-consensus agreement overall and by rubric section, or justify why the pooled figure suffices.
[Results, 'Synthetic-patient realism'; Online Methods, 'Embedding-based realism assessment'] The realism validation compares synthetic turns with the Amanda corpus only under the no-risk condition, because, as the manuscript itself states, the reference corpus 'contained no-risk conversations because users presenting with risk were not eligible to continue in the original studies.' The claim that 'these findings establish synthetic patients as a realistic and controlled basis' for evaluating high-risk conversations is therefore an extrapolation: the suicide, self-harm, domestic-violence, and substance-misuse synthetic conditions are never compared with any real-risk reference. The rubric already contains clinician-rated patient-context items ('whether the patient's responses were realistic,' 'whether the emotional tone was believable') that were collected on the 151 transcripts but excluded from the leaderboard composites; reporting aggregate clinician ratings on those items would provide a direct, within-study anchor for the realism of risk-active synthetic patients. Alternatively, the Discussion claim that the overlap data support the realism of risk-active patient language should be softened.
[Table 1 note; Results, 'Automated-judge calibration'] The claim that the judge 'reproduced the clinician-derived scoring standard with greater item-level consistency than clinicians achieved with one another' compares judge-consensus agreement (94.2%) with clinician-clinician pairwise agreement (89.4%). These two targets are not directly comparable: consensus is the modal or central response, so a judge that matches the modal value will mechanically out-agree an average pair of clinicians whenever ratings split 2-1 (the judge agrees with two of the three raters, whereas the clinicians agree with one another only one time in three on such items). At least part of the reported 4.9 percentage-point advantage is therefore a property of the comparison target rather than evidence of the judge's superior consistency. I suggest additionally reporting the judge's agreement with individual clinician ratings (judge versus the 478 raw rating rows) or the agreement of a re-derived consensus with held-out clinicians, so that judge and human performance are measured against comparable targets.
[Results, 'Leaderboard composition and score meaning'; Online Methods, 'Automated judge calibration and scoring'] The judge is openai/gpt-4o, a model from one of the providers whose systems occupy the top leaderboard ranks, and the patient agent used for the public evaluation is also GPT-4o. Self-preference bias in LLM-as-judge evaluation is a documented phenomenon in the evaluation literature, so the cross-provider comparison rests on the assumption that the frozen judge's agreement with clinician consensus does not vary with the identity of the target model. No reported analysis addresses this: judge-consensus agreement is stratified by rubric section and risk domain but not by the provider or family of the model that generated each calibration transcript. A concrete check would be to stratify the 6,751 judge-consensus comparisons by target model provider and show that agreement and the distribution of discrepancies are stable; if the calibration transcripts came from a small or unrepresentative set of target models, that should be stated as a limitation.

Minor objections (6)

[Throughout (typesetting)] Several passages have lost spaces between words, e.g., 'K-Benchevaluatessevendimensions' in the Main text, 'itmeasuredrubric-concordantconversation' in the Discussion, and 'Reasoningrefersheretoaprovider-exposedinference-timeconfiguration' in the Online Methods; these appear to be typesetting artifacts and should be corrected.
[Supplementary Table 5; Results, 'Therapeutic-prompt effects'] Supplementary Table 5 is titled 'Full Matched Effects of Therapeutic Prompting' but lists 16 pairs, whereas the text reports 61 matched default-versus-therapeutic pairs and a stricter 43-pair analysis; please clarify that the table is a selected subset and state where the complete 61-pair dataset is published.
[Figure 2; Supplementary Table 1] The top overall scores on the leaderboard differ by only 0.01-0.03 points (ranks 1-3 of Supplementary Table 1), and the bootstrap intervals in Figure 2 presumably overlap for such configurations; please report rank-stability statistics (e.g., the proportion of bootstrap replicates preserving adjacent ranks) or explicitly warn that near-tied overall rankings do not support ordinal comparisons.
[Table 1; 'Risk-severity operationalisation' in Online Methods] Exact agreement on the four-category Q1 severity item is only 65.7% among clinicians for suicide and 75.3% overall; reporting adjacent-category or weighted agreement for Q1 as well would clarify how much of the disagreement is ordinal rather than categorical, and would place the judge's 82.5% figure in a fairer context.
[Online Methods, 'Embedding-based realism assessment'] The realism reference corpus (Amanda) is described only by a citation to a manuscript under review (Vowels et al., 2026a); please state where the de-identified corpus is archived or how reviewers can access it, since the embedding-based realism comparison cannot otherwise be reproduced.
[Figure 4; Online Methods, 'Data analysis and reporting'] Figure 4 and the surrounding text use 'blended text cost' without defining the input/output token mix and pricing assumptions; a one-sentence definition in the Online Methods would make the cost comparison reproducible.

How Pith checks papers → · Full audit of this paper →

The paper

The paper claims

K-Bench provides a clinically calibrated benchmark that evaluates large language models in high-risk mental health conversations using synthetic patients and an automated judge that agrees with clinicians on 94.2% of items.

The reading

K-Bench is a benchmark designed to evaluate how safely large language models handle mental health conversations where risk can emerge gradually. The paper establishes that synthetic patient conversations, generated from lived-experience-derived material and a factorial design, are realistic enough to substitute for real users when compared with a no-risk reference corpus. It then shows that a frozen GPT-4o automated judge, calibrated against the consensus of six clinicians, reproduces clinician ratings with higher consistency than the clinicians themselves. Using this evaluation system on 125 model configurations from 33 models, K-Bench finds that leading models score above 95 on a composite of clinical judgement and risk exploration, while risk exploration separates strong from weak models more sharply than other competencies. These results matter because people increasingly turn to chatbots for mental health support, and existing benchmarks test single-turn or single-risk scenarios.

What carries the argument

The central object is K-Bench itself: a protected benchmark pipeline that combines (1) a 122-variable factorial vignette framework derived from lived-experience material, (2) synthetic patient-agent conversation generation with validated realism, (3) a 47-component clinician-developed rubric across seven dimensions, (4) item-level clinician consensus ground truth from 151 transcripts, and (5) a frozen GPT-4o automated judge calibrated to that consensus. The benchmark evaluates each model configuration on the same 200-vignette cohort in conversations of up to 20 turns, and withholds operational test materials to prevent gaming.

Extended reading notes

Core claim

The paper develops and validates K-Bench, a clinician-calibrated benchmark that evaluates LLM performance across seven dimensions of conversational competence (clinical judgement, risk exploration, ethical reasoning, supportive conversation, psychological knowledge, cultural competence, and autonomy and boundaries) using 200 multi-turn vignettes spanning suicide, self-harm, domestic violence, substance misuse, and no-risk presentations. The central discovery is that a frozen GPT-4o judge, when prompted with the same rubric used by clinicians, achieves 94.2% exact item-level agreement with the clinician consensus across 6,751 eligible comparisons, outperforming the 89.4% agreement among the clinicians themselves. This makes large-scale, reproducible evaluation feasible without requiring a clinician to read every transcript.

Load-bearing premise

The realism of synthetic patient conversations is validated only against a no-risk reference corpus (the Amanda dataset, which excluded users presenting with risk), yet the benchmark assumes that embedding-space overlap under no-risk conditions implies similar realism for conversations involving suicide, self-harm, domestic violence, and substance misuse.

Editorial extensions

If this is right

K-Bench provides a standard that future models can be measured against, enabling direct comparison of safety performance across providers, versions, prompts, and reasoning settings.
The benchmark's risk exploration dimension exposes safety-relevant weaknesses that overall scores alone would hide, guiding developers toward targeted improvements.
Therapeutic prompting helps weaker models but not stronger ones, suggesting that prompt engineering alone is insufficient for clinical safety.
The finding that elevated reasoning does not consistently improve risk performance implies that inference-time compute investments may not translate to safer mental health conversations.
Protected operational materials allow the benchmark to remain valid for new model releases, tracking genuine progress over time.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If synthetic patients validated only under no-risk conditions do not fully generalize to high-risk conversations where language may be more urgent or culturally specific, then the realism guarantees may be weaker than the benchmark claims. A direct empirical test using real high-risk conversations from crisis helplines would settle this.
The benchmark's restriction to API configurations rather than complete consumer products means that real-world safety may differ when moderation, memory, and escalation layers are present.
The 20-turn cap may miss how models handle deferred risks in longer interactions; future iterations with more turns could reveal additional failure modes.
The strong judge-clinician agreement suggests the rubric captures a reproducible standard for contemporaneous risk formulation, but not for predicting actual suicide attempts or self-harm episodes, which remains a separate challenge.

Original abstract

People increasingly use large language models (LLMs) for mental health support, yet their safety in evolving, high-risk conversations remains poorly characterised. We developed K-Bench, a clinician-calibrated, protected benchmark evaluating 125 model configurations representing 33 base models from 14 providers across a fixed cohort of 200 multi-turn vignettes involving suicide, self-harm, domestic violence, substance misuse, and no-risk presentations. Synthetic patient conversations showed substantial distributional overlap with real human-AI conversations. A frozen GPT-4o judge achieved 94.2% exact agreement with clinician consensus across 6,751 eligible item comparisons from 151 clinician-rated transcripts. Leading models combined strong supportive conversation with combined-risk scores above 95, whereas risk exploration exposed substantial variation among lower-performing configurations. Therapeutic prompting produced configuration-specific gains concentrated among weaker models, while elevated reasoning produced no average improvement. K-Bench combines broader clinical coverage and configuration-scale comparison with a continuously updated public leaderboard whose operational test materials are protected from direct optimisation. The leaderboard is available at www.k-bench.ai.

The compiled record

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 6,259 input tokens.

What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.37 · human participants 0.68 · controlled comparison 0.27 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.60 · mentions code or data availability 0.65.

The abstract, sentence by sentence

arXiv listing abstract v2. Results and the main claim are underlined; show every role.

People increasingly use large language models (LLMs) for mental health support, yet their safety in evolving, high-risk conversations remains poorly characterised. We developed K-Bench, a clinician-calibrated, protected benchmark evaluating 125 model configurations representing 33 base models from 14 providers across a fixed cohort of 200 multi-turn vignettes involving suicide, self-harm, domestic violence, substance misuse, and no-risk presentations. Synthetic patient conversations showed substantial distributional overlap with real human-AI conversations. A frozen GPT-4o judge achieved 94.2% exact agreement with clinician consensus across 6,751 eligible item comparisons from 151 clinician-rated transcripts. Leading models combined strong supportive conversation with combined-risk scores above 95, whereas risk exploration exposed substantial variation among lower-performing configurations. Therapeutic prompting produced configuration-specific gains concentrated among weaker models, while elevated reasoning produced no average improvement. K-Bench combines broader clinical coverage and configuration-scale comparison with a continuously updated public leaderboard whose operational test materials are protected from direct optimisation. The leaderboard is available at www.k-bench.ai.

background contribution method result interpretation main claim

What the classifier reports for each sentence. The number is the probability it gave the label.
#	sentence	role	the authors' own work	states the main conclusion	strength
s0	People increasingly use large language models (LLMs) for mental health support, yet their safety in evolving, high-risk conversations remains poorly characterised.	gap 0.63	0.05	0.04	asserted 0.99
s1	We developed K-Bench, a clinician-calibrated, protected benchmark evaluating 125 model configurations representing 33 base models from 14 providers across a fixed cohort of 200 multi-turn vignettes involving suicide, self-harm, domestic violence, substance misuse, and no-risk presentations.	contribution 0.69	0.97	0.08	asserted 1.00
s2	Synthetic patient conversations showed substantial distributional overlap with real human-AI conversations.	result 0.90	0.87	0.03	asserted 1.00
s3	A frozen GPT-4o judge achieved 94.2% exact agreement with clinician consensus across 6,751 eligible item comparisons from 151 clinician-rated transcripts.	result 0.80	0.89	0.06	asserted 1.00
s4	Leading models combined strong supportive conversation with combined-risk scores above 95, whereas risk exploration exposed substantial variation among lower-performing configurations.	result 1.00	0.93	0.36	asserted 1.00
s5	Therapeutic prompting produced configuration-specific gains concentrated among weaker models, while elevated reasoning produced no average improvement.	result 0.99	0.92	0.16	asserted 1.00
s6	K-Bench combines broader clinical coverage and configuration-scale comparison with a continuously updated public leaderboard whose operational test materials are protected from direct optimisation.	contribution 0.98	0.87	0.11	asserted 1.00
s7	The leaderboard is available at www.k-bench.ai.	contribution 0.90	0.40	0.02	asserted 0.99

What kind of paper the classifier reports this is: dataset or benchmark 1.00 · evidence from observation or experiment 0.96.

What was examined, and what was not

sources read

arXiv listing abstract v2

not examined

full text
figures and tables
references and citing sentences
relations to other papers

levels run

P abstract

not run

S sections
T body sentences
C citations
R cross-paper relations

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. Every number in the measurements table is the authors', checked by code. Record as JSON

Figures

**Figure 1.** Embedding-space comparison of real-user and synthetic patient turns (GPT-4o).

**Figure 2.** Overall and combined-risk performance across evaluated model configurations.

**Figure 3.** Rubric-dimension profiles for the best-performing configuration from each provider.

**Figure 4.** Combined-risk performance in relation to model inference cost.

**Figure 5.** Matched effects of therapeutic prompting and elevated reasoning on combined-risk

Figures reproduced from arXiv:2609.15855 by the authors.

References

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

What do AI chatbots do after users decline therapeutic redirection for suicide thoughts?
cs.HC 2026-10 accept novelty 8.0 of 10

Chatbots uniformly offer crisis referrals after suicide disclosures, but after refusal they vary widely in persistence, newer versions persist less, and those that stop sooner more often present themselves as the user...
SAFE-Teen: Evaluating Chatbot Safety Across Adolescent Mental Health Conversations
cs.CL 2026-11 accept novelty 6.0 of 10

Chatbot safety in teen mental health conversations depends heavily on how the teen responds, and even safety-prompts fail to eliminate risky behavior like giving diagnoses or missing indirect suicide cues.

Discussion

Disagree with the review, or know something it missed? Leave a comment, highlight any passage above to comment on it, or challenge this review formally.