{"id":"89dd4f14-efe0-4a7d-88d8-122c48e8cbeb","arxiv_id":"2610.40476","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Multi-turn reinforcement learning in simulated psychiatric intake environments improved a language model's structured history-taking, differential diagnosis, communication, and safety triage relative to its identically prompted base model, as judged by blinded board-certified psychiatrists.","lead":"The authors trained an AI psychiatric intake interviewer by having it practice against simulated patients, with rewards for thorough history-taking, rapport, and safety triage. In a double-blind study with 32 psychiatrists role-playing patients and 15 psychiatrists grading transcripts, the trained system beat its identical untrained twin in 76.7% of 60 matched scenario pairs, though serious gaps remained.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"The 76.7% OSCE win rate may be partly length-driven: trained-agent transcripts average 60 clinician turns versus 8 for the base model, and the reported rebuttal (ρ = 0.18, CI [−0.09, 0.43]) cannot exclude a systematic thoroughness cue given near-collinearity of length and condition.","rationale":"The reader's weakest-assumption pick, the fidelity premise, is real but not the most load-bearing. The authors explicitly scope the claim to controlled, standardized-patient settings: they state that psychiatrists role-playing patients may present in 'overly structured, textbook patterns' (b177) and call validation with professional standardized patients and real patient cohorts 'essential next steps' (b177; b195). The fidelity concern therefore targets real-patient transfer, a claim the paper does not make. The length confound targets the claim the paper does make: that simulation RL produced human-visible improvement in the blinded OSCE (b7; b51). Why the correlational rebuttal is insufficient: condition and length are near-collinear, since nearly every trained-agent transcript is 7.5x longer than its paired base transcript; between-scenario variance in the length difference is small, so ρ = 0.18 with CI [−0.09, 0.43] is consistent with a substantial length effect, and the partial Spearman ρ said to have been assessed (Section 4.7) is not reported in the main text. Only an experimental length match can separate the variables. The surrounding evidence reads consistently with a thoroughness-driven interpretation: the trained agent's largest gains are the history-coverage items that require long transcripts; the one global live-satisfaction measure (GMCPQ) is non-significant while every checklist-style measure shows large effects; and the agent's median length is 16 turns in simulated evaluation versus 60 in the human OSCE, an emergent interaction property the paper did not control. If the concern lands, the claim reduces to 'RL produces longer, more checklist-complete transcripts that experts rate as more thorough' — materially weaker than the abstract's claim of improved clinical examination skill, communication, and diagnostic formulation. Credit where due: a pre-specified primary endpoint with an exact sign test; a genuinely double-blind crossover with disjoint actor and evaluator pools; 48-hour washout; counterbalanced order; and honest reporting of residual failures (46.7% comprehensive treatment, 60.6% critical-question omission, 12.9% under-triage) and of non-significant WAI-SR and GMCPQ results. The abstract's automated and adversarial safety numbers also rely on the in-family Gemini 3.1 Pro autorater (acknowledged in b179), with no reported out-of-family check for the 256-scenario safety claims, though the intake-suite attenuation (0.270 to 0.165) shows the direction of that bias. Verdict remains CONDITIONAL, as the reader concluded, but the decisive condition should be a length-matched OSCE validation; that check is the one most likely to change the interpretation of the headline result.","tokens_in":54040,"tokens_out":21557,"duration_ms":199458,"concrete_test":"Re-run the OSCE on 20–30 of the 60 scenario pairs with the base model prompted identically except for an explicit full-length instruction (for example, 'conduct a complete intake; do not terminate before approximately 40 clinician turns; cover all five history domains, safety screening, rapport-building, and management planning'), keeping the trained agent, the 15 evaluators, and the blinding protocol unchanged. Decision rule: if the trained-agent win rate falls below roughly 60% (from 76.7%) or the PACES gap drops to d_z < 0.3, the headline effect is substantially length- or thoroughness-driven; if it holds above roughly 70% with d_z > 0.5, the length confound is refuted.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central external evidence is the blinded human OSCE: 76.7% win rate (46/60, p < 0.0001) and PACES 3.04 vs. 2.32 (d_z = 0.77) from 15 blinded psychiatrist raters (Section 2.1; b51). The load-bearing weakness is the encounter-length imbalance inside that study: the trained agent took a median of 60 clinician turns (max 165) versus 8 (max 17) for the base model. The primary composite is the unweighted sum of 18 PACES plus 4 differential-diagnosis items (Section 4.5); items such as systems review, past medical history, family history, medication reconciliation, and DDx comprehensiveness cannot be satisfied in an 8-turn transcript, and the trained agent's largest gains sit exactly on these thoroughness items (+1.21, +1.10, +0.82, +0.81, +0.64). The authors' rebuttal, a non-significant correlation between scenario-level score gains and turn-count differences (ρ = 0.18, CI [−0.09, 0.43]; Limitations), is underpowered: with N = 60, a CI reaching 0.43, and length near-collinear with condition across all 60 pairs, a between-scenario correlation cannot separate the two; the partial Spearman ρ controlling for turn count promised in Section 4.7 is not reported in the main text. Live ratings face the same cue because clinician-patients directly experienced the 60- versus 8-turn difference, and the only global live instrument, GMCPQ, is non-significant (d_z = 0.19, p = 0.091) while all checklist-style instruments show large effects. Each transcript also had a single grader, with different graders across the two arms of a pair (Appendix A.2), leaving rater heterogeneity in valuing thoroughness unquantified. This concern strikes at the in-scope relative claim; the reader's fidelity premise is real but explicitly acknowledged by the authors (b177) and bears on the deferred real-patient transfer, not on the controlled comparison the paper makes.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"The manuscript reports that multi-turn reinforcement learning (GRPO) against a generative patient simulator improves a Gemini 3.1 Flash-Lite model on structured text-based psychiatric intake relative to its identically prompted base model, with all evaluation in simulated or psychiatrist-enacted settings. The evidence has three parts: (i) a randomized, counterbalanced crossover OSCE in which 32 board-certified psychiatrists role-played standardized patients and a disjoint panel of 15 blinded psychiatrists graded de-identified transcripts, yielding the pre-specified primary endpoint of 76.7% pair win rate (46/60, p < 0.0001) and PACES clinical-skills scores of 3.04 vs. 2.32 (d_z = 0.77); (ii) an automated suite of 127 psychiatrist-validated intake scenarios graded by an LLM autorater, where the trained agent scored 0.706 vs. 0.379 on the clinical rubric and the advantage, though attenuated, persisted under an out-of-family judge; and (iii) 256 adversarial scenarios derived from two Delphi-consensus taxonomies, with under-triage reduced from 30.5% to 12.9% and residual failures still present. The paper is carefully scoped: the authors repeatedly state that the system is a research prototype, that real-patient validation is required, and that any clinical role would be strictly assistive and clinician-supervised.","tokens_in":54480,"tokens_out":14165,"duration_ms":112518,"significance":"If the central result holds, the paper makes a meaningful methodological contribution: it demonstrates human-visible transfer of simulation-trained conversational psychiatric interviewing skill relative to an identically prompted base model, using a reusable recipe of gated-disclosure patient simulation, multi-component rubric reward, GRPO training, and human OSCE validation. The study design has genuine strengths that deserve explicit credit: a pre-specified primary endpoint, randomized counterbalanced crossover with a 48-hour washout, blinded independent evaluators disjoint from the role-playing actors, 127 scenarios validated by 54 psychiatrists (Gwet's AC1 = 0.93), a cross-evaluator check with an out-of-family judge, and a three-way concordance analysis (Fig. 3c-f) that honestly quantifies where automated evaluation departs from human judgment (automated d_z = 2.11 vs. 0.71 for psychiatrists; scenario-level delta rho = 0.09). The Limitations section is unusually candid, acknowledging the in-family autorater circularity, the role-player fidelity premise, the text-only mental status examination, single-rater transcripts, and the absence of human clinicians as comparators.","major_comments":[{"comment":"The primary OSCE endpoint is potentially confounded with encounter length. The trained agent produced a median of 60 clinician turns (maximum 165) versus 8 (maximum 17) for the base model, and the largest PACES gains are concentrated in exactly the thoroughness items that cannot be satisfied in an 8-turn transcript (past medical history +1.21, review of systems +1.10, medication reconciliation +0.81, family history +0.64). The rebuttal reported in the Limitations, rho = 0.18 with 95% CI [-0.09, 0.43] between scenario-level score gains and turn-count differences, cannot separate length from condition: with N = 60 and length near-collinear with condition across all pairs, a non-significant between-scenario correlation is expected even if length is the main driver. The partial Spearman correlation controlling for clinician turn count promised in §4.7 is not reported in the main text. The observation that the only global live instrument, GMCPQ, was non-significant (d_z = 0.19, p = 0.091) while all checklist-style instruments showed large effects is consistent with a systematic thoroughness cue. Please report the partial correlation and at least one length-controlled sensitivity analysis (for example, ratings on matched-length transcript subsets or on length-truncated transcripts), and state the outcome of that analysis in the Limitations section.","section":"§2.1, §4.7, §3 Limitations","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The headline safety results (under-triage 12.9% vs. 30.5%, contraindications 0.8% vs. 6.2%, missed red flags 6.2% vs. 10.9%) are produced exclusively by the Gemini 3.1 Pro autorater, the same model family that supplied the RL training reward in §4.3. The authors acknowledge this circularity risk in the Limitations, and they mitigate it for the intake suite by an out-of-family judge (GPT-5.6-Luna), under which the trained-base gain attenuated from 0.270 to 0.165 (Section 2.2). However, no such independent check and no human adjudication is reported for the 256-scenario adversarial safety evaluation, so the paper's central safety claim rests on an in-family judge whose magnitude inflation is demonstrated, not merely hypothesized, by the Section 2.2 comparison. Given that the safety reduction is a headline contribution of the abstract, the adversarial evaluation needs either an out-of-family evaluator or a human-annotated subset of the 256 safety outcomes before those rates can be taken at face value.","section":"§2.3, §4.3","authors_acknowledge":"Third, the training reward and scenario reference rubrics were generated within the same model family (Gemini 3.1 Pro) that served as the primary automated evaluator, introducing circularity risks including verbosity bias, stylistic mimicry, and reward hacking58;59.","authors_acknowledge_p":0.82}],"minor_comments":[{"comment":"The abstract and Section 2.1 describe the OSCE as double-blind, but the live clinician-patient ratings are necessarily unblinded to encounter length, since the actor directly experiences the 60-turn versus 8-turn difference in real time. The Limitations paragraph acknowledges this cue; the protocol description in §4.5 would be more precise if it stated the asymmetry between the independent-evaluator blinding and the live-rating blinding.","section":"§2.1, §4.5","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Each pair's two transcripts were graded by different evaluators, and the 120 transcripts are nested within 15 raters; the paired statistics therefore include between-rater variance, and no rater-level clustering is modeled. This is acknowledged in Appendix A.6, but a brief quantification (for example, a rater-random-effects sensitivity analysis or the distribution of transcripts per rater) would let readers gauge how much of the within-pair difference could be attributable to rater leniency.","section":"§4.5, Appendix A.6","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The manuscript states that release of the evaluation materials is being 'explored.' Since the paper's contribution includes a validated scenario suite and two Delphi taxonomies, the authors should state clearly whether the 127 scenarios, rubrics, and taxonomy definitions will be released with the final version; the current phrasing leaves reproducibility and reuse of the methodology uncertain.","section":"Data Availability","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The piecewise turn-length penalty in Eq. (1) is zero below 30 clinician turns and saturates at 40, yet the trained agent's OSCE transcripts ran to a median of 60 and a maximum of 165 clinician turns while its simulated encounters had a median of 16. A sentence explaining this mismatch (for example, whether termination behavior failed to transfer to human patients, or whether the human actors' responsiveness changed the reward geometry) would clarify a point that is directly relevant to the length confound discussed above.","section":"§4.3, §4.4","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision","confidential_remarks":"This is a well-executed industry demonstration with an unusually candid limitations section, and the two issues I raised are fixable within the manuscript's scope: (1) reporting the promised partial-correlation and a length-controlled sensitivity analysis for the primary human endpoint, and (2) providing an independent-judge or human-annotated check on the adversarial safety rates. I would not ask the authors to add real-patient data; the careful scoping of all claims to controlled, supervised settings is appropriate and a strength. The within-model, developer-run evaluation is standard for this literature but limits independent verification, and the decision to withhold code is defensible for safety reasons; that makes prompt, maximal release of the evaluation materials (scenarios, rubrics, taxonomies) more important. The paper is a strong fit for a clinical-AI or machine-learning venue; a general-medicine referee pool may underweight the psychiatric-specific contributions such as the gated-disclosure scenario design and the Delphi consensus taxonomies."},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"Bottom line: this is a serious and unusually honest demonstration that multi-turn RL against a generative psychiatric patient simulator changes how an LLM conducts intake, in ways blinded psychiatrist raters prefer. What is genuinely new is the psychiatry-specific package: gated-disclosure scenarios, a 28-item clinical rubric, two Delphi taxonomies, 256 adversarial presentations, and a three-way human/AI-rater/simulator concordance analysis. The learning algorithm is ResidencyRL with GRPO, and the paper says so and reuses that framework's general telehealth corpus, so the novelty is incremental at the method level but real at the application and evaluation level.\n\nThe OSCE design is a real strength. Pre-specified primary endpoint tested with an exact sign test; randomized counterbalanced crossover with a 48-hour washout; 32 role-playing psychiatrists and 15 independent graders, blind to condition and hypothesis and disjoint from each other; standardized transcript formatting; and the authors report their absolute failures rather than only the wins. The limitations section names the autorater circularity, single-grader design, length confound, and synthetic scope. That is the right level of candor.\n\nThe main soft spot is encounter length. Median 60 versus 8 clinician turns is not a minor imbalance. An 8-turn base-model interview cannot plausibly cover systems review, past medical and family history, medication reconciliation, and differential comprehensiveness, and those are exactly the items where the trained agent gained most. The authors address this with a non-significant correlation (rho = 0.18, CI to 0.43), but with N=60 and length essentially collinear with condition, that correlation cannot separate length from training. The promised partial Spearman controlling for turn count is not in the main text. With one grader per transcript and different graders across the arms of each pair, rater-specific taste for thoroughness is also unmeasured. This does not sink the relative claim—longer, more thorough intake is a clinically sensible treatment effect—but it means d_z = 0.77 is an upper-bound reading, not a precise estimate.\n\nThe automated and adversarial headline numbers have a second caveat: the primary evaluator is the same Gemini 3.1 Pro family that generated the RL rewards. The authors disclose this, and the out-of-family GPT-5.6-Luna check reduces the gain from 0.270 to 0.165 while preserving significance. The abstract's \"nearly doubled\" framing should really carry that context.\n\nNo code, scenarios, or taxonomies are released, so independent audit is not currently possible. Citations look fair, including the ResidencyRL reuse.\n\nWho should read it: anyone working on RL in clinical dialogue, LLM evaluation methodology, or AI-for-mental-health safety. It deserves serious peer review. I would send it out, asking for a turn-budget-matched or length-controlled analysis, dual grading with IRR, out-of-family evaluation in the main text, and release of at least the taxonomies and scenario suite.","headline":"A credible, unusually honest OSCE study showing simulation RL changes psychiatric intake behavior; the win is real but partly a length/thoroughness effect, and the automated numbers need the in-family rater caveat in front of them.","tokens_in":55535,"tokens_out":4470,"would_cite":true,"duration_ms":49380,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"This paper claims that multi-turn reinforcement learning against simulated psychiatric patients improves a language model's intake interviewing, with blinded board-certified psychiatrist raters preferring the simulation-trained agent…","keywords":["psychiatric intake","reinforcement learning","patient simulator","gated disclosure","clinical communication","adversarial safety testing","OSCE evaluation","large language models"],"falsifier":"A concrete test is to rerun the same 60-scenario double-blind crossover with professional standardized patients trained to exhibit disorganized speech, tangentiality, and non-textbook symptom presentations, keeping prompts, scoring rubrics, and blinding identical, and check whether the simulation-trained agent's win rate over the base model reproduces the reported 76.7%. A complementary observational check is to measure, in real supervised intake transcripts, how often genuinely critical facts are actually withheld until rapport is built, compared with the simulator's gating schedule.","tokens_in":55344,"feed_emoji":"🧠","tokens_out":10080,"duration_ms":66814,"temperature":0.7,"pith_summary":"This paper claims that a language model can learn clinically meaningful psychiatric interviewing by training against simulated patients, and it backs the claim with head-to-head tests against its own untrained base model. In 60 matched, double-blind text consultations, board-certified psychiatrists role-playing patients encountered both models, and an independent panel of 15 blinded psychiatrists preferred the trained agent's transcript in 76.7% of pairs ($p < 0.0001$), with higher scores on history-taking, differential diagnosis, communication, and management planning. Across 127 psychiatrist-validated simulated scenarios the trained agent nearly doubled the base model's clinical rubric score (0.706 vs. 0.379), and under 256 adversarial presentations it more than halved under-triage of acute safety risks (12.9% vs. 30.5%). The point of interest is that the training signal comes entirely from simulated encounters with a patient simulator that withholds diagnostically critical facts until the clinician earns them through rapport and targeted questioning. The authors are explicit that the system is not ready for autonomous use and that validation with real patients is the necessary next step.","feed_headline":"77% of psychiatric intake matchups won by simulation-trained AI","feed_subtitle":"Blinded psychiatrists rated its history-taking, diagnosis, and communication above the identically prompted base model.","key_machinery":"The load-bearing object is the generative patient simulator: each scenario carries a disclosure architecture specifying which clinical facts are offered freely, which require a targeted question, and which unlock only after the clinician builds rapport, and the simulator's two-call turn architecture first reasons about the patient's hidden mood and communicative-reliability state (truthful, withholding, downplaying, exaggerating, forgetful) and then generates dialogue conditioned on that state, which is what makes the training signal a test of interviewing rather than of knowledge recall. The second piece is the reward structure: a hierarchical tree scored by a language-model evaluator over 28 rubric items spanning psychiatric management, diagnostic appropriateness, alliance and communication, psychiatric assessment, and documentation, combined with fixed penalties so severe that the policy must balance diagnostic thoroughness against rapport. The third piece is the optimizer, Group Relative Policy Optimization (GRPO), which updates the policy from group-relative advantages on self-generated trajectories without a separate critic. Together these components convert a prompting-only model into one whose elicited information and differential formulation psychiatrists rated substantially higher.","core_discovery":"The central claim is that multi-turn reinforcement learning in a simulated clinical environment transfers to human-judged psychiatric intake performance. The authors trained a lightweight language model using GRPO on encounters with a generative patient simulator whose hidden state tracks mood and communicative reliability, gating diagnostically critical facts behind rapport and targeted inquiry, with a reward tree that jointly scores a 28-item clinical rubric, patient-centered communication, and large fixed safety penalties for hallucinations, contraindications, premature closure, and under-triage. By comparing the trained agent against its identically prompted base checkpoint under identical prompts, tools, and reasoning budgets, the design isolates the effect of simulation training. The transfer evidence comes in three evaluations: a randomized double-blind crossover OSCE with 32 psychiatrists acting as patients and 15 independent psychiatrists grading transcripts (win rate 76.7%, $p < 0.0001$; PACES clinical skill 3.04 vs. 2.32, $d_z = 0.77$), an automated suite of 127 expert-validated scenarios (rubric 0.706 vs. 0.379), and 256 adversarial presentations in which under-triage fell from 30.5% to 12.9%.","pith_inferences":["If the fidelity premise holds, the same simulator-based training loop could be pointed at other rapport-gated professional conversations, such as social-work intake, crisis-line triage, or legal first interviews, where clients withhold decisive facts until trust is established, though the paper does not explore these domains.","The weak scenario-level correlation between simulated and human scores ($\\rho = 0.09$ on trained-minus-base differences) suggests the simulator teaches the right average behavior but may misallocate effort across individual cases, and calibrating the simulator to per-scenario human difficulty ratings is a direct, testable refinement the paper leaves implicit.","Part of the measured gain may be specific to the judge model that authored the training rubrics, since the gain attenuated from 0.270 to 0.165 under an out-of-family evaluator, and retraining with rewards from a hidden human-scored set would test how much of the human-visible improvement is reward-family-specific.","The reduced under-triage rate was achieved against concealment tactics written into the simulator, so whether the trained agent generalizes to concealment styles it never saw, such as disorganized or culturally specific ways of hiding crises, is open and testable with a held-out adversarial set generated independently of the training taxonomy."],"forward_implications":["Multi-turn RL against a gated-disclosure simulator is a general recipe for improving conversational clinical skills without adding new medical corpora, and the same pipeline that produced history-taking gains also cut automated hallucination rates from 46.5% to 3.1%.","Safety-weighted penalties visibly shape behavior, with contraindication recommendations falling from 6.2% to 0.8% and under-triage from 30.5% to 12.9% across 256 adversarial presentations, suggesting RL can reduce omission failures that post-hoc output filters cannot intercept.","Residual gaps mean any near-term deployment must be assistive and clinician-supervised, with process-level safeguards such as mandatory safety-question checklists, because a critical intake question was still omitted in 60.6% of simulated intakes.","Simulated evaluation reproduces the direction of training effects but inflates their size ($d_z = 2.11$ in simulation vs. 0.71 with human raters) and tracks human judgment only weakly on per-scenario differences ($\\rho = 0.09$), so automated arenas can order models but cannot substitute for human validation.","The 127-scenario expert-validated suite and the two Delphi-consensus taxonomies provide a reusable, clinician-endorsed audit methodology for conversational psychiatric systems, including a structured way to stress-test omission failures."],"supporting_citations":[{"why":"Supplies the reinforcement-learning-in-simulation framework, the scenario-generation pipeline, and the two-call patient simulator architecture that this paper extends to psychiatry.","marker":"[36]"},{"why":"Supplies the conversational-diagnosis benchmark, OSCE design, and the evaluation instruments (PACES items, differential diagnosis and management scales) adapted here.","marker":"[10]"},{"why":"The Royal College clinical-skills examination rubric used as the 18-item transcript scoring instrument for blinded psychiatrist evaluators.","marker":"[43]"},{"why":"The Kalamazoo consensus statement from which the 11-item patient-centered communication checklist (PCCBP) is drawn, a primary communication outcome.","marker":"[44]"},{"why":"The base language model whose identically prompted checkpoint serves as the within-model comparator isolating the effect of simulation training.","marker":"[45]"},{"why":"The language model used for scenario assembly and as the primary automated evaluator that computes the training and evaluation rewards.","marker":"[74]"},{"why":"The language model that powers the patient simulator for training and simulated evaluation encounters.","marker":"[79]"},{"why":"Introduces the Group Relative Policy Optimization (GRPO) update used to train the policy on self-generated trajectories.","marker":"[86]"}],"fun_headline_variants":["Simulation-trained AI wins 77% of psychiatric intake matchups","RL in simulation beats base model on psychiatric intake","Psychiatric intake AI improves after simulated patient training","Simulation training lifts psychiatric intake AI over base model","Trained in simulation, psychiatric intake AI tops base model in blind tests"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The load-bearing premise is that the simulator's gated-disclosure rules, whereby critical facts are revealed only after rapport and targeted questioning, faithfully reproduce how real patients communicate in psychiatric intake, and the authors themselves concede that psychiatrists role-playing patients may present symptoms in overly structured, textbook patterns.","fun_headline_variants_meta":{"raw":{"variants":["Simulation-trained AI wins 77% of psychiatric intake matchups","RL in simulation beats base model on psychiatric intake","Psychiatric intake AI improves after simulated patient training","Simulation training lifts psychiatric intake AI over base model","Trained in simulation, psychiatric intake AI tops base model in blind tests"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":9.4e-05,"raw_usage":{"cost":9.4003e-05,"is_byok":false,"cost_details":{"upstream_inference_cost":9.4003e-05,"upstream_inference_prompt_cost":3.7219e-05,"upstream_inference_completions_cost":5.6784e-05},"total_tokens":1557,"prompt_tokens":1245,"completion_tokens":312,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":1045,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":242}},"tokens_in":1245,"tokens_out":312,"cost_source":"openrouter_usage","duration_ms":4264,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":242,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-10-30T09:48:24.116456+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"falsifier":"A concrete test is to rerun the same 60-scenario double-blind crossover with professional standardized patients trained to exhibit disorganized speech, tangentiality, and non-textbook symptom presentations, keeping prompts, scoring rubrics, and blinding identical, and check whether the simulation-trained agent's win rate over the base model reproduces the reported 76.7%. A complementary observational check is to measure, in real supervised intake transcripts, how often genuinely critical facts are actually withheld until rapport is built, compared with the simulator's gating schedule.","supporting_citations":[{"cited_title":"ResidencyRL: Reinforcement Learning in Simulated Clinical Environments","cited_arxiv_id":"2608.07418","evidence_quote":"Supplies the reinforcement-learning-in-simulation framework, the scenario-generation pipeline, and the two-call patient simulator architecture that this paper extends to psychiatry."},{"cited_title":"Towards conversational diagnostic artificial intelligence.Nature, 642(8067):442–450, 2025","cited_arxiv_id":null,"evidence_quote":"Supplies the conversational-diagnosis benchmark, OSCE design, and the evaluation instruments (PACES items, differential diagnosis and management scales) adapted here."},{"cited_title":"PACES (Practical Assessment of Clinical Examination Skills).https://www.thefederation.uk/examinations/paces, 2026","cited_arxiv_id":null,"evidence_quote":"The Royal College clinical-skills examination rubric used as the 18-item transcript scoring instrument for blinded psychiatrist evaluators."},{"cited_title":"Essential elements of communication in medical encounters: The Kalamazoo consensus statement.Academic Medicine, 76(4):390–393, 2001","cited_arxiv_id":null,"evidence_quote":"The Kalamazoo consensus statement from which the 11-item patient-centered communication checklist (PCCBP) is drawn, a primary communication outcome."},{"cited_title":"Gemini 3.1 Flash-Lite Model Card.https://deepmind.google/ models/model-cards/gemini-3-1-flash-lite/, 2026","cited_arxiv_id":null,"evidence_quote":"The base language model whose identically prompted checkpoint serves as the within-model comparator isolating the effect of simulation training."},{"cited_title":"Gemini 3.1 Pro Model Card.https://deepmind.google/models/ model-cards/gemini-3-1-pro/, 2026","cited_arxiv_id":null,"evidence_quote":"The language model used for scenario assembly and as the primary automated evaluator that computes the training and evaluation rewards."},{"cited_title":"Gemini 3 Flash Model Card.https://storage.googleapis.com/ deepmind-media/Model-Cards/Gemini-3-Flash-Model-Card.pdf, 2025","cited_arxiv_id":null,"evidence_quote":"The language model that powers the patient simulator for training and simulated evaluation encounters."},{"cited_title":"The structure of ill structured problems","cited_arxiv_id":"2402.03300","evidence_quote":"Introduces the Group Relative Policy Optimization (GRPO) update used to train the policy on self-generated trajectories."}],"review_version":1}