{"id":"2cc86e23-ea84-4f25-a96b-4754edc52bde","arxiv_id":"2611.10997","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"A structured survey of open problems and recommendations for researchers and policymakers on the mental health effects of general-purpose chatbots.","lead":"This paper identifies and organizes the key open problems in evaluating and improving the effects of general-purpose chatbots on mental health. It provides 12 recommendations and 12 open questions spanning controlled evaluation, real-world evidence, and policy, aiming to guide future research.","discovery_kind":"review","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"No significant objection identified","rationale":"The reader's weakest assumption—comprehensiveness of the problem selection—matches the paper's own stated limitation. The paper meets its goal as a commentary: it synthesizes prior work, proposes a structured agenda, and is transparent about scope. No hidden assumptions, internal contradictions, or unsupported empirical claims weaken the argument. The central claim is appropriately bounded and supported by the cited literature. Therefore, no adjustment to the reader's ACCEPT verdict is warranted.","tokens_in":28546,"tokens_out":955,"duration_ms":10562,"concrete_test":"Conduct a systematic literature review following PRISMA guidelines on chatbot interaction and mental health outcomes, coding all identified problems and recommendations into the paper's three domains (controlled evaluation, real-world evidence, policy). If more than 15% of coded items fall outside these domains, the framing may omit critical areas. This would test whether the selective synthesis missed major themes.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper is an agenda-setting commentary, not a falsifiable hypothesis. It transparently acknowledges its unsystematic literature selection, self-imposed scope, and lack of empirical validation. The central claim that open problems fall into three interdependent domains and require coordinated effort is appropriately soft and grounded in cited evidence. The reader's identified weakest assumption—comprehensiveness of the framing—is already stated as a limitation by the authors. No hidden assumption or internal inconsistency undermines the central argument.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"This commentary identifies and organizes open problems in the evaluation and governance of general-purpose chatbots used for mental health support. The authors argue that the most critical challenges fall into three interdependent domains: controlled evaluation (construct validity, multi-turn interactions, simulation fidelity), real-world evidence (deployed product analysis, human review practices, longitudinal data collection), and policy (user protections, standards of care, research data governance). They distill 12 recommendations and 12 open questions, each tagged by stakeholder, and emphasize that progress requires coordinated effort across these domains. The paper is explicitly a selective synthesis shaped by the authors' expertise, not a systematic review.","tokens_in":28752,"tokens_out":1917,"duration_ms":21064,"significance":"The paper provides a timely, well-structured agenda for researchers, developers, and policymakers. Its organization of known challenges into three interacting areas and its concrete recommendations offer a useful roadmap for a rapidly evolving field. The explicit acknowledgment of limitations and positionality strengthens rather than weakens the contribution, as it clarifies the scope of the authors' claims. If the agenda is taken up, it could help align evaluation practices, real-world evidence collection, and regulatory policy in a domain where harms are increasingly documented but poorly understood.","major_comments":[{"comment":"The paper states (Limitations, §7) that the literature selection and problem identification were not systematic and reflect the author team's composition. This is an honest and appropriate caveat, but it means the set of open problems is necessarily incomplete. The authors should consider adding a brief sentence in the Introduction or Conclusion noting that other important problems (e.g., economic access, cross-cultural adaptation, long-term therapeutic alliance) may exist outside the current framing. This would further align the text with the already stated limitation.","section":"§1, §7","authors_acknowledge":"Given our focus on these harms, we may fail to consider the interaction between unreported harms and chatbots.","authors_acknowledge_p":0.71}],"minor_comments":[{"comment":"The paper uses stakeholder icons (🔬, ♂laptop-code, 🩺, /user-friends, /landmark) in the text. These may not render correctly in all publication formats. Please provide a text-only alternative (e.g., bracketed labels like [RESEARCHER]) in addition to or instead of the Unicode symbols.","section":"Throughout","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Recommendation 5 advises that evaluators justify simulation choices. The following sentence in the explanation—\"Targets, assessments, reference data, comparison features, and similarity metrics should be studied together\"—would benefit from a brief example or reference to a concrete simulation study to illustrate what 'studied together' means in practice.","section":"§4.2.1, Recommendation 5","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Figure 3 is described as a diagram of a chatbot product pipeline. Since the actual figure is not rendered in the arXiv HTML or PDF preview (only the caption appears), please ensure the figure is included in the submission and that it has an accessible text description in the caption.","section":"§5.1, Fig. 3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The discussion of data export and linearization (paragraph beginning \"Data export and interoperability\") mentions the need for a standard linearization approach but does not cite existing work on conversation tree linearization (e.g., from the chatbot data donation literature). A reference to a method like Moore et al. [104] (which is already cited elsewhere) would clarify the state of the art.","section":"§5.3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The suggestion to treat differing state/provincial chatbot regulations as natural experiments is promising. The authors could strengthen this by noting one concrete methodological challenge (e.g., that users can use VPNs or multi-state access) and a potential mitigation, even if brief.","section":"§6.1, Open Question 8","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The question about transparency and external oversight is well-posed, but the accompanying explanation lists disclosure criteria without a reference to existing audit frameworks (e.g., the algorithmic auditing literature from Raji et al. [141] is cited elsewhere). Adding a one-sentence connection to that literature would ground the question.","section":"§6.1, Open Question 10","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Several references (e.g., [18], [77], [89], [168]) are dated 2026, which is well in the future relative to the submission date (November 2026). All cited papers appear to be preprints or published works that are publicly accessible; this is acceptable. However, the authors should ensure that all references are in their final published form where possible, or consistently mark them as 'arXiv preprint' or 'in press'.","section":"References","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"Here's my take on this paper.\n\nIt does what it sets out to do: identify open problems in chatbots and mental health, organize them into three domains (controlled evaluation, real-world evidence, policy), and distill 12 recommendations and 12 open questions. The writing is clear, the scope is honest, and the authors are admirably upfront about their positionality and the unsystematic nature of their literature review. For a reader new to this area, this is a solid orientation. For an expert, it’s a useful check-list and framing device.\n\nWhat’s actually new here is the synthesis itself. The individual problems—lack of construct validity in benchmarks, need for multi-turn evaluation, privacy constraints on real-world data, governance gaps—are each discussed elsewhere. But pulling them together into a coherent agenda with explicit stakeholder icons and cross-referencing across domains is a genuine contribution. The paper earns credit for making the interdependence of evaluation, evidence, and policy concrete.\n\nThe soft spots are mostly the flip side of its strengths. Because the problems are selected rather than systematically surveyed, some readers will find gaps (e.g., the political economy of AI companies’ incentives gets only a brief reference in the policy section). The recommendations are sensible but sometimes vague—“develop standards” is easy to say, harder to specify. The policy coverage is US-heavy despite nods to EU and China. None of this undermines the paper’s central argument; it’s just worth noting that the agenda is shaped by the authors’ backgrounds and that other communities might prioritize differently.\n\nThe tone is measured throughout. No overclaiming, no scare-mongering. The paper acknowledges what it cannot do and cites its sources properly.\n\nWho benefits? Graduate students entering this space, policy staff looking for a structured landscape, and researchers wanting a shared vocabulary for interdisciplinary work. It won’t change a specialist’s mind, but it might help them coordinate.\n\nRecommendation: This deserves peer review. It is not a technical result, but agenda-setting pieces of this quality are legitimate contributions.","headline":"A well-structured, transparent agenda-setting paper that synthesizes known problems into a coherent framework; no empirical novelty but valuable for its scope and clarity.","tokens_in":29170,"tokens_out":1257,"would_cite":false,"duration_ms":11874,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"The most critical open problems in chatbots and mental health cluster into three interdependent domains: controlled evaluation, real-world evidence, and policy.","keywords":["chatbots","mental health","evaluation","AI governance","privacy","policy","real-world evidence","controlled evaluation"],"falsifier":"A systematic meta-review that identifies one or more critical open problems (e.g., data privacy, cultural adaptation, clinical liability) not covered by the three proposed domains would falsify the claim that this tripartite framing captures the most important challenges. Alternatively, a large-scale study demonstrating that controlled evaluation measures fail to correlate with independently assessed user outcomes would challenge the interdependence thesis.","tokens_in":30113,"feed_emoji":"🤖","tokens_out":4532,"duration_ms":39425,"temperature":0.7,"pith_summary":"This paper argues that understanding and mitigating the mental health effects of general-purpose chatbots requires tackling three interdependent sets of open problems: controlled evaluation of chatbot behavior, real-world evidence from deployed products, and evidence-based policy. The authors synthesize a broad literature to propose 12 concrete recommendations and 12 open questions organized around these domains. They claim that progress depends on connecting evaluation measures to real-world outcomes, validating user simulations against naturalistic data, translating evidence into governance and standards of care, and ensuring that all three areas advance together. A sympathetic reader would care because millions already use chatbots for emotional support, yet the field lacks validated methods to assess safety and efficacy, and policy is being made without sufficient evidence.","feed_headline":"12 open questions define roadmap for chatbot mental health safety","feed_subtitle":"Researchers outline coordinated action on evaluation, real-world evidence, and policy to protect users and improve well-being.","key_machinery":"The paper's central machinery is a tripartite framework that identifies controlled evaluation, real-world evidence, and policy as the three interacting pillars needed to understand and improve chatbot mental health effects. Within each pillar, the authors distill specific recommendations (12 total) and open questions (12 total) that serve as actionable targets for stakeholders. The framework itself highlights dependencies: evaluation needs real-world validation, real-world evidence requires governed data access and independent oversight, and policy must be informed by both.","core_discovery":"The central claim of this paper is that the most urgent challenges in chatbot mental health cluster into three interacting domains: (1) controlled evaluation, which must move from static benchmarks to ecologically valid, multi-turn assessments linked to clinical outcomes; (2) real-world evidence, which requires studying deployed products, their interfaces and human escalation systems, and collecting longitudinal chat log data under strong privacy protections; and (3) policy, which needs evidence-based standards of care, user protections, and governed data access. The authors argue that these domains are mutually dependent—valid evaluations require real-world validation, and effective policy depends on independently produced evidence—and that no single discipline can address them alone. The paper organizes the existing literature to distill 12 concrete recommendations and 12 open questions as actionable targets for researchers, developers, clinicians, users, and policymakers.","pith_inferences":["The paper's framework implicitly elevates independent evaluation as a bottleneck: without governed access to proprietary system components and real user logs, neither academic nor regulatory bodies can verify safety claims. This suggests that data governance agreements may be a prerequisite for progress in the other two domains.","The emphasis on general-purpose chatbots (rather than purpose-built mental health apps) broadens the regulatory challenge: these systems are not marketed as medical devices, yet the paper documents how they can drift into therapeutic roles, creating a liability gap that existing frameworks do not close.","A testable consequence of the paper's thesis is that in jurisdictions requiring systematic safety evaluations, measurable harms (e.g., crisis-related hospitalizations attributed to chatbot use) will be lower than in jurisdictions without such requirements, after controlling for baseline mental health outcomes."],"forward_implications":["Mental health chatbot benchmarks will need to specify constructs, test reliability, and validate against real-world outcomes rather than relying on static datasets alone.","User simulations used in safety evaluations must be validated against real user behavior using clinically relevant fidelity metrics before they can predict deployment risks.","Policy interventions such as required crisis-response reporting, component standards, and governed data access will become testable through natural experiments across jurisdictions.","Longitudinal chat log data, if shared under privacy-preserving governance models resembling genomic repositories, could provide new evidence on psychopathology and risk signals not available from clinical settings.","Human review and escalation practices by chatbot companies must be studied rigorously to determine which interventions actually connect users to care."],"supporting_citations":[{"why":"Provides a case study of simulated user behavior compared with real deployment, supporting the need for validation of simulations.","marker":"[71]"},{"why":"Demonstrates how real chat logs can reveal harmful patterns (delusional spirals), grounding the argument for real-world evidence.","marker":"[104]"},{"why":"Highlights limitations of static benchmarks and the need for multi-turn, expert-informed evaluation, directly supporting the controlled evaluation section.","marker":"[89]"},{"why":"Offers a clinically validated framework for auditing chatbot behavior, cited as a model for connecting evaluation to clinical outcomes.","marker":"[171]"},{"why":"Exemplifies recent state-level legislation restricting companion chatbots, used to anchor the policy discussion and demonstrate enacted mitigations.","marker":"[24]"}],"fun_headline_variants":["12 open questions for chatbot mental health safety","Chatbot mental health: three domains, 12 open problems","Safer chatbots: 12 research questions for mental health","Open problems in chatbot mental health evaluation and policy","12 open problems define roadmap for chatbot mental health"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The paper's selection of open problems and recommendations is based on a non-systematic literature review shaped by the author team's composition and experience; if other critical problems exist outside this framing, the research agenda would be incomplete.","fun_headline_variants_meta":{"raw":{"variants":["12 open questions for chatbot mental health safety","Chatbot mental health: three domains, 12 open problems","Safer chatbots: 12 research questions for mental health","Open problems in chatbot mental health evaluation and policy","12 open problems define roadmap for chatbot mental health"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":7.3e-05,"raw_usage":{"cost":7.343616e-05,"is_byok":false,"cost_details":{"upstream_inference_cost":7.343616e-05,"upstream_inference_prompt_cost":3.437056e-05,"upstream_inference_completions_cost":3.90656e-05},"total_tokens":1408,"prompt_tokens":972,"completion_tokens":436,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":256,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":360}},"tokens_in":972,"tokens_out":436,"cost_source":"openrouter_usage","duration_ms":4532,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":360,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-11-10T20:09:06.616974+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"falsifier":"A systematic meta-review that identifies one or more critical open problems (e.g., data privacy, cultural adaptation, clinical liability) not covered by the three proposed domains would falsify the claim that this tripartite framing captures the most important challenges. Alternatively, a large-scale study demonstrating that controlled evaluation measures fail to correlate with independently assessed user outcomes would challenge the interdependence thesis.","supporting_citations":[{"cited_title":"Johnson, Robert Friel, Cassidy Laidlaw, Luke Hewitt, Nari Johnson, Conrad Stosz, and Sarah Schwettmann","cited_arxiv_id":null,"evidence_quote":"Provides a case study of simulated user behavior compared with real deployment, supporting the need for validation of simulations."},{"cited_title":"Paech, Kevin Klyman, Stevie Chancellor, Eric Lin, Nick Haber, and Desmond C","cited_arxiv_id":null,"evidence_quote":"Demonstrates how real chat logs can reveal harmful patterns (delusional spirals), grounding the argument for real-world evidence."},{"cited_title":"Arora, Preston Bowman, Michael Sharman, Sahra Ghalebikesabi, Mikhail Trofimov, Vinnie Monaco, and Karan Singhal","cited_arxiv_id":null,"evidence_quote":"Highlights limitations of static benchmarks and the need for multi-turn, expert-informed evaluation, directly supporting the controlled evaluation section."},{"cited_title":"API Benchmark Scores Do Not Reliably Transfer to Chatbot Interfaces","cited_arxiv_id":"2609.08861","evidence_quote":"Offers a clinically validated framework for auditing chatbot behavior, cited as a model for connecting evaluation to clinical outcomes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Exemplifies recent state-level legislation restricting companion chatbots, used to anchor the policy discussion and demonstrate enacted mitigations."}],"review_version":1}