{"paper":{"title":"AI out-persuades expert humans in text-based dialogue","arxiv_id":"2606.16475","primary_cat":"cs.CY","submitted_at":"2026-06-15T09:40:28+00:00","version_compiled":"v2"},"notice":"Every quoted sentence is the authors' text from the arXiv listing abstract. Labels and probabilities are the outputs of classifier-1.13.0 on the question bank abstract-p1-20260923, exactly as returned. No sentence in this record was written by a language model.","schema":"pith.compiled-record/v0","status":"compiled","abstract":{"source":"arXiv listing abstract v2","sentences":[{"i":0,"p":1.0,"id":"s0","own":false,"role":"background","text":"Many societal decisions are settled by contests of persuasion.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.03,"main_claim_p":0.01,"quantitative":false,"quantitative_p":0.05,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.0,"background":1.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":1,"p":0.99,"id":"s1","own":false,"role":"gap","text":"Conversational AI is a powerful new entrant in these contests, but whether it can out-persuade skilled and highly incentivized humans has remained unclear.","hedge":{"p":0.5,"label":"asserted"},"own_p":0.05,"main_claim_p":0.03,"quantitative":false,"quantitative_p":0.21,"role_probabilities":{"gap":0.99,"method":0.0,"result":0.0,"background":0.01,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":2,"p":0.84,"id":"s2","own":true,"role":"method","text":"Here, in a series of four preregistered experiments (n = 18,978 conversations from 6,923 people), we pitted AI systems against a range of human persuaders, including laypeople, winners of a separately preregistered four-round online persuasion tournament, professional canvassers, and world championship debaters.","hedge":{"p":0.8,"label":"asserted"},"own_p":0.95,"main_claim_p":0.03,"quantitative":true,"quantitative_p":0.97,"role_probabilities":{"gap":0.0,"method":0.84,"result":0.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.16,"interpretation":0.0}},{"i":3,"p":1.0,"id":"s3","own":true,"role":"result","text":"We found that AI systems were reliably more persuasive than expert humans, even when expert humans chose their issues, researched in advance, underwent hours of live, structured practice, and were incentivized with {\\pounds}1,000 cash bonuses.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.97,"main_claim_p":0.77,"quantitative":true,"quantitative_p":0.86,"role_probabilities":{"gap":0.0,"method":0.0,"result":1.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":4,"p":0.91,"id":"s4","own":true,"role":"result","text":"In a follow-up study, AI's advantage persisted after experts received a coaching tool that let them practice against the AI that beat them, review their performance history, and see what AI would have said at key moments.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.94,"main_claim_p":0.05,"quantitative":false,"quantitative_p":0.12,"role_probabilities":{"gap":0.0,"method":0.01,"result":0.91,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.08,"interpretation":0.0}},{"i":5,"p":0.64,"id":"s5","own":true,"role":"interpretation","text":"We found converging evidence that AI's advantage stemmed from rapidly deploying larger quantities of information: after coaching, expert humans could tie an AI constrained to respond at human speeds and with human-length messages.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.96,"main_claim_p":0.07,"quantitative":true,"quantitative_p":0.51,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.36,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.64}},{"i":6,"p":0.99,"id":"s6","own":true,"role":"result","text":"In a final study, we show that AI's advantage extends to consequential real-world behavior: AI was more than 3x more effective than professional canvassers from a UK fundraising firm at raising real-money donations to Save the Children.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.96,"main_claim_p":0.12,"quantitative":true,"quantitative_p":0.99,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.99,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.01,"interpretation":0.0}},{"i":7,"p":0.89,"id":"s7","own":true,"role":"interpretation","text":"Together, these results establish that frontier AI systems out-persuade expert humans in text-based conversation, with significant implications for political communication.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.89,"main_claim_p":0.96,"quantitative":false,"quantitative_p":0.18,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.1,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.01,"interpretation":0.89}}]},"compiled":{"main_claim":{"p":0.98,"text":"Together, these results establish that frontier AI systems out-persuade expert humans in text-based conversation, with significant implications for political communication.","location":"abstract s7","hedge_strength":{"p":1.0,"label":"asserted"}},"provenance":{"route":{"via":"TypeSafe API","model":"classifier-1.13.0","endpoint":"https://api.typesafe.ai/v1/systemone","provider":"TypeSafe"},"packing":"one request per paper","requests":1,"input_tokens":6503,"cost_usd_reported":0.0},"compiled_at":"2026-11-09","bank_version":"abstract-p1-20260923","model_version":"classifier-1.13.0","abstract_states":{"claims_first":0.07,"states_limitation":0.16,"human_participants":0.98,"controlled_comparison":0.55,"randomized_assignment":0.08,"states_quantitative_result":0.98,"compares_with_prior_results":0.9,"mentions_code_or_data_availability":0.04},"evidential_basis":{"p":1.0,"value":"observation or experiment","probabilities":{"argument":0.0,"literature":0.0,"proof_or_derivation":0.0,"analysis_of_existing_data":0.0,"observation_or_experiment":1.0,"simulation_or_computation":0.0}},"contribution_type":{"p":1.0,"value":"empirical study","probabilities":{"review":0.0,"method_or_tool":0.0,"empirical_study":1.0,"simulation_study":0.0,"theoretical_result":0.0,"dataset_or_benchmark":0.0,"position_or_commentary":0.0}},"contribution_statement":{"p":0.98,"text":"Here, in a series of four preregistered experiments (n = 18,978 conversations from 6,923 people), we pitted AI systems against a range of human persuaders, including laypeople, winners of a separately preregistered four-round online persuasion tournament, professional canvassers, and world championship debaters.","location":"abstract s2"}},"coverage":{"tier":"abstract","levels_run":["P abstract"],"not_examined":["full text","figures and tables","references and citing sentences","relations to other papers"],"sources_read":["arXiv listing abstract v2"],"levels_not_run":["S sections","T body sentences","C citations","R cross-paper relations"],"sentences_examined":8,"sentences_in_abstract":8},"measured_accuracy":{"url":"https://pith.science/accuracy","measured_on":"2026-10-02","method":"two independent model families labelled a fresh random sample blind; the figure is how often the published label matches their consensus","sentence_role_agreement":0.92,"main_claim_agreement":0.908,"typed_citation_agreement":0.933,"typed_citation_agreement_95_interval":[0.917,0.945],"review_comments_accurate":0.758,"review_comments_accurate_95_interval":[0.684,0.82],"review_caught_admitted_error_found_or_related":0.448}}