{"paper":{"title":"From More to Better: Causal Inference and Digital Experimentation in the Age of AI","arxiv_id":"2611.16335","primary_cat":"cs.AI","submitted_at":"2026-11-12T05:56:52+00:00","version_compiled":"v1"},"notice":"Every quoted sentence is the authors' text from the arXiv listing abstract. Labels and probabilities are the outputs of classifier-1.13.0 on the question bank abstract-p1-20260923, exactly as returned. No sentence in this record was written by a language model.","schema":"pith.compiled-record/v0","status":"compiled","abstract":{"source":"arXiv listing abstract v1","sentences":[{"i":0,"p":1.0,"id":"s0","own":false,"role":"background","text":"Artificial intelligence (AI) is quickly expanding the capacity of organizations to produce and analyze data.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.05,"main_claim_p":0.02,"quantitative":false,"quantitative_p":0.06,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.0,"background":1.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":1,"p":0.36,"id":"s1","own":false,"role":"interpretation","text":"We argue that this expansion shifts the leverage of data scientists from performing individual analyses to assessing the credibility and importance of findings and determining when evidence generalizes across contexts.","hedge":{"p":0.96,"label":"asserted"},"own_p":0.2,"main_claim_p":0.52,"quantitative":false,"quantitative_p":0.37,"role_probabilities":{"gap":0.03,"method":0.0,"result":0.01,"background":0.3,"limitation":0.0,"future_work":0.0,"contribution":0.3,"interpretation":0.36}},{"i":2,"p":0.99,"id":"s2","own":true,"role":"contribution","text":"Drawing on a roundtable of academic researchers and industry practitioners, we examine three ways in which data scientists who work on causal inference and experimentation can help organizations turn more analytical capacity into better decisions: preserving and accumulating the learning value of experimentation beyond individual tests; choosing and combining experimental, observational, and synthetic evidence; and ensuring evaluation keeps pace with AI as its capabilities and uses evolve.","hedge":{"p":0.89,"label":"asserted"},"own_p":0.74,"main_claim_p":0.51,"quantitative":true,"quantitative_p":0.72,"role_probabilities":{"gap":0.0,"method":0.01,"result":0.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.99,"interpretation":0.0}},{"i":3,"p":0.57,"id":"s3","own":true,"role":"contribution","text":"In each area, we distinguish established practice from open questions.","hedge":{"p":0.91,"label":"asserted"},"own_p":0.78,"main_claim_p":0.04,"quantitative":false,"quantitative_p":0.26,"role_probabilities":{"gap":0.01,"method":0.4,"result":0.01,"background":0.0,"limitation":0.0,"future_work":0.01,"contribution":0.57,"interpretation":0.0}},{"i":4,"p":0.53,"id":"s4","own":true,"role":"interpretation","text":"We conclude with implications for the training and practice of data science and for organizations.","hedge":{"p":0.55,"label":"asserted"},"own_p":0.63,"main_claim_p":0.14,"quantitative":false,"quantitative_p":0.03,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.0,"background":0.0,"limitation":0.0,"future_work":0.44,"contribution":0.03,"interpretation":0.53}}]},"compiled":{"main_claim":{"p":0.79,"text":"We argue that this expansion shifts the leverage of data scientists from performing individual analyses to assessing the credibility and importance of findings and determining when evidence generalizes across contexts.","location":"abstract s1","hedge_strength":{"p":0.96,"label":"asserted"}},"provenance":{"route":{"via":"TypeSafe API","model":"classifier-1.13.0","endpoint":"https://api.typesafe.ai/v1/systemone","provider":"TypeSafe"},"packing":"one request per paper","requests":1,"input_tokens":4224,"cost_usd_reported":0.0},"compiled_at":"2026-11-13","bank_version":"abstract-p1-20260923","model_version":"classifier-1.13.0","abstract_states":{"claims_first":0.03,"states_limitation":0.26,"human_participants":0.33,"controlled_comparison":0.06,"randomized_assignment":0.04,"states_quantitative_result":0.02,"compares_with_prior_results":0.06,"mentions_code_or_data_availability":0.02},"evidential_basis":{"p":0.99,"value":"argument","probabilities":{"argument":0.99,"literature":0.01,"proof_or_derivation":0.0,"analysis_of_existing_data":0.0,"observation_or_experiment":0.0,"simulation_or_computation":0.0}},"contribution_type":{"p":1.0,"value":"position or commentary","probabilities":{"review":0.0,"method_or_tool":0.0,"empirical_study":0.0,"simulation_study":0.0,"theoretical_result":0.0,"dataset_or_benchmark":0.0,"position_or_commentary":1.0}},"contribution_statement":{"p":0.97,"text":"Drawing on a roundtable of academic researchers and industry practitioners, we examine three ways in which data scientists who work on causal inference and experimentation can help organizations turn more analytical capacity into better decisions: preserving and accumulating the learning value of experimentation beyond individual tests; choosing and combining experimental, observational, and synthetic evidence; and ensuring evaluation keeps pace with AI as its capabilities and uses evolve.","location":"abstract s2"}},"coverage":{"tier":"abstract","levels_run":["P abstract"],"not_examined":["full text","figures and tables","references and citing sentences","relations to other papers"],"sources_read":["arXiv listing abstract v1"],"levels_not_run":["S sections","T body sentences","C citations","R cross-paper relations"],"sentences_examined":5,"sentences_in_abstract":5},"measured_accuracy":{"url":"https://pith.science/accuracy","measured_on":"2026-10-02","method":"two independent model families labelled a fresh random sample blind; the figure is how often the published label matches their consensus","sentence_role_agreement":0.92,"main_claim_agreement":0.908,"typed_citation_agreement":0.933,"typed_citation_agreement_95_interval":[0.917,0.945],"review_comments_accurate":0.758,"review_comments_accurate_95_interval":[0.684,0.82],"review_caught_admitted_error_found_or_related":0.448,"review_comments_judged":{"judged_by":["gpt-5.6-luna","grok-4.7"],"judged_against":"Pith's own text copy of each paper: text extracted from the PDF, the first 90,000 characters","measured_on":"2026-10-04","accurate":113,"decided":149,"sampled_reviews":200,"split_by_lane":false,"by_lane":"https://pith.science/accuracy#by-lane"}}}