{"paper":{"title":"Is the Jeffreys' scale a reliable tool for Bayesian model comparison in cosmology?","arxiv_id":"1210.7652","primary_cat":"astro-ph.CO","submitted_at":"2012-10-29T13:25:12+00:00","version_compiled":"v2"},"notice":"Every quoted sentence is the authors' text from the arXiv listing abstract. Labels and probabilities are the outputs of classifier-1.13-20260917 on the question bank abstract-p1-20260923, exactly as returned. No sentence in this record was written by a language model.","schema":"pith.compiled-record/v0","status":"compiled","abstract":{"source":"arXiv listing abstract v2","sentences":[{"i":0,"p":1.0,"id":"s0","own":false,"role":"background","text":"We are entering an era where progress in cosmology is driven by data, and alternative models will have to be compared and ruled out according to some consistent criterium.","hedge":{"p":0.97,"label":"asserted"},"own_p":0.04,"main_claim_p":0.02,"quantitative":false,"quantitative_p":0.05,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.0,"background":1.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":1,"p":1.0,"id":"s1","own":false,"role":"background","text":"The most conservative and widely used approach is Bayesian model comparison.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.04,"main_claim_p":0.02,"quantitative":false,"quantitative_p":0.22,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.0,"background":1.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":2,"p":0.58,"id":"s2","own":true,"role":"method","text":"In this paper we explicitly calculate the Bayes factors for all models that are linear with respect to their parameters.","hedge":{"p":0.98,"label":"asserted"},"own_p":0.96,"main_claim_p":0.02,"quantitative":false,"quantitative_p":0.1,"role_probabilities":{"gap":0.0,"method":0.58,"result":0.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.42,"interpretation":0.0}},{"i":3,"p":0.66,"id":"s3","own":true,"role":"method","text":"We do this in order to test the so called Jeffreys' scale and determine analytically how accurate its predictions are in a simple case where we fully understand and can calculate everything analytically.","hedge":{"p":0.8,"label":"asserted"},"own_p":0.86,"main_claim_p":0.04,"quantitative":false,"quantitative_p":0.07,"role_probabilities":{"gap":0.0,"method":0.66,"result":0.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.34,"interpretation":0.0}},{"i":4,"p":0.63,"id":"s4","own":true,"role":"contribution","text":"We also discuss the case of nested models, e.g. one with $M_1$ and another with $M_2\\supset M_1$ parameters and we derive analytic expressions for both the Bayes factor and the Figure of Merit, defined as the inverse area of the model parameter's confidence contours.","hedge":{"p":0.98,"label":"asserted"},"own_p":0.95,"main_claim_p":0.03,"quantitative":false,"quantitative_p":0.23,"role_probabilities":{"gap":0.0,"method":0.37,"result":0.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.63,"interpretation":0.0}},{"i":5,"p":0.74,"id":"s5","own":true,"role":"result","text":"With all this machinery and the use of an explicit example we demonstrate that the threshold nature of Jeffreys' scale is not a \"one size fits all\" reliable tool for model comparison and that it may lead to biased conclusions.","hedge":{"p":0.84,"label":"asserted"},"own_p":0.95,"main_claim_p":0.94,"quantitative":false,"quantitative_p":0.11,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.74,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.26}},{"i":6,"p":0.9,"id":"s6","own":true,"role":"contribution","text":"Furthermore, we discuss the importance of choosing the right basis in the context of models that are linear with respect to their parameters and how that basis affects the parameter estimation and the derived constraints.","hedge":{"p":0.9,"label":"asserted"},"own_p":0.58,"main_claim_p":0.04,"quantitative":false,"quantitative_p":0.04,"role_probabilities":{"gap":0.0,"method":0.03,"result":0.01,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.9,"interpretation":0.06}}]},"compiled":{"main_claim":{"p":1.0,"text":"With all this machinery and the use of an explicit example we demonstrate that the threshold nature of Jeffreys' scale is not a \"one size fits all\" reliable tool for model comparison and that it may lead to biased conclusions.","location":"abstract s5","hedge_strength":{"p":0.84,"label":"asserted"}},"provenance":{"route":{"via":"OpenRouter","model":"classifier-1.13","endpoint":"https://openrouter.ai/api/v1/systemone","provider":"TypeSafe"},"packing":"one request per paper","requests":1,"input_tokens":5737,"cost_usd_reported":0.000241},"compiled_at":"2026-09-23","bank_version":"abstract-p1-20260923","model_version":"classifier-1.13-20260917","abstract_states":{"claims_first":0.05,"states_limitation":0.32,"human_participants":0.01,"controlled_comparison":0.06,"randomized_assignment":0.01,"states_quantitative_result":0.08,"compares_with_prior_results":0.54,"mentions_code_or_data_availability":0.02},"evidential_basis":{"p":1.0,"value":"proof or derivation","probabilities":{"argument":0.0,"literature":0.0,"proof_or_derivation":1.0,"analysis_of_existing_data":0.0,"observation_or_experiment":0.0,"simulation_or_computation":0.0}},"contribution_type":{"p":0.99,"value":"theoretical result","probabilities":{"review":0.0,"method_or_tool":0.0,"empirical_study":0.0,"simulation_study":0.0,"theoretical_result":0.99,"dataset_or_benchmark":0.0,"position_or_commentary":0.01}},"contribution_statement":{"p":0.8,"text":"In this paper we explicitly calculate the Bayes factors for all models that are linear with respect to their parameters.","location":"abstract s2"}},"coverage":{"tier":"abstract","levels_run":["P abstract"],"not_examined":["full text","figures and tables","references and citing sentences","relations to other papers"],"sources_read":["arXiv listing abstract v2"],"levels_not_run":["S sections","T body sentences","C citations","R cross-paper relations"],"sentences_examined":7,"sentences_in_abstract":7},"measured_accuracy":{"url":"https://pith.science/accuracy","measured_on":"2026-10-02","method":"two independent model families labelled a fresh random sample blind; the figure is how often the published label matches their consensus","sentence_role_agreement":0.92,"main_claim_agreement":0.908,"typed_citation_agreement":0.933,"typed_citation_agreement_95_interval":[0.917,0.945],"review_comments_accurate":0.758,"review_comments_accurate_95_interval":[0.684,0.82],"review_caught_admitted_error_found_or_related":0.448}}