{"paper":{"title":"Prevalence Bounds for Edited Watermarked Text: Power Envelopes and Calibration Sensitivity","arxiv_id":"2611.17035","primary_cat":"stat.ML","submitted_at":"2026-11-12T16:21:31+00:00","version_compiled":"v1"},"notice":"Every quoted sentence is the authors' text from the arXiv listing abstract. Labels and probabilities are the outputs of classifier-1.13.0 on the question bank abstract-p1-20260923, exactly as returned. No sentence in this record was written by a language model.","schema":"pith.compiled-record/v0","status":"compiled","abstract":{"source":"arXiv listing abstract v1","sentences":[{"i":0,"p":1.0,"id":"s0","own":false,"role":"background","text":"Watermark detectors decide whether one document came from a watermarked language model.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.05,"main_claim_p":0.02,"quantitative":false,"quantitative_p":0.04,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.0,"background":1.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":1,"p":0.79,"id":"s1","own":false,"role":"background","text":"Auditors often need a different number: the share of a corpus that did.","hedge":{"p":0.76,"label":"asserted"},"own_p":0.07,"main_claim_p":0.03,"quantitative":false,"quantitative_p":0.13,"role_probabilities":{"gap":0.21,"method":0.0,"result":0.0,"background":0.79,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":2,"p":0.99,"id":"s2","own":true,"role":"contribution","text":"We study this share for Gumbel-max and green-list (KGW) watermarks when every document may have been edited by an unknown amount.","hedge":{"p":0.94,"label":"asserted"},"own_p":0.84,"main_claim_p":0.05,"quantitative":false,"quantitative_p":0.11,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.0,"background":0.01,"limitation":0.0,"future_work":0.0,"contribution":0.99,"interpretation":0.0}},{"i":3,"p":0.71,"id":"s3","own":true,"role":"result","text":"Without restrictions on the edits, the upper end of the identified set is 1.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.66,"main_claim_p":0.08,"quantitative":true,"quantitative_p":0.97,"role_probabilities":{"gap":0.05,"method":0.0,"result":0.71,"background":0.01,"limitation":0.23,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":4,"p":0.79,"id":"s4","own":true,"role":"method","text":"Combining a known least-favourable next-token distribution with a sequential coupling, we derive a power envelope indexed by three budgets: a retention floor, an entropy budget, and a budget for in-place substitutions, which for Gumbel-max make edited positions look less watermarked than human text.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.94,"main_claim_p":0.21,"quantitative":true,"quantitative_p":0.84,"role_probabilities":{"gap":0.0,"method":0.79,"result":0.02,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.19,"interpretation":0.0}},{"i":5,"p":0.53,"id":"s5","own":true,"role":"contribution","text":"The envelope gives an identified set that is sharp for the alternatives it dominates, finite-sample bounds on the realized share of a fixed corpus with either the exact null or a null calibrated on an independent human sample, and confidence sequences for streams.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.94,"main_claim_p":0.27,"quantitative":true,"quantitative_p":0.63,"role_probabilities":{"gap":0.0,"method":0.03,"result":0.44,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.53,"interpretation":0.0}},{"i":6,"p":1.0,"id":"s6","own":true,"role":"result","text":"In Gumbel-max simulations where the envelope binds, the bounds cover, and the threshold that maximizes the gap between envelope and null is generally as tight as twelve thresholds.","hedge":{"p":0.94,"label":"asserted"},"own_p":0.92,"main_claim_p":0.12,"quantitative":true,"quantitative_p":0.98,"role_probabilities":{"gap":0.0,"method":0.0,"result":1.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":7,"p":1.0,"id":"s7","own":true,"role":"result","text":"On text from OPT-1.3B and Qwen3-1.7B, 12,000 fresh human documents scored under three keys show no robust departure from the exact null, and all bounds covered in resampling trials from finite pools of documents with random edits, substitutions and paraphrases; at a share of 0.2 the exact-null bound is about [0.14,0.38].","hedge":{"p":1.0,"label":"asserted"},"own_p":0.94,"main_claim_p":0.28,"quantitative":true,"quantitative_p":0.99,"role_probabilities":{"gap":0.0,"method":0.0,"result":1.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":8,"p":0.97,"id":"s8","own":true,"role":"limitation","text":"These results are fragile in two ways we make precise.","hedge":{"p":0.99,"label":"asserted"},"own_p":0.83,"main_claim_p":0.07,"quantitative":false,"quantitative_p":0.05,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.02,"background":0.0,"limitation":0.97,"future_work":0.0,"contribution":0.0,"interpretation":0.01}},{"i":9,"p":0.61,"id":"s9","own":true,"role":"result","text":"LLM-paraphrased KGW pools fall below the envelope, yet at 2000 documents the bounds still covered in these trials.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.92,"main_claim_p":0.06,"quantitative":true,"quantitative_p":0.98,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.61,"background":0.0,"limitation":0.39,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":10,"p":0.77,"id":"s10","own":true,"role":"limitation","text":"A simulated null error of 0.01, comparable to the largest deviation we observed at our thresholds, is likewise harmless at 2000 documents but breaks exact-null coverage at 100,000; a null calibrated on an independent human sample keeps coverage there, but needs a large reference sample to stay narrow.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.9,"main_claim_p":0.12,"quantitative":true,"quantitative_p":0.99,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.22,"background":0.0,"limitation":0.77,"future_work":0.0,"contribution":0.0,"interpretation":0.01}}]},"compiled":{"main_claim":{"p":0.65,"text":"On text from OPT-1.3B and Qwen3-1.7B, 12,000 fresh human documents scored under three keys show no robust departure from the exact null, and all bounds covered in resampling trials from finite pools of documents with random edits, substitutions and paraphrases; at a share of 0.2 the exact-null bound is about [0.14,0.38].","location":"abstract s7","hedge_strength":{"p":1.0,"label":"asserted"}},"provenance":{"route":{"via":"TypeSafe API","model":"classifier-1.13.0","endpoint":"https://api.typesafe.ai/v1/systemone","provider":"TypeSafe"},"packing":"one request per paper","requests":1,"input_tokens":8523,"cost_usd_reported":0.0},"compiled_at":"2026-11-13","bank_version":"abstract-p1-20260923","model_version":"classifier-1.13.0","abstract_states":{"claims_first":0.05,"states_limitation":0.97,"human_participants":0.39,"controlled_comparison":0.85,"randomized_assignment":0.06,"states_quantitative_result":0.98,"compares_with_prior_results":0.71,"mentions_code_or_data_availability":0.03},"evidential_basis":{"p":0.65,"value":"proof or derivation","probabilities":{"argument":0.0,"literature":0.0,"proof_or_derivation":0.65,"analysis_of_existing_data":0.0,"observation_or_experiment":0.13,"simulation_or_computation":0.22}},"contribution_type":{"p":0.99,"value":"theoretical result","probabilities":{"review":0.0,"method_or_tool":0.01,"empirical_study":0.0,"simulation_study":0.0,"theoretical_result":0.99,"dataset_or_benchmark":0.0,"position_or_commentary":0.0}},"contribution_statement":{"p":0.76,"text":"Combining a known least-favourable next-token distribution with a sequential coupling, we derive a power envelope indexed by three budgets: a retention floor, an entropy budget, and a budget for in-place substitutions, which for Gumbel-max make edited positions look less watermarked than human text.","location":"abstract s4"}},"coverage":{"tier":"abstract","levels_run":["P abstract"],"not_examined":["full text","figures and tables","references and citing sentences","relations to other papers"],"sources_read":["arXiv listing abstract v1"],"levels_not_run":["S sections","T body sentences","C citations","R cross-paper relations"],"sentences_examined":11,"sentences_in_abstract":11},"measured_accuracy":{"url":"https://pith.science/accuracy","measured_on":"2026-10-02","method":"two independent model families labelled a fresh random sample blind; the figure is how often the published label matches their consensus","sentence_role_agreement":0.92,"main_claim_agreement":0.908,"typed_citation_agreement":0.933,"typed_citation_agreement_95_interval":[0.917,0.945],"review_comments_accurate":0.758,"review_comments_accurate_95_interval":[0.684,0.82],"review_caught_admitted_error_found_or_related":0.448,"review_comments_judged":{"judged_by":["gpt-5.6-luna","grok-4.7"],"judged_against":"Pith's own text copy of each paper: text extracted from the PDF, the first 90,000 characters","measured_on":"2026-10-04","accurate":113,"decided":149,"sampled_reviews":200,"split_by_lane":false,"by_lane":"https://pith.science/accuracy#by-lane"}}}