{"paper":{"title":"How Feedback-Free Self-Evolution Reshapes Agent Skills: A Systematic Study","arxiv_id":"2611.25534","primary_cat":"cs.AI","submitted_at":"2026-11-19T05:32:18+00:00","version_compiled":"v1"},"notice":"Every quoted sentence is the authors' text from the arXiv listing abstract. Labels and probabilities are the outputs of classifier-1.13.0 on the question bank abstract-p1-20260923, exactly as returned. No sentence in this record was written by a language model.","schema":"pith.compiled-record/v0","status":"compiled","abstract":{"source":"arXiv listing abstract v1","sentences":[{"i":0,"p":1.0,"id":"s0","own":false,"role":"background","text":"Agent skills are documents in which experts write down procedural knowledge, and models increasingly maintain them by rewriting them without feedback.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.06,"main_claim_p":0.03,"quantitative":false,"quantitative_p":0.06,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.0,"background":1.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":1,"p":0.99,"id":"s1","own":false,"role":"gap","text":"Such rewriting could adapt a skill to the model that uses it or erode the expert knowledge that makes it useful, yet how skills evolve under this feedback-free self-evolution has not been studied.","hedge":{"p":0.97,"label":"hedged_possibility"},"own_p":0.05,"main_claim_p":0.03,"quantitative":false,"quantitative_p":0.05,"role_probabilities":{"gap":0.99,"method":0.0,"result":0.0,"background":0.01,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":2,"p":0.98,"id":"s2","own":true,"role":"method","text":"We let models rewrite expert skills round after round without any task, run or judgment and run agents built on the same models with the rewritten skills.","hedge":{"p":0.87,"label":"asserted"},"own_p":0.96,"main_claim_p":0.03,"quantitative":false,"quantitative_p":0.09,"role_probabilities":{"gap":0.0,"method":0.98,"result":0.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.02,"interpretation":0.0}},{"i":3,"p":1.0,"id":"s3","own":true,"role":"result","text":"Our experiments show that the skills lose value rather than improve, as every model we test performs worse with them on every benchmark, by 15 points on average on SkillsBench after thirty rounds.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.97,"main_claim_p":0.79,"quantitative":true,"quantitative_p":0.99,"role_probabilities":{"gap":0.0,"method":0.0,"result":1.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":4,"p":0.67,"id":"s4","own":true,"role":"result","text":"Sampling each model's prior with skills it writes from the topic alone, we find that rewriting pulls skills toward this prior and gives up first the knowledge outside it.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.94,"main_claim_p":0.12,"quantitative":false,"quantitative_p":0.1,"role_probabilities":{"gap":0.0,"method":0.03,"result":0.67,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.02,"interpretation":0.28}},{"i":5,"p":0.94,"id":"s5","own":true,"role":"interpretation","text":"Putting this knowledge back recovers much of the lost pass rate, so an expert's skill is valuable largely for what the model does not write on its own.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.92,"main_claim_p":0.32,"quantitative":false,"quantitative_p":0.16,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.06,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.94}},{"i":6,"p":0.59,"id":"s6","own":true,"role":"contribution","text":"Building on this insight, SkillDiff marks this knowledge in place and raises the pass rate on SkillsBench by 4.8 points on average without feedback or any change to the skill's text.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.96,"main_claim_p":0.18,"quantitative":true,"quantitative_p":0.99,"role_probabilities":{"gap":0.0,"method":0.07,"result":0.34,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.59,"interpretation":0.0}},{"i":7,"p":0.46,"id":"s7","own":false,"role":"contribution","text":"Project repository: https://github.com/LitaoGuo/How-Self-Evolution-Reshapes-Skills.","hedge":{"p":0.94,"label":"unclear"},"own_p":0.31,"main_claim_p":0.02,"quantitative":false,"quantitative_p":0.05,"role_probabilities":{"gap":0.0,"method":0.12,"result":0.01,"background":0.14,"limitation":0.01,"future_work":0.26,"contribution":0.46,"interpretation":0.0}}]},"compiled":{"main_claim":{"p":0.86,"text":"Our experiments show that the skills lose value rather than improve, as every model we test performs worse with them on every benchmark, by 15 points on average on SkillsBench after thirty rounds.","location":"abstract s3","hedge_strength":{"p":1.0,"label":"asserted"}},"provenance":{"route":{"via":"TypeSafe API","model":"classifier-1.13.0","endpoint":"https://api.typesafe.ai/v1/systemone","provider":"TypeSafe"},"packing":"one request per paper","requests":1,"input_tokens":6321,"cost_usd_reported":0.0},"compiled_at":"2026-11-20","bank_version":"abstract-p1-20260923","model_version":"classifier-1.13.0","abstract_states":{"claims_first":0.48,"states_limitation":0.29,"human_participants":0.08,"controlled_comparison":0.84,"randomized_assignment":0.02,"states_quantitative_result":0.99,"compares_with_prior_results":0.73,"mentions_code_or_data_availability":0.95},"evidential_basis":{"p":0.99,"value":"observation or experiment","probabilities":{"argument":0.0,"literature":0.0,"proof_or_derivation":0.0,"analysis_of_existing_data":0.0,"observation_or_experiment":0.99,"simulation_or_computation":0.01}},"contribution_type":{"p":0.99,"value":"empirical study","probabilities":{"review":0.0,"method_or_tool":0.01,"empirical_study":0.99,"simulation_study":0.0,"theoretical_result":0.0,"dataset_or_benchmark":0.0,"position_or_commentary":0.0}},"contribution_statement":{"p":0.94,"text":"We let models rewrite expert skills round after round without any task, run or judgment and run agents built on the same models with the rewritten skills.","location":"abstract s2"}},"coverage":{"tier":"abstract","levels_run":["P abstract"],"not_examined":["full text","figures and tables","references and citing sentences","relations to other papers"],"sources_read":["arXiv listing abstract v1"],"levels_not_run":["S sections","T body sentences","C citations","R cross-paper relations"],"sentences_examined":8,"sentences_in_abstract":8},"measured_accuracy":{"url":"https://pith.science/accuracy","measured_on":"2026-10-02","method":"two independent model families labelled a fresh random sample blind; the figure is how often the published label matches their consensus","sentence_role_agreement":0.92,"main_claim_agreement":0.908,"typed_citation_agreement":0.933,"typed_citation_agreement_95_interval":[0.917,0.945],"review_comments_accurate":0.758,"review_comments_accurate_95_interval":[0.684,0.82],"review_caught_admitted_error_found_or_related":0.448,"review_comments_judged":{"judged_by":["gpt-5.6-luna","grok-4.7"],"judged_against":"Pith's own text copy of each paper: text extracted from the PDF, the first 90,000 characters","measured_on":"2026-10-04","accurate":113,"decided":149,"sampled_reviews":200,"split_by_lane":false,"by_lane":"https://pith.science/accuracy#by-lane"}}}