{"paper":{"title":"SEABench: Benchmarking Endogenous Misalignment In Self-Evolving Agents","arxiv_id":"2609.35596","primary_cat":"cs.CR","submitted_at":"2026-09-28T16:46:25+00:00","version_compiled":"v1"},"notice":"Every quoted sentence is the authors' text from the arXiv listing abstract. Labels and probabilities are the outputs of classifier-1.13.0 on the question bank abstract-p1-20260923, exactly as returned. No sentence in this record was written by a language model.","schema":"pith.compiled-record/v0","status":"compiled","abstract":{"source":"arXiv listing abstract v1","sentences":[{"i":0,"p":1.0,"id":"s0","own":false,"role":"background","text":"Self-evolving LLM agents have gained prominence for their ability to improve after deployment by modifying their harness, including their controller instructions, memory management protocols, and reusable tools and skills, in response to user and environment feedback.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.05,"main_claim_p":0.03,"quantitative":false,"quantitative_p":0.04,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.0,"background":1.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":1,"p":0.77,"id":"s1","own":false,"role":"background","text":"However, locally useful updates may persist into later tasks where they produce unsafe behavior, even without direct adversarial influence.","hedge":{"p":0.89,"label":"hedged_possibility"},"own_p":0.1,"main_claim_p":0.11,"quantitative":false,"quantitative_p":0.05,"role_probabilities":{"gap":0.23,"method":0.0,"result":0.0,"background":0.77,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":2,"p":0.92,"id":"s2","own":true,"role":"contribution","text":"To study this risk, we introduce SEABench, a benchmark for studying endogenous misalignment arising from agent self-evolution, with 48 longitudinal task sequences that span multiple evolution surfaces, task domains, and harm types in a rich personal-assistant environment.","hedge":{"p":0.97,"label":"asserted"},"own_p":0.91,"main_claim_p":0.06,"quantitative":true,"quantitative_p":0.95,"role_probabilities":{"gap":0.0,"method":0.08,"result":0.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.92,"interpretation":0.0}},{"i":3,"p":0.98,"id":"s3","own":true,"role":"method","text":"To account for the stochasticity inherent in agentic operations, we provide an adaptive trajectory discovery pipeline that probes for failures while preserving original task intent and supports causal attribution through paired non-evolving agents and attribution scores.","hedge":{"p":0.99,"label":"asserted"},"own_p":0.94,"main_claim_p":0.03,"quantitative":false,"quantitative_p":0.11,"role_probabilities":{"gap":0.0,"method":0.98,"result":0.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.02,"interpretation":0.0}},{"i":4,"p":1.0,"id":"s4","own":true,"role":"result","text":"Our evaluation across multiple recent LLMs, evolution surfaces, and harm types reveals that self-evolution indeed increases task completion rates but often at the cost of safety failures that are absent for paired non-evolving baseline agents.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.97,"main_claim_p":0.78,"quantitative":true,"quantitative_p":0.89,"role_probabilities":{"gap":0.0,"method":0.0,"result":1.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":5,"p":1.0,"id":"s5","own":true,"role":"result","text":"We also show that qualitatively different safety behaviors emerge across evolution surfaces and harm types.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.95,"main_claim_p":0.07,"quantitative":false,"quantitative_p":0.09,"role_probabilities":{"gap":0.0,"method":0.0,"result":1.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":6,"p":0.86,"id":"s6","own":true,"role":"result","text":"Further, we show that this divergence in safety behavior is reflected in agents' chain-of-thought reasoning, which yields an effective monitoring strategy that can mitigate unsafe behavior with a low false positive rate.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.95,"main_claim_p":0.16,"quantitative":false,"quantitative_p":0.22,"role_probabilities":{"gap":0.0,"method":0.01,"result":0.86,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.06,"interpretation":0.07}}]},"compiled":{"main_claim":{"p":1.0,"text":"Our evaluation across multiple recent LLMs, evolution surfaces, and harm types reveals that self-evolution indeed increases task completion rates but often at the cost of safety failures that are absent for paired non-evolving baseline agents.","location":"abstract s4","hedge_strength":{"p":1.0,"label":"asserted"}},"provenance":{"route":{"via":"TypeSafe API","model":"classifier-1.13.0","endpoint":"https://api.typesafe.ai/v1/systemone","provider":"TypeSafe"},"packing":"one request per paper","requests":1,"input_tokens":5781,"cost_usd_reported":0.0},"compiled_at":"2026-09-29","bank_version":"abstract-p1-20260923","model_version":"classifier-1.13.0","abstract_states":{"claims_first":0.07,"states_limitation":0.21,"human_participants":0.11,"controlled_comparison":0.98,"randomized_assignment":0.03,"states_quantitative_result":0.79,"compares_with_prior_results":0.93,"mentions_code_or_data_availability":0.07},"evidential_basis":{"p":0.98,"value":"observation or experiment","probabilities":{"argument":0.0,"literature":0.0,"proof_or_derivation":0.0,"analysis_of_existing_data":0.0,"observation_or_experiment":0.98,"simulation_or_computation":0.02}},"contribution_type":{"p":1.0,"value":"dataset or benchmark","probabilities":{"review":0.0,"method_or_tool":0.0,"empirical_study":0.0,"simulation_study":0.0,"theoretical_result":0.0,"dataset_or_benchmark":1.0,"position_or_commentary":0.0}},"contribution_statement":{"p":1.0,"text":"To study this risk, we introduce SEABench, a benchmark for studying endogenous misalignment arising from agent self-evolution, with 48 longitudinal task sequences that span multiple evolution surfaces, task domains, and harm types in a rich personal-assistant environment.","location":"abstract s2"}},"coverage":{"tier":"abstract","levels_run":["P abstract"],"not_examined":["full text","figures and tables","references and citing sentences","relations to other papers"],"sources_read":["arXiv listing abstract v1"],"levels_not_run":["S sections","T body sentences","C citations","R cross-paper relations"],"sentences_examined":7,"sentences_in_abstract":7},"measured_accuracy":{"url":"https://pith.science/accuracy","measured_on":"2026-10-02","method":"two independent model families labelled a fresh random sample blind; the figure is how often the published label matches their consensus","sentence_role_agreement":0.92,"main_claim_agreement":0.908,"typed_citation_agreement":0.933,"typed_citation_agreement_95_interval":[0.917,0.945],"review_comments_accurate":0.758,"review_comments_accurate_95_interval":[0.684,0.82],"review_caught_admitted_error_found_or_related":0.448,"review_comments_judged":{"judged_by":["gpt-5.6-luna","grok-4.7"],"judged_against":"Pith's own text copy of each paper: text extracted from the PDF, the first 90,000 characters","measured_on":"2026-10-04","accurate":113,"decided":149,"sampled_reviews":200,"split_by_lane":false,"by_lane":"https://pith.science/accuracy#by-lane"}}}