{"paper":{"title":"SciToolAgent-Evo: An Ontology-Aware Self-Evolving Agent for Open-World Scientific Tool Acquisition","arxiv_id":"2607.28692","primary_cat":"cs.AI","submitted_at":"2026-07-30T08:47:11+00:00","version_compiled":"v1"},"notice":"Every quoted sentence is the authors' text from the arXiv listing abstract. Labels and probabilities are the outputs of classifier-1.13.0 on the question bank abstract-p1-20260923, exactly as returned. No sentence in this record was written by a language model.","schema":"pith.compiled-record/v0","status":"compiled","abstract":{"source":"arXiv listing abstract v1","sentences":[{"i":0,"p":1.0,"id":"s0","own":false,"role":"background","text":"Large language model (LLM) agents have been increasingly adopted in scientific research for organizing and invoking specialized computational tools.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.05,"main_claim_p":0.02,"quantitative":false,"quantitative_p":0.09,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.0,"background":1.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":1,"p":0.67,"id":"s1","own":false,"role":"gap","text":"However, their reliance on predefined tool spaces with static semantics limits their applicability to open-world scientific workflows, where tool requirements, capabilities, and boundaries evolve dynamically.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.06,"main_claim_p":0.03,"quantitative":false,"quantitative_p":0.08,"role_probabilities":{"gap":0.67,"method":0.0,"result":0.0,"background":0.31,"limitation":0.02,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":2,"p":1.0,"id":"s2","own":true,"role":"contribution","text":"To this end, we propose SciToolAgent-Evo, an ontology-aware self-evolving agent for open-world scientific tool acquisition.","hedge":{"p":0.96,"label":"asserted"},"own_p":0.78,"main_claim_p":0.08,"quantitative":false,"quantitative_p":0.03,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":1.0,"interpretation":0.0}},{"i":3,"p":0.99,"id":"s3","own":true,"role":"method","text":"Driven by an evolving memory of skills, experiences, and an ontologized tool graph, it distills generalizable knowledge from contrastive trajectories during accumulation, whereas during inference, it formulates active requests and utilizes a LinUCB-based bandit gate to dynamically balance exploration and exploitation.","hedge":{"p":0.99,"label":"asserted"},"own_p":0.85,"main_claim_p":0.06,"quantitative":false,"quantitative_p":0.05,"role_probabilities":{"gap":0.0,"method":0.99,"result":0.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.01,"interpretation":0.0}},{"i":4,"p":0.94,"id":"s4","own":true,"role":"method","text":"Once a novel tool is acquired, its scientific ontology is completed online for seamless integration into the known graph.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.75,"main_claim_p":0.04,"quantitative":false,"quantitative_p":0.04,"role_probabilities":{"gap":0.0,"method":0.94,"result":0.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.06,"interpretation":0.0}},{"i":5,"p":0.94,"id":"s5","own":true,"role":"contribution","text":"Moreover, we introduce OpenSciToolBench, a benchmark containing 900 realistic tasks across four difficulty levels.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.93,"main_claim_p":0.03,"quantitative":true,"quantitative_p":0.99,"role_probabilities":{"gap":0.0,"method":0.06,"result":0.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.94,"interpretation":0.0}},{"i":6,"p":0.95,"id":"s6","own":true,"role":"result","text":"Extensive evaluations show that SciToolAgent-Evo achieves state-of-the-art performance, validating its robustness and generalization.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.96,"main_claim_p":0.86,"quantitative":true,"quantitative_p":0.64,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.95,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.05}}]},"compiled":{"main_claim":{"p":1.0,"text":"Extensive evaluations show that SciToolAgent-Evo achieves state-of-the-art performance, validating its robustness and generalization.","location":"abstract s6","hedge_strength":{"p":1.0,"label":"asserted"}},"provenance":{"route":{"via":"TypeSafe API","model":"classifier-1.13.0","endpoint":"https://api.typesafe.ai/v1/systemone","provider":"TypeSafe"},"packing":"one request per paper","requests":1,"input_tokens":5614,"cost_usd_reported":0.0},"compiled_at":"2026-09-23","bank_version":"abstract-p1-20260923","model_version":"classifier-1.13.0","abstract_states":{"claims_first":0.07,"states_limitation":0.1,"human_participants":0.04,"controlled_comparison":0.24,"randomized_assignment":0.02,"states_quantitative_result":0.82,"compares_with_prior_results":0.63,"mentions_code_or_data_availability":0.06},"evidential_basis":{"p":0.89,"value":"observation or experiment","probabilities":{"argument":0.0,"literature":0.0,"proof_or_derivation":0.0,"analysis_of_existing_data":0.0,"observation_or_experiment":0.89,"simulation_or_computation":0.11}},"contribution_type":{"p":1.0,"value":"method or tool","probabilities":{"review":0.0,"method_or_tool":1.0,"empirical_study":0.0,"simulation_study":0.0,"theoretical_result":0.0,"dataset_or_benchmark":0.0,"position_or_commentary":0.0}},"contribution_statement":{"p":0.96,"text":"To this end, we propose SciToolAgent-Evo, an ontology-aware self-evolving agent for open-world scientific tool acquisition.","location":"abstract s2"}},"coverage":{"tier":"abstract","levels_run":["P abstract"],"not_examined":["full text","figures and tables","references and citing sentences","relations to other papers"],"sources_read":["arXiv listing abstract v1"],"levels_not_run":["S sections","T body sentences","C citations","R cross-paper relations"],"sentences_examined":7,"sentences_in_abstract":7},"measured_accuracy":{"url":"https://pith.science/accuracy","measured_on":"2026-10-02","method":"two independent model families labelled a fresh random sample blind; the figure is how often the published label matches their consensus","sentence_role_agreement":0.92,"main_claim_agreement":0.908,"typed_citation_agreement":0.933,"typed_citation_agreement_95_interval":[0.917,0.945],"review_comments_accurate":0.758,"review_comments_accurate_95_interval":[0.684,0.82],"review_caught_admitted_error_found_or_related":0.448,"review_comments_judged":{"judged_by":["gpt-5.6-luna","grok-4.7"],"judged_against":"Pith's own text copy of each paper: text extracted from the PDF, the first 90,000 characters","measured_on":"2026-10-04","accurate":113,"decided":149,"sampled_reviews":200,"split_by_lane":false,"by_lane":"https://pith.science/accuracy#by-lane"}}}