Pith.

Artificial Intelligence · arXiv:2607.28692 v1 · submitted 30 Jul 2026 · 36 references

SciToolAgent-Evo: An Ontology-Aware Self-Evolving Agent for Open-World Scientific Tool Acquisition

Yuqi Tang, Chenyi Zhou, Libin Wang, Keyan Ding, Qiang Zhang, Huajun Chen

The paper says

“Large language model (LLM) agents have been increasingly adopted in scientific research for organizing and invoking specialized computational tools.”

more of the abstract

However, their reliance on predefined tool spaces with static semantics limits their applicability to open-world scientific workflows, where tool requirements, capabilities, and boundaries evolve dynamically. To this end, we propose SciToolAgent-Evo, an ontology-aware self-evolving agent for open-world scientific tool acquisition. Driven by an evolving memory of skills, experiences, and an ontologized tool graph, it distills generalizable knowledge from contrastive trajectories during accumulation, whereas during inference, it formulates active requests and utilizes a LinUCB-based bandit gate to dynamically balance exploration and exploitation. Once a novel tool is acquired, its scientific ontology is completed online for seamless integration into the known graph. Moreover, we introduce OpenSciToolBench, a benchmark containing 900 realistic tasks across four difficulty levels. Extensive evaluations show that SciToolAgent-Evo achieves state-of-the-art performance, validating its robustness and generalization.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 3 Aug 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. A solid systems paper with honest ablations, but the 'open-world acquisition' claim is partly an artifact of a searchable unknown-tool catalog; the method still transfers out-of-domain and deserves referee time. This line is wrong

3 major objections · 5 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 36 references say.

The paper says, at §4.1 Definition 1; Algorithm 1 line 26; Appendix H

The review did not quote this passage.

Pith says: Objection 1 of 3 · major

The open-world setting is modeled with a queryable TU. Definition 1 says each unknown tool u ∈ TU is "initially under-specified, with only a minimal set of basic attributes exposed," and Algorithm 1's exploration step is SearchUnknown(r_j, TU) — a retrieval over those exposed attributes.

more of objection 1

Appendix H shows that all baselines are given the same TU. Thus "unknown" means "not in the known graph but present in a retrievable index." The bandit gate decides when to retrieve from this index; it does not discover or synthesize tools absent from the environment. All headline results (Table 1, Figure 3, Figure 4, Table A2) are generated inside this setup. This is a legitimate but much narrower problem than "open-world scientific tool acquisition." The paper should either reframe the contribution as catalog-based acquisition, or add an experimental condition where unknown tools are not directly searc

The paper says, at §3.2–3.3; Appendix G

The review did not quote this passage.

Pith says: Objection 2 of 3 · major

The main benchmark's reference data and the final evaluator share the same model family as the evaluated agent. OpenSciToolBench's reference tool paths and questions are generated by GPT-5.1; final answer accuracy is judged by GPT-5-mini (Appendix G); the agent is GPT-5.4-mini.

more of objection 2

Human validation checks question clarity, level alignment, tool-chain coherence, and "answerability" — that an answer can be obtained — but does not independently verify the scientific correctness of the reference answer or of the agent's answer. Consequently, the reported 12.11/9.55-point gains on OpenSciToolBench could partly reflect agreement with GPT-family generation priors rather than verifiable scientific tool-use ability. Please (i) report exact-match or execution-derived correctness, (ii) add a human-verified gold-answer subset, and (iii) report judge agreement for the GPT-5-mini evaluation. The external

The paper says, at §5.5; Table 3; Table A1

The review did not quote this passage.

Pith says: Objection 3 of 3 · major

The full-pipeline row in Table 3 (and Table A1) is inconsistent with the SciToolAgent-Evo row in Table 1 for the same backbone and dataset. Table 3 reports Overall=67.22 with Level-3=60.81 and Level-4=36.84, while Table 1 reports Overall=64.28 with Level-3=58.82 and Level-4=32.89.

more of objection 3

If both are answer accuracy under the same split, one is wrong; if Table 3 is tool accuracy or a different configuration, the axis labels are missing. Since the ablation conclusions in §5.5 are the main evidence for component contributions, this discrepancy must be reconciled before the paper can be accepted.

5 minor objections

The paper says, at Tables 1–2, Appendix G

Pith says: minor objection 1 of 5

All results are reported as mean±std over 3 runs, yet several entries have 0.00 standard deviation. Please clarify whether these are identical due to deterministic decoding and provide seeds or significance tests for the main comparisons.

The paper says, at Appendix B3

Pith says: minor objection 2 of 5

The OOD experiments on m&m's and RestBench would be stronger if the authors specified whether knowledge accumulation and bandit updates are run on those benchmarks' training sets, and how TU is defined there. Without this, the "out-of-domain" claim is ambiguous.

show 3 more

The paper says, at Figure 3

Pith says: minor objection 3 of 5

Figure 3 shows four level accuracies plus overall accuracy for each split, but the text only discusses the overall number. Please add clear bar labels, error bars, and a pointer to which bar is "overall."

The paper says, at Eq. (5), Algorithm 1 line 26

Pith says: minor objection 4 of 5

The semantic similarity S(r_j, t) and the SearchUnknown retrieval mechanism are not operationalized. Please define the embedding/similarity used for both known-graph retrieval and unknown-catalog search.

The paper says, at §3.3, Appendix D

Pith says: minor objection 5 of 5

The human validation criterion "Answerability" only checks whether an answer can be obtained from the chain, not whether the stored reference answer is scientifically correct. Please report inter-annotator agreement and the number of samples rejected at each validation stage.

What would settle it

Construct an ablation where TU is not searchable: give the agent the known graph and a black-box execution oracle but no attribute descriptions of unknown tools; allow discovery only by generating candidate tool calls (e.g., from a package API or by writing code) and observing success/failure. Run SciToolAgent-Evo and the same baselines under this condition. If unknown-tool hit rate or overall answer accuracy falls to near the no-exploration baseline, the sparse-attribute catalog is the load-bearing assumption. Alternatively, a simpler probe: in Algorithm 1, replace SearchUnknown(rj, TU) with a function that returns the true tool only when rj contains its full description, and measure the drop on the 30/70 split. A large drop demonstrates that exposed minimal attributes — not the bandit policy — carry the open-world result.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 3 Aug 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic agrees 53 s · 22,897 tokens read, 5,557 written
  3. Editor wrote the verdict above 20 s · 23,782 tokens read, 1,938 written
  4. Referee major revision: 3 major objections, 5 minor 1 min 49 s · 23,106 tokens read, 11,028 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

What you should know up front: this is a real systems contribution, but the headline overstates what the math actually does. The unknown-tool set is queryable by construction. Definition 1 gives every unknown tool a sparse set of exposed basic attributes, and the explore step in Algorithm 1 is SearchUnknown(rj, TU) — a retrieval over exactly those attributes. So "acquisition" is better described as triggered retrieval from a partial index, not discovery of unobserved capabilities. The reader's stress-test note is correct, and it is load-bearing for the framing, though not for the engineering underneath.

The genuinely new piece is OpenSciToolBench: 900 tasks, four difficulty levels, human-validated, not present in the cited literature. The two-stage design — contrastive distillation during accumulation, LinUCB-gated acquisition with online ontology completion — is a coherent combination that goes beyond the component papers. I want to give real credit here: the ablations are systematic and isolate each module's contribution, the OOD results on m&m's and RestBench are the most convincing evidence that the mechanism transfers beyond the authors' own benchmark, and the appendices include the prompts and hyperparameters needed to reproduce the pipeline.

Now the soft spots, in proportion. The primary benchmark is self-generated: GPT-5.1 writes the questions and reference chains, GPT-5-mini scores the answers, and the agent uses the same model family. Part of the measured "SOTA" may be self-agreement rather than scientific competence. Baselines receive no accumulation phase, even though roughly fifteen points of Evo's margin comes from training on the benchmark's own split. Table 3's full-pipeline overall score (67.22) disagrees with Table 1's answer accuracy for the same configuration (64.28), which needs reconciliation. Level-4 reference chains include unseeded random generation, making fixed-reference scoring ill-posed. And no code or benchmark data is released. None of these are individually fatal, but together they mean the measured numbers should be read as "effective retrieval-triggering policy under favorable conditions," not open-world acquisition as advertised.

Who is this for? Anyone working on LLM tool use, agent evaluation, or AI-for-science. It deserves a serious referee, but with clear expectations: release code and data, re-run baselines with an accumulation phase, fix the table, and either soften the open-world language or redesign the unknown-tool set so it is genuinely unobservable. I would send it to review. As it stands, it is a good paper that oversells its central claim; with the artifacts and a reframing, it is a solid benchmark-plus-method contribution.

The strongest objection

The headline claim is acquisition of tools absent from a fixed tool space in open-world workflows. The formalization, however, makes unknown tools observable by construction: Definition 1 says each u in TU is 'under-specified, with only a minimal set of basic attributes exposed,' and Algorithm 1's exploration step is SearchUnknown(rj, TU), a retrieval over exactly those exposed attributes. Baselines are given the same TU (Appendix H). So 'unknown' means 'not in the known graph yet present in a queryable index,' and the bandit gate's explore action is triggered retrieval from that index rather than discovery of an unobserved capability. The paper's strongest evidence for open-world generalization — the 30/70 split at 59.01% and Unk. Tool Hit of 88.24% after accumulation — is generated entirely inside this searchable-catalog setup. If a genuinely open world contains tools that must be discovered (via code generation, external registries, or reasoning that a capability exists), SciToolAgent-Evo has no mechanism: SearchUnknown requires the target to be describable and indexed. The claimed 12.11/9.55-point SOTA margins therefore validate a retrieval-triggering policy, not open-world acquisition as the paper frames it. This is load-bearing because it determines what problem the paper has actually solved.

The reader's weakest assumption is the most load-bearing condition for the paper's central claim. The system's mechanisms — contrastive accumulation, LinUCB gate, ontology migration — are coherent, and the out-of-domain results on m&m's and RestBench suggest the retrieval-and-planning machinery transfers. But those OOD benchmarks are also closed tool-catalog tasks, so they do not repair the open-world framing. The numerical inconsistency between Table 3's full-pipeline 67.22 and Table 1's 64.28 answer accuracy is real and should be corrected, and the reliance on GPT-family generation/scoring for the primary benchmark is a secondary concern; however, external benchmarks and honest ablations show a mechanism beyond pure self-agreement. The observability premise is the gate: if unknown tools can be found only by genuinely unobservable actions, the exploration branch breaks. The proposed black-box-discovery experiment would settle whether the premise lands. Since the reader already conditionalizes on this assumption, my recommendation is no change.

The referee's report

The contribution is conditionally significant. The architecture is coherent, the ablations are systematic, and the paper provides detailed prompts, pseudocode, and cost analyses, which support reproducibility. The out-of-domain experiments on m&m's and RestBench are a genuine strength and suggest that the modular design may transfer beyond the in-house benchmark. However, the headline claim of "open-world scientific tool acquisition" is substantially narrower than stated: unknown tools are represented as a queryable sparse-attribute catalog, and the main benchmark is generated and judged within the same GPT family as the evaluated agent. If these issues are addressed by reframing or additional experiments, the work would be a solid contribution to catalog-based tool acquisition; in its present form, the scope is overstated.

Pith's reading of the paper (a machine's summary, not the authors')

SciToolAgent-Evo aims to make large-language-model-powered scientific agents work when the toolset they start with is incomplete. The authors' central claim is that an agent can acquire genuinely missing tools if it accumulates reusable skills and experience rules from contrastive rollouts, then at inference time uses a contextual-bandit gate to decide, for each tool need, whether to exploit a known tool or explore the unknown pool. On a new 900-task benchmark with four difficulty levels, the approach improves overall answer accuracy by about 12 and 10 points over the runner-up under two leading language-model backbones, with smaller but consistent gains on an existing scientific tool-use benchmark. Ablations show the biggest drops come from removing tool migration and knowledge accumulation, supporting the claim that the self-evolving memory, not just the planner, is what drives performance.

What carries the argument. The load-bearing mechanism is an evolving memory plus a linear contextual-bandit gate. The memory has three parts: a skill library (task-level workflow templates), an experience pool (domain-specific and general tool-selection rules), and an ontologized tool graph (known tools and their relationships, with each tool described by basic attributes and scientific applicability). The gate is the decision point: for every active tool request, it constructs a four-dimensional context vector from the retrieved candidates and the request itself, and uses a linear upper-confidence-bound (LinUCB) policy to choose between exploiting a known tool and exploring the unknown pool. This combination does thr

Core claim. Open-world tool acquisition is modeled as a decision between exploiting a known tool graph and exploring a pool of under-specified unknown tools. The agent's mechanism is a two-phase loop: it first distills reusable skills and experience rules from contrastive rollouts (a deterministic trial versus exploratory ones, with the critical divergence critiqued); then, during inference, it converts each task into atomic tool requests and runs a linear contextual-bandit gate on four features (known-tool feasibility, candidate margin, ontology alignment, prior exploration success) to pick explore or exploit. Successfully used unknown tools get their ontology completed and are migrated into the known

The premise it rests on. The whole open-world claim rests on unknown tools being at least minimally exposed in a queryable catalogue; if a missing tool is completely unobservable — no index entry, no searchable attributes — the exploration step in the algorithm cannot find it, and 'acquisition' reduces to deciding when to search a partial catalogue.

If the paper is right (Pith's inferences, not the authors')

If the design is correct, agents should get better with experience: the authors report planning accuracy and hit rates on initially unknown tools rising steadily across accumulation checkpoints.

The explore/exploit gate is doing real work: replacing it with a fixed threshold or with an LLM judge consistently lowers accuracy, especially on harder multi-tool tasks.

Tool acquisition is genuinely about discovery from a pool: when 70 percent of tools start unknown, accuracy is competitive; when only 40 percent start unknown, accuracy rises, which is consistent with the memory and gate becoming less essential as the known graph grows.

The same agent pattern transfers beyond scientific tools: out-of-domain experiments on multimodal tool use and RESTful API planning show the largest tool-F1 and completion-rate gains among the compared methods.

The method keeps inference cheap relative to interactive baselines, using fewer total tokens and lower API cost per task, so the accuracy gains are not bought with unbounded exploration.

The model of 'unknown' tools is a sparse catalogue: every unknown tool has a minimal set of exposed attributes and is findable through a search operation. That makes this an open-catalogue setting; a genuinely invisible tool (no attributes, no queryable index) is outside the current mechanism, and the strongest form of the open-world claim would need a test where some required tools are completely

Nothing in the mechanism is science-specific, so the same bandit-gated memory loop is a plausible general recipe for any domain with a large, evolving tool or API space; out-of-domain results hint at this, but a purpose-built test with adversarial tool descriptions would make it precise.

The four-dimensional context vector is hand-designed; learning these features end-to-end from the reward signal, or adding a measure of the agent's uncertainty about a request, is a natural next experiment that could strengthen the gate's decisions.

The accumulation phase is a parameter-free substitute for fine-tuning. A head-to-head comparison against a model fine-tuned on the same successful trajectories would reveal whether the memory mechanism captures most of the benefit or whether gradient updates add something the memory cannot.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Large language model (LLM) agents have been increasingly adopted in scientific research for organizing and invoking specialized computational tools. However, their reliance on predefined tool spaces with static semantics limits their applicability to open-world scientific workflows, where tool requirements, capabilities, and boundaries evolve dynamically. To this end, we propose SciToolAgent-Evo, an ontology-aware self-evolving agent for open-world scientific tool acquisition. Driven by an evolving memory of skills, experiences, and an ontologized tool graph, it distills generalizable knowledge from contrastive trajectories during accumulation, whereas during inference, it formulates active requests and utilizes a LinUCB-based bandit gate to dynamically balance exploration and exploitation. Once a novel tool is acquired, its scientific ontology is completed online for seamless integration into the known graph. Moreover, we introduce OpenSciToolBench, a benchmark containing 900 realistic tasks across four difficulty levels. Extensive evaluations show that SciToolAgent-Evo achieves state-of-the-art performance, validating its robustness and generalization.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-09-23

The authors' main claim, as the classifier picked it out 1.00

“Extensive evaluations show that SciToolAgent-Evo achieves state-of-the-art performance, validating its robustness and generalization.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.10 · human participants 0.04 · controlled comparison 0.24 · randomized assignment 0.02 · states quantitative result 0.82 · compares with prior results 0.63 · mentions code or data availability 0.06

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Large language model (LLM) agents have been increasingly adopted in scientific research for organizing and invoking specialized computational tools. background 1.00 0.05 0.02 asserted 1.00
However, their reliance on predefined tool spaces with static semantics limits their applicability to open-world scientific workflows, where tool requirements, capabilities, and boundaries evolve dynamically. gap 0.67 0.06 0.03 asserted 1.00
To this end, we propose SciToolAgent-Evo, an ontology-aware self-evolving agent for open-world scientific tool acquisition. contribution 1.00 0.78 0.08 asserted 0.96
Driven by an evolving memory of skills, experiences, and an ontologized tool graph, it distills generalizable knowledge from contrastive trajectories during accumulation, whereas during inference, it formulates active requests and utilizes a LinUCB-based bandit gate to dynamically balance exploration and exploitation. method 0.99 0.85 0.06 asserted 0.99
Once a novel tool is acquired, its scientific ontology is completed online for seamless integration into the known graph. method 0.94 0.75 0.04 asserted 1.00
Moreover, we introduce OpenSciToolBench, a benchmark containing 900 realistic tasks across four difficulty levels. contribution 0.94 0.93 0.03 asserted 1.00
Extensive evaluations show that SciToolAgent-Evo achieves state-of-the-art performance, validating its robustness and generalization. result 0.95 0.96 0.86 asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.89

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 5,614 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (4)
Figure 1 of the paper
Figure 1. Overview of OpenSciToolBench. The benchmark
Figure 2 of the paper
Figure 2. Overview of the SciToolAgent-Evo framework.
Figure 4 of the paper
Figure 4. Performance comparison across knowledge accu

Showing 3 of the paper's 4 figures. Figures are the authors', reproduced from the paper.

References (36)
  1. [1] The question must evaluate implicit grounding fromscientificterminologytoatool-capabilitychain on Pith
  2. [2] ChatGPT for robotics: Design principles and model abilities.Ieee Access, 12: 55682–55696. Wang, H.; Qian, C.; Zhong, W.; Chen, X.; Qiu, J.; Huang, S.; Jin, B.; Wang, M.; Wong, K.-F.; and Ji, H. 2025a. OTC: Optimal tool calls via reinforcement learning.arXiv–2504. Wang, L.; Ma,... on PithPith's reviewarXiv
  3. [3] tool_path should contain a single tool or a very short tool chain arranged in the correct execution order. 3.Thequestionmaynotdescribethesolvingstepsor Method LLM Calls↓Input Tokens↓Output Tokens↓Total Tokens↓Latency (s)↓API Cost / Task ($)↓ ReAct 8.2 72,115.85 427.85 72,543.7... on Pith
  4. [4] The Parameter field should contain only the sub- stance name or ID selected in the question on Pith
  5. [5] Ensure natural, diverse phrasing and avoid template-like wording. Strictly output the response invalid JSONformat only. The requiredJSONformat is as follows: { "tool_path": ["tool_selected", "xxx"], "question": "question text", "Parameter": "substance name" } Level-2 (2–4 tool... on Pith
  6. [6] Ensure natural, diverse phrasing and avoid template-like wording. Strictly output the response invalid JSONformat only. The requiredJSONformat is as follows: { "tool_path": ["tool_selected", "xxx"], "question": "question text", "Parameter": ["substance name 1", "substance name... on Pith
  7. [7] tool_path should contain a short tool chain ar- rangedinthecorrectexecutionorder,supportingen- tityrepresentationconversion,propertycomputation, or property comparison on Pith
  8. [8] The question must not directly mention any tool name,butshouldusestandardscientificterminology to clearly imply the target property to be evaluated; it must not describe the solving steps or reveal any tool outputs, numerical results, or answer hints on Pith
  9. [9] The Parameter field should contain only the ref- erenced substance names or IDs, with a consistent format on Pith
  10. [10] The question must not use comparison criteria that are unnatural in real scientific queries, such as median; instead, it should use more realistic scien- tific constraints such as maximum/minimum values, threshold-based judgments, or structural inclusion on Pith
  11. [12] The question must evaluate deep scientific rea- soning, including property derivation, comprehen- sive threshold-based judgment, candidate filtering, or selection of the optimal object satisfying all con- ditions on Pith
  12. [13] tool_path should contain a multi-tool chain ar- ranged in the correct execution order, supporting cross-attribute analysis and constraint-based reason- ing over multiple candidate objects on Pith
  13. [14] The question must not directly mention any tool name,butshouldusestandardscientificterminology toclearlydescribethecandidateobjects,targetprop- erties,andscientificconstraints;itmustnotdescribe thesolvingstepsorrevealanytooloutputs,numerical results, or answer hints. 4.ThePara... on Pith
  14. [15] The question must not use comparison criteria that are unnatural in real scientific queries, such as median; instead, it should use scientific constraints suchasmaximum/minimumvalues,threshold-based judgments, or structural inclusion on Pith
  15. [16] Ensure natural, diverse phrasing and avoid template-like wording. Strictly output the response invalid JSONformat only. The requiredJSONformat is as follows: { "tool_path": ["tool_selected", "xxx"], "question": "question text", "Parameter": ["candidate substance 1", "candidate... on Pith
  16. [17] tool_path should contain a relatively long tool chain arranged in a reasonable execution order, sup- portingmultidimensionalanalysis,evidenceintegra- tion, and scientific decision-making on Pith
  17. [18] The question must not directly mention any tool name,butshouldusestandardscientificterminology to describe the research background, analysis objec- tive, and evaluation criteria; it must not describe the specific solving steps or reveal any tool outputs, nu- merical results, o... on Pith
  18. [19] The Parameter field should contain the specific substances, sequences, or other scientific objects se- lected in the question. 6.Ensurenatural,coherent,andsituatedinarealistic research context. Strictly output the response invalid JSONformat only. The requiredJSONformat is as ... on Pith
  19. [20] Check whether the question is clearly stated and free of obvious ambiguity on Pith
  20. [21] Check whether tool_path covers the scientific op- erations required by the question on Pith
show 16 more
  1. [22] Check whether the input-output dependencies be- tween adjacent tools in tool_path are valid and can form an executable chain on Pith
  2. [23] Judge as “No” if any condition fails; otherwise, “Yes”. Question: {question} Reference tool chain: {tool_path} Parameter: {parameter} Strictly output the response invalid JSONformat only. The requiredJSONformat is as follows: { "decision": "Yes / No", "reason": "brief explanat... on Pith
  3. [24] During this stage, locally deployed Qwen3-Embedding- 0.6B is used as the retrieval model (max_tools= 5), while Qwen3-1.7B is used as the reranking model. We implement a LinUCB-based bandit gate with a 4D context vector. The explorationcoefficientαinitializesat0.8anddecaysby0.0... on Pith
  4. [25] Each tool capability must be an atomic computa- tional primitive, used only to derive one property or perform one transformation. It must not merge mul- tiple properties or operations, nor include decision logic such as comparison, filtering, or aggregation. 2.Usethefewestandm... on Pith
  5. [26] If the task involves state or representation trans- formation, tool capabilities must follow data-form evolution, ensuring causal coherence between steps without skipping intermediate forms on Pith
  6. [27] Tool descriptions should be generic capability de- scriptions and must not include specific substance names, numerical values, candidate answers, task conclusions, or instantiated information. 5.Ifthetaskinvolvesmultiplesubstances,sequences, or scientific objects, describe onl... on Pith
  7. [28] Use goal-orientedbackward reasoning by firstde- termining the final output internally, then inferring the required inputs and tool capabilities, and finally outputting only the tool capability descriptions. 7.Eachtooldescriptionmustbehighlyconcise,with no more than 15 English ... on Pith
  8. [29] known_top1_feasibility(0.0–1.0): the overall feasibility of top-1. •1.0: Function fully matches, with aligned I/O and Category. •0.75: Function strongly matches, with minor I/O or Category mismatch. •0.5:Functionpartiallymatches,withmoderateI/O or Category mismatch. •0.25: Fun... on Pith
  9. [30] general_prompt:One general principle applica- ble to scientific tool planning on Pith
  10. [31] domain:Strictly choose one from the following: biology, chemistry, materials on Pith
  11. [32] domain_prompt:A robust tool mapping rule tar- geting the diagnosed error. The core expression must follow: “When requiring [specific scientific context/con- straint], select a tool that provides [specific func- tional capability].” [RULE]: 1.Allrulesmustprovideclearandactionab... on Pith
  12. [33] Limit each rule to 25 words. Strictly output the response invalid JSONformat only. The requiredJSONformat is as follows: { "general_prompt": "1 universal actionable rule", "domain": "biology/chemistry/materials", "domain_prompt": "domain-specific rule" } Prompt for Generating ... on Pith
  13. [34] Remove concrete parameters: do not include nu- merical values, thresholds, filtering conditions, or specific substance names on Pith
  14. [35] Skill naming: the key Ni should use short snake_case and reflect only the property/function combination on Pith
  15. [36] Skill description: the key Zi must be a coherent sequence of atomic computational primitives. Use connectives to form one fluent description within 20 words. Strictly output the response invalid JSONformat only. The requiredJSONformat is as follows: { "Ni": "property_combinati... on Pith
  16. [2024] Appworld: A controllable world of apps and people for benchmarking interactive coding agents. InProceedings ofthe62ndAnnualMeetingoftheAssociationforComputa- tional Linguistics (Volume 1: Long Papers), 16022–16076. Vemprala, S. H.; Bonatti, R.; Bucker, A.; and Kapoor, A on Pith
Cited by 2 Pith papers

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment