Pith.

Artificial Intelligence · arXiv:2609.36927 v1 · submitted 29 Sep 2026 · 79 references

Neuro-Symbolic Computer Use: Learning Reusable Policies for Reliable and Efficient Execution

Hyewon Suh, Thanh Minh Nguyen, Chih-Lun Lee, Darrow Hartman, Lizhao Liu, Xin Eric Wang, Ang Li, Jiachen Yang

The paper says

“Many computer tasks recur: the same workflow runs many times, with new inputs and from different starting states. Current computer-use agents re-plan every step of every run, which makes them costly and unreliable on such tasks.”

more of the abstract

We introduce neuro-symbolic computer use, in which a recurring workflow is executed by a learned policy rather than re-derived by an agent on each run. The policy fixes the decisions that are stable across runs (ordering, variables, loops, and branches) in executable code, and delegates observation-dependent decisions, such as grounding and state checks, to neural models. We learn these policies with neuro-symbolic policy iteration: starting from one agent trajectory, it executes the policy, diagnoses failures with task-completion and step-level judges, and revises the code with a coding model informed by an agent's continuation from the point of failure, without access to the benchmark evaluator. Iterating on generated parameter and initial-state variants makes the policy reusable, and a pre-action verifier guards each state-mutating step at deployment. On OSWorld-Verified and ScienceBoard, the learned policies achieve the highest Pass^3 of all methods in all four settings, 3.6-15.8 points above the base agent, while cutting per-run cost by 15-217× and latency by 3.4-5.1×. On OSWorld-Verified, policies built only on variants transfer to the held-out original tasks, exceeding AutoRPA by 8.6-17.5 points in Pass^3.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 30 Sep 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. Externally measured Pass3 and cost claims hold up and are honestly disclosed; the judge-driven learning loop is the weakest validated piece, but raw evidence reaching the healer mitigates how much that matters. This line is wrong

3 major objections · 4 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 79 references say.

The paper says, at §4.1-4.3, Tables 1-2

The review did not quote this passage.

Pith says: Objection 1 of 3 · major

The headline claim 'highest Pass^3 of all methods in all four settings' rests on margins that are thin in two settings and no uncertainty is reported. On OSWorld with Opus 5, NSPI is 72.0 vs 68.4 for the base agent (3.6 points); on OSWorld with Terra, NSPI is 50.4 vs 47.1 for Batch2Code (3.3 points).

more of objection 1

With 361 tasks, the standard error of a 3.6-point Pass^3 gap is roughly 3 points, and the Table 2 generalization gaps (60.0 vs 51.4 on n=35 initial-state families) have even larger relative uncertainty. Since Pass^3 is estimated from only three trials per task, I request bootstrap confidence intervals or pairwise significance tests for the Table 1 and Table 2 comparisons, and a scoping of the 'highest Pass^3' sentence to what the data supports. The cost and latency claims (orders of magnitude) are unaffected by this concern.

The paper says, at §3.1.1, §4.7

The review did not quote this passage.

Pith says: Objection 2 of 3 · major

The paper never reports the termination causes of the policy-iteration loop: the fraction of final policies that passed the dual judge versus the fraction returned at the N=5 budget cap, nor how external Pass^3 correlates with termination cause. Given the low judge recall (Table 5: only 11.4-40.1% of externally reliable policies are accepted), a large share of final policies must have been rejected on every iteration and returned only when the budget ran out.

more of objection 2

This matters for two load-bearing claims: (i) the mechanism claim that judge acceptance drives reliability (as opposed to the loop's unguided search occasionally producing correct code at the last iteration), and (ii) the cost attribution in Table 3, where construction cost is attributed to judge-guided refinement. Reporting the pass-at-cap distribution and its correlation with external outcomes would resolve both; if cap-exhausted returns dominate, the execute-diagnose-revise narrative needs to be restated.

The paper says, at §4.7, Table 6, Appendix B.3

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.85):

“This limitation also hides the most common failure, a missing library, because the error appears only in the script output; our sample leaves these failures out, so the counts above are more favorable to the step-level judge.”

Pith says: Objection 3 of 3 · major

The step-level judge's localization, the healer's primary input in Eq. (4), is correct on 7/10 visible failures and wrong on all 6 invisible failures, and the annotation sample deliberately excludes the most common failure class (missing Python library), which is invisible to the step-level judge (b348).

more of objection 3

The paper acknowledges these counts are favorable to the judge, but it does not quantify the consequence: for the dominant failure class, the healer receives a wrong t* and an unmeasured fraction of incomplete diagnoses. This does not threaten the externally measured Pass^3, but it limits the claim that the judge diagnostics drive the improvement, and it should be stated explicitly in Section 4.7 rather than only in the appendix. A concrete test would be to measure healing success conditioned on t* being correct versus wrong on a sample that includes script-only failures.

4 minor objections

The paper says, at Abstract vs §1 (b30)

Pith says: minor objection 1 of 4

The abstract reports gains of 3.6-15.8 points over the base agent, which matches Table 1, but the introduction (b30) reports 4.5-13.9 points; the two ranges should be reconciled.

The paper says, at §4.2 (b117)

Pith says: minor objection 2 of 4

On OSWorld-Terra, Pass^1 for NSPI is 53.6 vs 55.9 for the base agent; the sentence 'its Pass^1 remains comparable to the base agent, so the flatter curve is not obtained by starting from a weaker policy' is mostly right, but the 2.3-point gap should be acknowledged rather than glossed, since the Pass^3 advantage there comes partly from a slightly lower starting point.

show 2 more

The paper says, at §2.1, Eq. (2)

Pith says: minor objection 3 of 4

The identity Pass^k = p^k assumes the k executions are independent; this independence assumption should be stated explicitly, since rollouts of a deterministic code policy may not be independent across runs in practice.

The paper says, at §3.2.1

Pith says: minor objection 4 of 4

Typo: 'the content of taks setup files' should read 'task setup files'.

What would settle it

Run NSPI on ~60–80 OSWorld tasks twice per task from identical seeds: once with the paper's LLM dual judge and once with the external benchmark scorer as an oracle judge at every iteration (same healer, same N/K budgets). Compare final Pass^3 and construction cost. Additionally disaggregate returned policies by termination cause (all rollouts passed the judge vs. N-budget exhaustion) and tabulate external success within each group. If the oracle-judge version yields materially higher Pass^3, the LLM judge is confirmed as the bottleneck and the reader's condition (judge calibration or shipped loop) stands; if Pass^3 is similar, the judge is not the load-bearing reliability mechanism, and the paper's described role for the judge needs re-characterization either way.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 30 Sep 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic agrees 1 min 13 s · 25,635 tokens read, 11,434 written
  3. Editor wrote the verdict above 26 s · 26,686 tokens read, 3,551 written
  4. Referee major revision: 3 major objections, 4 minor 1 min 5 s · 26,107 tokens read, 9,046 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

The headline results are real as measured, and the paper is unusually honest about its own weak spots. Pass3 is scored by the benchmark evaluators, not the internal judges, so judge errors can't inflate those numbers, and the cost ratios are internally consistent with Table 3. The paper flags its own limitations clearly — judge recall (11.4–40.1%, so most correct policies get rejected), the localizer being wrong on all 6 invisible failures, the verifier's 70.8% wrong-pass rate on perturbed states, and the OSWorld-V2 degradation where NSPI trails the base agent. That level of disclosure earns real credit.

What's actually new is narrower than the framing: trajectory-to-script conversion with iterative refinement is AutoRPA's territory, and the paper says so. The new pieces are Pass3-focused iteration (k-of-k rather than pass@1), variant-based intra-task generalization with a held-out original task, and the calibrated pre-action verifier with the token-logprob threshold. The generalization experiment — building a policy only on synthetic variants and testing on the held-out originals, beating AutoRPA by 8.6–17.5 points — is the cleanest result in the paper.

The stress test's concern about judge reliability is real but, on reading, somewhat softened. The localizer being wrong on t* doesn't doom the loop, because the healer receives the failed rollout, the error log, the screenshots, and an agent continuation from the failure state — not just the judge's diagnosis. The judge may function more as a noisy stopping signal than a precise critic, and the system appears robust to that. What the paper doesn't tell you is how often the final policy comes from a judge-accepted iteration versus a budget-exhausted return-of-last-policy, and with N=5 that distinction matters for the claim that iteration is what produces reliability. That breakdown is missing, and low judge recall inflating construction cost is acknowledged but quantified only indirectly.

Two things would firm this up: shipping the code and policy-iteration logs, and reporting the judged-return versus cap-exhausted-return split. Neither is a rejection-level problem. The economics of recurring workflow automation are a real niche, and the break-even analysis (3–27 runs) is the kind of claim practitioners can actually use. Deserves a serious referee with revisions requested, not a desk reject.

The strongest objection

The headline Pass^3 numbers are measured against external OSWorld/ScienceBoard evaluators, so judge errors cannot inflate them; at worst they push NSPI toward extra iterations and cap-exhausted failures. The load-bearing concern is the method's core mechanism. NSPI claims to learn policies by policy iteration in which the dual judge is the critic (Sec 3.1.2): it stops the loop (all K rollouts pass) and points the healer at the failing step. The evidence that this judge is a useful learning signal is thin. Table 5: recall 11.4–40.1%, so the judge rejects most externally reliable policies, and specificity 69.3–94.4%, meaning 5.6–30.7% of unreliable policies are accepted as final. Table 6, on 19 tasks, shows localizer t* correct on only 7/10 visible failures and wrong 6/6 on invisible ones; B.3 admits the sample excludes the most common failure (missing library), which is hidden from the step-level judge. With N=5 iterations and return-of-last-policy on budget exhaustion, the high final Pass^3 may be driven by cap-exhausted returns where the healer happened to produce correct code, rather than by judge acceptance. If judge acceptance is not the mechanism producing reliability, the stated execute-diagnose-revise loop and the construction-cost figures (Table 3, break-even 3.1–26.8 runs) rest on an unvalidated signal. This is a correctness risk for the method's central claim as a ground-truth-free iteration procedure, not a challenge to the externally measured numbers, which the authors also defend via the disclosed judge analysis.

I read the central claim as two-part: (1) NSPI policies achieve the highest externally measured Pass^3 in all four settings at 15–217x lower cost, and (2) NSPI is a ground-truth-free policy-iteration method in which a dual LLM judge supplies the evaluation and localization signal. Part (1) is defended: Pass^3 is computed by the benchmark evaluators on fresh trials, all methods get the same three-trial protocol, the reported judge analyses (Tables 5–6, B.3) are honestly disclosed and show only conservative bias that can lower, not raise, the measured numbers. I found no leakage that touches the Table 1 loop: the converter, healer, and judges receive the task and trajectories, not the evaluation script; only the Section 4.3 variant generator receives the evaluation script (B.2.1), which is an AutoRPA-comparison fairness issue specific to the transfer claim, not the main claim. Part (2) is where the argument is least secure, and here I agree with the reader. The judge is the loop's only stopping criterion and its only localization signal, yet the supporting evidence is weak: recall of 11.4–40.1% means the judge rejects most externally reliable policies, so many correct policies must be returned by budget exhaustion rather than acceptance; specificity of 69.3–94.4% means 5.6–30.7% of unreliable policies are accepted and returned and then fail externally, contributing construction failures; the step-level localization is validated on a 19-task sample that explicitly excludes the most common failure mode, and t* is wrong in 6/6 invisible-failure cases. The paper's claim that conservatism 'increases construction cost without hurting reliability' is not fully supported, because a false-accepted policy stops refinement early and becomes a final failure, and because low recall inflates C_pi in Table 3 and therefore the break-even counts. The reader correctly distinguishes this from the externally measured headline, and I agree with the CONDITIONAL verdict: the numbers stand, but the method's core mechanism should be demonstrated via judge calibration against the true scorer or by shipping the iteration loop, code, and data. My proposed oracle-judge ablation would directly determine whether the judge signal or the healer/code representation is carrying the reliability, which settles whether the reader's condition is the right one. I therefore keep the verdict as CONDITIONAL (UNCHANGED).

The referee's report

If the headline results hold, this is a substantial practical contribution: it turns the expensive per-run replanning of computer-use agents into a cheap, reliable, reusable artifact for recurring workflows, with an explicit amortized-cost model (Eq. 3) and break-even analysis. The evaluation design is methodologically sound at its core: the headline Pass^3 numbers are measured against the external OSWorld/ScienceBoard scorers, not the internal judges, so the main claims are not circular. The Pass^k estimator in Eq. (2) is correctly stated as unbiased, and the Batch2Code ablation (conversion-only, no iteration) plus the failed-seed recovery results (62 tasks, Figure 4) provide real evidence that iteration contributes to reliability beyond mere determinism. The paper is unusually honest about its weaknesses: the OSWorld-V2 shortfall, the judge recall numbers, the four judge limitations in Appendix B.3, and the verifier wrong-pass rates are all disclosed.

Pith's reading of the paper (a machine's summary, not the authors')

Contemporary computer-use agents replan every step of every run, so a workflow executed a thousand times is rediscovered each time. This paper argues that a recurring workflow should instead be compiled once into a neuro-symbolic policy: executable code that fixes the decisions stable across runs (ordering, variables, loops, branches), with neural primitives for the observation-dependent parts, such as grounding a click to screen coordinates or checking whether a condition holds. To build such a policy from a single agent trajectory without any ground-truth reward, it introduces neuro-symbolic policy iteration, an execute–diagnose–explore–revise loop driven by a dual judge — a task-completion checker that probes the machine's end state and a step-level judge that localizes the first divergence — and a coding model that repairs the policy. On OSWorld-Verified and ScienceBoard the learned policies achieve the highest Pass$^3$, the share of tasks solved in all three independent runs, in all four benchmark–model settings (3.6–15.8 points above the base agent), while cutting per-run cost 15–217$\times$ and latency 3.4–5.1$\times$, and policies learned only on synthetic parameter and initial-state variants transfer to held-out originals better than the AutoRPA script baseline. The economic claim is that the one-time construction cost is recovered after roughly 3–27 executions, after which every run is cheap.

What carries the argument. The central object is the neuro-symbolic policy: a program whose control flow (loops, conditionals, variables, recursion) is ordinary code, and whose observation-dependent steps call neural primitives — a grounding action such as \texttt{click("File menu")} that maps a natural-language description to screen coordinates, and a condition check \texttt{state\_satisfies(...)} that lets the code branch on the current screen. The argument is carried by the neuro-symbolic policy iteration loop, whose critic is a dual judge: a task-completion judge that probes persistent machine state with read-only tools (shell commands, file reads, database queries) to check whether the task was really done, and a step-level judge whose observer VLMs describe each screen transition blind to the task and the code's intent, followed by a localizer that returns the first divergent step and a named failure mode. On a failure, the base agent continues from the failed state and a coding-model healer rewrites the whole policy, and at deployment a pre-action verifier — a calibrated one-token VLM that blocks a state-mutating primitive when the renormalized probability of the \texttt{no} answer crosses threshold $\theta$ — guards each write to the machine. The dual judge is the load-bearing critic, because iteration converges to whatever the judges approve.

Core claim. Central claim: a workflow's durable structure can be separated from its state-dependent decisions and fixed in a neuro-symbolic program, so reliability under repetition is written into the policy rather than bought from an LLM on every run. NSPI starts from one base-agent trajectory, which need not even be successful, converts it into an initial code policy, and then iterates: execute the policy from fresh states, judge each rollout with the dual judge, ask the base agent to continue from the failure point, and have a coding model rewrite the policy from the diagnosis and the continuation — never consulting the benchmark evaluator until the returned policy is scored. The payoff is a flatter Pass$^1$$\to$Pass$^3$ curve, a 3.4-point average drop versus 12.4 points for the base agent, at 15–217$\times$ lower per-run cost, with policies refined only on variants still exceeding AutoRPA by 8.6–17.5 Pass$^3$ points on held-out originals. The authors state plainly that this holds across the two benchmarks and two base models tested, and that on the longer-horizon OSWorld-V2 the policies keep 96% of their Pass$^1$ at $k=3$ but trail the base agent by 5.6 points in strict Pass$^3$.

The premise it rests on. The loop converges to whatever the dual judge approves, and the judge is only as good as the probes it chooses to run: it accepts just 11.4–40.1% of genuinely reliable policies (Table 5), so a misaligned judge would inflate construction cost and could admit final policies that the external evaluator rejects.

If the paper is right (Pith's inferences, not the authors')

If the central claim holds, any recurring workflow — expense reports, invoice processing, spreadsheet formatting — becomes an amortized asset: one trajectory yields a policy whose construction cost is recovered in roughly 3–27 executions, after which each run is 15–217$\times$ cheaper and 3.4–5.1$\times$ faster than the planning agent.

Reliability under repetition improves because the procedure is fixed in code: Pass$^3$ beats every compared method in all four tested settings, with a flatter Pass$^1$$\to$Pass$^3$ drop (3.4 points on average versus 12.4 for the base agent).

Policies can be made reusable across task instances by refining them on synthetic parameter and initial-state variants and then freezing them; on OSWorld such policies transfer to held-out original tasks, exceeding AutoRPA by 8.6–17.5 Pass$^3$ points.

Failed seed trajectories are usable: across 356 tasks where the base agent failed all three attempts, NSPI still produced a policy that passed all three final runs on 62 tasks, at 18–31% higher healing cost.

A pre-action verifier can catch unanticipated environment drift at deployment, gating each state-mutating step behind a calibrated \textquotedblleft should this action run now?\textquotedblright{} check with an adjustable wrong-block versus wrong-pass trade-off.

The technique's value grows with the recurrence ratio and the price of the planner: the break-even count tracks the planner's per-run cost (Claude Opus 5 breaks even roughly 2–2.4$\times$ sooner in dollars than GPT-5.6 Terra), so the method should pay off most where frontier-LLM planning is priciest — a relation Table 3 shows but the paper does not generalize.

Because the paper's own judge accepts only 11.4–40.1% of genuinely reliable policies, construction cost — not final correctness — is the real bottleneck, and a better-calibrated judge should shorten the payback period and extend the method to long-horizon tasks, where the OSWorld-V2 result (losing 5.6 Pass$^3$ points to the base agent) is attributed partly to the step-level judge's inability to lo

The neuro-symbolic representation suggests a division of labor the paper does not exploit: the code part can be audited like software while the neural primitives can be swapped for stronger grounding or vision models without rewriting the procedure.

As a testable corollary, the same reliability-and-cost profile should appear in other settings that mix deterministic and state-dependent structure, such as multi-step web forms or data-cleaning pipelines, if the method transfers beyond desktop automation.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Many computer tasks recur: the same workflow runs many times, with new inputs and from different starting states. Current computer-use agents re-plan every step of every run, which makes them costly and unreliable on such tasks. We introduce neuro-symbolic computer use, in which a recurring workflow is executed by a learned policy rather than re-derived by an agent on each run. The policy fixes the decisions that are stable across runs (ordering, variables, loops, and branches) in executable code, and delegates observation-dependent decisions, such as grounding and state checks, to neural models. We learn these policies with neuro-symbolic policy iteration: starting from one agent trajectory, it executes the policy, diagnoses failures with task-completion and step-level judges, and revises the code with a coding model informed by an agent's continuation from the point of failure, without access to the benchmark evaluator. Iterating on generated parameter and initial-state variants makes the policy reusable, and a pre-action verifier guards each state-mutating step at deployment. On OSWorld-Verified and ScienceBoard, the learned policies achieve the highest Pass^3 of all methods in all four settings, 3.6-15.8 points above the base agent, while cutting per-run cost by 15-217× and latency by 3.4-5.1×. On OSWorld-Verified, policies built only on variants transfer to the held-out original tasks, exceeding AutoRPA by 8.6-17.5 points in Pass^3.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-09-30

The authors' main claim, as the classifier picked it out 0.99

“On OSWorld-Verified and ScienceBoard, the learned policies achieve the highest Pass^3 of all methods in all four settings, 3.6-15.8 points above the base agent, while cutting per-run cost by 15-217$\times$ and latency by 3.4-5.1$\times$.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.08 · states limitation 0.24 · human participants 0.04 · controlled comparison 0.93 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.98 · mentions code or data availability 0.04

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Many computer tasks recur: the same workflow runs many times, with new inputs and from different starting states. background 1.00 0.07 0.02 asserted 1.00
Current computer-use agents re-plan every step of every run, which makes them costly and unreliable on such tasks. background 0.77 0.06 0.04 asserted 1.00
We introduce neuro-symbolic computer use, in which a recurring workflow is executed by a learned policy rather than re-derived by an agent on each run. contribution 0.99 0.80 0.24 asserted 1.00
The policy fixes the decisions that are stable across runs (ordering, variables, loops, and branches) in executable code, and delegates observation-dependent decisions, such as grounding and state checks, to neural models. method 0.95 0.70 0.08 asserted 1.00
We learn these policies with neuro-symbolic policy iteration: starting from one agent trajectory, it executes the policy, diagnoses failures with task-completion and step-level judges, and revises the code with a coding model informed by an agent's continuation from the point of failure, without access to the benchmark evaluator. method 0.99 0.95 0.05 asserted 0.99
Iterating on generated parameter and initial-state variants makes the policy reusable, and a pre-action verifier guards each state-mutating step at deployment. method 0.98 0.90 0.06 asserted 1.00
On OSWorld-Verified and ScienceBoard, the learned policies achieve the highest Pass^3 of all methods in all four settings, 3.6-15.8 points above the base agent, while cutting per-run cost by 15-217$\times$ and latency by 3.4-5.1$\times$. result 1.00 0.96 0.58 asserted 1.00
On OSWorld-Verified, policies built only on variants transfer to the held-out original tasks, exceeding AutoRPA by 8.6-17.5 points in Pass^3. result 1.00 0.95 0.22 asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.93

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-30: 1 request, 6,507 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (8)
Figure 1 of the paper
Figure 1. Recurring computer workflows require reliable repeated execution with generalizability at low cost. In Neuro-Symbolic Policy Iteration: a baseline agent’s exploration seeds an initial neuro￾symbolic policy, which preserves the workflow’s stable structure in code while retaining neural primitives for observation-dependent decisions. Our framework iteratively executes, evaluates, diagnoses, and improves the policy with additional agent continuation. The resulting policy is reused across workflows and guarded with pre-action verifiers during run-time deployment. where Pass^k (Yao et…
Figure 2 of the paper
Figure 2. Example of NSPI converting agent trajectory into a reusable, abstracted neuro-symbolic policy and using judge feedback to identify and heal failures. The resulting policy combines state￾dependent control flow, task parameters in code, and neural primitives for grounding (Section 4.1). 3. Neuro-Symbolic Policy Iteration NSPI first learns a policy that solves a task reliably (Section 3.1) and then generalizes it beyond the instance it was learned on (Section 3.2);
Figure 3 of the paper
Figure 3. Repeated-execution reliability and execution cost on ScienceBoard with GPT-5.6-Terra. (a) NSPI maintains high Pass^k as k increases, while agent-based baselines degrade more sharply and the Batch2Code conversion-only baseline remains consistent but less accurate. (b) NSPI achieves highest Pass^3 with efficient per-run execution cost over baselines. Policy-construction cost is analyzed separately in Section 4.4. Terra Opus Terra Opus 0 20 40 60 80 100 p ass 3 (%) ScienceBoard OSWorld (a) Terra Opus Terra Opus 0 20 40 60 80 100 Judge rejects (%) ScienceBoard OSWorld (b) Terra Opus…
Figure 4 of the paper
Figure 4. NSPI can recover reliable policies even from unsuccessful seed trajectories, but at higher cost. (a) Pass^3, with base-agent performance on the same tasks. (b) Judge rejection rate during policy iteration. (c) One-time healing cost with the relative increase when initialization uses failed seeds. on ScienceBoard with GPT-5.6 Terra. NSPI maintains its success rate as 𝑘 increases while moving execution to a much lower-cost regime. Across all four settings, the resulting policies reduce per-run cost by 15-217x and run 3.4–5.1× faster than the corresponding base agents. 4.3. Policy…
Figure 5 of the paper
Figure 5. Policy structures associated with successful reuse across workflow instances. Labels show the number of successful policies out of total applicable policies
Figure 6 of the paper
Figure 6. A learned binary-output pre-action verifier 𝑉 runs before every state-mutating primitive during policy deployment. Reusing the plan’s most recent stashed screenshot and accessibility tree, 𝑉 emits top-20 answer-token logprobs; the yes/no log-mass is renormalized to 𝑝no and the action is blocked iff 𝑝no ≥ 𝜃. A block raises a script-level error at the offending line, triggering the fallback hierarchy (agent hand-off, code repair, halt). observations (the probe results of 𝐽task and the observer descriptions) separately from its interpreta￾tions (verdicts, failure hypotheses, and…
Figure 7 of the paper
Figure 7. Given an original target task and environment configuration (initial state and evaluation script), we call an LLM to generate synthetic task variants that differ in the initial state and task parameters to match. The validator rejects a parameter variant if its instruction is unchanged, if the rebound evaluation script uses a different scoring function, or if the script still compares a changed output against the original reference file, because a correct solution could never match that file. It marks variants that pass but carry a warning, e.g., an edited evaluation script on an…
Figure 8 of the paper
Figure 8. Pre-action verifier error rates at 𝜃 = 0.78. Rollout-level wrong-block rate on correct policies, by environment, base model used for initial trajectory seed, and verifier model. that does not exist: a “Blue” theme in GIMP and a photo in a folder that has none. Both judges suggest how to complete the task instead of reporting that it is infeasible. Fourth, 𝐽task trusts saved files and the output of its own tools more than the running application. Saved files can lag the state of the application, and tool output can contain text that is not in the file. B.4. Precondition checks The…

Figures are the authors', reproduced from the paper.

References (79)
  1. [1] Agent s: An open agentic framework that uses computers like a human , author=. International Conference on Learning Representations , volume= on Pith
  2. [2] Agent s2: A compositional generalist-specialist framework for computer use agents , author=. arXiv preprint arXiv:2504.00906 , year= on PithPith's reviewarXiv
  3. [3] Minghao Chen and Xinyi Hu and Zhou Yu and Yufei Yin , booktitle=. Auto. 2026 , url= on Pith
  4. [4] Web world models , author=. arXiv preprint arXiv:2512.23676 , year= on PithPith's reviewarXiv
  5. [5] Dynamic programming and markov processes. , author=. 1960 , publisher= on Pith
  6. [6] Fikes, Richard E and Nilsson, Nils J , journal= on Pith
  7. [7] Code as policies: Language model programs for embodied control , author=. 2023 IEEE International conference on robotics and automation (ICRA) , pages=. 2023 , organization= on Pith
  8. [8] Vimo: A generative visual gui world model for app agents , author=. arXiv preprint arXiv:2504.13936 , year= on PithPith's reviewarXiv
  9. [9] Growing with your embodied agent: A human-in-the-loop lifelong code generation framework for long-horizon manipulation skills , author=. arXiv preprint arXiv:2509.18597 , year= on PithPith's reviewarXiv
  10. [10] Code as agent harness , author=. arXiv preprint arXiv:2605.18747 , year= on PithPith's reviewarXiv
  11. [11] 2026 , howpublished = on Pith
  12. [12] ALRM: Agentic LLM for Robotic Manipulation , author=. arXiv preprint arXiv:2601.19510 , year= on PitharXiv
  13. [13] Reinforcement Learning: An Introduction , author =. 2018 , edition = on Pith
  14. [14] Lifelong language-conditioned robotic manipulation learning , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume= on Pith
  15. [15] Self-improving reactive agents based on reinforcement learning, planning and teaching , author=. Machine learning , volume=. 1992 , publisher= on Pith
  16. [16] Human-level control through deep reinforcement learning , author=. nature , volume=. 2015 , publisher= on Pith
  17. [17] Claude Opus 5 System Card , author =. 2026 , howpublished = on Pith
  18. [18] Sweigart, Al , year = on Pith
  19. [19] 2020 , howpublished = on Pith
  20. [20] 2004 , howpublished = on Pith
show 59 more
  1. [21] 2012 , howpublished = on Pith
  2. [22] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments , author=. 2024 , eprint= on Pith
  3. [23] OSWorld2. 0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks , author=. arXiv preprint arXiv:2606.29537 , year= on PithPith's reviewarXiv
  4. [24] WebArena: A Realistic Web Environment for Building Autonomous Agents , author=. ICLR , year= on Pith
  5. [25] VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks , author=. arXiv preprint arXiv:2401.13649 , year= on PithPith's reviewarXiv
  6. [26] Scienceboard: Evaluating multimodal autonomous agents in realistic scientific workflows , author=. arXiv preprint arXiv:2505.19897 , year= on PithPith's reviewarXiv
  7. [27] Mind2Web: Towards a Generalist Agent for the Web , author=. NeurIPS , year= on Pith
  8. [28] WorldGUI: An Interactive Benchmark for Desktop GUI Automation from Any Starting Point , author=. 2026 , eprint= on Pith
  9. [29] OS-Marathon: Benchmarking Computer-Use Agents on Long-Horizon Repetitive Tasks , author=. 2026 , eprint= on Pith
  10. [30] ProSoftArena: Benchmarking Hierarchical Capabilities of Multimodal Agents in Professional Software Environments , author=. 2025 , eprint= on Pith
  11. [31] SCUBA: Salesforce Computer Use Benchmark , author=. 2025 , eprint= on Pith
  12. [32] -bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains , author=. 2024 , eprint= on Pith
  13. [33] Evaluating Large Language Models Trained on Code , author=. 2021 , eprint= on Pith
  14. [34] OpenAI GPT-5 System Card , author=. 2025 , eprint= on Pith
  15. [35] Claude Sonnet 4.6 System Card , author=. 2026 , howpublished= on Pith
  16. [36] Kimi K2.5: Visual Agentic Intelligence , author=. 2026 , eprint= on Pith
  17. [37] Qwen3-VL Technical Report , author=. 2025 , eprint= on Pith
  18. [38] OpenCUA: Open Foundations for Computer-Use Agents , author=. 2025 , eprint= on Pith
  19. [39] UI-TARS: Pioneering Automated GUI Interaction with Native Agents , author=. 2025 , eprint= on Pith
  20. [40] Scaling Agents for Computer Use , author=. 2026 , eprint= on Pith
  21. [41] On the Reliability of Computer Use Agents , author=. arXiv preprint arXiv:2604.17849 , year= on PithPith's reviewarXiv
  22. [42] Horace He and Thinking Machines Lab , title =. Thinking Machines Lab: Connectionism , year = on Pith
  23. [43] Efficient Memory Management for Large Language Model Serving with PagedAttention , author=. Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles , year= on Pith
  24. [44] SGLang: Efficient Execution of Structured Language Model Programs , author=. 2024 , eprint= on Pith
  25. [45] Tabassi, Elham , year=. Artificial Intelligence Risk Management Framework (AI RMF 1.0) , url=. doi:10.6028/nist.ai.100-1 , abstractNote= on Pithdoi:10.6028/nist.ai.100-1
  26. [46] Ethics Guidelines for Trustworthy AI , author =. 2019 , url = on Pith
  27. [47] DO-178C Software Compliance for Aerospace & Defense , author =. 2023 , url = on Pith
  28. [48] Introduction to Functional Safety and IEC 61508 , author =. 2019 , url = on Pith
  29. [49] ISO 26262 Functional Safety Standard for Automotive Applications , author =. 2020 , url = on Pith
  30. [50] IEC 62304: Medical Device Software Life Cycle Processes , author =. 2022 , url = on Pith
  31. [51] Note on the Sampling Error of the Difference Between Correlated Proportions or Percentages , volume=. Psychometrika , author=. 1947 , pages=. doi:10.1007/BF02295996 , number= on Pithdoi:10.1007/bf02295996
  32. [52] Frank Wilcoxon , journal =. Individual Comparisons by Ranking Methods , urldate = on Pith
  33. [53] PAL: Program-aided Language Models , author=. 2023 , eprint= on Pith
  34. [54] Executable Code Actions Elicit Better LLM Agents , author=. 2024 , eprint= on Pith
  35. [55] Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks , author=. 2023 , eprint= on Pith
  36. [56] APRICOT: Active Preference Learning and Constraint-Aware Task Planning with LLMs , author=. 2024 , eprint= on Pith
  37. [57] Bayesian Preference Elicitation with Language Models , author=. 2024 , eprint= on Pith
  38. [58] Active Preference Inference using Language Models and Probabilistic Reasoning , author=. 2024 , eprint= on Pith
  39. [59] Reflexion: Language Agents with Verbal Reinforcement Learning , author=. 2023 , eprint= on Pith
  40. [60] Self-Refine: Iterative Refinement with Self-Feedback , author=. 2023 , eprint= on Pith
  41. [61] Generating code world models with large language models guided by monte carlo tree search , author=. Advances in Neural Information Processing Systems , volume= on Pith
  42. [62] Leveraging In-Context Learning for Language Model Agents , author=. 2025 , eprint= on Pith
  43. [63] Gdpval: Evaluating ai model performance on real-world economically valuable tasks , author=. International Conference on Learning Representations , volume= on Pith
  44. [64] Worldcoder, a model-based llm agent: Building world models by writing code and interacting with the environment , author=. Advances in Neural Information Processing Systems , volume= on Pith
  45. [65] Programmatically interpretable reinforcement learning , author=. International conference on machine learning , pages=. 2018 , organization= on Pith
  46. [66] Voyager: An open-ended embodied agent with large language models , author=. arXiv preprint arXiv:2305.16291 , year= on PithPith's reviewarXiv
  47. [67] Agent Workflow Memory , author=. Forty-second International Conference on Machine Learning , year= on Pith
  48. [68] Inducing Programmatic Skills for Agentic Tasks , author=. Second Conference on Language Modeling , year= on Pith
  49. [69] Agent world model: Infinity synthetic environments for agentic reinforcement learning , author=. arXiv preprint arXiv:2602.10090 , year= on PithPith's reviewarXiv
  50. [70] Generalized out-of-distribution detection: A survey , author=. International Journal of Computer Vision , volume=. 2024 , publisher= on Pith
  51. [71] WALL-e: World alignment by neurosymbolic learning improves world model-based LLM agents , author=. Advances in Neural Information Processing Systems , volume= on Pith
  52. [72] Quantifying Generalization in Reinforcement Learning , author=. Proceedings of the 36th International Conference on Machine Learning , volume=. 2019 , series= on Pith
  53. [73] Skillweaver: Web agents can self-improve by discovering and honing skills , author=. arXiv preprint arXiv:2504.07079 , year= on PithPith's reviewarXiv
  54. [74] Ui-venus-1.5 technical report , author=. arXiv preprint arXiv:2602.09082 , year= on PitharXiv
  55. [75] 2026 , howpublished= on Pith
  56. [76] Robotic process automation , author=. Business & information systems engineering , volume=. 2018 , publisher= on Pith
  57. [77] Bengio, Yoshua and LeCun, Yann , booktitle =. Scaling Learning Algorithms Towards on Pith
  58. [78] Hinton, Geoffrey E. and Osindero, Simon and Teh, Yee Whye , journal =. A Fast Learning Algorithm for Deep Belief Nets , volume = on Pith
  59. [79] Deep learning , author=. 2016 , publisher= on Pith
Cited by 2 Pith papers

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment