PatchWorld: Gradient-Free Optimization of Executable World Models for Agent Environments

Haoyu Huang; Hong Ting Tsang; Jeff Z. Pan; Jiaxin Bai; Jiaxuan Xiong; Lihui Liu; Tianqing Fang; Tianshi Zheng; Yangqiu Song; Yifei Dong

REVIEW 1 major objections 1 minor 2 cited by

Executable Python world models induced from trajectories enable effective planning in POMDPs without internal LLM calls.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-30 10:57 UTC pith:DJM62QHS

load-bearing objection PatchWorld shows a workable gradient-free code-repair method for turning trajectories into inspectable Python world models that support planning in text POMDPs, though the partial-observability handling needs closer checks. the 1 major comments →

arxiv 2605.30880 v4 pith:DJM62QHS submitted 2026-05-29 cs.CL cs.AI

PatchWorld: Gradient-Free Optimization of Executable World Models for Agent Environments

Jiaxin Bai , Yue Guo , Yifei Dong , Jiaxuan Xiong , Tianshi Zheng , Yixia Li , Tianqing Fang , Yufei Li

show 8 more authors

Yisen Gao Haoyu Huang Zhongwei Xie Hong Ting Tsang Zihao Wang Lihui Liu Jeff Z. Pan Yangqiu Song

This is my paper

classification cs.CL cs.AI

keywords executable world modelsPOMDPscode repairplanningtext agentsbelief statespartial observabilitygradient-free optimization

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper aims to establish that a gradient-free framework can convert offline trajectories into executable Python code serving as world models for partially observable environments. This is done through counterexample-guided code repair to create symbolic belief-state programs that are inspectable and patchable. If successful, this approach allows for planning with higher success rates while avoiding LLM calls during prediction, and it reveals a tradeoff between observation fidelity and decision utility. Readers would care because it provides a transparent, repairable alternative to traditional black-box models in agent planning tasks.

Core claim

PatchWorld induces executable Python world models from offline trajectories using counterexample-guided code repair, resulting in symbolic belief-state programs whose action updates enable effective planning under partial observability, with the Simple variant reaching 76.4% macro success in live one-step lookahead across seven environments without LLM calls inside the prediction module.

What carries the argument

Counterexample-guided code repair for generating symbolic belief-state programs from trajectories.

Load-bearing premise

Offline trajectories contain enough information for counterexample-guided code repair to produce executable world models whose action updates support effective planning under partial observability.

What would settle it

If the induced executable models are deployed in planning and fail to achieve higher success rates than black-box alternatives in the AgentGym environments, the central claim would be falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Symbolic programs enable inspection, replay, and local patching of world models.
Planning achieves high success rates with no internal LLM invocations.
A residual-memory bias improves observation fidelity but reduces decision utility, exposing a fidelity-utility tradeoff.
The method outperforms other evaluated approaches in code-based planning scores.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The approach could be tested in environments with more complex partial observability to see if code repair scales.
Automated methods for selecting biases might resolve the observed tradeoff between fidelity and utility.
This framework might connect to program synthesis techniques for other decision problems beyond text agents.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

PatchWorld shows a workable gradient-free code-repair method for turning trajectories into inspectable Python world models that support planning in text POMDPs, though the partial-observability handling needs closer checks.

read the letter

PatchWorld turns offline trajectories into executable Python world models using counterexample-guided code repair. It reports 76.4% macro success on one-step lookahead planning across seven AgentGym environments, with no LLM calls inside the prediction module.

The new piece is the gradient-free loop that produces symbolic belief-state programs instead of black-box predictors. The work does well by shipping code, running a clear comparison against other code-based methods, and explicitly calling out the fidelity-utility tradeoff when a residual-memory bias is added. That tradeoff observation is honest and worth having on record.

The soft spot is the POMDP partial-observability issue. The abstract gives no detail on how counterexamples are extracted or whether the trajectories contain enough distinguishing observations to recover action-conditioned latent dynamics. If the data are ambiguous, the repair process could converge to programs that match surface observations but misrepresent the transitions needed for planning. The single success number also lacks visible variance or controls, which makes the result harder to weigh.

This paper is for people working on symbolic or code-based world models for agents. Readers who want inspectable models and are willing to accept the current experimental limits will find the framework and the tradeoff discussion useful.

It deserves a serious referee. The method is concrete, the code is public, and the tradeoff point engages a real tension in the area. More experimental detail would strengthen it, but the core idea is worth review time.

Referee Report

1 major / 1 minor

Summary. The paper introduces PatchWorld, a gradient-free framework that induces executable Python world models from offline trajectories via counterexample-guided code repair for POMDP text-agent environments. PatchWorld-Simple is reported to achieve the highest code-based planning score, with 76.4% macro success in live one-step lookahead across seven AgentGym environments, without LLM calls inside the world-model prediction module. A human-specified residual-memory bias is shown to improve observation fidelity at the cost of decision utility, exposing a fidelity-utility tradeoff in executable world models.

Significance. If the central claims hold under rigorous validation, the work is significant for demonstrating that symbolic, inspectable belief-state programs can be induced for planning under partial observability, offering an alternative to black-box models with advantages in replayability and local patching. Code availability is a positive for reproducibility. The identified tradeoff between fidelity and utility is a useful conceptual contribution for future executable world-model research.

major comments (1)

[Abstract] The headline 76.4% macro success claim (abstract) rests on the assumption that counterexample-guided repair from offline trajectories recovers action-conditioned latent dynamics sufficiently for planning. In POMDPs the same observation sequence can arise from multiple latent trajectories; without explicit analysis or experiments showing that the collected trajectories contain distinguishing counterexamples for decision-relevant quantities, it is unclear whether the repaired programs avoid matching surface observations while misrepresenting dynamics. This is load-bearing for the performance comparison.

minor comments (1)

The abstract states that 'code is available' but provides no details on reproduction steps, environment versions, or exact data splits used for the 76.4% figure; adding these would improve clarity.

Simulated Author's Rebuttal

1 responses · 0 unresolved

Thank you for your review and the recommendation for major revision. We address the major comment regarding the abstract claim as follows.

read point-by-point responses

Referee: [Abstract] The headline 76.4% macro success claim (abstract) rests on the assumption that counterexample-guided repair from offline trajectories recovers action-conditioned latent dynamics sufficiently for planning. In POMDPs the same observation sequence can arise from multiple latent trajectories; without explicit analysis or experiments showing that the collected trajectories contain distinguishing counterexamples for decision-relevant quantities, it is unclear whether the repaired programs avoid matching surface observations while misrepresenting dynamics. This is load-bearing for the performance comparison.

Authors: We agree that in POMDPs, observation sequences may not uniquely determine the latent trajectory, raising the possibility that programs could match observations without correctly capturing the underlying dynamics. Our approach mitigates this through counterexample-guided repair, where the program is iteratively corrected based on execution mismatches with the full trajectory data, including action effects on subsequent observations. This encourages the program to learn action-conditioned updates rather than mere surface matching. Furthermore, the evaluation involves live one-step lookahead planning, where the model's predictions directly influence action selection in the real environment; the high success rate of 76.4% suggests that the dynamics are captured at a level sufficient for planning. We did not include an explicit analysis of distinguishing counterexamples in the submitted manuscript. We will add a discussion of this point and any supporting experiments in the revision. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper describes PatchWorld as inducing executable symbolic belief-state programs from offline trajectories via counterexample-guided code repair, with performance evaluated on separate live one-step lookahead planning tasks (76.4% macro success). No equations, predictions, or claims in the abstract or description reduce the central results to fitted parameters by construction, self-definitional loops, or load-bearing self-citations. The method is presented as relying on external data and iterative repair without the reported planning utility being equivalent to the input trajectories. This is the most common honest finding for self-contained empirical frameworks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

No details on free parameters, axioms, or invented entities are available from the abstract alone.

pith-pipeline@v0.9.1-grok · 5790 in / 1098 out tokens · 47668 ms · 2026-06-30T10:57:34.565618+00:00 · methodology

0 comments

read the original abstract

World models for interactive text agents must typically be learned from observation-action trajectories alone. Specifically, the environment returns text observations after each action, but does not expose a ground-truth latent state nor an inspectable transition model.A research gap remains in how to induce executable code as a world model in this black-box setting for prediction and agent decision making. We introduce PatchWorld, a gradient-free framework that turns offline trajectories into executable Python world models through counterexample-guided code repair.Instead of predicting the next observation with a black-box model, PatchWorld induces symbolic belief-state programs whose action updates can be inspected, replayed, and locally patched. Across seven AgentGym environments, PatchWorld-Simple achieves the highest code-based decision-making score among evaluated methods (76.4% macro success in live one-step lookahead), matching or exceeding LLM-based lookahead while invoking no LLM calls inside the world-model prediction module itself. We further find that a human-specified residual-memory bias improves surface observation fidelity but weakens agent decision-making utility. This reveals a tradeoff in executable world models, since improving observation fidelity can come at the expense of action-discriminative dynamics, and vice versa. Code is available at https://github.com/HKBU-KnowComp/PatchWorld.

Figures

Figures reproduced from arXiv: 2605.30880 by Haoyu Huang, Hong Ting Tsang, Jeff Z. Pan, Jiaxin Bai, Jiaxuan Xiong, Lihui Liu, Tianqing Fang, Tianshi Zheng, Yangqiu Song, Yifei Dong, Yisen Gao, Yixia Li, Yue Guo, Yufei Li, Zhongwei Xie, Zihao Wang.

**Figure 2.** Figure 2: Core executable interface implemented by [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Figure 3: Fidelity–utility Pareto frontier. Points compare one-step Token F1 ( [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Scaling GUI Agents with Visual State Transitions
cs.AI 2026-07 conditional novelty 6.0

A joint inverse-forward pretraining stage on visual screen transitions improves GUI-agent fine-tuning by 0.6–6.2 percentage points across three benchmarks.
VisualPatchWorld: Code World Models as Latent Structured Representations for Planning
cs.CL 2026-07 conditional novelty 5.0

A two-level induction procedure—active-probe sketch selection plus multi-step rollout fitting—recovers executable code world models that improve CEM planning over prior code baselines on four LeWM tasks.