{"id":"eb0c37b9-b158-46bf-8a29-2e17619e2611","arxiv_id":"2605.30880","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Offline text-agent trajectories can be compiled into executable belief-state world models via LLM-guided counterexample repair, reaching 76.4% macro one-step-lookahead success with zero prediction-time LLM calls.","lead":"PatchWorld turns agent logs into inspectable Python programs that predict what happens next, and repairs those programs whenever predictions fail. On seven text-agent environments it beats other code-based world models on one-step planning while running zero LLM calls during lookahead prediction.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Planning metric does not isolate world-model quality: Wordle's induced model hard-codes one target yet scores 100%, and Maze's 6 instances carry the ReAct advantage.","rationale":"The reader identified the shared reranker/fallback as a potential confound and noted the Maze test split's fragility. I agree, but the strongest concrete evidence comes from the manuscript itself: the induced Wordle model in Appendix P hard-codes a single target word, which cannot be correct in a POMDP where the target is hidden and varies per episode. Its 100% success under the shared planner shows the evaluation harness can fully mask predictor failures. This, together with the Maze calculation, makes the 'matching/exceeding LLM-based lookahead' component of the central claim unsupported. Importantly, the code-based comparison is more robust: PatchWorld-Simple remains above PoE-World and WorldCoder even after excluding Maze and Wordle. Therefore the paper still merits conditional acceptance, pending a robustness check that removes non-informative environments and reports override rates; I do not move the reader's verdict.","tokens_in":40046,"tokens_out":11758,"duration_ms":119353,"concrete_test":"From the saved RQ3 planning logs (the same logs used for Appendix K), recompute macro episode success for PatchWorld-Simple, ReAct, LLM-Direct, PoE-World, and WorldCoder after (a) removing Maze and (b) removing both Maze and Wordle; report per-environment Wilson intervals and the fraction of episodes in which the gated reranker actually overrode the ReAct default for each method. Separately, run PatchWorld's induced Wordle program on the held-out Wordle episodes and measure how often its predicted feedback matches the true feedback; if match rate is near chance while episode success stays 100%, the planning harness is masking predictor quality.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Load-bearing concern: the planning evaluation does not establish that the world model, rather than the surrounding selector/fallback machinery, drives the headline 76.4% macro success. Appendix P (Figure 11) shows the induced Wordle model hard-codes `target_word = 'frame'` in `init_belief` and never recovers the hidden target from feedback; `readout_observation` therefore emits feedback for 'frame' for every episode. Yet Table 11 reports 100% Wordle success for PatchWorld-Simple, identical to ReAct. A predictor that is wrong for most held-out episodes can still yield ceiling success under the gated reranker with ReAct fallback (Appendix H), so Wordle contributes no information about the model. Second, the +2.0 pp macro edge over ReAct is carried by Maze, whose test split has 6 instances (Table 6; one episode = 16.7 pp). Excluding Maze drops PatchWorld-Simple to ~74.0, below LLM-Direct (75.8) and equal to ReAct (74.4). Excluding both Maze and Wordle, PatchWorld-Simple still beats the code-based baselines but does not beat ReAct. Thus the 'matching or exceeding LLM-based lookahead' and 'useful for action selection' parts of the central claim are not robustly supported; only 'best code-based program predictor' survives.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"PatchWorld induces executable Python belief-state world models from offline trajectories in text-agent environments. It uses an LLM to synthesize an initial program, then performs counterexample-guided repair by replaying trajectories, extracting typed counterexamples, asking the LLM for candidate patches, and accepting only patches that lower a severity-weighted replay score. It evaluates two variants: PatchWorld-Simple (pure symbolic) and PatchWorld-Residual (with a train-only exact-key residual memory). Across seven AgentGym environments, it reports PatchWorld-Residual as the best code-based next-observation predictor (macro Token F1 0.69) and PatchWorld-Simple as the best code-based planner (76.4% macro episode success in one-step lookahead with zero LLM calls at prediction), and claims a fidelity–utility Pareto frontier. The paper includes per-environment tables, a five-seed variance study, a component ablation, a coverage diagnostic, and complete induced programs.","tokens_in":40400,"tokens_out":9432,"duration_ms":93697,"significance":"If the planning claim were robust, the paper would make a valuable contribution: a gradient-free, inspectable, executable alternative to neural next-observation predictors, with evidence on multiple environments and backbones, careful instance-level splits, five-seed variance, and released code. The fidelity result (PatchWorld-Residual 0.69 macro Token F1, with 0.019 seed std) appears solid. However, the headline planning result is not robustly supported: the Wordle component is an artifact, Maze has six test instances, and the macro difference over ReAct/LLM-Direct is within noise. Thus the significance as a planning method is currently unproven, though the framework and fidelity contribution remain interesting.","major_comments":[{"comment":"The Wordle row is invalid as evidence and inflates the headline. Figure 11's init_belief hard-codes target_word='frame'; readout_observation always returns feedback for 'frame', and correct_belief never updates the target. For any held-out episode with a different target, predictions are wrong, yet Table 11 reports 100% PatchWorld-Simple success identical to ReAct, which the gated fallback (Appendix H) can produce regardless of the predictor. Combined with Maze's 6-instance test split (Table 6), these two environments carry the macro result: excluding Maze and Wordle, PatchWorld-Simple's macro success over the remaining five environments is 66.9%, below ReAct (67.5%) and essentially tied with LLM-Direct (66.0%). The abstract's 'matching or exceeding LLM-based lookahead' and 'useful for action selection' are not supported. Please re-analyze on the robust subset and report per-environment","section":"Table 11; Appendix P"},{"comment":"The planning protocol does not isolate the world model. A shared Qwen3-Coder-480B selector overrides the ReAct default only when it finds 'usable contrast', otherwise falls back. This gating machinery is identical across all predictors and is never ablated. A predictor that is wrong on most episodes (Wordle) can still yield ceiling success via fallback, so the 76.4% macro number may measure the selector rather than the world model. Please add a no-lookahead control with the same selector, and an ablation that replaces the selector with a fixed rule or random ranking, to demonstrate that predicted next-observation differences change action selection.","section":"Appendix H"},{"comment":"The fidelity–utility tradeoff is not robustly established. PatchWorld-Residual's lower macro success (72.9 vs 76.4) is driven by Maze (83.3 vs 100, n=6) and Wordle (tied); on the other five environments the per-environment signs are mixed, and excluding Maze and Wordle the residual's macro is 65.4 vs 66.9 for Simple, within run-to-run variation. No significance test is reported. Appendix K only rules out identical-string collapse; it does not show that residual memory causally weakens action-discriminative contrast. Please report per-environment confidence intervals and a formal test, or temper the tradeoff claim.","section":"§5.4; Table 12; Figure 3"},{"comment":"The reported replay-error reduction is tautological. Candidate patches are accepted only if Score(c;B) improves, so the final validation error is lower by construction. Table 5's '-Repair loop' also reports validation F1, not held-out test F1. To support the repair-loop contribution, report a held-out test ablation (full pipeline vs R=0) or explicitly label Table 15 as an optimization diagnostic rather than evidence of generalization.","section":"§5.5; Table 15"}],"minor_comments":[{"comment":"The abstract says 'matching or exceeding LLM-based lookahead' while Appendix J explicitly says 'parity' with LLM-Direct. Please align the wording with the statistical evidence.","section":"Abstract vs Appendix J"},{"comment":"The caption should state that Avg. F1 is validation Token F1, not held-out test F1, to avoid implying test-set ablation.","section":"Table 5"},{"comment":"The 'Pareto frontier' is drawn through only two points; since many points are not on a convex hull, clarify that it is a schematic frontier rather than a fitted Pareto surface.","section":"Figure 3"},{"comment":"The 'usable contrast' criterion for overriding the ReAct default is not specified. Please define it or provide the exact threshold used, since it controls the fallback behavior that drives the headline numbers.","section":"Appendix H, step 4"},{"comment":"The code comment 'Fixed target for consistent testing' should not appear in a paper claiming generalization. At minimum, disclose this limitation in the main text and exclude Wordle from the planning claim unless the model is repaired.","section":"Figure 11"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for the journal. The Wordle hard-coded target in Appendix P is a serious oversight that undercuts the central planning claim; the authors should address it directly. The fidelity results are more convincing, and the released code and held-out protocol are strengths. I see no evidence of citation manipulation or scope mismatch."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"PatchWorld is a solid, honest empirical paper. The core loop — offline trajectories, contrastive mining, LLM synthesis, execution-grounded counterexamples, validation-gated repair — is a credible new assembly for code world models under partial observability. The fidelity results are the strongest part: PatchWorld-Residual beats the best code baseline (WorldCoder/PoE-World) on held-out Token F1 (0.69 vs 0.63) with five-seed variance and instance-level splits, and the residual memory is train-only, so leakage is not a concern. The repair-loop validation reduction in Table 15 is tautological given the accept-only-if-improves gate, so it should be read as a sanity check, not evidence — the paper presents it that way.\n\nThe planning claim needs more care. The 76.4% macro success is a real number, and the gap over WorldCoder (64.4) and PoE-World (69.3) is larger than the reported standard deviations. But the 'matching or exceeding LLM-based lookahead' framing in the abstract goes beyond what the experiment supports. The +2 pp over ReAct is driven almost entirely by Maze, whose test split has 6 instances; the Wordle induced model in Appendix P hard-codes target_word = 'frame' and never recovers the true target, yet scores 100% under the shared reranker with ReAct fallback, so Wordle is uninformative about model quality. The gap over LLM-Direct (75.8) is within the paper's own ±0.8 standard deviations. Appendix J is honest about this ('parity'); the abstract is not. That is a fixable presentation issue, not a fatal flaw.\n\nThe fidelity-utility frontier is a two-point interpretation, but the value-equivalence argument and the Appendix K diagnostic (the residual does not collapse predicted-text diversity) give it enough support to take seriously. The limitations section is appropriately narrow: one-step lookahead, one model per environment, language-only. The Risks appendix is a welcome acknowledgment that induced models can encode wrong rules.\n\nBottom line: this is a serious piece of work that deserves a careful referee. The authors should be asked to soften the abstract, report per-environment n and paired significance tests for the planning results, and ideally vary the selector or add a no-selector control. I would bring it to a reading group to discuss the evaluation-design tradeoffs. Send it to peer review.","headline":"Solid, honest paper; fidelity results and code-vs-code planning gains are credible, but the 'matching LLM lookahead' claim is fragile and should be softened.","tokens_in":40932,"tokens_out":5747,"would_cite":true,"duration_ms":59249,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that offline agent trajectories can be compiled, through counterexample-guided code repair, into executable Python world models that predict next observations and support planning at zero LLM inference cost, matching or ex","keywords":["executable world models","program synthesis","counterexample-guided repair","belief state","text agents","partial observability","planning with lookahead","fidelity–utility tradeoff"],"falsifier":"Run the same one-step lookahead planning protocol with the world-model predictor replaced by a constant predictor that returns the same generic string for every candidate action, keeping the candidate generator and reranker fixed. If macro episode success stays near 76.4% or within a few points of the no-lookahead baseline, the reported gain is not attributable to the world model's action-conditioned predictions. A complementary check would corrupt the lookahead predictions only for the action that the planner ultimately selects and measure the success drop.","tokens_in":39932,"feed_emoji":"⚙️","tokens_out":3711,"duration_ms":39716,"temperature":0.7,"pith_summary":"The paper asks whether a text agent's environment can be captured as an executable Python program induced purely from logged observation–action trajectories, without ever seeing the simulator's hidden state or transition rules. It proposes PatchWorld, a gradient-free loop in which an LLM writes a belief-state program, replay exposes concrete counterexamples, and the LLM proposes patches that are accepted only when they lower a replay-error score. The paper's central claim is that this works: the purely symbolic variant reaches 76.4% macro episode success in live one-step lookahead across seven environments, matching or exceeding LLM-based lookahead while using zero LLM calls inside the prediction module. A second claim is that surface fidelity and planning utility are not the same objective: adding a train-only residual memory improves next-observation match but lowers planning success, exposing a Pareto frontier. A sympathetic reader would care because cheap, inspectable, locally repairable world models could make agent simulation and diagnosis practical without per-step LLM inference.","feed_headline":"Offline logs become runnable world models; 76.4% plan success","feed_subtitle":"Executable belief-state programs match LLM lookahead on seven agent tasks and reveal a fidelity–utility tradeoff.","key_machinery":"The central object is the executable belief-state program c, which separates parsing, belief correction, dynamics, and readout through the functions parse_observation, init_belief, correct_belief, predict_belief, readout_observation, and extract_valid_action_forms. The predict-and-correct cycle is: ŝ0 = gc(⊥, o0); ŝt+1 = fc(ŝt, at); ôt+1 = ρc(ŝt+1, at); ŝt+1 = gc(ŝt+1, ot+1). Induction is a counterexample-guided loop: replay the program against logged trajectories, cluster and prioritize typed failures (load errors, parser exceptions, belief-update mismatches, readout mismatches), prompt an LLM to emit complete replacement programs, and accept a patch only when its severity-weighted replay s","core_discovery":"Under partial observability, finite trajectories underdetermine the environment's latent structure, yet PatchWorld shows that an LLM acting as a symbolic optimizer can search the space of executable belief-state programs and converge to models that both predict next observations and support action selection. The induced program implements a predict-and-correct cycle: parse the observation into a belief state, predict the next belief from an action, render a predicted observation, then correct the belief when the true observation arrives. Repair is driven by typed counterexamples from full-trajectory replay, and a candidate patch is accepted only if it improves a severity-weighted replay scor","pith_inferences":["A natural testable extension is to use PatchWorld's induced programs as disagreement generators: the points where a program's predictions diverge from the logged environment could be mined to bootstrap better neural predictors or to flag environment instances where the code model is unreliable.","The fidelity–utility frontier suggests that benchmark suites for text agents should report both a reconstruction score and a decision-utility score for every world model, since a model that excels at one axis can be worse on the other, as shown by the residual variant's opposite ordering on the two metrics.","If the cost of induction continues to fall, offline logs from deployed agents could become a renewable source of inspectable simulators for sandboxing, safety auditing, and counterfactual 'what if' analysis — uses that a black-box neural world model supports less directly.","The repair loop's validation gate suggests that similar acceptance rules could stabilize other LLM self-improvement schemes: allowing a proposed change only when it improves a formal, replay-grounding score may prevent the regression failures seen in unconstrained self-correction."],"forward_implications":["Executable world models induced from offline logs can be used at inference time with zero LLM calls inside the prediction module, cutting lookahead prediction cost from tens of thousands of tokens per task to zero while roughly matching LLM-based planning accuracy.","The same induced program supports both next-observation prediction and action selection, so a single inspectable artifact can serve simulation, diagnosis, and planning without retraining.","The fidelity–utility divergence means that next-token metrics like Token F1 and BLEU-4 can mis-rank world models for decision making; planning-oriented evaluation needs an action-contrastive axis.","Counterexample-guided repair with a validation gate is an effective inductive bias: it reduces replay errors by 12–79% depending on environment and backbone, and the strongest single component in the ablation is contrastive transition selection, not the repair loop itself.","A human-specified residual-memory bias improves surface fidelity but can weaken planning utility, showing that world-model design involves a deliberate tradeoff rather than a single optimum."],"fun_headline_variants":["PatchWorld codes worlds from offline logs","Executable world models via code repair","Repair code to mimic agent environments","World models from black-box trajectories","Gradient-free lookahead beats LLM on 7 tasks"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The headline planning result assumes that the shared reranker genuinely uses the world model's predicted next-observation text to decide between candidate actions; if the reranker mostly falls back to the default action or ignores the lookahead predictions, the 76.4% success rate measures the selector, not the world model.","fun_headline_variants_meta":{"raw":{"variants":["PatchWorld codes worlds from offline logs","Executable world models via code repair","Repair code to mimic agent environments","World models from black-box trajectories","Gradient-free lookahead beats LLM on 7 tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00016,"raw_usage":{"total_tokens":1070,"prompt_tokens":749,"completion_tokens":321,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":493,"completion_tokens_details":{"reasoning_tokens":255}},"tokens_in":493,"tokens_out":321,"duration_ms":4264,"temperature":1.0,"reasoning_tokens":255,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T12:45:03.288512+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same one-step lookahead planning protocol with the world-model predictor replaced by a constant predictor that returns the same generic string for every candidate action, keeping the candidate generator and reranker fixed. If macro episode success stays near 76.4% or within a few points of the no-lookahead baseline, the reported gain is not attributable to the world model's action-conditioned predictions. A complementary check would corrupt the lookahead predictions only for the action that the planner ultimately selects and measure the success drop.","supporting_citations":[],"review_version":3}