Pith. sign in

REVIEW 4 major objections 4 minor 16 references

CoEx -- Co-evolving World-model and Exploration

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CoEx claims that LLM agents plan better when the world model is an explicit neurosymbolic belief state that co-evolves with subgoal-level exploration, and reports gains over ReAct, AdaPlanner, and HiAgent on ALFWorld, PDDL, and Jericho.

desk verdict CoEx is a sensible architecture for online world-model adaptation in LLM agents, but the paper's own Table 4 contradicts its headline claim of outperforming all existing paradigms. read the letter →

arxiv 2507.22281 v1 pith:74K6OMN2 submitted 2025-07-29 cs.AI cs.CL

Minsoo Kim , Seung-won Hwang This is my paper
classification cs.AIcs.CL
keywords LLMagentsworldmodelsubgoalplanningexplorationneurosymbolicbeliefstateverificationandsynthesisALFWorldtext-basedgames
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes CoEx, an LLM agent architecture whose central move is to make the world model an explicit, editable belief state instead of an implicit function of the LLM's parameters or context. A subgoal-level Planner generates goals, including deliberate exploratory goals, and delegates execution to an Actor; after each subgoal, a verification-and-synthesis module distills the raw experience into a neurosymbolic belief state combining code-based symbolic memory with structured textual memory. The paper's claim is that this co-evolution keeps planning aligned with the true state of a partially observable environment and overcomes the exploitation bias and limited adaptation of monolithic agents like ReAct. CoEx reaches a 93.28% total success rate on ALFWorld, exceeding ReAct, Reflexion, and AdaPlanner, and it outperforms ReAct on PDDL and Jericho in both success and progress rates. A sympathetic reader would take this as evidence that online world-model learning and subgoal-level exploration reinforce each other rather than pulling the agent in opposite directions.

What carries the argument

The load-bearing machinery is the neurosymbolic belief state together with the two-stage update that refreshes it. $m_k$ is implemented as a Python class whose methods parse observations into symbolic predicates such as agent location, held objects, and on/clear relations, giving exact low-level state tracking without calling the LLM. $\ell_k$ is generated by an LLM in two stages: Verification answers consistency and outcome questions about the subgoal trace, and Synthesis writes a status line, a justification, and a list of learned facts. The Planner is then a state machine whose states are these belief states; the Actor executes each subgoal in a bounded reason-act loop and signals completion or requests replanning. Because the update fires once per subgoal rather than per action, the extra computation is small, with the paper estimating that 70% of token cost stays in the Actor.

What would settle it

Run CoEx on ALFWorld while logging every learned fact and status line that the verification stage writes, then compare each against the simulator's true state. If a large share of false updates still leads to task success, belief-state fidelity is not the mechanism; if false updates reliably precede failed replanning, the paper's mechanism is confirmed.

Watch

Extended reading notes

Core claim

The core discovery is that an LLM agent can be organized as a closed loop in which planning and world-model updates happen at the same subgoal granularity. The belief state is written $b_k=(m_k,\ell_k)$, where $m_k$ is a code-based, object-oriented symbolic memory updated deterministically from observations, and $\ell_k$ is a structured textual memory produced by the LLM-based Verification and Synthesis module after each subgoal attempt. The Planner is an LLM prompted as a state machine over this belief state: it reads its history $H_k$ and the latest $b_k$, emits a subgoal, and can regenerate the whole plan when new facts arrive. The paper argues that this design lets exploration be expressed as ordinary subgoals, and lets the outcome of exploration be written back into a persistent world model, directly addressing the two failure modes it identifies in monolithic LLM agents. Its experimental claim is that this architecture outperforms existing agent paradigms on ALFWorld, PDDL, and Jericho.

Load-bearing premise

The load-bearing premise is that the language model that checks and writes the belief-state updates reports what actually happened during a subgoal; if it quietly invents facts about the world, the Planner builds on a corrupted model and the co-evolution story fails.

Editorial extensions

If this is right

  • Online world-model learning becomes a drop-in planning component: an agent can explore, record what it learns, and replan on the same clock, without an offline training phase.
  • Exploratory behavior no longer has to be demonstrated at the action level; the Planner can issue an exploratory subgoal such as 'investigate the trap door' as a first-class step, which is how CoEx handles open-ended discovery in Jericho.
  • Explicit belief states make the agent's model inspectable and correctable: a wrong learned fact can in principle be overwritten by a later verification-and-synthesis update rather than persisting implicitly in context.
  • The same architecture transfers across environment formats, from text-based embodied tasks to classical symbolic planning to interactive fiction, suggesting the mechanism is about the planning-update loop rather than a single benchmark's quirks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the paper's strongest falsifiable prediction is that belief-state fidelity is the causal driver; feeding the verification module ground-truth labels instead of its own readings should preserve or improve performance, while injecting a single false learned fact should measurably degrade planning.
  • Editorial inference: the symbolic memory's predicates on PDDL could be handed directly to a classical planner, so a natural extension is to test whether the co-evolution loop helps most when subgoal semantics are fuzzy, as in text games, or also when planning could be fully symbolic.
  • Editorial inference: the decomposition of task-level exemplars into a skill library is a hidden variable, and transferring CoEx to a new environment likely depends on the breadth of that library; an ablation varying the skill library size would separate the contribution of the belief state from the contribution of the Actor's prior knowledge.
  • Editorial inference: if the update cost is as low as reported, the architecture suggests a general recipe for long-horizon LLM agents, subgoal-level planning, explicit memory, and verification at update boundaries, that could extend to domains such as web navigation or embodied instruction following.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes CoEx, a hierarchical LLM agent architecture in which a subgoal-level Planner generates dynamic subgoal plans, an Actor executes them through action-level reasoning, and a neurosymbolic belief state (comprising code-based symbolic memory and LLM-synthesized textual memory) is updated after each subgoal attempt. The design aims to mitigate what the authors call exploitation bias and limited adaptation in monolithic LLM agents. CoEx is evaluated on ALFWorld, PDDL (Gripper and Blocksworld), and Jericho, benchmarked against ReAct, Reflexion, AdaPlanner, ExpeL, WALL-E, and HiAgent. The core claim is that CoEx 'outperforms existing agent paradigms in planning and exploration.'

Significance. The paper proposes a plausible and cleanly articulated architecture: separating subgoal-level planning from action-level execution, and making the world model explicit and updatable through a two-stage verification-and-synthesis process. The design is concrete, with pseudocode, prompts, and an example trajectory, and the cost analysis in RQ2 is a useful practical addition. If the empirical claims were fully supported, this would be a valuable contribution to LLM-agent research. However, the central quantitative claim is not supported by the paper's own data: the ALFWorld aggregate in Table 4 shows WALL-E outperforming CoEx, and the comparisons on PDDL and Jericho are much narrower than the abstract's unqualified claim. These issues must be resolved before the contribution can be assessed fairly.

major comments (4)
  1. [§5.2, Table 4] The text states that CoEx 'achiev[es] the highest total success rate of 93.28%' on ALFWorld, but Table 4 reports WALL-E at 95.00% total, and WALL-E is also higher on Clean (97.00% vs 83.87%) and Heat (100% vs 91.3%). Since §2 explicitly categorizes WALL-E as an existing LLM agent paradigm, the abstract and conclusion claim that CoEx 'outperforms existing agent paradigms' is contradicted by the paper's own reported numbers. The later statement that CoEx 'achieves competitive performance with WALL-E' is accurate, but it does not reconcile with the 'highest total success rate' phrasing. This is a load-bearing inconsistency in the paper's headline claim and must be corrected.
  2. [§5.2, Tables 5 and 6] The claim of outperforming 'existing agent paradigms' is supported on PDDL by comparisons only against ReAct and HiAgent, and on Jericho only against ReAct. The other baselines discussed in §2 and evaluated on ALFWorld—WALL-E, Reflexion, AdaPlanner, and ExpeL—are not run on these benchmarks. The cross-benchmark generalization of the headline claim therefore rests on a single task suite (ALFWorld), where CoEx is not the top performer. Please either add the missing baselines or explicitly qualify the claim as applying only to the methods actually compared in each setting.
  3. [§4.4 and §5.5] The paper acknowledges in §5.5 that 'LLM-based QA may introduce marginal inaccuracies during world model updates,' but it does not measure how often the Verification and Synthesis module (Stages 1 and 2) produces incorrect or hallucinated belief-state updates, nor whether such errors compound over the subgoal sequence. Because the Planner conditions all future subgoals on this belief state, reliability of the update mechanism is load-bearing. Please add a quantitative or qualitative error analysis of belief updates (e.g., a human evaluation of a sample of verification outputs and learned facts, or a measurement of error propagation).
  4. [§5.1 and §5.2] All results are reported as single-point success rates without confidence intervals, error bars, or significance tests, despite the substantial stochasticity of sampling from GPT-4o-mini across the Planner, Actor, and verification modules. The gap between CoEx (93.28%) and WALL-E (95.00%) on ALFWorld is small, and without multiple runs or a significance test, the ordering of the two methods cannot be treated as reliable. At minimum, the paper should report the number of runs or seeds used and discuss the sensitivity of the results.
minor comments (4)
  1. [Throughout] There are several typos: 'effictiveness' in the introduction, 'myoptic' and 'W ALL-E' in §2, 'representave' in §5.1, 'maximimum' in §5.1, 'is is' in §4.4, 'promisining' in the Limitations section, and 'AlfWorld' (inconsistent capitalization) in §2.
  2. [Algorithms 1 and 2] The pseudocode is internally inconsistent: Algorithm 1 line 16 assigns the return of ExecuteSubgoalEpisode to (εk, mk+1), but Algorithm 2's ExecuteSubgoalEpisode returns only Hexec; additionally, Algorithm 1 line 17 calls v.BeliefUpdate while Algorithm 2 defines the update through separate Verification and Synthesis procedures. Please align the naming and return values so the pseudocode matches the prose.
  3. [Table 4] The table formatting is inconsistent (e.g., '100.00 %' with a space, and varying decimal places). Also, 'Picktwo' should be consistently capitalized (the task is 'PickTwo' in the ALFWorld literature).
  4. [§4.2] The sentence 'using an example from theZork1 of the Jericho text-based game' is missing a space and is awkwardly phrased; please rephrase.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: CoEx's claims rest on external benchmark comparisons and independent environment evaluation, not on self-citations, fitted inputs, or definitional equivalences.

full rationale

The paper's derivation chain is empirical rather than analytic. CoEx is an architecture (Planner, Actor, and neurosymbolic belief state) evaluated against external environments (ALFWorld, PDDL, Jericho) and external baselines (ReAct, Reflexion, AdaPlanner, HiAgent, ExpeL, WALL-E). No equation is derived from an assumption that contains the conclusion, no parameter is fit to a subset of data and then relabeled as a prediction, and no load-bearing claim is grounded in a self-citation. The belief-state loop in which an LLM-based verification and synthesis module updates the belief state and the Planner conditions on it is a design feedback loop, not a circular derivation: success is measured independently by environment reward, and the results would stand or fall regardless of how the belief state is generated. The appendix disclosure that Gemini 2.5 pro was used to implement symbolic-memory code with manual revision is an implementation detail, not a fit to test outcomes. The Limitations section's concession that 'LLM-based QA may introduce marginal inaccuracies during world model updates' is an acknowledged robustness caveat, not a circular step. The paper does contain a non-circular correctness concern: Section 5.2 calls 93.28% the 'highest total success rate' on ALFWorld even though Table 4 lists WALL-E at 95.00%, and the abstract's unqualified claim that CoEx 'outperforms existing agent paradigms' is stronger than the reported tables support. That internal inconsistency affects the accuracy of the headline, but it does not make the result circular, because the comparisons remain external evidence. Overall score 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces a software architecture component, the neurosymbolic belief state, but no new physical entities, unobserved forces, or formal postulates. The central claim depends on hand-set hyperparameters (step limits, LLM choice) and on the reliability of LLM-generated belief updates in partially observable environments.

free parameters (3)
  • max_subgoal_steps = 35
    The actor's subgoal execution episode is capped at 35 steps (Section 5.1, Algorithm 2). This hand-set limit affects whether a subgoal is marked complete or triggers replanning, and it is not swept in any experiment.
  • max_total_steps = 100 (ALFWorld, PDDL), 150 (Jericho)
    Total step budgets per environment set in Section 5.1. These are hand-chosen and could favor an agent that replans at the subgoal level rather than one that must act more densely.
  • LLM backbone = GPT-4o-mini
    All CoEx components use GPT-4o-mini (Section 5.1). Performance depends on this choice, and the same model is not explicitly reported for all baselines, so part of the observed gap may be model-driven rather than architecture-driven.
assumptions (4)
  • domain assumption Action-level in-context policy learning induces exploitation bias and myopic planning horizons
    The motivation for CoEx relies on this premise, citing Monea et al. (2025) and Dai et al. (2024). It is treated as a given, though the link between ICL demonstrations and exploration behavior in complex environments is not formally established in this paper.
  • domain assumption An explicit belief state updated by LLM-based verification and synthesis is a reliable world model for planning
    Section 4.4 assumes that LLM-generated status lines, justifications, and learned facts will be accurate enough to improve future subgoal planning. The paper notes inaccuracies are possible, but does not bound their frequency or effect.
  • domain assumption Object-oriented symbolic memory can be programmatically updated from raw text observations across ALFWorld, PDDL, and Jericho
    The paper provides a working Python memory for PDDL Blocksworld, but states that ALFWorld and Jericho use similar code-based symbolic memories without providing those implementations. It assumes a generic object-oriented representation generalizes across domains.
  • domain assumption Subgoal-level planning improves exploration and task success compared to action-level planning
    This is the central design hypothesis, tested only indirectly through benchmark comparisons. The paper does not provide a controlled experiment where only planning granularity changes while all else is held fixed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CoEx -- Co-evolving World-model and Exploration." pith.science (2026). https://pith.science/paper/74K6OMN2

@misc{pith2026250722281,
  author       = {Pith},
  title        = {Pith review of: CoEx -- Co-evolving World-model and Exploration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/74K6OMN2}},
  note         = {Machine review of arXiv:2507.22281}
}
read the original abstract

Planning in modern LLM agents relies on the utilization of LLM as an internal world model, acquired during pretraining. However, existing agent designs fail to effectively assimilate new observations into dynamic updates of the world model. This reliance on the LLM's static internal world model is progressively prone to misalignment with the underlying true state of the world, leading to the generation of divergent and erroneous plans. We introduce a hierarchical agent architecture, CoEx, in which hierarchical state abstraction allows LLM planning to co-evolve with a dynamically updated model of the world. CoEx plans and interacts with the world by using LLM reasoning to orchestrate dynamic plans consisting of subgoals, and its learning mechanism continuously incorporates these subgoal experiences into a persistent world model in the form of a neurosymbolic belief state, comprising textual inferences and code-based symbolic memory. We evaluate our agent across a diverse set of agent scenarios involving rich environments and complex tasks including ALFWorld, PDDL, and Jericho. Our experiments show that CoEx outperforms existing agent paradigms in planning and exploration.

Figures

Figures reproduced from arXiv: 2507.22281 by the authors.

Figure 1
Figure 1. Landscape of LLM agent paradigms along the di [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

16 extracted references · 14 canonical work pages

  1. [1]

    Read the new_belief text from the most recentanalysis_feedback message

    work page

  2. [2]

    Identify all receptacle IDs mentioned as existing in the room within that belief text (e.g., cabinet 1, fridge 1,countertop 1)

    work page

  3. [3]

    Prioritize likely locations if possible

    Create a list of these known receptacle IDs. Prioritize likely locations if possible

    work page

  4. [4]

    Populate the SEARCH_LOCATIONS: field with this exact list. Example: SEARCH_LOCATIONS: [cabinet 1, fridge 1, countertop 1] Below are examples of a similar task: {{task exemplars}} Instance Prompt (ALFWorld) {{task_room}}{{task}} Exploration strategies: - When searching for an object, try to search in the most likely location first, and if not found, try to...

    work page

  5. [5]

    Find and take [object]

    Processing Subgoal Feedback: - After a subgoal attempt, you will receive an ’analysis_feedback’ message (role: assistant). - Parse the JSON content of this message. - Use the new_belief field to understand the current world state before planning the next subgoal. EXECUTE_SUBGOAL Action Format: You MUST issue subgoals using the following multi-line format ...

    work page

  6. [9]

    - Parse the JSON content of this message

    Processing Subgoal Feedback: - After a subgoal attempt, you will receive an ’analysis_feedback’ message (role: assistant). - Parse the JSON content of this message. - Use the new_belief field to understand the current world state before planning the next subgoal. EXECUTE_SUBGOAL Action Format: You MUST issue subgoals using the following multi-line format ...

    work page

  7. [10]

    The action is not valid and therefore takes no effect. Please check valid actions

    Processing Subgoal Feedback: - After a subgoal attempt, you will receive an ’analysis_feedback’ message (role: assistant). - Parse the JSON content of this message. - Use the new_belief field to understand the current world state before planning the next subgoal. EXECUTE_SUBGOAL Action Format: You MUST issue subgoals using the following multi-line format ...

    work page

  8. [11]

    Status:

    A concise status line reflecting the current progress relative to the plan (starting with "Status: ")

    work page

Show all 16 references
  1. [12]

    and the justification for the status line

  2. [13]

    status_line

    A list of concise new facts learned or hypotheses formed about the environment/task based only on the last subgoal’s execution, especially failures or unexpected outcomes. Focus on actionable insights or constraints. Respond ONLY with a valid JSON object containing the keys " ...

  3. [14]

    Status:

    Generate status_line: Create a single sentence starting with "Status: " that reflects the ’Last Subgoal Outcome’ and progress relative to the ’Latest Overall Plan’

  4. [15]

    Provide justification: Briefly explain the reasoning for the status line

  5. [16]

    Confirmed: Kettle is in Cabinet

    Generate learned_facts: Based only on the Q&A and outcome of the last subgoal, list any new, concise, and actionable facts learned or hypotheses formed (e.g., "Confirmed: Kettle is in Cabinet.", "Error: Agent tried incorrect action ’open fridge drawer.’"). Output an empty list...

  6. [2008]

    running", Hexec ← ExecutorInitHistory(ek) 3: Param: max_sub_steps 4: while t <max_sub_steps do 5: at ← α(a|ek, Hexec) 6: if at contains SUBGOAL COMPLETED then 7: status ←

    An object-oriented representation for efficient reinforcement learning. In Proceedings of the 25th In- ternational Conference on Machine Learning, ICML ’08, page 240–247, New York, NY , USA. Association for Computing Machinery. David Ha and Jürgen Schmidhuber. 2018. Recurrent ...

    2024 arXiv

  7. [2018]

    CoRR, abs/1806.11532

    Textworld: A learning environment for text- based games. CoRR, abs/1806.11532. Zhenwen Dai, Federico Tomasi, and Sina Ghiassian

    arXiv

  8. [2024]

    In The Twelfth International Conference on Learning Representations

    In-context exploration-exploitation for rein- forcement learning. In The Twelfth International Conference on Learning Representations. Nicola Dainese, Matteo Merler, Minttu Alakuijala, and Pekka Marttinen. 2024. Generating code world mod- els with large language models guided ...

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.