{"id":"efd66622-8591-455f-a899-8d17afc366dc","arxiv_id":"2504.15785","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"WALL-E 2.0 improves LLM agents by encoding learned environment rules as executable code that corrects an LLM world model, lifting ALFWorld success to 98%.","lead":"This paper proposes WALL-E 2.0, an agent that learns an environment's symbolic rules, knowledge graphs, and scene graphs from its own exploration, then converts them into Python code that corrects a large language model's world-model predictions. The method is tested in two game-like environments and beats several existing agents on task success.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The binary success/failure world model assumes deterministic transitions, but Mars includes stochastic outcomes (Appendix F); this unexamined determinism limits the central alignment claim and the cover-rate evidence.","rationale":"The reader identified the binary success/failure and determinism assumption as the weakest point; I agree and sharpen it with direct evidence from the paper itself. Appendix F admits that stochastic transitions exist in Mars, and Section 4.3 shows that Survival outcomes depend on random creature spawns. This is not a speculative edge case: it is part of the benchmark where the central claim is demonstrated. The cover-rate metric, which is the paper's main evidence that symbolic knowledge drives the gain, is not well-defined for stochastic transitions because a deterministic rule cannot have a fixed correct label for a probabilistic outcome. The empirical results may still hold in the deterministic majority of transitions, and the ablations and cover-rate analysis provide real support for the mechanism in that regime. Therefore the appropriate verdict remains CONDITIONAL: the authors should either scope their claims to deterministic environments or extend rule learning to handle stochastic dynamics. No change from the reader's verdict is needed; the concern is important but does not overturn the paper's contribution within its stated benchmarks.","tokens_in":28771,"tokens_out":9185,"duration_ms":87318,"concrete_test":"On the Mars Survival setting, extract all transitions where the same action is attempted from the same observation at night or with hostile creatures present. For each distinct state-action pair, execute it N=100 times (with seeds controlling creature movement) and record the actual success distribution. Compare the aligned world model's binary prediction against these empirical success probabilities. If the model's predictions agree with the outcome less than, say, 80% of the time on the stochastic subset—or if the model always predicts failure while the true success rate is between 20% and 80%—then the deterministic alignment assumption fails exactly where the paper admits it cannot work.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central assertion—that learned symbolic knowledge aligns the LLM world model with environment dynamics—rests on the Section 3.1.1 reduction of every transition to a binary success/failure flag, justified by 'ot+1 can be derived from the success or failure of an action.' That reduction is only valid if the environment is deterministic. The paper's own Appendix F concedes the opposite: 'resource gathering at night in Mars often fails due to hostile creatures but can sometimes succeed,' and Section 4.3 reports high variance in Survival because cows can shoot arrows and initial spawns vary. On such transitions, no deterministic code rule can be correct across repeated attempts; the rule learner's response is to classify the scenario as a failure, which is a policy bias rather than a world model. The cover-rate metric (Section 4.4) is also ill-defined for stochastic transitions: a rule that always predicts failure will 'cover' actual-failure cases but will be wrong on actual-success cases, so the reported 0.0% to 92.3% cover rate depends on how the stochastic subset is sampled. Because both evaluated benchmarks are not purely deterministic, the claimed improvement cannot be attributed to a faithful predictive world model; it may be an overly conservative refusal policy plus LLM prior. This does not invalidate the empirical results, but it does mean the paper's 'world alignment' claim is narrower than stated: it aligns with deterministic affordances, not with POMDP transition probabilities.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes WALL-E 2.0, a training-free pipeline that aligns an LLM-based world model with environment dynamics by learning symbolic knowledge (action rules, knowledge graphs, scene graphs) from exploration trajectories, converting that knowledge into executable code rules, and using a maximum-coverage pruning step to keep the rule set compact. The aligned world model is used inside an agent that applies model-predictive control (MPC), where the LLM agent proposes actions and the world model checks them against the learned rules. Experiments on Mars and ALFWorld report improved reward/score and success rates over prior LLM agents and RL baselines, and ablations suggest the symbolic knowledge and its placement in the world model are the main contributors.","tokens_in":29075,"tokens_out":10725,"duration_ms":90653,"significance":"If the results hold, WALL-E 2.0 offers a practical, interpretable way to correct LLM priors in environments with counter-commonsense mechanics without fine-tuning, and the code-rule mechanism provides a verifiable interface between symbolic constraints and LLM predictions. The paper includes useful ablations (Tables 3–5) and a diagnostic cover-rate measure (Section 4.4) that directly tracks how much of the LLM's misprediction is corrected by learned rules. The reported ALFWorld success rate of 98% is notable, though its statistical robustness needs verification. The significance is tempered by the narrow scope of the 'world model' (binary success/failure prediction), the acknowledged inability to handle stochastic transitions, and a mismatch between the MPC framing and the one-step implementation.","major_comments":[{"comment":"The binary success/failure world model in Section 3.1.1 assumes deterministic transitions, but the paper's own Appendix F concedes that Mars includes stochastic outcomes (e.g., resource gathering at night can sometimes succeed) and Section 4.3 reports high variance from cows shooting arrows. On such transitions, a learned rule that always predicts failure is a policy bias rather than a predictive world model, and the cover-rate metric (Eq. 9) is not well-defined for stochastic transitions because a failure-predicting rule covers only the failure realizations. The authors should either restrict the world-model predictions and cover-rate evaluation to deterministic transitions, model stochastic outcomes probabilistically, or explicitly justify the conservative-failure treatment as a safety policy rather than an accurate world model.","section":"3.1.1, Appendix F, Eq. (9)"},{"comment":"The abstract and introduction claim an 'LLM-based MPC framework' with look-ahead optimization of future steps, but Algorithm 2 is a one-step action feasibility filter: the loop only replans a single action until it passes the code-rule checks, and Appendix E.1 confirms 'We employ a one-step MPC approach.' This is a mismatch between the stated contribution and the implemented method; either implement multi-step look-ahead planning or reframe the contribution as a one-step predictive filter.","section":"3.2, Appendix E.1"},{"comment":"The text states that WALL-E 2.0 surpasses IfR 'in every aspect' (Figure 5), but Table 1 shows IfR outperforming WALL-E 2.0 on the TERR. TASK. row (reward 6.9±1.8 vs 5.8±2.2; score 7.1±2.5 vs 4.7±2.0) and on the SURV. TASK. reward row (3.3±1.4 vs 3.2±1.4). The superiority claim should be restricted to average results or the specific settings where it holds.","section":"4.3, Table 1"},{"comment":"ALFWorld results are reported as single success rates without error bars or trial counts, despite using GPT-3.5-Instruct with stochastic sampling. The 98% versus RAFA's 95% difference may be within run-to-run noise. The authors should report multiple runs with standard deviations or confidence intervals and specify the number of trials per task type.","section":"Table 2, Figure 4"},{"comment":"The abstract and conclusion say WALL-E 2.0 surpasses baselines in Mars by 16.1%–51.6% of 'success rate', but Section 4.2 and Table 1 define and report 'Reward' and 'Score', not success rate. The 16.1%–51.6% range corresponds to reward improvements. The metric name should be corrected throughout the abstract and conclusion.","section":"Abstract, Conclusion, Section 4.2"}],"minor_comments":[{"comment":"The caption has duplicate stage numbering: it lists '(2) learning new symbolic knowledge...; (4) translating symbolic knowledge to code; and (4) Code rule set pruning.' The numbering should be corrected to 2, 3, 4.","section":"Figure 3 caption"},{"comment":"In the prompt text, 'Please use you knowledge' should be 'Please use your knowledge'.","section":"Appendix B.1"},{"comment":"In the structured observation format, 'inital_state' appears as a key in the ALFWorld action rules (Appendix D.2); this should be 'initial_state' for consistency with the main text.","section":"Appendix C.2"},{"comment":"The text reports that applying symbolic knowledge in the world model yields 'about a 46.3% improvement' and in the agent 'about a 30.9% gain', but the numbers in Table 4 give (6.7−3.8)/6.7 ≈ 43.3% for the world-model case and (5.5−3.8)/5.5 ≈ 30.9% for the agent case; please clarify the formula or adjust the percentages.","section":"Section 4.5, Table 4"},{"comment":"The values of REPLAN LIMIT (Algorithm 2) and the context window k (Section 3.1.2) are never specified; please report their values for reproducibility.","section":"Algorithm 2, Section 3.1.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a preliminary arXiv submission that extends WALL-E 1.0 with knowledge graph and scene graph components. The core empirical idea—learning symbolic constraints from trajectories and injecting them as code rules into an LLM world model—is plausible and the ablations are informative. However, the manuscript currently overclaims on two load-bearing points: the 'MPC' framework is a one-step filter, and the binary success/failure world model cannot handle the stochastic transitions that the authors themselves acknowledge. The ALFWorld comparison also lacks statistical confidence information. These issues are addressable in a revision, but they need substantial work before the paper meets the standards of a journal publication. I would recommend the editor treat the paper as a promising but not-yet-ready submission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a solid incremental systems paper with a real empirical result and a narrower claim than the title suggests. The method combines known ingredients—rule extraction, knowledge/scene graphs, code-based execution, MPC with an LLM—but the specific integration and the demonstration that learned symbolic rules, not just the LLM prior, drive gains on Mars and ALFWorld are worth taking seriously.\n\nWhat's new: WALL-E 2.0 extends WALL-E 1.0 with knowledge graphs, scene graphs, code-rule pruning, and an MPC loop. The ablations show these components matter: dropping either rules or KG/SG hurts performance, and the cover-rate analysis ties gains to rule coverage. ALFWorld 98% after four iterations is concrete, and the Mars score improvements are large. The paper also honestly reports high variance in Survival and concedes in Appendix F that the rule learner cannot handle stochastic outcomes.\n\nSoft spots: the central issue is the deterministic-transition assumption. Section 3.1.1 justifies binary success/failure prediction by saying 'ot+1 can be derived from the success or failure of an action.' That is only true if the environment is deterministic. Mars includes stochastic outcomes—cows shooting arrows, night resource gathering—so the aligned world model is not a faithful model of those transitions; it is a conservative refusal policy plus the LLM prior. The cover-rate metric inherits this problem: a rule that always predicts failure will 'cover' actual failures but be wrong on successes, so the 0.0% to 92.3% number depends on how the stochastic subset is sampled. This does not invalidate the empirical results, but it does mean the paper overstates 'world alignment' with POMDP dynamics. Also, the abstract/conclusion says 'success rate' when Mars metrics are reward and score, and ALFWorld results lack error bars or trial counts. On some Mars settings (e.g., Surv. Task, Terr. Task) WALL-E 2.0 is not better than baselines.\n\nOverall: a legitimate contribution to LLM-agent research, but the main claim needs qualification. The empirical gains are likely real; the mechanism is narrower than stated. For readers building deterministic-rule world models, it is useful; for those hoping for POMDP world models, it is a cautionary example.\n\nRecommendation: send it to peer review, but require a revision that either restricts the claims to deterministic environments or explicitly handles stochastic transitions, and adds missing statistical detail. A solid paper with one load-bearing weakness, not a desk reject.","headline":"Solid incremental systems paper with real empirical gains, but the 'world alignment' claim is narrower than stated because the rule-based world model only covers deterministic transitions.","tokens_in":29643,"tokens_out":2240,"would_cite":true,"duration_ms":19717,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper contends that extracting symbolic world rules as executable code aligns an LLM world model with environment dynamics, letting an agent reach 98% success on ALFWorld after four iterations without any training.","keywords":["LLM agents","world models","world alignment","neurosymbolic learning","action rules","knowledge graphs","scene graphs","model-predictive control"],"falsifier":"Take a deterministic benchmark, inject one stochastic transition — for example, make a mining action succeed with probability 0.5 because hostile creatures sometimes appear — and measure the code rules' cover rate on mispredicted transitions over iterations. If the rule learner labels the same action sometimes success and sometimes failure, the cover rate will stall near chance and agent success will degrade far below the reported 98% and 15.3%, confirming that the binary deterministic-transition assumption is the load-bearing premise.","tokens_in":28586,"feed_emoji":"🤖","tokens_out":8298,"duration_ms":72764,"temperature":0.7,"pith_summary":"This paper contends that a pretrained LLM can be turned into an accurate world model for an agent without finetuning, by learning the environment's own symbolic knowledge from exploration. The knowledge — action rules, a knowledge graph, and a scene graph — is extracted by prompting the LLM to reason inductively over real and predicted trajectories, compiled into executable code rules, and used to override the LLM's raw success/failure predictions when they conflict with the environment. The agent plans by asking this neurosymbolic world model to check candidate actions, receiving feedback and suggestions, and replanning until the model approves. On Mars and ALFWorld, the authors report state-of-the-art results, including 98% success on ALFWorld after four iterations and an average Mars score rising from 4.0% to 15.3%. If correct, the method shows that a small amount of complementary symbolic knowledge, not the LLM's prior alone, is what makes LLM-based world models reliable.","feed_headline":"LLM agent reaches 98% success by learning the world's rules","feed_subtitle":"Training-free symbolic rules fix the model's wrong predictions; Mars score jumps from 4% to 15.3%.","key_machinery":"The load-bearing mechanism is the neurosymbolic learning pipeline together with the world-model overlay. It runs in four stages: compare real and predicted trajectories and sort transitions into correctly and incorrectly predicted; use LLM inductive reasoning to extract action rules, a knowledge graph of requires/consumes/enables constraints, and a scene graph of spatial and global facts; translate these into executable code rules of the form `def CodeRule_m(obs, action, KG/SG): return feedback, suggestion, flag`; and prune the rule set with a greedy maximum-coverage algorithm that keeps only rules correcting mispredicted transitions. During model-predictive control, the MAPEXECUTE wrapper applies active code rules to the LLM world model's prediction; when a rule disagrees, the rule's output overrides the LLM and the agent uses the generated feedback to propose a new action. The world model's predictive task is deliberately reduced to binary success/failure on the assumption that the next observation follows deterministically from that outcome.","core_discovery":"The paper's central claim is that the gap between an LLM's prior knowledge and an environment's actual dynamics is narrow but decisive, and that the missing piece can be learned as compact symbolic knowledge. WALL-E 2.0 performs world alignment by running the LLM as a world model, comparing its predicted trajectories against real ones, and limiting the prediction task to whether each action transition succeeds or fails. It then prompts the LLM for inductive reasoning to extract action rules, a knowledge graph, and a scene graph, translates them into Python functions, and prunes them by solving a maximum set coverage problem over the initially mispredicted transitions. During planning, these code rules override the LLM's predictions when they disagree and supply feedback and suggestions to the agent, which replans in a model-predictive-control loop. The authors attribute the performance gains to this alignment: learned rules raise the cover rate of failed predictions from 0.0% to 92.3%, and the method outperforms all compared baselines on Mars and reaches 98% success in ALFWorld after four iterations.","pith_inferences":["The binary success/failure design implies the approach is best suited to rule-governed, deterministic environments; extending the rules to stochastic transitions would require probabilistic success rates rather than hard flags, a step the paper identifies as future work.","Since the code rules are compact and environment-specific, a natural transfer test would be to take a pruned rule set learned in one world and apply it to a structurally similar world, measuring whether alignment transfers without new exploration.","The maximum-coverage pruning objective is defined on the LLM's current mispredictions, so the selected rules may patch the model's specific errors rather than capture the full dynamics; validating rules on held-out mispredictions would separate these two explanations.","The MPC loop's efficiency rests on the LLM being a good action proposer; in environments where the LLM's heuristics are poor, the world model can veto bad actions but cannot invent good ones, so the ceiling may be set by the proposer's prior."],"forward_implications":["No RL or finetuning is needed to align an LLM world model: an iteratively updated set of code rules is sufficient for substantial prediction and planning gains in a new environment.","Because pruning keeps only rules that fix real mispredictions, the agent's knowledge stays compact and it does not need to carry the full history of past trajectories in context.","Encoding rules as executable code makes the world model's checks deterministic and reduces the variance of prompt-based rule following that hurts natural-language-rule methods.","Combining action rules with a knowledge graph and a scene graph is necessary: ablations show each component contributes, and the improvement comes mostly from applying the symbolic knowledge to the world model rather than to the agent's prompt.","The method is designed for environments that contradict the LLM's priors, and the reported gains are largest where other methods rely on misleading commonsense."],"supporting_citations":[{"why":"Supplies the Mars environment, its eight modified world types, and the IfR and Skill Library baselines that WALL-E 2.0 must outperform on reward and score.","marker":"Tang et al., 2024b"},{"why":"Supplies the ALFWorld benchmark with its 134 testing tasks and six task types, the setting for the reported 98% success rate.","marker":"Shridhar et al., 2020b"},{"why":"Defines WALL-E 1.0, the direct predecessor whose rule-based world alignment this paper extends with knowledge graphs, scene graphs, and model-predictive control, and which serves as a baseline.","marker":"Zhou et al., 2024"},{"why":"Provides RAFA, the strongest ALFWorld LLM baseline at 95% average success that WALL-E 2.0 surpasses.","marker":"Hao et al., 2023"},{"why":"Provides AdaPlanner, a skill-library ALFWorld baseline at 91% success whose trajectory-memory approach contrasts with WALL-E 2.0's inductive symbolic knowledge.","marker":"Sun et al., 2024"},{"why":"Provides DreamerV3, the RL world-model baseline trained for one million environment steps, against which WALL-E 2.0's 5,600-step budget is compared.","marker":"Hafner et al., 2023"}],"fun_headline_variants":["Training-free neurosymbolic rules lift LLM agents to 98%","LLM agent learns world's rules on the fly, hits 98% success","Symbolic world alignment: LLM agent beats baselines by 51.6%","No training, just rules: LLM agent reaches 98% in ALFWorld"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's rules assume the next state is determined by whether an action succeeds or fails; if the environment is stochastic, or a successful action can lead to different next states, the learned world model cannot faithfully predict what happens next.","fun_headline_variants_meta":{"raw":{"variants":["Training-free neurosymbolic rules lift LLM agents to 98%","LLM agent learns world's rules on the fly, hits 98% success","Symbolic world alignment: LLM agent beats baselines by 51.6%","No training, just rules: LLM agent reaches 98% in ALFWorld"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000699,"raw_usage":{"total_tokens":3215,"prompt_tokens":1062,"completion_tokens":2153,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":678,"completion_tokens_details":{"reasoning_tokens":2075}},"tokens_in":678,"tokens_out":2153,"duration_ms":15148,"temperature":1.0,"reasoning_tokens":2075,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:17:56.854888+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a deterministic benchmark, inject one stochastic transition — for example, make a mining action succeed with probability 0.5 because hostile creatures sometimes appear — and measure the code rules' cover rate on mispredicted transitions over iterations. If the rule learner labels the same action sometimes success and sometimes failure, the cover rate will stall near chance and agent success will degrade far below the reported 98% and 15.3%, confirming that the binary deterministic-transition assumption is the load-bearing premise.","supporting_citations":[],"review_version":1}