{"id":"f596c793-4bbd-4638-ab5f-9fc606e2bd3c","arxiv_id":"2601.03509","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"An evolving network of executable skills, repaired by LLM-based 'symbolic backpropagation' and gated by maturity, beats flat skill libraries on Minecraft and Crafter.","lead":"PSN treats an agent's skills as executable computer programs wired into a growing network, and uses LLMs to repair, stabilize, and reorganize those programs as new tasks arrive. Results on Minecraft and Crafter suggest this program-network approach adapts faster and forgets less than flat skill-memory agents such as Voyager.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Architecture-over-LLM claim is untested: all experiments use one LLM backend, so the central attribution is not established.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing concern: the 'architecture over LLM' thesis is asserted in Section 2 but never tested, because all experiments use gpt-5-mini-2025-08-07 and no backbone is varied. This is the single most important soft spot because the paper's headline contribution—that PSN's mechanisms, not the LLM, drive continual skill acquisition—rests on it. The evidence that does exist, notably PSN outperforming Voyager* with the same LLM and the PSN-w/o-Optimizer ablation, shows the full system is better than a flat library with the same backend, but it does not disentangle the architectural structure from the LLM-instantiated repair operators. The paper is still plausible and contains useful ablations, but the central causal claim is underdetermined. The correct verdict remains CONDITIONAL: accept only if artifacts are released, baselines are rerun under identical conditions, and ideally a backend-sensitivity check is provided. I therefore do not change the reader's verdict, but I flag the same assumption as the primary reason for conditionality.","tokens_in":38144,"tokens_out":3498,"duration_ms":37616,"concrete_test":"Run PSN and the same-LLM flat skill-library baseline (Voyager*) on the Minecraft tech-tree curriculum using at least two different LLM backends (e.g., gpt-5-mini and a Claude-class model), keeping prompts, API wrappers, and evaluation identical. If the PSN-vs-Voyager* gap in iterations-to-diamond and skill-retention rate is qualitatively preserved across backends, the 'architecture over LLM' claim gains support; if the gap collapses or reverses with one backend, the central attribution fails. As a secondary check, give Voyager* the same execution-trace feedback and REFLECT-style repair prompts on individual skills, without the compositional network, to see whether the trace-based credit assignment across skill compositions—not merely LLM repair capability—is what produces the advantage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that PSN's architectural scaffolding—executable program graphs, trace-based credit assignment, maturity gating, and rollback refactoring—drives continual learning, with Section 2 explicitly stating that learning dynamics 'emerge from the architectural choices of PSN rather than from the internal mechanisms of the LLM' and that the framework can be 'instantiated with different code generation backends while preserving its continual learning properties.' Yet no experiment varies the backend: all operators (CODEGEN, REFLECT, PATCH) use gpt-5-mini-2025-08-07 (§4.1). The PSN-vs-Voyager* comparison in Table 1 controls for LLM at a single value, but it does not test the claimed LLM-independence. Moreover, the ablation 'PSN w/o Optimizer' removes both REFLECT and gating, but since REFLECT is an LLM-prompted repair operator, this ablation conflates the architectural credit-assignment structure with the presence and format of LLM-generated repair. A flat skill library that receives the same execution traces and the same LLM repair prompts could plausibly reproduce much of the gain, which would undermine the 'architecture not LLM' thesis. Baseline comparability is also weakened by Table 1 inheriting ReAct/Reflexion/AutoGPT numbers from the Voyager paper, likely with different LLMs and harnesses. The most load-bearing gap is therefore that the central attribution is asserted but not experimentally isolated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Programmatic Skill Networks (PSN), a framework for continual skill acquisition in embodied environments where skills are represented as executable symbolic programs organized in a directed invocation graph. Three mechanisms are proposed: REFLECT for trace-based credit assignment across skill compositions, maturity-aware update gating (Eq. 6) that gradually stabilizes reliable skills, and online canonical refactoring with rollback validation. The authors also present an interpretive analogy to neural-network training, where credit assignment is likened to backpropagation, gating to learning-rate scheduling/freezing, and refactoring to architecture search. Experiments on MineDojo and Crafter report faster tech-tree progression, better skill retention, and more compact skill libraries than flat skill-library baselines such as Voyager.","tokens_in":38575,"tokens_out":3350,"duration_ms":35673,"significance":"If the empirical claims are supported, PSN would be a meaningful step toward structured, inspectable continual skill acquisition in open-ended embodied agents. The conceptual contributions—trace-based symbolic credit assignment over executable programs, maturity-gated updates, and rollback-validated refactoring—are well-motivated and could be useful beyond the specific LLM implementation. The paper is also transparent in its Limitations section about batch-size-one training and the absence of formal convergence guarantees. However, the current evidence does not establish the paper's central attribution that the architectural scaffolding, rather than the LLM backend, drives the observed gains. The lack of released code/data, small number of runs, and reliance on inherited baseline numbers also limit the strength of the empirical conclusions.","major_comments":[{"comment":"The central claim that PSN's learning dynamics 'emerge from the architectural choices of PSN rather than from the internal mechanisms of the LLM' is not experimentally isolated. All operators (CODEGEN, REFLECT, PATCH) use gpt-5-mini-2025-08-07 in every experiment; no run varies the backend. The ablation 'PSN w/o Optimizer' removes the optimizer, REFLECT, and gating, but it does not control for the presence or format of LLM-generated repair prompts. A flat skill library that receives the same execution traces and the same REFLECT/PATCH prompts could plausibly reproduce much of the gain. To support the attribution, the authors should either vary the backbone (e.g., a different LLM) or compare against a flat-library baseline that feeds identical trace and repair information to the LLM.","section":"§2 (LLM implementation), §4.1"},{"comment":"The headline result—faster tech-tree progression—is under-supported. The ReAct, Reflexion, and AutoGPT numbers are inherited from the Voyager paper, likely with different LLMs and harnesses; the comparison is not apples-to-apples. Only three runs are reported per method, and the obsidian result comes from a single successful run (1/3). While PSN's diamond result (51±9, 3/3) is better than Voyager's (102, 1/3), the variance and run count are too small to establish a robust difference, especially for the later stages. More runs and a fully matched baseline suite are needed.","section":"Table 1, §4.2"},{"comment":"The claimed benefits in cumulative reward, retention rate, and skill-library compactness are presented without error bars or confidence intervals. Figure 3, Figure 4, and Figure 6 appear to show single trajectories, so the reader cannot assess run-to-run variability. Since the paper's key qualitative conclusions (reduced forgetting, compact repertoire, higher reward) rest on these curves, the authors should report multiple seeds with interval estimates or at least overlay individual run curves.","section":"Figures 3–6, §4.3–4.4"},{"comment":"The offline-refactor comparison does not isolate the claimed benefit of online refactoring. Voyager-R is produced by Claude Opus 4.5, a different LLM from the gpt-5-mini used in PSN, and is evaluated on a single fixed task sequence with no error bars. Moreover, the reported refactor turns 58 skills into 65 skills (7 generic + 20 wrappers + 38 unchanged), so the comparison is not a clean test of 'compactness' or of online vs. offline structural change. The 0.6875 vs. 0.8462 difference is suggestive but not statistically grounded.","section":"§4.4 (Offline Refactor vs. Online Refactor)"},{"comment":"Maturity-aware update gating is a core mechanism, yet the hand-set constants in Eq. (6) (pivot 0.6, γ=5, ϵ=0.1), the rolling repair buffer of 5, and the 20% rollback threshold are used without sensitivity analysis. The paper's qualitative conclusions about stabilization and plasticity would be more robust if the authors reported how performance changes when these hyperparameters are varied. At a minimum, some motivation or a small sensitivity study is needed, because these choices directly affect the claimed stability-plasticity tradeoff.","section":"Eq. (6), §2.4"}],"minor_comments":[{"comment":"Typographical and naming inconsistencies: the paper alternates between 'Voyager' and 'Voyager' in text and table captions (e.g., Table 1 uses 'Voyager*' but the abstract uses 'MineDojo' with inconsistent hyphenation). Please standardize.","section":"Throughout"},{"comment":"The description of the Crafter environment as 'a Mineflayer-like Python API system' is vague; without code or a precise API specification, the reader cannot assess whether the implementation faithfully simulates the target environment.","section":"§4.1"},{"comment":"The prompt templates are useful, but the relation between the 'hybrid REFLECT' rule-based component and pure LLM-based reflection is unclear. Please describe the rule set or provide a reference.","section":"Appendix D"},{"comment":"The Limitations section honestly acknowledges batch-size-one optimization and the lack of formal projection guarantees. These are appropriate caveats, but they further argue for more extensive empirical validation rather than weaker wording.","section":"Limitations"}],"recommendation":"major_revision","confidential_remarks":"The paper's central 'architecture over LLM' claim needs a direct experimental test. If a flat skill library with identical trace-based prompts matches PSN's gains, the main thesis would collapse; if not, the paper would be much stronger. The small number of runs and inherited baselines are also a serious concern for a journal-level claim. I would encourage the editor to require either code/data release or a substantial increase in evidence before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is a genuine framework contribution whose central empirical claim is not actually tested. Worth reading for the design, but do not take the 'architecture not LLM' headline at face value.\n\nWhat's new: PSN integrates online trace-based credit assignment over a graph of executable program skills with maturity-aware update gating and rollback-validated refactoring. That specific combination is not in the prior work they cite. The two-phase optimizer (top-down feedback propagation, bottom-up patch application) is cleanly specified, and the appendices give real prompt templates, an algorithm, and actual code diffs from training runs. That is reproducible evidence in the sense that a reader can see exactly what the operators do. The refactor casebook is also useful.\n\nWhere it slips: the paper repeatedly says learning dynamics 'emerge from the architectural choices of PSN rather than from the internal mechanisms of the LLM.' That claim is untested. All experiments use gpt-5-mini-2025-08-07. No run varies the backbone. The ablation 'PSN w/o Optimizer' removes both REFLECT and the gating, so it conflates the graph structure with the presence of an LLM repair operator. A flat skill library that gets the same execution traces and the same repair prompts could plausibly reproduce much of the gain; that baseline is not run. This is the load-bearing gap.\n\nBaseline comparability is also weaker than the table suggests. Most numbers for ReAct, Reflexion, AutoGPT, and Voyager are taken from the Voyager paper, likely with different LLMs and harnesses. Only Voyager* is controlled for LLM, and it does not reach diamond. The statistical base is thin: three runs, obsidian from a single successful run, and no error bars on Figures 3–6. No code or data is released, though the paper says it plans to open-source. The hand-set constants (0.6, gamma=5, epsilon=0.1) get no sensitivity analysis—that's minor, but worth noting.\n\nCredit where due: the paper is honest about its limitations (batch-size-one, no formal guarantees), clearly written, and the related work is fair. The framework is coherent and the empirical results, while under-supported, are suggestive.\n\nWho this is for: anyone building LLM agents with persistent skills. A serious referee should see it, but the revision must release artifacts, rerun baselines under identical conditions, add variance reporting, and test at least one different LLM backend before the central attribution is accepted.","headline":"A coherent and genuinely novel framework for programmatic skill networks, but the central 'architecture not LLM' claim is untested—send it to review, not to press.","tokens_in":38988,"tokens_out":2457,"would_cite":false,"duration_ms":24412,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Programmatic skill networks claim faster, more stable continual learning than flat skill libraries: skills are executable programs in a graph, repaired via trace-based credit assignment and reorganized by rollback-validated refactoring.","keywords":["continual skill acquisition","programmatic skills","credit assignment","structural refactoring","maturity-aware gating","embodied agents","open-ended environments","LLM agents"],"falsifier":"Run PSN on the same two environments with two different LLM backends, or with a deliberately degraded repair operator, while keeping all scaffolding fixed; if tech-tree progression, Crafter reward, or retention differences shrink or reverse, the architecture-over-LLM claim is falsified. A minimal test is to disable REFLECT for a held-out set of new tasks and compare success rates against the full PSN on identical task streams.","tokens_in":38023,"feed_emoji":"🧩","tokens_out":5097,"duration_ms":50748,"temperature":0.7,"pith_summary":"The paper introduces the Programmatic Skill Network (PSN), where each skill is an executable symbolic program with preconditions and effects, and skills invoke one another, forming a directed graph. PSN's claim is that continual skill acquisition is driven by its scaffolding: fault localization over execution traces (REFLECT), maturity-aware gating that stabilizes reliable skills while keeping uncertain ones plastic, and canonical structural refactoring with rollback validation that keeps the network compact. The authors argue these mechanisms explain why PSN progresses faster through a Minecraft tech tree, earns higher cumulative reward in Crafter, retains mastered skills better, and maintains a smaller skill library than flat skill-library baselines. They also observe that these mechanisms mirror neural-network training—backpropagation, learning-rate scheduling, and architecture search—suggesting general principles for continual learning in symbolic systems. The paper itself flags batch-size-one online learning and the absence of formal convergence or projection guarantees as limitations.","feed_headline":"Programmatic skill networks beat flat libraries in open worlds","feed_subtitle":"Skills that compose, repair, and refactor themselves let an agent progress further with fewer attempts and less forgetting.","key_machinery":"The central object is the Programmatic Skill Network: a directed graph whose nodes are executable skills (code with control flow, parameters, preconditions, and postconditions) and whose edges are invocation links. Three mechanisms carry the argument: REFLECT, a trace-based fault-localization operator that propagates failure signals top-down along the execution trace and emits symbolic 'gradients' applied bottom-up as code patches; a maturity-aware update gate that lowers update probability as a skill's empirical success rate rises, with a soft pivot at 0.6 and a minimum update probability of 0.1; and an online refactor module that applies five canonical rewrites—parametric coverage, behavio","core_discovery":"On the paper's own terms, the discovery is that an embodied agent can continually improve not by relying on an LLM's memory or planning alone, but by storing skills as executable programs in a network and treating failures as signals for structured credit assignment. Given a failure trace, REFLECT decomposes responsibility along invocation links and produces localized repair proposals that are applied bottom-up as code patches; on success, a refactor module applies a fixed set of canonical rewrites—merging duplicates, extracting common subskills, and synthesizing missing abstractions—validated by rollback if performance drops. The reported evidence is that PSN unlocks diamond-tier tools in a","pith_inferences":["If the architecture-over-LLM claim is right, the same scaffolding should transfer to cheaper or weaker code-generation backends; a direct test is to vary the LLM while holding all PSN mechanisms fixed and measure whether the gains persist.","The five canonical refactor cases are pre-enumerated; a natural extension is to have the system discover new rewrite patterns from execution traces instead of relying on a fixed casebook.","The trace-based credit assignment is effectively a discrete analogue of backpropagation; this primitive could be reused in neuro-symbolic systems where differentiable modules sit below programmable controllers.","The paper's batch-size-one setup and lack of a projection guarantee invite stress tests: running parallel task streams or adversarial task orderings would reveal whether the maturity gate and rollback validation remain stable under higher throughput."],"forward_implications":["Persistent executable skills with pre/postconditions enable compositional generalization: unseen tasks can be solved by backward-chaining existing skills rather than synthesizing new ones from scratch.","Trace-based credit assignment is load-bearing for long-horizon tasks: ablating the optimizer reduces PSN to roughly flat-library performance on early and mid-tier tools and prevents reliable diamond or obsidian progression.","Maturity gating prevents oscillatory behavior: without it, converged skills are repeatedly modified by downstream failures, lowering cumulative task success.","Online refactoring outperforms offline compression: refactoring a static skill library with a strong LLM yields lower task success (0.6875) than PSN's online rollback-validated refactoring (0.8462).","PSN's dynamics resemble neural-network training—credit assignment as backpropagation, maturity gating as learning-rate scheduling, refactoring as architecture search—which suggests stability-plasticity tradeoffs are general principles for symbolic learning systems."],"fun_headline_variants":["Skills as code let agents adapt and generalize in open-ended tasks","Programmatic skill networks self-repair and refactor for open worlds","Evolving program skills: agents repair and reuse in MineDojo and Crafter","Composable skill programs enable continual learning without forgetting","From failures to fixes: programmatic skill networks learn continuously"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The claim that PSN's architecture, not the LLM backend, produces the learning gains rests on the untested premise that the REFLECT, CODEGEN, and PATCH operators behave stably across models; all experiments use a single LLM, so if code-generation or repair quality is the real driver, the framework's generality would fail.","fun_headline_variants_meta":{"raw":{"variants":["Skills as code let agents adapt and generalize in open-ended tasks","Programmatic skill networks self-repair and refactor for open worlds","Evolving program skills: agents repair and reuse in MineDojo and Crafter","Composable skill programs enable continual learning without forgetting","From failures to fixes: programmatic skill networks learn continuously"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000238,"raw_usage":{"total_tokens":1302,"prompt_tokens":653,"completion_tokens":649,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":397,"completion_tokens_details":{"reasoning_tokens":562}},"tokens_in":397,"tokens_out":649,"duration_ms":7085,"temperature":1.0,"reasoning_tokens":562,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T12:15:29.009728+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run PSN on the same two environments with two different LLM backends, or with a deliberately degraded repair operator, while keeping all scaffolding fixed; if tech-tree progression, Crafter reward, or retention differences shrink or reverse, the architecture-over-LLM claim is falsified. A minimal test is to disable REFLECT for a held-out set of new tasks and compare success rates against the full PSN on identical task streams.","supporting_citations":[],"review_version":1}