{"id":"f5bd80da-1d3e-4e2a-876a-12ad521f81fe","arxiv_id":"1908.01362","paper_version":2,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"ASNets learn generalized planning policies from small instances and solve all 18,300 large Blocksworld test instances after training on 50 small ones.","lead":"ASNets are neural networks that learn a general policy for a planning domain by copying a traditional planner on a few small problems, then use that policy to solve much larger problems without search. This paper extends the architecture and shows it solves thousands of large Blocksworld instances and several other planning domains, with some limitations.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim rests on the unproven assumption that small training problems contain the structure needed for large-instance transfer; the headline Blocksworld result may be sensitive to training curriculum and test-set selection, so a fresh held-out replication is needed.","rationale":"The reader's weakest-assumption analysis correctly identifies the small-to-large transfer assumption as the load-bearing point. I agree that the paper provides strong empirical support on seven domains and discloses failures on others. My concern sharpens the reader's point: the two strongest pieces of evidence, especially the 18,300-instance Blocksworld result, do not fully separate the learned policy's transfer ability from the choice of training curriculum, hyperparameter selection, per-state LM-cut heuristic inputs, and the possibility that the test set was used to guide development. This is a standard robustness concern for empirical learning papers, not a fatal flaw. The paper is unusually transparent about its assumptions and limitations, and the public code makes a direct replication feasible. Because the central claim is explicitly an empirical claim and the existing evidence is substantial, I would not change the ACCEPT verdict; I would, however, recommend that future work or a revision report seed variance and a strict held-out evaluation. The concrete test above would settle whether the 100% coverage is a reliable property or an artifact of a favorable training/test configuration.","tokens_in":55188,"tokens_out":9014,"duration_ms":104675,"concrete_test":"Using the released code, retrain the Section 5.5 Blocksworld policy from three fresh random seeds, each with a newly drawn set of 50 training problems of 8-10 blocks, then evaluate on a freshly generated held-out set of 18,300 instances drawn from the same BWSTATES/BWKSTATES distribution. Report coverage per seed and per size/tower-count stratum. If any seed fails to reach near-100% coverage, the headline result is seed- or test-selection-dependent rather than a robust property of the learned policy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that imitation on a handful of small problems yields a generalized reactive policy that transfers to much larger instances. The load-bearing assumption, stated in Section 4.1, is that training problems contain structural elements representative of those in larger problems. This is not formally justified, and Section 5.3.3 reports failures on n-Puzzle and Sokoban, showing the assumption can fail. The strongest evidence is the 18,300-instance Blocksworld result, but that result was produced after explicitly modifying hyperparameters and training setup for the extended experiment (Section 5.5), and no seed variance or separate validation set is reported. The 18,300 test instances may therefore have functioned as a development target, inflating the apparent transfer reliability. Additionally, the policy receives per-state LM-cut landmark and action-count inputs (Section 3.5), so part of the apparent generalization may come from an external heuristic rather than the learned weights. The paper is transparent about limitations and shares code, so this is a robustness concern, not an accusation; but the transfer assumption remains the least secure load-bearing point.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Action Schema Networks (ASNets), a graph-convolutional architecture for learning generalized policies in (P)PDDL planning domains. The network is structured by the relatedness of ground actions and propositions, with weight sharing across the same lifted action schemas and predicates, so that a fixed set of parameters can be applied to problems of arbitrary size. Policies are trained by imitation learning from a teacher planner on a handful of small instances, then executed greedily on much larger held-out instances. The paper extends the earlier ASNet architecture with position-aware pooling and skip connections, adds LM-cut landmark and action-count inputs to mitigate the receptive-field limitation, and reports experiments on seven probabilistic and deterministic domains, an extended Blocksworld evaluation of 18,300 instances, ablations, and an interpretability study using L1-regularized sparse policies.","tokens_in":55446,"tokens_out":5131,"duration_ms":55394,"significance":"If the results hold, this is an important contribution to generalized planning and to structured deep learning for planning. The architecture is a clean and general way to obtain size-invariant neural policies with strong inductive biases, the comparison against strong non-learning baselines is extensive and fairly presented, and the authors honestly report domains where the approach fails (n-Puzzle, Sokoban, Parking, Thoughtful). The paper ships code, provides detailed cost/coverage tables, and demonstrates a genuinely interpretable policy for Triangle Tireworld. The main limitations are the absence of a formal generalization guarantee and the reliance on hand-engineered heuristic input features, both of which the paper acknowledges; the empirical strength of the flagship 18,300-instance result is the key remaining question.","major_comments":[{"comment":"The headline claim that the trained policy solves all 18,300 test instances is reported after modifying the hyperparameters and training setup from Section 5.1 (20 instead of 16 channels, 50 instead of 25 training problems, six instead of two hours, dropout 0.3 instead of 0.1). No seed variance or separate validation set is reported, and the text does not describe a model-selection protocol that would rule out using the extended test set to choose these settings. As a result, the 100% coverage figure is best interpreted as an optimistic point estimate rather than a reliable generalization rate. Please report coverage over multiple training seeds and clarify the selection protocol, or soften the claim accordingly.","section":"Section 5.5"},{"comment":"The receptive-field limitation means the architecture alone cannot solve large instances in most domains; the LM-cut landmark features and action-count features are essential, as shown by the 'no h.' rows in Table 1 and the receptive-field experiments in Appendix C. This does not invalidate the method, but it does mean the claimed generalization is a property of the ASNet-plus-heuristic-features system rather than of the learned weights alone. The abstract and conclusion should qualify the claim, for example by stating that the policy is learned on top of a domain-independent landmark heuristic.","section":"Section 3.5"},{"comment":"The hyperparameters were selected by automated tuning and manual interpolation to maximize coverage on the benchmark problems, and the same benchmark suite is later used to evaluate the final system. This creates a selection bias that is not quantified. Please report a sensitivity analysis over the flagged hyperparameters (dropout, L2 coefficient, learning rate, hidden dimension) or a nested evaluation with a separate validation set, so that the comparison against untuned off-the-shelf baselines is not systematically disadvantaged.","section":"Section 5.1.1"}],"minor_comments":[{"comment":"The caption says 'four drive actions related to at(shakey, kitchen)', but the text in Section 3.3 uses at(shakey, hall) for this example; the caption should be corrected to match the text.","section":"Figure 4 caption"},{"comment":"The sentence 'the ELU is simply the identify function on [0,∞)' contains a typo: 'identify' should be 'identity'.","section":"Section 6.2"},{"comment":"The column layout of Table 2 is confusing; please add explicit labels for the instance groups (e.g., '18 blocks, constrained towers', '25 blocks, random towers') so that the row structure matches the 18,300 total.","section":"Table 2"},{"comment":"The phrase 'containing structural elements representative of those found in larger problems' is vague; a brief concrete characterization (e.g., range of tower counts, goal configurations) would help readers understand the transfer assumption.","section":"Section 4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper has already appeared in JAIR 68 (2020), so this review should focus on the version at hand. The main risk to the published record is the unsupported precision of the 18,300-instance claim: if the authors can supply a multi-seed evaluation or clearly state the selection protocol, the claim would be much more robust. The reliance on LM-cut features is a significant caveat for how the contribution is framed in the abstract, but it is transparently discussed in the body. I do not see any indication of misconduct; the paper is honest about failures and limitations."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this for the empirical case that small-instance imitation training can yield policies that transfer to much larger problems in the same domain. The ASNet architecture itself is an incremental extension of the authors' 2018 paper — position-aware pooling, skip connections — but the paper earns its keep through the scale and honesty of the evaluation. The 18,300-instance Blocksworld result is the kind of thing the field needed: trained on 50 instances with 8–10 blocks, the policy solves all 18,300 with 18–50 blocks. They also report clear failures (n-Puzzle, Sokoban) and ablation results that isolate which components matter.\n\nThe main soft spot is the one you'd expect from an empirical generalisation claim: the paper assumes small training problems contain the structural elements needed for larger ones. That is stated plainly in Section 4.1 and it is not formally justified. It holds on seven domains but fails on two, so it is a real assumption, not a stylistic one. The large Blocksworld experiment is also somewhat tuned: Section 5.5 describes modified hyperparameters, more training time, and a different training set; with the Section 5.1 settings, some instances fail. No seed variance is reported, and the 18,300 test instances may have functioned as a development target. This doesn't sink the paper, because the authors disclose all this, but it does mean the 'near 100% coverage' claim is narrower than it sounds.\n\nA second, minor point: the network receives LM-cut landmark features and action counts as inputs. The 'no h.' ablations show that these external heuristics are important. So part of the generalisation is coming from a domain-independent heuristic that the planner already knows, not purely from the learned weights. The paper is upfront about this, and the ablation makes it visible.\n\nOverall this is a well-executed, carefully reported empirical contribution. The math is straightforward — weight-sharing over relatedness positions — and the citation pattern is appropriate. I see no load-bearing flaw, just a caveat about how much to extrapolate from the large-scale result.\n\nWho is it for: researchers working on learning generalised policies or neural planners. A serious editor should send it to referees; I would take it. My recommendation: accept, with a request for seed variance or a genuinely held-out test set if feasible.","headline":"A solid, transparent empirical paper on learning generalised planning policies; the headline Blocksworld result is real but rests on a tuning choice and an unproven transfer assumption.","tokens_in":55934,"tokens_out":1901,"would_cite":true,"duration_ms":22342,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By tying network weights to planning-domain schemas, the paper claims a policy learned on small problems transfers to much larger ones.","keywords":["generalised planning","action schema networks","deep learning","PPDDL","imitation learning","reactive policies","Blocksworld","probabilistic planning"],"falsifier":"Train an ASNet on Blocksworld problems whose goal towers contain misplaced blocks only near the top, then test on an instance where the only misplaced block sits at the bottom of a tall tower; if the network's actions loop or fail there, the transfer assumption is violated. A more direct check is the paper's own receptive-field experiment: a fixed-depth ASNet should fail on a chain-length problem longer than its number of proposition layers.","tokens_in":55009,"feed_emoji":"🧱","tokens_out":5343,"duration_ms":59687,"temperature":0.7,"pith_summary":"The paper claims that a single neural network with weights tied to the schemas of a planning domain can learn a generalised reactive policy by imitating a traditional planner on a few small problems, and that this policy then solves much larger instances from the same domain. The load-bearing evidence is an experiment in which training on 50 Blocksworld problems with 8 to 10 blocks yielded a policy that solved all 18,300 test instances with 18 to 50 blocks. Across seven probabilistic and deterministic domains, the network matches or beats heuristic-search planners in cumulative coverage once its two-hour training period is accounted for, solving several instances the planners could not solve within three hours. The paper also reports that sparsity regularisation can shrink a trained policy to a small set of human-readable equations, and it explicitly notes domains where this transfer recipe fails, including n-Puzzle and Sokoban.","feed_headline":"One neural policy solves 18,300 Blocksworld instances","feed_subtitle":"A graph-based network imitates a planner on tiny cases, then transfers that control knowledge to much larger problems.","key_machinery":"The central mechanism is the action-proposition relatedness graph: each action module receives, at each layer, the hidden representations of propositions appearing at unique positions in its action schema, while each proposition module pools over related actions grouped by schema and position. Weight tying across modules instantiated from the same action schema or predicate makes the parameter set $\theta$ independent of problem size, and first-layer inputs add proposition truth values, goal indicators, action applicability, LM-cut landmark indicators, and action counts. The output layer uses a masked softmax to produce a distribution over applicable actions. This construction is what lets a fixed set of weights act as a generalised policy over an entire lifted-SSP domain.","core_discovery":"An ASNet is a neural network instantiated over the graph of ground actions and propositions of a (P)PDDL problem, with shared weights that depend only on action schemas and predicates, so the same parameter set applies to every instance of a domain. Trained by imitation learning against the value estimates of a heuristic-search teacher on small problems, the resulting reactive policy is claimed to transfer to much larger instances, with the strongest result being 18,300 of 18,300 Blocksworld test instances solved. The paper presents the architecture's position-sensitive pooling, skip connections, and heuristic input features as improvements that make such transfer possible, while acknowledging structural limits: no support for quantifiers or arbitrary goal formulae, a receptive-field limit on chains of reasoning, and failure to produce reliable policies for n-Puzzle and Sokoban.","pith_inferences":["An implication the author leaves implicit is that the practical bottleneck for this approach is training-set design: to generalise, the small training problems must contain the rare structures (deep towers, dead-end configurations, long chains) that will appear in larger instances, and the paper's failures on n-Puzzle and Sokoban illustrate that this is not automatic.","A natural extension would be to combine ASNet policies with search at test time, using the learned policy to guide rather than replace a planner; the paper mentions this direction but does not evaluate it.","Because the sparse Triangle Tireworld policy reduces to eight nonzero weights, a further step would be to verify such policies formally against a specification of the domain, turning learned control knowledge into a checked artefact rather than an empirical claim.","The ASNet recipe of schema-derived graph structure plus shared weights and heuristic inputs could plausibly transfer to numeric and concurrent planning domains, though the paper leaves those extensions open. "],"forward_implications":["A policy trained on small instances can solve problems that heuristic-search planners cannot solve within a three-hour budget, as demonstrated on Blocksworld, CosaNostra Pizza, Triangle Tireworld, and other domains.","The same architecture can handle both probabilistic and deterministic PPDDL domains because probabilistic semantics only enter through training data; the network itself reads the same relational structure.","Heuristic input features such as LM-cut landmarks and action-count history are essential in several domains, and removing them sharply reduces coverage.","The receptive-field limit implies that a fixed-depth network cannot generalise across domains where correct actions require reasoning over arbitrarily long relatedness chains, as shown by the paper's own chain-length experiments.","Sparsity regularisation can reduce a policy to very few nonzero parameters, producing a lifted policy that a human can read and check against the domain definition. "],"supporting_citations":[{"why":"Supplies the original ASNet architecture that this paper extends with position-sensitive pooling, skip connections, and heuristic inputs.","marker":"Toyer et al. (2018)"},{"why":"Defines PPDDL, the formalism in which the domains and lifted-SSP families are specified.","marker":"Younes & Littman (2004)"},{"why":"Provides the Blocksworld instance generator and hardness analysis used for the training and the 18,300-instance evaluation.","marker":"Slaney & Thiébaux (2001)"},{"why":"Supplies LRTDP, the default teacher planner and one of the probabilistic baselines.","marker":"Bonet & Geffner (2003)"},{"why":"Supplies the LM-cut heuristic, which yields the landmark features fed to ASNets and also serves as a planner baseline.","marker":"Helmert & Domshlak (2009)"},{"why":"Introduces Triangle Tireworld, the probabilistic domain used both for evaluation and for the interpretable sparse-policy analysis.","marker":"Little & Thiébaux (2007)"},{"why":"Provides the IPC 2008 learning-track domains Gold Miner and Matching Blocksworld used in the evaluation.","marker":"Fern, Khardon, & Tadepalli (2011)"}],"fun_headline_variants":["ASNets: deep learning for scalable planning policies","Neural policy from small cases solves 18k instances","Generalised planning via action-schema networks","Imitate small, solve big: ASNets transfer to 18k","Interpretable neural policies for planning domains"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The small training problems are representative of the larger test problems, so the structure needed for a general policy is present in the training set; the paper offers strong empirical evidence in some domains but no formal transfer guarantee.","fun_headline_variants_meta":{"raw":{"variants":["ASNets: deep learning for scalable planning policies","Neural policy from small cases solves 18k instances","Generalised planning via action-schema networks","Imitate small, solve big: ASNets transfer to 18k","Interpretable neural policies for planning domains"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00094,"raw_usage":{"total_tokens":3998,"prompt_tokens":904,"completion_tokens":3094,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":520,"completion_tokens_details":{"reasoning_tokens":3016}},"tokens_in":520,"tokens_out":3094,"duration_ms":24315,"temperature":1.0,"reasoning_tokens":3016,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T15:14:53.563172+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train an ASNet on Blocksworld problems whose goal towers contain misplaced blocks only near the top, then test on an instance where the only misplaced block sits at the bottom of a tall tower; if the network's actions loop or fail there, the transfer assumption is violated. A more direct check is the paper's own receptive-field experiment: a fixed-depth ASNet should fail on a chain-length problem longer than its number of proposition layers.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the original ASNet architecture that this paper extends with position-sensitive pooling, skip connections, and heuristic inputs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines PPDDL, the formalism in which the domains and lifted-SSP families are specified."},{"cited_title":"\\ \\ Thi \\'e baux, S","cited_arxiv_id":null,"evidence_quote":"Provides the Blocksworld instance generator and hardness analysis used for the training and the 18,300-instance evaluation."},{"cited_title":"\\ \\ Geffner, H","cited_arxiv_id":null,"evidence_quote":"Supplies LRTDP, the default teacher planner and one of the probabilistic baselines."},{"cited_title":"\\ \\ Domshlak, C","cited_arxiv_id":null,"evidence_quote":"Supplies the LM-cut heuristic, which yields the landmark features fed to ASNets and also serves as a planner baseline."},{"cited_title":"\\ \\ Thi \\'e baux, S","cited_arxiv_id":null,"evidence_quote":"Introduces Triangle Tireworld, the probabilistic domain used both for evaluation and for the interpretable sparse-policy analysis."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the IPC 2008 learning-track domains Gold Miner and Matching Blocksworld used in the evaluation."}],"review_version":1}