{"id":"f1ee9bb2-239f-4731-aae9-4625ac8cbaea","arxiv_id":"2608.03653","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A three-stage LLM search framework converts execution evidence into structural policies and discovers network dismantling programs with strong quality and low runtime.","lead":"AutoSND automatically searches for network dismantling heuristics by having an LLM generate candidate programs, log how each performs, and turn those logs into structural rules that guide the next round of search. On 12 real networks and three million-node networks, its two chosen programs complete every network with near-best fragmentation quality at about 2 to 3 seconds per run.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Single-seed, single-proxy search makes the claimed superiority of AutoSND-Q/S a favorable-draw possibility rather than a demonstrated method-level result; multi-seed/multi-proxy reruns are needed.","rationale":"The reader's weakest assumption correctly flags the single 500-node Powerlaw_500 search proxy and the transfer question. I agree that this is the most load-bearing point, but I would sharpen it: the deeper issue is not only transfer across evaluation graphs but the absence of any repeated-search statistics. Since the final Q/S candidates are selected after 500 stochastic LLM generations on one proxy, the reported external results are conditioned on one search trajectory and one proxy graph. A single successful draw is an existence proof, not a demonstration of 'better search performance and stability' as a property of the method. The paper does provide useful supporting evidence: the CoreHD-root rerun and DeepSeek-v4-flash rerun show the process is not entirely locked to one initial heuristic or one LLM, and the ablations give internal consistency. Those checks, however, hold the proxy fixed and do not quantify variance. If multi-seed/multi-proxy runs reproduce the Table 1 means within a small range and retain 12/12 coverage, the concern is resolved and the conditional acceptance can move toward full acceptance. If not, the headline comparative claim should be weakened to an existence result. The reader's CONDITIONAL verdict already reflects the need for more evidence, so I do not move the verdict; I only make the required condition more specific: release and analyze multi-seed, multi-proxy runs before treating the Table 1 ordering as definitive.","tokens_in":31181,"tokens_out":8555,"duration_ms":97964,"concrete_test":"Run the complete AutoSND pipeline with the same budgets (B1=300, B2=10, B3=200), generation settings, and endpoint-selection rules on at least 10 independent seeds and at least 4 proxy graphs: Powerlaw_500, ER(500, p≈0.004), WS(500, k=8, p=0.08), and one small real graph such as hamster or CEnew. Freeze Q/S per run as in Appendix A.3, then evaluate every frozen candidate on the 12 main graphs with the strict complete-sequence ANC(GCC) protocol. Report the distribution (min/median/max) of mean ANC, coverage, and average rank, plus a win-rate comparison against ERA, Clade-AHD, and GND. If the median external mean ANC exceeds about 11.1 or any proxy/seed combination drops below 12/12 coverage in more than one or two runs, the single-run Table 1 result is not a stable method-level claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"All empirical support for the central comparison rests on one stochastic execution of the pipeline. The frozen endpoints AutoSND-Q (cd9e3818033d) and AutoSND-S (0ade8d3405c2) are taken from a single source search on the 500-node Powerlaw_500 proxy with seed 42 (Table 15, Appendix E.1, Section 5.1.4). The search draws 500 LLM candidates at temperature 0.2 and selects endpoints on a proxy-specific ANC@30%-runtime frontier (Appendix A.3). Thus Table 1's 'lowest two means' and the Fig. 3 stability curves are outcomes of one favorable-looking draw from a high-variance process, not estimates of the method's typical behavior. No repeated-search statistics are reported; ablations, the CoreHD-root test, and the DeepSeek-v4-flash variation all use the same Powerlaw_500 proxy and do not quantify seed-to-seed or proxy-to-proxy variability. The abstract's 'search performance and stability' claim is therefore underdetermined: a single successful trajectory shows existence of a discovery path, not that AutoSND reliably produces competitive heuristics, and it leaves open that the external quality and coverage are selection artifacts of that particular proxy graph. This is the load-bearing assumption behind the comparative and stability claims.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AutoSND, a three-stage framework for automatic heuristic discovery for network dismantling. Stage I performs a broad LLM-driven tree search from an HDA root, archiving candidate code, execution metrics, and runtime. Stage II applies LLM-Struct to every archived candidate, aligns structures with quality/runtime/execution states, and compiles a structural policy (reuse/avoid/bound) from positive and negative evidence. Stage III performs policy-conditioned tree search with four branch seed types and selects quality- and speed-prioritized endpoints (AutoSND-Q/S) from a Pareto frontier. The main experiments evaluate the two frozen candidates on 12 real networks and 3 large networks, reporting lower mean ANC(GCC) than 18 baselines, near-100% candidate validity, complete coverage, and low runtime. Ablations test the contribution of Stage II and structural-family diversity, and a component-level analysis interprets the final programs as residual-degree backbones with bounded local corrections.","tokens_in":1598,"tokens_out":1577,"duration_ms":61325,"significance":"If the empirical claims hold, the paper makes a useful contribution to LLM-based automatic heuristic design by converting execution evidence into structural-level constraints, a step beyond scalar-fitness or language-reflection feedback. The strengths are substantial: the code is released; the large-network runs are strict complete-sequence verifications with hashes and independently recomputed ANC; and the component ablations in Appendix D.1 directly test which code structures matter. The lineage analysis in Section 5.3.2 and the structural-policy tracing in Appendix E provide unusually detailed provenance. The main risk is that the central comparative claim is established from a single search trajectory, which I detail below.","major_comments":[{"comment":"The entire comparative evaluation rests on one stochastic execution: search seed 42 on the single 500-node Powerlaw_500 proxy. The frozen endpoints AutoSND-Q (cd9e3818033d) and AutoSND-S (0ade8d3405c2) are selected from that run. Temperature-0.2 LLM generation is stochastic, so the reported mean ANC, coverage, and validity are outcomes of a single draw, not estimates of typical behavior. The DeepSeek-v4-flash and CoreHD-root experiments use the same seed/proxy, so they do not address seed sensitivity. The abstract's 'search performance and stability' claim is therefore underdetermined. Please run multiple search seeds (e.g., 5) and at least one additional proxy family (ER/WS/SBM), and report the distribution of external metrics.","section":"§5.1.4, §A.3, §E.1"},{"comment":"The 12-graph means for AutoSND-Q/S (10.99/10.93) are close to Clade-AHD (11.19) and ERA (11.12), yet no confidence intervals, standard errors, or significance tests are given. The claim that AutoSND is 'better' is based on point estimates over 12 graphs, some of which are near-ties. The average rank comparison (3.50 vs 4.92) also lacks a measure of variability. Please provide per-graph paired differences, standard errors, or a paired test against the strongest baselines.","section":"Table 1, §5.2.1"},{"comment":"All AHD baselines are local reproductions under a unified interface, not original implementations. The validity, coverage, and runtime comparisons may be influenced by reproduction choices. For example, LLM4CN completes 9/12 and HSEvo 9/12; these may not reflect the published methods' behavior. Unless the reproductions are validated against known outputs or the limitation is explicitly stated, the 'AutoSND vs. published methods' comparison is not a strict comparison. Please add a fidelity check or treat the comparison as 'AutoSND vs. local re-implementations' throughout.","section":"§5.1.3, §C.2"},{"comment":"The ablation interpretation is partially underspecified. The w/o-Stage-II variant has slightly lower mean ANC on the 8 common graphs (the text says 'slightly lower') but falls to 8/12 coverage. The conclusion 'Stage II mainly supports cross-network executability' rests on coverage differences over a small, non-random set of 8 graphs; no test is given. Similarly, the w/o-family variant changes both seed diversity and seed selection procedure, so it does not isolate the structural-family restriction. Please clarify what is held fixed and add error bars or tests for the ablation comparisons.","section":"§5.3.1, Fig. 4"}],"minor_comments":[{"comment":"The notation for complete-sequence ANC in Eq. (2) uses N both as node count and as the number of removed nodes, while the search metric in Eq. (7) uses K. A brief sentence distinguishing the full-sequence metric from the search-time ANC@30% would improve readability.","section":"§3.1, Eqs. (2)-(7)"},{"comment":"The figure caption states 'The red solid line denotes AutoSND-S' but does not label the curves for ERA, FunSearch, Clade-AHD, MCTS-AHD, and AlphaEvolve. Please add a legend or explicit color mapping.","section":"Fig. 3 caption"},{"comment":"The appendix states that PH and hamster have identical undirected simple edge sets and that Collaboration and GrQC are isomorphic, yet both pairs appear as distinct rows in Table 6. Clarify why both names are retained and whether the main-test label differs from the archive label only by provenance.","section":"§C.1, Table 6"}],"recommendation":"major_revision","confidential_remarks":"The core architecture is sound and the provenance apparatus is unusually thorough, but the headline claim is based on one seed and one proxy, and the AHD baselines are local reproductions. I would not accept without multi-seed/multi-proxy evidence and explicit statistical treatment of the 12-graph means. The stress-test concern from the reader about a single favorable draw is real and is the primary reason for this recommendation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is Stage II: compiling execution records into explicit structural policies that condition later generation. That is a real step beyond scalar fitness or language reflection, and the paper does a lot right. The three-stage tree search with archived execution evidence, LLM-Struct annotations, contrastive positive/negative evidence, and a compiled \"reuse, avoid, bound\" constraint is a solid mechanism. The discovered AutoSND-Q/S are complete executable heuristics, not just scoring functions, and the evaluation is broad: 12 real networks, 3 large real networks, component ablations, lineage analysis, a CoreHD-root robustness check, a model-swap experiment, and an influence-maximization transfer. The candidates themselves are competitive and fast, and the paper ships code and a reproduction manifest with hashes, which earns real trust.\n\nThe soft spot is precisely what the stress-test note flags: the central claim of \"better search performance and stability\" is underdetermined. All endpoints in Table 1 come from a single search run with seed 42 on Powerlaw_500. There are no repeated-search statistics, and the ablations mostly use the same proxy and seed. The CoreHD-root runs with seeds 42 and 43 give some additional evidence, but the improvement over CoreHD is only 1.2% on Q, so it does not substitute for repeated full-pipeline runs. Without multi-seed and ideally multi-proxy searches, the Table 1 margins could be a favorable draw. That does not invalidate the existence result—the paper shows a discovery path exists—but it does not support the reliability claim as stated. Also, several AHD baselines are local reproductions rather than original implementations, so the exact head-to-head margins should be read with care, though the appendix documents the reproductions unusually well.\n\nWho is this for? Researchers in LLM-based heuristic discovery and network dismantling. The mechanism is worth discussing and the paper deserves a serious referee. The revision should ask for multi-seed runs (at least three to five seeds) and confidence intervals for the 12-network means before the comparative claims are treated as definitive.\n\nRecommendation: send it to peer review with that request. The core idea is sound and the reported reproducibility artifacts make the work worth engaging with.","headline":"AutoSND's Stage II structural-policy induction is a real step forward for LLM-based heuristic discovery, but the paper's headline comparison rests on a single search seed on one synthetic proxy, so the reliability claim is not yet demonstrated.","tokens_in":32001,"tokens_out":2705,"would_cite":true,"duration_ms":32990,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Machine-found heuristics beat hand-built rules on 12 real networks","keywords":["network dismantling","automatic heuristic design","large language models","structural policy induction","tree search","complex networks","program synthesis"],"falsifier":"Re-run the full AutoSND search with the proxy replaced by an Erdős–Rényi or Watts–Strogatz 500-node graph on the same budget and seed, freeze the resulting Q/S candidates, and evaluate them on the 12 main graphs. If the frozen candidates drop below 12/12 coverage or their mean 100×ANC(GCC) rises above the best native baseline (about 12.5), then the transfer from the power-law proxy does not follow as described.","tokens_in":31163,"feed_emoji":"🕸️","tokens_out":5529,"duration_ms":57720,"temperature":0.7,"pith_summary":"AutoSND is a three-stage automated search that treats a complete network dismantling program—not just a node-scoring rule—as the object to discover. It first runs a broad tree search from a simple degree-based heuristic, archiving quality, runtime, execution states, and code structures; then it compiles these records into an explicit structural policy (reuse, avoid, bound) that governs local signals, neighborhood access, and update ranges; finally it continues tree search under that policy. The two frozen candidates, AutoSND-Q and AutoSND-S, achieve the lowest mean accumulated normalized connectivity among 18 methods completing all 12 real networks, finish all 12 within seconds, and also return valid removal sequences on million-node networks. If true, this shows execution evidence can be converted into reusable structural guidance, turning LLM-based heuristic design from isolated candidate screening into interpretable, transferable program discovery.","feed_headline":"Machine-found heuristics beat hand-built rules on 12 real networks","feed_subtitle":"A three-stage LLM search turns execution evidence into structural rules, yielding fast complete dismantling programs.","key_machinery":"The central object is the complete removal-sequence heuristic h: G ↦ π, combined with the Execution-to-Evidence Structural Policy Induction. Stage II uses LLM-Struct to give each archived candidate a unified description z_i over a vocabulary of structural features (degree, frontier, weak_tie, two_hop, boundary, redundancy, phase, heap), then forms contrastive evidence C1 = P ∪ N (positive versus poor-quality/slow/failed candidates) and statistics T1, and compiles them into an explicit policy Π = (A+, A-, B): what structures to reuse, what to avoid, and how to bound neighborhood access and state updates. Π is embedded in the Stage III generation prompt, so subsequent search is constrained by","core_discovery":"AutoSND claims that a three-stage LLM-driven tree search can discover complete executable network dismantling heuristics whose quality and runtime are competitive with—and on the tested networks better than—expert-designed and learning-based baselines. Stage II is the key step: every archived candidate is annotated with a unified structural description, aligned with its ANC@30%, runtime, and execution state, and compiled into an explicit structural policy Π = (A+, A-, B) that says what to reuse, what to avoid, and how to bound neighborhood access and state updates. Stage III continues generation conditioned on this policy and selects quality- and speed-prioritized endpoints from the quality-","pith_inferences":["The explicit policy induction could be decoupled from tree search and used to distill a corpus of previously run candidate programs into design rules for other algorithm-discovery settings; the paper tests this only for network dismantling and one influence-maximization transfer.","The comparison depends on a single 500-node synthetic proxy; searching on a small portfolio of proxies spanning power-law, Erdős–Rényi, Watts–Strogatz, and stochastic block model graphs would test the transfer claim more directly than the current single-proxy design.","The final score formula resembles classical dismantling ideas (degree, CI, CoreHD) but with explicit caps on access and updates; ablating the caps alone would quantify how much of the gain comes from bounded computation versus the local signals themselves.","A practical test is to use the induced policy Π as a prompt-level prior for zero-shot generation without tree search, comparing validity and quality to see whether the policy itself, rather than the search process, carries the discovered knowledge."],"forward_implications":["Complete heuristics, not scoring functions: the discovered objects output a full removal sequence, so they can be used directly in robustness and vulnerability analysis.","Structural policies make search interpretable: Stage II turns one run's execution evidence into explicit reuse/avoid/bound rules, preventing costly whole-graph rescans from recurring in later generations.","Transfer across scales: candidates frozen from a 500-node synthetic power-law proxy produce valid complete sequences on 12 real networks and on million-node networks, suggesting the induced structure is not overfit to proxy size.","A reusable pattern: the final programs support a testable heuristic-design principle—residual degree backbone plus bounded local corrections plus restricted update scope—that can seed future manual or automatic designs.","Cross-task feasibility: the paper reports that the same three-stage process transfers to influence maximization, improving quality and runtime over compared controls, though this is presented as a limited feasibility result."],"supporting_citations":[{"why":"Defines the ANC(GCC) connectivity metric and supplies the three large real networks (Facebook, YouTube, Flickr) used for scalability evaluation.","marker":"[8]"},{"why":"Supplies the simple HDA heuristic used as the Stage I root and as a native baseline.","marker":"[13]"},{"why":"Formalizes the network dismantling task and the removal-sequence evaluation that AutoSND targets.","marker":"[3]"},{"why":"Provides the LLM program-search paradigm that AutoSND extends with evidence-to-structural-policy guidance.","marker":"[25]"},{"why":"A reflection-based LLM hyper-heuristic baseline and prior work on using historical experience to guide later generation.","marker":"[28]"},{"why":"Tree-search baseline (MCTS-AHD) that AutoSND builds on and compares against under a unified interface.","marker":"[32]"},{"why":"Clade-level selection baseline relevant to AutoSND's Stage III branch seed selection.","marker":"[15]"},{"why":"Prior LLM-based critical-node identification approach that AutoSND contrasts with complete-program search.","marker":"[18]"}],"fun_headline_variants":["AutoSND: LLM search mines execution data for faster network dismantling","From run logs to rules: AutoSND auto-discovers network dismantling heuristics","Three-stage LLM search turns execution traces into structural policies","AutoSND beats hand-crafted dismantling heuristics via evidence-driven search","LLM writes network dismantling heuristics from observed execution evidence"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The search is run and the final candidates are frozen using one 500-node synthetic power-law graph as the only proxy; the reported quality and coverage assume that structural patterns learned on this single graph transfer to real networks of different sizes, densities, and formation mechanisms.","fun_headline_variants_meta":{"raw":{"variants":["AutoSND: LLM search mines execution data for faster network dismantling","From run logs to rules: AutoSND auto-discovers network dismantling heuristics","Three-stage LLM search turns execution traces into structural policies","AutoSND beats hand-crafted dismantling heuristics via evidence-driven search","LLM writes network dismantling heuristics from observed execution evidence"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000884,"raw_usage":{"total_tokens":3656,"prompt_tokens":747,"completion_tokens":2909,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":491,"completion_tokens_details":{"reasoning_tokens":2813}},"tokens_in":491,"tokens_out":2909,"duration_ms":21479,"temperature":1.0,"reasoning_tokens":2813,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T15:13:07.094887+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the full AutoSND search with the proxy replaced by an Erdős–Rényi or Watts–Strogatz 500-node graph on the same budget and seed, freeze the resulting Q/S candidates, and evaluate them on the 12 main graphs. If the frozen candidates drop below 12/12 coverage or their mean 100×ANC(GCC) rises above the best native baseline (about 12.5), then the transfer from the power-law proxy does not follow as described.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the simple HDA heuristic used as the Stage I root and as a native baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Formalizes the network dismantling task and the removal-sequence evaluation that AutoSND targets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the LLM program-search paradigm that AutoSND extends with evidence-to-structural-policy guidance."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Clade-level selection baseline relevant to AutoSND's Stage III branch seed selection."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior LLM-based critical-node identification approach that AutoSND contrasts with complete-program search."}],"review_version":1}