{"id":"a39c4a7e-3134-4b47-be08-fe8188ccbd05","arxiv_id":"2506.11456","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Fast p-KGFN generates node candidates from a single EIFN optimization and evaluates the p-KGFN acquisition function on a small discrete set, achieving up to a 16x speedup with competitive optimization quality.","lead":"The paper presents a faster way to run Bayesian optimization on function networks where some parts can be evaluated independently and at different costs. It cuts the computational cost of the previous p-KGFN algorithm by up to 16 times while staying close in optimization quality, which could make partial-evaluation optimization practical for real design problems.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 21-point heuristic set A in Section 4.2 replaces the continuous maximizer of ν_{n+1}(x; z_k) without error bounds or validation; if it omits the true maximizer, p-KGFN values are biased and the competitive query-efficiency claim is unsupported.","rationale":"The paper has genuine empirical support: code is available, experiments use 30 trials, and ablations show robustness to parameter choices. The central concern is not that the heuristic is obviously wrong, but that no check establishes that the small discrete set A is adequate for approximating the p-KGFN acquisition. This is the weakest link because the entire speedup comes from evaluating p-KGFN on A rather than solving the continuous problem, while the algorithm's decisions are driven by those values. I agree with the reader's weakest assumption: the set A may omit the true maximizer of the updated posterior mean. The proposed test would settle whether this matters in practice. I also note a secondary reproducibility gap: the batch-TS subset selection in Section 4.2 is defined as an argmax over subsets, but the algorithm used to solve it is not described; this should be clarified but is less central than the adequacy of A. The reader's CONDITIONAL verdict is appropriate, so no verdict change is needed.","tokens_in":14397,"tokens_out":8045,"duration_ms":84110,"concrete_test":"On AckMat and FreeSolv, for 30 random trials and a few iterations, compute both max_{x∈A} ν_{n+1}(x; ẑ_{n,k}) and max_{x∈X} ν_{n+1}(x; ẑ_{n,k}) (via multi-start L-BFGS or DIRECT on the exact GP posterior) for each candidate ẑ_{n,k}, using the same Monte Carlo samples of y_k. Record the relative error and whether the argmax node changes. If the argmax changes or the relative error exceeds ~5% in more than a small fraction of iterations, the discretization materially affects node selection and the query-efficiency claim needs qualification.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2 evaluates the p-KGFN acquisition (Eq. 4) by solving max_{x∈A} ν_{n+1}(x; z_k) instead of max_{x∈X} ν_{n+1}(x; z_k), with A = S_T ∪ S_L ∪ {x*_n}. With the experimental settings M=N_T=N_L=10, this is a 21-point discrete set in a continuous input space (7D for AckMat). The paper states this discretization is inherited from Buathong et al. (2024), but for Fast p-KGFN it is load-bearing because node selection in Algorithm 1 (step 10) uses these approximate α values to choose both node and input. If the updated posterior mean's maximizer is not near any point in A, the Monte Carlo estimate of the expected improvement is biased downward, and the algorithm may prefer a candidate whose true p-KGFN value is lower. The ablation in Appendix B only compares compositions of heuristic components of A; it never checks whether A contains points near the true continuous maximizer or how α values would change with a much richer set. Thus the central claim of maintaining competitive query efficiency is not yet supported by analysis; it rests on an unquantified heuristic.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes Fast p-KGFN, an accelerated variant of the p-KGFN acquisition function for Bayesian optimization of function networks with partial evaluations. Instead of solving a nested optimization problem for each node, the method generates one candidate input per node by optimizing a modified EIFN acquisition, draws a single realization from each GP posterior to compute intermediate outputs, and then evaluates the p-KGFN acquisition on a small discrete set A built from batch Thompson sampling, local points, and the current posterior mean maximizer. Experiments on AckMat, FreeSolv, and Manu report objective values close to p-KGFN with up to a 16x reduction in per-iteration runtime, along with ablations on the discrete set composition and its parameters.","tokens_in":14754,"tokens_out":5087,"duration_ms":49952,"significance":"If the claims hold, the paper is practically valuable because it substantially reduces the computational overhead of p-KGFN, making partial-evaluation BOFN applicable to problems where objective evaluations are only moderately expensive. The paper's strengths include reproducible code, results averaged over 30 trials with confidence intervals, runtime comparisons, and parameter ablations that show robustness. However, the central approximation, replacing the continuous inner maximization in p-KGFN with a small discrete set A, is heuristic and unquantified, and the paper does not describe how the batch Thompson subset-selection problem is solved. These issues temper the strength of the central empirical claim.","major_comments":[{"comment":"The replacement of max_{x∈X} ν_{n+1}(x; z_k) with max_{x∈A} ν_{n+1}(x; z_k) is load-bearing for node selection, but the paper provides no analysis or empirical validation of this approximation. The ablation in Appendix B only compares different compositions of A; it never checks whether A contains points near the true continuous maximizer, nor how the selected action would change with a much richer discrete set. Because the central claim of maintaining competitive query efficiency depends on this approximation, please add an experiment or analysis that quantifies its error, for example by comparing the objective values obtained against a variant that uses a much larger discrete set or continuous inner optimization on at least one problem, such as AckMat.","section":"§4.2, Eq. (4), Algorithm 1 step 10"},{"comment":"The method requires solving ST ∈ arg max_{S⊂X, |S|=N_T} (1/M) Σ_{j=1}^M max_{x∈S} f̂^A_j(x), but the paper does not specify how this combinatorial optimization over a continuous space is performed, what candidate pool is searched, or whether the solution is exact or approximate. This missing algorithmic detail is necessary for reproducibility and also affects the quality of A; please state the algorithm used (e.g., greedy selection from the set of sampled realizations' maximizers) and, if it is approximate, discuss the impact on the discrete set quality.","section":"§4.2, batch Thompson sampling subset selection"}],"minor_comments":[{"comment":"The label \"ActMat\" should be \"AckMat\" in the table header and the appendix section heading.","section":"Table 1 and Appendix A.1"},{"comment":"The phrase \"with replacing y*_{n,K} by ν*_n\" should be \"by replacing y*_{n,K} with ν*_n\".","section":"Algorithm 1, step 4"},{"comment":"The statement that \"a set containing Y_{J(k)} is known\" is vague; the main text should give a concrete example of how such a set is obtained (as the appendix does for the test problems) or explicitly refer to the appendix.","section":"§2"},{"comment":"The caption says \"comparing Fast p-KGFN against 8 baselines,\" but only seven baselines are listed alongside Fast p-KGFN; this should be corrected to \"7 baselines\" or the list should be adjusted.","section":"Appendix D, caption"},{"comment":"The ablation would be more informative if it reported the discrete set sizes for each configuration (for example, the default has |A|=21 while Thompson+Local has |A|=20), since the performance differences could be confounded with the size of A.","section":"Appendix B, Figure 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is a natural extension of the authors' ICML 2024 p-KGFN work and reuses much of that paper's experimental setup. The main weakness is the unquantified discrete-set approximation in Section 4.2, which the stress-test correctly identifies as load-bearing. I do not think rejection is warranted, because the empirical evidence is positive and the missing validation is within the scope of the manuscript to add. The missing description of the batch Thompson subset-selection solver is a more basic issue that should also be fixed. The paper fits the journal's scope well."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take. Fast p-KGFN is a pragmatic speedup of p-KGFN that replaces the expensive per-node nested acquisition optimization with two heuristics: generate one network candidate via a modified EIFN, and evaluate p-KGFN on a small discrete set A built from batch Thompson samples, local points, and the current posterior mean maximizer. On the three test problems (AckMat, FreeSolv, Manu), it achieves 4–16x speedups while remaining close to p-KGFN in final objective value. The ablations on A's composition and on the hyperparameters are useful and suggest the method is not knife-edge. Code is provided. That is the positive case, and it is a real one.\n\nThe soft spots are the two approximations. First, using A in place of X for the inner max in Eq. (4) is a standard KG discretization, but here A has only 21 points by default (10+10+1) in a continuous input space (7D for AckMat). The paper gives no evidence that A contains points near the maximizers of the updated posterior mean across Monte Carlo samples. The ablation in Appendix B only compares which ingredients of A to keep; it never asks whether a much richer A changes the decisions. If A omits the right maximizer for a node, the p-KGFN value is biased and node selection can go wrong. I don't think this sinks the paper—the empirical results are plausible—but it should be stated plainly as a heuristic, and ideally checked by comparing against a larger A on a couple of problems.\n\nSecond, the batch Thompson sampling subset selection is under-specified. Algorithm 1 says to solve argmax over S subset of X with |S|=N_T of an average of maxima over sampled functions. That is a combinatorial/infinite-dimensional problem; the paper does not say how it is solved (greedy? random candidate generation?). This is a reproducibility gap, not a fatal one.\n\nThe paper also inherits p-KGFN's assumptions (e.g., you can query any node with arbitrary inputs), and the conclusion acknowledges that. I'd like to see a note that the query-efficiency claim is empirical and problem-specific, not a guarantee.\n\nNet: this deserves a serious referee. It's a useful, honest engineering paper in the BO subfield. For a conference, the missing algorithmic detail and the unquantified discretization error should be fixed before acceptance.","headline":"A pragmatic speedup of p-KGFN with real empirical gains, but the two approximations that make it fast are under-analyzed and one subroutine is under-specified.","tokens_in":15168,"tokens_out":3301,"would_cite":false,"duration_ms":32378,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single global candidate replaces nested per-node search, giving up to a 16x speedup in Bayesian optimization of function networks while keeping query choices nearly unchanged.","keywords":["Bayesian optimization","function networks","partial evaluations","knowledge gradient","acquisition function","Gaussian process","computational speedup","expected improvement"],"falsifier":"For one of the paper's test networks, evaluate a node candidate $z_k$ at a point whose hypothetical observation shifts the updated posterior mean maximizer far from $A$ (for example, into a region not covered by local points and not sampled by batch Thompson draws), then compute $\\alpha_{n,k}(\\hat{z}_{n,k})$ using $A$ and using a dense global optimization of $\\nu_{n+1}$; if the rankings of nodes change and Fast p-KGFN selects a node that a correct p-KGFN value would not, the central speedup claim is refuted.","tokens_in":14229,"feed_emoji":"⚡","tokens_out":4729,"duration_ms":44276,"temperature":0.7,"pith_summary":"The paper tries to establish that the expensive nested acquisition optimization of p-KGFN can be replaced by a much cheaper procedure without losing much query efficiency. It proposes Fast p-KGFN, which generates one candidate network input via a modified expected-improvement problem, propagates it through sampled surrogate functions to form node-specific candidates, and then evaluates p-KGFN on a small discrete set rather than solving a nested continuous optimization problem for every node. Numerical experiments on three function-network problems report objective values close to p-KGFN per unit evaluation cost while cutting acquisition runtime by up to about 16x. If true, this widens the applicability of cost-aware partial evaluations to problems where the original acquisition cost dominated the savings.","feed_headline":"One global candidate replaces nested search: 16x faster network BO","feed_subtitle":"Fast p-KGFN keeps p-KGFN's query choices while cutting acquisition compute by evaluating candidates on a small discrete set.","key_machinery":"The load-bearing mechanism is a three-step shortcut: (1) a modified EIFN acquisition function $\\mathbb{E}[(y_K(x) - \\nu_n^*)^+ \\mid D_n]$ is optimized once to obtain a network-level candidate $\\hat{x}_n$; (2) one posterior sample of every node function is propagated through the DAG to simulate intermediate outputs $\\hat{y}_k(\\hat{x}_n)$, which are concatenated with the relevant external inputs to assemble node-specific candidates $\\hat{z}_{n,k}$; (3) the p-KGFN acquisition $\\alpha_{n,k}(\\hat{z}_{n,k})$ is evaluated over the small discretization $A = S_T \\cup S_L \\cup \\{x_n^*\\}$ instead of over $X$, where $S_T$ comes from a batch Thompson sampling scheme that selects $N_T$ points maximizing average performance across $M$ sampled networks and $S_L$ are random points near $x_n^*$. The nested optimization of the original method is thereby replaced by one continuous solve plus finite enumeration.","core_discovery":"The central claim is that the computational bottleneck of p-KGFN, namely optimizing a nested Monte Carlo acquisition function separately for every node, is avoidable. Fast p-KGFN replaces this with one optimization of EIFN modified to use the current posterior-mean maximum $\\nu_n^*$ as the baseline, then uses one shared GP sample path through the network to produce a candidate input for each node, and finally approximates the p-KGFN value of each candidate by maximizing the updated posterior mean over a discrete set $A$ containing batch-Thompson points, local points, and the current maximizer $x_n^*$. The paper reports that this variant achieves optimization performance close to p-KGFN and consistently beats full-evaluation baselines, while speeding up acquisition computation up to 16x.","pith_inferences":["The same 'one global solve plus discrete enumeration' template could accelerate other acquisition functions whose nested expectation requires solving an inner continuous optimization for each sampled outcome.","The speedup should grow roughly linearly in the number of nodes, since the original method solves one nested acquisition problem per node while the fast method solves one EIFN problem regardless of the network size; this is a testable scaling prediction.","The batch-Thompson representative subset idea, choosing a small set that performs well on average across many sampled networks, may be useful beyond p-KGFN as a cheap construction of promising sets in multi-fidelity or multi-objective settings.","A possible failure mode the paper leaves open is that on strongly multimodal function networks, the single EIFN candidate plus local neighborhood may systematically miss promising nodes whose inputs are far from the current maximizer, so the method may degrade where many separated optima exist."],"forward_implications":["Per-iteration acquisition runtime drops by 4.24x to 16.03x across the reported problems, with the largest speedup when p-KGFN is most expensive.","Optimization quality stays close to p-KGFN and, in the reported comparisons, consistently beats EIFN, showing that partial evaluations still earn their keep.","The benefits of partial evaluations grow as the evaluation cost of downstream nodes increases, in both p-KGFN and Fast p-KGFN.","The current posterior-mean maximizer $x_n^*$ in the discrete set is essential: ablations show that removing it degrades performance sharply.","The method applies whenever intermediate node outputs can be queried at arbitrary feasible inputs without first evaluating parents."],"supporting_citations":[{"why":"Defines the original p-KGFN acquisition function, the partial-evaluation problem setup, the test problems AckMat, FreeSolv, and Manu, and the local-point discretization baseline that Fast p-KGFN extends.","marker":"(Buathong et al., 2024)"},{"why":"Supplies the EIFN acquisition function and the conditionally independent Gaussian process model for function networks that Fast p-KGFN modifies and uses for inference.","marker":"(Astudillo and Frazier, 2021a)"},{"why":"Source of the discretization idea of optimizing over a finite set instead of a continuous space, which Fast p-KGFN adopts for fast acquisition computation.","marker":"(Scott et al., 2011)"},{"why":"Provides a further discretization approach for knowledge-gradient computation that the paper leverages to evaluate p-KGFN quickly.","marker":"(Ungredda et al., 2022)"},{"why":"Supplies the Gaussian process regression machinery used for the surrogate models of each node in the network.","marker":"(Williams and Rasmussen, 2006)"}],"fun_headline_variants":["16x speedup for network BO with one global simulation","Fast p-KGFN: one global candidate set, up to 16x faster","One Monte Carlo run, node-wise candidates: 16x less computation","Network BO without nested acquisition: 16x faster","p-KGFN accelerated: single global candidate per node, 16x less compute"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything rests on the heuristic that the small discrete set $A$ contains a point close to the true maximizer of each updated posterior mean $\\max_{x \\in X} \\nu_{n+1}(x; z_k)$; when that fails the p-KGFN estimate is wrong and the algorithm may pick the wrong node or input, and the paper provides no error bound for this approximation.","fun_headline_variants_meta":{"raw":{"variants":["16x speedup for network BO with one global simulation","Fast p-KGFN: one global candidate set, up to 16x faster","One Monte Carlo run, node-wise candidates: 16x less computation","Network BO without nested acquisition: 16x faster","p-KGFN accelerated: single global candidate per node, 16x less compute"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000535,"raw_usage":{"total_tokens":2546,"prompt_tokens":896,"completion_tokens":1650,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":512,"completion_tokens_details":{"reasoning_tokens":1555}},"tokens_in":512,"tokens_out":1650,"duration_ms":11793,"temperature":1.0,"reasoning_tokens":1555,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:04:34.636042+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For one of the paper's test networks, evaluate a node candidate $z_k$ at a point whose hypothetical observation shifts the updated posterior mean maximizer far from $A$ (for example, into a region not covered by local points and not sampled by batch Thompson draws), then compute $\\alpha_{n,k}(\\hat{z}_{n,k})$ using $A$ and using a dense global optimization of $\\nu_{n+1}$; if the rankings of nodes change and Fast p-KGFN selects a node that a correct p-KGFN value would not, the central speedup claim is refuted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the original p-KGFN acquisition function, the partial-evaluation problem setup, the test problems AckMat, FreeSolv, and Manu, and the local-point discretization baseline that Fast p-KGFN extends."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of the discretization idea of optimizing over a finite set instead of a continuous space, which Fast p-KGFN adopts for fast acquisition computation."},{"cited_title":"Efficient computation of the Knowledge Gradient for Bayesian Optimization","cited_arxiv_id":"2209.15367","evidence_quote":"Provides a further discretization approach for knowledge-gradient computation that the paper leverages to evaluate p-KGFN quickly."}],"review_version":1}