{"id":"6b687cf3-7d2e-4b34-b905-f950bb24601f","arxiv_id":"2607.22238","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"PolyBO fits a polynomial to the observed experimental data, labels random search-space points with its predictions, and adds them to the Bayesian-optimization surrogate, cutting iterations to a target regret on high-dimensional benchmarks.","lead":"This paper introduces PolyBO, a Bayesian optimization variant that creates extra 'pseudo-experimental' data points from a polynomial model trained on the few real data points, and uses them to guide the search. On 20-dimensional synthetic benchmarks it reports needing 42% fewer iterations to match plain Bayesian optimization, and 96% fewer on a simulated material-composition problem.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 4 is singular in the reported high-dimensional regime: degree-4 polynomial in 20D has 10,626 coefficients but training starts with 2 points; no regularization is described, so the algorithm as specified cannot produce the reported results.","rationale":"The reader's verdict is CONDITIONAL, and my concern supports that conclusion, so no verdict change is needed. The reader's weakest_assumption focused on the HEANN simulator being a faithful proxy for real experiments, which is a valid external-validity concern. However, the more elementary internal issue is that the high-dimensional experiments cannot be reproduced from the method as written because the normal equation is singular. The reader's rationale briefly mentions 'the stated polynomial fit is underdetermined in high dimensions,' so my concern overlaps with part of their reasoning, but it was not the weakest_assumption they selected. I treat this as the most load-bearing concern because it affects both the synthetic and real-world headline claims: if the numerical solve is undocumented, the reported speedups could be an artifact of an unspecified pseudo-inverse or regularization, and the experiments cannot be independently checked. The concrete test is straightforward: inspect the code and, if regularization is present, sweep λ to see whether the speedup survives. This is not an accusation of misconduct; it is a request for a missing methodological detail that is essential for reproducibility.","tokens_in":18727,"tokens_out":4432,"duration_ms":47533,"concrete_test":"Inspect the released repository (github.com/funalab/PolyBO) and locate the polynomial regression weight computation. Verify whether it implements Eq. 4 literally or uses np.linalg.pinv/lstsq or Ridge. Then rerun the D=20 BBOB suite (F01–F24) with the literal normal equation replaced by an explicitly documented ridge solve (vary λ ∈ {0, 1e-8, 1e-4, 1e-2}) and recompute the median I50. If no ridge/pinv is present and the code crashes or uses an undocumented fallback, the reported 42% median reduction is not established by the manuscript as written; if performance is insensitive to λ over 1e-8–1e-4, the unstated regularization is not load-bearing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"PolyBO's key experiments (D=20 BBOB, k=2 initial points, p=4) require solving Eq. 4: w = (Φ^T Φ)^−1 Φ^T y. The feature vector Φ(x) includes all power and interaction terms up to degree 4, giving C(20+4,4) = 10,626 coefficients. With only k+t experimental points (t starts at 1, max 100), Φ^T Φ is 10,626×10,626 with rank at most n ≪ 10,626, so the inverse in Eq. 4 does not exist. The manuscript states the normal equation is solved analytically but does not mention pseudo-inverse, ridge regularization, or any rank-deficiency handling in Methods or Algorithm 1. The same issue affects the real-world D=10 case (C(14,4)=1,001 parameters vs. initial k=20). Consequently, the published algorithm as written cannot be executed in the regime where the 42% median speedup is reported; if the public code silently uses a pseudo-inverse or regularizer, the reported improvements may depend on that unstated choice, and the label 'low-capacity polynomial regression' is misleading for p=4, D=20. This is a reproducibility and specification gap at the core of the method, independent of whether the NN surrogate is valid.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces PolyBO, a Bayesian optimization method that augments the GP surrogate with pseudo-experimental points generated by an adaptively refit polynomial regression model. At each iteration, a degree-p polynomial is fit to the observed experimental data, m' input locations are sampled uniformly in the search space, their responses are predicted by the polynomial, and the resulting pseudo-data are combined with the experimental data to train the GP; EI (or GP-UCB) is then used to select the next point. The method is evaluated on 24 BBOB functions in D=2,5,10,20 with 30 seeds against vanilla BO, BOPP, and (at D=20) TSBO, reporting a median 42% reduction in iterations at D=20. It is also applied to a high-entropy-alloy composition problem using a neural-network simulator, reporting a 96% reduction relative to BO-EI/RL-DQN. The paper includes hyperparameter sensitivity and ablation studies.","tokens_in":19142,"tokens_out":9036,"duration_ms":80907,"significance":"If the mechanism is sound, PolyBO is an attractive low-cost way to inject exploration into BO in high-dimensional, small-data settings, and the benchmark evaluation is extensive: 24 BBOB functions, 30 seeds, four dimensions, several baselines, plus sensitivity and ablation analyses. The authors state that the code is available. However, the central algorithmic specification is incomplete: Eq. (4) is singular in the exact regime where the headline speedup is reported, and the 'real-world' application rests on an unvalidated neural-network simulator. The significance is therefore conditional on resolving these specification and validation gaps.","major_comments":[{"comment":"The normal equation w = (Phi^T Phi)^{-1} Phi^T y is not well-defined for the reported settings. With D=20, p=4, the polynomial feature map has C(24,4)=10,626 coefficients; with initial dataset size k=2 and at most 100 iterations, the number of training points is at most 101, so Phi^T Phi is rank-deficient and has no inverse. The same holds for the D=10 HEA experiment (C(14,4)=1,001 parameters, k=20) throughout the early iterations in which the reported I1500 is achieved. The paper never mentions a pseudo-inverse, ridge term, or rank-deficiency handling. Consequently, Algorithm 1 as written cannot be executed in the regime where the 42% median speedup is reported. Moreover, the label 'low-capacity polynomial regression' (Abstract, Methods) is misleading for p=4,D=20: the model has far more parameters than data. The authors must specify the actual solver (e.g., Moore-Penrose pseudoinverse,","section":"Materials and Methods, Eq. (4); Algorithm 1; Tables 1-3"},{"comment":"The 'real-world' claim is unsupported. The objective values come from the neural-network predictor of Xian et al. [46], not from physical synthesis experiments, and the paper provides no evidence that this predictor reproduces the true response surface of high-entropy alloys. The reported 96% reduction is a reduction in simulator iterations, not in wall-clock experiment time; the Methods define optimization time solely as the number of optimization iterations. Without validation against physical measurements or at least a clear statement of the simulator's fidelity, the abstract's claim that PolyBO 'achieves efficient optimization in settings where each experiment requires a long time' and the 'real-world' language in the Results overstate what is demonstrated. Please either temper the real-world framing or add validation/discussion of the simulator's accuracy.","section":"Application to a real-world material composition optimization problem; Results"}],"minor_comments":[{"comment":"The I50 metric is computed relative to the simple regret of vanilla BO at iteration 50, but for many functions at D=5 (and some at D=2) the reference regret is never reached within the 100-iteration budget, producing dashes in Table 1. The reported median reduction of 6% at D=5 is based on only four calculable functions (F04, F05, F06, F16), so the statement that PolyBO's low-dimensional performance is poor is much less robust than the high-dimensional claim. Please report the number of functions used for each median and discuss the censoring (unreached reference) explicitly.","section":"Performance comparison with vanilla BO; Table 1"},{"comment":"The proposed-point entropy PE is defined as the differential entropy of the predictive distribution at the proposed point; the formula assumes a Gaussian predictive distribution. Please state that assumption explicitly when introducing PE.","section":"Methods, Eq. (7)"},{"comment":"For the discrete constrained HEA problem, the pseudo-data sampling step says 'uniformly sample from G_delta' but does not specify how the equality constraint is enforced for pseudo-data. Please clarify the sampling procedure.","section":"Supplementary Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is promising and the benchmark effort is substantial, but the unstated rank-deficient solver for Eq. (4) is a serious reproducibility gap at the core of the method. I recommend major revision, asking the authors to specify the exact linear-algebra routine used (pseudo-inverse, ridge, etc.), verify that the reported results are reproducible with that specification, and reassess the 'real-world' claim in light of the simulator-only evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: PolyBO is a genuinely different way to generate pseudo-data for BO — fit a polynomial to the current experimental data, sample pseudo-points across the whole search space, refit every iteration, discard old pseudo-data — and the benchmark sweep is broader than most papers in this niche: 24 BBOB functions, 30 seeds, D=2/5/10/20, with BOPP and TSBO baselines where appropriate. The ablation and sensitivity analyses are honest and useful; the authors show low-D gains are marginal and that m' and p matter.\n\nThe problem is the core fitting step. With D=20 and p=4, the polynomial has 10,626 coefficients, but the algorithm starts with k=2 and never has more than ~102 experimental points. Eq. 4 writes w=(Φ^T Φ)^-1 Φ^T y, and that inverse does not exist in this regime. No ridge, pseudo-inverse, or rank-deficient solve is mentioned in Methods or Algorithm 1. The code is public, so the gap may be in the writeup rather than the software, but as published the paper cannot produce its own headline numbers. That's a reproducibility issue at the center of the method, not a stylistic one. The D=10 real-world case has the same issue early on.\n\nThe real-world claim also needs a caveat: '96% shorter optimization time' is measured on a neural-network simulator of high-entropy alloys, in iterations, not wall-clock, and the surrogate is not validated against physical measurements. That's a reasonable proxy for a methods paper, but it is not evidence about real experiments.\n\nThe I50 metric comes from the authors' own prior work. That is not disqualifying — the paper also shows simple-regret curves — but I'd want an independent or at least more standard metric before leaning on the 42% number.\n\nBottom line: the idea is worth engaging with, and the evaluation is above average for this area. But a referee should insist on a precise statement of how Eq. 4 is solved when Φ^T Φ is singular, and ideally a rerun with explicit regularization and sensitivity to it. The paper deserves peer review, not desk rejection, but it needs a revision that makes the fitting step reproducible. I'd read the code before citing the speedups.","headline":"Promising pseudo-data BO idea with broad BBOB coverage, but the high-D results rest on an underspecified normal equation that is singular as written, so verify the code before trusting the 42%/96% speedups.","tokens_in":19560,"tokens_out":3766,"would_cite":true,"duration_ms":38336,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Augmenting Bayesian optimization with polynomial-regression pseudo-data cuts required iterations by a median of 42% in 20 dimensions and 96% on a simulated alloy task.","keywords":["Bayesian optimization","pseudo-experimental data","polynomial regression","high-dimensional optimization","small-data optimization","experimental design","material composition","exploration-exploitation tradeoff"],"falsifier":"An independent re-implementation on the same 24 noiseless benchmark functions at D=20 should reproduce a median I50 near 29 (interquartile range comparable to Table 1); a median above 50 would refute the core claim. For the alloy result, a reader could run PolyBO and the reinforcement-learning baseline on 50–100 real high-entropy alloy syntheses and check whether the median number of real experiments to match the baseline's final figure of merit is still near 59 rather than near 1500.","tokens_in":18684,"feed_emoji":"🧪","tokens_out":6212,"duration_ms":52319,"temperature":0.7,"pith_summary":"The paper aims to establish that when real experiments are slow and scarce, Bayesian optimization can be made to converge to a target performance in fewer iterations by having a low-capacity polynomial regression model, refit on the current experimental data at every iteration, generate pseudo-experimental points across the search space, and by training the Gaussian process surrogate on the combined real-plus-pseudo dataset. This matters because experimental optimization problems in biology and materials science often have dozens of variables and each evaluation can take weeks; if the claim holds, PolyBO turns a typical 50-experiment campaign into roughly 30 experiments on 20-dimensional problems and, on the simulated alloy composition task, a 1500-experiment search into a median of 59 experiments. The paper's own evidence is measured in optimization iterations rather than wall-clock time, and the 'real-world' alloy result is obtained on a neural-network simulator, not in a physical laboratory.","feed_headline":"Pseudo-data from polynomial fits cut optimization iterations by 42%","feed_subtitle":"Adaptive polynomial pseudo-labels beat vanilla BO by a median 42% in 20-dimensional benchmarks and 96% on a simulated alloy task.","key_machinery":"The load-bearing object is the p-th degree polynomial regression model fp(x) with power and interaction terms, fit by solving the normal equation on the current experimental data. At each iteration it produces m' pseudo-experimental points (x', fp(x')) at uniformly sampled locations in the search space; the Gaussian process surrogate is trained on D0:t-1 ∪ D'_t, and the pseudo-set is regenerated from scratch each iteration with constant size m'. The polynomial is chosen because its low capacity keeps it stable when the experimental sample is tiny (initial set size k=2), while its flexibility still reshapes the posterior in high dimensions; the per-iteration reset prevents early low-quality p","core_discovery":"The paper's central claim is that a per-iteration refresh of pseudo-experimental data generated by an adaptively updated polynomial regression model shifts Bayesian optimization toward earlier exploration in high-dimensional search spaces, letting it locate promising regions within the first 20–40 iterations and reach the simple regret that vanilla BO attains in 50 iterations after a median of 29 iterations at D=20 (a 42% reduction). On the simulated 10-element high-entropy-alloy composition problem with discrete and constrained search space, PolyBO reached the performance of the reinforcement-learning baseline after a median of 59 iterations versus 1500, a 96% reduction. The mechanism is th","pith_inferences":["A direct extension the paper leaves implicit is that the same pseudo-data augmentation should transfer to any surrogate (not only a Gaussian process) whenever the pseudo-labels come from a low-capacity regressor refit each round; the argument depends on the update structure, not on the particular kernel.","The paper's 42% and 96% figures are guarantees only in iteration counts; converting them to wall-clock savings requires the per-iteration overhead of fitting the polynomial and sampling m' points to be negligible relative to the experiment time, which is plausible for multi-week experiments but unproven.","The strong real-world claim rests on the neural-network alloy predictor; a direct test would be to run PolyBO against the reinforcement-learning baseline on a small set of physical syntheses. If the simulator overestimates smoothness, the iteration advantage should shrink on rugged response surfaces, consistent with the paper's own observation that gains are smallest on functions like Sharp Ridge ","A practically useful calibration rule would be to set m' in the 5–25 range and choose an even degree p—but the paper's memory scaling warning suggests that in very high dimensions (D≫20) even p=4 may be computationally prohibitive, so adaptive degree selection remains an open problem."],"forward_implications":["If the claim is correct, on 20-dimensional benchmark landscapes PolyBO reaches the simple regret of vanilla BO at 50 iterations after a median of 29 iterations, implying nearly half the number of real experiments for the same outcome.","On the discrete 10-element alloy composition problem, PolyBO reaches the performance of the reinforcement-learning baseline after a median 59 iterations rather than 1500—a 96% reduction—which would translate into drastically fewer synthesis runs if the simulator is faithful.","The performance gain is robust to the acquisition function (EI vs GP-UCB) and to the pseudo-set size up to m'=200, with an optimum around m'=5–25, so the method does not require fine-tuning of these hyperparameters.","The polynomial degree should be even (p=4,6,8) for best results on the tested functions; p=1 is competitive only when the true response is nearly linear, and degree increases memory cost roughly as the square of the number of polynomial terms.","Both parts of the update mechanism—discarding previous pseudo-data and keeping m' constant—are necessary; ablations that keep old pseudo-data or scale m' by iteration converge worse."],"fun_headline_variants":["PolyBO: pseudo-data slashes optimization iterations by 42%","Pseudo-experimental data speeds up Bayesian optimization by 42%","Adaptive polynomial pseudo-labels cut BO iterations by 42% to 96%","96% fewer iterations for alloy optimization with PolyBO","PolyBO uses pseudo-experiments to cut optimization time by 42%"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the neural-network predictor used for the high-entropy-alloy problem reproduces the true experimental response surface; if the simulator does not match real synthesis behavior, the measured 96% reduction in iterations will not carry over to the laboratory.","fun_headline_variants_meta":{"raw":{"variants":["PolyBO: pseudo-data slashes optimization iterations by 42%","Pseudo-experimental data speeds up Bayesian optimization by 42%","Adaptive polynomial pseudo-labels cut BO iterations by 42% to 96%","96% fewer iterations for alloy optimization with PolyBO","PolyBO uses pseudo-experiments to cut optimization time by 42%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000711,"raw_usage":{"total_tokens":3059,"prompt_tokens":785,"completion_tokens":2274,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":529,"completion_tokens_details":{"reasoning_tokens":2181}},"tokens_in":529,"tokens_out":2274,"duration_ms":15682,"temperature":1.0,"reasoning_tokens":2181,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T05:21:54.282438+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"An independent re-implementation on the same 24 noiseless benchmark functions at D=20 should reproduce a median I50 near 29 (interquartile range comparable to Table 1); a median above 50 would refute the core claim. For the alloy result, a reader could run PolyBO and the reinforcement-learning baseline on 50–100 real high-entropy alloy syntheses and check whether the median number of real experiments to match the baseline's final figure of merit is still near 59 rather than near 1500.","supporting_citations":[],"review_version":1}