{"id":"a6d2dd10-ee94-4af6-8d49-7db18eb58868","arxiv_id":"2507.06529","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"A decision transformer, trained offline on ROI-filtered, early-stopped GP-ensemble rollouts and refined by sparse real evaluations, is proposed as a non-myopic policy for Bayesian optimization.","lead":"DRO trains a decision transformer on simulated Bayesian optimization rollouts, generated by an ensemble of Gaussian processes and a pool of acquisition functions, to directly propose next query points that minimize final regret. The paper reports lower simple regret than standard BO baselines across synthetic and hyperparameter tuning benchmarks, including high-dimensional Ackley problems.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim that DRO learns a non-myopic policy that directly minimizes final simple regret is not yet supported: every training action comes from a myopic acquisition function, and the return-conditioning signal may be inert, reducing DRO to an ensemble of myopic policies.","rationale":"The reader's weakest_assumption focuses on transfer from simulated GP trajectories with returns computed against each GP's own estimated optimum, and on the fixed target return 1.0. That is a real symptom, and I agree it is under-supported. However, the more fundamental issue is that even with a perfectly specified GP and an oracle optimum, the training distribution contains no non-myopic behavior to distill: all actions are generated by myopic acquisitions, and the decision transformer is an offline return-conditioned behavioral clone of those policies. The central claim therefore requires evidence that the return-to-go signal actually changes the action distribution and that the induced changes reduce real simple regret. The proposed return-ablation test directly settles this. This concern does not accuse the authors of misconduct; it identifies an unverified load-bearing assumption in the argument. Since the manuscript currently lacks this evidence, as well as code and statistical tests, the correct verdict remains CONDITIONAL: acceptance should require the return ablation and a comparison showing the learned actions differ from, and improve upon, the pool acquisitions. The reader and I overlap on the transfer/target mismatch but diverge on whether the main problem is model misspecification or the absence of any non-myopic expert signal in the training data, hence 'partial'.","tokens_in":17806,"tokens_out":6447,"duration_ms":83898,"concrete_test":"Run the same DRO pipeline on Ackley 10D and LunarLander with the return-to-go replaced by a constant (or by returns randomly permuted across trajectories), keeping architecture, data, and ROIs identical. If the return-ablated variant matches DRO's final best objective and convergence curve, then the return-conditioning is inert and the 'direct regret' mechanism is not responsible for the gains; if DRO is significantly better (with error bars and a paired test), the concern is resolved. As a secondary check, record whether the transformer's proposed action under target return 1.0 differs from the top candidates of the pool acquisitions at the same state.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The load-bearing step is the claim in §4.5 and §6 that the decision transformer \"directly minimizes final simple regret\" and learns a non-myopic policy. In Algorithm 1 and Appendix D.1, every action in every training trajectory is generated by maximizing a conventional one-step acquisition (EI, UCB, PI, MES) inside the ROI of an ensemble GP (D.4), and the return-to-go is computed as the final simulated best value minus each GP's own optimistic optimum f~m^* (D.1). There are no demonstrations from a non-myopic or multi-step planner, and no policy-improvement step based on rollout returns. The transformer is thus trained by return-conditioned imitation; it can only re-weight and interpolate actions already proposed by myopic acquisition functions, and it cannot introduce actions far outside the support of those demonstrators. The fixed target return 1.0 used at inference is not calibrated to the real objective's scale, so even the conditioning signal is not a well-defined regret target. Consequently, the plotted empirical gains in §6, which lack significance tests, may come from ROI filtering and ensemble diversity alone rather than from direct regret optimization. This is a correctness risk: the method's central novelty, non-myopic direct regret minimization, is not yet established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Direct Regret Optimization (DRO), a Bayesian optimization method that trains a decision transformer offline on simulated rollouts from an ensemble of Gaussian processes (GPs) with varying hyperparameters. Simulated trajectories are restricted to an adaptive Region of Interest (ROI), generated by rotating conventional acquisition functions (EI, UCB, PI, MES), and terminated by a Bayesian early stopping criterion. At inference, the transformer proposes the next query conditioned on a target return-to-go, and real evaluations update the GP ensemble. The authors claim that DRO jointly learns the surrogate and a non-myopic acquisition policy and explicitly minimizes final simple regret, supporting this with experiments on Ackley functions (2D–20D), three hyperparameter tuning tasks, LunarLander, and ablations on ROI filtering and ensemble size.","tokens_in":18075,"tokens_out":7127,"duration_ms":74291,"significance":"The intended contribution is timely: replacing hand-crafted myopic acquisition functions with an end-to-end learned, return-conditioned policy would address a recognized weakness of standard BO, and the concrete design of ensemble-based ROI-constrained rollouts plus dense offline training is a plausible recipe. The paper has clear strengths: it evaluates on external benchmarks, includes ablations for ROI filtering and ensemble size, and describes the simulated-data pipeline in enough detail to be reproduced with effort. If the central mechanism were verified, DRO would be a meaningful practical contribution. However, the paper currently overstates its central claim: no multi-step planning step is present in the training pipeline, the return-to-go target is not calibrated to real objective scale, and the empirical evidence consists of shaded mean curves without significance tests or numerical tables. These gaps are load-bearing for the abstract's claim that DRO 'explicitly targets minimizing the multi-step regret.'","major_comments":[{"comment":"The central claim that DRO learns a non-myopic policy that directly minimizes final simple regret is not supported by the described training procedure. In every simulated rollout, each action is obtained by maximizing a one-step acquisition function (EI, UCB, PI, or MES) over the current GP posterior within the ROI, and the decision transformer is trained to predict those actions conditioned on a return-to-go. This is return-conditioned imitation of myopic demonstrators: there is no multi-step planner, no rollout-based policy improvement, and no return-maximization step in Algorithm 1. As described, the transformer's action distribution is restricted to the support of the myopic base acquisitions, so any claimed non-myopic advantage must be empirically isolated. Please add an ablation that varies or removes the return conditioning—for example, compare DRO against an unconditional imitation baseline, a version with shuffled return-to-go values, and a true multi-step lookahead policy—to demonstrate that the return signal is doing causal work.","section":"§4.5, Algorithm 1, Appendix D.4"},{"comment":"The return-to-go used for training is defined relative to an estimate \tilde f^*_m described as 'an optimistic estimate of the true optimum from GP_m's perspective or a normalized target,' while inference conditions on a fixed target return-to-go of 1.0. The paper never specifies how \tilde f^*_m is computed, how the return-to-go is normalized across GP models and objective scales, or how the value 1.0 corresponds to zero simple regret on the real objective. Since the real objective's scale is unknown and the ensemble GPs can disagree about the optimum, the training signal is not demonstrably aligned with the final simple regret that the paper claims to minimize. Please specify the exact return-to-go formula, the normalization procedure, and the calibration of the inference target, and ablate over target values.","section":"Appendix D.1, §4.5"},{"comment":"The empirical claim of 'consistently outperforms BO baselines' rests on mean curves with ±1 standard error shading, but the paper provides no tables of final means and variances and no significance tests. In several panels, such as Adam Iris and Adam Breast Cancer in Figure 3, the shaded bands overlap substantially, so the qualitative claim is not supported for those tasks. Please report per-benchmark final simple-regret (or best-objective) means and standard errors, the number of trials, and pairwise significance tests across seeds (e.g., Wilcoxon signed-rank tests or bootstrap confidence intervals), and state explicitly which benchmarks support the 'consistent outperformance' claim.","section":"§5.1, §6"},{"comment":"The paper explicitly states that 'full theoretical analysis of DRO is future work,' and the two propositions provided concern only ROI preservation for base acquisitions and convergence of expected improvement under converging simple regret. Neither proposition addresses the regret of the learned decision-transformer policy. In light of the abstract's claim that DRO 'explicitly targets minimizing the multi-step regret,' this is a gap between the stated objective and the provided support. Either derive a formal statement of the DRO training objective and a regret or consistency argument for the learned policy, or rephrase the abstract, §4.5, and §6.1 to present DRO as a heuristic distillation method rather than a directly regret-minimizing method.","section":"§4.7"}],"minor_comments":[{"comment":"The y-axis tick labels in Figure 3 appear garbled (e.g., '10 1', '2 × 10 1'); please render the exponent notation with standard formatted tick labels.","section":"Figure 3"},{"comment":"Reference [4] contains editorial annotations such as 'Often cited tutorial; relevant for discussing integrated acquisition functions'; such annotations should be removed from the reference list.","section":"References"},{"comment":"Algorithm 1 refers to 'ROI parameters (e.g., β_t)', while Appendix D.4 states that the UCB/LCB constraint in the ROI definition uses κ = 6.0; please unify the notation and clarify which parameter controls the ROI.","section":"Algorithm 1 vs. Appendix D.4"},{"comment":"For the HPO tasks, the paper reports simple regret but does not state how the unknown global maximum f^* is estimated for XGBoost, Adam, or the other tasks; this affects the interpretation of the y-axis in Figure 3.","section":"§5.1"},{"comment":"Algorithm 1 retrains or fine-tunes the decision transformer at every real iteration, but the paper does not report the wall-clock or computational cost of this training relative to the cost of real evaluations; please add this information, since BO is motivated by expensive function evaluations.","section":"Algorithm 1 and §5.1"}],"recommendation":"major_revision","confidential_remarks":"The main risk is that the paper's headline novelty—non-myopic direct regret minimization—is not yet established by the described method or experiments. The proposed pipeline is a return-conditioned distillation of myopic acquisition functions, and the empirical evidence lacks significance testing and code release. I would encourage the editor to request the additional ablations and quantitative results as part of the revision, and to ask the authors to clarify how the return-to-go is normalized in practice. The related-work discussion would also benefit from a sharper positioning against EARL-BO and prior meta-learned acquisition approaches."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper combines known pieces into a genuinely new recipe: train a decision transformer on GP-ensemble simulated BO rollouts, constrained to a region of interest and early-stopped, then use it as an acquisition policy with sparse online refinement. Each ingredient has precedents, but the specific per-problem transformer-on-rollouts setup isn't in the cited literature. The ablations (ROI filtering, ensemble size, acquisition rotation) are useful and suggest the design choices matter. Credit also for the honest disclaimer that full theoretical analysis is future work; the propositions are modest consistency checks, not guarantees for DRO itself.\n\nThe soft spot is the central claim. The paper says DRO 'directly minimizes final simple regret' and learns a non-myopic policy. But every training action in a rollout is chosen by a conventional one-step acquisition (EI, UCB, PI, MES) inside the ROI. The transformer is trained to imitate those actions conditioned on a return-to-go that, as the appendix shows, is essentially the simulated final regret relative to each GP's own optimistic optimum. That makes DRO a return-conditioned imitation learner, not a policy optimizer. Unless the transformer can combine or reweight the demonstrators in a way that provably improves regret, the 'direct regret optimization' tag is not supported. Also problematic: at inference the target return is fixed at 1.0, with no discussion of how that maps to the real objective's scale. If the conditioning signal is uncalibrated, the transformer may just output whatever the myopic policies would do anyway.\n\nThe experiments are plot-only: no tables with means and variances, no significance tests, no code, and no comparison to the RL-based non-myopic BO baselines (EARL-BO, FSAF, RLABO) that appear in the related work. That makes the 'consistently outperforms' claim hard to verify. These are fixable issues, but they are real.\n\nThis paper is for BO researchers working on learned acquisition functions and meta-BO. It is a reasonable idea to try, and the ablations give useful signal. I would not cite it as evidence for non-myopic direct regret minimization, but I might cite it as related work in the decision-transformer-for-BO space.\n\nSend it to peer review, with the expectation of major revision. The recipe is worth discussing; the claims need to be scaled back or empirically substantiated with code, significance tests, RL baselines, and a demonstration that the learned policy actually does something non-myopic. If the authors can show that DRO's actions differ from the myopic demonstrators in a way that improves multi-step regret, the paper would be much stronger.","headline":"A novel combination of GP-ensemble rollouts and decision transformers for BO, but the central claim of direct regret minimization is not yet supported by the evidence.","tokens_in":18636,"tokens_out":3147,"would_cite":true,"duration_ms":33831,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DRO trains a transformer on simulated GP rollouts to directly minimize final simple regret in Bayesian optimization.","keywords":["Bayesian optimization","decision transformer","simple regret","non-myopic acquisition","Gaussian process ensemble","region of interest","offline reinforcement learning","dense training sparse learning"],"falsifier":"Run DRO on a benchmark whose true optimum sits in a narrow attraction basin that RBF GPs with lengthscales in [0.1, 10] are unlikely to cover, e.g., a multimodal function with a very sharp global peak; if the policy's final simple regret is no better than a myopic EI baseline in that setting, the simulation-to-real transfer assumption is falsified. A second check is to condition the transformer on target returns other than 1.0 and see whether its regret tracks the target; a flat response would show the return signal is not doing the claimed work.","tokens_in":17564,"feed_emoji":"🎯","tokens_out":8369,"duration_ms":88939,"temperature":0.7,"pith_summary":"The paper tries to show that Bayesian optimization can be built around a learned query policy instead of hand-crafted acquisition functions. It proposes Direct Regret Optimization (DRO), which trains a decision transformer offline on many simulated BO runs generated by an ensemble of Gaussian processes, restricted to adaptive regions of interest and stopped early when expected improvement is small. The transformer is trained to output the next query point that minimizes the final simple regret after a multi-step horizon, and at deployment it is conditioned on a target return-to-go of 1.0. Results on the Ackley function up to 20 dimensions, three hyperparameter-tuning tasks, and a LunarLander controller are reported to show that DRO reaches lower simple regret than standard GP-BO, TuRBO, PFNs4BO, and SCoreBO, with ablations indicating ROI filtering and ensemble size drive the gain.","feed_headline":"Transformer learns non-myopic Bayesian optimization from GP rollouts","feed_subtitle":"DRO replaces hand-crafted acquisition functions with a transformer that learns from simulated BO trajectories to cut simple regret.","key_machinery":"The central object is the decision transformer, a sequence model that maps histories of (state, action, return-to-go) triples to the next action. State features include each ensemble GP's hyperparameters, the best value found, the iteration number, and coordinates of the best points; actions are the query points chosen by a conventional acquisition during simulation; the return-to-go is the simulated simple regret relative to that GP's estimated optimum. Simulations are generated by an ensemble of Gaussian processes with varied hyperparameters, constrained to the adaptive Region of Interest $\\hat{X}_{m,t} = \\{x : \\mathrm{UCB}_{m,t}(x) \\ge \\max_{x'} \\mathrm{LCB}_{m,t}(x')\\}$, and truncated by a Bayesian early-stop rule based on maximum expected improvement falling below $\\delta = 10^{-4}$. At inference, the transformer is conditioned on a target return of 1.0 (zero regret) and directly proposes the next real query.","core_discovery":"DRO claims that a decision transformer can learn a non-myopic BO policy by distilling simulated trajectories rather than by optimizing a myopic acquisition at query time. The training signal is the return-to-go computed from each simulating GP's own estimate of the optimum; the policy is evaluated by the regret it achieves on the real objective. The paper asserts this policy consistently achieves lower final simple regret than the baselines and that the adaptive region-of-interest constraint and the diversity of the GP ensemble are the components that make the simulations informative. A second claimed component is the dense-training--sparse-learning split: abundant offline simulated rollouts train the policy, while the small number of real evaluations only updates the GPs and refines the policy online.","pith_inferences":["The paper defers a full theoretical analysis; its stated regret guarantee covers an ROI-constrained base acquisition, not the learned transformer policy itself, so the empirical transfer result is the load-bearing evidence.","A natural extension the authors do not test is varying the target return-to-go at inference to trace an exploration-exploitation frontier; if regret does not respond to the target, the return conditioning is not the active mechanism.","If the approach generalizes, the same distillation recipe could be applied with a high-fidelity simulator in place of GP rollouts, which would remove the GP-ensemble misspecification concern in domains where simulators exist.","The paper's robustness argument is most exposed when the GP ensemble's kernel family is wrong in a structured way, so a deliberate misspecification experiment (e.g., periodic or discontinuous objectives) would sharpen the claim."],"forward_implications":["Users can drop the per-problem choice of acquisition function and its hyperparameters, since the transformer embodies the acquisition policy.","Constraining simulated rollouts to the adaptive ROI is necessary for the method's performance; removing it degrades results substantially.","The ensemble size matters: more GPs (M=5 or M=10) yield more diverse training trajectories and better final objective values than M=1.","The dense-training/sparse-learning split lets the policy be trained largely offline on cheap simulations, with only a few real evaluations used to correct the GP ensemble online.","The same simulated-trajectory distillation recipe transfers, in principle, to constrained BO and multi-fidelity BO by changing what the simulator models and what the action represents."],"supporting_citations":[{"why":"Supplies the decision transformer sequence-modeling architecture that maps state, action, and return-to-go histories to actions.","marker":"[6]"},{"why":"Provides the Gaussian-process and acquisition-function implementation used to build the simulation engine and baselines.","marker":"[2]"},{"why":"Defines the TuRBO trust-region baseline and is one of the ROI-filtering inspirations the paper cites.","marker":"[12]"},{"why":"Provides the PFNs4BO transformer-surrogate baseline and the Bayesmark HPO evaluation setup.","marker":"[29]"},{"why":"Provides the SCoreBO baseline that handles GP hyperparameter misspecification, the comparison most relevant to DRO's robustness claim.","marker":"[19]"},{"why":"Supplies the YAHPO Gym benchmark used for the XGBoost hyperparameter tuning experiments.","marker":"[33]"},{"why":"Supplies the Gymnasium LunarLander environment used as the complex-simulation benchmark.","marker":"[40]"},{"why":"Underpins the ROI identification lemma that Proposition 1 extends to a regret guarantee for ROI-constrained acquisition.","marker":"[46]"},{"why":"Underpins the expected-improvement convergence lemma used to justify the Bayesian early-stop criterion.","marker":"[32]"}],"fun_headline_variants":["Transformer learns non-myopic BO from GP rollouts","DRO: transformer distills BO policy from simulated trajectories","Non-myopic BO via transformer trained on simulated rollouts","Simulated BO rollouts teach transformer to minimize regret","Direct regret optimization: transformer for non-myopic BO"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that training against approximate GP models, where success is measured by each model's own guess of the optimum, transfers to the real objective when the model is asked to produce a high target return. If the GP ensemble is systematically wrong about where the optimum is, or if the target return of 1.0 is not on the same scale as the real rewards, the learned policy could be optimizing a phantom objective.","fun_headline_variants_meta":{"raw":{"variants":["Transformer learns non-myopic BO from GP rollouts","DRO: transformer distills BO policy from simulated trajectories","Non-myopic BO via transformer trained on simulated rollouts","Simulated BO rollouts teach transformer to minimize regret","Direct regret optimization: transformer for non-myopic BO"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000229,"raw_usage":{"total_tokens":1457,"prompt_tokens":899,"completion_tokens":558,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":515,"completion_tokens_details":{"reasoning_tokens":480}},"tokens_in":515,"tokens_out":558,"duration_ms":6599,"temperature":1.0,"reasoning_tokens":480,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:01:32.810150+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run DRO on a benchmark whose true optimum sits in a narrow attraction basin that RBF GPs with lengthscales in [0.1, 10] are unlikely to cover, e.g., a multimodal function with a very sharp global peak; if the policy's final simple regret is no better than a myopic EI baseline in that setting, the simulation-to-real transfer assumption is falsified. A second check is to condition the transformer on target returns other than 1.0 and see whether its regret tracks the target; a flat response would show the return signal is not doing the claimed work.","supporting_citations":[{"cited_title":"Botorch: A framework for efficient monte-carlo bayesian opti- mization","cited_arxiv_id":null,"evidence_quote":"Provides the Gaussian-process and acquisition-function implementation used to build the simulation engine and baselines."},{"cited_title":"Scal- able global optimization via local bayesian optimization","cited_arxiv_id":null,"evidence_quote":"Defines the TuRBO trust-region baseline and is one of the ROI-filtering inspirations the paper cites."},{"cited_title":"Pfns4bo: In-context learning for bayesian optimization","cited_arxiv_id":null,"evidence_quote":"Provides the PFNs4BO transformer-surrogate baseline and the Bayesmark HPO evaluation setup."},{"cited_title":"Self-correcting bayesian opti- mization through bayesian active learning","cited_arxiv_id":null,"evidence_quote":"Provides the SCoreBO baseline that handles GP hyperparameter misspecification, the comparison most relevant to DRO's robustness claim."},{"cited_title":"Yahpo gym-an efficient multi-objective multi-fidelity benchmark for hyperparameter optimization","cited_arxiv_id":null,"evidence_quote":"Supplies the YAHPO Gym benchmark used for the XGBoost hyperparameter tuning experiments."},{"cited_title":"Finding interior optimum of black-box con- strained objective with bayesian optimization","cited_arxiv_id":null,"evidence_quote":"Underpins the ROI identification lemma that Proposition 1 extends to a regret guarantee for ROI-constrained acquisition."},{"cited_title":"Regret for expected improvement over the best-observed value and stopping condition","cited_arxiv_id":null,"evidence_quote":"Underpins the expected-improvement convergence lemma used to justify the Bayesian early-stop criterion."}],"review_version":1}