{"id":"6427fbaa-5347-4990-8977-9abafd282ebc","arxiv_id":"2501.15554","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"BoTier is a smooth, auto-differentiable hierarchical scalarization that encodes tiered preferences in Bayesian optimization and, on the paper's benchmarks, reaches threshold-satisfying solutions faster than Chimera and EHVI.","lead":"This paper introduces BoTier, a new scalarization that lets Bayesian optimization encode tiered preferences, for example maximizing reaction yield before minimizing reagent cost. It is implemented as an auto-differentiable composite objective in the BoTorch library and benchmarked on analytical surfaces and emulated chemistry problems.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (2) saturates: once every objective is above its threshold, Ξ = sum_i t_i exactly, so the exact scalarization is flat (zero gradient) and discontinuous at threshold crossings; benchmark metrics reward this saturation, so the claimed general superiority for maximizing hierarchical objectives is…","rationale":"The reader correctly flags the smooth-approximation transfer, but the more fundamental issue is visible in the exact Eq. (2) itself. The statement that min(ψ_i,t_i) preserves continuity is false at superordinate threshold crossings, and the cap makes Ξ constant on the entire set of threshold-satisfying points. Because the paper frames its examples as maximizing yield and minimizing cost/temperature, and compares against Chimera (which adds a primary-objective term once all thresholds are met), this flatness is not cosmetic. It means the smooth finite-k approximation is not a numerical convenience but the mechanism that produces any preference among feasible points, and the reported benchmarks measure time-to-threshold-satisfaction as the headline success criterion. On that criterion BoTier’s flatness is aligned with the metric, so the broader claim of having 'never performed worse ... particularly as a composite objective' is only supported for satisficing, not for continued maximization. The chemistry-emulator results also lack a k-sensitivity analysis, unlike the analytical surfaces. This does not invalidate the engineering contribution—the batch-evaluable, auto-differentiable BoTorch implementation is a real and useful addition—but it changes the scope of what is claimed. A conditional verdict is appropriate: the paper should either add a term that activates when all thresholds are met (e.g., Chimera-style primary-objective continuation), or explicitly reframe BoTier as a threshold-satisfaction tool with correspondingly narrower empirical claims. If the proposed post-threshold test shows Chimera also saturates, the concern would be weakened; if it shows BoTier is flat, the paper should be revised accordingly.","tokens_in":13792,"tokens_out":8610,"duration_ms":85012,"concrete_test":"On the DH4* analytical surface (or the Suzuki–Miyaura emulator), set the primary threshold t1 low enough that a large fraction of the search space satisfies it, and keep t2 (and t3) as in Tables S1/S4. Run 50 independent BoTier-composite and Chimera-composite campaigns; after the first evaluation that satisfies all thresholds, record the best observed primary objective ψ1 at each subsequent evaluation. If BoTier’s post-threshold ψ1 curve is flat while Chimera’s continues to rise, the saturation of Eq. (2) is confirmed and the 'never performed worse' claim does not cover actual objective maximization beyond thresholds.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"In Eq. (2), each objective enters as min(ψ_i,t_i). For any point with ψ_j ≥ t_j for all j, Ξ = Σ_i t_i exactly, independent of the values of ψ_i. The exact objective therefore has zero gradient on the entire feasible set defined by the thresholds and cannot distinguish a point that barely meets every threshold from one that far exceeds the primary objective. This contradicts the stated task of maximizing objectives such as yield, and it means BoTier is a satisficing scalarization rather than the hierarchical maximization objective claimed in Sections 1 and 2. Furthermore, the claim in §2.2 that min(ψ_i,t_i) preserves continuity is false: at a superordinate threshold crossing, the Heaviside product switches subordinate terms on or off, producing a jump (for N=2, as ψ1 crosses t1, Ξ jumps by roughly t2). The implemented smooth H and smooth min (SI Eqs. S3–S4) remove both the zero gradient and the jump, so the benchmarked objective is qualitatively different from the exact one, and Figs. S13/S18 only vary k on analytical surfaces, not on the chemistry emulators. The paper’s success metric—evaluations needed to satisfy all thresholds—is exactly the quantity the flat objective targets, so the claim that BoTier 'never performed worse' does not test post-threshold optimization, which is where Eq. (2) gives no guidance.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces BoTier, a hierarchical scalarization composite objective for multi-objective Bayesian optimization. The scalarization in Eq. (2) is designed to reflect a user-defined hierarchy over objectives, where each objective contributes only if all superordinate objectives satisfy their thresholds; the paper claims this preserves continuity and is consistent with the existing Chimera scalarization. The authors provide an auto-differentiable implementation using smooth approximations of the Heaviside and min functions, integrate it with BoTorch, and benchmark it on analytical surfaces and emulated chemistry problems (Suzuki-Miyaura coupling, benzylation, enzymatic alkoxylation, silver nanoparticle synthesis). The central claims are that BoTier is a robust and flexible composite objective that never performs worse than existing methods and often performs notably better, especially when used as a composite objective.","tokens_in":14062,"tokens_out":4204,"duration_ms":36336,"significance":"If the claims hold, BoTier would be a practically useful contribution to multi-objective Bayesian optimization, particularly for autonomous experimentation, because it extends hierarchical scalarization to the composite-objective setting and provides an open-source, auto-differentiable implementation. The paper's strengths include the release of a PyPI/GitHub package, reproducible benchmark scripts, systematic comparisons against Chimera, penalty-based scalarization, and EHVI, and explicit acknowledgment of the role of the smoothing parameter. The benchmark suite covers both analytical and real-world emulated problems, and the paper is generally clearly written. However, the exact formulation in Eq. (2) and the actually implemented smooth version differ in properties that are load-bearing for the paper's stated claims, and the benchmark success metric is defined in terms of the very thresholds BoTier targets. These issues require careful treatment before the claims of general superiority can be accepted.","major_comments":[{"comment":"The claim that Eq. (2) preserves continuity is incorrect for the exact formulation. At a superordinate threshold crossing, e.g., as ψ1 crosses t1 in the N=2 case, the Heaviside product ∏_{j<i} H(ψ_j - t_j) switches the subordinate term min(ψ2,t2) on or off, producing a jump of magnitude approximately t2 (or min(ψ2,t2)). Additionally, on the region where all ψ_i ≥ t_i, Eq. (2) equals Σ_i t_i exactly, so the exact scalarization has zero gradient and cannot distinguish between a point that barely satisfies all thresholds and one that far exceeds the primary objective. The implementation described in SI Section 1.1 uses smooth sigmoid and soft-min approximations (SI Eqs. S3 and S4), which remove both the discontinuity and the flat saturation, but this means the benchmarked objective is not Eq. (2) as stated. The manuscript should either redefine BoTier as the smooth objective from the outset, clearly state the exact-vs-smooth distinction, or provide a rigorous argument for why the smooth version faithfully represents the intended hierarchy without introducing qualitatively different behavior.","section":"§2.2, Eq. (2)"},{"comment":"The success metric in the benchmarks is the number of experiments required to satisfy the first n objectives (Fig. 2 bottom rows; Figs. S9-S18; Figs. S20-S26). This metric is exactly what the flat region of Eq. (2) is designed to satisfy, so the paper's claim that BoTier 'never performed worse, but often performed notably better' is only established for reaching thresholds, not for continued maximization of objectives above their thresholds. Since the stated task in the introduction includes maximizing objectives such as yield, the benchmarks should also report post-threshold progress (e.g., the best yield achieved among all feasible points within the budget). Without such measurements, the claim of general superiority over EHVI and other baselines is too broad.","section":"§3, Figs. 2-3; SI §S2.2, §S3"},{"comment":"The sensitivity analyses for the smoothing parameter k (Figs. S13 and S18) are performed only on the analytical surfaces, not on the chemistry emulators. The paper's central benchmarks on real-life problems (Section 3, Fig. 3) therefore rely on an unverified assumption that the smooth approximations in SI Eqs. S3-S4 preserve the ranking and optimizer of the exact threshold objective for those emulator surfaces. The authors should either provide sensitivity results on the emulated problems or supply a formal or quantitative argument (e.g., Lipschitz-type bounds relating k to objective perturbations) that the analytical-surface results transfer. As written, the conclusion that BoTier is robust across all investigated cases is not fully supported by the presented experiments.","section":"SI §S1.1, Figs. S13 and S18"},{"comment":"The consistency check between BoTier and Chimera reports Spearman rank correlations of only 0.38 and 0.58 for the BNH and BNH* surfaces, which the text attributes to 'numerical inconsistency' near thresholds. Since these are exactly the surfaces where the paper's own benchmarks show BoTier diverging from Chimera most strongly, the dismissal is not adequately justified. The manuscript should quantify how the low rank correlation affects the optimization trajectories, or at least acknowledge that the two scalarizations can rank points very differently in threshold-dense regions, rather than presenting the overall high correlations as evidence that BoTier is equivalent to Chimera.","section":"SI §S4, Table S11"}],"minor_comments":[{"comment":"There is a typo in the first sentence: 'satisifies' should be 'satisfies'.","section":"Abstract / §1"},{"comment":"SI Eq. S4 is written for the max function, but the scalarization in Eq. (2) uses min. The text says 'analogs for the Heaviside step function and the max function'; it should either derive the soft-min explicitly or clarify that min(x1,x2) = -max(-x1,-x2) is used.","section":"SI §S1.1"},{"comment":"The sentence 'In every case, using BoTier as a composite objective further accelerated optimization, which we initially attributed to the the surrogate model not needing to re-discover known correlations' contains a duplicated 'the'.","section":"§3"},{"comment":"The figures report the 'best observed value of Ξ', but Ξ in the implementation is the smooth approximation. The captions should state which version of Ξ is plotted, since the exact and smooth versions may differ noticeably in regions near thresholds.","section":"Fig. 2 and Fig. 3"},{"comment":"The description of benchmark runs says 'Each optimization campaign starts by randomly drawing a single seed experiment'. Clarify whether the seed is drawn uniformly at random or via a fixed random seed for reproducibility, since 50 independent runs are reported.","section":"SI §S2.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a methods contribution with a clear practical motivation and a ready-to-use implementation. The main concerns are conceptual rather than experimental: the exact formulation in Eq. (2) is not the objective that is benchmarked, and the paper's continuity claim is wrong for the exact form. The authors need to reconcile the exact and smooth formulations, and to validate the smoothing parameter on the emulated problems or provide a theoretical transfer argument. The benchmark metric should be supplemented with post-threshold maximization performance to justify the 'never performed worse' claim. These are fixable within the scope of a revision; I do not see a fundamental flaw that would require rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi — quick take on the BoTier preprint.\n\nThe genuinely useful piece is the scalarization in Eq. (2) combined with a smooth, batch-friendly, auto-differentiable implementation in BoTorch. That's a real practical advance over the original Chimera, which is black-box and not auto-differentiable. The authors ship code and reproduce scripts, which is good practice, and the benchmarks across synthetic surfaces and chemistry emulators show BoTier gets to threshold-satisfying points faster than Chimera and EHVI in most cases. The composite-objective integration is clean.\n\nThe soft spot is bigger than the authors admit. The exact Eq. (2) saturates: once every psi_i is above its threshold, Xi = sum t_i exactly, independent of the individual values. So it gives zero gradient and cannot distinguish a point that barely clears thresholds from one that far exceeds the primary objective. That contradicts the paper's framing of maximizing yield, and the §2.2 claim that min preserves continuity is simply false — at a superordinate threshold crossing the Heaviside product switches subordinate terms on, so Xi jumps. The smooth H and min approximations in SI S3–S4 remove both the flatness and the jump, which means the benchmarked objective is qualitatively different from the exact one. The k-sensitivity analysis only covers analytical surfaces, not the chemistry emulators. The success metric is exactly the threshold satisfaction that the smooth objective targets, so the 'never performed worse' claim holds for satisficing, not for any form of continued improvement beyond thresholds.\n\nThat's a fixable paper, not a fatal flaw. The method is likely useful for its intended use case: hierarchical satisficing in experiment planning. But the text needs to say that explicitly, correct the continuity assertion, and ideally add post-threshold behavior or at least a discussion of why satisficing is the right goal. Significance testing beyond standard error bands would also help, and the low Spearman rank on BNH (0.38) deserves more comment than 'numerical inconsistency.'\n\nIf this lands on my desk, I'd send it to review — it's an honest, well-packaged engineering contribution with public code, and the conceptual gap is addressable. I'd ask the authors to be straight about what the objective actually does.","headline":"BoTier is a useful, well-packaged threshold scalarization for hierarchical MOO, but the exact objective saturates and is discontinuous, so the paper's framing and continuity claim need fixing.","tokens_in":14625,"tokens_out":3257,"would_cite":true,"duration_ms":29919,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"BoTier is a threshold-gated composite objective that benchmarks show reaches all objectives in fewer experiments than existing multi-objective Bayesian optimization methods.","keywords":["multi-objective optimization","Bayesian optimization","composite objective","hierarchical scalarization","threshold-based scalarization","experiment planning","reaction optimization","BoTier"],"falsifier":"Run the same BoTier Bayesian-optimization campaigns on the four emulated chemistry problems with the exact, discontinuous score and with the smooth approximation at several $k$ values, and compare the fraction of campaigns that satisfy all thresholds within the 50-evaluation budget; if the smooth version underperforms the exact version on any problem, the equivalence assumption fails.","tokens_in":13541,"feed_emoji":"🎯","tokens_out":7067,"duration_ms":62053,"temperature":0.7,"pith_summary":"The paper introduces BoTier, a composite objective that encodes a user-defined hierarchy of satisfaction thresholds over both experiment outcomes and input parameters, and claims that it speeds up Bayesian optimization in multi-objective scientific problems. The central design is the scalarization $\\Xi = \\sum_{i=1}^N \\left( \\min(\\psi_i,t_i) \\prod_{j<i} H(\\psi_j-t_j) \\right)$, which lets each objective contribute only after every higher-priority objective has cleared its threshold, and which can be evaluated separately for each candidate point from model posterior samples. On four analytical surfaces and four emulated chemistry problems, the paper reports that BoTier reaches conditions satisfying all thresholds at budgets where Chimera, a penalty-based scalarization, and Pareto-front mapping (expected hypervolume improvement) often fail, and that the composite formulation consistently matches or beats the black-box version. The authors present BoTier as an auto-differentiable, batch-evaluable alternative that removes the batch-dependence and non-differentiability of the Chimera scalarization.","feed_headline":"Tiered-objective solver reaches all targets faster than baselines","feed_subtitle":"A threshold-gated composite score lets Bayesian optimization honor objective priorities without re-learning known input costs.","key_machinery":"The central object is the BoTier scalarization $\\Xi = \\sum_{i=1}^N \\min(\\psi_i,t_i) \\prod_{j<i} H(\\psi_j-t_j)$, a sum of capped objectives gated by threshold crossings. The product of step functions $H(\\psi_j-t_j)$ ensures a lower-priority objective contributes only after all higher-priority objectives meet their thresholds; once a subordinate objective is below its threshold, $\\min(\\psi_i,t_i)$ makes that objective the active driver, and when all thresholds are met, $\\Xi$ accumulates the threshold constants and the top objective dominates. To make the score optimizable, the paper approximates $H$ and $\\min$ by smooth logistic and softmax-like functions with a user-set sharpness $k$, and evaluates the composite objective through Monte-Carlo integration over posterior samples from independent Gaussian-process surrogates. That smooth-composite construction is what carries the claim: it preserves the ranking of Chimera in the reported correlation checks while making the objective auto-differentiable and batch-evaluable.","core_discovery":"The central claim is that tiered preferences in multi-objective optimization can be captured by the threshold-gated sum $\\Xi$, and that optimizing $\\Xi$ as a composite objective is both practically usable and sample-efficient. Unlike Chimera, whose value for one point depends on the other points in the batch through running maxima over the search space, $\\Xi$ gives each candidate an independent score once thresholds are fixed. Replacing the Heaviside factors and the $\\min$ with smooth, $k$-parameterized analogs makes the score differentiable, so it can be differentiated through Monte-Carlo samples of Gaussian-process posteriors and optimized with standard expected-improvement acquisition. The paper's benchmark-supported conclusion is that, for the problems studied, BoTier never performed worse and often performed notably better than existing multi-objective methods, especially when used as a composite objective.","pith_inferences":["The paper fixes the smoothness parameter $k$ and checks sensitivity only on analytic surfaces; a natural extension is an adaptive or annealed $k$ that sharpens as the budget grows, which could make the smooth score track the exact threshold objective more faithfully near crossings.","The observation that joint multi-output surrogates rarely helped suggests that modeling each objective independently may be sufficient whenever a scalarization separates objectives; testing this on higher-dimensional objective spaces would tell whether the gain is intrinsic to composite scalarization or to the surrogate split.","The 'never performed worse' statement is tied to the paper's threshold choices; thresholds that place many Pareto-optimal points near the satisfaction boundary could compress the ranking differences, so the claim should be read as benchmark-specific.","The same gating construction could in principle be reused for hierarchical constraints or multi-fidelity settings, though the paper does not develop those directions."],"forward_implications":["If BoTier works as claimed, experiment planners can encode a known priority structure over outcomes and input costs without mapping the full Pareto front, saving experimental budget on uninteresting trade-off regions.","The composite formulation means input-dependent objectives enter the score directly, so a surrogate never has to relearn known correlations between inputs and objectives.","Because the score is auto-differentiable and built on Monte-Carlo posterior samples, it plugs into standard Bayesian optimization loops with expected-improvement acquisition.","The benchmark evidence implies that, for threshold-based hierarchies, hierarchical scalarization can find all-satisfying conditions faster than non-hierarchical penalty methods and Pareto-oriented EHVI.","Even when all objectives depend only on experiment outputs, composite use of BoTier matched or beat black-box scalarization in the studied cases."],"supporting_citations":[{"why":"Supplies the composite-objective paradigm BoTier is built on: scalarizing posterior samples rather than raw observations.","marker":"[23]"},{"why":"Introduces Chimera, the hierarchical scalarization BoTier is designed to replace while preserving ranking behavior.","marker":"[24]"},{"why":"Provides the Bayesian optimization and Monte-Carlo acquisition machinery that makes BoTier's auto-differentiable composite objective usable.","marker":"[25]"},{"why":"Defines the penalty-based threshold scalarization used as a non-hierarchical baseline.","marker":"[27]"},{"why":"Supplies the Expected Hypervolume Improvement Pareto-oriented baseline for comparisons.","marker":"[28]"},{"why":"Describes the emulation strategy used to benchmark BoTier on real chemical reactions.","marker":"[29]"},{"why":"Provides the silver nanoparticle synthesis dataset used as one emulated optimization problem.","marker":"[30]"},{"why":"Provides the continuous-flow benzylation dataset used as another emulated optimization problem.","marker":"[31]"}],"fun_headline_variants":["BoTier: Tiered objectives beat baselines in multi-objective BO","Threshold-gated sum encodes objective priorities in BoTier","Differentiable composite objective speeds up Bayesian optimization","BoTier: Prioritize outcomes and inputs without relearning costs","Hierarchy-aware optimizer handles competing objectives efficiently"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The benchmarks assume that the smooth stand-ins for the step function and the min function give the same ranking of candidate experiments as the exact, discontinuous threshold score, for every user-set threshold; the paper tests this sensitivity only on the analytic test surfaces, not on the chemistry emulators.","fun_headline_variants_meta":{"raw":{"variants":["BoTier: Tiered objectives beat baselines in multi-objective BO","Threshold-gated sum encodes objective priorities in BoTier","Differentiable composite objective speeds up Bayesian optimization","BoTier: Prioritize outcomes and inputs without relearning costs","Hierarchy-aware optimizer handles competing objectives efficiently"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000274,"raw_usage":{"total_tokens":1587,"prompt_tokens":843,"completion_tokens":744,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":459,"completion_tokens_details":{"reasoning_tokens":663}},"tokens_in":459,"tokens_out":744,"duration_ms":7158,"temperature":1.0,"reasoning_tokens":663,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:12:17.670907+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same BoTier Bayesian-optimization campaigns on the four emulated chemistry problems with the exact, discontinuous score and with the smooth approximation at several $k$ values, and compare the fraction of campaigns that satisfy all thresholds within the 50-evaluation budget; if the smooth version underperforms the exact version on any problem, the equivalence assumption fails.","supporting_citations":[{"cited_title":"Astudillo and P","cited_arxiv_id":null,"evidence_quote":"Supplies the composite-objective paradigm BoTier is built on: scalarizing posterior samples rather than raw observations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces Chimera, the hierarchical scalarization BoTier is designed to replace while preserving ranking behavior."},{"cited_title":"Balandat, B","cited_arxiv_id":null,"evidence_quote":"Provides the Bayesian optimization and Monte-Carlo acquisition machinery that makes BoTier's auto-differentiable composite objective usable."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the penalty-based threshold scalarization used as a non-hierarchical baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Expected Hypervolume Improvement Pareto-oriented baseline for comparisons."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Describes the emulation strategy used to benchmark BoTier on real chemical reactions."},{"cited_title":"Mekki-Berrada, Z","cited_arxiv_id":null,"evidence_quote":"Provides the silver nanoparticle synthesis dataset used as one emulated optimization problem."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the continuous-flow benzylation dataset used as another emulated optimization problem."}],"review_version":1}