{"id":"40529cd8-afca-473a-a3ee-dc523cf32aee","arxiv_id":"2506.03815","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Nested adaptive grid designs hit the optimal worst-case run count v^{-(p-1)} for classifying monotonic binary simulations, an order of magnitude fewer than the v^{-p} needed by static designs.","lead":"This paper proposes adaptive grid designs that test points on nested grids and skip runs whose binary outcomes are implied by monotonicity, and proves these designs need only v^{-(p-1)} runs to shrink the uncertain region to volume v, versus roughly v^{-p} for any static design. The motivation is an ice-breaking simulation where each run takes nearly 10 hours, so lowering the run-count exponent is practically important.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 3's proof uses geometric grid levels (2^l+1) while Algorithm 1 and the theorem statement use arithmetic levels (2l+1); under the arithmetic reading the displayed bound gives m ~ v^{-p}, so the claimed v^{-(p-1)} rate is unproven for the algorithm as written.","rationale":"The reader correctly identified the Theorem 3 notation inconsistency in their rationale ('unless the intended formula is 2^l+1') but did not make it the weakest assumption, choosing instead the exact-monotonicity assumption. My stress-test elevates the Theorem 3 inconsistency to the primary load-bearing concern because it directly threatens the paper's headline theoretical contribution: the v^{-(p-1)} optimal-rate claim for the grouped-adaptive grid design. If the algorithm really uses arithmetic levels (2l+1), the displayed bound gives m_GG(g)=O(g^p) and V(U)=O(1/g), implying m_GG=O(v^{-p}), no better than static designs. The paper's subsequent asymptotic text silently switches to a geometric interpretation (2^{g(p-1)}), which is the only way to obtain v^{-(p-1)}. This is not a cosmetic typo: the algorithm, theorem statement, proof, and the numerical experiments (which use D_SG,3,125, an arithmetic level) are mutually inconsistent. A reader cannot tell which design is being analyzed or recommended. The lower-bound theorems (2 and 4) and the overall framing appear sound, and the monotonicity concern raised by the reader is a legitimate application-level limitation, but it is secondary to the internal inconsistency that undermines the central claim as stated. Because the main theoretical result is unproven for the described algorithm and the text suggests the proof concerns a different construction, the submitted manuscript does not currently support its headline claim; a substantial revision is required, hence REJECT rather than CONDITIONAL.","tokens_in":30377,"tokens_out":28394,"duration_ms":263850,"concrete_test":"Implement Algorithm 1 exactly as written with arithmetic grid levels D_SG,2,9, D_SG,2,25, D_SG,2,49, D_SG,2,81 (i.e., 3x3, 5x5, 7x7, 9x9) on the monotone function f(x)=1 if x1+x2>=1, -1 otherwise. For each level g, record m_GG(g), the number of evaluations until all grid outputs are known, and V(U). If m_GG(g) grows as g^2 while V(U) decays as 1/g, then m_GG ~ V(U)^{-2}, refuting the claimed v^{-1} rate for p=2. Then repeat with geometric levels 3x3, 5x5, 9x9, 17x17; if m_GG(g) grows as 2^g and V(U) as 2^{-g}, the v^{-(p-1)} rate requires the geometric generator, confirming that Algorithm 1 must be changed to D_SG,p,(2^l+1)^p.","verdict_should_be":"REJECT","load_bearing_attack":"The central rate claim rests on Theorem 3, but the theorem's statement, its proof, and Algorithm 1 describe different grid sequences. Algorithm 1 generates candidate sets D_SG,p,(2l+1)^p, i.e., grid levels with 2l+1 points per dimension (after an initial 2^p corner grid). The displayed bound in Theorem 3 is 2^p + sum_{l=1}^g p(2l+1)^{p-1}, which for p=2 is O(g^2). At level g the grid spacing is h=1/(2g), so the worst-case uncertain volume is V(U)=O(1/g). Hence m_GG(g)=O(g^2)=O(V(U)^{-2}), i.e., O(v^{-p}) for p=2, the same order as static designs, not the claimed v^{-(p-1)}. The paper's own asymptotic interpretation immediately after Theorem 3 instead claims m_GG(g) is O(2^{g(p-1)}) and that O(v^{-(p-1)}) evaluations suffice. This geometric behavior cannot follow from the displayed arithmetic sum; it would require grid levels 2^l+1 per dimension, for which the per-level increment is about p(2^{l+1})^{p-1} and the sum is geometric. The proof in Appendix C.3 compounds the confusion by setting 'm=2l' and analyzing a refinement from m+1 to 2m+1 points per dimension, which is exactly geometric doubling, not the arithmetic progression used in the algorithm. Thus either the algorithm is wrong (it does not achieve the advertised rate) or the theorem and proof are about a different, geometric-grid method that the paper never defines. Until this is resolved, the central claim that the grouped-adaptive grid design needs only O(v^{-(p-1)}) evaluations is unsupported for the design described and implemented in Sections 2-5.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies experimental design for deterministic, binary, monotone computer simulations. It measures design quality by the worst-case volume V(U) of the region whose outcome is not logically forced by the evaluated runs, and it derives lower bounds on the number of runs required by any static design (Theorem 2) and any adaptive design (Theorem 4). It then proposes grouped-adaptive grid (GG) and fully adaptive grid (AG) designs, and proves an upper bound for GG (Theorem 3) that is intended to show that O(v^{-(p-1)}) runs suffice to guarantee V(U)≤v, matching the adaptive lower bound and improving on the static O(v^{-p}) rate. Numerical comparisons on synthetic test functions, road-crash simulation data, and an ice-breaking finite-element simulation demonstrate practical gains for the proposed adaptive grid methods.","tokens_in":30756,"tokens_out":16003,"duration_ms":174143,"significance":"The worst-case rate results are the main contribution. If the intended geometric-grid reading of Theorem 3 is confirmed, the paper provides the first matching-order lower and upper bounds for this class of problems, with a constructive design that is simple to implement and substantially cheaper than static grids. The paper also includes detailed proofs, comparisons with many existing design methods, and two real simulation applications, which strengthen its practical relevance. The main reservation is that the theoretical optimality statement applies to the grouped variant GG, while the fully adaptive variant AG, which is the method recommended for applications, has no supporting theorem; in addition, the grid-level notation in the central upper-bound theorem needs to be made unambiguous.","major_comments":[{"comment":"The grid-level notation must be disambiguated. In the manuscript as reproduced, Algorithm 1 and Theorem 3 state levels with (2l+1) points per dimension, while the proof in Appendix C.3 sets m=2^l and analyzes refinement from m+1 to 2m+1 points per dimension, and the asymptotic sentence after Theorem 3 claims m_GG(g)=O(2^{g(p-1)}). If read arithmetically, the displayed bound sum_{l=1}^g p(2l+1)^{p-1} is O(g^p), so with grid spacing O(1/g) the resulting number of runs is O(v^{-p}), not the claimed O(v^{-(p-1)}); the theorem would then not support the paper's central claim. The surrounding text and proof indicate that the intended construction is geometric, with 2^l+1 points per dimension, in which case the advertised rate follows. Because this is the load-bearing upper bound, the published version must use unambiguous notation throughout and align Algorithm 1, the group definitions, Theorem 3, and the proof; I recommend explicitly writing 2^l+1 rather than 2l+1 and stating the initial level l=0 corresponds to the 2^p corner grid.","section":"Section 2.3, Theorem 3, Algorithm 1, Appendix C.3"},{"comment":"The theoretical optimality result is proved only for the grouped-adaptive grid design GG, not for the fully adaptive grid design AG. Section 2.3 explicitly says that no rigorous theory is established for AG, yet the abstract and conclusions use the phrase 'adaptive grid designs' without qualification when claiming a v^{-(p-1)} evaluation rate that matches the best possible adaptive design. Since AG is the method recommended in the numerical sections and the applications, the paper should either scope the optimality claim to GG or clearly state that the optimality statement is conjectural for AG, so that readers do not attribute a proven rate to the algorithm actually used in practice.","section":"Abstract, Section 2.3, Section 6"}],"minor_comments":[{"comment":"The displayed limit in Theorem 5 is garbled; it should be lim_{n→∞} E[V{U(D_MC,p,n, f_tilde)}]/n^{-1/p}, not the expression with an extra factor of n in the numerator.","section":"Appendix C.5, Theorem 5"},{"comment":"When l=0, the candidate set is D_SG,p,2^p, the 2^p corner grid. This should be stated explicitly, because a literal reading of '(2l+1)^p' with l=0 would give a degenerate one-point grid that is not covered by the definition of D_SG,p,n.","section":"Algorithm 1"},{"comment":"The support vector classifier with a Gaussian kernel is not guaranteed to produce monotone predictions. Since the theoretical bounds concern V(U), not classifier accuracy, the paper should clarify that V(U) is the design-efficiency criterion and that the downstream classifier is not assumed to respect monotonicity.","section":"Sections 3-5"},{"comment":"The ice-breaking application assumes exact monotonicity of the finite-element simulator from domain knowledge. A brief caveat that small numerical non-monotonicities could invalidate the skipping step and hence the V(U) interpretation would be useful, since the paper later says it plans to study violations of the monotonicity assumption.","section":"Section 5"}],"recommendation":"major_revision","confidential_remarks":"The arithmetic-versus-geometric grid issue in Theorem 3 may be partly an artifact of flattened superscripts in the version I reviewed, but it is exactly the kind of ambiguity that can invalidate the central claim if it survives into the published PDF. I recommend asking the authors to verify that Algorithm 1, Theorem 3, and the proof all consistently display 2^l+1, and to add a short remark distinguishing the proven optimality of GG from the empirical status of AG. The paper is otherwise a solid contribution to a practically important design problem."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe useful part of arXiv:2506.03815 is the pair of worst-case lower bounds: Theorem 2 says any static design needs Ω(v^{-p}) runs, and Theorem 4 says any adaptive design needs Ω(v^{-(p-1)}) runs to push worst-case uncertain volume under v. Those arguments look coherent, they are genuinely new for this problem class, and they make the paper worth a serious referee's time.\n\nThe soft spot is load-bearing. The advertised upper bound, Theorem 3, does not match the algorithm it analyzes. Algorithm 1 uses grids with 2l+1 points per dimension (an arithmetic progression), and the displayed bound m_GG(g) ≤ 2^p + Σ p(2l+1)^{p-1} is polynomial in g. For p=2 that gives m=O(g^2), and with spacing 1/(2g) the worst-case uncertain volume is O(1/g), so m=O(v^{-2}) — same order as static designs, not the v^{-1} claimed. The text right after Theorem 3 instead says m is O(2^{g(p-1)}), and the proof in C.3 sets m=2l and refines from m+1 to 2m+1 points per dimension, which is geometric doubling. So either the algorithm is wrong or the theorem and proof describe a different, geometric-grid method that the paper never defines. The central claim that the adaptive grid design matches the adaptive lower bound is unsupported as printed.\n\nOther soft spots are minor by comparison: the fully adaptive AG variant is recommended on numerics without a proof, and no code or data are shipped. The numerical studies and the ice-breaking application are real work, but they do not fill a gap in the theory.\n\nThis is fixable. The lower bounds stand on their own even if the upper bound is temporarily withdrawn. The authors need to pick one grid sequence — geometric (2^l+1) is the one their proof actually supports — restate Theorem 3 accordingly, rerun the experiments for that algorithm, or else soften the abstract's rate claim. Until that happens, the paper's headline result should not be cited as established.\n\nI'd send it to review; the referee needs to push the authors on the Theorem 3/Algorithm 1 mismatch and check the revised proof. It is not a desk-reject, but it is not acceptable in current form.","headline":"Useful worst-case lower bounds for static and adaptive designs, but the paper's advertised upper bound for the adaptive grid design does not match its own algorithm; the central rate claim is unsupported as printed.","tokens_in":31275,"tokens_out":4146,"would_cite":true,"duration_ms":40892,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62K05","62L05"],"pacs":[],"model":"deepseek-v4-flash","headline":"Nested adaptive grids that skip logically forced runs trace a monotone simulator's success–failure boundary at the best possible adaptive rate — about $v^{-(p-1)}$ runs in $p \\ge 2$ dimensions, an order of magnitude better than any static…","keywords":["adaptive design","computer experiments","monotonic binary simulation","deterministic simulation","uncertainty quantification","space-filling design","minimax lower bound","ice-breaking simulation"],"falsifier":"Run the ice-breaking finite-element model at pairs of inputs ordered by the paper's reparameterization (higher velocity, lower thickness, lower modulus all push the output up); the first pair with a 'break' at the lower input and a 'no break' at the higher input refutes the monotonicity premise on which the skipping logic and the $V(U)$ guarantee rest. Separately, run the fully adaptive grid method on a known monotone test function with boundary $\\sum_k x_k = p/2$ for $p = 2$ and $p = 3$, record the evaluations needed to reach $V(U) \\le v$ for $v = 10^{-1}, 10^{-2}, 10^{-3}$, and check that the count grows as $v^{-(p-1)}$ rather than $v^{-p}$.","tokens_in":30130,"feed_emoji":"🧊","tokens_out":27408,"duration_ms":268430,"temperature":0.7,"pith_summary":"The paper asks how many runs of an expensive deterministic simulator are needed to find the boundary between \"success\" and \"failure\" outputs when the response is binary and monotone in the inputs. It proves that the answer depends on whether runs can be chosen adaptively: any static design needs on the order of $v^{-p}$ runs in $p \\ge 2$ dimensions to bring the uncertain volume below $v$, any adaptive design needs at least on the order of $v^{-(p-1)}$ runs, and the proposed adaptive grid designs attain the $v^{-(p-1)}$ rate. The net claim is that adaptivity buys a full power of $1/v$ — an order of magnitude in the worst-case exponent — and that no adaptive procedure of any kind can do better. The benefit matters because the motivating applications cost hours per run: the paper's adaptive scheme reproduces the information of a 125-run grid with 29 runs in the ice-breaking simulator, and it outperforms standard static and active-learning designs on test functions, a road-crash simulation, and the ice-breaking problem.","feed_headline":"Adaptive grids use an order of magnitude fewer simulator runs","feed_subtitle":"For ice-breaking and crash simulators, the saving can turn months of supercomputing into days.","key_machinery":"The engine of the method is logical run-skipping under total monotonicity. After a point $x$ is evaluated, the outcome is forced at every point in the lower orthant $[0, x]$ if $f(x) = -1$ and in the upper orthant $[x, 1]$ if $f(x) = 1$, so the uncertain region $U(D, f)$ is the complement of these covered orthants and its volume $V(U)$ is the paper's measure of classification error. The adaptive grid design is a named class of designs: sequences of nested full grids with $(2^l + 1)^p$ points per grid ($3^p, 5^p, 9^p, 17^p, \\ldots$), run from coarse to fine, in which only points that fall inside the current uncertain region get evaluated. The grouped variant (GG, Algorithm 1) processes one whole resolution before updating $U$ and carries the proved rate; the fully adaptive variant (AG, Algorithm 2) updates $U$ after every run and chooses next the point that maximizes the number of remaining candidates it will render skippable, $\\min\\{\\mathrm{card}(A_x), \\mathrm{card}(B_x)\\}$ — numerically stronger, without a matching theorem. The lower bounds are proved by an adversarial monotone threshold function on $\\sum_k x_k$ that forces any design with few points to leave thin uncovered slices of the input box, which makes the static and adaptive exponents $n^{-1/p}$ and $n^{-1/(p-1)}$ unavoidable.","core_discovery":"The central claim is that the worst-case cost of classifying a monotone binary deterministic simulator is governed by the dimension of the boundary, not of the input space, provided runs may be chosen adaptively. For $p \\ge 2$, the volume of the uncertain region $V(U)$ after $n$ runs is at most a constant times $n^{-1/(p-1)}$ for the proposed adaptive grid design (Theorem 3), while no adaptive design can force it below a constant times $n^{-1/(p-1)}$ (Theorem 4) and no static design can beat a constant times $n^{-1/p}$ (Theorems 1 and 2). In terms of runs needed to guarantee $V(U) \\le v$, that is order $v^{-(p-1)}$ for the adaptive grid, order $v^{-(p-1)}$ for the best possible adaptive design, and order $v^{-p}$ for the best possible static design — so the adaptive grid is worst-case optimal up to constants among all adaptive procedures and better than any static design by a full power of $1/v$; for $p = 1$ the comparison is $\\log(1/v)$ versus $1/v$. The paper's grouped-adaptive grid design carries this rate guarantee; its fully adaptive variant is shown numerically to be even better, and the authors state that they could not prove a matching theorem for it. Applications to a road-crash simulator and an ice-breaking finite-element model, where one run costs about ten hours on 120 cores, confirm the practical gain.","pith_inferences":["Editorial inference: the $v^{-(p-1)}$ rate reads as the cost of locating a $(p-1)$-dimensional boundary to accuracy $v$, whereas static designs pay to resolve the whole $p$-dimensional box; if that geometric reading is right, skipping schemes built on monotone covering should be near-optimal wherever the target is a low-codimension manifold, not only for monotone binary simulators.","Editorial inference: the same orthant-skipping logic transfers, via the paper's own reparameterizations, to simulators that are monotone decreasing in some coordinates or defined on rectangular domains, and to shape-constrained classification problems beyond deterministic simulation.","Editorial inference: the paper's numerical comparisons show the grid's edge fading by $p = 6$, where the entropy-based active learning baseline overtakes it, so a hybrid that applies adaptive grids in a few coordinates and entropy acquisition in the rest is a natural, testable next step that the paper does not propose.","Editorial inference: the lower bounds are driven by adversarial step-shaped boundaries, so on smooth monotone simulators the realized saving over static designs could be larger than the worst-case comparison; a direct measurement on smooth simulators would quantify how much the minimax guarantee understates the practical gain."],"forward_implications":["In $p \\ge 2$ dimensions no static design — grid, Latin hypercube, Monte Carlo, or space-filling — can achieve better than order $v^{-p}$ runs, so the $n^{-1/p}$ static rate is a genuine floor (Theorems 1 and 2).","No adaptive design of any kind can beat order $v^{-(p-1)}$ runs, and the adaptive grid attains it, so the proposed method is minimax-rate optimal up to constants among all adaptive strategies (Theorems 3 and 4).","On the ice-breaking simulator, 29 adaptive grid runs reproduce the uncertainty level of the 125-run static grid — about 12 days of simulation instead of close to two months — and the road-crash results show the same pattern across all 40 reconstructed crash occasions.","For $p = 1$ the adaptive advantage is exponential: roughly $\\log(1/v)$ runs versus order $1/v$ runs for any static design.","Because all runs in one grid resolution can be evaluated in parallel, the grouped variants (GG, GI) offer a parallel-friendly route, whereas the fully adaptive variants (AG, AI) are sequential but empirically stronger."],"supporting_citations":[{"why":"supplies the adaptive Monte Carlo baseline (AMC) from work on monotonic reliability methods, which the efficiency analysis compares with the proposed designs.","marker":"[24]"},{"why":"support vector classification, the classifier that converts designed runs into predictions in every numerical comparison.","marker":"[16]"},{"why":"provides the road-crash data set (40 reconstructed rear-end crashes, 1005 input combinations) used as the ordinal-input application.","marker":"[4]"},{"why":"provides the ice-breaking water-exiting vehicle simulation that motivates the study and anchors the final application.","marker":"[2]"},{"why":"entropy-based active learning (ALE), the strongest competitor in the numerical comparisons for 2 to 6 dimensions.","marker":"[26]"},{"why":"Latin hypercube design, the canonical space-filling design whose univariate projections and ordered pairs are compared with grid designs.","marker":"[18]"},{"why":"partitioned active learning with Gaussian process models (PALC), an adaptive baseline in the comparisons.","marker":"[23]"},{"why":"minimum energy designs, a sequential sampling baseline compared with the proposed methods.","marker":"[25]"},{"why":"optimal sliced Latin hypercube designs, used to seed the initial designs of the active-learning baselines.","marker":"[19]"}],"fun_headline_variants":["Adaptive grids cut simulator runs by an order of magnitude","Adaptive grid provably optimal for monotone sims","Adaptive grid beats static designs by full power","Adaptive grids slash simulator cost months to days","Adaptive grid finds critical edge in monotone sims"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the simulator's output is exactly monotone — non-decreasing in every input coordinate after reparameterization — so that an unevaluated point's outcome is logically forced by comparisons with evaluated points; in the ice-breaking application this is assumed from domain knowledge, and any small numerical non-monotonicity would let a skipped run be misclassified and would void the $V(U)$ guarantee.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive grids cut simulator runs by an order of magnitude","Adaptive grid provably optimal for monotone sims","Adaptive grid beats static designs by full power","Adaptive grids slash simulator cost months to days","Adaptive grid finds critical edge in monotone sims"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000804,"raw_usage":{"total_tokens":3583,"prompt_tokens":1049,"completion_tokens":2534,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":665,"completion_tokens_details":{"reasoning_tokens":2457}},"tokens_in":665,"tokens_out":2534,"duration_ms":21487,"temperature":1.0,"reasoning_tokens":2457,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:55:25.273136+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the ice-breaking finite-element model at pairs of inputs ordered by the paper's reparameterization (higher velocity, lower thickness, lower modulus all push the output up); the first pair with a 'break' at the lower input and a 'no break' at the higher input refutes the monotonicity premise on which the skipping logic and the $V(U)$ guarantee rest. Separately, run the fully adaptive grid method on a known monotone test function with boundary $\\sum_k x_k = p/2$ for $p = 2$ and $p = 3$, record the evaluations needed to reach $V(U) \\le v$ for $v = 10^{-1}, 10^{-2}, 10^{-3}$, and check that the count grows as $v^{-(p-1)}$ rather than $v^{-p}$.","supporting_citations":[{"cited_title":"de Rocquigny","cited_arxiv_id":null,"evidence_quote":"supplies the adaptive Monte Carlo baseline (AMC) from work on monotonic reliability methods, which the efficiency analysis compares with the proposed designs."},{"cited_title":"Active sampling: A machine-learning-assisted framework for finite population inference with optimal subsamples.Technometrics, 0(0):1–12, 2024","cited_arxiv_id":null,"evidence_quote":"provides the road-crash data set (40 reconstructed rear-end crashes, 1005 input combinations) used as the ordinal-input application."},{"cited_title":"A numerical simulation method for ice-breaking and cavitation effects on the water-exiting vehicle.Ocean Engineering, 314:119659, 2024","cited_arxiv_id":null,"evidence_quote":"provides the ice-breaking water-exiting vehicle simulation that motivates the study and anchors the final application."},{"cited_title":"Lewis and William A","cited_arxiv_id":null,"evidence_quote":"entropy-based active learning (ALE), the strongest competitor in the numerical comparisons for 2 to 6 dimensions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Latin hypercube design, the canonical space-filling design whose univariate projections and ordered pairs are compared with grid designs."},{"cited_title":"Partitioned active learning for heterogeneous systems.Journal of Computing and Information Science in Engineering, 23(4):041009, 01 2023","cited_arxiv_id":null,"evidence_quote":"partitioned active learning with Gaussian process models (PALC), an adaptive baseline in the comparisons."},{"cited_title":"Roshan Joseph, Tirthankar Dasgupta, Rui Tuo, and C","cited_arxiv_id":null,"evidence_quote":"minimum energy designs, a sequential sampling baseline compared with the proposed methods."},{"cited_title":"Myers, and William A","cited_arxiv_id":null,"evidence_quote":"optimal sliced Latin hypercube designs, used to seed the initial designs of the active-learning baselines."}],"review_version":1}