{"id":"876d14ff-9eea-4563-a940-928db3d9f901","arxiv_id":"2509.10149","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Uniform sampling inside the highest density region of the input distribution often improves surrogate-based failure probability estimates in low dimensions, but it can worsen global mean-square accuracy.","lead":"Researchers propose training surrogate models on points spread uniformly inside the high-probability region of the input distribution rather than on random draws from that distribution. The goal is better approximations in the tails, which matter for rare-event and reliability calculations, at the cost of some accuracy near the mode.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Abstract's 'globally outperform' claim is contradicted by the paper's own RMSE results: HDR loses on Franke and loses for every α when d>3, and the RRIE ranking excludes noise-level runs.","rationale":"The reader's weakest assumption was LAR comparability; I see that as a real confound but secondary. The most load-bearing issue is that the paper's own headline claim is internally contradicted by its reported figures. A claim that is contradicted by the paper's own data cannot be accepted as stated, regardless of basis-selection details. The LAR confound could explain differential RLOO, but RMSE on a fixed validation set with adaptive LAR is still an end-to-end measure; the d-dimensional study is controlled and already shows natural winning for d>3. The appendix derivations (A and B) are correct and the implementation is validated on Gaussian HDRs, so the contribution has independent support; the problem is calibrating the scope of the claim. The reader's CONDITIONAL verdict is therefore appropriate, but the condition should be explicit: either weaken 'globally outperform' to 'competitive in low dimension and useful for reliability,' or provide the stratified re-analysis above. I would keep the verdict CONDITIONAL/UNCHANGED; no need to reject the paper.","tokens_in":31815,"tokens_out":5840,"duration_ms":70295,"concrete_test":"Obtain the per-replication data (or rerun the benchmark) and compute, for RMSE and RRIE, the median and win counts stratified by dimension (d≤3 vs d>3), with no ε̄_RRIE disqualification, and with LAR fixed to the same polynomial degree/index set for both sampling schemes. If HDR's aggregate RMSE advantage disappears or reverses in d>3 and the RRIE ranking reverses without disqualification, the abstract must be restricted to 'low-dimensional reliability problems' rather than 'global outperform.'","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that HDR designs 'globally outperform' natural designs in RMSE and failure-probability estimation is not supported by the paper's own figures. First, on the Franke problem (§4.1, Fig. 10) HDR worsens RMSE by roughly a factor 2 for both PCE (1.56×10⁻¹→3.12×10⁻¹) and PCK (3.2×10⁻⁴→6.5×10⁻⁴); the text explains that the validation set drawn from the natural distribution penalises HDR, but this explanation applies to the whole benchmark, not just Franke. Second, in the controlled d-dimensional study (§4.3, Fig. 14b) the paper states: 'natural sampling outperforms HDR for any tested value of α as soon as d>3.' Since the benchmark includes problems of dimension 4,5,6,7,9 and 10, 'global' performance requires either d≤3 or a metric-specific reading that the abstract does not provide. Third, the RRIE ranking (§4.2, Fig. 13c) removes all runs with ε_RRIE below the noise threshold ε̄_RRIE before ranking; this disqualifies the best-performing runs, so the RRIE win counts do not measure 'outperformance' over the full distribution of replications. Taken together, the headline claim overstates the empirical record; the evidence supports a qualified claim (HDR competitive or better in low dimension, especially for reliability), not the unqualified 'globally outperform' statement.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a heuristic experimental-design strategy for surrogate modelling: instead of sampling training points according to the input distribution ('natural sampling'), it generates points uniformly inside the highest density region (HDR) of the input density at probability level 1−α. The HDR is computed numerically via a reliability-style root-finding method, and uniform samples are obtained by affine mapping of the unit cube to a PCA-based bounding box followed by acceptance–rejection. The method is tested on PCE and PCK surrogates for nine problems, with 100 repetitions per setting (about 1,400 surrogates), using three metrics: relative leave-one-out error, relative mean-square error on an external validation set, and a relative reliability-index error based on importance sampling. The paper claims that the HDR-based designs 'globally outperform' natural designs in both RMSE and failure-probability estimation. The appendices provide correct closed-form results for Gaussian HDRs and for a synthetic d-dimensional test function with exactly prescribed failure probability.","tokens_in":32162,"tokens_out":4530,"duration_ms":54811,"significance":"If the headline claim were supported, the method would offer a cheap, black-box way to improve surrogate-based reliability estimates. The paper has real strengths: a large and carefully replicated benchmark, inclusion of a full-scale finite-element geotechnical problem, transparent reporting of the heuristic nature of the method, and machine-checkable appendix proofs for the Gaussian HDR formula and the d-dimensional test function. However, the central empirical claim is not supported as stated: the paper's own results show HDR is worse on the Franke problem for RMSE, and worse than natural sampling for every tested α once d>3. The RRIE ranking also excludes the most accurate runs. The method may still be useful as a heuristic for low-dimensional reliability problems, but the paper needs substantial qualification and additional analysis before the 'global outperformance' claim can stand.","major_comments":[{"comment":"The abstract states that HDR designs 'globally outperform' natural designs 'both in terms of relative mean square error as well as in estimating the probability of failure.' This is contradicted by the paper's own evidence. In §4.1, Fig. 10, HDR roughly doubles RMSE on the Franke problem for both PCE (1.56×10⁻¹ → 3.12×10⁻¹) and PCK (3.2×10⁻⁴ → 6.5×10⁻⁴). In §4.3, Fig. 14b, the text explicitly says 'natural sampling outperforms HDR for any tested value of α as soon as d>3.' Because the benchmark includes d=4,5,6,7,9,10, the 'global' claim cannot be maintained. The abstract and conclusions should be revised to a qualified statement, e.g. 'often competitive or better in low dimension, especially for reliability metrics,' or 'with the exception of RMSE in higher dimensions.'","section":"Abstract and §4.1, §4.3"},{"comment":"The RRIE ranking used to support the failure-probability claim excludes all runs with ε_RRIE below the noise threshold ε̄_RRIE. These are the best-performing runs—those where the surrogate is indistinguishable from the reference at the Monte Carlo noise level. The displayed win counts (33 vs 25) therefore do not compare the two methods over the full distribution of replications; they compare only the noisier runs. The authors note that disqualification 'mechanically narrows the separation,' but this does not cure the selection bias. The paper should report the proportion of sub-threshold runs for each method (the right-axis numbers in Fig. 13c are aggregated) or treat sub-threshold runs as ties, and state whether HDR's advantage survives that analysis.","section":"§4.2, Fig. 13c"},{"comment":"The paper itself acknowledges a load-bearing comparability problem: 'the LARS algorithm might converge to different multi-index sets, resulting in a different selection of elements of the polynomial bases. In this context, the question of whether the results are truly comparable is legitimate.' If HDR sampling systematically produces sparser or simpler bases, then lower LOO and RMSE could reflect model complexity rather than sampling quality. The benchmark is presented as a user-perspective comparison, which is legitimate, but the causal claim that HDR sampling 'outperforms' requires isolating the sampling effect. The authors should report the distribution of selected basis sizes or run a fixed-basis comparison for at least a subset of problems.","section":"§4.2, LARS/comparability"},{"comment":"For problem F6 (sheet pile wall), the reference reliability index is not an external value: it is obtained from a PCK-based active-learning reliability scheme, as stated in §3.2.4. The table also marks the problem as not reproducible. The RRIE metric for this problem therefore compares the surrogate against another surrogate-based estimate, not against a reference 'truth.' This introduces a circularity risk for one of the seven benchmark problems in the pooled ranking. The paper should either flag this explicitly as a limitation in the RRIE analysis, or provide an independent Monte Carlo validation for F6 at a coarser tolerance.","section":"§3.2.4 and Table 1"}],"minor_comments":[{"comment":"The initial sample-size formula appears to be the reciprocal of the volume of the unit-diameter d-ball. The volume is π^{d/2}/(2^d Γ(d/2+1)), so the expected number of samples to obtain n accepted points is n·2^d Γ(d/2+1)/π^{d/2}. The formula as written, Γ(d/2+1)/(π^{d/2}2^d)·n, is much smaller; for d=2 it gives n/(4π)≈0.08n instead of about 1.27n. This is not fatal because the loop increments, but it makes the initial guess misleading.","section":"Algorithm 2, line 4"},{"comment":"The text says the vertical ranking 'allows us to identify under which conditions HDR performs best,' but the displayed bars aggregate over problems and training sizes. A breakdown by problem and n would better support the 'conditions' language.","section":"§4.2, Fig. 13"},{"comment":"The sensitivity analysis reports a maximum relative error of 6% for the HDR level ℓ. This is fine for the method, but the paper should state explicitly that this uncertainty propagates into the boundary of the sampling region and therefore into all subsequent comparisons; a sentence in the limitations would be helpful.","section":"§3.1.1"},{"comment":"The notation ̃ẑ or ̃g is used inconsistently for surrogate predictions; for example Eq. (14) writes ̃g with a hat on i, while the text elsewhere uses ̃g(𝒙). Please standardize.","section":"Eq. (14) and Eq. (15)"}],"recommendation":"major_revision","confidential_remarks":"The paper is honest about its heuristic nature and the benchmark is extensive, but the central 'globally outperform' claim is not supported by the reported figures. The comparability issue with LARS basis selection and the RRIE exclusion of sub-threshold runs are additional load-bearing concerns that need either additional analyses or substantial qualification. The theoretical contributions (Gaussian HDR formula and d-dimensional test function) are correct and useful, so the manuscript is salvageable within its own scope, but not in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the HDR-sampling idea is a genuine new combination, and the benchmark is substantial, but the abstract overstates what the evidence supports. The paper says HDR designs \"globally outperform\" natural sampling on RMSE and failure-probability error. Its own figures show natural sampling wins RMSE when d>3, and on the Franke problem HDR roughly doubles RMSE. That is not a global win; it is a qualified win at low dimensions and for probability-of-failure estimation.\n\nWhat is actually new: sampling uniformly inside the HDR of the input distribution to train PCE/PCK surrogates, with a PCA bounding box acceptance/rejection sampler. The appendices with the Gaussian HDR proof and the d-dimensional test function are correct and useful. The benchmark is extensive \\u2014 100 repetitions across 9 problems, 1400 surrogates \\u2014 and the authors are transparent about implementation details and limitations, including the curse of dimensionality and the heuristic choice of alpha.\n\nThe soft spots are proportionate to the claims. The \"globally outperform\" line is the main problem. Beyond the d>3 RMSE loss, the RRIE ranking disqualified the low-error runs (those below the noise threshold) before ranking, which mechanically masks what would likely be the strongest HDR results; the win counts then only cover a subset of replications. The paper itself raises the question of whether LAR-based basis selection makes the two sampling schemes truly comparable, which is a legitimate methodological worry for the LOO and RMSE comparisons. Also, the reference reliability index for the sheet pile wall (F6) comes from an active-learning surrogate, so that comparison is partly self-referential. None of this kills the paper, but together it means the abstract needs to say \"competitive in low dimensions, often better for reliability\" instead of \"globally outperform.\"\n\nWho the paper is for: practitioners doing surrogate-based reliability analysis in low dimensions (d<=20, moderately correlated inputs). It is a useful heuristic with supporting implementation and a clear benchmark. It deserves a serious peer review; the right outcome is a revision with more careful claims.","headline":"A useful heuristic with an overbroad headline: the abstract's 'globally outperform' is contradicted by the paper's own RMSE results, but the low-dimensional reliability gains and the substantial benchmark make this worth reviewing after the claims are tempered.","tokens_in":32653,"tokens_out":1805,"would_cite":true,"duration_ms":20539,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62N05"],"pacs":[],"model":"deepseek-v4-flash","headline":"Training surrogate models on uniformly drawn samples inside the input distribution's highest density region improves approximation and failure-probability estimates compared with sampling according to the distribution itself.","keywords":["highest density region","surrogate modelling","uniform design","reliability analysis","importance sampling","polynomial chaos expansion","polynomial chaos Kriging","rare event estimation"],"falsifier":"Repeat the eight-problem benchmark with a fixed polynomial basis for both sampling schemes. If the HDR advantage in leave-one-out error, RMSE, or reliability-index error shrinks or vanishes, the reported gains come from different LAR-selected bases, not from the sampling design.","tokens_in":31657,"feed_emoji":"🎯","tokens_out":7890,"duration_ms":77714,"temperature":0.7,"pith_summary":"The paper asks how to choose training points for a surrogate model when the downstream goal is uncertainty propagation and rare-event estimation. It proposes a heuristic: instead of sampling according to the input distribution, which clusters points near the mode, draw points uniformly inside the highest density region (HDR) of that distribution. Across nine benchmark problems and 100 repetitions per configuration, HDR-based designs usually beat natural sampling on leave-one-out error, relative mean-square error, and the error in the estimated reliability index. The gain is largest for polynomial chaos Kriging, and the underlying reason is that uniform HDR sampling pushes training information into the tails, where failure probabilities are decided.","feed_headline":"Uniform sampling in high-density regions beats natural sampling","feed_subtitle":"Evenly spread points inside the density core improve failure-probability estimates for surrogates.","key_machinery":"The central object is the highest density region (HDR), a compact superlevel set of the input density that contains a chosen probability mass 1−α. The machinery has two parts: a numerical estimate of the threshold ℓ (a closed chi-squared form for Gaussian vectors, otherwise a Monte-Carlo-based root find) and a uniform sampler inside the HDR built from an affine map to a PCA-based bounding box plus acceptance–rejection. The HDR does the conceptual work: uniform sampling inside it redistributes training effort from the mode into the tails, exactly where reliability estimates are decided.","core_discovery":"The central claim is that the experimental design can be chosen to serve the downstream task. For a random vector X with density f_X, the authors define the (1−α)-HDR as the superlevel set {x: f_X(x) ≥ ℓ} that carries probability 1−α, and sample uniformly inside it using an affine PCA-based bounding box followed by acceptance–rejection. They train polynomial chaos expansion (PCE) and polynomial chaos Kriging (PCK) surrogates on these points and compare them with surrogates trained on natural samples, holding other settings fixed. The reported result is that HDR designs globally outperform natural designs on all three error metrics, with especially pronounced improvements in the relative reli","pith_inferences":["A fixed-basis benchmark (no LAR selection) would separate sampling quality from basis-selection effects; the paper does not run this, but it would settle whether HDR's advantage is partly an automatic sparsification effect.","Because the validation set is drawn from the original distribution, the RMSE comparison penalises HDR by construction; a density-weighted or tail-focused error metric would likely show a larger global gain.","Starting from an HDR design could accelerate active-learning schemes for rare events; the paper's high-dimensional toy with Pf=10^-6 hints at this but leaves it untested.","For multimodal distributions, the HDR may be a union of components and the single PCA bounding box becomes inefficient; component-wise sampling or a transport-based sampler would be the natural extension."],"forward_implications":["HDR-based experimental designs are generated from the input distribution alone, so they can be inserted into existing PCE and PCK workflows with no extra model evaluations.","In the pooled ranking over eight benchmark problems and five experimental-design sizes, HDR sampling takes more top ranks than natural sampling on all three error metrics, and no case showed natural sampling significantly outperforming HDR.","Failure-probability estimates improve: HDR-trained surrogates give lower relative reliability-index errors, which matters when Pf is as small as 10^-6.","As α increases, the training region shrinks and the leave-one-out error becomes optimistic; the paper recommends small α and cautious interpretation of RLOO under HDR sampling.","The acceptance–rejection sampler is subject to the curse of dimensionality, so the method is recommended for low-dimensional and moderately correlated inputs (tested up to d=20)."],"fun_headline_variants":["Highest-density sampling boosts failure-probability accuracy","Uniform core sampling outshines natural draws for rare events","Sample in high-density regions to sharpen reliability estimates","Dense-region sampling improves surrogate failure estimates"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The comparison assumes that error differences reflect sampling quality rather than the fact that LAR basis selection may converge to different polynomial bases under each sampling scheme—a caveat the paper itself raises in Section 4.2.","fun_headline_variants_meta":{"raw":{"variants":["Highest-density sampling boosts failure-probability accuracy","Uniform core sampling outshines natural draws for rare events","Sample in high-density regions to sharpen reliability estimates","Dense-region sampling improves surrogate failure estimates"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00035,"raw_usage":{"total_tokens":1742,"prompt_tokens":730,"completion_tokens":1012,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":474,"completion_tokens_details":{"reasoning_tokens":963}},"tokens_in":474,"tokens_out":1012,"duration_ms":8753,"temperature":1.0,"reasoning_tokens":963,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T18:06:08.929791+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Repeat the eight-problem benchmark with a fixed polynomial basis for both sampling schemes. If the HDR advantage in leave-one-out error, RMSE, or reliability-index error shrinks or vanishes, the reported gains come from different LAR-selected bases, not from the sampling design.","supporting_citations":[],"review_version":1}