{"id":"3700913a-06f1-493d-90e6-cdad46ba6b0f","arxiv_id":"2607.27427","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"low","formal_verification":"none","parameter_count":5,"one_line_summary":"Interleaving greedy trace-residual landmark selection with hyperparameter updates yields Nyström GPs that match exact GP accuracy far more stably than random landmarks on standard test functions.","lead":"The paper introduces a greedy Nyström landmark scheme for Gaussian process regression that picks points by residual kernel error and re-optimizes hyperparameters after each addition. It offers a practical way to keep GP accuracy while cutting the usual cubic cost on moderate computer-experiment data.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Missing ablation of interleaving vs static greedy/pivoted-Cholesky leaves the distinctive claim under-supported on small smooth benchmarks.","rationale":"The reader correctly flags that defaults, discrete landmarks, and n≤1000 smooth LHD settings are the soft underbelly of the “near-exact + linear scaling” claim, and already lists static pivoted Cholesky among missing baselines. The single most load-bearing gap for the paper’s distinctive contribution is sharper: interleaving is asserted as the key novelty (§3.5) but never isolated. A static greedy/pivoted-Cholesky + one-shot fit is the minimal control that would show whether the sequential loop in Algorithm 1 is doing real work. That, plus the weak random-only baseline and the one reversal in Table 3, keeps the verdict CONDITIONAL rather than ACCEPT; it does not rise to REJECT because the math (Woodbury, residual criterion, complexity) is standard and correct and the empirical direction vs random is mostly favorable. No change to CONDITIONAL; confidence in that call remains high. Code release and the static ablation (or larger-n + stronger sparse-GP baselines) would be the natural conditions.","tokens_in":12387,"tokens_out":593,"duration_ms":33901,"concrete_test":"On the same Borehole/Piston/Steel Column LHD splits, run static pivoted-Cholesky (or one-shot greedy residual) to the adaptive method’s final m, then optimize (2) once; if S-RMSPE and seed-SD match adaptive within sampling noise, interleaving is not necessary for the accuracy claim and the strongest claim should be narrowed.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The strongest claim (near-exact GP accuracy at linear cost, significantly beating random) rests on Algorithm 1’s interleaving of trace-residual landmark growth with repeated hyperparameter fits (§3.3–3.5). §3.5 correctly notes that the selection rule (3) is equivalent to pivoted Cholesky on K, and positions interleaving as the primary novelty. Yet §4 only compares against uniform random Nyström of matched final m (and full GP). There is no static baseline that (i) builds the same greedy/pivoted-Cholesky landmark set under fixed or once-estimated θ and (ii) optimizes the profiled Nyström likelihood (2) a single time. Without that control, observed gains on the five smooth VLSE LHD problems (n≤1000) cannot be attributed to interleaving rather than greedy coverage alone. Defaults δθ=10^{-3}, δr=10^{-4} and discrete landmarks⊂X further limit the claim’s reach; Table 3 already shows random beating adaptive on Wing Weight (polynomial). Linear-scaling advantage is also unshown where O(n^3) is truly infeasible.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes an adaptive Nyström approximation for Gaussian process regression in which landmark points are chosen greedily to maximize the kernel trace residual, and landmark expansion is interleaved with re-optimization of the profiled Nyström likelihood (via Woodbury and the matrix-determinant lemma). Landmarks are restricted to the observed design, and stopping is controlled by relative change in θ and mean residual. On five VLSE benchmark functions (n≤1000, Matérn-1.5, constant and polynomial means), the method is compared to uniform random Nyström of matched final m and to exact GP, and is reported to reach near-exact predictive accuracy with greater stability than random selection while scaling as O(T E n m²).","tokens_in":12630,"tokens_out":1414,"duration_ms":30010,"significance":"If the interleaving procedure reliably yields near-exact GP accuracy at linear cost in n, the work would be a useful, implementable contribution for large-scale computer experiments, where practitioners often prefer discrete landmarks drawn from the design and derivative-free optimizers. The algorithmic reductions in §3.1 are standard and correctly stated, the selection rule is cleanly tied to residual variance / pivoted Cholesky (§3.5), and the empirical protocol (unified custom R code, matched-m random baseline, S-RMSPE on held-out LHDs, 10 seeds) is transparent. The distinctive claim, however, is the value of interleaving relative to static greedy selection; that claim is only partially supported by the present evidence, so the incremental significance over known pivoted-Cholesky / residual-greedy Nyström is currently modest.","major_comments":[{"comment":"§3.5 identifies interleaving of landmark growth with hyperparameter updates (Algorithm 1) as the primary novelty relative to static pivoted Cholesky on K. Section 4 compares only to uniform random Nyström of matched final m and to full GP. There is no static control that (i) builds the same greedy/pivoted-Cholesky landmark set under fixed or once-estimated θ and (ii) optimizes the profiled likelihood (2) a single time. Without that ablation, gains on the five smooth LHD problems cannot be attributed to interleaving rather than greedy coverage alone, so the central methodological claim is under-supported.","section":"§3.5, Algorithm 1; §4"},{"comment":"The abstract and §3.4–§5 claim linear scaling and a framework for large-scale experiments where exact O(n³) inference is infeasible. All reported runs use n∈{500,1000}. In Table 2 (OTL, polynomial mean) adaptive Nyström is slower than full GP (30.66s vs 23.28s), which the text correctly attributes to sequential overhead at modest n. No experiment with n large enough that full GP is impractical is provided, so the scaling claim remains an asymptotic argument rather than a demonstrated advantage.","section":"Abstract; §3.4; Table 2; §5"},{"comment":"Table 3 (Wing Weight, polynomial mean) shows random Nyström with lower S-RMSPE than adaptive (0.0112 vs 0.0169). The text notes possible high-d clustering of greedy landmarks but does not quantify when residual-greedy selection fails or how defaults δθ=10^{-3}, δr=10^{-4} and m_max interact with that failure mode. Given that the abstract asserts the method “significantly outperforms random landmark selection in both accuracy and stability,” this counterexample needs either a clearer scope restriction or additional diagnostics (e.g., residual maps, length-scale trajectories) before the claim is left unqualified.","section":"Abstract; Table 3; §4.2"},{"comment":"§3.3 defaults (δθ=10^{-3}, δr=10^{-4}, landmarks ⊂ X, modest m0/m_max) are presented as consistently balancing accuracy and cost, yet the weakest modeling assumption is that the resulting low-rank surrogate remains faithful when the residual surface is multimodal or length-scales shift sharply. The five smooth VLSE functions with maximin LHDs do not stress this regime. At minimum, a sensitivity study over (δθ, δr) and a brief comparison to free inducing-point / variational sparse GP baselines (already cited in §3.5) would better bound the claim.","section":"§3.3; §4 experimental design"}],"minor_comments":[{"comment":"Figure 1 caption and §4.1 refer to “Eq. (17)” for the residual, but the residual selection criterion is equation (3) in §3.2; numbering appears inconsistent.","section":"§4.1; Figure 1"},{"comment":"Abstract and introduction say “five benchmark functions,” but §4.1–4.2 detail Borehole plus four others (Piston, OTL, Wing Weight, Steel Column)—fine, yet Borehole results are only in Figure 1 and not tabulated like Tables 1–4; a short table for Borehole would aid comparison.","section":"§4"},{"comment":"Typo in §2: “conditional variane” → “variance.”","section":"§2"},{"comment":"References include future-dated items (e.g., R Core Team 2026, lhs 2026, arXiv date 2026); verify bibliographic years for consistency with the venue.","section":"References"},{"comment":"Predictive variance formula after Algorithm 1 drops the usual mean-correction term present in the exact GP variance in §2; state explicitly whether this is intentional under the Nyström approximation or an omission.","section":"§3.3"}],"recommendation":"major_revision","confidential_remarks":"The technical core is competent methods work, not flawed algebra. I would not reject on correctness grounds. The skeptic’s missing-ablation point is load-bearing for the novelty claim and is the main reason I chose major_revision rather than minor_revision. If the authors add a static greedy/pivoted-Cholesky control and either larger-n timing or a clearly scoped claim, this could become a solid, incremental journal paper. Fit is appropriate for a computational statistics / computer-experiments audience; novelty relative to the large Nyström and sparse-GP literature is narrower than the abstract suggests."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The one thing worth knowing is that this is a clean, modest methods note for computer-experiment GPs. The math is standard and correctly done: Nyström + Woodbury/determinant lemma for the profiled likelihood, residual selection that the author herself equates to pivoted Cholesky, and the usual O(nm²) story. What is actually packaged as new is Algorithm 1—grow landmarks by max residual, then warm-start re-optimize θ and η, stop on dual tolerances. That interleaving is the claimed novelty; the rest is assembly of known parts.\n\nWhat it does well: the write-up is clear, the complexity accounting is honest, and the five VLSE benchmarks (n≤1000) show adaptive usually beating matched-size random Nyström in S-RMSPE and seed stability, often near full GP with m a small fraction of n. Tables are readable; the author flags the OTL case where sequential overhead exceeds a single O(n³) fit. No circularity games.\n\nSoft spots in proportion. The stress-test lands: §4 never runs the natural control—static pivoted-Cholesky / greedy residual landmarks under fixed or once-fit θ, then one likelihood optimization. Without it you cannot attribute gains to interleaving rather than greedy coverage alone. Defaults δθ=10^{-3}, δr=10^{-4} and landmarks restricted to X are under-explored; Wing Weight polynomial already has random slightly ahead. Linear scaling is asserted, not shown where n makes exact GP impossible. No code, no leverage-score / VFE / FITC baselines. None of that is fatal for a methods paper; it just means the strongest abstract sentence is only partly supported.\n\nWho it is for: people already doing sparse/Nyström GPs on smooth computer experiments who want a practical sequential recipe. Not a theory advance and not a new modeling paradigm. I would send it to peer review; a referee can demand the static ablation, larger n, and code. Worth a look if you work in this lane; I would not rearrange a reading group around it unless someone is actively building scalable kriging tools.","headline":"Solid engineering paper: greedy residual Nyström interleaved with hyperparameter fits beats random on small smooth benchmarks, but the distinctive claim lacks the obvious static-greedy control.","tokens_in":13283,"tokens_out":529,"would_cite":false,"duration_ms":15857,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62M20","62G08","65F30","68T05"],"pacs":[],"model":"grok-4.5","headline":"Greedy trace-residual landmarks, updated with the kernel itself, let Gaussian process regression match exact accuracy at linear cost in sample size.","keywords":["Gaussian process regression","Nyström approximation","landmark selection","trace residual","scalable GPR","computer experiments","Woodbury identity","hyperparameter optimization"],"falsifier":"On a larger or less smooth computer-experiment suite, run the same adaptive loop against exact GP and random Nyström with matched landmark count: if adaptive S-RMSPE stays far above exact GP, or fails to beat random selection with lower variance across random starts, the central claim fails.","tokens_in":13184,"feed_emoji":"📈","tokens_out":998,"duration_ms":23847,"temperature":0.7,"pith_summary":"Gaussian process regression gives reliable predictions and uncertainty, but the usual matrix work grows as the cube of the number of data points, so it stalls on large computer experiments. A standard fix is the Nyström low-rank shortcut that rebuilds the kernel from a small set of landmark points; the catch is that random landmarks often miss the structure that matters. This paper’s claim is that you can choose those landmarks greedily by repeatedly picking the training point with the largest leftover kernel residual, and that you should re-estimate the kernel hyperparameters after each addition so the residual is measured under the current covariance. On five standard benchmark simulators the adaptive loop reaches accuracy close to a full Gaussian process while using only a small fraction of the points as landmarks, and it is more stable than random selection. The practical payoff is a linear-in-n procedure that still behaves like exact GP inference for the kinds of smooth design problems common in computer experiments.","feed_headline":"Adaptive landmarks make large GP regression match exact accuracy","feed_subtitle":"Greedy residual picks, updated with the kernel, beat random Nyström at linear cost in n","key_machinery":"The adaptive Nyström loop: at each step add the training point that maximizes the residual variance r(x_i) = K(x_i,x_i) − k_Z(x_i)^T W^{-1} k_Z(x_i) (equivalently the pivoted-Cholesky pivot), then re-minimize the Woodbury-reduced Nyström profiled likelihood until both the hyperparameter change and the mean residual fall below fixed tolerances.","core_discovery":"Interleaving greedy maximization of the kernel trace residual with hyperparameter re-optimization produces a Nyström landmark set that yields predictive accuracy comparable to exact Gaussian process regression and clearly better and more stable than uniform random landmarks, while the dominant cost stays linear in the training size.","pith_inferences":["The equivalence to pivoted Cholesky under a moving kernel suggests the method could inherit pivoted-Cholesky error bounds if those bounds were re-proved with slowly changing length-scales.","Because landmarks stay discrete and inside the training set, the approach may be easier to warm-start or parallelize than variational inducing-point methods that move continuous coordinates.","A natural stress test left open by the paper is multimodal or discontinuous response surfaces, where greedy residual peaks might cluster and leave large unexplored regions.","Automated schedules for the residual and hyperparameter tolerances could remove the remaining hand-tuned knobs the authors flag for future work."],"forward_implications":["For smooth computer experiments, near-exact GP predictions can be obtained with landmark counts on the order of a few percent of n once the residual criterion and interleaved updates are used.","Uniform random landmark selection is an unreliable baseline for Nyström GPR accuracy and stability under the same computational budget.","Restricting landmarks to observed design points still leaves enough flexibility for the residual-greedy rule to cover the kernel feature space on the reported problems.","The same residual-driven expansion can be stopped by simple relative tolerances, giving a practical accuracy–cost dial without continuous inducing-point optimization.","When n is only a few hundred, sequential re-optimization can erase the asymptotic gain; the method’s advantage appears as n grows."],"fun_headline_variants":["Adaptive Nyström landmarks match exact GP accuracy at linear cost","Greedy trace-residual landmarks beat random Nyström in GP regression","Interleaved landmark growth and kernel tuning rivals exact GPR","Adaptive greedy Nyström stabilizes large-scale GP predictions","Residual-maximizing landmarks close the gap to exact GP inference"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The method assumes that stopping when hyperparameter change and average residual are small, with landmarks forced to be a subset of the observed design, is enough to keep the low-rank surrogate faithful for near-exact predictions in general—not only on the smooth Latin-hypercube benchmarks tested.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive Nyström landmarks match exact GP accuracy at linear cost","Greedy trace-residual landmarks beat random Nyström in GP regression","Interleaved landmark growth and kernel tuning rivals exact GPR","Adaptive greedy Nyström stabilizes large-scale GP predictions","Residual-maximizing landmarks close the gap to exact GP inference"]},"model":"grok-4.5","effort":"low","cost_usd":0.003017,"raw_usage":{"total_tokens":1016,"prompt_tokens":661,"num_sources_used":0,"completion_tokens":66,"cost_in_usd_ticks":30168000,"prompt_tokens_details":{"text_tokens":661,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":289,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":661,"tokens_out":66,"duration_ms":6394,"temperature":1.0,"reasoning_tokens":289,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-31T01:15:12.221817+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On a larger or less smooth computer-experiment suite, run the same adaptive loop against exact GP and random Nyström with matched landmark count: if adaptive S-RMSPE stays far above exact GP, or fails to beat random selection with lower variance across random starts, the central claim fails.","supporting_citations":[],"review_version":1}