{"id":"7dc930ea-f569-4f9e-b618-eac5fe771fa5","arxiv_id":"2608.01268","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Moment-based detection of an epsilon-scale, mass-f distribution change requires polynomial degree at least log(1/f)/(2 epsilon), which yields a bandwidth rule for kernel shift tests.","lead":"Distribution shift detection usually starts with choosing a statistic; this paper proves a scale law linking the required moment order to the spatial scale and mass of the change, and turns it into a rule for setting the bandwidth of an MMD kernel test. The rule is tested on real embedding streams against persistent homology, where the authors report a cost gap rather than a power gap.","discovery_kind":"first_principles","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Bandwidth rule rests on unproved Gaussian tightness that the body itself disclaims, while the abstract asserts it as fact","rationale":"The paper does two things: it proves a one-sided lower bound on polynomial degree, and it uses an unproved matching upper bound by Gaussian probes to output a bandwidth rule. The lower bound (Theorem 1 and the Chebyshev argument) appears internally coherent: the Gauss-quadrature construction is explicit, and the annulus example correctly demonstrates that the bound cannot certify sufficiency. The fragile step is the transition from 'moments of order N resolve scale 1/N' to 'Gaussian test functions attain O(log(1/f)/epsilon)'. The manuscript is unusually candid: Section 3 states that the Gaussian attainment is a fitted exponent, not a proof, that the Bernstein-ellipse argument has not been carried out, and that tightness is not claimed. But the abstract and Section 5 assert attainment as fact, and the bandwidth calibration in Section 6.5 is presented as the law's practical content. If the correct rate for Gaussian probes were, say, epsilon^{-2} log(1/f), the law would not predict sigma ~= epsilon; it would predict a bandwidth much smaller than the feature scale or a different detector. The empirical sigma*/epsilon measurements are suggestive but cannot substitute for the missing asymptotic argument: they use n=26 after excluding one replicate, an argmax over a multimodal AUC curve, W=200, and a compensation-of-biases step (sigma = epsilon_hat/2) that the paper itself describes as two errors cancelling. So the central practical claim is conditional on an unproved tightness result. This is exactly the kind of limitation the reviewing rule asks us to flag, and the reader already identified it. I do not see a more load-bearing concern: the lower-bound proof is not obviously flawed, and the empirical sections are self-critical and reproducible in principle. The appropriate disposition remains conditional: accept the one-sided law and the honest negative results, but require either a proof of Gaussian attainment or a softening of the headline claim before the bandwidth rule is stated as a consequence of the law.","tokens_in":15686,"tokens_out":12710,"duration_ms":123712,"concrete_test":"Derive a rigorous bound on the required degree N*(epsilon,f) for the Gaussian probe g_sigma(x)=exp(-||x||^2/(2*sigma^2)) by bounding its Chebyshev coefficients on a Bernstein ellipse with parameter depending on epsilon/sigma. Concretely: establish an inequality ||g_sigma - p_N||_inf <= C rho^{-N} with rho = 1 + c(epsilon/sigma)^2 (or the correct ellipse parameter), then solve for the N at which the error falls below f. If the resulting N*(epsilon,f) is Theta(epsilon^{-1} log(1/f)) with the predicted constant, Gaussian attainment holds and the bandwidth rule has a foundation; if it is Theta(epsilon^{-2} log(1/f)) or Theta(epsilon^{-1/2} log(1/f)), the attainment claim fails and sigma ~= epsilon is unsupported by the law.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is two-sided: the lower bound N* >= log(1/f)/(2*epsilon) is paired with the assertion that Gaussian test functions attain this rate, and that attainment is what converts the law into the bandwidth rule sigma ~= epsilon for RBF-MMD. The lower-bound side is in reasonable shape: Theorem 1's Gauss-quadrature construction is explicit, and the Chebyshev contrast argument is plausible. The load-bearing weak point is the tightness side. In Section 3, after Theorem 2, the paper states that Gaussian attainment is supported only by 'numerical evidence—a fitted exponent, not a proof,' that closing the gap 'requires bounding the Chebyshev coefficients of an epsilon-wide Gaussian via a Bernstein ellipse argument,' and that 'we do not claim tightness.' Yet the abstract and Section 5 (P2) state flatly that 'the upper bound is attained by Gaussian test functions, the RKHS witness of an RBF kernel.' The calibration rule depends exactly on this unproved attainment: if the Gaussian probe actually requires N* ~ epsilon^{-2} log(1/f), or any exponent worse than epsilon^{-1}, then there is no theoretical reason for sigma ~= epsilon to be near-optimal. The empirical sigma*/epsilon measurements cannot settle the asymptotic question: they use in-sample argmax AUC over a multimodal objective, n=26 after excluding one replicate, window size W=200, and a hand-set 1/2 correction in the data-driven closure. The paper is admirably explicit about these caveats, but that makes the gap between the body's scope note and the headline claim the decisive issue: the practical output is conditional on a proof that is flagged as missing.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper develops a 'scale law' for moment-based detection of distribution shift: a feature of spatial scale ε carrying mass fraction f cannot be certified by polynomial (moment) tests of degree below about log(1/f)/(2ε), and a Gauss-quadrature construction gives N* ≥ 4b−1 for a b-scale topology. The same section proves a scale-relative upper bound under an f-margin hypothesis and exhibits an annulus matching all moments through fourth order while carrying H1. The practical prediction is that RBF-MMD should use bandwidth σ ≈ ε because Gaussian test functions are presented as attaining the rate. The empirical part measures σ*/ε with median 1.12 (IQR 1.01–1.52, n=26) on embedding streams, tests an adaptive adversary evading mean/covariance/k-NN/kurtosis, and compares persistent homology summaries, finding that total persistence dominates the first landscape and that a cost gap of roughly 116× kurtosis remains. The body repeatedly disclaims the Gaussian-tightness proof and the data-driven closure, while the abstract asserts attainment as fact.","tokens_in":15968,"tokens_out":6402,"duration_ms":58979,"significance":"If the lower bound holds, it is a valuable and clean result: the cost of moment-based detection is governed by feature fineness and mass, not feature count, and the paper supports it with explicit constructions, archived reproducible code, and an unusually honest discussion of repeated cells, post-hoc formulation, and excluded replicates. The calibration rule σ ≈ ε is practically useful, and the adversarial experiment with an MMD-aware adversary is a useful stress test. However, the two-sided form of the law — attainment by Gaussian test functions — is unproved and is exactly the premise on which the bandwidth rule rests; the empirical calibration evidence is partly in-sample and relies on a hand-set correction. These issues are fixable by reframing or by a proof, so the work is worthy of revision.","major_comments":[{"comment":"The abstract and conclusion assert that 'the upper bound is attained by Gaussian test functions' and that this is what predicts the bandwidth, but §3 states that Gaussian attainment is supported only by 'numerical evidence—a fitted exponent, not a proof', that closing the gap 'requires bounding the Chebyshev coefficients of an ε-wide Gaussian via a Bernstein ellipse argument', and that 'we do not claim tightness'. This is a load-bearing inconsistency: the calibration rule σ ≈ ε is derived precisely from the unproved attainment, so as written the paper's headline promise exceeds its proof. Either provide the Bernstein ellipse bound, or rephrase the abstract, P2, and conclusion to say the bandwidth rule is an empirical conjecture consistent with the lower bound. The lower bound can stand either way.","section":"Abstract; §3 after Theorem 2; §5 P2"},{"comment":"The empirical validation of the calibration rule is partly circular and should be presented as such. σ* is defined as the in-sample argmax AUC on the same test windows that are then used to report the median 1.12; with a multimodal AUC(σ) curve and n=26 after excluding one replicate, this measures the best in-sample peak, not the predictive performance of the rule. The closing-the-loop result AUC ≥ 0.95 is obtained with σ_pred = ε_hat/2, where the factor 1/2 is admitted to be a correction that makes the loop close by 'partial compensation of two errors'. This is not a validation of the rule as stated. Please report out-of-sample or cross-validated bandwidth selection, or explicitly label the measured constant as a fitted calibration constant rather than a test of the law.","section":"§6.5"},{"comment":"The scale-relative upper bound is weaker than the surrounding text sometimes suggests: it assumes the f-margin condition on both radii, which the authors admit 'encodes a good part of the conclusion', and it proves agreement of the nerve of a fixed ε-net, not of the Čech complex of the sample; the interleaving argument is explicitly not carried out. This does not invalidate the lower-bound law, but the abstract's claim of a scale law constraining 'any moment-based choice' is supported mainly by the lower bound, and the upper side should be segregated as conditional.","section":"§3, Theorem 2"}],"minor_comments":[{"comment":"The text refers to 'the annulus of §4', but the annulus counterexample is given in §3 after Remark 1; correct the cross-reference.","section":"§5 (H1)"},{"comment":"The symbol b is overloaded: it denotes both the Mardia kurtosis statistic and the complexity parameter in Theorem 1; consider a different symbol for one of them to reduce confusion.","section":"Throughout"},{"comment":"The column header 'ms/window×kurtosis' is ambiguous; the text clarifies it, but the header should say 'cost relative to kurtosis' or similar.","section":"Table 2"},{"comment":"The parenthetical explanation of the floor-protocol ratios is accurate but interrupts the main line; consider moving it to a footnote.","section":"§6.5"}],"recommendation":"major_revision","confidential_remarks":"The disclosure of order-of-discovery and repeated-cell variation is exemplary, and the lower-bound construction appears sound from the proof sketch. The main risk is the abstract/conclusion overclaim about Gaussian attainment, which is the only genuinely load-bearing gap; I suggest the editor seek a proof consultation on the Bernstein-ellipse point before acceptance. The paper's lack of institutional affiliation is not relevant to the scientific assessment."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: the lower-bound results are real, and the bandwidth rule is a good idea that the paper currently over-sells. The body openly says the Gaussian attainment is unproved, but the abstract and conclusion state it as fact. That gap is the thing to press.\n\nWhat's new: the Gauss-quadrature construction giving N* ≥ 4b−1 for b-scale topology is clean and I don't recall it in the literature. The Chebyshev lower bound N* ≥ log(1/f)/(2ε) is plausible and self-contained. The annulus with matched mean, covariance, and all fourth-order moments yet H1 ≠ 0 is a good counterexample that correctly kills any \"fourth-order suffices\" claim. The empirical finding that the persistence summary matters more than the filtration is also new and believable: total persistence at recall 0.75 vs 0.00 for the first landscape is a big split.\n\nThe paper's honesty is unusual and worth crediting: repeated-cell variation, post-hoc hypothesis ordering, the excluded replicate, the hand-set 1/2 correction, and the line that the Gaussian rate is \"a fitted exponent, not a proof.\" That makes the soft spots easier to diagnose.\n\nThe soft spots are real. The load-bearing weakness is the tightness side. The bandwidth rule σ ≈ ε depends on Gaussian test functions attaining O(log(1/f)/ε). The body says this is unproved and \"we do not claim tightness,\" but the abstract says the upper bound is attained by Gaussian test functions. That overstates the evidence. If Gaussian probes actually cost ε^{-2} log(1/f), the calibration rule loses its theoretical reason. The empirical validation is also partly circular: bandwidth is selected as argmax AUC on the same test windows, then reported as successful. The data-driven closure uses σ_pred = ε_hat/2, an explicit fudge factor, and only on two configurations. n=26 with one exclusion is thin, though the paper is transparent about it.\n\nNone of this sinks the lower-bound side. The Gauss-quadrature argument is explicit, the annulus checks out, and the empirical section is far more careful than the average arXiv ML paper. I'd send it to a serious referee. The fixes are straightforward: soften the attainment claim in the abstract, report out-of-sample bandwidth selection, and either prove or properly flag the Bernstein ellipse gap. This paper deserves engagement, not a desk reject.","headline":"Solid lower-bound and counterexample work; the bandwidth rule is oversold in the abstract and needs a softer claim, but the paper deserves a serious referee.","tokens_in":16533,"tokens_out":3726,"would_cite":true,"duration_ms":30662,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G10","62R40","41A50"],"pacs":[],"model":"deepseek-v4-flash","headline":"A scale law fixes how fine a distribution change moments can detect, and sets the kernel bandwidth.","keywords":["distribution shift","moment tests","scale law","MMD bandwidth calibration","persistent homology","kernel two-sample tests","adversarial detection","Chebyshev extremal problem"],"falsifier":"Compute the Chebyshev coefficients of an $\\varepsilon$-wide Gaussian by an elliptic-contour argument and check whether the minimal degree needed to separate mass fraction $f$ from zero grows as $\\log(1/f)/\\varepsilon$; if the measured exponent in $\\varepsilon$ is not close to 1, the tightness of the scale law is refuted. A complementary experiment: run the paper's controlled-deformation protocol below $\\varepsilon \\approx 0.28$ times the inter-point spacing with larger mass fractions; if no interior AUC maximum appears near $\\sigma \\approx \\varepsilon$, the calibration rule has a resolution floor the paper could not rule out.","tokens_in":15424,"feed_emoji":"📏","tokens_out":19838,"duration_ms":150917,"temperature":0.7,"pith_summary":"This paper asks how fine a change in a data stream moment-based tests can certify. The answer is a scale law: a feature of spatial scale $\\varepsilon$ carrying mass fraction $f$ requires polynomial test degree $N^* \\ge \\log(1/f)/(2\\varepsilon)$, so cost is governed by feature fineness, not feature count. The law is one-sided: it never shows a given order suffices, and the paper exhibits an annulus with the same mean, covariance, and all fourth-order moments as a filled disk while still carrying a hole. On the constructive side, Gaussian test functions — the reproducing-kernel Hilbert space witness of a radial-basis-function kernel — attain the rate, making them the predicted near-optimal probe for a maximum-mean-discrepancy test; this yields a calibration rule: set the kernel bandwidth to the feature scale, with measured $\\sigma^*/\\varepsilon$ median 1.12. Against an adversary optimized against the defender's mean, covariance, k-NN, and kurtosis statistics, only the bandwidth-matched kernel test still detects.","feed_headline":"A scale law fixes the resolution limit of moment-based shift tests","feed_subtitle":"Detecting a feature of scale ε and mass f costs degree log(1/f)/(2ε); set the RBF bandwidth to the feature scale.","key_machinery":"The argument runs on three linked objects. The Chebyshev extremal problem supplies the lower bound: a degree-$N$ polynomial localised at scale $\\varepsilon$ has contrast at most $T_N((1+\\varepsilon^2)/(1-\\varepsilon^2)) \\approx e^{2N\\varepsilon}$, so separating mass $f$ from zero requires $N \\gtrsim \\log(1/f)/(2\\varepsilon)$. Gauss quadrature supplies the topology-specific bound: replacing a rotation-invariant ball law by $b$ concentric spheres with Gauss weights matches all moments through degree $4b-2$, proving $N^* \\ge 4b-1$ for a $b$-scale topology, and showing the cost is packing- rather than count-driven. The constructive side is the RKHS witness of an RBF kernel — a Gaussian test function of width matched to $\\varepsilon$ — which attains the rate and turns the bound into the bandwidth calibration rule $\\sigma^* \\approx \\varepsilon$ for MMD.","core_discovery":"On its own terms, the paper establishes a lower bound on moment-based detection of distribution shift: certifying a feature of spatial scale $\\varepsilon$ carrying mass fraction $f$ requires polynomial tests of degree $N^* \\ge \\log(1/f)/(2\\varepsilon)$, proved through the extremal problem for polynomial contrast, and a quadrature construction shows $N^* \\ge 4b-1$ for a $b$-scale topology. The cost is set by the fineness and mass of the feature, not by the number of features. The bound is one-sided, and the annulus counterexample shows that equality through order four does not imply topological triviality. The constructive counterpart is that Gaussian test functions — the reproducing-kernel Hilbert space witness of a radial-basis-function kernel — attain the rate, so the law predicts the optimal bandwidth of a maximum-mean-discrepancy test is the feature scale; the paper measures $\\sigma^*/\\varepsilon$ with median 1.12 and interquartile range [1.01, 1.52] over three settings and three scales (n=26). Against an adversary optimized against the defender's own statistics, only the bandwidth-matched kernel test detects the change.","pith_inferences":["The paper does not prove that Gaussian probes attain the $O(\\log(1/f)/\\varepsilon)$ rate; an elliptic-contour bound on the Chebyshev coefficients of an $\\varepsilon$-wide Gaussian is the natural next step that would turn the numerical support into a theorem.","Because the lower bound constrains only polynomial (moment) tests, it points toward non-polynomial witnesses — for example compactly supported or adaptively shaped kernels — that could beat the moment bound; the paper's MMD result is one concrete instance for Gaussian kernels.","The annulus counterexample implies that any claim of redundancy between low-order moments and topological features must be tested against adversarially optimized deformations; extending the paper's audit to other embedding families would check how generic the redundancy is.","The persistence-summary effect suggests a design rule for monitoring pipelines: use scale-averaged summaries such as total persistence for coarse, single-scale alarms, and reserve scale-localized summaries for problems where the relevant scale is unknown; the paper's law predicts exactly this ordering."],"forward_implications":["Moment-based detectors have an intrinsic resolution limit: order-$N$ moments resolve scale about $1/N$, so features finer than that cannot be certified by moments of that order, and a feature of mass fraction $f$ adds a $\\log(1/f)$ factor.","Cost is set by fineness, not feature count: at fixed scale $\\varepsilon$, adding more features raises the required moment order only slowly (empirically from 4 to 8 as $b$ goes from 1 to 16 in $d=2$), because a single degree-$N$ polynomial resolves scale $1/N$ uniformly; packing many features into a bounded domain forces their scale down, which is what drives the cost.","The bandwidth of an RBF-kernel MMD test should be set to the feature scale: measured $\\sigma^*/\\varepsilon$ has median 1.12 with interquartile range [1.01, 1.52] over three settings and three scales under one protocol, and a bandwidth predicted from a data-driven scale estimate reaches AUC $\\ge 0.95$.","An adversary optimized against mean, covariance, k-NN, and kurtosis can evade all of those statistics, but a bandwidth-matched kernel test still detects the change at AUC 1.00.","On the covariance-preserving attack, total persistence reaches recall 0.75 at FPR 1% where the first persistence landscape reaches 0.00, and the surviving gap against cheap statistics is computational cost (about 116 times kurtosis), not detection power."],"supporting_citations":[{"why":"defines the kernel two-sample test whose RBF bandwidth the scale law calibrates.","marker":"[14]"},{"why":"supplies the multivariate kurtosis statistic used as a baseline and as the empirical redundancy target.","marker":"[17]"},{"why":"defines persistence landscapes, the summary whose choice dominates the paper's topological results.","marker":"[4]"},{"why":"provides the benchmark of shift detection on learned representations that frames the task.","marker":"[19]"},{"why":"supplies the intrinsic-dimension estimator used to justify PCA reduction before topological computation.","marker":"[11]"},{"why":"provides bootstrap confidence intervals for persistence diagrams used to compare summaries and baselines.","marker":"[12]"}],"fun_headline_variants":["Scale law sets the cost of detecting distribution shift","Feature fineness rules shift detection, not feature count","RBF bandwidth = feature scale: a law for MMD shift tests","Moments vs topology: a scale law decides shift detection","Small shifts, big cost: a scale law for detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The calibration rule rests on the assumption that bell-shaped (Gaussian) probe functions actually attain the predicted detection rate, which the paper supports only by numerical fits rather than proof; if that attainment fails, the bandwidth rule loses its theoretical foundation even though the one-sided lower bound may still hold.","fun_headline_variants_meta":{"raw":{"variants":["Scale law sets the cost of detecting distribution shift","Feature fineness rules shift detection, not feature count","RBF bandwidth = feature scale: a law for MMD shift tests","Moments vs topology: a scale law decides shift detection","Small shifts, big cost: a scale law for detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000796,"raw_usage":{"total_tokens":3611,"prompt_tokens":1158,"completion_tokens":2453,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":774,"completion_tokens_details":{"reasoning_tokens":2372}},"tokens_in":774,"tokens_out":2453,"duration_ms":17418,"temperature":1.0,"reasoning_tokens":2372,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:10:11.806462+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the Chebyshev coefficients of an $\\varepsilon$-wide Gaussian by an elliptic-contour argument and check whether the minimal degree needed to separate mass fraction $f$ from zero grows as $\\log(1/f)/\\varepsilon$; if the measured exponent in $\\varepsilon$ is not close to 1, the tightness of the scale law is refuted. A complementary experiment: run the paper's controlled-deformation protocol below $\\varepsilon \\approx 0.28$ times the inter-point spacing with larger mass fractions; if no interior AUC maximum appears near $\\sigma \\approx \\varepsilon$, the calibration rule has a resolution floor the paper could not rule out.","supporting_citations":[{"cited_title":"A kernel two-sample test.Journal of Machine Learning Research, 13(25):723–773, 2012","cited_arxiv_id":null,"evidence_quote":"defines the kernel two-sample test whose RBF bandwidth the scale law calibrates."},{"cited_title":"Measures of multivariate skewness and kurtosis with applications.Biometrika, 57(3):519–530, 1970","cited_arxiv_id":null,"evidence_quote":"supplies the multivariate kurtosis statistic used as a baseline and as the empirical redundancy target."},{"cited_title":"Statistical topological data analysis using persistence landscapes.Journal of Machine Learning Research, 16(1):77–102, 2015","cited_arxiv_id":null,"evidence_quote":"defines persistence landscapes, the summary whose choice dominates the paper's topological results."},{"cited_title":"Failing loudly: An empirical study of methods for detecting dataset shift","cited_arxiv_id":null,"evidence_quote":"provides the benchmark of shift detection on learned representations that frames the task."},{"cited_title":"Estimating the intrinsic dimension of datasets by a minimal neighborhood information.Scientific Reports, 7(1):12140, 2017","cited_arxiv_id":null,"evidence_quote":"supplies the intrinsic-dimension estimator used to justify PCA reduction before topological computation."}],"review_version":2}