{"id":"3668004a-eb1a-4f8d-8a32-03b04d1abfc8","arxiv_id":"2412.09369","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Conformalized randomized prior operators give per-location calibrated uncertainty intervals for wavelet and spiking wavelet neural operators, with a Gaussian process extension for zero-shot super-resolution UQ.","lead":"This paper combines randomized prior ensembles with split conformal prediction to produce uncertainty bounds for neural operators, including spiking variants, and adds a Gaussian process extension for super-resolution. Tested on four PDE datasets, the method reports better coverage than quantile-based baselines, though some implementation details and guarantees need clarification.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The coverage claim is not verifiable because Eq. (7) and Algorithm 1 insert an unspecified width multiplier z; if z≠1 the conformal proof does not cover the reported band.","rationale":"The reader's weakest-assumption analysis already identifies the unspecified multiplier z, and my independent reading confirms that this is the most load-bearing gap in the central claim. The conformal Lemma 1 is valid only for the band µ ± qs; the actual algorithm announces µ ± zqs. If z is a fixed ex ante constant with z ≥ 1, the method is still valid but conservative, and the reported 'desired coverage' language would be misleading only in the sense that the realized coverage is guaranteed to be above 95%, not calibrated to exactly 95%. If z is chosen post hoc to make coverage look good, the empirical comparison is not a clean test of conformal calibration. Either way, the missing value makes the experiment impossible to reproduce from the manuscript. Secondary concerns, such as the post hoc selection of the best 10 of 20 RP copies in Example IV and the fact that per-location coverage is not a joint field-wide guarantee, are real but less central: the stated claim is explicitly per-location, and the selection issue affects only one example. Because the identified gap is addressable by specification and code release, and because the reader already marked the paper CONDITIONAL, my stress-test does not change the verdict.","tokens_in":19994,"tokens_out":3130,"duration_ms":34023,"concrete_test":"Set z = 1 in Eq. (7) and Algorithm 1, then recompute the per-location coverage counts in Tables 3–7 on the same trained models and calibration/test splits. If every location still has ≥ 95% coverage, the unspecified multiplier is not load-bearing; if any location drops below 95%, the reported coverage depends on the hidden z. As a secondary check, ask the authors to report the exact z used for every example and verify that z ≥ 1, so that the conformal guarantee is at least conservative.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that CRP-WNO and CRP-VSWNO give the required 95% coverage at every solution-grid location (Tables 3–7). But the calibrated band is defined in Eq. (7) as Cp = [µ − zqs, µ + zqs], and Algorithm 1 step 9 uses the same expression. Lemma 1 proves a coverage guarantee for the score e = |y−µ|/s and the set {e ≤ q}, i.e. the band [µ − qs, µ + qs]. Introducing z changes the induced score to |y−µ|/(zs). The conformal quantile q is computed from the unmodified scores, so the guarantee in Lemma 1 applies to that modified score only if z = 1. For z > 1 the band is conservative and still achieves at least 1−α, but for z < 1 the guarantee fails. The manuscript never states the value of z, never restricts z ≥ 1, and Appendix B says code will be released only after acceptance. Consequently, the reported per-location coverages are not reproducible from the text, and the 'tighter bounds' comparison against RCQNO in Table 8 is confounded because interval width scales linearly with z. The headline claim therefore rests on an unspecified implementation detail rather than on the stated conformal-calibration argument.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Conformalized Randomized Prior Operator (CRP-O) framework that combines randomized-prior ensembles with split conformal prediction to produce calibrated prediction bands for wavelet and spiking neural operators. The method is applied element-wise on the output grid, with a Gaussian-process extension for zero-shot super-resolution. Four PDE test cases (Burgers, Darcy on rectangular and triangular domains, and Helmholtz) are used to claim that the conformalized bands achieve 95% coverage at every grid location except one location in the fourth example, and that they outperform RP, Q-WNO, CQ-WNO, and RCQNO baselines.","tokens_in":20285,"tokens_out":5717,"duration_ms":61994,"significance":"The paper addresses a timely and practical problem: distribution-free uncertainty quantification for neural operators, including energy-efficient spiking variants. The core idea of using an RP ensemble as a heuristic uncertainty measure and then applying split conformal calibration is sensible, and the standard conformal lemma is invoked in substance. The four PDE experiments cover nontrivial settings, and the comparison with RCQNO is a useful practical addition. However, the central coverage claim currently rests on an unspecified width multiplier z in the prediction band, the super-resolution extension is heuristic rather than guaranteed, and the empirical protocol has several reproducibility gaps. If these issues are resolved, the framework would be a solid contribution to the neural-operator UQ literature.","major_comments":[{"comment":"The prediction band is defined as Cp = [mu - z q s, mu + z q s], but the conformal guarantee in Lemma 1 applies to the band [mu - q s, mu + q s] induced by the score e = |y - mu|/s. If z is not fixed to 1, the calibrated score is effectively |y - mu|/(z s), and the quantile q computed from the unmodified scores does not provide the stated coverage: for z < 1 the guarantee fails, while for z > 1 the band is conservative. The manuscript never gives the value of z, so the reported per-location coverages and the interval widths in Table 8 are not reproducible from the text, and the comparison with RCQNO is confounded. The authors must either set z = 1 (and state it explicitly) or, if z is chosen a posteriori, prove a conformal guarantee for the modified score and report the chosen z.","section":"Section 3.2, Eq. (7), Algorithm 1 step 9"},{"comment":"The Gaussian-process extension interpolates the conformal parameter q from the coarse calibration grid to a finer prediction grid and then uses the GP predictive mean as the calibrated q at new locations. The split-conformal guarantee, however, applies to the q computed directly from calibration scores at the original grid locations; the GP-interpolated values are not quantiles of calibration scores for the fine-grid locations. The coverage improvements shown in Figs. 3(a) and 5(a) are therefore empirical observations, not consequences of Lemma 1. The paper should explicitly state that the super-resolution extension is a heuristic (which the text partially does) and should not claim a calibrated guarantee for the fine grid without additional analysis.","section":"Section 3.3, Figs. 3 and 5"},{"comment":"The manuscript states that for E-IV the authors train 20 RP copies and 'select the best 10', but it does not specify the selection criterion (e.g., validation loss, calibration score, or test error). If the selection uses the test set, the reported uncertainty calibration is invalid. Additionally, Table 8 compares RCQNO with n = 500 calibration samples against CRP-WNO with n = 150 calibration samples, so the reported interval widths are not directly comparable; the difference in calibration set size could affect the conformal quantile and hence the band width. The comparison should be made at matched calibration sample sizes or the dependence on n should be discussed and quantified.","section":"Section 4.4, E-IV and Table 8"},{"comment":"The headline claim is that CRP-WNO and CRP-VSWNO achieve at least 95% observed coverage at all (or all but one) grid locations. Because the test set contains only 100 samples, the binomial uncertainty in an observed 95% coverage is large: e.g., 95 successes out of 100 gives a 95% Clopper-Pearson lower bound of about 88.6%. The paper should report confidence intervals for the coverage proportions, or repeat the evaluation over multiple test splits, before claiming that 'the required coverage is achieved at all locations'.","section":"Section 4, Tables 3–7"}],"minor_comments":[{"comment":"The acronym SCP is expanded as 'Stochastic Cross-Validation Procedure', but the correct term is 'Split Conformal Prediction'; this should be fixed.","section":"Section 3.2"},{"comment":"The quantile definition is written as Quantile({e}, ceil((1-alpha)(n+1))/n) in Eq. (9) but as q = e_{ceil((1-alpha)(n+1))} in Eq. (11). These are different formulas; the notation should be aligned and the order-statistic indexing clarified.","section":"Eqs. (9) and (11)"},{"comment":"The proof writes the sorted scores as e1 <= ... <= en and then states P(et <= ei) = i/(n+1), reusing the symbol ei for both the raw and sorted scores; this is a notational flaw in an otherwise standard argument.","section":"Lemma 1 proof"},{"comment":"There is a typo in 'thetth time step'; it should read 'the t-th time step'.","section":"Eq. (3)"},{"comment":"The symbol ⊘ is used for element-wise division but is not defined; please define it in the text or in the algorithm.","section":"Algorithm 1, step 2"},{"comment":"The domain is given as x, z in [0,690] with delta x = delta z = 70, but the output is described as a 70 x 70 grid; the relation between these numbers should be clarified.","section":"Section 4.4, Eq. (31)"},{"comment":"The nomenclature in Table 1 uses CQ-WNO, but the text at the start of Section 4 refers to 'CRQ-WNO'; this inconsistency should be fixed.","section":"Section 4, Table 1 and text"},{"comment":"The statement that sample code 'is to be released after acceptance' is incompatible with the reproducibility needs of the central coverage claim, especially given the unspecified z; please release code or provide complete implementation details.","section":"Appendix A"}],"recommendation":"major_revision","confidential_remarks":"The core methodology is sound in principle and the empirical scope is appropriate for a stat.ML audience. The main blocker is the unspecified z in Eq. (7), which makes the central coverage claim unverifiable; this is fixable but requires a clear restatement of the algorithm or an explicit proof for the modified score. I also recommend that the editor ask the authors to justify the 'best 10 of 20' selection in E-IV and to either match calibration sizes in the RCQNO comparison or discuss the dependence on n. With those changes, the paper could be suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read it straight through. The core move is simple: take the ensemble spread from randomized-prior wavelet/spiking operators, use it as the nonconformity score, and calibrate with split conformal prediction per output-grid location. That is a natural wrapper, and I believe it is the first time it is applied to VSWNO; the GP-based extension of q to unseen super-resolution grids is also new, though heuristic. The empirical story is consistent across four PDE examples: the conformalized RP versions hit roughly 95% per-location coverage, while quantile-based baselines often miss entire regions. That counts for something. The paper also cites the two prior conformal-operator papers (Moya et al., Ma et al.) and does not oversell conditional coverage.\n\nThe soft spots are real, and one is load-bearing. Eq. (7) and Algorithm 1 define the calibrated band as mu ± z q s, but the conformal quantile q is computed from scores |y-mu|/s. Lemma 1 proves coverage for mu ± q s. Unless z=1, the guarantee is for the modified score |y-mu|/(z s); for z>1 the band is conservative and still covers, but for z<1 it does not. The manuscript never states z, never restricts z≥1, and the code is promised only after acceptance. So the headline coverage claims are not reproducible from the text, and the width comparison against RCQNO in Table 8 is confounded by an unknown multiplicative factor. This is easy to fix: state z=1, or divide calibration scores by z and rerun. But as written it needs to be fixed before the guarantee is taken seriously.\n\nSecond, E-IV says the authors trained 20 randomized-prior copies and selected the best 10. That post hoc selection compromises the RP spread as an uncertainty heuristic; it should be justified or removed. Third, all coverage tables come from a single 100-sample test set with no error bars. With 100 samples, per-location coverage has a binomial standard error around 2%, so exact counts of '0 failures' are noisier than they look. Fourth, the GP super-resolution step has no coverage guarantee and should be described as an empirical extension, not as part of the conformal argument.\n\nThe Lemma 1 proof has a small order-statistic indexing slip (Eq. 13 should be an inequality or phrased with ranks), but the standard conformal result is invoked correctly in substance. That is minor.\n\nWho is this for? Practitioners doing UQ for neural operators, especially spiking/edge models. Conformal theorists will not learn much. The method is likely to work, and the paper deserves a serious referee, but I would send it back for major revision: specify z, release code, clean up E-IV, and add error bars or repeated splits. My verdict is conditional, leaning positive.","headline":"A useful, mostly sound conformal-wrapper paper for spiking neural operators whose headline coverage claim is not verifiable as written because the width multiplier z is never specified.","tokens_in":20781,"tokens_out":3435,"would_cite":false,"duration_ms":34594,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G15","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that randomized-prior ensembles plus split conformal calibration yield per-location 95% prediction bands for wavelet and spiking neural operators across four PDE benchmarks, missing at only one grid location in the…","keywords":["conformal prediction","uncertainty quantification","neural operator","spiking neural network","wavelet neural operator","randomized prior","operator learning","super-resolution"],"falsifier":"Recompute per-location coverage with $z$ set explicitly to 1 in Algorithm 1, using the same trained RP operators, calibration sets, and test sets; if Examples I-III no longer achieve at least 95% coverage at all grid locations, the paper's central claim depends on the unspecified multiplier rather than on randomized priors plus split conformal calibration.","tokens_in":19792,"feed_emoji":"⚡","tokens_out":8587,"duration_ms":77523,"temperature":0.7,"pith_summary":"The paper introduces a distribution-free uncertainty-quantification wrapper, the Conformalized Randomized Prior Operator (CRP-O), that attaches calibrated prediction bands to wavelet and spiking neural operators. Randomized-prior ensembles provide a per-location mean and standard deviation; split conformal prediction then rescales this spread by a location-dependent quantile so that the bands meet a target coverage of 95% in the experiments. Across four PDE benchmarks (Burgers, Darcy on rectangular and triangular domains, and seismic Helmholtz wave propagation), the conformalized operators CRP-WNO and CRP-VSWNO achieve the target coverage at every grid location except one, whereas quantile-based baselines undercover at many locations. A Gaussian-process extension transfers the calibrated quantile from the training grid to finer super-resolution grids without retraining. The paper argues that a good initial uncertainty estimate is what lets conformal calibration succeed with a meager calibration set.","feed_headline":"Conformal wrapper gives spiking neural operators 95% coverage","feed_subtitle":"Burgers, Darcy, and seismic benchmarks mostly hit the 95% per-location target, unlike quantile baselines.","key_machinery":"The load-bearing object is the randomized-prior operator ensemble: $n_c$ copies of the base operator (vanilla WNO or VSWNO) are trained on the same data with different initializations, each augmented by a frozen prior network whose parameters enter the loss as $\\lambda L_{\\mathrm{prior}}(\\phi)$. The ensemble spread supplies a heuristic uncertainty $s(u_t)$ alongside the mean $\\mu(u_t)$; the normalized residual $|y-\\mu|/s$ is the conformal score. Split conformal prediction turns that score into a per-location multiplier $q$ (one quantile per element of the output grid), and the final band is $\\mu \\pm z q s$, with $z$ a user-set width parameter that the paper never assigns a value. For super-resolution, a Gaussian process regression model treats $q$ as a function of grid location and predicts it on the finer grid, so no retraining or recalibration is needed. VSWNO is the spiking variant of the wavelet neural operator, replacing continuous activations with variable spiking neurons (VSNs) that emit sparse, event-driven outputs; its uncertainty is the paper's main target.","core_discovery":"On its own terms, the paper's central claim is that randomized-prior (RP) operators give a sufficiently informative initial uncertainty estimate that split conformal calibration can turn into per-location calibrated bands: the final band is $C_p = [\\mu - z q s, \\mu + z q s]$, where $\\mu$ and $s$ are the ensemble mean and standard deviation of $n_c$ copies of the base operator, and $q$ is computed element-wise as the $\\lceil(1-\\alpha)(n+1)\\rceil/n$ quantile of the calibration scores $|y-\\mu|/s$. In Examples I-III (Burgers, Darcy rectangular, Darcy triangular), both CRP-WNO and CRP-VSWNO achieve $\\geq 95\\%$ coverage at every point of the solution grid, and in Example IV (Helmholtz) they miss at a single grid location in one frequency component. The same per-location performance is not obtained by uncalibrated RP operators, by quantile-trained WNO (Q-WNO), or by conformalized quantile WNO (CQ-WNO), which reach about 95% only on average. Against the three-way-split risk-controlling quantile neural operator (RCQNO), CRP-WNO produces tighter intervals and lower NMSE with fewer calibration samples. The paper additionally claims that mapping $q$ from the training grid to a finer grid with a Gaussian process preserves coverage in zero-shot super-resolution, demonstrated in the first two examples.","pith_inferences":["The value of the width multiplier $z$ is never reported; if $z$ is not fixed in advance, the conformal guarantee in Lemma 1 applies to $\\mu \\pm q s$, not $\\mu \\pm z q s$, so the reported coverage could silently depend on an unexamined widening of the bands, and a clean test would report coverage with $z=1$.","Element-wise calibration over a large solution grid raises a multiple-testing concern: with more than seven thousand locations, a few failures would be expected by chance even under a valid method, so the paper's all-locations pass is stronger than the formal guarantee requires.","The Gaussian-process mapping assumes the conformal quantile $q(x)$ varies smoothly over the spatial domain; near sharp features such as the Burgers shock or across velocity discontinuities in the Helmholtz field, this smoothness may fail, which is testable by checking super-resolution coverage near discontinuities.","The choice of the prior network (a smaller WNO inside RP-VSWNO) is an unexamined degree of freedom; ablating the prior's architecture and the weight $\\lambda$ would reveal how much of the benefit comes from the prior rather than from ensembling alone."],"forward_implications":["CRP-WNO and CRP-VSWNO achieve at least 95% per-location coverage on Burgers (1024 of 1024 locations), Darcy rectangular (7225 of 7225), and Darcy triangular (1102 of 1102), and at all but one of 4900 locations in the Helmholtz example.","Quantile-based baselines, though near 95% on average, undercover at hundreds to thousands of locations, supporting the paper's hypothesis that the initial uncertainty heuristic, not calibration alone, determines per-location performance.","The Gaussian-process extension gives zero-shot super-resolution coverage: bands calibrated at 1024 points remain valid at 2048 points in Example I, and bands calibrated on an 85 by 85 grid remain valid on a 141 by 141 grid in Example II.","CRP-WNO gives tighter calibrated intervals and lower NMSE than RCQNO on the Helmholtz problem while using 150 calibration samples instead of 500.","Only a two-way train/calibration split is needed, unlike RCQNO's three-way split, so the method remains usable when data are limited."],"supporting_citations":[{"why":"It supplies the randomized-prior ensemble training scheme (frozen prior network plus data loss) that generates the initial uncertainty estimates.","marker":"[37]"},{"why":"It states the split conformal prediction guarantee and quantile construction that the paper's Lemma 1 relies on.","marker":"[29]"},{"why":"It introduces the variable spiking wavelet neural operator (VSWNO), the spiking base architecture whose uncertainty CRP-VSWNO quantifies.","marker":"[27]"},{"why":"It introduces the wavelet neural operator (WNO), the base architecture used by CRP-WNO and as the prior network inside RP-VSWNO.","marker":"[23]"},{"why":"It presents the conformalized quantile DeepONet baseline and the quantile-regression training that Q-WNO and CQ-WNO are built from and compared against.","marker":"[31]"},{"why":"It defines RCQNO, the risk-controlling quantile neural operator baseline compared in Example IV, and motivates the paper's two-way split design.","marker":"[32]"},{"why":"It provides the Burgers and rectangular-Darcy benchmark datasets used in Examples I and II.","marker":"[22]"},{"why":"It supplies the Gaussian process regression machinery used to map the conformal parameter $q$ onto the super-resolution grid.","marker":"[40]"},{"why":"It provides the triangular-domain Darcy benchmark dataset used in Example III.","marker":"[46]"}],"fun_headline_variants":["Distribution-free uncertainty for spiking neural operators","Conformalized spiking operators achieve 95% coverage","Zero-shot super-resolution UQ for spiking neural operators","Randomized priors plus conformal prediction calibrate operator UQ"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the multiplier $z$ in the band $\\mu \\pm z q s$ is fixed in advance; the paper never gives a value for $z$, and the conformal guarantee covers $\\mu \\pm q s$, so an unexamined choice of $z$ could produce the reported coverage.","fun_headline_variants_meta":{"raw":{"variants":["Distribution-free uncertainty for spiking neural operators","Conformalized spiking operators achieve 95% coverage","Zero-shot super-resolution UQ for spiking neural operators","Randomized priors plus conformal prediction calibrate operator UQ"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000327,"raw_usage":{"total_tokens":1903,"prompt_tokens":1094,"completion_tokens":809,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":710,"completion_tokens_details":{"reasoning_tokens":743}},"tokens_in":710,"tokens_out":809,"duration_ms":8199,"temperature":1.0,"reasoning_tokens":743,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:06:12.262035+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute per-location coverage with $z$ set explicitly to 1 in Algorithm 1, using the same trained RP operators, calibration sets, and test sets; if Examples I-III no longer achieve at least 95% coverage at all grid locations, the paper's central claim depends on the unspecified multiplier rather than on randomized priors plus split conformal calibration.","supporting_citations":[],"review_version":1}