{"id":"5d9f9e67-deeb-4730-9b6b-8d32e378f0e5","arxiv_id":"2504.15240","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"low","formal_verification":"none","parameter_count":4,"one_line_summary":"Conformal prediction applied to ensembles of KANs, FBKANs, and MFKANs yields prediction intervals that empirically hit the target 95% coverage on four synthetic problems.","lead":"The paper wraps Kolmogorov-Arnold Networks in a standard statistical wrapper, conformal prediction, to produce prediction intervals with coverage guarantees. The idea is a direct extension of the same authors' earlier conformalized DeepONet work, and the experiments are all synthetic benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Coverage guarantee and empirical claims are standard and sound, but only one coverage realization per experiment is reported; repeated-seed coverage statistics are needed to assess whether the reported values reflect the guarantee rather than noise.","rationale":"I agree with the reader's overall conditional verdict and with the identification of exchangeability as the formally load-bearing assumption; the theory is standard and correctly invoked, and the experiments respect the i.i.d. assumption. However, I do not think exchangeability is the most decision-relevant weak point, because the paper explicitly states the datasets are i.i.d. and the experiments are synthetic. The more consequential weakness is that the empirical evidence section reports a single realization of the coverage for each configuration. The conformal guarantee is a probabilistic statement, and the reported coverage values are random; the finite-sample variance is large enough that the numerical differences among models and the deviations from 95% are all within noise. The reader also requested repeated-seed coverage statistics, so this is a partial agreement rather than a disagreement: the reader's requested check is essentially the same as mine, but the reader framed the primary concern as exchangeability, whereas I locate the load-bearing issue in the single-realization empirical evidence. The central theoretical claim of the paper is sound, so the verdict should remain conditional rather than being rejected: the authors should add repeated-seed coverage experiments, publish code and data, and soften the 'consistently' language unless the repeated-seed results support it.","tokens_in":14343,"tokens_out":1862,"duration_ms":15481,"concrete_test":"Re-run Experiment 1 (or Experiment 4) with, say, 30 independent seeds, where each seed redraws the train/calibration/test split and retrains the ensembles of KAN and FBKAN models, then record the conformalized coverage for each seed. Report the mean and standard deviation of coverage across seeds for each model. If the across-seed standard deviation is approximately sqrt(alpha(1-alpha)/n_test) ~ 0.7% and the two models' coverage distributions overlap substantially (e.g., both are within a one-sigma band around 95%), then the paper's cross-method comparisons and 'consistently achieves' language are not supported by the current evidence, and the claims should be softened to 'coverage is in line with the guaranteed level in these single runs.' If instead the distributions separate by more than 2 sigma, the comparative claims gain support.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that Conformalized-KANs provide prediction intervals with at least 1-alpha coverage, relying on the standard split-conformal exchangeability theorem. That theorem is applied correctly: the nonconformity scores are computed on a held-out calibration set, the trained ensemble is fixed before calibration, and the reported intervals are of the required form. The exchangeability assumption is satisfied in the experiments because all data are synthetic i.i.d. draws. The theoretical claim is thus internally sound; it is a direct application of established theory. The load-bearing weakness is empirical rather than theoretical: each reported coverage number (e.g., 96.10%, 95.50%, 96.01%, 95.44%) comes from a single test set and a single trained ensemble. Under the conformal guarantee, coverage at fixed alpha is random and, for n_cal=500 and n_test=1000, the standard deviation of the empirical coverage is roughly sqrt(0.95*0.05/1000) ~ 6.9e-3, i.e., about 1.4% at two sigma. The differences across methods and deviations from 95% are therefore all consistent with ordinary sampling noise. The ablation studies vary ensemble size and subdomain count but never report multiple independent realizations of the full data split and training procedure. Consequently, the experiments demonstrate that the method runs and produces plausible intervals, but they do not yet substantiate the stronger claims that Conformalized-KANs 'consistently achieve the target 95% coverage,' that FBKANs 'consistently' produce sharper intervals, or that ensemble KANs systematically underestimate uncertainty. The existence of one realization at 94.85% (FBKAN 2D) already shows finite-sample variation around the nominal level; without repeated-seed statistics the apparent differences between methods in Tables 1-4 cannot be distinguished from noise.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Conformalized-KANs, a UQ framework that combines an ensemble of Kolmogorov-Arnold Networks (KANs) with split conformal prediction. The ensemble mean is used as the point predictor and the ensemble standard deviation as a heuristic uncertainty scale; conformal calibration then yields intervals of the form [mu_M(x) - qhat*sigma_M(x), mu_M(x) + qhat*sigma_M(x)] with a claimed coverage guarantee P(f(x_test) in C_alpha(x_test)) >= 1-alpha. The method is applied to standard KANs and to two KAN variants, FBKANs and MFKANs, in four experiments: a 1-D function, a 2-D function, a multi-fidelity problem, and a wave-equation PDE. The reported empirical coverage values are close to 95%, and ablations study the effect of ensemble size, number of subdomains, and calibration-set size. The theoretical content is a correct application of standard split-conformal theory; the main weaknesses are empirical reporting (single-run coverage statistics) and some algorithmic details in the definition of the nonconformity score.","tokens_in":14760,"tokens_out":5916,"duration_ms":56415,"significance":"If the empirical claims are made rigorous, this would be a useful and practical application of established conformal-prediction theory to a popular class of SciML models. The paper's strength is that it correctly identifies the standard split-conformal theorem and applies it in the right way: the ensemble is trained before calibration, the nonconformity scores are computed on held-out calibration data, and the intervals have the correct conformal form. The application to FBKANs and MFKANs broadens the scope of conformal UQ to recent KAN variants. The coverage guarantee itself is not new, so the contribution is primarily empirical and architectural. The current evidence, however, is weakened by the absence of repeated-seed or multi-split statistics: all reported coverage values are single realizations, and under the nominal 95% level with test sets of size 1000 the sampling noise is roughly +/-1.4 percentage points at two standard deviations, which is comparable to the observed differences between methods.","major_comments":[{"comment":"The nonconformity score is defined as s_j = |f(x_j) - mu_M(x_j)| / sigma(x_j). For the conformal guarantee to hold, this score must be a well-defined, fixed function on all calibration and test points. If the ensemble standard deviation is zero or numerically negligible for any point, the score is undefined or extremely large, which can make the conformal quantile and the resulting intervals unstable. The paper should state a safeguard such as sigma_epsilon(x) = max(sigma_M(x), epsilon) and discuss how this affects the coverage guarantee. In addition, Algorithm 1 uses inconsistent notation: line 4 divides by sigma(x_j) and line 8 uses sigma(xtest), while Section 3.1 defines sigma_M; this should be corrected.","section":"Section 3.2, Algorithm 1"},{"comment":"Each reported coverage value (e.g., 96.10%, 95.50%, 96.01%, 95.44%) comes from a single data split and a single trained ensemble. For a test set of size 1000 and nominal coverage 0.95, the standard error of the empirical coverage is about 0.7 percentage points, so two-standard-deviation intervals span roughly 1.4 percentage points. The differences between methods and the deviations from 95% in Tables 1-4 are therefore all within ordinary sampling noise. The claims that Conformalized-KANs 'consistently achieve the target 95% coverage' and that FBKANs 'consistently outperform KANs' are not substantiated by single-run statistics. Please report coverage means and standard deviations across multiple independent data splits and model initializations.","section":"Section 4, Tables 1-4"},{"comment":"The ablation studies visualize coverage and width curves as functions of ensemble size M and the number of subdomains L, but the paper does not state whether each plotted curve is a single realization or an average over repeated runs. If these are single runs, the same sampling-noise critique applies to the ablation conclusions, including the claim that the conformalized variants 'consistently' maintain coverage near the target. The figure captions and text should clarify the number of independent realizations and include error bars or confidence bands.","section":"Appendix A, Figures 6-8"}],"minor_comments":[{"comment":"There are typographical errors: 'multifideilty' should be 'multi-fidelity' in the abstract, and 'parition' should be 'partition' in Section 2.2.","section":"Abstract and Section 2.2"},{"comment":"The penalty term involving lambda_alpha is garbled in the displayed equation ('+ lambda_alpha alpha^n + w' appears as '+ λααn +w'). Please rewrite the equation so that the exponent on alpha is clear.","section":"Section 2.3, Eq. (3)"},{"comment":"The text says the 1.96-sigma interval gives 'approximately 95% confidence, assuming the predictions follow a normal distribution.' This is only valid for Gaussian predictive distributions, and the paper correctly notes this limitation; however, the point is stated somewhat informally and could be clarified.","section":"Section 3.1"},{"comment":"For the PDE experiment, the calibration and test sets should be drawn from the same distribution over (x,t) for the exchangeability assumption to hold. The paper should state explicitly how the 1200 calibration points and 10000 test points are sampled, especially because the training uses physics-informed losses with initial, boundary, and residual data.","section":"Section 4.5"},{"comment":"The sentence 'Given that the data is exchangeable, the method assures coverage' would be clearer if it specified that the guarantee holds over the randomness in the calibration data for a fixed test point, rather than suggesting a deterministic statement for any test set.","section":"Section 3.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a straightforward application of split conformal prediction to KAN ensembles; the theoretical novelty is minimal, but the practical value is real if the empirical validation is solid. The main barrier is that all coverage results are single-run realizations, so the central empirical claims are currently over-stated relative to the evidence. The authors should also clarify the division-by-sigma issue in the nonconformity score. These are fixable within the manuscript's scope. The paper also relies heavily on the authors' own prior conformalized-DeepONet work; the novelty framing should be careful not to overstate the methodological advance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a competent, clearly-written application of split conformal prediction to KAN ensembles and their FBKAN/MFKAN variants. The coverage guarantee is standard theory, correctly applied, and the experiments show the expected result: conformalized intervals land near 95% coverage while raw ensemble intervals often undercover. What is genuinely new is narrow—the specific wrapper plus demonstrations on these architectures. That is fine for a methods-applied paper, but the empirical claims go slightly past what the evidence supports.\n\nThe theory section is sound. The authors fix the trained ensemble, compute nonconformity scores on a held-out calibration set, take the empirical quantile, and build intervals. Exchangeability is satisfied in their synthetic i.i.d. experiments, and they correctly cite the standard guarantees. The ablations on ensemble size, subdomain count, and calibration-set size are a genuine plus. They are also honest that the non-conformal 1.96-sigma intervals assume normality and do not guarantee coverage.\n\nThe main soft spot is the one the stress-test flags: every coverage number in Tables 1–4 comes from a single test set and a single trained ensemble. For n_test=1000, the two-sigma sampling spread around 95% is about 1.4 percentage points, so 96.10%, 95.50%, 96.01%, and 95.44% are all indistinguishable from 95%, and the apparent differences between KAN and FBKAN widths are not backed by any repeat-seed variance. The text says conformalized models “consistently achieve” target coverage; strictly, they produce one realization consistent with it. That is fixable: report mean and standard deviation across 10–20 seeds, or show that deviations fall within the binomial band. There are also minor weaknesses: the nonconformity score divides by sigma(x_j) with no handling for zero ensemble variance; there is no benchmark against existing Bayesian KAN UQ; and no code or data are included.\n\nWho is this for? Practitioners working with KANs who want a plug-in UQ layer and are comfortable with split conformal prediction. It is not a theoretical advance and does not change how conformal prediction or KANs work, but it is a legitimate engineering contribution.\n\nRecommendation: I would send it to peer review with a request for repeated-seed coverage statistics and code/data release. It deserves referee time; it is neither a desk reject nor a breakthrough.","headline":"Competent application of split conformal prediction to KAN ensembles; theory is sound, but the empirical claims need repeated-seed statistics to support them.","tokens_in":15270,"tokens_out":2158,"would_cite":false,"duration_ms":19513,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","62G15"],"pacs":[],"model":"deepseek-v4-flash","headline":"Conformal prediction gives Kolmogorov-Arnold network ensembles prediction intervals with a distribution-free coverage guarantee.","keywords":["conformal prediction","Kolmogorov-Arnold networks","uncertainty quantification","ensemble methods","coverage guarantees","scientific machine learning","multi-fidelity models","domain decomposition"],"falsifier":"Rerun the conformalized ensemble on exchangeable calibration and test data with a different seed or base model and check whether the empirical coverage on a large test set stays between $1-\\alpha$ and $1-\\alpha+1/(n+1)$. The claim would be falsified if coverage fell systematically below $1-\\alpha$; the paper's own experiments are this check for its four test problems.","tokens_in":14141,"feed_emoji":"🎯","tokens_out":8674,"duration_ms":71570,"temperature":0.7,"pith_summary":"The paper claims that wrapping a KAN ensemble in split conformal prediction turns the ensemble spread into prediction intervals with a distribution-free coverage guarantee: for any test input, the true output lands in the interval with probability at least $1-\\alpha$, provided the calibration data are exchangeable with the test data. The method is demonstrated on a 1-D function, a 2-D function, a multi-fidelity problem, and a wave-equation PDE, where the conformalized intervals reach roughly 95% coverage while raw ensemble intervals often under-cover. A secondary finding is that domain-decomposed FBKANs produce considerably narrower conformal intervals than standard KANs at the same coverage. This matters for scientific machine learning because it attaches a rigorous, user-chosen reliability statement to KAN predictions in data-sparse settings.","feed_headline":"KAN ensembles get guaranteed-coverage prediction intervals","feed_subtitle":"Calibrating ensemble spread on exchangeable data hits the target 95% coverage across 1D, 2D, multi-fidelity, and PDE tests.","key_machinery":"The load-bearing object is split conformal prediction applied to a KAN ensemble. Conformal prediction is a distribution-free wrapper: it uses a held-out calibration set to estimate how badly the model's own uncertainty measure misses, then widens the ensemble interval by the calibration quantile. The ensemble standard deviation serves as the denominator of the nonconformity score, so the final interval has the shape $\\mu_M \\pm \\hat{q}_\\alpha \\sigma_M$, and the conformal theorem converts the empirical quantile into the coverage guarantee. For FBKANs the ensemble is built from domain-decomposed sub-KANs stitched by a partition of unity; for MFKANs a frozen low-fidelity KAN feeds an ensemble of linear and nonlinear high-fidelity KANs.","core_discovery":"The central claim is that ensembles of KAN-based models, including FBKANs and MFKANs, can be conformalized to yield calibrated prediction intervals with guaranteed coverage without assuming a particular error distribution. Concretely, after training $M$ models from different random initializations, the ensemble mean $\\mu_M(x)$ and standard deviation $\\sigma_M(x)$ define a heuristic uncertainty. The paper's procedure computes nonconformity scores $s_j = |f(x_j)-\\mu_M(x_j)|/\\sigma(x_j)$ on held-out calibration data and rescales the ensemble spread by the empirical quantile $\\hat{q}_\\alpha = \\lceil(n+1)(1-\\alpha)\\rceil/n$ of these scores, producing $C_\\alpha(x) = [\\mu_M(x)-\\hat{q}_\\alpha\\sigma_M(x),\\ \\mu_M(x)+\\hat{q}_\\alpha\\sigma_M(x)]$. The paper states that exchangeability of calibration and test data gives $P(f(x_{\\mathrm{test}}) \\in C_\\alpha(x_{\\mathrm{test}})) \\ge 1-\\alpha$, and its four experiments report empirical coverage near the target 95%.","pith_inferences":["The wrapper should transfer to any KAN variant that can be ensembled, such as physics-informed or wavelet KANs, as long as the ensemble standard deviation remains a meaningful uncertainty heuristic; a direct test would be to rerun the same conformal procedure with a different base architecture.","The guarantee is marginal over exchangeability, not conditional on the input; users who need per-input reliability in heterogeneous domains would have to calibrate locally or conditionally, which the paper does not develop.","A natural extension is local or adaptive calibration: binning calibration points by input region and computing region-specific quantiles could shrink the widest intervals while preserving coverage, especially where FBKAN subdomains localize error.","For multi-fidelity problems, conformalized MFKAN interval width could serve as an acquisition signal for choosing where to collect new high-fidelity data, since the intervals reflect where the surrogate is least certain."],"forward_implications":["At miscoverage level $\\alpha = 0.05$, conformalized KAN, FBKAN, and MFKAN intervals reach roughly 95% empirical coverage across all four test problems, correcting the under-coverage of raw ensemble intervals.","With $n$ exchangeable calibration points, coverage is at least $1-\\alpha$ and at most $1-\\alpha + 1/(n+1)$, independent of the data distribution.","Because the ensemble standard deviation is the only model-specific ingredient, the same wrapper can be applied to other KAN variants beyond FBKANs and MFKANs.","Domain decomposition improves interval sharpness: conformalized FBKANs give narrower intervals than standard conformalized KANs at the same coverage.","Larger calibration sets reduce coverage fluctuations, while ensemble size mainly influences interval width rather than the coverage guarantee."],"supporting_citations":[{"why":"Defines KANs and the spline/grid construction that all experiments use as the base model.","marker":"[3]"},{"why":"Introduces FBKANs, the domain-decomposed KAN variant the paper wraps with conformal prediction.","marker":"[28]"},{"why":"Introduces MFKANs, the multi-fidelity KAN variant the paper wraps.","marker":"[29]"},{"why":"Provides the conformal prediction theorem that yields the coverage guarantee for quantile-based intervals.","marker":"[40]"},{"why":"Supplies the nonconformity-score construction that the paper adapts to ensemble standard deviations.","marker":"[41]"},{"why":"Gives the split-conformal framework and the coverage bounds used in Section 3.2.","marker":"[42]"},{"why":"Demonstrates conformal prediction for deep operator networks, the methodological template the paper follows.","marker":"[43]"},{"why":"States the Kolmogorov-Arnold representation theorem that motivates the KAN architecture.","marker":"[5]"}],"fun_headline_variants":["Conformalized KANs guarantee prediction intervals","KAN ensembles get calibrated prediction intervals","Guaranteed coverage for Kolmogorov-Arnold Networks","Conformal prediction for KANs: reliable uncertainty","KANs with conformalized ensembles: coverage guaranteed"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The coverage guarantee rests entirely on the calibration data being exchangeable with the test data; the paper assumes i.i.d. draws, so if the test inputs come from a different distribution, are correlated with the calibration set, or arrive in a non-exchangeable order, the stated coverage can fail.","fun_headline_variants_meta":{"raw":{"variants":["Conformalized KANs guarantee prediction intervals","KAN ensembles get calibrated prediction intervals","Guaranteed coverage for Kolmogorov-Arnold Networks","Conformal prediction for KANs: reliable uncertainty","KANs with conformalized ensembles: coverage guaranteed"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00018,"raw_usage":{"total_tokens":1316,"prompt_tokens":971,"completion_tokens":345,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":587,"completion_tokens_details":{"reasoning_tokens":270}},"tokens_in":587,"tokens_out":345,"duration_ms":3524,"temperature":1.0,"reasoning_tokens":270,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:28:55.345583+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the conformalized ensemble on exchangeable calibration and test data with a different seed or base model and check whether the empirical coverage on a large test set stays between $1-\\alpha$ and $1-\\alpha+1/(n+1)$. The claim would be falsified if coverage fell systematically below $1-\\alpha$; the paper's own experiments are this check for its four test problems.","supporting_citations":[{"cited_title":"Springer, 2005","cited_arxiv_id":null,"evidence_quote":"Provides the conformal prediction theorem that yields the coverage guarantee for quantile-based intervals."},{"cited_title":"Conformalized-deeponet: A distribution- free framework for uncertainty quantification in deep operator networks","cited_arxiv_id":null,"evidence_quote":"Demonstrates conformal prediction for deep operator networks, the methodological template the paper follows."},{"cited_title":"On the representations of continuous functions of many variables by superposition of continuous functions of one variable and addition","cited_arxiv_id":null,"evidence_quote":"States the Kolmogorov-Arnold representation theorem that motivates the KAN architecture."}],"review_version":1}