{"id":"cc41c62c-31d6-436b-bd17-4c38519b7a88","arxiv_id":"2412.16738","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"KKANs, a two-block KART-based architecture with MLP inner functions and basis-function outer functions, universally approximate continuous functions and empirically outperform MLP and cKAN baselines in regression, PINN, and operator-learning benchmarks.","lead":"This paper introduces KKANs, a two-block neural network that uses MLPs inside and basis functions outside, inspired by the Kolmogorov-Arnold representation theorem. The authors show the architecture matches or beats standard MLPs and KANs on function fitting, physics-informed PDE solving, and operator learning tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The empirical 'outperform' claims rely on uncontrolled comparisons: in Table 6 only KKAN receives the WNadResNet enhancement, and in Table 5 the KAN baselines use Dirichlet boundary conditions while KKAN uses periodic hard constraints; the universal approximation theorem is not the weak point.","rationale":"The reader's CONDITIONAL verdict is reasonable, but the specific weakest assumption stated is not quite right: Table 5 lists the MLP baselines as Periodic, not Dirichlet, so the boundary-condition mismatch is between KKAN and the KAN baselines rather than between KKAN and the MLP baselines. The broader theme, that some headline comparisons are confounded by unequal problem or architecture treatment, is correct and load-bearing. I find a stronger instance in Section 4.3: the operator-learning comparisons give KKAN the WNadResNet enhancement while withholding it from DeepONet and DeepOcKAN, and the resulting advantage over QR-DeepONet is numerically tiny. The universal approximation theorem is sound as a generic denseness statement, though the phrase 'regardless of basis function selection' is overbroad if taken literally; it should be read as 'for any basis family whose finite linear spans are dense in C.' Since the internal PIML comparison (Table 4) is fair and shows KKAN competitive, the central architectural idea is not invalidated, but the empirical superiority claims need deconfounding and statistics. The proposed concrete test would settle whether the operator-learning advantage is architectural or due to the enhancement, and whether the PIML comparison to KAN variants survives a matched boundary-condition setting. Overall verdict remains CONDITIONAL; no change is needed.","tokens_in":37991,"tokens_out":20957,"duration_ms":194923,"concrete_test":"Re-run the Burgers operator-learning benchmark from Table 6, with WNadResNet removed from the KKAN models and, separately, with the same enhancement added to the MLP and cKAN baselines, using identical optimizer schedules and five random seeds per configuration. Report mean and standard deviation of relative L2 error; if QR-DeepOKKAN does not beat QR-DeepONet by a margin larger than the seed noise, the claim that KKAN outperforms MLPs in operator learning should be weakened. In parallel, retrain the KAN baselines in Table 5 with the same periodic hard-constraint embedding used for KKAN; if their errors drop below KKAN's 2.56e-5, the PIML comparison claim also needs revision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The theoretical core is not where the argument is most vulnerable. Theorem 1 and its Appendix A proof are a correct, standard denseness argument: if the inner and outer ansatz classes are dense in C(I_z), the two-block superposition is dense in C(E^d). No circularity or internal inconsistency appears there. The load-bearing weakness is in the empirical support for the headline claim that KKANs outperform MLPs and KANs in operator learning and PIML. In Table 6 (Burgers operator learning), the KKAN models are built with the weight-normalized adaptive ResNet enhancement (WNadResNet, Appendix C.2), while the MLP and cKAN baselines do not receive this or an equivalent enhancement. The QR-DeepOKKAN advantage over QR-DeepONet is only 2.66e-2 versus 2.72e-2, so the claimed superiority could easily be an artifact of the enhancement rather than the KKAN architecture. No error bars or repeated-seed statistics are reported for these runs, so the margin is not known to be significant. In Table 5 (Allen-Cahn PIML), KKAN enforces periodic boundary conditions as a hard architectural constraint, whereas the listed KAN baselines are trained with Dirichlet boundary conditions; because the Allen-Cahn problem is periodic, those baselines are solving a harder, mismatched problem, and the comparison is not architecture-only. Note the reader's summary says MLP baselines used Dirichlet conditions, but Table 5 shows the MLP rows are Periodic; the Dirichlet mismatch is in the KAN rows. The internal PIML comparisons in Table 4, where all models share the same boundary treatment, are fairer and provide partial support, but the operator-learning comparison and the SOTA KAN comparison both need deconfounding before the central empirical claim is accepted.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes Kurkova-Kolmogorov-Arnold Networks (KKANs), a two-block architecture in which MLP-based inner functions expand each input dimension and outer functions are linear combinations of basis functions, following Kurkova's approximate version of the Kolmogorov-Arnold representation theorem. It proves a universal approximation theorem for such two-block superpositions under a denseness assumption on the inner and outer ansatz spaces, extends the architecture to physics-informed machine learning and DeepONet/QR-DeepONet operator learning, and proposes a self-scaled residual-based attention (ssRBA) method. The paper benchmarks KKANs against MLPs and cKANs on discontinuous and smooth function approximation, the Allen-Cahn equation, and Burgers operator learning, and analyzes learning dynamics through signal-to-noise ratio and geometric complexity, identifying fitting, transition, and diffusion stages.","tokens_in":38347,"tokens_out":6866,"duration_ms":61520,"significance":"If the empirical claims held, KKAN would be an attractive drop-in architecture for scientific machine learning: it is simple, follows the KART structure more closely than stacked KANs, and the stated universal approximation result is proved under a clean denseness assumption. The strength of the paper is its theoretical core: Appendix A gives a correct and standard denseness argument, and the appendices contain substantial implementation detail, including hyperparameter tables and a concrete pseudo-code for ssRBA. However, several load-bearing empirical comparisons are confounded by unequal enhancements or boundary-condition treatments, and most benchmark tables report single runs without error bars. The universal approximation claim is also stated more broadly than the theorem supports. These issues make the headline 'KKANs outperform MLPs and KANs' claims currently under-supported, although they are plausibly fixable with additional experiments and careful qualification.","major_comments":[{"comment":"The paper claims universal approximation 'regardless of the choice of basis function' and 'for a general class of functions.' Theorem 1 (Eq. (7)) is correct only under the stated assumption that the ansatz classes A_M^g and A_M^psi are dense in C(I_g) and C(I_psi). That assumption is not satisfied by the fixed finite-dimensional implementations in Appendix B.3.2: for example, the RBF outer block uses D centers fixed on a grid with fixed width sigma, so its span is a finite-dimensional subspace and is not dense in C(I_g) for any fixed D; the same holds for a fixed polynomial degree in the Chebyshev, Legendre, or sine bases. The 'regardless of basis function' statement therefore needs to be qualified, for instance by requiring a family of bases that becomes dense as D grows, or by proving density for each implemented basis family with appropriate scaling of centers and widths.","section":"Abstract; Theorem 1, Section 2.2; Appendix B.3.2"},{"comment":"In the Burgers operator-learning comparison, only the KKAN rows are listed with the architecture enhancement WNadResNet; the DeepONet and DeepOcKAN baselines do not receive this or an equivalent enhancement. The resulting advantage of QR-DeepOKKAN over QR-DeepONet is small (2.66e-2 versus 2.72e-2), and no repeated-seed statistics are reported. The claim that KKANs outperform MLPs in operator learning is therefore not established as an architecture effect. Please either apply the same enhancement to the baselines or ablate WNadResNet from KKAN, and report mean and standard deviation over multiple seeds.","section":"Table 6; Appendix F.3.1"},{"comment":"The external KAN baselines in Table 5 (AcNet, KAN [52] implementation, and KAN [41] implementation) are trained with Dirichlet boundary conditions, while KKAN+ssRBA enforces periodicity through a hard architectural constraint. For the Allen-Cahn equation, whose exact solution is periodic, this makes the KAN comparison mismatched and harder for the baselines. Consequently, the statement that KKAN 'outperforms all KAN-based formulations' is not an architecture-only conclusion. Please either implement KAN baselines with the same periodic embedding or explicitly restrict the claim to the periodic-constraint setup and list the boundary-condition mismatch as a limitation.","section":"Table 5; Section 4.2.1"},{"comment":"The paper asserts a 'strong correlation' between geometric complexity and SNR and identifies three 'universal' learning stages across all architectures and tasks. The evidence is qualitative visual inspection of a small set of problems, without correlation coefficients, repeated seeds, or a statistical test, and the word 'universal' overreaches beyond the four benchmark settings. Please either quantify the claimed correlation and the staging across multiple runs or soften the claims to observations on the tested problems.","section":"Section 5; Figures 11-14"}],"minor_comments":[{"comment":"The abstract says KKANs outperform 'the original KANs,' but the benchmarks use cKANs (Chebyshev KANs) rather than the original B-spline KAN; the original KAN is not included in Tables 2-6. Please adjust the wording or add the original KAN baseline.","section":"Abstract; Section 4"},{"comment":"These tables report point estimates from presumably single runs, so the observed margins (for example, 5.86e-3 versus 1.26e-2 in Table 2, or 3.07e-5 versus 3.52e-5 in Table 4c) cannot be assessed for statistical significance. Adding repeated-seed statistics would materially strengthen the empirical claims.","section":"Tables 2, 3, 4, and 6"},{"comment":"The RBA update rule contains a typographical error: 'lambda_{alpha,i} <- gamma_k lambda_{alpha,i} + eta ||r_{alpha,i}|/max_j|r_{alpha,j}|' should read 'eta |r_{alpha,i}| / max_j |r_{alpha,j}|'.","section":"Algorithm 1, line 11"},{"comment":"The comparison with the errors reported in [41] is indirect because those models predict only the final time u(x,1), while the present models predict the full solution history u(x,t). The text notes this, but the juxtaposition of numbers in the same table may still mislead; a separate comparison on the final time would be clearer.","section":"Table 6 footnote"},{"comment":"No code or data availability statement is provided, which limits reproducibility of the benchmarks. Please include a statement or repository link.","section":"Overall"}],"recommendation":"major_revision","confidential_remarks":"The self-citation pattern is noticeable, especially for the SNR/stage framework and RBA, but the main theorem is grounded in external KART and Cybenko results, and I do not see a circularity problem. The main issue is fairness of the empirical comparisons: Table 6 gives KKAN an enhancement not given to baselines, and Table 5 compares periodic hard constraints against Dirichlet-constrained KAN variants. These are fixable within the scope of the manuscript, but they need to be addressed before the performance claims can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short of it: KKAN is a sensible two-block architecture—MLP inner, basis-function outer—that tracks KART closely and comes with a correct, if standard, universal approximation theorem. The empirical section is where the claim that 'KKANs outperform MLPs and KANs' gets shaky. In Table 6, the winning KKAN operator model is built with WNadResNet while the MLP and cKAN baselines are not; in Table 5, the KAN baselines are trained with Dirichlet boundary conditions on a periodic problem, so they are solving a harder mismatch than the KKAN rows. The reader's take pointed at MLP baselines for the boundary mismatch, but the stress-test is right: the MLP rows are periodic; the KAN rows are Dirichlet. Either way, the comparisons are not architecture-only.\n\nWhat is genuinely new: the two-block design, the use of a trainable Chebyshev expansion inside the inner MLP (ebMLP), and the extension of the IB stage analysis to function approximation and operator learning. The ssRBA weighting scheme is a reasonable incremental extension of RBA. The universal approximation theorem is correct under the stated density assumption, but the 'regardless of basis function' phrasing is overbroad—fixed-width RBFs with centers on a grid do not form a dense subset of C(I). So the theorem is fine, the marketing around it is not.\n\nThe learning-dynamics analysis is qualitative. The three stages are identified by eye, and the 'strong correlation' between geometric complexity and SNR is not quantified. That is a minor concern; this kind of analysis is often qualitative.\n\nMissing error bars are a real weakness. Most function-approximation and operator tables report single runs, so we don't know if the 2.66e-2 vs 2.72e-2 difference in Table 6 is noise.\n\nAll that said, the architecture is promising, the theory is sound as stated, and the range of benchmarks is useful. This deserves a serious referee, but the authors should be asked to (i) deconfound Table 6 by either giving all models the same enhancements or none, (ii) retrain or re-state the KAN baselines in Table 5 under periodic conditions, and (iii) report repeated-seed statistics for the key claims.\n\nI'd bring it to reading group to discuss the comparison-fairness issue, and I'd cite the architecture (not the empirical claims) in my own work.\n\nRecommendation: accept for peer review with major revision.","headline":"A promising KART-inspired architecture with correct but standard theory, undermined by deconfounded empirical comparisons that leave the headline 'outperforms' claim unproven.","tokens_in":38940,"tokens_out":3334,"would_cite":true,"duration_ms":27934,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","41A30","65D15"],"pacs":[],"model":"deepseek-v4-flash","headline":"KKAN is a universal approximator, regardless of basis choice, and outperforms MLPs and KANs in function approximation and operator learning while matching optimized MLPs in PIML.","keywords":["Kolmogorov-Arnold representation theorem","KKAN","universal approximation","physics-informed machine learning","neural operators","information bottleneck theory","geometric complexity","self-scaled residual-based attention"],"falsifier":"A single controlled experiment would settle the empirical claim: train KKAN and MLP on Allen-Cahn with identical periodic hard-constraint embeddings, parameter budgets, optimizers, and collocation points, and check whether the MLP reaches or beats KKAN's reported relative L2 error of about $3\\times10^{-5}$; for the universality claim, the analogous check is to find a continuous function and a finite dense basis family for which KKAN error does not drop below $\\varepsilon$ as the number of features grows.","tokens_in":37762,"feed_emoji":"🧠","tokens_out":7458,"duration_ms":60964,"temperature":0.7,"pith_summary":"KKAN is proposed as a two-block architecture—MLP-based inner functions feeding linear combinations of basis functions as outer functions—that adheres to Kurkova's approximate version of the Kolmogorov-Arnold representation theorem rather than to the deeply nested KAN stack. The paper's central claim is that this structure is a universal approximator for any continuous function on the unit cube, with no restriction on the choice of dense basis family, and that it beats MLPs and Chebyshev-KANs on function regression and operator learning while matching fully optimized MLPs in physics-informed problems. If correct, KKAN is a drop-in architecture that gives practitioners a principled reason to pick basis functions (Chebyshev, Legendre, RBFs, sin-series) without changing the outer training loop. The paper also claims that all these architectures learn through three universal stages—fitting, transition, and diffusion—and that diffusion, when the gradient signal-to-noise ratio is high, is where generalization is won; its self-scaled residual-based attention is designed to keep the SNR high and prolong that stage.","feed_headline":"Two-block KKAN beats KANs and MLPs on SciML benchmarks","feed_subtitle":"A KART-faithful universal approximator, with self-scaled attention that sustains convergence through the diffusion stage.","key_machinery":"The load-bearing construction is the approximate KART set $K^{m,d}_M$: sums of the form $\\sum_{q=0}^m G_q(\\sum_{p=1}^d \\Psi_{p,q}(x_p))$, where $G_q$ and $\\Psi_{p,q}$ are univariate functions chosen from dense subsets $A^g_M$ and $A^\\psi_M$ of $C(I)$. This set is dense in $C([0,1]^d)$ whenever both ansatz classes are dense, so any universal one-dimensional approximator can fill the inner and outer blocks. The KKAN implementation uses an MLP-based inner block (with two trainable Chebyshev embedding layers) and a linear combination of basis functions—Chebyshev, Legendre, sin-series, Chebyshev-grid, or RBFs—as the outer block. The second mechanism is self-scaled residual-based attention, which raises the memory coefficient $\\gamma$ in stages to keep the gradient signal-to-noise ratio high during the late diffusion phase.","core_discovery":"The paper's central claim is that a two-block network built directly from Kurkova's approximate Kolmogorov-Arnold representation is a universal approximator and a practical drop-in architecture. Specifically, for any continuous function $f$ on the unit cube and any $\\varepsilon>0$, there are univariate inner functions $\\Psi_{p,q}$ and outer functions $G_q$ drawn from dense ansatz classes such that $\\|f - \\sum_{q=0}^m G_q(\\sum_{p=1}^d \\Psi_{p,q}(x_p))\\|_\\infty < \\varepsilon$. The theorem does not depend on which dense basis the outer block uses. Empirically, the paper reports that KKANs outperform MLPs and Chebyshev-KANs on smooth and discontinuous function regression, reach high accuracy on the Allen-Cahn PIML benchmark, and beat the same baselines when embedded in DeepONet and QR-DeepONet for the Burgers operator.","pith_inferences":["Beyond the paper: if the density assumption is the only requirement for universality, then any one-dimensional universal approximator—wavelets, splines, RBFs, polynomials—can be dropped into the outer block, making KKAN a modular front-end for spectral-style basis selection.","Beyond the paper: the observed three-stage learning curve and the SNR peak during diffusion suggest a practical early-stopping or learning-rate-scheduling rule based on SNR rather than validation loss; this is testable across architectures.","Beyond the paper: since ssRBA works by progressively raising the memory term $\\gamma$ to sustain SNR, the same mechanism should transfer to plain MLPs and KAN variants; a simple test is to apply ssRBA to the MLP baselines in Table 5 under matched boundary conditions."],"forward_implications":["Because the universal approximation theorem holds for any dense one-dimensional ansatz, KKAN inherits the KART representation without committing to B-splines or Chebyshev polynomials; the same architecture can switch bases by changing the outer-block linear combination.","In function approximation benchmarks, KKAN reaches relative L2 errors of $5.86\\times10^{-3}$ (discontinuous) and $1.74\\times10^{-4}$ (smooth), outperforming MLPs and cKANs at comparable parameter counts and per-iteration cost.","In PIML, KKAN+ssRBA solves Allen-Cahn to $2.28\\times10^{-5}$ relative L2 error, beats the KAN variants compared, and is competitive with optimized MLPs; it also trains faster than the larger MLP variant in the full-batch setting.","In operator learning, KKAN-based DeepONet and QR-DeepONet reach $3.07\\times10^{-2}$ and $2.66\\times10^{-2}$ relative L2 errors on the Burgers operator, the best among compared models, and QR reparameterization helps KKAN while hurting cKAN.","Across all models and tasks, training proceeds through fitting, transition, and diffusion stages; the diffusion stage coincides with high SNR and optimal generalization, and ssRBA prolongs it by maintaining SNR."],"supporting_citations":[{"why":"Supplies the approximate-representation principle that turns the exact Kolmogorov-Arnold theorem into a trainable dense-ansatz network.","marker":"[55, 56]"},{"why":"Introduces KANs and the outer-block linear-combination-of-basis-functions idea that KKAN adopts.","marker":"[23]"},{"why":"Provides the recursive Chebyshev-KAN (cKAN) baseline used throughout the benchmarks.","marker":"[39]"},{"why":"Supplies the sin-series basis function that performs best on the discontinuous benchmark and serves as a KAN alternative baseline.","marker":"[52]"},{"why":"Provides the self-adaptive PINN weighting scheme used as a baseline and the boundary-weight setting for the Allen-Cahn experiments.","marker":"[22]"},{"why":"Defines residual-based attention, the method that ssRBA extends with progressive memory scaling.","marker":"[63]"},{"why":"Supplies the geometric-complexity metric (discrete Dirichlet energy) used to link model simplicity to generalization.","marker":"[61]"},{"why":"Establishes the fitting-transition-diffusion learning-stage framework that the paper extends to function approximation and operator learning.","marker":"[62]"},{"why":"Provides the QR-DeepONet reparameterization and two-stage training procedure that KKAN integrates.","marker":"[107]"},{"why":"Defines the DeepONet operator-learning framework that KKAN extends into DeepOKKAN.","marker":"[3]"}],"fun_headline_variants":["KKAN is a two-block universal approximator that outperforms KANs and MLPs","KKAN reveals universal learning stages: fitting, transition, and diffusion","Self-scaled attention in KKAN sustains high SNR and uniform convergence","KKAN beats KANs and MLPs on regression, PIML, and operator learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The empirical claim that KKAN matches or beats fully optimized MLPs in PIML rests on benchmark comparisons being apples-to-apples; in the Allen-Cahn comparison, KKAN enforces periodic boundary conditions as hard architectural constraints while several MLP baselines were trained with soft Dirichlet conditions, so part of the accuracy gap may come from the problem setup rather than the architecture.","fun_headline_variants_meta":{"raw":{"variants":["KKAN is a two-block universal approximator that outperforms KANs and MLPs","KKAN reveals universal learning stages: fitting, transition, and diffusion","Self-scaled attention in KKAN sustains high SNR and uniform convergence","KKAN beats KANs and MLPs on regression, PIML, and operator learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001394,"raw_usage":{"total_tokens":5655,"prompt_tokens":975,"completion_tokens":4680,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":591,"completion_tokens_details":{"reasoning_tokens":4592}},"tokens_in":591,"tokens_out":4680,"duration_ms":28935,"temperature":1.0,"reasoning_tokens":4592,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:15:16.222325+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A single controlled experiment would settle the empirical claim: train KKAN and MLP on Allen-Cahn with identical periodic hard-constraint embeddings, parameter budgets, optimizers, and collocation points, and check whether the MLP reaches or beats KKAN's reported relative L2 error of about $3\\times10^{-5}$; for the universality claim, the analogous check is to find a continuous function and a finite dense basis family for which KKAN error does not drop below $\\varepsilon$ as the number of features grows.","supporting_citations":[],"review_version":1}