{"id":"d00b5a8e-dcca-4e01-8959-78a4f773097b","arxiv_id":"2412.13571","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"PowerMLP is a ReLU-power MLP that trains about 40x faster than KAN in the reported benchmarks and often beats it, but the main proof that KANs are contained in PowerMLPs at the same depth is flawed.","lead":"PowerMLP replaces KAN's slow recursive splines with power-of-ReLU activations, runs roughly 40 times faster in its tests, and often matches or beats KAN accuracy. The paper's headline proof that the two architectures are equally expressive, however, has a gap at the same-depth inclusion step.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 2's same-depth inclusion K⊂P is false for the final layer: PowerMLP's last layer is affine (Eq. 6), so a depth-1 nonlinear KAN (e.g., a spline computing x^2) cannot be represented by any depth-1 PowerMLP.","rationale":"The reader's weakest assumption identifies the same flaw, and I agree. The central claim of the paper—that PowerMLP is a drop-in, theoretically justified replacement for KAN—depends on Corollaries 3 and 7, both built on Theorem 2. If Theorem 2's same-depth inclusion fails, the paper does not establish that PowerMLP has at least KAN's expressive power at the same depth; it only suggests that a depth-augmented embedding might be possible. The d=1 counterexample is direct and does not rely on subtle parameter counting; it contradicts the statement of Theorem 2 as written. The reverse inclusion (Theorem 6) appears more defensible, and the empirical training-time advantage is plausible, so the paper contains a useful empirical kernel. But the advertised theoretical equivalence is not sound. Repairing it would require either allowing the PowerMLP output layer to be nonlinear (which weakens the MLP-like speed story) or restating the inclusion as K_d ⊂ P_{d+1}, which weakens the same-depth, same-parameter corollaries. The reader's REJECT verdict is therefore appropriate and does not need to be changed.","tokens_in":19292,"tokens_out":16570,"duration_ms":148452,"concrete_test":"Set d=1, n=m=1, k=3, G=3. Define a KAN layer with u=0 and spline coefficients representing φ(x)=x^2 on the grid interval [-1,1]; this is exactly a K_1 network. By Eq. (6), every depth-1 PowerMLP is ωx+γ, an affine map, so P_1 contains no nonlinear function. Check whether the authors can exhibit a depth-1 PowerMLP equal to x^2; if they cannot, Theorem 2 is false. A more general check: for any d, let the last KAN layer be a nonlinear spline and attempt to verify the claimed same-depth inclusion; the construction requires a (d+1)-th linear layer to absorb the final β.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central theoretical guarantee rests on Theorem 2, which claims K_{d,w,k,G,p} ⊂ P_{d,w^2(G+k),k,p}. The proof represents a KAN layer as αb(x)+βσ_k(ωx+γ) (Eq. 11) and then states that β can be absorbed into the next layer's weight. This absorption is only legitimate when there is a next layer and that next layer is linear. In a KAN, every layer is nonlinear; in a PowerMLP, the last layer is explicitly affine (Eq. 6). Hence the β of the last KAN layer cannot be absorbed without adding a layer, changing the depth count. The failure is not merely technical: for d=1, a KAN is a single spline layer, while a depth-1 PowerMLP is affine, so any nonlinear spline (for example, x^2 on [-1,1] represented by a (3,3)-spline) is in K but not in P. Thus the same-depth inclusion is false as stated, and Corollary 3's 'strictly larger function space' has no valid proof. The proof's parameter accounting is also inconsistent: β is written as m×mn(G+k) while the text counts only mn(G+k) parameters, but the decisive flaw is the missing post-activation linear map in the final layer. The empirical speed results may survive, but the theoretical equivalence claim does not.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PowerMLP, a feedforward network whose hidden layers compute αℓ b(xℓ) + σ_k(ωℓ xℓ + γℓ) and whose last layer is affine, as a faster MLP-style alternative to Kolmogorov-Arnold Networks. The theoretical sections claim that B-splines can be written as linear combinations of powers of ReLU, that any KAN is contained in a PowerMLP of the same depth and comparable width (Theorem 2), and that the reverse inclusion holds over bounded intervals using a two-layer KAN construction (Theorem 6), yielding an equality of function spaces (Corollary 7) and an approximation bound (Corollary 8). The paper also provides a FLOPs comparison and experiments on function fitting, knot theory, classification, and training time, reporting training speeds about 40 times faster than KAN.","tokens_in":19531,"tokens_out":16310,"duration_ms":148900,"significance":"The practical motivation is credible: a drop-in network that trains much faster than KAN while retaining comparable accuracy would be useful, and the empirical study is substantial: the experiments use matched parameter budgets, repeated timings, ablations, and released code. The reverse inclusion PowerMLP ⊆ KAN over intervals (Theorem 6) appears to be a valid construction. However, the central theoretical contribution is not established. The key inclusion Theorem 2 is false as stated, and Corollary 3 and Corollary 7 both depend on it. As a result, the advertised claims that PowerMLPs define a strictly larger or equal function space than KANs are unsupported. The empirical speed and accuracy results may still be of interest, but they do not compensate for the failure of the main theoretical assertion in a paper whose abstract, title, and Figure 1 foreground that assertion.","major_comments":[{"comment":"The proof of Theorem 2 does not establish the stated inclusion K_{d,w,k,G,p} ⊂ P_{d,w^2(G+k),k,p}. The proof models a KAN layer as αb(x) + βσ_k(ωx + γ) in Eq. (11), but Definition 3 (Eq. (6)) contains no post-activation matrix β: the only term added after σ_k is αb(x), where α multiplies the fixed basis function b(x), not the activated features. The assertion that β can be absorbed into the next layer is not a reparameterization of the same network, because the next PowerMLP layer applies σ_k to an affine function of its input, so the absorbed β would enter inside the next nonlinearity, whereas in the KAN the multiplication by β occurs before the next layer's spline activation. For the final layer there is no next layer, and Eq. (6) makes the final PowerMLP layer affine. Consequently, a depth-1 KAN whose single layer is a nonlinear spline (e.g., a (3,3)-spline representing x^2 on [-1,1]) cannot be represented by any depth-1 PowerMLP. The same-depth inclusion is therefore false as stated, and Corollary 3 and Corollary 7, which both rely on Theorem 2, are not established.","section":"Section 4.2 / Definition 3 (Eq. 6), Eq. 11"},{"comment":"The proof's parameter accounting is inconsistent with the PowerMLP architecture. The proof introduces β ∈ R^{m×mn(G+k)} and counts mn(G+k) parameters for β, but those parameters do not appear in Definition 3. If β is instead absorbed into the following layer, that layer's weight matrix changes, so the claimed equality of parameter counts for a same-depth network is lost. Thus the subscript p in P_{d,w^2(G+k),k,p} is not supported by the proof.","section":"Appendix A.1, parameter accounting"},{"comment":"Even setting aside the failure of Theorem 2, the argument for strict inclusion over R^n is incomplete. The text says that a spline function is zero outside a certain interval and that PowerMLPs include all polynomials, but a KAN layer also contains the residual basis function b(x) in Eq. (4), which is nonzero and unbounded outside that interval. To conclude that KANs cannot represent all polynomials, the paper would need to analyze the effect of composing b(x) with spline layers; this is not done. The strictness claim is therefore unsupported independently of the Theorem 2 gap.","section":"Section 4.2, Corollary 3"}],"minor_comments":[{"comment":"The dimension of γℓ is stated as R^{mℓ×1}, but ωℓ is m_{ℓ+1}×mℓ, so ωℓ xℓ + γℓ lives in R^{m_{ℓ+1}×1}; the bias dimension should be corrected.","section":"Definition 3, Eq. (6)"},{"comment":"Eq. (11) writes βσ_{k-1}(ωx + γ) while the text and the rest of the proof use σ_k; this is presumably a typo and should be made consistent.","section":"Eq. (11) and surrounding text"},{"comment":"The proof uses σ_0(x) as the binary step function, but Eq. (5) defines σ_k only for k ∈ Z_+; σ_0 should be defined explicitly or the base case should be stated as a convention.","section":"Lemma 1 proof"},{"comment":"The reported speedup ratios vary from about 24× to 95× across the eight tasks; writing 'about 40 times faster' is only an average, and the range or standard deviation should be reported.","section":"Section 5.3, Table 3"},{"comment":"The accuracy comparisons in Figure 5 appear as single points without error bars or repetition information; adding variability would strengthen the empirical claims.","section":"Section 5.2, Figure 5"},{"comment":"The related work mentions KAN variants such as Wav-KAN and rKAN, but the experiments compare only against KAN version 0.2.5; a comparison with at least one faster KAN implementation would make the speed claim more robust.","section":"Related Work / Section 5"}],"recommendation":"reject","confidential_remarks":"The central theorem of the manuscript, Theorem 2, is false as stated, and the two main theoretical corollaries depend on it. This is not a local proof gap that can be repaired while keeping the same architecture; the PowerMLP definition would need to be changed (for example, by adding a post-activation matrix), which would require reworking the theory and experiments. The empirical speed and accuracy results are interesting, and a resubmission that reframes the paper as an empirical study or proves only a weaker, depth-changing representation theorem could be viable. In the current form, however, the advertised theoretical contribution is unsupported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"PowerMLP is a simple MLP with ReLU-k activations plus a basis function, and the empirical story is believable: roughly 40x training speedup over KAN on several tasks, with competitive accuracy and code provided. The construction itself is a clean repackaging of the classical B-spline-as-ReLU-k identity, and the FLOPs comparison is transparent. The ablation showing both the basis and ReLU-k matter is genuinely useful.\n\nThe theory is not sound as stated. Theorem 2 claims K_{d,w,k,G,p} ⊂ P_{d,w^2(G+k),k,p}. The proof expresses a KAN layer as αb(x)+βσ_k(ωx+γ) and then says β can be absorbed into the next layer. But PowerMLP layers have no post-activation β; only the final layer is affine. For depth 1, a KAN layer is a nonlinear spline while a depth-1 PowerMLP is affine, so a spline like x^2 on [-1,1] is in K but not in P. The absorption argument implicitly adds a layer, breaking the same-depth claim. Corollary 3's strict containment over R^n then has no valid proof. Theorem 6, showing PowerMLP can be represented by a KAN over bounded intervals, appears correct, so the paper has a partial equivalence, not the advertised one.\n\nThe experiments are suggestive but not tight. The knot theory KAN numbers are taken from Liu et al. rather than run in the same pipeline; several figures lack error bars; and the learning-rate search is done for MLP/PowerMLP but not for KAN. The speed advantage is consistent with the FLOPs analysis, so I expect the empirical core would survive fairer comparisons.\n\nBottom line: there is a useful empirical contribution here, but the central theoretical guarantee fails. I would send it to peer review because the topic is timely and the empirical evidence deserves checking, but the authors need to either weaken the theory to allow a small depth overhead or present it as an approximation. In current form it should not be accepted.","headline":"Useful empirical speedup and a clean architecture, but the central same-depth equivalence with KAN is false; worth a careful revision, not acceptance as-is.","tokens_in":20177,"tokens_out":4472,"would_cite":false,"duration_ms":40046,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","41A15"],"pacs":[],"model":"deepseek-v4-flash","headline":"PowerMLP, an MLP-style network using k-th powers of ReLU as activations, is claimed to match KAN's function space on bounded boxes while training roughly 40 times faster.","keywords":["Kolmogorov-Arnold Networks","PowerMLP","ReLU-k activation","B-spline representation","function space equivalence","FLOPs comparison","efficient neural networks","spline-free approximation"],"falsifier":"Train a one-hidden-layer KAN with a $(3,3)$-grid and compare the functions it can express with a one-hidden-layer PowerMLP of width $w^2(G+k)$: if there exists a KAN function that no same-depth PowerMLP of that width reproduces, the inclusion in Theorem 2 fails as stated. Concretely, check whether the $\\beta$ matrix in Eq. (11) can always be absorbed into the following layer without changing the number of layers, since removing that absorption step breaks the depth equality in the theorem.","tokens_in":19024,"feed_emoji":"⚡","tokens_out":5174,"duration_ms":38893,"temperature":0.7,"pith_summary":"The paper introduces PowerMLP, an MLP-style network whose hidden layers compute $\\sigma_k(\\omega x+\\gamma)+\\alpha b(x)$, where $\\sigma_k$ is the $k$-th power of ReLU. The central claim is that PowerMLP is a drop-in replacement for Kolmogorov-Arnold Networks (KANs): over $\\mathbb{R}^n$ it defines a strictly larger function space than KAN, and over any bounded box $[-E,E]^n$ it defines exactly the same function space. Because B-splines can be written as finite linear combinations of ReLU powers, PowerMLP avoids the recursive de Boor-Cox evaluation that makes KAN slow, and its FLOP count is over ten times lower. The authors report that PowerMLP trains about 40 times faster than KAN while matching or beating its accuracy across function fitting, knot theory, classification, and language tasks. If correct, the result gives practitioners a faster architecture with at least the expressive power of KAN.","feed_headline":"PowerMLP matches KAN's power at 40x speed","feed_subtitle":"Writing B-splines as sums of ReLU powers removes KAN's iterative spline cost while keeping its expressiveness.","key_machinery":"The load-bearing identity is Lemma 1: a $k$-order B-spline on a knot sequence $(t_j,\\dots,t_{j+k+1})$ equals $\\sum_i \\frac{t_{j+k+1}-t_j}{\\prod_{l\\neq i}(t_l-t_i)}\\,\\sigma_k(x-t_i)$. This expresses the recursive de Boor-Cox construction as a single fixed linear combination of ReLU powers, which is what lets PowerMLP evaluate splines without iteration; the same identity is inverted in Lemmas 4 and 5 to show splines can also reproduce affine maps and ReLU-$k$, giving the reverse inclusion over bounded intervals.","core_discovery":"The paper's central discovery is that KAN's learnable spline activations are redundant in a precise sense: every $(k,G)$-spline is a linear combination of $k$-th powers of ReLU, so any KAN layer can be rewritten as a PowerMLP layer with the same number of trainable parameters and a width inflated by a factor of $G+k$. This yields the inclusion $K_{d,w,k,G,p} \\subset P_{d,w^2(G+k),k,p}$, and since splines vanish outside their support while ReLU powers generate all polynomials, PowerMLPs form a strictly larger function space over $\\mathbb{R}^n$. Conversely, over any bounded interval, affine maps and ReLU-$k$ functions can be represented by splines, so every PowerMLP layer is a two-layer KAN; the two architectures therefore define the same function space on $[-E,E]^n$. The paper also derives closed-form FLOP counts showing KAN needs over ten times more operations per parameter, and validates the theory with experiments reporting roughly 40-fold faster training.","pith_inferences":["Beyond the paper: if the equivalence on bounded intervals holds, then approximation and representation results for ReLU-$k$ MLPs transfer to KANs and vice versa, a two-way bridge the paper only sketches through Corollary 8.","Beyond the paper: the FLOP comparison counts forward-pass arithmetic only; a fairer training-cost comparison would include backward passes and memory access, so the reported 40x wall-clock speedup may depend on implementation details beyond FLOPs.","Beyond the paper: the strictly-larger-over-$\\mathbb{R}^n$ claim suggests that MLP-type networks need not sacrifice expressive power to gain speed, which could inform architecture search beyond KAN itself."],"forward_implications":["Any architecture that uses KAN layers can swap in PowerMLP layers and keep, or exceed, the representable function space while cutting training time by roughly 40x in the reported settings.","The approximation guarantee for smooth functions on $[-1,1]^n$ (Corollary 8) transfers to KANs via the equivalence on bounded intervals, giving a depth and parameter bound for KANs that goes beyond the depth-2 Kolmogorov-Arnold representation.","PowerMLP has FLOP counts close to plain MLP, so MLP-optimized hardware and training pipelines can be reused without the spline-evaluation overhead.","The ablation indicates the basis function is needed for training stability at depth and ReLU-$k$ for expressive power, so both components are load-bearing in practice."],"supporting_citations":[{"why":"Defines KAN, its spline activation, and the claim that KANs train 10x slower than MLPs; this is the baseline power and speed comparison.","marker":"Liu et al. (2024)"},{"why":"Identifies k-order B-splines as k-th divided differences of truncated powers, the classical fact PowerMLP exploits.","marker":"Curry and Schoenberg (1947)"},{"why":"Gives the representation of B-splines as linear combinations of powers of ReLU, which is the non-iterative formula at the core of PowerMLP.","marker":"Greville (1969)"},{"why":"Supplies the de Boor-Cox recursion that KAN uses for spline evaluation; PowerMLP's speed claim is measured against this recursion.","marker":"Cox (1972)"},{"why":"Standard reference for B-spline properties, including the partition-of-unity property used in Lemma 4.","marker":"de Boor (1978)"},{"why":"Establishes that ReLU-k networks can represent polynomials, used for the strict containment over R^n and for the approximation bound in Corollary 8.","marker":"Li, Tang, and Yu (2020a)"},{"why":"Provides the FLOP count for a KAN layer that the paper compares against PowerMLP's closed-form count.","marker":"Yu, Yu, and Wang (2024)"}],"fun_headline_variants":["PowerMLP: KAN's speed fix","ReLU powers beat KAN's splines at 40x speed","PowerMLP ditches slow splines, keeps KAN's power","Spline-free KAN alternative trains 40x faster","PowerMLP: KAN expressiveness at MLP speed"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof of Theorem 2 rewrites a KAN layer as $\\beta\\sigma_k(\\omega x+\\gamma)$ and then assumes the post-activation matrix $\\beta$ can be folded into the next layer without changing the depth count, but the PowerMLP layer as defined has no such post-activation linear map, so the same-depth inclusion depends on a depth re-counting that the definition does not state.","fun_headline_variants_meta":{"raw":{"variants":["PowerMLP: KAN's speed fix","ReLU powers beat KAN's splines at 40x speed","PowerMLP ditches slow splines, keeps KAN's power","Spline-free KAN alternative trains 40x faster","PowerMLP: KAN expressiveness at MLP speed"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000565,"raw_usage":{"total_tokens":2693,"prompt_tokens":975,"completion_tokens":1718,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":591,"completion_tokens_details":{"reasoning_tokens":1632}},"tokens_in":591,"tokens_out":1718,"duration_ms":11459,"temperature":1.0,"reasoning_tokens":1632,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:01:24.649841+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a one-hidden-layer KAN with a $(3,3)$-grid and compare the functions it can express with a one-hidden-layer PowerMLP of width $w^2(G+k)$: if there exists a KAN function that no same-depth PowerMLP of that width reproduces, the inclusion in Theorem 2 fails as stated. Concretely, check whether the $\\beta$ matrix in Eq. (11) can always be absorbed into the following layer without changing the number of layers, since removing that absorption step breaks the depth equality in the theorem.","supporting_citations":[{"cited_title":"B.; and Schoenberg, I","cited_arxiv_id":null,"evidence_quote":"Identifies k-order B-splines as k-th divided differences of truncated powers, the classical fact PowerMLP exploits."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the representation of B-splines as linear combinations of powers of ReLU, which is the non-iterative formula at the core of PowerMLP."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the de Boor-Cox recursion that KAN uses for spline evaluation; PowerMLP's speed claim is measured against this recursion."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Standard reference for B-spline properties, including the partition-of-unity property used in Lemma 4."}],"review_version":1}