REVIEW 1 major objections 3 minor 12 references
Optimized Piecewise Affine Abstractions of Neural Networks with Learnable Activation Functions
T0 review · 1 major / 3 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read Verifying a KAN reduces to deciding how many linear pieces each activation gets, with error allocated like a knapsack budget.
desk verdict The knapsack-style allocation of PWA pieces across KAN units is a real and useful idea, with an honest 59/65 tightness win over uniform allocation, but the paper's central soundness claim depends on a discretization-to-continuum correction that the main text never says was applied. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing identity is Theorem 1: |y − ŷ| ≤ Σ W(i)j,k e(i)j,k, with the weights W(i)j,k defined as the sum over all paths from unit ψ(i)j,k to the network output of the products of downstream Lipschitz constants. This additive decomposition is what converts a network-level approximation problem into a per-unit budgeting problem: once each unit has a dynamic-programming 'trade-off table' listing the best error achievable with k linear pieces, choosing how many pieces to give every unit under a total error budget is a multi-choice knapsack problem. The weighted error bound therefore determines both the objective and the feasibility of the allocation, and the resulting piecewise-linear ne
What would settle it
Take any single KAN activation on a bounded interval, build the trade-off table with a coarse discretization step, and omit the discretization correction; then compute max_z |ψ(z) − ψ̂(z)| densely (or by exact optimization). If the dense or true error exceeds the tabulated e, the abstraction is unsound. A network-level falsifier is to verify a small KAN with the optimized method, then evaluate the original network with a global optimizer over the input box: any true output outside the claimed interval refutes the soundness claim.
Extended reading notes
Core claim
On its own terms, the paper establishes that the best verified output range for a KAN is obtained by treating abstraction quality as a scarce resource. For every activation ψ(i)j,k, replacing it by a piecewise-linear function with error e(i)j,k contributes at most W(i)j,k · e(i)j,k to the final output error, where W(i)j,k is the sum, over every path from that activation to the output, of the products of the Lipschitz constants of the downstream activations along that path. Summing these contributions bounds the global error, so minimizing total error for a fixed number of pieces is exactly a multi-choice knapsack problem. The paper computes per-activation trade-off tables with dynamic progra
Load-bearing premise
The whole soundness argument rests on each tabulated per-activation error e(i)j,k being a genuine upper bound on the continuous error between the activation and its piecewise-linear approximation over the entire input range, not merely the error on the discrete grid used by the dynamic program.
Editorial extensions
If this is right
- If the central bound holds, optimizing piece allocation rather than using a fixed number of segments per activation gives tighter verified output ranges; the paper reports 59 of 65 benchmarks improved over uniform allocation.
- Range verification of a KAN becomes solvable by standard MILP solvers, so nonlinear learned activations no longer require a bespoke verifier.
- The abstraction can be computed once and reused: sensitivity analysis that sweeps many input perturbations becomes cheap because the per-unit tables and knapsack solution are amortized.
- The same pipeline applies to any activation satisfying the paper's mild assumptions (bounded domain, differentiability, and an interval procedure for derivatives), including spline, Chebyshev, Fourier, and other KAN variants.
- The tightness guarantee is quantitative: for a chosen δ, the returned output interval is within δ of the true attainable range, so users control the precision/cost trade-off explicitly.
Reading between the lines
- The paper leaves implicit that the knapsack view also gives a dual formulation: instead of fixing total pieces and minimizing error, one could fix an error budget and ask for the cheapest MILP (fewest total pieces), yielding the same DP machinery as a Pareto frontier over verification cost and bound tightness.
- Because the error decomposition only uses path products of Lipschitz constants and the additivity of KAN layers, a natural testable extension is applying the same weighted-budget allocation to other additive or residual architectures; the paper does not claim this, but the Theorem 1 proof structure suggests it.
- A straightforward experiment to stress the soundness claim is to compare the verified interval against exact nonlinear global optimization on small KANs: if the discretization correction is omitted or the grid is too coarse, the verified interval may exclude true outputs, so the correction is not a formality but load-bearing.
- The reported sensitivity numbers (output changes of 8.6°C and 23.8° in weather and prosthetic models under 0.01 perturbations) suggest that practitioners should treat these KAN models as highly non-robust; that is an editorial reading of the paper's application study, not a claim the paper generalizes.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a verification framework for Kolmogorov-Arnold Networks (KANs) in which each univariate activation is replaced by a piecewise affine (PWA) approximation with a certified error. A dynamic programming routine is used to compute per-unit error-vs-piece trade-off tables, Theorem 1 derives a global error bound as a weighted sum of per-unit errors with weights given by path products of downstream Lipschitz constants, and the resulting allocation problem is formulated as a multi-choice knapsack problem. The abstracted network is then encoded in a MILP for range verification. The experiments compare the optimized abstraction against a vanilla uniform-piece abstraction on 65 KAN benchmarks and also compare KAN verification with MLP verification using Gurobi and LiRPA. The central claim is that the optimized abstraction produces tighter output intervals than the uniform allocation on 59 of 65 benchmarks at comparable cost.
Significance. If the identified gaps are fixed, the paper would make a useful contribution: the path-weighted error propagation in Theorem 1 is a clean structural idea, the reduction to multi-choice knapsack gives a principled way to allocate a limited number of PWA pieces, and the experimental corpus is broad and relevant. The paper also explicitly targets a gap in the literature, since KAN verification is relatively unexplored. However, the current manuscript does not fully establish the soundness of the reported verification bounds: the connection between the discretized DP error and the continuous error needed for verification is not shown to be enforced in the experiments, and the DP recurrence as written does not implement the claimed optimization. These issues are load-bearing for the paper's main claim and must be resolved before the results can be accepted.
major comments (1)
- [§6, MILP encoding] The MILP constraints in Section 6 add an independent ±e slack at each unit, producing an over-approximation by allowing the per-unit errors to vary independently. Theorem 1 instead gives a single weighted global slack. The paper does not explain how the knapsack allocation, which is derived from the weighted bound, relates to the actual MILP bound. This may be an intended design choice, but it should be stated explicitly, since otherwise the connection between the theoretically optimized allocation and the empirically reported output widths is unclear.
minor comments (3)
- [§4] Typo: 'singplePieceError' should be 'singlePieceError'.
- [§7.1] The choice δ = 3.4 × δ_min is not motivated. Since this parameter controls the trade-off between tightness and the number of pieces, a brief explanation or an ablation would help the reader interpret the results.
- [General] The full-text title, 'Optimal Abstractions for Verifying Properties of Kolmogorov-Arnold Networks (KANs)', differs from the arXiv title. These should be harmonized.
Circularity Check
No significant circularity: the error-propagation bound and knapsack allocation are derived from the KAN structure, not from the verification target; reported tighter bounds are an optimization outcome, not a fitted prediction.
full rationale
The derivation chain is self-contained. Lemma 1 bounds each unit's error using the Lipschitz constant ψ′_max, and Theorem 1 combines these via path products W(i)j,k over the network DAG; neither statement assumes the output range being verified. Problem 2's DP (Bellman) minimizes a per-unit PWA error, and Problem 3's knapsack minimizes total piece count subject to the error budget δ; the MILP in Section 6 encodes the same per-unit error bounds. No parameter is fit to the benchmark output widths, and the tightness comparison to the vanilla allocation is a consequence of optimizing the allocation, not a conclusion imported from the benchmarks. The heuristic δ=3.4×δmin is ad hoc but not circular. Self-citations (e.g., Kushner et al. 2020 for the sensitivity setup, Dutta et al. 2019 as related work) are not load-bearing for the main result. The one genuine concern is a soundness gap rather than circularity: Section 4 defines the DP objective as the discretized error e(Δ) and says the Appendix A correction 'can be used,' but Section 7 never states that the correction or a sufficiently small Δ was applied; if it was not, the reported bounds are not guaranteed over-approximations. That is a missing implementation/proof detail, not a case where a prediction reduces by construction to its inputs.
Assumptions & free parameters
free parameters (5)
- grid spacing Δ
- target error budget δ = 3.4×δmin =
3.4×δmin
- max pieces per unit (Smax) =
262 or 175 depending on model size
- MIP gap tolerance εgap =
0.15 or 0.2
- integer scaling constant c for knapsack weights
assumptions (4)
- domain assumption Assumption 1: each ψ is zero outside [−L,L], continuous and differentiable, and a procedure DERIVATIVEINTERVAL returns a valid interval bound on the derivative with tolerance ε.
- domain assumption The discretized-error trade-off table can be converted to a continuous error bound using the correction in Theorem 4 (Appendix A).
- domain assumption The KAN is a DAG of univariate functions and addition with unit edge weights; error propagation weights W are sums over paths of products of downstream Lipschitz constants.
- standard math Multi-choice knapsack DP runs in O(W N k) for integer weights.
Cite this review
Pith. "Pith review of Optimized Piecewise Affine Abstractions of Neural Networks with Learnable Activation Functions." pith.science (2026). https://pith.science/paper/AMZIB4YT
@misc{pith2026260206737,
author = {Pith},
title = {Pith review of: Optimized Piecewise Affine Abstractions of Neural Networks with Learnable Activation Functions},
year = {2026},
howpublished = {\url{https://pith.science/paper/AMZIB4YT}},
note = {Machine review of arXiv:2602.06737}
}
read the original abstract
We present a generalized framework for the range verification of neural networks featuring non-linear activation functions. Our approach first constructs an ``optimized piecewise affine abstraction" of the network that replaces each non-linear activation function by a piecewise affine (PWA) function plus a bounded error. Such PWA functions are readily amenable to existing neural network verification techniques using specializations of linear arithmetic SMT solvers and mixed-integer optimization approaches. However, there are infinitely many ways to abstract each node, with a natural tradeoff between the number of pieces used, the global error bound, and the complexity of the resulting verification problem. We propose a dynamic programming (DP) algorithm to systematically compute the optimized PWA abstraction for general activation functions, guaranteeing tighter output bounds. The algorithm combines a local DP approximation at each node with a global error bound, yielding a variant of the knapsack problem for deciding how to allocate a fixed budget on the total number of pieces across units so as to minimize the worst-case error bound between the network and its approximation. Although the knapsack problem is itself NP-hard, we can use pseudo-polynomial DP algorithms as well as approximation schemes to solve it efficiently. Crucially, our approach is broadly applicable to diverse networks consisting of non-linear activations, including standard Multi-Layer Perceptrons (MLPs) and recently proposed architectures such as Kolmogorov-Arnold Networks (KANs). Over a series of KAN benchmarks spanning 20 to 22,000 parameters, our approach yields output bounds that are consistently of smaller width than uniform PWA allocation. The overall time taken is roughly comparable while the overhead for computing the optimized abstraction is subsumed by the time taken to compute output bounds.
Figures
Reference graph
Works this paper leans on
-
[1]
ItemI k i,1 with weightw k i,1 = 0and valuev k i,1 = 0, corresponding to not selecting the itemj
-
[2]
ItemI k i,2 with weightw k i,2 =w j and valuev k i,2 =v j, corresponding to selecting itemj If the knapsack problem has a subset S satisfying the constraints of the MOK problem, then for each i choosing item I i i,2 if j∈S and I i i,1 otherwise, will lead to a valid MOK solution with total weight ≤W and value ≥V . Conversely, any feasible solution that co...
-
[5]
URL https://openreview.net/forum? id=Ozo7qJ5vZi. Lomuscio, A. and Maganti, L. An approach to reachabil- ity analysis for feed-forward relu neural networks, 2017. URLhttps://arxiv.org/abs/1706.07351. Mahara, A., Rishe, N. D., and Deng, L. The dawn of kan in image-to-image (i2i) translation: Integrating kolmogorov- arnold networks with gans for unpaired i2i...
arXiv 2017
-
[11]
If the unit’s input is that of the entire KAN, its range is known as part of the problem input and thus we can compute Mz directly from the input bound
-
[12]
To computeM y for a given KAN, we recall the linearized function bψ(z) = 0z≤ −L aiz+b i z∈[z i, zi+1], i∈ {0,
If the unit’s input is a sum node that involves the outputs of k units, the value of Mz for the unit is simply the sum of the value ofM y’s for the units whose outputs connect to the input through the summation node. To computeM y for a given KAN, we recall the linearized function bψ(z) = 0z≤ −L aiz+b i z∈[z i, zi+1], i∈ {0, . . . , ℓ−1} 0z≥L Note...
-
[1969]
doi: Link:”10.1080/01621459.1969.10501038”. Bhattacharjee, S. S. TorchKAN: Simplified KAN model with variations. https://github.com/1ssb/ torchkan/, 2024. Bojarski, M., Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., Zhang, X., Zhao, J., and Zieba, K. End to end learning for self-driving car...
arXiv 1969
-
[2016]
Weather data from the Max Planck Institute for Biogeochemistry, Jena, Germany. Accessed 28 January 2026. MOSEK ApS.MOSEK Optimization Suite, 2022. URL https://www.mosek.com/. Version 10.0. Polo-Molina, A., Alfaya, D., and Portela, J. Monokan: Certified monotonic kolmogorov-arnold network, 2024. URLhttps://arxiv.org/abs/2409.11078. Samala, M., Rattanakoch,...
arXiv 2026
-
[2019]
cc/paper_files/paper/2019/file/ 0a9fdbb17feb6ccb7ec405cfb85222c4-Paper
URL https://proceedings.neurips. cc/paper_files/paper/2019/file/ 0a9fdbb17feb6ccb7ec405cfb85222c4-Paper. pdf. Sprecher, D. A. A numerical implementation of kol- mogorov’s superpositions.Neural Networks, 9(5):765– 772, 1996. SS, S., AR, K., R, G., and KP, A. Chebyshev polynomial- based kolmogorov-arnold networks: An efficient archi- tecture for nonlinear f...
arXiv 2019
Show all 12 references
-
[2021]
Version 20.1.0
URL https://www.ibm.com/products/ ilog-cplex-optimization-studio . Version 20.1.0. Irfan, A., Julian, K. D., Wu, H., Barrett, C., Kochenderfer, M. J., Meng, B., and Lopez, J. Towards verification of neural networks for small unmanned aircraft collision avoidance. In2020 AIAA/I...
2020
-
[2025]
Howard, A
URL https://openreview.net/forum? id=N9hl7CShRA. Howard, A. A., Jacob, B., Murphy, S. H., Hein- lein, A., and Stinis, P. Finite basis kolmogorov- arnold networks: domain decomposition for data-driven and physics-informed problems. arXiv (Cornell University), Jun 2024. doi: htt...
2024 doi
-
[2158]
URL https://proceedings
PMLR, 2022. URL https://proceedings. mlr.press/v205/majd23a.html. Max Planck Institute for Biogeochemistry. Jena climate dataset (2009–2016). https://www.kaggle. com/datasets/mnassrib/jena-climate,
2022
-
[3396]
Gurobi Optimization, LLC
IEEE, 2017. Gurobi Optimization, LLC. Gurobi Optimizer Reference Manual, 2024. URLhttps://www.gurobi.com. Heiderich, B., Schumacher, M.-L., and Huber, M. Training and verifying robust kolmogorov-arnold networks. In ICLR 2025 Workshop: VerifAI: AI Verification in the Wild,
2017
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.