Pith. sign in

REVIEW 1 major objections 3 minor 12 references

Optimized Piecewise Affine Abstractions of Neural Networks with Learnable Activation Functions

T0 review · 1 major / 3 minor · reviewed 2026-08-04 · deepseek-v4-flash

Pith's one-line read Verifying a KAN reduces to deciding how many linear pieces each activation gets, with error allocated like a knapsack budget.

desk verdict The knapsack-style allocation of PWA pieces across KAN units is a real and useful idea, with an honest 59/65 tightness win over uniform allocation, but the paper's central soundness claim depends on a discretization-to-continuum correction that the main text never says was applied. read the letter →

arxiv 2602.06737 v2 pith:AMZIB4YT submitted 2026-02-06 cs.LG cs.AIcs.LO

classification cs.LGcs.AIcs.LO
keywords Kolmogorov-Arnoldnetworksneuralnetworkverificationpiecewiseaffineabstractionrangeanalysisknapsackproblemdynamicprogrammingmixed-integerlinearLipschitzerrorpropagation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Kolmogorov-Arnold networks replace fixed activations with learned univariate functions, which makes formal verification hard because the usual ReLU-style reasoning does not apply. The paper's central claim is that this difficulty can be managed by replacing every activation with a piecewise-linear approximation that carries a bounded error, and then choosing how many linear pieces to spend on each activation as a global budget-allocation problem. The key theorem states that the total approximation error of the whole network is bounded by a weighted sum of per-activation errors, where each weight counts all output paths through downstream Lipschitz constants. That structure turns the choice of approximations into a multi-choice knapsack problem, solvable by dynamic programming, and the resulting abstraction is encoded as a mixed-integer linear program for range verification. Across 65 KAN benchmarks, the optimized allocation yields tighter verified output ranges than a uniform piece allocation, with comparable overall verification time.

What carries the argument

The load-bearing identity is Theorem 1: |y − ŷ| ≤ Σ W(i)j,k e(i)j,k, with the weights W(i)j,k defined as the sum over all paths from unit ψ(i)j,k to the network output of the products of downstream Lipschitz constants. This additive decomposition is what converts a network-level approximation problem into a per-unit budgeting problem: once each unit has a dynamic-programming 'trade-off table' listing the best error achievable with k linear pieces, choosing how many pieces to give every unit under a total error budget is a multi-choice knapsack problem. The weighted error bound therefore determines both the objective and the feasibility of the allocation, and the resulting piecewise-linear ne

What would settle it

Take any single KAN activation on a bounded interval, build the trade-off table with a coarse discretization step, and omit the discretization correction; then compute max_z |ψ(z) − ψ̂(z)| densely (or by exact optimization). If the dense or true error exceeds the tabulated e, the abstraction is unsound. A network-level falsifier is to verify a small KAN with the optimized method, then evaluate the original network with a global optimizer over the input box: any true output outside the claimed interval refutes the soundness claim.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that the best verified output range for a KAN is obtained by treating abstraction quality as a scarce resource. For every activation ψ(i)j,k, replacing it by a piecewise-linear function with error e(i)j,k contributes at most W(i)j,k · e(i)j,k to the final output error, where W(i)j,k is the sum, over every path from that activation to the output, of the products of the Lipschitz constants of the downstream activations along that path. Summing these contributions bounds the global error, so minimizing total error for a fixed number of pieces is exactly a multi-choice knapsack problem. The paper computes per-activation trade-off tables with dynamic progra

Load-bearing premise

The whole soundness argument rests on each tabulated per-activation error e(i)j,k being a genuine upper bound on the continuous error between the activation and its piecewise-linear approximation over the entire input range, not merely the error on the discrete grid used by the dynamic program.

Editorial extensions

If this is right

  • If the central bound holds, optimizing piece allocation rather than using a fixed number of segments per activation gives tighter verified output ranges; the paper reports 59 of 65 benchmarks improved over uniform allocation.
  • Range verification of a KAN becomes solvable by standard MILP solvers, so nonlinear learned activations no longer require a bespoke verifier.
  • The abstraction can be computed once and reused: sensitivity analysis that sweeps many input perturbations becomes cheap because the per-unit tables and knapsack solution are amortized.
  • The same pipeline applies to any activation satisfying the paper's mild assumptions (bounded domain, differentiability, and an interval procedure for derivatives), including spline, Chebyshev, Fourier, and other KAN variants.
  • The tightness guarantee is quantitative: for a chosen δ, the returned output interval is within δ of the true attainable range, so users control the precision/cost trade-off explicitly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the knapsack view also gives a dual formulation: instead of fixing total pieces and minimizing error, one could fix an error budget and ask for the cheapest MILP (fewest total pieces), yielding the same DP machinery as a Pareto frontier over verification cost and bound tightness.
  • Because the error decomposition only uses path products of Lipschitz constants and the additivity of KAN layers, a natural testable extension is applying the same weighted-budget allocation to other additive or residual architectures; the paper does not claim this, but the Theorem 1 proof structure suggests it.
  • A straightforward experiment to stress the soundness claim is to compare the verified interval against exact nonlinear global optimization on small KANs: if the discretization correction is omitted or the grid is too coarse, the verified interval may exclude true outputs, so the correction is not a formality but load-bearing.
  • The reported sensitivity numbers (output changes of 8.6°C and 23.8° in weather and prosthetic models under 0.01 perturbations) suggest that practitioners should treat these KAN models as highly non-robust; that is an editorial reading of the paper's application study, not a claim the paper generalizes.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 3 minor

Summary. The paper proposes a verification framework for Kolmogorov-Arnold Networks (KANs) in which each univariate activation is replaced by a piecewise affine (PWA) approximation with a certified error. A dynamic programming routine is used to compute per-unit error-vs-piece trade-off tables, Theorem 1 derives a global error bound as a weighted sum of per-unit errors with weights given by path products of downstream Lipschitz constants, and the resulting allocation problem is formulated as a multi-choice knapsack problem. The abstracted network is then encoded in a MILP for range verification. The experiments compare the optimized abstraction against a vanilla uniform-piece abstraction on 65 KAN benchmarks and also compare KAN verification with MLP verification using Gurobi and LiRPA. The central claim is that the optimized abstraction produces tighter output intervals than the uniform allocation on 59 of 65 benchmarks at comparable cost.

Significance. If the identified gaps are fixed, the paper would make a useful contribution: the path-weighted error propagation in Theorem 1 is a clean structural idea, the reduction to multi-choice knapsack gives a principled way to allocate a limited number of PWA pieces, and the experimental corpus is broad and relevant. The paper also explicitly targets a gap in the literature, since KAN verification is relatively unexplored. However, the current manuscript does not fully establish the soundness of the reported verification bounds: the connection between the discretized DP error and the continuous error needed for verification is not shown to be enforced in the experiments, and the DP recurrence as written does not implement the claimed optimization. These issues are load-bearing for the paper's main claim and must be resolved before the results can be accepted.

major comments (1)
  1. [§6, MILP encoding] The MILP constraints in Section 6 add an independent ±e slack at each unit, producing an over-approximation by allowing the per-unit errors to vary independently. Theorem 1 instead gives a single weighted global slack. The paper does not explain how the knapsack allocation, which is derived from the weighted bound, relates to the actual MILP bound. This may be an intended design choice, but it should be stated explicitly, since otherwise the connection between the theoretically optimized allocation and the empirically reported output widths is unclear.
minor comments (3)
  1. [§4] Typo: 'singplePieceError' should be 'singlePieceError'.
  2. [§7.1] The choice δ = 3.4 × δ_min is not motivated. Since this parameter controls the trade-off between tightness and the number of pieces, a brief explanation or an ablation would help the reader interpret the results.
  3. [General] The full-text title, 'Optimal Abstractions for Verifying Properties of Kolmogorov-Arnold Networks (KANs)', differs from the arXiv title. These should be harmonized.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the error-propagation bound and knapsack allocation are derived from the KAN structure, not from the verification target; reported tighter bounds are an optimization outcome, not a fitted prediction.

full rationale

The derivation chain is self-contained. Lemma 1 bounds each unit's error using the Lipschitz constant ψ′_max, and Theorem 1 combines these via path products W(i)j,k over the network DAG; neither statement assumes the output range being verified. Problem 2's DP (Bellman) minimizes a per-unit PWA error, and Problem 3's knapsack minimizes total piece count subject to the error budget δ; the MILP in Section 6 encodes the same per-unit error bounds. No parameter is fit to the benchmark output widths, and the tightness comparison to the vanilla allocation is a consequence of optimizing the allocation, not a conclusion imported from the benchmarks. The heuristic δ=3.4×δmin is ad hoc but not circular. Self-citations (e.g., Kushner et al. 2020 for the sensitivity setup, Dutta et al. 2019 as related work) are not load-bearing for the main result. The one genuine concern is a soundness gap rather than circularity: Section 4 defines the DP objective as the discretized error e(Δ) and says the Appendix A correction 'can be used,' but Section 7 never states that the correction or a sufficiently small Δ was applied; if it was not, the reported bounds are not guaranteed over-approximations. That is a missing implementation/proof detail, not a case where a prediction reduces by construction to its inputs.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. Its free parameters are algorithmic: grid spacing, error budget heuristic, max pieces, MIP gap, and an unstated integer-scaling constant. The axioms are domain assumptions about the KAN and the validity of the discretization-to-continuous error lift, the latter being the most fragile.

free parameters (5)
  • grid spacing Δ
    Discretization step used to build the trade-off table; controls the gap between discretized and continuous error. Its value is not reported in the experiments.
  • target error budget δ = 3.4×δmin = 3.4×δmin
    Heuristic choice for the knapsack weight budget. No principled justification; it directly affects bound tightness and runtime (Section 7.1).
  • max pieces per unit (Smax) = 262 or 175 depending on model size
    User-set upper limit on PWA pieces per unit, scaled by model size (Section 7.1).
  • MIP gap tolerance εgap = 0.15 or 0.2
    Gurobi MIP gap; affects both the tightness and formal soundness of MILP-derived bounds (Section 7.1).
  • integer scaling constant c for knapsack weights
    Required to make pseudo-polynomial DP applicable to real-valued weights W×e; never specified. Rounding would introduce additional error not accounted for.
assumptions (4)
  • domain assumption Assumption 1: each ψ is zero outside [−L,L], continuous and differentiable, and a procedure DERIVATIVEINTERVAL returns a valid interval bound on the derivative with tolerance ε.
    Needed to compute Lipschitz constants L and the discretization-error correction (Section 3, Assumption 1).
  • domain assumption The discretized-error trade-off table can be converted to a continuous error bound using the correction in Theorem 4 (Appendix A).
    The paper does not state explicitly that this correction is applied in the experiments; if it is not, the MILP uses e as a continuous bound and the verification is unsound (Section 4, Theorem 4, Section 6).
  • domain assumption The KAN is a DAG of univariate functions and addition with unit edge weights; error propagation weights W are sums over paths of products of downstream Lipschitz constants.
    Used in Theorem 1 and its proof in Appendix B, which implicitly assumes |w|=1 on summation edges.
  • standard math Multi-choice knapsack DP runs in O(W N k) for integer weights.
    Standard result (Kellerer et al. 2004), but requires integer weights; the paper's weights W×e are generally real and no valid integer scaling is supplied (Section 5.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Optimized Piecewise Affine Abstractions of Neural Networks with Learnable Activation Functions." pith.science (2026). https://pith.science/paper/AMZIB4YT

@misc{pith2026260206737,
  author       = {Pith},
  title        = {Pith review of: Optimized Piecewise Affine Abstractions of Neural Networks with Learnable Activation Functions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AMZIB4YT}},
  note         = {Machine review of arXiv:2602.06737}
}
read the original abstract

We present a generalized framework for the range verification of neural networks featuring non-linear activation functions. Our approach first constructs an ``optimized piecewise affine abstraction" of the network that replaces each non-linear activation function by a piecewise affine (PWA) function plus a bounded error. Such PWA functions are readily amenable to existing neural network verification techniques using specializations of linear arithmetic SMT solvers and mixed-integer optimization approaches. However, there are infinitely many ways to abstract each node, with a natural tradeoff between the number of pieces used, the global error bound, and the complexity of the resulting verification problem. We propose a dynamic programming (DP) algorithm to systematically compute the optimized PWA abstraction for general activation functions, guaranteeing tighter output bounds. The algorithm combines a local DP approximation at each node with a global error bound, yielding a variant of the knapsack problem for deciding how to allocate a fixed budget on the total number of pieces across units so as to minimize the worst-case error bound between the network and its approximation. Although the knapsack problem is itself NP-hard, we can use pseudo-polynomial DP algorithms as well as approximation schemes to solve it efficiently. Crucially, our approach is broadly applicable to diverse networks consisting of non-linear activations, including standard Multi-Layer Perceptrons (MLPs) and recently proposed architectures such as Kolmogorov-Arnold Networks (KANs). Over a series of KAN benchmarks spanning 20 to 22,000 parameters, our approach yields output bounds that are consistently of smaller width than uniform PWA allocation. The overall time taken is roughly comparable while the overhead for computing the optimized abstraction is subsumed by the time taken to compute output bounds.

Figures

Figures reproduced from arXiv: 2602.06737 by the authors.

Figure 1
Figure 1. An illustration of the key steps of our approach ψb(i) j (z)|, as the number of pieces in ψb(i) j varies. 2. Error Analysis: We characterize the worst case error emax = maxx∈Rn |fN − ˆfN | between the KAN and its PWA approximation. Theorem 1 of the paper shows that this error bound can be written as a weighted com￾bination of the error bounds obtained for each unit. 3. Knapsack Formulation: We show a surprising conn… view at source ↗
Figure 2
Figure 2. A two-layer KAN with three inputs x = (x1, x2, x3) and three outputs y = (y1, y2, y3), i.e., K = 2 with n1 = 3, n2 = 2, n3 = 3. Each layer is shaded differently. KANs are inspired by the well-known Kolmogorov-Arnold representation theorem in real-analysis that states that any continuous function f : R n → R can be expressed as a com￾position of univariate functions and addition (Kolmogorov, 1957; Arnold, 1958). Ther… view at source ↗
Figure 3
Figure 3. Input perturbation radius versus average output bound width on four tasks for KAN Optimized and Empirical bound widths [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 2 linked inside Pith

  1. [1]

    ItemI k i,1 with weightw k i,1 = 0and valuev k i,1 = 0, corresponding to not selecting the itemj

  2. [2]

    ItemI k i,2 with weightw k i,2 =w j and valuev k i,2 =v j, corresponding to selecting itemj If the knapsack problem has a subset S satisfying the constraints of the MOK problem, then for each i choosing item I i i,2 if j∈S and I i i,1 otherwise, will lead to a valid MOK solution with total weight ≤W and value ≥V . Conversely, any feasible solution that co...

  3. [5]

    Lomuscio, A

    URL https://openreview.net/forum? id=Ozo7qJ5vZi. Lomuscio, A. and Maganti, L. An approach to reachabil- ity analysis for feed-forward relu neural networks, 2017. URLhttps://arxiv.org/abs/1706.07351. Mahara, A., Rishe, N. D., and Deng, L. The dawn of kan in image-to-image (i2i) translation: Integrating kolmogorov- arnold networks with gans for unpaired i2i...

  4. [11]

    If the unit’s input is that of the entire KAN, its range is known as part of the problem input and thus we can compute Mz directly from the input bound

  5. [12]

    To computeM y for a given KAN, we recall the linearized function bψ(z) =    0z≤ −L aiz+b i z∈[z i, zi+1], i∈ {0,

    If the unit’s input is a sum node that involves the outputs of k units, the value of Mz for the unit is simply the sum of the value ofM y’s for the units whose outputs connect to the input through the summation node. To computeM y for a given KAN, we recall the linearized function bψ(z) =    0z≤ −L aiz+b i z∈[z i, zi+1], i∈ {0, . . . , ℓ−1} 0z≥L Note...

  6. [1969]

    Bhattacharjee, S

    doi: Link:”10.1080/01621459.1969.10501038”. Bhattacharjee, S. S. TorchKAN: Simplified KAN model with variations. https://github.com/1ssb/ torchkan/, 2024. Bojarski, M., Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., Zhang, X., Zhao, J., and Zieba, K. End to end learning for self-driving car...

  7. [2016]

    Accessed 28 January 2026

    Weather data from the Max Planck Institute for Biogeochemistry, Jena, Germany. Accessed 28 January 2026. MOSEK ApS.MOSEK Optimization Suite, 2022. URL https://www.mosek.com/. Version 10.0. Polo-Molina, A., Alfaya, D., and Portela, J. Monokan: Certified monotonic kolmogorov-arnold network, 2024. URLhttps://arxiv.org/abs/2409.11078. Samala, M., Rattanakoch,...

  8. [2019]

    cc/paper_files/paper/2019/file/ 0a9fdbb17feb6ccb7ec405cfb85222c4-Paper

    URL https://proceedings.neurips. cc/paper_files/paper/2019/file/ 0a9fdbb17feb6ccb7ec405cfb85222c4-Paper. pdf. Sprecher, D. A. A numerical implementation of kol- mogorov’s superpositions.Neural Networks, 9(5):765– 772, 1996. SS, S., AR, K., R, G., and KP, A. Chebyshev polynomial- based kolmogorov-arnold networks: An efficient archi- tecture for nonlinear f...

Show all 12 references
  1. [2021]

    Version 20.1.0

    URL https://www.ibm.com/products/ ilog-cplex-optimization-studio . Version 20.1.0. Irfan, A., Julian, K. D., Wu, H., Barrett, C., Kochenderfer, M. J., Meng, B., and Lopez, J. Towards verification of neural networks for small unmanned aircraft collision avoidance. In2020 AIAA/I...

  2. [2025]

    Howard, A

    URL https://openreview.net/forum? id=N9hl7CShRA. Howard, A. A., Jacob, B., Murphy, S. H., Hein- lein, A., and Stinis, P. Finite basis kolmogorov- arnold networks: domain decomposition for data-driven and physics-informed problems. arXiv (Cornell University), Jun 2024. doi: htt...

  3. [2158]

    URL https://proceedings

    PMLR, 2022. URL https://proceedings. mlr.press/v205/majd23a.html. Max Planck Institute for Biogeochemistry. Jena climate dataset (2009–2016). https://www.kaggle. com/datasets/mnassrib/jena-climate,

  4. [3396]

    Gurobi Optimization, LLC

    IEEE, 2017. Gurobi Optimization, LLC. Gurobi Optimizer Reference Manual, 2024. URLhttps://www.gurobi.com. Heiderich, B., Schumacher, M.-L., and Huber, M. Training and verifying robust kolmogorov-arnold networks. In ICLR 2025 Workshop: VerifAI: AI Verification in the Wild,

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.