REVIEW 2 cited by
ReLU-KAN: New Kolmogorov-Arnold Networks that Only Need Matrix Addition, Dot Multiplication, and ReLU
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Limited by the complexity of basis function (B-spline) calculations, Kolmogorov-Arnold Networks (KAN) suffer from restricted parallel computing capability on GPUs. This paper proposes a novel ReLU-KAN implementation that inherits the core idea of KAN. By adopting ReLU (Rectified Linear Unit) and point-wise multiplication, we simplify the design of KAN's basis function and optimize the computation process for efficient CUDA computing. The proposed ReLU-KAN architecture can be readily implemented on existing deep learning frameworks (e.g., PyTorch) for both inference and training. Experimental results demonstrate that ReLU-KAN achieves a 20x speedup compared to traditional KAN with 4-layer networks. Furthermore, ReLU-KAN exhibits a more stable training process with superior fitting ability while preserving the "catastrophic forgetting avoidance" property of KAN. You can get the code in https://github.com/quiqi/relu_kan
Forward citations
Cited by 2 Pith papers
-
Sinusoidal Approximation Theorem for Kolmogorov-Arnold Networks
The paper states universal approximation theorems for sine-based Kolmogorov-Arnold networks, but the proof relies on an invalid lemma and an unproven matrix invertibility.
-
Leveraging KANs for Expedient Training of Multichannel MLPs via Preconditioning and Geometric Refinement
Training in a B-spline KAN basis is equivalent to preconditioned gradient descent on a multichannel ReLU MLP, and geometric refinement plus trainable knots accelerate and improve training.
Discussion (0). Sign in to comment.