{"id":"1056cc90-743b-4d7d-a936-1f04ba3901d8","arxiv_id":"2501.00709","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"Replacing the weight matrix in SGCN with a KAN layer yields competitive but generally not superior performance on signed community detection and link sign prediction, with high variance across datasets.","lead":"This paper swaps the standard linear layer in a Signed Graph Convolutional Network for a Kolmogorov-Arnold Network layer and tests the new model on community detection and link sign prediction. Across seven signed graphs, the KAN-enhanced model performs comparably to the standard SGCN, with differences that are often small relative to run-to-run variation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported KASGCN-vs-SGCN differences sit within run-to-run noise; without significance or equivalence tests, the 'comparable' claim rests on absence of evidence, and the Congress AUC drop (-12.29%, Table 5) contradicts the 'consistent improvement' observation.","rationale":"The reader's weakest assumption correctly identifies the absence of significance tests and the reliance on a single hyperparameter configuration. My stress-test narrows this to the inferential step from 10-run means to the paper's hedged but still positive conclusion. The claimed 'competitive or comparable' performance is not falsified by the tables, but it is also not established in any statistical sense: the reported standard deviations are large enough that most observed gains and losses are consistent with chance. The Congress AUC result is the most concrete challenge to the stronger 'drop-in replacement' framing because it is the largest degradation and appears in the primary link-sign-prediction table. A paired bootstrap or equivalence test would settle whether this degradation is real or noise. I do not think this concern changes the reader's CONDITIONAL verdict; it reinforces it. I also note, but do not rely on, that the SGCN equations in Section 1.2 contain apparent typos (hU at l=1 aggregates from N+ instead of N-, and the l>1 hU update uses W_B(l) instead of W_U(l)), which further motivates an artifact release or a careful re-derivation during revision.","tokens_in":13442,"tokens_out":6091,"duration_ms":62293,"concrete_test":"Run a paired bootstrap over the 10 per-run scores for every dataset and metric in Tables 2-5 (and ideally Tables 6-9) to obtain 95% confidence intervals for the mean difference SGCN - KASGCN, or a TOST equivalence test with a margin such as 0.02 in AUC/F1/Q. Use the raw per-run values, not the summary means. If the Congress interval excludes zero in the negative direction, the 'no degradation' version of the claim fails for that dataset; if all intervals include zero, the honest conclusion is 'no detectable difference' rather than 'comparable' or 'consistent improvement'.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that KASGCN is a drop-in replacement for SGCN requires that KASGCN performance is not materially worse than SGCN. The only evidence is 10-run means and standard deviations in Tables 2-5 and 6-9, with no paired significance or equivalence tests. The observed differences are mostly within one standard deviation of the run-to-run variation (e.g., Congress AUC 0.570 +/- 0.055 vs 0.499 +/- 0.046 in Table 5; BitcoinAlpha F1 0.721 +/- 0.042 vs 0.690 +/- 0.094). Thus the data are compatible with both 'no difference' and 'moderate degradation' on individual graphs. The aggregate pattern favoring KASGCN on AUC is driven by small, non-significant gains. The one clear negative outlier, Congress AUC (-12.29%), directly contradicts the paper's observation in Section 3.2 of 'consistent improvement in AUC' and is not explained away by the reported spread. Without a pre-specified margin and a paired test, the headline conclusion is underdetermined by the evidence.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes KASGCN, an extension of the Signed Graph Convolutional Network (SGCN) in which the linear transformation in each layer is replaced by a Kolmogorov-Arnold Network (KAN) layer. The authors evaluate KASGCN against standard SGCN on seven signed graphs for two downstream tasks: community detection via K-means++ on node embeddings and link sign prediction via multinomial logistic regression. They also compare several KAN variants (Fourier, Laplace, Wavelet) and examine training efficiency and embedding similarity. The central empirical claim is that KASGCN achieves competitive or comparable performance to SGCN, with context-dependent variability.","tokens_in":13571,"tokens_out":4984,"duration_ms":47511,"significance":"If the claims are substantiated, the paper provides an early empirical datapoint on whether KAN layers can serve as drop-in replacements for linear transformations in signed GNNs. The work is honest in reporting means and standard deviations over 10 runs and in hedging its conclusions as context-dependent. However, the strength of the evidence is limited by the absence of significance or equivalence testing, the use of a clustering quality metric that conflates two components with different scales, and fixed hyperparameters without sensitivity analysis. The reported differences are often within run-to-run noise, so the 'comparable' conclusion is currently underdetermined.","major_comments":[{"comment":"The observation that KASGCN shows 'consistent improvement in AUC across most datasets' is not supported by the data: the Congress dataset shows a 12.29% AUC drop (0.570 ± 0.055 vs 0.499 ± 0.046), and the gains on other datasets are small relative to the reported standard deviations. Without paired significance tests (e.g., Wilcoxon signed-rank or paired t-test over the same seeds) or confidence intervals for the differences, the aggregate pattern could be consistent with random noise. Please report paired tests and effect sizes, and explicitly discuss the Congress outlier.","section":"Section 3.2, Table 5"},{"comment":"The clustering quality metric Q = posin + negout assigns equal weight to two components that have different scales and variances across datasets. For example, in Table 3 (K = 10), PPI shows KASGCN increasing negout from 0.784 ± 0.011 to 0.936 ± 0.009 while decreasing posin from 0.422 ± 0.037 to 0.364 ± 0.037; the reported +7.79% gain is driven primarily by the negout term. The authors should justify why equal weighting is appropriate or report the components separately with a clear interpretation of trade-offs.","section":"Section 3.1, Eq. (5)"},{"comment":"All hyperparameters (layers = [32,32], grid size = 5, spline order = 3, learning rate = 0.001, etc.) are fixed across datasets and models, and no sensitivity analysis is provided. Since the paper's central claim is that KASGCN is 'comparable' and 'context-dependent,' the observed differences could be an artifact of suboptimal KAN hyperparameters rather than an intrinsic property of the architecture. A sensitivity analysis over at least grid size and hidden layer size on one or two datasets is needed to support the robustness of the conclusions.","section":"Section 2.2, Tables 2-5"},{"comment":"The gain percentages in Tables 2-5 are computed as ratios of noisy quantities, but no uncertainty is propagated into the gains. For instance, the Congress AUC gain of -12.29% in Table 5 is derived from means with standard deviations of 0.055 and 0.046, and the confidence interval for the gain would likely include zero or even positive values. Please report absolute differences with confidence intervals, or apply a bootstrap/delta method to the gains, so that the reader can judge the stability of each reported gain.","section":"Section 3.1, Tables 2-5"},{"comment":"The efficiency claim that 'KAGCN takes significantly longer' (the text appears to mean KASGCN) is supported only by a visual inspection of Figure 2, with no quantitative runtime values in the text or a table. Please provide numerical results, such as mean training time and standard deviation for each aggregator layer, to make the efficiency comparison concrete and reproducible.","section":"Section 3.3, Figure 2"}],"minor_comments":[{"comment":"The text refers to 'KAGCN' in several places (e.g., 'KAGCN takes a significantly longer time' in Section 3.3 and the heading 'KAGCN Robustness' in Section 3.5); these should be corrected to 'KASGCN'.","section":"Sections 3.3 and 3.5"},{"comment":"The title appears as 'KAN KAN B UFF SIGNED GRAPH NEURAL NETWORKS?' which is likely a formatting artifact; it should be corrected to a readable phrase (e.g., 'KAN-Buff Signed Graph Neural Networks?').","section":"Title"},{"comment":"In Table 4, the PPI gain is reported as '8.74' without a percent sign; in Table 5, the WikiRFA F1 gain is reported as '-0.434' which should likely be '-0.43%'. Please ensure consistent formatting of percentages.","section":"Tables 4-5"},{"comment":"The reference [Anonymous, 2024] is an anonymous citation; a journal submission should cite the actual authors and venue for this work on GraphKAN.","section":"References"},{"comment":"The abstract states that KANs offer 'improved accuracy and interpretability with fewer parameters,' but the paper does not evaluate interpretability or parameter efficiency in any experiment; please soften this claim or add supporting evidence.","section":"Abstract and Section 1.1"},{"comment":"The near-zero cosine similarity between SGCN and KASGCN embeddings is an interesting finding, but the paper does not explain why such different embeddings can still yield comparable downstream performance. A brief discussion would be useful.","section":"Section 3.4"},{"comment":"No code availability statement is provided. Given that the KAN implementations rely on external repositories, a link to the experimental code would substantially improve reproducibility.","section":"Overall"}],"recommendation":"major_revision","confidential_remarks":"The paper is a proof-of-concept empirical study with a narrow novelty (first KAN integration into signed GNNs). The central 'comparable' claim is not yet supported by adequate statistical evidence, and the clustering quality metric needs justification. These issues are fixable within the manuscript's scope, so I recommend major revision rather than rejection. The paper would benefit from paired significance tests, a sensitivity analysis, and a clearer treatment of the PPI and Congress outliers."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe takeaway: this is a modest, honest paper. It does the first KAN-for-signed-GNN integration, replacing the weight matrix in SGCN with a KAN layer, and shows across seven signed graphs that the resulting KASGCN is roughly comparable to SGCN on community detection and link sign prediction. The authors do not oversell it; their conclusion says 'competitive or comparable' and 'context-dependent.'\n\nWhat's genuinely new: no prior work applies KANs to signed networks. The architecture is a direct transplant of the KAGNN idea, but that is fine for a proof of concept. The paper also compares four KAN variants (Original, Fourier, Laplace, Wavelet) and includes efficiency measurements and embedding similarity analysis. The experimental reporting is transparent: 10 runs with means and standard deviations, and the datasets are standard signed graph benchmarks.\n\nThe soft spots are real but not fatal. The main one is statistical: the differences between KASGCN and SGCN are mostly within one standard deviation, so the 'comparable' claim is an absence of evidence rather than demonstrated equivalence. No significance or equivalence tests are run. The Congress AUC drop (-12.29%) is a clear counterexample to the 'consistent improvement in AUC' phrasing, even if 'across most datasets' is technically accurate. The hyperparameters are fixed without sensitivity analysis, and no code or data are released, so the experiments are hard to build on. Comparison to other signed GNN baselines is missing; the only baseline is SGCN.\n\nI agree with the stress-test: the aggregate pattern is suggestive but underdetermined. The paper would be strengthened by paired tests or a pre-specified margin, error propagation in the gain percentages, and a sensitivity sweep on grid size and spline order.\n\nWho should read this: anyone working on KAN-based GNNs or signed graph learning. It is a legitimate data point, not a deep result.\n\nRecommendation: it deserves a serious referee. The questions it asks are valid, the evidence is honestly reported, and the weaknesses are fixable. I would not desk-reject it, but I would send it back for statistical rigor before publication.\n\nBest,\n[Your name]","headline":"An honest, incremental empirical study showing KAN layers can substitute for the linear map in SGCN with roughly comparable results; the main weakness is that the headline comparison is statistically underpowered.","tokens_in":14246,"tokens_out":2539,"would_cite":false,"duration_ms":24408,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"KAN layers can replace the linear transformations in Signed Graph Convolutional Networks with no systematic loss in link-sign prediction or community-detection quality.","keywords":["Kolmogorov-Arnold Networks","signed graph neural networks","signed community detection","link sign prediction","graph representation learning","balance theory","KASGCN"],"falsifier":"Run the same comparisons with per-dataset hyperparameter search and paired statistical tests on the 10 runs: if the Congress 12.29% AUC loss is reproduced and most of the reported gains fall within one standard deviation of run-to-run variation, the claim of competitive parity would no longer hold.","tokens_in":13119,"feed_emoji":"⚖️","tokens_out":5400,"duration_ms":47190,"temperature":0.7,"pith_summary":"Signed graphs encode trust and distrust, support and opposition, and Signed Graph Convolutional Networks (SGCNs) learn node embeddings by aggregating neighbors through two separate channels that reflect structural balance. The paper asks whether replacing the linear weight matrices in SGCN with Kolmogorov-Arnold Network (KAN) layers, which use learnable univariate spline functions instead of fixed activations, yields better or even comparable embeddings. It introduces KASGCN and tests it on community detection and link sign prediction across seven signed graphs. The central finding is parity: KASGCN mostly matches SGCN, with small AUC gains on most graphs and graph-dependent F1 results, while some KAN variants (Fourier, Wavelet) frequently lead and LaplaceKASGCN consistently underperforms. This matters because KANs promise greater expressivity with fewer parameters, so a drop-in replacement would offer a practical upgrade if the parity holds.","feed_headline":"KASGCN: swapping SGCN weights for KAN layers keeps results competitive","feed_subtitle":"New KASGCN architecture swaps SGCN linear layers for splines and stays on par across seven signed graphs","key_machinery":"The carrying object is the KAN layer $\\varphi$, a learnable univariate function built from B-splines, with a grid size and spline order controlling the approximation. In KASGCN it replaces the linear transformation $W$ in the SGCN recurrence for both the positive ($h^B$) and negative ($h^U$) node representations, with variants using Fourier, Laplace, and Wavelet bases instead of B-splines. The architecture keeps SGCN's balance-theory message passing, so the KAN layer is the only change from the baseline, which lets the experiments isolate the effect of the spline-based transformation.","core_discovery":"The paper claims that a KAN layer can take over the role of the weight matrix in the SGCN update equations without breaking the balance-theoretic structure that gives SGCN its meaning. In KASGCN, the linear transforms $W^{(l)}$ for updating positive and negative representations are replaced by KAN layers $\\varphi^{(l)}$, while the two-channel aggregation over positive and negative neighbors is preserved. In experiments on seven signed graphs, KASGCN achieves competitive or comparable performance to SGCN across signed community detection and link sign prediction: it improves AUC on six of seven datasets for link sign prediction, with gains up to 3.46% on WikiElec, but F1 improvements are mixed and one graph, Congress, shows a 12.29% AUC drop. The authors interpret the results as context-dependent effectiveness rather than general superiority of KAN-based signed GNNs.","pith_inferences":["Because the embeddings are near-orthogonal yet downstream scores are similar, KASGCN may be capturing different node features than SGCN; a natural test is whether KASGCN embeddings preserve balance-theoretic cycle structure as faithfully as SGCN's.","The hyperparameters were fixed once for all graphs, so tuning grid size, spline order, or layer width per dataset could change both absolute performance and the variant ranking; the observed parity is therefore a lower bound on what KAN-based signed GNNs could achieve.","A paired significance test across the 10 runs, rather than averaged metrics alone, would tell whether gains like 2-3% AUC are real or within run-to-run noise; the paper reports no such test."],"forward_implications":["KAN layers can act as a drop-in substitute for the linear map in SGCN: for AUC, KASGCN is not systematically worse than SGCN on the seven tested signed graphs.","The choice of KAN variant matters more than the choice between SGCN and KASGCN: FourierKASGCN and WaveletKASGCN often finish first in community detection or link-sign prediction, while LaplaceKASGCN is consistently last.","Near-zero average cosine similarity (between $-0.02$ and $0.07$) between SGCN and KASGCN embeddings shows the two models produce geometrically different embeddings, yet those different geometries lead to similar downstream performance.","Training time grows with the number of aggregator layers for KASGCN but not for SGCN, so parity comes at a computational cost that increases with depth."],"supporting_citations":[{"why":"Defines SGCN, the baseline model whose balance-theory message passing and weight matrices KASGCN modifies.","marker":"[Derr et al., 2018]"},{"why":"Introduces the original Kolmogorov-Arnold Network with learnable univariate spline functions that KASGCN uses as its layer.","marker":"[Liu et al., 2024]"},{"why":"Shows how to integrate KAN layers into unsigned graph neural networks, supplying the template for replacing weight matrices with KAN layers.","marker":"[Bresson et al., 2024]"},{"why":"Provides the Konect collection from which most of the signed graph datasets (BitcoinAlpha, BitcoinOTC, WikiElec, Chess, Congress) are taken.","marker":"[Kunegis, 2013]"},{"why":"Supplies the WikiRFA and PPI signed network datasets used in the evaluation.","marker":"[He et al., 2022]"},{"why":"Provides the efficient B-spline KAN implementation used for the original KAN layer in KASGCN.","marker":"[Blealtan, 2024]"},{"why":"Provides the Toy-KAN implementation used for the KAN variants, including LaplaceKASGCN.","marker":"[Yin et al., 2024]"},{"why":"Defines the FourierKAN variant that is tested as FourierKASGCN.","marker":"[Xu et al., 2024]"},{"why":"Defines the WaveletKAN variant that is tested as WaveletKASGCN.","marker":"[Bozorgasl and Chen, 2024]"}],"fun_headline_variants":["KAN layers replace SGCN weights, AUC up on 6/7 graphs","KASGCN: KAN-based signed GNNs competitive, but context matters","Splines in SGCN: modest gains, one 12% AUC drop","KAN-enhanced SGCN: promising but not universally better","KAN KAN replace SGCN? Sort of."],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's conclusion rests on the assumption that a single fixed hyperparameter configuration (layers [32,32], grid size 5, spline order 3, learning rate 0.001, seed 42) and 10-run averages without significance tests are enough to judge which model is better.","fun_headline_variants_meta":{"raw":{"variants":["KAN layers replace SGCN weights, AUC up on 6/7 graphs","KASGCN: KAN-based signed GNNs competitive, but context matters","Splines in SGCN: modest gains, one 12% AUC drop","KAN-enhanced SGCN: promising but not universally better","KAN KAN replace SGCN? Sort of."]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00032,"raw_usage":{"total_tokens":1783,"prompt_tokens":905,"completion_tokens":878,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":521,"completion_tokens_details":{"reasoning_tokens":782}},"tokens_in":521,"tokens_out":878,"duration_ms":7963,"temperature":1.0,"reasoning_tokens":782,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:44:04.328933+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same comparisons with per-dataset hyperparameter search and paired statistical tests on the 10 runs: if the Congress 12.29% AUC loss is reproduced and most of the reported gains fall within one standard deviation of run-to-run variation, the claim of competitive parity would no longer hold.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Konect collection from which most of the signed graph datasets (BitcoinAlpha, BitcoinOTC, WikiElec, Chess, Congress) are taken."},{"cited_title":"Efficient-KAN: An Efficient Implementation of Kolmogorov-Arnold Network","cited_arxiv_id":null,"evidence_quote":"Provides the efficient B-spline KAN implementation used for the original KAN layer in KASGCN."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Toy-KAN implementation used for the KAN variants, including LaplaceKASGCN."}],"review_version":1}