{"id":"db02f0ad-ebc5-4635-b16a-2266c06cf04f","arxiv_id":"2502.15704","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"EMK-KEN, a Mamba-KAN hybrid classifier, is reported to outperform six baselines on knowledge-value classification across ten citation datasets.","lead":"A new model called EMK-KEN combines two neural-network techniques, Mamba and KAN, to score how much knowledge a scientific paper contributes using citation-network data. The authors report higher accuracy, F1, and AUC than six baseline models on ten citation datasets from computer science and biomedicine.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported SOTA claim is not internally reproducible: Table 3's 'Whole model' results differ significantly from Table 2's EMK-KEN results on the same six datasets (e.g., Oxytocin 73.68 vs. 77.41 ACC), so the empirical superiority cannot be trusted until the discrepancy is resolved.","rationale":"I focused on the empirical claim because the paper's contribution is 'outperforms state-of-the-art in effectiveness and robustness'; the architecture details are secondary if the experiments cannot be trusted. The Table 2 vs. Table 3 discrepancy is the strongest evidence of unreliability because it is internal to the manuscript: the same named model earns materially different scores on the same datasets. A leakage-based explanation is plausible but depends on unstated feature definitions; the table conflict is already documented in the text. This does not prove the method is bad, but it means the central claim cannot be accepted as written. The reader's CONDITIONAL verdict is appropriate: the discrepancy is resolvable by releasing code/data/hyperparameters and reconciling the two tables. I therefore keep the verdict unchanged. The reader flagged leakage as the weakest assumption and mentioned the table conflict only in passing; I elevate the table conflict to the primary concern, so my agreement is partial.","tokens_in":17148,"tokens_out":6823,"duration_ms":74190,"concrete_test":"Run one fixed protocol: release the exact train/validation/test split, seeds, and per-dataset d_state1/d_state2 values from Figure 6, then re-run EMK-KEN on Citation Networks-V12 and Oxytocin using the Table 2 and Table 3 'Whole model' settings. If the ACC differences (1.21 and 3.73 points) persist under an identical configuration, the tables describe different setups and the main comparison is undefined. As a second verification, inspect the Fmeta-dimensional metadata vector to determine whether in-degree or reference-network node/edge counts are present as input features; if they are, the Section 3 label construction makes the classification task trivially learnable and the reported gains are not meaningful.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4's ablation table (Table 3) reports a 'Whole model' row described as the complete EMK-KEN, but its numbers do not match the EMK-KEN row in the main results (Table 2) on any of the six overlapping datasets. Examples (ACC): DBLP-V13 70.93±0.85 vs. 72.03±0.38; Citation Networks-V12 82.43±0.36 vs. 83.64±0.35; OGBN-ArXiv 84.12±0.46 vs. 81.89±0.88; Biomedicine 81.51±0.33 vs. 83.33±0.67; Oxytocin 73.68±0.27 vs. 77.41±0.65; SNAP-HEP-TH 76.20±0.57 vs. 75.31±0.39. For Oxytocin and Citation Networks-V12 the gaps are far outside the reported standard deviations. The paper gives no explanation (e.g., different epochs, seeds, or hyperparameters), and no code, data splits, or per-dataset d_state settings are provided. If the two tables are supposed to describe the same model, at least one set of results is not reproducible; if they differ by design, the paper does not say so. Since the central claim of consistent SOTA effectiveness rests entirely on Table 2, this internal inconsistency is a load-bearing correctness problem. Separately, Section 3 labels datasets (1)-(6) using in-degree and reference-network size without stating whether those quantities are in the input metadata, leaving an unresolved leakage risk; but the table conflict is directly observable and does not depend on that ambiguity.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes EMK-KEN, a model for assessing the knowledge value of academic papers from citation networks. The model combines a Mamba-based sequence processor (MamST) with a KAN-based network unit (KNU) to capture semantic features from node metadata and text embeddings and structural information from the citation graph. The authors introduce a Knowledge Quantization/Quantification Index (KQI) computed from graph structural entropy and use it as classification labels for two of the ten datasets. They report results on ten citation-network datasets, claiming that EMK-KEN outperforms six baseline models in accuracy, F1, and AUC. The central claim is that EMK-KEN is both more effective and more robust than existing GNN- and Mamba-based methods.","tokens_in":17576,"tokens_out":4516,"duration_ms":47685,"significance":"If the reported results were reproducible, the paper would make a useful contribution by demonstrating a practical combination of Mamba and KAN for citation-network classification, with a linear-time sequence model and learnable nonlinear activations. The KQI-based labeling scheme, specified in Section 3, is an interesting attempt to operationalize 'knowledge value' from structural entropy. The paper also provides a fairly detailed description of the two-module architecture and an ablation study over six architectural components. However, the experimental evidence is currently not self-contained: no code, data splits, baseline configurations, or final hyperparameter values are given, and the main results table conflicts with the ablation table. These issues prevent the reader from verifying the central SOTA claim, so the significance of the contribution cannot be assessed on the evidence presented.","major_comments":[{"comment":"The 'Whole model' row of the ablation study (Table 3) is inconsistent with the EMK-KEN row in the main results (Table 2) on every dataset that appears in both tables. For example, ACC values differ as follows: DBLP-V13 70.93±0.85 vs 72.03±0.38; Citation Networks-V12 82.43±0.36 vs 83.64±0.35; OGBN-ArXiv 84.12±0.46 vs 81.89±0.88; Biomedicine 81.51±0.33 vs 83.33±0.67; Oxytocin 73.68±0.27 vs 77.41±0.65; SNAP-HEP-TH 76.20±0.57 vs 75.31±0.39. Some of these gaps exceed the reported standard deviations by a wide margin. This is load-bearing because the paper's central claim of consistent superiority over baselines rests on Table 2. The authors must either explain why the two tables describe different configurations (e.g., different hyperparameters, epochs, or seeds) or provide code and experimental logs that resolve the discrepancy. Without such an explanation, at least one set of results is unreliable.","section":"Section 4, Tables 2 and 3"},{"comment":"The paper states that for datasets (1) through (6) the reference network was classified 'based on the in-degree of the central node, as well as the number of nodes and edges in the reference network, before being fed into the model for prediction.' It does not specify whether these structural quantities are also included in the metadata feature vector given to the model. If in-degree and edge/node counts are part of the input, the classifier can directly memorize the ground-truth label from the same features, making the reported performance trivial. The paper also does not describe the train/test split, which matters particularly for datasets (7) and (8) where KQI is computed on the full graph; if the same graph structure is used during training and testing, label information may leak between neighborhoods. Please clarify the exact input features and the split procedure for each dataset.","section":"Section 3, label construction and feature leakage"},{"comment":"The hyperparameter analysis (Figure 6) shows that performance is sensitive to d_state1 and d_state2 and that optimal values are selected per dataset, but the final values used for the results in Table 2 are never reported. The regularization coefficient λ and all training details (learning rate, batch size, epochs, optimizer, dropout rates) are also omitted. Without these values, and without code or data splits, the results in Table 2 cannot be reproduced or independently verified. The authors should provide the exact configuration for every dataset, either in the paper or as supplementary material.","section":"Section 4, hyperparameter analysis"},{"comment":"The text alternates between 'Knowledge Quantization Index' and 'Knowledge Quantification Index' (the latter appears in the body around Equations 14 and 15). More substantively, the KQI is computed from graph-structural quantities (in-degrees, out-degrees, volume terms in Equations 14 and 15) and is then used as the classification label for the same citation graph that the model consumes. If these same structural quantities appear in the node features, the label is a deterministic function of the input, which would make the classification task ill-posed as a measure of knowledge evaluation. Please clarify which features are used and whether KQI is the only label construction for datasets (9) and (10).","section":"Section 3, KQI definition and terminology"}],"minor_comments":[{"comment":"The text contains typos: 'd estate2' should be 'd_state2', and 'seq 1en' should be 'seq_len' where it appears in the metadata processing description.","section":"Section 2.3 and Figure 6"},{"comment":"The terms 'Knowledge Quantization Index' and 'Knowledge Quantification Index' are used inconsistently; please choose one and use it throughout.","section":"Section 3"},{"comment":"Algorithm 1 writes 'Hmeta = W fcFmeta + bfc', which omits the ReLU activation present in Eq. (1) and does not match the dimension mapping described in the text. Please align the pseudo-code with the equation.","section":"Algorithm 1 and Eq. (1)"},{"comment":"Some dataset statistics look suspiciously identical (e.g., avgdegreemean for ACMHypertextECHT and OGBN-ArXiv are both 3.209401105); please verify that the values are correct and that the table columns correspond to the datasets listed in the text.","section":"Table 1"},{"comment":"The caption does not define d_state1 and d_state2; please add a sentence explaining that these are the state dimensions for the metadata and text branches of the Mamba module.","section":"Figure 6 caption"}],"recommendation":"major_revision","confidential_remarks":"The paper presents an interesting model combination, but the empirical evidence is not yet reliable. The Table 2 vs Table 3 discrepancy is a serious internal inconsistency that must be resolved before any claim of state-of-the-art performance can be accepted. I would encourage the authors to release code and data splits; without them, the area of citation-network knowledge evaluation will be no better off than before. This manuscript is borderline for the journal's scope; if the empirical issues are resolved, it could be a reasonable fit."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The new thing here is the specific combination: Mamba handles metadata and text embeddings separately, then KAN processes the central node representation, with labels derived from a structural-entropy KQI index. That is a reasonable architecture to try, and the paper describes it clearly enough to reimplement in principle. The benchmark suite is wide (ten datasets, three metrics, six baselines), and the ablation study at least attempts to isolate each component. If the numbers held up, this would be a useful contribution for large-scale literature screening.\n\nBut the numbers do not hold up as reported. Table 3's \"Whole model\" row is supposed to be the complete EMK-KEN, yet it disagrees with the EMK-KEN row in Table 2 on all six overlapping datasets, sometimes by a lot: Oxytocin 73.68 vs 77.41 ACC, OGBN-ArXiv 84.12 vs 81.89, DBLP-V13 70.93 vs 72.03. The text gives no explanation (different epochs? seeds? hyperparameters?), and the gaps are far outside the reported standard deviations. Since the central claim of consistent SOTA effectiveness rests entirely on Table 2, this is a load-bearing inconsistency, not a cosmetic typo.\n\nBeyond that, the paper provides no code, no data splits, no baseline configurations, and no per-dataset hyperparameter settings (d_state1, d_state2, lambda, KQI thresholds), so none of the numbers can be independently checked. The hyperparameter sensitivity section even says different datasets need different values, but the actual values are not given. There is also an unresolved leakage risk: datasets 1-6 are labeled from in-degree and node/edge counts of the reference network, and the paper never states whether those quantities are part of the input metadata. If they are, the model could memorize the target. I am not asserting it does, but the paper should rule it out.\n\nWhat is good: the writing is readable, the architecture rationale is coherent, and the ablation table (inconsistency aside) shows each component matters. The problems are empirical transparency and internal consistency, both fixable in principle. As it stands, though, the evidence does not support the abstract's \"extensive experiments show it outperforms SOTA.\"\n\nI would not cite this yet, and I would not send it to a serious referee until the authors release code and data, report the actual hyperparameters, and reconcile Tables 2 and 3. Desk reject with an invitation to resubmit after that work is done.","headline":"Plausible Mamba+KAN architecture, but the SOTA claim is undercut by an internal table conflict and missing artifacts.","tokens_in":18077,"tokens_out":1941,"would_cite":false,"duration_ms":21147,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A hybrid of Mamba's selective state-space layers and KAN's learnable activations classifies the knowledge value of papers in citation networks more accurately than six existing models on all ten benchmark datasets tested.","keywords":["citation networks","knowledge value assessment","Mamba","Kolmogorov-Arnold networks","node classification","structural entropy","Knowledge Quantization Index","text embeddings"],"falsifier":"Compute the same ten benchmark experiments with node features stripped of in-degree, node count, edge count, and any derived structural statistics, and with a temporal train/test split so KQI is only computed from training data; if EMK-KEN's margin over the baselines shrinks to the noise level, the performance claim is an artifact of label leakage rather than knowledge-value learning.","tokens_in":16968,"feed_emoji":"📈","tokens_out":7544,"duration_ms":80418,"temperature":0.7,"pith_summary":"EMK-KEN is an attempt to make knowledge-value assessment of academic papers fast, accurate, and transferable across research fields. It claims that a hybrid of Mamba, which processes node metadata and text embeddings as sequences in linear time, and KAN, whose learnable activations capture structural patterns in citation neighborhoods, classifies a paper's knowledge level better than whole-graph GNN approaches. If the claim holds, citation-network evaluation no longer needs to model the entire graph, which would make large-scale literature assessment practical and less computationally costly. The paper supports the claim with experiments on ten datasets spanning computer science and biomedicine, using three label schemes and reporting the best accuracy, F1, and AUC among seven models on every comparison.","feed_headline":"Hybrid Mamba-KAN model leads ten citation benchmarks","feed_subtitle":"Selective state-space text processing plus learnable KAN activations beats six baselines across 10 datasets.","key_machinery":"The central mechanism is the pairing of a Mamba layer with a KAN layer. Mamba is a selective state-space sequence model: it projects and splits the input, applies a causal convolution, and updates a hidden state through learned matrices $A$, $B$, $C$, and $D$, giving linear-time processing of long sequences and selective propagation or forgetting of information. KAN (Kolmogorov-Arnold network) replaces fixed activation functions with learnable B-spline interpolation, providing fine-grained nonlinear transformations with relatively few parameters. The connective step is the central-node extraction: only the target paper's Mamba outputs are passed to KAN, so the model learns from a paper's direct reference neighborhood instead of the whole citation graph. The same Mamba-KAN pipeline is applied separately to metadata and to text embeddings, and the two KAN outputs are fused for classification.","core_discovery":"EMK-KEN is a two-module network. The Mamba Long Order Processor (MamST) takes each node's metadata through a feature preprocessor (MetaFP) with relative position encoding, and takes text embeddings from a pretrained language model; both streams go through Mamba's selective state-space layers with causal convolution. The KANflex Neural Unit (KNU) then extracts the central node's output from each stream and applies a KAN layer, whose B-spline basis functions serve as learnable activations, to capture the structural information of the citation neighborhood. The metadata and text outputs are concatenated and passed through a linear layer with softmax to produce the knowledge-level classification. The paper claims this design beats six comparison models on ten datasets, with labels defined by citation-network complexity (in-degree of the central node and counts of nodes and edges in the reference network), by dataset-provided classes, and by the Knowledge Quantization Index (KQI) derived from graph structural entropy; for example, on Citation Networks-V12 it reports 83.64% accuracy versus 80.78% for MambaMLP.","pith_inferences":["The fairest control the paper does not report is to strip in-degree, node counts, edge counts, and derived structural statistics from the node features, and to recompute KQI labels using only training-time graphs; if the accuracy margin vanishes, the reported advantage comes from label leakage rather than from learning knowledge value.","The same architecture could be transferred to other networked artifacts with metadata, text, and local structure, such as patents, clinical guidelines, or software dependency graphs, because it never needs the full graph.","A natural external validation would test whether KQI classes and EMK-KEN's predictions track future citations or expert judgments of importance; the paper does not connect knowledge value to downstream impact."],"forward_implications":["On the ten datasets tested, the model reports the highest accuracy, F1, and AUC under all three label schemes, including the structural-entropy-based KQI labels.","Because Mamba processes sequences in linear time and only the central node goes to KAN, the approach scales to very large citation networks without full-graph message passing.","Ablation experiments attribute the gains to specific components: removing the causal convolution, state-space model, Mamba block, KAN layer, or the post-KAN dropout each lowers accuracy, often sharply.","The Knowledge Quantization Index, computed from graph structural entropy, is used as a classification target, so the model is effectively predicting a quantified knowledge value rather than raw citation counts."],"supporting_citations":[{"why":"Supplies the Mamba selective state-space layer that processes metadata and text embeddings in linear time.","marker":"[23]"},{"why":"Supplies the KAN layer with learnable B-spline activations used to capture structural information.","marker":"[27]"},{"why":"Supplies the pretrained language model embeddings used as the text input to the Mamba branch.","marker":"[32]"},{"why":"Source of the Citation Networks-V12 dataset and part of the in-degree-based classification scheme used for the first six datasets.","marker":"[40]"},{"why":"Basis for classifying the first six datasets by in-degree of the central node and by node and edge counts in the reference network.","marker":"[52]"},{"why":"Supplies the OGBN-ArXiv dataset and its provided classification labels.","marker":"[48]"},{"why":"Defines the Knowledge Quantization Index used to label the AMN and Biomedicine datasets.","marker":"[53]"},{"why":"Provides the theory and methodology of quantifying knowledge that supports KQI as a label.","marker":"[54]"}],"fun_headline_variants":["Mamba-KAN hybrid tops 10 citation benchmarks","EMK-KEN: two-module network beats six baselines","Mamba plus KAN leads citation value ranking","Two-module Mamba-KAN wins on 10 citation sets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the structural quantities used to build classification labels are not visible to the classifier; if in-degree or node and edge counts are among the node features, or if the Knowledge Quantization Index is computed on graphs that include test nodes, the model can memorize the labels and the reported gains become an artifact.","fun_headline_variants_meta":{"raw":{"variants":["Mamba-KAN hybrid tops 10 citation benchmarks","EMK-KEN: two-module network beats six baselines","Mamba plus KAN leads citation value ranking","Two-module Mamba-KAN wins on 10 citation sets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000695,"raw_usage":{"total_tokens":3128,"prompt_tokens":916,"completion_tokens":2212,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":532,"completion_tokens_details":{"reasoning_tokens":2153}},"tokens_in":532,"tokens_out":2212,"duration_ms":18299,"temperature":1.0,"reasoning_tokens":2153,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:48:07.896270+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the same ten benchmark experiments with node features stripped of in-degree, node count, edge count, and any derived structural statistics, and with a temporal train/test split so KQI is only computed from training data; if EMK-KEN's margin over the baselines shrinks to the noise level, the performance claim is an artifact of label leakage rather than knowledge-value learning.","supporting_citations":[{"cited_title":"Scibert: A pretrained language model for scientific text,","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained language model embeddings used as the text input to the Mamba branch."},{"cited_title":"An overview of microsoft academic service (mas) and applications,","cited_arxiv_id":null,"evidence_quote":"Source of the Citation Networks-V12 dataset and part of the in-degree-based classification scheme used for the first six datasets."},{"cited_title":"Identifying node role in social network based on multiple indicators,","cited_arxiv_id":null,"evidence_quote":"Basis for classifying the first six datasets by in-degree of the central node and by node and edge counts in the reference network."},{"cited_title":"Ogbn-arxiv (processed for pyg)","cited_arxiv_id":null,"evidence_quote":"Supplies the OGBN-ArXiv dataset and its provided classification labels."},{"cited_title":"Quantifying the value of knowledge within the context of product development,","cited_arxiv_id":null,"evidence_quote":"Defines the Knowledge Quantization Index used to label the AMN and Biomedicine datasets."},{"cited_title":"A theory and methodology to quantify knowledge,","cited_arxiv_id":null,"evidence_quote":"Provides the theory and methodology of quantifying knowledge that supports KQI as a label."}],"review_version":1}