{"id":"7f0d3649-46c8-4ffc-83c9-d0c73953958e","arxiv_id":"2502.05864","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A node-wise multi-view ensemble distillation method lets plain MLPs match or beat multiplex GNN teachers on 5 of 6 datasets while running 35-89x faster.","lead":"The paper trains MLPs to imitate multiplex graph neural networks by distilling soft labels from each edge-type view, getting near-GNN accuracy at MLP speed. It also learns per-node weights that decide how much to trust each view, making the distillation slightly interpretable.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Transductive accuracy parity relies on training the student on soft labels for the evaluation nodes; on a purely inductive split MGFNN+ drops 3.9–11.2 points below the teacher, so the deployment claim is not established.","rationale":"The reader's weakest assumption is that node features carry enough information about labels for an MLP to succeed. My concern is closely related but more specific about the evidence: the transductive protocol provides the student with the teacher's soft label for every evaluation node, so the favorable 5/6 result does not test whether features alone generalize. Table 3's inductive rows are the direct test, and they show substantial drops (up to 11.24 points on ArXiv, 7.14 on MAG). The paper does report these inductive numbers and briefly acknowledges distribution shift, but the abstract and central claim are stated without that scope restriction, and the interpolated 'prod' metric obscures the inductive deficit by weighting transductive accuracy at 80%. This is the single most load-bearing issue because the motivating use case is inference efficiency in latency-sensitive applications, which typically involves queries whose teacher soft labels were not available at distillation time. Other issues, such as the entropy regularization notation, the low-rank reparameterization with m in {1,2,3}, and the overlap with the authors' prior work, are addressable or secondary. Because the paper's contribution is still valuable for static transductive graphs and the inductive limitations can be fixed by scoping the claims, the reader's CONDITIONAL verdict remains appropriate; no verdict change is needed.","tokens_in":13869,"tokens_out":9505,"duration_ms":102445,"concrete_test":"Withhold teacher soft labels for a random 20% of unlabeled nodes on ArXiv and MAG, train MGFNN+ only with labels for training and observed nodes, then evaluate only on the withheld nodes (repeat across 5 seeds). Compare withheld-node accuracy to the transductive numbers in Table 2 (78.25 on ArXiv, 60.32 on MAG) and to the inductive numbers in Table 3 (66.75 on ArXiv, 55.79 on MAG). If withheld-node accuracy is close to the inductive numbers rather than the transductive numbers, target-node pseudo-label exposure drives the claimed parity and the headline should be conditioned on transductive/static deployment.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that an MLP can replace the MGNN with little or no accuracy loss is supported mainly by transductive experiments. In those experiments, teacher soft labels z_v are generated for every node, including the unlabeled/test nodes, and the student is trained to match them via the KL term over all V in Eq. (6). Thus the student has already seen the teacher's prediction on each evaluation node; transductive accuracy measures fit to those target pseudo-labels, not true generalization to unseen queries. When the same student is evaluated on nodes whose soft labels were not used during distillation (Table 3, ind rows), it falls behind RSAGE by 5.32, 3.87, 4.65, 4.75, 11.24, and 7.14 points on ACM, IMDB, IMDB5K, DBLP, ArXiv, and MAG, respectively. The paper's production metric 'prod = 0.2*ind + 0.8*tran' dilutes these deficits, and the abstract's unqualified 'accurate' claim omits this caveat. Since the deployment scenario motivating the work is latency-sensitive inference on new/unseen nodes, the relevant accuracy is the inductive one, where the paper's own assumption that node features alone suffice is weakest. This is not an external disagreement about consensus; it is an internal tension, because the transductive protocol makes that assumption appear true by construction.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MGFNN and MGFNN+, MLP students distilled from multiplex GNN (MGNN) teachers. MGFNN distills from the teacher's final soft labels, while MGFNN+ additionally distills per-view soft labels using node-wise ensemble coefficients learned through a low-rank reparameterization with entropy regularization. Experiments on six multiplex graph datasets report accuracy close to or above the teacher MGNN in transductive and 'production' settings, together with 35.40x–89.14x inference speedups over the teacher, and visualizations of node-wise coefficients for interpretability.","tokens_in":14183,"tokens_out":5058,"duration_ms":51658,"significance":"If the accuracy claims are properly scoped, the paper makes a useful contribution to GNN-to-MLP distillation for multiplex graphs. Its concrete strengths are that it is the first to target multiplex GNNs in this distillation paradigm, the low-rank node-wise coefficient design gives a parameter-efficient way to combine per-view teachers, the ablation against view-wise ensemble variants is informative, and the evaluation covers six datasets, multiple teacher architectures, inductive split rates, and feature noise. The released code is another positive. However, the headline 'accurate' claim is currently supported mainly by transductive numbers in which every evaluation node's teacher soft label is used during student training, and the inductive results show consistent deficits; this makes the deployment claim, and the unqualified abstract language, stronger than the evidence justifies.","major_comments":[{"comment":"The transductive protocol generates soft labels for every node in V (including the nodes later used for evaluation), and Eq. (6) trains the student on the KL term over all v in V. Thus the reported transductive accuracy measures fit to teacher pseudo-labels on the evaluation nodes, not generalization to unseen queries. In the inductive rows of Table 3, MGFNN+ trails RSAGE by 5.32, 3.87, 4.65, 4.75, 11.24, and 7.14 points on ACM, IMDB, IMDB5K, DBLP, ArXiv, and MAG. The interpolated metric prod = 0.2*ind + 0.8*tran dilutes these deficits, and the abstract's unqualified 'accurate' claim omits them. The deployment claim should be restricted to the transductive setting unless the inductive behavior is analyzed and addressed.","section":"Section 5.1, Table 3, Abstract"},{"comment":"The entropy regularization is not well-defined as written. The text defines c = (1/(n(r+1))) sum_{v,i} c_i^v, which is a scalar, but then writes H(c) = - sum_{i=1}^{r+1} c_i log c_i, which requires c_i as a vector of per-view averages. The loss in Eq. (6) is therefore ambiguous and cannot be reproduced as a precise objective. Please define c_i = (1/n) sum_v c_i^v, state the normalization (e.g., whether c is a probability vector), and rewrite Eq. (6) accordingly.","section":"Section 4.2, Eq. (6)"},{"comment":"The information-theoretic justification in Section 4.3 rests on the assumption that I(X[v]; y_v | E[v]) is large, but the paper provides no empirical measure of this quantity or of feature-structure correlation. The paper's own inductive results on ArXiv and MAG (Table 3) show the regime where the assumption fails, and Section 5.2 attributes the drop to 'distribution shift.' This is an internal tension: the paper uses the transductive setting, where teacher soft labels for test nodes are available, to support a general 'accurate' claim, while the inductive setting is the one most relevant to the latency-sensitive deployment scenario. Please either add an analysis of when feature-only students can match MGNNs or substantially qualify the claims.","section":"Section 4.3, Section 5.2, Table 3"}],"minor_comments":[{"comment":"The text says 'we decompose C = ST, where U in R^{n x m} and T in R^{m x (r+1)}' but the first matrix is later used as S. Please fix the variable name to S.","section":"Section 4.2, Eq. (5)"},{"comment":"The last contribution bullet contains a duplicated phrase: 'Experiments on six datasets show MGFNNs that MGFNNs achieve...' Please correct this typo.","section":"Section 1, Contributions"},{"comment":"The sentence 'these view-wise coefficients only reflect the global importance of each view without but make no specific discrimination for individual nodes' has a grammatical error and should be rewritten.","section":"Section 4.2"},{"comment":"The caption contains a typo: 'Classificatiom' should be 'Classification'.","section":"Table 2 caption"},{"comment":"The y-axis appears to show duplicated tick labels ('60 65 70' repeated). Please check the figure export.","section":"Figure 5(b)"}],"recommendation":"major_revision","confidential_remarks":"The central idea is reasonable and the experiments are substantial, but the paper's main claim is overstated because transductive accuracy in this setup includes teacher soft labels for the evaluation nodes. The fix is within scope: re-scope the claims, report inductive results prominently, and repair the entropy-regularization definition. I therefore recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, it does something genuinely new: it is the first GNN-to-MLP distillation framework aimed at multiplex graphs, and the low-rank reparameterization of node-wise ensemble coefficients is a clean, practical trick. Second, the central accuracy claim is not as solid as the abstract suggests. The transductive results—where MGFNN+ matches or beats the MGNN teacher on 5/6 datasets—are obtained by training the student on teacher soft labels for every node, including the test nodes. That is not a generalization measurement; it is fitting to pseudo-labels. The inductive results in Table 3 are the honest ones, and there the student falls behind RSAGE by 3.9 to 11.2 points across all six datasets. The paper's \"prod = 0.2*ind + 0.8*tran\" metric hides this, and calling the method \"accurate\" without qualification is an overclaim.\n\nWhat is good: the motivation is sound, the method is clearly explained, the ablation against view-wise ensemble variants (MEAN, PARA, ATTN, GRAD) is reasonable, and the speedups of 35-89x are real. The code is public. The low-rank coefficient learning is an elegant way to avoid O(n) parameters while staying node-adaptive. All of that deserves credit.\n\nSoft spots, in rough order of importance. The evaluation protocol is the main issue: if the deployment scenario is latency-sensitive inference on new nodes, the inductive accuracy is the relevant metric, and it is consistently worse. The discussion of ArXiv/MAG attributes the gap to distribution shift, but the same pattern appears on the small datasets too, so that explanation is incomplete. Second, there is a mathematical inconsistency in the entropy regularization: c is defined as a scalar average, then H(c) is written as a sum over i of c_i log c_i. The intent is probably a per-view average coefficient vector, but as written it is incoherent. Fixable, but sloppy. Third, the paper does not discuss its relationship to the authors' own prior work on heterogeneous graph distillation (ref [12]), which may be a close cousin. Fourth, there are no adapted strong baselines like NOSMOG or VQGraph for multiplex graphs, so the comparison to the state of the art in GNN-to-MLP distillation is incomplete.\n\nWho is this for? Researchers working on graph inference acceleration and practitioners who care about multiplex graphs in latency-sensitive settings. It deserves a serious referee, but the referee should push for a major revision: report inductive accuracy as the primary result, reframe the claims, fix the entropy notation, and clarify the relationship with [12]. If those are addressed, the paper would be a solid contribution; as is, the headline does not match the evidence.","headline":"First multiplex GNN-to-MLP distillation with a neat low-rank node-wise ensemble, but the headline accuracy claim is built on a transductive protocol that leaks teacher predictions into the evaluation nodes; the inductive numbers tell a more cautious story.","tokens_in":838,"tokens_out":1109,"would_cite":true,"duration_ms":33586,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that multiplex-GNN knowledge can be moved into a feature-only MLP by distilling each view's soft labels with per-node weights, matching or beating the teacher on five of six transductive datasets while running…","keywords":["knowledge distillation","multiplex graph neural networks","GNN-to-MLP distillation","inference acceleration","node-wise multi-view ensemble","low-rank reparameterization","multiplex graphs","efficiency and accuracy"],"falsifier":"Retrain on a multiplex graph whose labels are a pure function of network position (for instance, class equals degree bucket or triangle membership) with node features randomly shuffled so features carry no label information: the paper's account predicts MGFNN+ collapses toward the vanilla MLP level while the teacher keeps its accuracy. Observing instead that the student tracks the teacher would refute the paper's explanation of why the method works.","tokens_in":13681,"feed_emoji":"⚡","tokens_out":14935,"duration_ms":128567,"temperature":0.7,"pith_summary":"The paper seeks to settle a deployment trade-off: multiplex graph neural networks (MGNNs) are accurate but slow at inference because they must fetch and aggregate neighbors across every edge type, while plain MLPs are fast but weak. Its proposal is to distill the teacher MGNN's knowledge into an MLP student that sees only node features, so that fast inference keeps most of the accuracy. The first model, MGFNN, does this with the teacher's final soft labels alone; the second, MGFNN+, also distills each view-specific GNN's own soft labels and learns a per-node weight vector that decides how much to trust each view. The reported result is that the student matches or beats the teacher on five of six transductive datasets, improves over vanilla MLPs by about ten percentage points on average, and runs 35.40×–89.14× faster than the teacher. If the claim holds, latency-sensitive applications on multiplex graphs can switch to feature-only models without sacrificing accuracy.","feed_headline":"Distilled MLPs match multiplex GNNs, 35-89x faster","feed_subtitle":"Node-wise multi-view ensemble distillation pushes per-view semantics into a feature-only MLP, enabling 35-89x faster inference.","key_machinery":"The load-bearing mechanism is a low-rank reparameterization of the node-wise ensemble coefficient matrix $C \\in \\mathbb{R}^{n \\times (r+1)}$, where $n$ is the number of nodes and $r$ the number of views. Rather than learning one weight per node per teacher, which would cost $O(n(r+1))$ parameters and be hard to optimize because each row receives gradients only from its own node, the paper factors $C = ST$: $T \\in \\mathbb{R}^{m \\times (r+1)}$ holds $m$ globally shared 'base view-wise weight assigners,' and $S = \\tanh(HW)$ is computed from the student MLP's last hidden layer $H$, so each node's coefficient vector is a weighted combination of the base assigners. This reduces the parameter cost to $O(m(r+1) + hm)$, makes every coefficient differentiable with respect to shared weights $W$, and ties the coefficients to node features, which is what makes them interpretable per node. A mean-entropy regularization term $\\mathcal{H}(\\bar{c})$ on the average coefficient vector stops the optimizer from collapsing a node onto a single teacher, preserving diversity among the distilled views.","core_discovery":"The central claim is that the teacher's fused prediction $z_v$ is a bottleneck rather than a ceiling. Because an MGNN's integration layer combines the view-specific predictions $z_v^i$ with global, node-agnostic weights, it discards node-specific information; a student that learns its own per-node mixing of the same view predictions can therefore end up at or above the teacher's accuracy. An oracle experiment supports this premise: on ACM, an ideal ensemble that is correct whenever any one view-specific GNN is correct reaches 97.55% versus the fused teacher's 87.53%, quantifying the lost headroom. MGFNN+ harvests this headroom by treating each view-specific GNN together with the whole MGNN as $r+1$ teachers, minimizing a per-node KL loss $\\sum_i c_v^i \\mathrm{KL}(\\hat{y}_v, z_v^i)$ with coefficients learned from the student's own hidden features and regularized by mean-entropy maximization. In the reported experiments the student matches or beats the teacher on five of six transductive datasets (89.10% versus 87.92% on ACM) and remains competitive on five of six production settings, while the inductive columns on the two largest datasets, ArXiv and MAG, drop sharply (66.75% versus 77.99% on ArXiv), which the paper attributes to a training-test distribution shift that feature-only students cannot bridge without neighbor information.","pith_inferences":["The oracle gap the paper quantifies (an ideal ensemble at 97.55% versus the fused teacher's 87.53% on ACM) indicates the teacher's fusion layer, not the view encoders, is the accuracy ceiling; a student trained on per-view logits could plausibly be pushed closer to that ideal bound than the 1–2 points MGFNN+ presently gains, since the view predictions disagree enough to correct one another.","Because the base assigners $T$ are node-agnostic, they could in principle transfer across multiplex graphs that share a feature space, letting a new domain initialize its ensemble behavior after seeing only a few nodes — an extension the paper does not test.","The same per-node coefficients can serve as a view-relevance diagnostic: a node whose weight concentrates on one view is likely poorly served by the other edge types, so the mechanism could flag noisy or misaligned views in a multiplex dataset before deployment."],"forward_implications":["Distillation can make a feature-only student not merely match but surpass its multiplex-GNN teacher: MGFNN+ beats RSAGE on five of six transductive datasets by 0.56–2.09 points, because per-node ensembling escapes the teacher's fused-prediction bottleneck.","Inference cost becomes almost dataset-independent: MGFNNs take about 0.18 ms per ten nodes on every dataset, yielding 35.40×–89.14× speedups over the RSAGE teacher and roughly 35× over a neighbor-sampled teacher, which is what latency-sensitive graph services need.","The distillation transfers across teacher architectures: with RGCN, RGAT, or HAN as the teacher instead of RSAGE, MGFNN+ still surpasses each teacher's transductive accuracy on the tested datasets (ACM, IMDB, and ArXiv).","The learned per-node coefficients are usable as an interpretability tool: on MAG, for example, node $v_1$ weights the PAP view at 0.3881 while node $v_3$ weights PSP at 0.4058, showing which edge type each node actually relies on.","The boundary of the accuracy parity is inductive generalization on large, shifting graphs: on the ArXiv and MAG inductive splits the student trails the teacher by 11.24 and 7.14 points respectively, so the method's promise is strongest where test nodes resemble training nodes."],"supporting_citations":[{"why":"Supplies the base GNN-to-MLP distillation recipe (KL on soft labels plus the transductive/inductive evaluation protocol) that MGFNN extends to multiplex graphs.","marker":"[29]"},{"why":"Provides the knowledge-distillation objective — soft targets and KL divergence — used to train the student MLP.","marker":"[7]"},{"why":"GraphSAGE is the view encoder from which the paper builds its RSAGE teacher model.","marker":"[6]"},{"why":"Defines the attention-based multiplex GNN integration paradigm the paper aims to surpass, and supplies the ACM, IMDB, and DBLP datasets.","marker":"[19]"},{"why":"Supplies the information-theoretic decomposition of I(G[v]; y_v) into feature and edge terms used to argue when a feature-only student can succeed.","marker":"[1]"},{"why":"Supplies the two large datasets (ArXiv and MAG) whose inductive splits reveal the method's accuracy boundary.","marker":"[8]"},{"why":"One of the view-wise adaptive ensemble distillation baselines (gradient space) that MGFNN+ is ablated against.","marker":"[3]"},{"why":"Multi-teacher GNN-to-MLP distillation baseline (attention-based view-wise ensembling) used in the ablation comparison.","marker":"[23]"}],"fun_headline_variants":["Node-wise distillation: MLPs match multiplex GNNs at 35-89x speed","Per-node ensemble distillation turns MGNN into fast MLP","35-89x faster: MLP students match multiplex GNN teachers","MLPs beat teachers by exploiting per-view headroom","Node-wise view ensemble: MLPs match MGNNs, 35-89x faster"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise, stated in Section 4.3, is that node features and structural roles are highly correlated in real graphs; the method works only if a node's features alone already carry most of the information needed to predict its label, and if labels are decided by network position, such as degree or triangle membership, the feature-only student has nothing to learn from and the accuracy advantage over a plain MLP disappears.","fun_headline_variants_meta":{"raw":{"variants":["Node-wise distillation: MLPs match multiplex GNNs at 35-89x speed","Per-node ensemble distillation turns MGNN into fast MLP","35-89x faster: MLP students match multiplex GNN teachers","MLPs beat teachers by exploiting per-view headroom","Node-wise view ensemble: MLPs match MGNNs, 35-89x faster"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000617,"raw_usage":{"total_tokens":2950,"prompt_tokens":1115,"completion_tokens":1835,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":731,"completion_tokens_details":{"reasoning_tokens":1738}},"tokens_in":731,"tokens_out":1835,"duration_ms":12798,"temperature":1.0,"reasoning_tokens":1738,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T17:37:02.320960+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain on a multiplex graph whose labels are a pure function of network position (for instance, class equals degree bucket or triangle membership) with node features randomly shuffled so features carry no label information: the paper's account predicts MGFNN+ collapses toward the vanilla MLP level while the teacher keeps its accuracy. Observing instead that the student tracks the teacher would refute the paper's explanation of why the method works.","supporting_citations":[{"cited_title":"In: ICLR (2022)","cited_arxiv_id":null,"evidence_quote":"Supplies the base GNN-to-MLP distillation recipe (KL on soft labels plus the transductive/inductive evaluation protocol) that MGFNN extends to multiplex graphs."},{"cited_title":"arXiv preprint (2015)","cited_arxiv_id":null,"evidence_quote":"Provides the knowledge-distillation objective — soft targets and KL divergence — used to train the student MLP."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the attention-based multiplex GNN integration paradigm the paper aims to surpass, and supplies the ACM, IMDB, and DBLP datasets."},{"cited_title":"In: ICLR (2021)","cited_arxiv_id":null,"evidence_quote":"Supplies the information-theoretic decomposition of I(G[v]; y_v) into feature and edge terms used to argue when a feature-only student can succeed."},{"cited_title":"NeurIPS (2020)","cited_arxiv_id":null,"evidence_quote":"Supplies the two large datasets (ArXiv and MAG) whose inductive splits reveal the method's accuracy boundary."},{"cited_title":"NeurIPS (2020)","cited_arxiv_id":null,"evidence_quote":"One of the view-wise adaptive ensemble distillation baselines (gradient space) that MGFNN+ is ablated against."},{"cited_title":"In: DASFAA","cited_arxiv_id":null,"evidence_quote":"Multi-teacher GNN-to-MLP distillation baseline (attention-based view-wise ensembling) used in the ablation comparison."}],"review_version":1}