{"id":"fad1b00f-56b4-49dc-b56d-cec4a0e470ab","arxiv_id":"2507.19031","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"ProGMLP trains a sequence of MLPs with progressive distillation and mixup, enabling flexible accuracy-speed trade-offs for graph node classification in one framework.","lead":"This paper introduces ProGMLP, a framework that trains a sequence of lightweight MLP models to imitate a graph neural network, allowing users to stop early for speed or run more models for accuracy. It matters because graph AI models are often too slow for real-time systems, and this offers a practical accuracy-versus-latency dial without retraining.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Trade-off curves count students, not cost: ProGMLP students take concatenated hidden input, so equal-#students comparisons in Fig. 3 overstate the accuracy-latency advantage.","rationale":"The reader's stopping-rule concern (Eq. 11 is batch-level and unvalidated) is legitimate, but I see an even more direct threat to the central claim: the paper's accuracy-cost curves compare methods at equal numbers of students while ignoring that ProGMLP students are architecturally more expensive due to the concatenated hidden-state input. This affects the core 'efficient trade-offs' claim even for the fixed-k operating points that are actually evaluated, and it is not addressed by any reported latency comparison against the ensemble baselines. The stopping-rule issue affects the 'on-demand' mechanism, but if the cost axis itself is incomparable, the central contribution is unverified regardless of how the stop decision is made. Both issues are fixable with additional experiments, so the verdict remains CONDITIONAL rather than REJECT: the method may well work, but the evidence as presented does not establish the headline efficiency advantage.","tokens_in":17874,"tokens_out":7964,"duration_ms":88879,"concrete_test":"On CS and ogbn-arxiv with the GCN teacher, measure wall-clock inference time (or FLOPs) for ProGMLP, E-GLNN, and AdaGMLP at every number of executed students from 1 to 10, using identical hardware, batch sizes, and hidden dimensionality, then replot Figure 3 with mean latency or FLOPs on the x-axis and accuracy on the y-axis. If ProGMLP's accuracy at equal latency is not consistently above the baselines across the operating range, the claimed efficiency trade-off is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is an efficient, on-demand accuracy-latency trade-off, but the experiments measure inference cost only as the number of executed students. Section III-C defines X_k = CONCAT(X, H_{k-1}), so every ProGMLP student (including the first) has input dimension d + d' rather than d. Section III-G reports per-student forward cost O(N d'(d + d' + C)), while a baseline 2-layer MLP student costs O(N d'(d + C)); at the stated d' = 128, each ProGMLP student has roughly double the first-layer FLOPs of a same-width baseline. Section IV-F nonetheless plots accuracy against '# Executed Students' for K = 10 for all methods, and Section IV-I reports latency only against teacher GNNs, not against E-GLNN or AdaGMLP. Consequently, Figure 3 and Tables II-III do not establish that ProGMLP has a better accuracy-latency trade-off: the apparent advantage may be purchased by extra per-student capacity and compute, not by the progressive distillation design. Since the paper's headline contribution is precisely 'efficient trade-offs', this missing cost-controlled comparison is load-bearing.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"ProGMLP proposes a progressive GNN-to-MLP distillation framework in which K MLP students are trained sequentially, each initialized from the previous student's parameters and fed the concatenation of the raw features and the previous student's hidden representation (Section III-C). Training combines a Progressive Knowledge Distillation term, a Progressive Mixup Augmentation term, and an optional confidence-based early exit at inference (Section III-F). The framework is evaluated on eight graph datasets with GCN, GAT, and GraphSAGE teachers, against ensemble G2M baselines (E-GLNN, AdaGMLP) and non-ensemble G2M baselines (KRD, HGMD, NOSMOG), including a large-scale latency study and an inductive setting.","tokens_in":18094,"tokens_out":10005,"duration_ms":97273,"significance":"The idea of offering a spectrum of accuracy-latency operating points within GNN-to-MLP distillation is timely, and the paper is generally clearly written. If the trade-off claims were established under cost-controlled comparisons, ProGMLP would be a useful contribution: it introduces concrete mechanisms (PTS, PKD, PMA), provides a code link, evaluates on eight datasets with three teacher architectures, and includes ablations that support the role of the progressive components. Comparing against AdaGMLP, a strong same-group baseline, is legitimate and not circular. However, the current evidence does not yet establish the central 'efficient trade-off' claim because inference cost is measured by student count rather than compute or latency, and the adaptive-stopping mechanism is batch-level rather than per-instance. These issues are fixable with additional experiments or by narrowing the claims.","major_comments":[{"comment":"The inference-cost metric used in Figure 3 is '# Executed Students', but ProGMLP students are not cost-equivalent to the baselines. Section III-C defines X_k = CONCAT(X, H_{k-1}), so every student has input dimension d + d' rather than d, and Section III-G gives per-student forward cost O(N d'(d + d' + C)). This is larger than the O(N d'(d + C)) cost of a same-width 2-layer MLP baseline by an additional O(N d'^2) term, a gap that is largest when the feature dimension is small (e.g., ogbn-products with d=128). Equal-#students curves therefore overstate ProGMLP's accuracy-latency advantage. Table II compounds this by comparing ProGMLP (whose K is not stated) against E-GLNN_2/4 and AdaGMLP_2/4 without matching total compute, and Table IV reports latency only against teacher GNNs, not against other G2M students. Please add accuracy-versus-FLOPs and/or accuracy-versus-measured-latency curves for all methods, and include at least one cost-matched comparison (e.g., equal total forward FLOPs per operating point).","section":"Section IV-F / Fig. 3; Section III-C / III-G"},{"comment":"The on-demand inference claim in the introduction and abstract is not supported by the implemented stopping rule. Eq. (11) computes c_k as the mean over all unlabeled nodes of the per-node maximum softmax probability, yielding a single batch-level scalar; the early-exit decision is therefore the same for the entire node set and cannot adapt per request or per node. This is a dataset-level budget selector rather than the per-instance dynamic mechanism suggested by the edge/mobile examples in Section I and by Figure 1. The authors should either provide a per-instance confidence protocol (with experiments measuring accuracy and latency at the instance level) or explicitly reframe ProGMLP as offering K dataset-level operating points selected before batch inference, removing the per-request interpretation. In addition, the paper should explain how tau_conf is calibrated to a latency or accuracy target; the current hyperparameter analysis on CS (Section IV-L) only shows accuracy robustness, not runtime adaptability.","section":"Section III-F.2 / Eq. (11) / Section I"},{"comment":"The claim that ProGMLP 'consistently outperforms' E-GLNN and AdaGMLP is not statistically supported. In several dataset/teacher combinations the standard deviations overlap substantially with the strongest baseline and the accuracy gap is small (e.g., GCN/Pubmed: ProGMLP 77.42±0.35 vs AdaGMLP_4 77.21±0.66; GraphSAGE/Computers: ProGMLP 80.21±1.21 vs AdaGMLP_4 79.23±1.06). Please report paired significance tests or exact confidence intervals over the 10 runs, and state in how many of the evaluated settings ProGMLP is significantly better. This matters because Table II is one of the primary empirical bases for the framework's advantage over ensemble G2M methods.","section":"Section IV-G / Table II"}],"minor_comments":[{"comment":"In Eq. (10), the moving-average recurrence has ell_bar_t on both sides; the right-hand side should use the instantaneous loss ell_t. In the text following Eq. (9), lambda is said to be updated from the 'current mixup loss calculated by Eq. (6)', but Eq. (6) is the PKD loss; the reference should be Eq. (8).","section":"Eq. (10) / Eq. (9)"},{"comment":"The notation 'kβ' in Eq. (6) is ambiguous: it should be defined explicitly as either k·β or β^k, since the surrounding text only says that later students receive higher weights.","section":"Eq. (6)"},{"comment":"The parameter search space in Section IV-E lists the number of students as {2, 3, ..., 6}, while Section IV-F sets K = 10 for all methods; these settings should be reconciled and the final K reported for every experiment table.","section":"Section IV-E / Section IV-F"},{"comment":"Algorithm 1 uses a generic variable E for the number of epochs, whereas Section III-F.1 defines E1 and E2; the notation should be aligned.","section":"Algorithm 1 / Section III-F.1"},{"comment":"The header 'DATASETS STATICS' contains a typo and should read 'DATASETS STATISTICS'.","section":"Table I"},{"comment":"The ablation text reports the Computers accuracy dropping to 69.31% when PMA is removed and states the original value as 77.92%, but Table II reports GCN+ProGMLP on Computers as 78.82±0.44; please clarify which teacher/setting the ablation uses and correct the inconsistency.","section":"Section IV-K"},{"comment":"For the comparison with non-ensemble G2M methods, please state the number of students K used for ProGMLP and whether the reported accuracy is after executing all K students or after the early-exit criterion; this is needed to interpret the comparison with single-student methods.","section":"Section IV-H / Table III"},{"comment":"The Index Terms line ('component, formatting, style, styling, insert') appears to be placeholder text and should be replaced with actual keywords.","section":"Index Terms"}],"recommendation":"major_revision","confidential_remarks":"The main gap is the cost-controlled evaluation of the central trade-off claim, together with the batch-level nature of the early-exit mechanism. If the authors add accuracy-versus-FLOPs/latency curves and either implement a per-instance protocol or revise the on-demand claims, the paper could become acceptable. The use of AdaGMLP as a baseline is legitimate and not a circularity issue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read ProGMLP carefully, and the stress-test note is right: the central efficiency claim is not established. The paper's own setup gives each ProGMLP student concatenated input CONCAT(X, H_{k-1}), so input dimension is d + d' instead of d. At d' = 128, each student's first layer costs about twice as many FLOPs as a same-width baseline MLP. Section III-G even writes the per-student forward cost as O(N d'(d + d' + C)). Yet Figure 3 plots accuracy against # executed students, and Table IV reports latency only against GNN teachers, not against E-GLNN or AdaGMLP. Tables II and III compare ProGMLP (which uses up to 10 students or stops early) against E-GLNN2/4 and AdaGMLP2/4, so those are not cost-matched either. The apparent trade-off advantage may simply be purchased with extra per-student capacity. That is a load-bearing problem for the paper's hook.\n\nTo be fair, the paper does have real substance. The training framework — sequential students initialized from the previous one, hidden-state propagation, progressive mixup with an adaptive mixing ratio, and confidence-weighted stopping — is a new combination in the G2M setting. The ablations show each component contributes, especially PTS. The code is public on GitHub, which makes the work reproducible, and the writing is clear enough that the method can be reimplemented without agonizing.\n\nThe softer spots are also worth naming. The inference-time stopping rule in Eq. (11) is a batch-level average of per-node max softmax over all unlabeled nodes. That is not a per-instance dynamic algorithm, so the “on-demand per-request” deployment story in the introduction overstates what is actually evaluated. The claim of being the first to enable flexible trade-offs in G2M is also a bit strong, since an early-stopping ensemble of any G2M method gives a trade-off curve; the paper does compare against such baselines, which mitigates this, but the framing still overreaches.\n\nThese are fixable issues. The core progressive-distillation idea may still be useful, and the accuracy-only comparisons in Table III show competitive or slightly better numbers against single-student G2M methods. A serious referee should ask for cost-controlled trade-off curves — FLOPs or measured latency against accuracy for all methods — and either a per-instance inference protocol or an explicit statement that the stopping rule is prespecified at the batch level. As written, I would not accept the efficiency claims, but the paper deserves a genuine peer review rather than a desk reject.","headline":"A plausible progressive G2M distillation framework whose headline efficiency claim is not yet supported, because the trade-off curves count students rather than cost.","tokens_in":18658,"tokens_out":2106,"would_cite":false,"duration_ms":21894,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ProGMLP claims that a progressive chain of distilled MLP students gives graph inference an adjustable accuracy-latency dial.","keywords":["GNN-to-MLP distillation","progressive training","knowledge distillation","mixup augmentation","early-exit inference","node classification","graph neural networks","accuracy-latency trade-off"],"falsifier":"A concrete test: run ProGMLP with a per-node early exit rule, where each node stops at the first student whose own maximum softmax probability exceeds the threshold, and compare the resulting accuracy-latency curve against the paper's batch-level rule on the same data. If the per-node rule does not match or beat the batch rule in accuracy at equal average latency, then the batch-level confidence statistic is not a faithful controller of per-request trade-offs and the on-demand claim is unsupported.","tokens_in":17647,"feed_emoji":"⚖️","tokens_out":11263,"duration_ms":103720,"temperature":0.7,"pith_summary":"This paper tries to establish that GNN-to-MLP knowledge distillation need not be a static, single-point trade-off. ProGMLP trains a sequence of MLP students, each initialized from the previous student's parameters and fed the previous student's hidden representation, so that later students progressively refine the distilled knowledge. The claim is that this single trained framework provides a spectrum of operating points on the accuracy-latency curve: a user can stop after a few students for low latency or run all students for maximum accuracy, without retraining. The paper supports the claim with experiments on eight real-world graph datasets, reporting both higher peak accuracy than the compared G2M baselines and large inference speedups, including a 21x speedup over the GCN teacher on ogbn-arxiv.","feed_headline":"A chain of MLP students puts GNN accuracy on a dial","feed_subtitle":"Stop early for speed or run all students for peak accuracy from one trained framework.","key_machinery":"The load-bearing mechanism is the Progressive Training Structure: a chain of MLP students in which $f_{k+1}$ inherits $\\theta_k$ from $f_k$ and the input to each student is the concatenation of raw features $X$ with the previous student's hidden representation $H_{k-1}$. This turns the sequence into a recurrent-like refinement process rather than an independent ensemble. On top of it sit Progressive Knowledge Distillation, which weights each student's supervised-plus-distillation loss by $k^\\beta$ so later students are pushed toward higher accuracy, and Progressive Mixup Augmentation, which makes mixed training samples progressively harder by raising the mixing ratio $\\lambda_k$ when the student's moving-average mixup loss falls. The inference mechanism that realizes the trade-off is the confidence-based stopping rule: $c_k$ is the mean of per-node maximum softmax probabilities over all unlabeled nodes, and inference halts at the first student with $c_k \\geq \\tau_{\\mathrm{conf}}$, with Eq. (12) combining the predictions of the executed students by confidence-derived weights.","core_discovery":"The central claim is that progressive training of multiple MLP students yields an anytime-inference system for GNN-to-MLP distillation. Concretely, student $f_k$ is trained with the previous student's parameters as initialization, receives $\\mathrm{CONCAT}(X,H_{k-1})$ as input (with $H_0$ the zero matrix), and is optimized by a loss that combines a cross-entropy-plus-KL distillation term weighted by $k^\\beta$ and a mixup term whose mixing ratio $\\lambda_k$ rises adaptively as the student's moving-average loss drops. At inference the students are evaluated in sequence; the chain stops at the first student whose average maximum softmax probability over unlabeled nodes, $c_k$ from Eq. (11), reaches a confidence threshold $\\tau_{\\mathrm{conf}}$, and the final prediction is the confidence-weighted sum of the executed students' predictions (Eq. (12)). The paper reports that on six medium-scale and two large-scale datasets, ProGMLP reaches near-peak accuracy with only a few students, continues to improve as more students run, beats the ensemble baselines GLNN and AdaGMLP on nearly all datasets, and on ogbn-arxiv improves on the teacher while being up to 21x faster.","pith_inferences":["The paper validates the confidence threshold at the batch level, so a per-instance early-exit protocol (stopping each node when its own softmax confidence crosses the threshold) is a natural testable extension the paper does not run; without it, the on-demand per-request flexibility claim remains a batch-level property.","Because each student's input includes the previous student's hidden representation, the framework is close to an adaptive computation graph; it could be combined with input-dependent gating to decide per node how many students to run, which would turn the current global stopping rule into a local one.","The progressive-mixup schedule is driven by the student's own loss, which couples curriculum difficulty to the model's state; a direct implication is that the same schedule could be applied to other distillation settings, such as GNN-to-GNN compression, though the paper does not test that."],"forward_implications":["If the reported results hold, a single trained ProGMLP model can replace separate fast and accurate models: setting the confidence threshold or the maximum number of executed students selects where the deployment sits on the accuracy-latency curve.","The progressive structure means early-exit predictions are not wasted: each executed student contributes to the confidence-weighted final prediction, so partial computation always yields a usable output.","On large graphs, the framework's inference cost scales with the number of students executed rather than with graph neighborhood size, which is the source of the reported speedups over GNN teachers.","Removing the progressive training structure hurts accuracy most (about 4% on Coauthor CS in the ablation), so the chain-like initialization and hidden-state passing, not just the ensemble size, carries the gain.","In the inductive setting, the distilled students can generalize to unseen nodes better than the GNN teachers, suggesting the progressive mixup and distillation improve robustness beyond transductive accuracy."],"supporting_citations":[{"why":"Supplies the GCN teacher architecture whose soft predictions are distilled into the MLP students.","marker":"[1]"},{"why":"Supplies the GAT teacher architecture used to test distillation from attention-based GNNs.","marker":"[2]"},{"why":"Supplies the GraphSAGE teacher and the inductive evaluation setup used in the generalization experiments.","marker":"[3]"},{"why":"Defines the foundational single-student G2M baseline that ProGMLP must outperform on accuracy and flexibility.","marker":"[9]"},{"why":"Presents the most related multi-MLP ensemble framework whose accuracy-latency curves ProGMLP claims to dominate.","marker":"[13]"},{"why":"Provides the knowledge-distillation objective (KL divergence) used in the Progressive Knowledge Distillation loss.","marker":"[38]"},{"why":"Supplies the Cora and Pubmed citation datasets used in the transductive and inductive evaluations.","marker":"[48]"},{"why":"Supplies the Amazon and Coauthor datasets used for the medium-scale node-classification experiments.","marker":"[49]"},{"why":"Supplies the ogbn-arxiv and ogbn-products large-scale benchmarks used to demonstrate speedups and scalability.","marker":"[50]"}],"fun_headline_variants":["Anytime GNN accuracy from a chain of MLPs","Dial GNN accuracy with progressive MLP students","Stop early or run all: MLP students tune GNN speed","Progressive MLPs beat teacher at 21x speed","On-demand GNN accuracy from MLP student chain"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the average of the maximum softmax probabilities over all unlabeled nodes is a reliable and controllable proxy for deciding when to stop the student chain, even though this stopping signal is batch-level and the paper's on-demand per-request flexibility claims would need the decision to be made per input.","fun_headline_variants_meta":{"raw":{"variants":["Anytime GNN accuracy from a chain of MLPs","Dial GNN accuracy with progressive MLP students","Stop early or run all: MLP students tune GNN speed","Progressive MLPs beat teacher at 21x speed","On-demand GNN accuracy from MLP student chain"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000308,"raw_usage":{"total_tokens":1814,"prompt_tokens":1051,"completion_tokens":763,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":667,"completion_tokens_details":{"reasoning_tokens":682}},"tokens_in":667,"tokens_out":763,"duration_ms":7440,"temperature":1.0,"reasoning_tokens":682,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:02:27.622976+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete test: run ProGMLP with a per-node early exit rule, where each node stops at the first student whose own maximum softmax probability exceeds the threshold, and compare the resulting accuracy-latency curve against the paper's batch-level rule on the same data. If the per-node rule does not match or beat the batch rule in accuracy at equal average latency, then the batch-level confidence statistic is not a faithful controller of per-request trade-offs and the on-demand claim is unsupported.","supporting_citations":[{"cited_title":"Graph-less neural networks: Teaching old mlps new tricks via distillation,","cited_arxiv_id":null,"evidence_quote":"Defines the foundational single-student G2M baseline that ProGMLP must outperform on accuracy and flexibility."},{"cited_title":"Adagmlp: Adaboosting gnn-to- mlp knowledge distillation,","cited_arxiv_id":null,"evidence_quote":"Presents the most related multi-MLP ensemble framework whose accuracy-latency curves ProGMLP claims to dominate."}],"review_version":2}