{"id":"a6b0016c-86aa-4406-86d3-8461e09bc8e1","arxiv_id":"2506.01231","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"GC splits a graph supernet into sub-supernets by grouping modules with similar gradient contributions, and UGAS searches combined MPNN and graph transformer architectures; the searched GNNs beat several baselines.","lead":"Supernet weight-sharing in neural architecture search degrades when different subnetwork modules pull shared weights in conflicting directions. This paper introduces a cheap gradient-contribution measure to split a supernet into sub-supernets, and pairs it with a unified search space covering both message-passing and transformer-style graph networks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The partition objective in Eq. (19) uses cosine similarities that can be negative, but the stated Stoer-Wagner solver is only valid for nonnegative edge weights; without a specified transformation, Algorithm 2 is not well-defined and the claim that it minimizes Eq. (19) is unsupported.","rationale":"The reader's weakest assumption concerns whether early-stage gradient contributions remain representative throughout training. That is a genuine empirical concern, but it presupposes that the partition computation itself is correct. The signed-min-cut problem is more fundamental: every GC experiment depends on the min-cut step, and the paper's stated solver is not valid for the stated edge weights. This is an internal consistency issue rather than a disagreement with external consensus, so it is appropriate to flag it as a correctness risk. The concern is settled by inspecting the public code, so the right response is to keep the verdict conditional: if the implementation documents a valid transformation of the signed similarities, the concern dissolves; if it does not, the central partition-quality claim is unsupported. I do not think the concern alone justifies outright rejection, because the released code may handle the issue and the empirical results could still be reproducible.","tokens_in":19783,"tokens_out":10543,"duration_ms":136726,"concrete_test":"Inspect the released Zenodo implementation for the min-cut step and run it on a signed similarity matrix such as the one in Figure 4. Enumerate all 2^4 cuts and compare the implementation's returned cut and objective value against the true minimizer of Eq. (19). Also check whether weights are clamped, shifted, or converted before calling Stoer-Wagner. If the implementation rejects negative weights, or if its returned cut differs from the true minimum, the method is not the one described and the Table 4 results require re-examination.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the min-cut over gradient-contribution similarities. Eq. (18) defines S_{l,i,j} as a cosine similarity, which is signed; Figure 4 even shows negative edge weights. Eq. (19) then asks for the cut minimizing the sum of these signed similarities. This is not the standard global minimum-cut problem solved by Stoer-Wagner, whose correctness requires nonnegative edge weights. Moreover, because the number of cross edges depends on the side sizes, adding a constant to make all weights nonnegative changes the minimizer unless the cut is balanced; the paper specifies no balance constraint or transformation. Thus Algorithm 2 as written is not a well-specified instance of the Minimum Cut Problem, and the partition scores gamma_l used to select the top-k layers are not necessarily actual minimum cuts. The SOTA partition-quality claim in Table 4 depends directly on this step, so either the released code silently clamps, shifts, or converts the weights, or the optimization claimed in the paper is not actually performed.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Gradient Contribution (GC), a supernet partitioning method for few-shot neural architecture search. GC decomposes the Vector-Jacobian Product during backpropagation to compute cosine similarities between gradient contributions of modules in adjacent layers, then formulates the partitioning of each layer as a minimum-cut problem over these similarities, selecting the top-k layers with the smallest cut weights to split the supernet into 2^k sub-supernets. The paper also introduces UGAS, a unified graph neural architecture search framework that searches over both MPNNs and graph transformers, with GA and DARTS implementations. Experiments on ten graph datasets claim that GC achieves state-of-the-art partitioning quality and time efficiency (about 21% of GM's time), and that UGAS+GC finds architectures outperforming manually designed GNNs and existing NAS methods.","tokens_in":19958,"tokens_out":5695,"duration_ms":61203,"significance":"If the claims hold, this is a valuable contribution to few-shot NAS: it offers a principled, gradient-based way to identify conflicting modules and partition a supernet with little extra training cost, and it broadens GNAS search spaces to include both MPNNs and graph transformers. The time-efficiency comparison against GM is clear and compelling, and the release of source code is a strength. However, the headline partitioning-quality claim rests on a single reduced dataset with overlapping error bars, and the minimum-cut formulation has a technical gap concerning signed edge weights. The unified MPNN/GT search space is a useful conceptual step, and the ablation in Table 6 supports the benefit of joint search.","major_comments":[{"comment":"The cosine similarity S_{l,i,j} in Eq. (18) is signed, and Eq. (19) minimizes the sum of these signed weights over a cut. The Stoer-Wagner algorithm is only correct for nonnegative edge weights; with negative weights, the problem is not the standard global minimum-cut problem, and the algorithm's optimality guarantee does not apply. Figure 4 explicitly shows negative edge weights (e.g., -0.7, -0.5). The paper does not specify a transformation (shifting, clamping, or a balanced-cut formulation) that would make the optimization well-defined. Consequently, the partition scores gamma_l and the top-k layer selection are not necessarily actual minimizers of Eq. (19), and the SOTA partitioning-quality claim in Table 4 is not supported by the stated optimization. The authors should either prove their implementation solves Eq. (19) (e.g., by specifying the exact transformation in the released code) or reformulate the objective so that the solver is applicable.","section":"Section 4.4, Eq. (18)-(19), Algorithm 2"},{"comment":"The proof of Proposition 1 assumes that the minimum-cut algorithm minimizes the cross-group similarity S_cross. Given the signed-weight issue in Section 4.4, this premise is not established. Moreover, the derivation of E[S_cross] in Eq. (29) is not fully justified (e.g., the probability expression and the treatment of self-similarities), and the conclusion that at least one group has above-average internal similarity is nearly definitional. The paper should either correct this proof to handle signed weights rigorously or present it as a heuristic motivation rather than a formal guarantee.","section":"Appendix B, Proposition 1"},{"comment":"The headline partitioning-quality result is based on a single dataset (Cluster, reduced to 30% of its original size), and the reported differences are within overlapping error bars (GC Kendall 0.54±0.17 vs. GM 0.42±0.22; Spearman 0.73±0.19 vs. 0.61±0.16). Given that the paper claims state-of-the-art partitioning quality, additional datasets or a statistical significance test are needed to support this claim. Without such evidence, the conclusion that GC reliably outperforms prior partitioning methods remains tentative.","section":"Section 5.3.2 and Appendix D, Table 4"},{"comment":"Many baseline numbers in Tables 2 and 3 are taken from the original papers (marked with †) without matched training protocols, while the UGAS results are obtained in this work. The claimed improvements over manually designed GNNs are modest (e.g., average accuracy improvements of 0.27% on BGNN), so cross-paper differences in training budgets, hyperparameters, and evaluation procedures could account for the gaps. The authors should either rerun the baselines under the same protocol for the key comparisons, or explicitly acknowledge and bound the impact of unmatched protocols on the claimed superiority.","section":"Tables 2 and 3 (symbol †)"},{"comment":"The partition criterion is stated to use \"early-stage gradients,\" but the exact checkpoint (epoch or iteration) at which the gradients are recorded is never specified. This is a free parameter that can affect the resulting partition and the reproducibility of the method. The paper should report the specific early-stage checkpoint used in all experiments and, ideally, show sensitivity of the final ranking correlations to this choice.","section":"Appendix F"}],"minor_comments":[{"comment":"The minimization in Eq. (2) is written as \"min ∑ ...\" without specifying the variable; it should read \"min_{Γ⊆X} ∑ ...\" for clarity.","section":"Section 3.2, Eq. (2)"},{"comment":"The sentence \"the supernet training time of GC is approximately four times that of GM\" is confusing because Figure 5 shows that GC's total time is much smaller; clarify that the comparison refers to the supernet training component only, while the total partitioning time is what is summarized by the 21% figure.","section":"Section 5.3.3 and Figure 5"},{"comment":"Table 7 reports ranking consistency values without standard deviations or the number of repetitions; adding this information would make the stability of the early-stage gradient claim more interpretable.","section":"Appendix F, Table 7"},{"comment":"The similarity matrix in Figure 4 displays negative entries, which directly relates to the major concern about signing in the min-cut formulation; a brief note on how negative weights are handled in practice would help the reader.","section":"Figure 4"},{"comment":"The line \"Compute the partition score γ_l\" is vague; it should reference Eq. (19) and the specific solver used, especially given the signed-weight issue.","section":"Algorithm 2, line 3"},{"comment":"The paper uses \"few-shot NAS\" to describe methods that partition a supernet into sub-supernets; this terminology is used inconsistently in the literature, and a one-sentence clarification of the intended meaning would improve accessibility.","section":"Abstract and Introduction"}],"recommendation":"major_revision","confidential_remarks":"The decisive issue is the signed-weight minimum-cut problem in Section 4.4. If the released code silently clamps, shifts, or otherwise transforms the cosine similarities, the paper's central optimization claim may still be salvageable, but it must be stated and analyzed. The efficiency result is solid, and the unified MPNN/GT search space is a useful contribution. The partitioning-quality evidence is thin (one reduced dataset, overlapping error bars) and should be strengthened before the SOTA claim is accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"For you, in one breath: this paper is worth reading for the VJP-decomposition trick and the unified MPNN/GT search space, but the min-cut step has a real gap, and the partition-quality claim rests on one reduced dataset. I'd engage with it, but not before checking the released code.\n\nThe genuinely new bit is GC. Instead of GM's retraining-based gradient matching, they keep the backward computation graph, decompose the VJP into per-module contributions, and compute cosine similarities between those contributions. That is a real computational shortcut, and the time comparison (about 21% of GM's cost) is the cleanest result in the paper. The UGAS unified search space is also a legitimate advance: it is the first GNAS framework I know of that co-searches MPNNs and GTs, and the ablation in Table 6 shows the joint search helps on both small and long-range graphs. The authors shipped code, which is more than many papers do.\n\nNow the soft spots, in proportion. The most serious is the min-cut formulation. Equation (18) produces signed cosine similarities, and Figure 4 even shows negative edge weights. Stoer-Wagner is only correct for nonnegative weights, and shifting the weights by a constant does not preserve the minimizer unless the cut is balanced. The paper specifies no clamping, shifting, or balance constraint. So the partition scores γ_l used to pick the top-k layers are not necessarily actual minimum cuts, and the SOTA partition-quality claim in Table 4 is not supported by the text alone. Maybe the code silently clamps or shifts, but that needs to be stated. This is a load-bearing gap, not a nitpick.\n\nThe empirical case for partition quality is also thinner than the abstract's \"SOTA\" language. Table 4 is one dataset (Cluster, reduced to 30%) with overlapping error bars. Many baselines in Table 2 are borrowed from original papers rather than re-run under matched training protocols. And the abstract says UGAS+GC outperforms NAS methods on all datasets, but the DARTS variant is not best on Molsider in Table 3 (the GA variant is, so the claim is salvageable if read charitably, but it is overstated as written).\n\nAppendix B is the weakest link. The proof shows, at best, that a min-cut groups similar items together; it does not show that this mitigates the weight-coupling problem, and the expected-value argument assumes a uniformly random partition, which is not what the algorithm produces. I would not rely on that proof.\n\nWho gets value: researchers working on few-shot NAS or graph architecture search, especially those who want a cheaper alternative to GM. The paper deserves a serious referee, but I would ask the authors to specify the weight transformation in Algorithm 2 and re-run at least one additional dataset for Table 4. Read it, cite the unified space if you work in GNAS, but do not adopt GC until the code's handling of negative weights is cleared up.","headline":"GC's VJP decomposition is a genuinely cheaper gradient-based partition method and the unified MPNN/GT search space is a useful step, but the min-cut formulation is underspecified for signed cosine weights and the headline partition-quality evidence is thinner than the abstract suggests.","tokens_in":20543,"tokens_out":2424,"would_cite":true,"duration_ms":29502,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that the weight coupling problem in few-shot neural architecture search can be traced to conflicting gradient directions between adjacent-layer modules, and that a single Vector-Jacobian Product decomposition during…","keywords":["neural architecture search","few-shot NAS","supernet partitioning","weight coupling","gradient contribution","graph neural architecture search","message passing neural networks","graph transformers"],"falsifier":"Train a supernet, record the GC partition from an early backward pass, then measure for many sampled subnets the accuracy gap between inherited-weight evaluation and from-scratch retraining; if the early-pass partition is no better than a random partition at reducing ranking error, or if partitions computed at later checkpoints give noticeably higher ranking correlation, the central assumption fails.","tokens_in":19565,"feed_emoji":"🔀","tokens_out":5221,"duration_ms":52694,"temperature":0.7,"pith_summary":"The paper targets the weight coupling problem in few-shot neural architecture search, where a supernet's shared weights are used to judge subnets that will later be trained alone. It argues that the real cause is conflicting gradient directions: modules in one layer pull the shared weights of the preceding layer in opposite ways. To exploit this, it proposes Gradient Contribution (GC), which reads gradient directions off the supernet's normal backward pass with a decomposed Vector-Jacobian Product, and then uses a minimum cut over cosine similarities to send conflicting modules to different sub-supernets. Wrapped in the UGAS framework that searches message-passing and graph-transformer modules together, the paper reports that GC gives the best ranking correlation between inherited-weight and from-scratch evaluations, uses about 21 percent of the time of the gradient-matching baseline, and that UGAS+GC beats twelve hand-designed GNNs and other NAS methods on the benchmarks tested.","feed_headline":"Gradient-conflict split beats hand-designed GNNs at one fifth the runtime","feed_subtitle":"A one-pass gradient decomposition tells which GNN modules fight, so searched architectures track from-scratch quality more closely.","key_machinery":"The load-bearing object is the gradient contribution $C_{l,j} = (\\partial y_{l,j} / \\partial y_{l-1})^{\\top} \\nabla_{y_{l,j}} \\mathcal{L}$, obtained by decomposing the Vector-Jacobian Product of a layer's backpropagation. It measures how each module in layer $l$ alone would push the weights of layer $l-1$. GC computes cosine similarities $S_{l,i,j}$ between these vectors, treats the modules as graph nodes with $S$ as edge weights, and applies the Stoer-Wagner minimum cut algorithm to split each layer into two sets; the top-$k$ lowest-cut layers are partitioned, generating $2^k$ sub-supernets. The argument that this works rests on a proof that a minimum-cut partition cannot leave both sides with below-average internal gradient similarity.","core_discovery":"The central claim is that the weight-coupling error of a supernet can be predicted from one local quantity: the cosine similarity between the gradient contributions that adjacent-layer modules send back to the shared preceding layer. GC computes each module's gradient contribution $C_{l,j} = (\\partial y_{l,j} / \\partial y_{l-1})^{\\top} \\nabla_{y_{l,j}} \\mathcal{L}$, forms a similarity graph per layer, and solves a minimum cut to split modules whose gradients conflict. Selecting the $k$ layers with the lowest cut weights yields $2^k$ sub-supernets, so each subnet inherits weights from a sub-supernet with less internal gradient conflict. The paper reports that this makes inherited weights track from-scratch performance more closely than existing few-shot partitioners, with Kendall and Spearman correlations of 0.54 and 0.73 on the Cluster dataset, and that architectures searched under UGAS outperform hand-designed GNNs and other NAS methods.","pith_inferences":["The appendix shows cut weights are stable across training stages, but that does not by itself prove early-stage gradient conflicts predict final subnet ranking divergence; a direct test would partition at a late checkpoint and compare the searched results.","Because GC only needs cosine similarities among modules within one layer, applying it to non-graph search spaces such as convolutional or transformer blocks is a natural extension the paper does not test.","The minimum-cut formulation treats each layer's partition independently; interactions between two partitioned layers could in principle cancel or amplify conflicts, so a joint multi-layer partition is a testable alternative.","Portability to other training regimes is untested: the ranking quality might depend on the particular optimizer, schedule, and early-training checkpoint used to collect the gradient contributions."],"forward_implications":["Few-shot NAS can rank subnets from inherited weights with less distortion: the reported 0.54 Kendall and 0.73 Spearman correlations on Cluster mean high-performing subnets are less likely to be discarded and poor ones less likely to be overestimated.","Partitioning cost drops to roughly one fifth of that of the gradient-matching baseline because no temporary supernets are pruned and retrained; the partition is read off the forward and backward passes already being run.","The search space can include structurally different modules, message-passing networks and graph transformers, so the search, not the search-space designer, decides which inductive biases each layer should use.","The partition logic is generic: any one-shot NAS search space with parallel modules in a layer could be split by gradient-direction conflicts, not just GNN backbones.","The ablation studies indicate $k=2$, giving four sub-supernets, is the reported sweet spot: deeper partitioning improves weight fidelity but shrinks module counts and raises search cost."],"supporting_citations":[{"why":"The gradient-matching few-shot partitioner that GC is directly compared against; supplies the baseline for partitioning quality, ranking correlation, and time efficiency.","marker":"[16]"},{"why":"Introduced few-shot NAS with supernet partitioning into sub-supernets, the paradigm this work builds on and compares against.","marker":"[47]"},{"why":"The Stoer-Wagner minimum cut algorithm used to split each layer's modules into two sets.","marker":"[36]"},{"why":"Survey of automatic differentiation that supplies the Vector-Jacobian Product formulation GC decomposes.","marker":"[2]"},{"why":"GraphGPS provides the encoding block and overall backbone structure of UGAS and serves as a strong hand-designed baseline.","marker":"[34]"},{"why":"DARTS is the differentiable search paradigm in which UGAS is also implemented to validate generality.","marker":"[24]"},{"why":"The one-shot NAS baseline whose weight-sharing evaluation GC improves through supernet partitioning.","marker":"[28]"}],"fun_headline_variants":["Gradient-driven split finds better GNNs in fraction of time","Conflict-sensing NAS beats hand-designed GNNs at 1/5 runtime","One-pass gradient split for GNN search: SOTA at lower cost","Partition by gradient conflict: faster, better GNN search","Gradient-aware split: better GNNs, less compute"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The partitioning decision is made from gradient directions measured once in early supernet training, and the method assumes those early conflicts still predict which modules will fight each other when subnets are trained later.","fun_headline_variants_meta":{"raw":{"variants":["Gradient-driven split finds better GNNs in fraction of time","Conflict-sensing NAS beats hand-designed GNNs at 1/5 runtime","One-pass gradient split for GNN search: SOTA at lower cost","Partition by gradient conflict: faster, better GNN search","Gradient-aware split: better GNNs, less compute"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000565,"raw_usage":{"total_tokens":2714,"prompt_tokens":1014,"completion_tokens":1700,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":630,"completion_tokens_details":{"reasoning_tokens":1607}},"tokens_in":630,"tokens_out":1700,"duration_ms":11219,"temperature":1.0,"reasoning_tokens":1607,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:46:53.286293+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a supernet, record the GC partition from an early backward pass, then measure for many sampled subnets the accuracy gap between inherited-weight evaluation and from-scratch retraining; if the early-pass partition is no better than a random partition at reducing ranking error, or if partitions computed at later checkpoints give noticeably higher ranking correlation, the central assumption fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The Stoer-Wagner minimum cut algorithm used to split each layer's modules into two sets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The gradient-matching few-shot partitioner that GC is directly compared against; supplies the baseline for partitioning quality, ranking correlation, and time efficiency."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduced few-shot NAS with supernet partitioning into sub-supernets, the paradigm this work builds on and compares against."},{"cited_title":"Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind","cited_arxiv_id":null,"evidence_quote":"Survey of automatic differentiation that supplies the Vector-Jacobian Product formulation GC decomposes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"GraphGPS provides the encoding block and overall backbone structure of UGAS and serves as a strong hand-designed baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DARTS is the differentiable search paradigm in which UGAS is also implemented to validate generality."},{"cited_title":"Guan, Barret Zoph, Quoc V","cited_arxiv_id":null,"evidence_quote":"The one-shot NAS baseline whose weight-sharing evaluation GC improves through supernet partitioning."}],"review_version":1}