{"id":"81243060-32a5-4c74-8190-14c1a80b12b4","arxiv_id":"2504.18696","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"Prototypical networks are more robust than GCNs when active few-shot vertex classification is run from a completely unlabeled graph without a class oracle.","lead":"This paper tests whether prototypical graph networks can label vertices from an unlabeled graph under a tight annotation budget, using active learning without a class oracle. It finds prototypical networks lose only about 1% accuracy when the oracle is replaced by clustering, while graph convolutional networks lose about 9%.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 9% vs 1% oracle-robustness gap may be confounded: k-medoids pseudo-labels are derived from GPN's Euclidean-trained embeddings but from GCN's logits, so the comparison conflates model type with clustering-space suitability.","rationale":"The reader's weakest assumption concerns whether k-medoids pseudo-labels align with true classes, especially on heterophilic graphs or with early random embeddings. My concern is related but more specific: even on the homophilic datasets tested, the comparison between GCN and GPN in the Unbalanced Sampling experiment changes the clustering space along with the model type. Section VI explicitly states that GCN and label propagation effectively cluster logits, which are not a metric space, while GPN embeddings are Euclidean by design. That means the headline difference in oracle-drop sensitivity may reflect the representation used for k-medoids rather than a fundamental advantage of prototypical models. This does not invalidate the paper's more narrow claim that the proposed GPN-based active-learning pipeline is robust; it does weaken the abstract's broader causal attribution. The fix is a controlled experiment with a shared clustering representation, which is feasible and would settle the concern. Since the finding remains conditional rather than false, I do not move the reader's verdict; the condition is reinforced.","tokens_in":18059,"tokens_out":11460,"duration_ms":120252,"concrete_test":"Re-run the Unbalanced Sampling experiment on Cora, CiteSeer, PubMed, and ogb-arXiv with a single fixed clustering representation for pseudo-label assignment (for example, Deep Graph Infomax embeddings or raw features) used for both GPN and GCN, keeping each model's training loss and prediction head unchanged. If GCN's average Planetoid oracle-drop penalty falls materially below 9%, or GPN's 1% penalty changes, the reported robustness gap is not attributable to the prototypical paradigm alone. Also report cluster purity (adjusted Rand index against true labels) for the two models at each budget step to quantify how much of the gap is driven by clustering-space geometry.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central robustness claim is that dropping the class oracle costs the discriminative GCN about 9% on Planetoid while the prototypical GPN loses only 1% (Tables II and III, with Table V averages). But the pseudo-label mechanism in Section IV.B is not model-independent: k-medoids runs on the current model's embedding space, and Section VI concedes that for the discriminative model this is effectively k-medoids on logits, 'which is sub-optimal, as these do not represent a metric space', whereas GPN embeddings are trained with Euclidean prototype losses and are said to be 'designed to represent Euclidean space'. Thus the measured asymmetry combines two changes at once: the learning paradigm and the geometric suitability of the representation being clustered. The result may be true for the specific GPN-based pipeline, but it does not isolate 'prototypical versus discriminative' as the cause of the 1% versus 9% robustness difference. This is a confound rather than a circularity; it is addressable by a controlled experiment that holds the clustering representation fixed.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses cold-start active learning for vertex classification on a fully unlabeled graph. It compares a prototypical model (GPN) with a discriminative model (GCN) under three decreasingly realistic assumptions: a balanced class oracle, k-medoids pseudo-labeling without an oracle, and an estimated number of classes. Label propagation is optionally used to expand the labeled set. The authors report 10-seed experiments on Cora, CiteSeer, PubMed, Reddit2, and ogb-arXiv, concluding that prototypical models outperform discriminative models with small label budgets and that dropping the class oracle costs GCN about 9% accuracy on Planetoid while GPN loses only about 1%. Source code is provided.","tokens_in":18312,"tokens_out":16777,"duration_ms":147989,"significance":"The setting is timely and practical: few-shot node classification usually assumes a class oracle or pre-labeled auxiliary classes, and the paper attacks the harder cold-start scenario. The study is reasonably broad in datasets and assumption levels, uses 10 seeds with standard deviations, includes an imperfect-annotator robustness check, and releases code. If the central comparison survives scrutiny, the recommendation of a prototypical model for cold-start active learning on homophilic graphs would be a useful, actionable result. However, as detailed below, the headline robustness gap is currently confounded, selective in its averaging, and the experimental bookkeeping needs clarification before the claims are fully supported.","major_comments":[{"comment":"The abstract states that 'the average performance of discriminative models decreases by 9%' while 'prototypical models only lose 1%' when the class oracle is dropped, but Table V shows that these are the Planetoid-only numbers in parentheses, not the all-dataset averages. Across all five datasets, the GPN average rises from 53.3 (balanced) to 55.0 (k-medoids), while the GCN average falls from 41.5 to 38.3; the Planetoid-only changes are 61.6 to 60.4 for GPN and 56.9 to 47.9 for GCN. Please either qualify the abstract to say 'on the Planetoid datasets' or use the all-dataset averages, because the current wording overstates a result that changes sign in aggregate for the prototypical model.","section":"Abstract and Table V"},{"comment":"The oracle-robustness comparison is confounded. In the Unbalanced Sampling experiment, pseudo-labels used for sampling come from k-medoids clustering of GPN Euclidean-trained embeddings, but for the GCN the same step is applied to logits, and Section VI explicitly concedes that logits 'do not represent a metric space' and calls this sub-optimal. The 9% versus 1% gap therefore combines two changes at once: the learning paradigm and the geometric suitability of the representation being clustered. To attribute the gap to prototypical versus discriminative learning, run a controlled condition with the clustering representation held fixed across both models (for example, cluster both on the same DGI or raw-feature representation, or augment the GCN with a metric-compatible embedding objective) and show that the robustness difference persists.","section":"Section IV.B and Section VI"},{"comment":"The budget and per-class sample accounting are internally inconsistent. Section IV.B fixes the budget at B = |C| * 5 over five sampling rounds and Algorithm 1 stops when |Vl| exceeds B, but Appendix D says that for Reddit2 and ogb-arXiv the model trains on 10 samples per class in each iteration. Five such rounds would give 50 samples per class, exceeding both the stated budget and the abstract's 'fewer than 20 samples per class' scope. In the Unknown Number of Classes experiment the estimated k differs from |C| (11 versus 7 on Cora, 13 versus 6 on CiteSeer), yet the text says the sampler draws k vertices per round while keeping B = |C| * 5; these statements cannot both hold. Please report the exact per-round sample count, the number of rounds actually executed, and the total budget for each dataset and experiment.","section":"Section IV.B, Appendix D, Algorithm 1"},{"comment":"The label-propagation hyperparameters alpha = 0.9 and the entropy threshold 0.2 were chosen from pre-experiments, but the manuscript does not report whether those pre-experiments used a held-out split. Because all reported accuracies are on the same five datasets, selecting these values on the same test data would make the label-propagation and absolute-accuracy results optimistic. Please add a sensitivity analysis over alpha and the entropy threshold, or describe a validation procedure that does not use test labels.","section":"Section III"},{"comment":"The Unknown Number of Classes experiment is run only on Cora, CiteSeer, and PubMed; Appendix E states that the DGI-based class estimation failed on Reddit2 and ogb-arXiv after 48 hours. The contribution claim that 'estimating the number of classes does not decrease performance' is therefore not supported for large graphs, and the abstract's 'across all experiments' phrasing overreaches. In addition, the elbow estimates are far from the true class counts (11 versus 7 on Cora, 13 versus 6 on CiteSeer), so a single point estimate is thin evidence for the claim; a sensitivity analysis over k would be more convincing.","section":"Section IV.B, Appendix E, Table IV"}],"minor_comments":[{"comment":"The total loss is written as L = Lr + lambda(Le + Lc), but the intra-class term was defined as Lp; please rename one of them for consistency.","section":"Section III"},{"comment":"The procedure says every experiment starts by assigning pseudo-labels via k-medoids clustering, but the Balanced Sampling experiment uses the ground-truth partition; please clarify that the k-medoids step only applies to the Unbalanced and Unknown settings.","section":"Section IV.B"},{"comment":"The legend labels 'LP-own' in Figures 6-8 are never defined in the text; please define this strategy or rename it.","section":"Section IV.B and Appendix I"},{"comment":"FeatProp reports identical values in all three experiments; please state explicitly whether it is rerun under each setting or is a static baseline, and explain how a static baseline is comparable across experiments.","section":"Tables II-IV"},{"comment":"The LP+Random and LP+Entropy rows are identical in several columns (e.g., Cora in Table II); please explain why entropy sampling degenerates to random sampling for the LP baseline.","section":"Tables II and III"},{"comment":"Please correct the many typos and stray fragments, including 'Appendix E.y having' and 'se an estimation' in Section IV.B, 'Appendix HDeviations' in Section V, and 'Pubmed' in Section VI, and fix 'Sogli et al.' to 'Solgi et al.' in Section II.B.","section":"Various"}],"recommendation":"major_revision","confidential_remarks":"I recommend major revision rather than rejection. The experimental core is reproducible and the paper attacks a real cold-start limitation, but the headline robustness result needs re-scoping to Planetoid, de-confounding of the clustering representation, and corrected budget reporting. These issues are addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a useful empirical comparison of prototypical versus discriminative models for cold-start vertex classification, with a real confound that the authors half-acknowledge. The headline number—GCN loses 9% when you drop the class oracle, GPN loses 1%—is backed by 10-seed runs on five datasets, and they ship code. That is more than most papers in this area.\n\nWhat is new is the systematic setup: start without labels, add active learning and label propagation, and relax assumptions in three stages. Prior work combines active learning with GPN or does open-world graph AL, but I do not see the direct comparison under these three oracle assumptions anywhere. The paper is also honest about its limits: homophily-only, transductive, and no held-out validation for two key hyperparameters.\n\nThe main soft spot is the confound in the robustness comparison. In the unbalanced sampling experiment, k-medoids clustering runs on GPN's Euclidean-trained embeddings but on GCN's logits. The authors say in Section VI that clustering logits is 'sub-optimal, as these do not represent a metric space.' So the 9% versus 1% gap conflates the learning paradigm with how cluster-friendly the representation is. A controlled experiment that clusters the same representation type (for instance, GCN's penultimate embeddings) would tell you whether prototypical models are inherently more robust or just have a better space for k-medoids. Without that, the causal claim stays open. To be fair, in the balanced-sampling experiment GPN also beats GCN on the small datasets, so model type matters independently; it is the specific robustness gap that is not cleanly attributed.\n\nMinor issues: alpha and the entropy threshold are chosen on the same datasets they are evaluated on; the unknown-classes experiment covers only three small datasets; and the class estimates are far off (Cora 11 vs 7, CiteSeer 13 vs 6) yet accuracy does not collapse, which is interesting but under-tested. The citation pattern is fine; they cite the earlier active-learning-plus-GPN work, so there is no narrative overclaim.\n\nI would bring this to a reading group and would cite the pipeline comparison with a caveat about the confound. It deserves a serious referee: the experiments are reproducible and the confound is addressable. I would not desk reject it.","headline":"Useful empirical comparison of cold-start pipelines, but the headline robustness gap between GPN and GCN is confounded by clustering different representation spaces.","tokens_in":18840,"tokens_out":4140,"would_cite":true,"duration_ms":37878,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Cold-start graph classification with tiny label budgets favors prototypical networks over GCNs, and the gap widens when the class oracle is replaced by clustering: prototypes lose 1% average accuracy while GCNs lose 9%.","keywords":["active learning","few-shot learning","vertex classification","graph neural networks","prototypical networks","cold start","k-medoids clustering","label propagation"],"falsifier":"Run the \"Unbalanced Sampling\" procedure on a heterophilic graph (homophily below roughly 0.3) and compare the prototypical model's accuracy loss against the balanced-oracle setting; or, once a few labels exist, compute the adjusted Rand index between the $k$-medoids pseudo-labels and the true labels after the first sampling round. If the prototype model loses as much as or more than the GCN's 9%, or the first-round pseudo-labels are near chance, the central robustness claim is limited to graphs whose embeddings cluster by class.","tokens_in":17857,"feed_emoji":"🏷️","tokens_out":11447,"duration_ms":93061,"temperature":0.7,"pith_summary":"Given a graph with no labels at all and a small budget for a human annotator, which model should you train? The paper argues for prototypical networks: models that assign each vertex the class of the nearest prototype (a representative embedding per class) rather than learning a decision boundary like a GCN. Its headline evidence is robustness under realism: when the idealized class oracle, which guarantees balanced per-class sampling, is replaced by $k$-medoids clustering of the model's embeddings, the GCN loses 9% average accuracy while the prototypical network loses only 1%. The same pattern holds when the number of classes is unknown and must be estimated, costing both models about another 1%. For any pipeline that must annotate an unlabeled graph from scratch with fewer than 20 labels per class, this makes prototype-based models the safer default.","feed_headline":"Drop the class oracle: prototypes pay 1%, GCNs pay 9%","feed_subtitle":"With fewer than 20 labels per class, prototypical networks beat GCNs across all three cold-start settings.","key_machinery":"The central mechanism is replacing the class oracle with $k$-medoids clustering: at each round, the current model's vertex embeddings are clustered into $k$ pseudo-classes and the active learning strategy draws vertices from these clusters, so that sampling a cluster medoid approximates sampling a representative vertex per class without knowing any labels. The prototypical model represents each class $c$ by a PageRank-weighted prototype $r_c = \\sum_{v \\in V_{l,c}} h_v \\, e_{s_v}$ over the labeled embeddings, and classifies a vertex by the closest prototype under Euclidean distance, with logits $p(c|v) = e^{-d(h_v,r_c)} / \\sum_{c' \\in C} e^{-d(h_v,r_{c'})}$. Training uses a three-part loss, an intra-class prototype distance term plus Euclidean and cosine regularizers that push prototypes apart. Label propagation, $Y' = \\alpha D^{-1/2} A D^{-1/2} Y + (1-\\alpha) Y$ with uncertain pseudo-labels filtered out by a normalized-entropy threshold of 0.2, exploits homophily to multiply the effective labeled set. For the third experiment, the class count is estimated by $k$-means over $k \\in [2,100]$ with the elbow rule on Deep Graph Infomax embeddings.","core_discovery":"The paper claims that in the cold-start regime, a transductive, fully unlabeled graph annotated round by round by a human-in-the-loop, prototypical models outperform discriminative models (GCNs) whenever fewer than 20 labeled samples per class are available. This is established through three experiments that successively relax assumptions: \"Balanced Sampling\" keeps a class oracle that supplies one labeled vertex per class per round; \"Unbalanced Sampling\" drops the oracle and instead partitions the embedding space with $k$-medoids clustering into pseudo-classes from which vertices are drawn; \"Unknown Number of Classes\" additionally estimates the class count by $k$-means with the elbow rule on Deep Graph Infomax embeddings. The central quantitative claim is that dropping the oracle costs the GCN 9% average accuracy on the Planetoid datasets but costs the prototypical network only 1%, with a further 1% for both when the class count is estimated. The paper also claims that label propagation on the homophilic graphs artificially extends the training set and helps most in early rounds, that medoid sampling is the best of the four active learning strategies tested, and that the prototypical model tolerates up to roughly 20% annotation noise.","pith_inferences":["I would expect the 1% versus 9% robustness gap to shrink or invert on heterophilic graphs (outside the tested homophily range of 0.44–0.77), because $k$-medoids pseudo-clusters will no longer track true classes; this is a direct, testable extension.","A cheap operational check follows from the paper's mechanism: once a handful of labels exist, compute the adjusted Rand index between $k$-medoids pseudo-labels and true labels; if it is near chance early on, the class-oracle-free advantage of prototypical models is unlikely to appear on that graph.","The results imply a reusable default recipe for cold-start graph annotation pipelines, prototype backbone, medoid sampler, label propagation with entropy filtering, that would be worth adopting as a standard baseline in future graph active learning studies."],"forward_implications":["When annotating a graph from scratch with fewer than 20 labeled samples per class, use a prototypical network rather than a GCN; the prototype model wins across all three experimental settings.","A class oracle is replaceable: $k$-medoids pseudo-label clustering costs the prototypical model only 1% average accuracy, so few-shot vertex classification does not require balanced per-class sampling.","The medoid active learning strategy outperforms random, entropy, and PageRank sampling on average, making it the default choice among the four tested.","Label propagation on homophilic graphs increases accuracy by effectively enlarging the training set, with the largest gains in the early annotation rounds.","Estimating the number of classes rather than assuming it costs about 1% more on average, so the method remains viable when the class count is unknown."],"supporting_citations":[{"why":"Supplies the prototypical network paradigm, class prototypes as means of support embeddings, which the paper adapts to graphs.","marker":"[7]"},{"why":"The GCN is the discriminative backbone and the main comparison model throughout the experiments.","marker":"[8]"},{"why":"Graph Prototypical Networks, the direct basis for the paper's prototypical model for attributed networks.","marker":"[12]"},{"why":"FeatProp, the active-learning baseline the paper must beat, which also uses k-medoids instance selection.","marker":"[21]"},{"why":"Provides the label propagation procedure used to artificially extend the labeled training set.","marker":"[28]"},{"why":"Source of the three-part prototypical loss, intra-class distance plus Euclidean and cosine prototype regularizers.","marker":"[30]"},{"why":"Defines the homophily measure used to characterize the datasets and justify label propagation.","marker":"[31]"},{"why":"Deep Graph Infomax embeddings used for estimating the number of classes in the third experiment.","marker":"[39]"}],"fun_headline_variants":["Prototypes beat GCNs when labels are scarce: 1% vs 9% loss without oracle","Without a class oracle, prototypes lose 1% accuracy; GCNs lose 9%","Active few-shot vertex classification: prototypes beat GCNs under 20 labels/class","Human-in-the-loop labeling: prototypes survive oracle removal, GCNs don't","No class oracle? Prototypes lose just 1%, GCNs 9% in few-shot classification"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that $k$-medoids clustering of the model's current embeddings produces pseudo-classes that align with the true classes well enough that sampling one medoid per pseudo-class approximates balanced per-class sampling; the paper only tests homophilic graphs (homophily 0.44–0.77), and if the embeddings do not cluster by true class, the reported 1% robustness to dropping the class oracle fails.","fun_headline_variants_meta":{"raw":{"variants":["Prototypes beat GCNs when labels are scarce: 1% vs 9% loss without oracle","Without a class oracle, prototypes lose 1% accuracy; GCNs lose 9%","Active few-shot vertex classification: prototypes beat GCNs under 20 labels/class","Human-in-the-loop labeling: prototypes survive oracle removal, GCNs don't","No class oracle? Prototypes lose just 1%, GCNs 9% in few-shot classification"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000741,"raw_usage":{"total_tokens":3397,"prompt_tokens":1125,"completion_tokens":2272,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":741,"completion_tokens_details":{"reasoning_tokens":2152}},"tokens_in":741,"tokens_out":2272,"duration_ms":14361,"temperature":1.0,"reasoning_tokens":2152,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:12:00.816310+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the \"Unbalanced Sampling\" procedure on a heterophilic graph (homophily below roughly 0.3) and compare the prototypical model's accuracy loss against the balanced-oracle setting; or, once a few labels exist, compute the adjusted Rand index between the $k$-medoids pseudo-labels and the true labels after the first sampling round. If the prototype model loses as much as or more than the GCN's 9%, or the first-round pseudo-labels are near chance, the central robustness claim is limited to graphs whose embeddings cluster by class.","supporting_citations":[{"cited_title":"Prototypical networks for few- shot learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the prototypical network paradigm, class prototypes as means of support embeddings, which the paper adapts to graphs."},{"cited_title":"Graph prototypical networks for few-shot learning on attributed networks,","cited_arxiv_id":null,"evidence_quote":"Graph Prototypical Networks, the direct basis for the paper's prototypical model for attributed networks."},{"cited_title":"Active learning for graph neural networks via node feature propagation,","cited_arxiv_id":null,"evidence_quote":"FeatProp, the active-learning baseline the paper must beat, which also uses k-medoids instance selection."},{"cited_title":"Combining label propagation and simple models out-performs graph neural net- works,","cited_arxiv_id":null,"evidence_quote":"Provides the label propagation procedure used to artificially extend the labeled training set."},{"cited_title":"Open-world graph active learning for node classification,","cited_arxiv_id":null,"evidence_quote":"Source of the three-part prototypical loss, intra-class distance plus Euclidean and cosine prototype regularizers."},{"cited_title":"Large scale learning on non-homophilous graphs: New benchmarks and strong simple methods,","cited_arxiv_id":null,"evidence_quote":"Defines the homophily measure used to characterize the datasets and justify label propagation."},{"cited_title":"Deep Graph Infomax,","cited_arxiv_id":null,"evidence_quote":"Deep Graph Infomax embeddings used for estimating the number of classes in the third experiment."}],"review_version":1}