{"id":"e508dc1c-2de6-4129-ac5c-93471934e003","arxiv_id":"2504.12715","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A vector-quantized graph autoencoder with annealed code selection and a hierarchical second-layer codebook outperforms 16 self-supervised baselines on link prediction and node classification.","lead":"This paper introduces a graph autoencoder that uses vector quantization with two fixes: a temperature-based annealing schedule to use more of the codebook, and a second-layer codebook that groups similar codes. On seven link-prediction benchmarks and eight node-classification benchmarks it reports the best average performance among 16 self-supervised baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Node-classification SOTA rests on unmatched baseline numbers and untested margins; link prediction also lacks matched-environment confirmation.","rationale":"The central claim is the empirical superiority assertion in the abstract. The strongest evidence is Table 1 (average rank 1.00) and Table 2 (average rank 1.25). For that claim to hold, HQA-GAE's numbers must be comparable to the baselines' numbers. Section 5.3 and Appendix A.3 reveal that Table 2 is assembled from multiple sources, and footnote 2 explicitly says MaskGAE and Bandana numbers are environment-sensitive. This makes the comparison uncontrolled precisely where the margins are small. On node classification, HQA-GAE's wins over the next-best method are frequently sub-significant, and it is second on two datasets, so the abstract overstates. I am not claiming fraud or that the method is weak; the correct response is to require a matched re-run and significance testing. The reader's weakest assumption about the Section 4.1 mechanism is interesting but secondary: even if VQ's role were fully explained, the empirical comparison would still need to be sound. The paper deserves credit for releasing code and for a consistent ablation story (annealing utilization curves, 2-layer vs 1-layer clustering), but those do not resolve the cross-source comparison issue. The verdict remains conditional pending the matched re-run.","tokens_in":23653,"tokens_out":12477,"duration_ms":129115,"concrete_test":"Run all 16 baselines using their public implementations under the exact environment, 5-fold splits, and seeds used for HQA-GAE, then apply paired significance tests (e.g., paired t-test or Wilcoxon signed-rank) between HQA-GAE and each baseline per dataset. If HQA-GAE no longer achieves average rank 1.00 in Table 1 or 1.25 in Table 2 under matched conditions, the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5.3 and Appendix A.3 show that Table 2 mixes results from at least three sources: DGI, GIC, MVGRL, BGRL, ARVGA, GraphMAE, MaskGAE, and S2GAE are taken from the S2GAE paper; Bandana's numbers are taken from Bandana; and VGAE, SeeGera, and HQA-GAE are run by the authors. The paper's own footnote 2 states that MaskGAE and Bandana results from a single public split are 'notably affected by software versions, CUDA environments, and random seeds,' yet Table 2 uses published numbers from those methods without a matched environment. HQA-GAE's node-classification margins over the runner-up are within one standard deviation on Cora, PubMed, Photo, Computers, and Physics (e.g., Cora 88.78±1.03 vs 88.59±1.35; PubMed 88.49±0.53 vs 88.16±0.57), and HQA-GAE is second on CS and ogbn-arxiv. No paired significance test is reported, so the abstract's claim that the model 'outperforms 16 representative baseline methods in ... node classification' is not established for node classification; the conclusion itself retreats to 'comparable performance.' Because the central claim is empirical superiority, an uncontrolled comparison is the load-bearing weakness.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper extends VQ-VAE to graph self-supervised learning, proposing HQA-GAE, which uses a GNN encoder, a softmax-based annealed code selection (Eqs. 4-5), and a hierarchical two-layer codebook (Eqs. 6-9) to address codebook underutilization and codebook space sparsity. In experiments on eight datasets, it reports average rank 1.00 on link prediction (Table 1) and 1.25 on node classification (Table 2), and it includes ablations on codebook size, temperature decay, loss weights, encoder choice, clustering quality, efficiency, and t-SNE visualizations.","tokens_in":23944,"tokens_out":6388,"duration_ms":58213,"significance":"The paper addresses a real gap in applying VQ-VAE to graph SSL and proposes two concrete, well-motivated mechanisms: annealed code selection to counteract winner-take-all underutilization, and a second-layer codebook to reduce sparsity. Strengths include the public code release, broad benchmark coverage, extensive ablations, a linear-complexity analysis, and a falsifiable claim about codebook utilization. If the empirical results survive matched-environment evaluation and the method specification is completed, HQA-GAE would be a competitive and conceptually novel graph SSL approach. The main limitation is that the headline empirical claim currently rests on uncontrolled baseline comparisons and untested margins, especially for node classification.","major_comments":[{"comment":"Table 2 mixes node-classification results from at least three different sources: DGI, GIC, MVGRL, BGRL, ARVGA, GraphMAE, MaskGAE, and S2GAE are taken from the S2GAE paper; Bandana's numbers are taken from Bandana; and only VGAE, SeeGera, and HQA-GAE are run by the authors. Footnote 2 states that MaskGAE and Bandana results are 'notably affected by software versions, CUDA environments, and random seeds,' yet those published numbers are used in the table without re-running in a matched environment. On five of the eight datasets (Cora, PubMed, Photo, Computers, Physics), HQA-GAE's margin over the runner-up is within one standard deviation (e.g., Cora 88.78±1.03 vs 88.59±1.35; PubMed 88.49±0.53 vs 88.16±0.57), and HQA-GAE is second on CS and ogbn-arxiv. No paired significance test is reported. The abstract's claim that the model 'outperforms 16 representative baseline methods in self-supervised link prediction and node classification' is therefore not established for node classification; the conclusion's more cautious 'comparable performance' is closer to the evidence. The authors should either re-run all baselines under the same protocol or substantially temper the node-classification claim.","section":"§5.3, Table 2, Appendix A.3"},{"comment":"The annealed code selection is the central novelty, but the forward pass is incompletely specified. The paper defines a softmax probability p_{i,j} over codes and a decaying temperature T_k, but never states whether training selects a code by sampling from this distribution, by the expected codebook embedding, or by argmax (with straight-through gradients), nor how the selection is made at inference. This distinction matters because sampling and expected-embedding behave differently under the annealing schedule and under the stop-gradient used in Eq. (9). Without this specification, the method is not fully reproducible and the ablation in Figure 3 cannot be interpreted unambiguously.","section":"§4.2, Eqs. (4)-(5)"},{"comment":"The paper's stated mechanism—that vector quantization 'forces the model to leverage the structural difference when reconstructing their raw features'—is supported only by the bar-chart comparison in Figure 2. That figure has no error bars, does not report the number of seeds, and reports no statistical test. Moreover, the comparison uses an MLP encoder, whereas the final HQA-GAE uses a GCN encoder, so Figure 2 does not directly establish the benefit of VQ in the model configuration actually used in Tables 1 and 2. Please add error bars or statistical tests and an ablation that removes VQ from the full GCN-based model.","section":"§4.1, Figure 2"},{"comment":"For link prediction, most baseline numbers in Table 1 are taken from the Bandana paper rather than run under the authors' environment; only DGI and GIC are reported as author-run. Given the sensitivity to software and CUDA versions documented in Footnote 2, the link-prediction superiority claim also lacks matched-environment confirmation. The margins are larger than in node classification, which reduces the severity of this concern, but a matched re-run or an explicit statement of protocol equivalence is needed before the abstract's claim is fully supported for link prediction as well.","section":"§5.2, Table 1, Appendix A.3"}],"minor_comments":[{"comment":"The denominator of the softmax in Eq. (4) is written with the summation index i, but it should sum over the codebook index j; as written, p_{i,j} is not a normalized distribution over codes.","section":"Eq. (4)"},{"comment":"The citation to S2GAE in §5.3 and Appendix A.3 is given as [49] (Bandana) rather than [36]; additionally, the baseline name 'AGVGA' appears in §5.1 and A.3, while the tables and reference [27] use 'ARVGA'.","section":"§5.3, Appendix A.3, References"},{"comment":"Table 4 contains typos in dataset names: 'Cara', 'CitrSeer', and 'Cumputers' should be 'Cora', 'CiteSeer', and 'Computers'.","section":"Table 4"},{"comment":"The abstract says the model 'outperforms 16 representative baseline methods in self-supervised link prediction and node classification,' while the conclusion says it 'shows comparable performance in node classification'; these statements are inconsistent and should be aligned.","section":"Abstract vs. Conclusion"},{"comment":"Figure 2 should state the number of runs and show error bars or confidence intervals; without them, the phrase 'significantly enhance' in Section 4.1 is not substantiated.","section":"Figure 2"},{"comment":"Appendix A.4 refers to the GitHub repository for the remaining hyperparameters; for archival reproducibility, the per-dataset values of M, C, T0, gamma, lambda, alpha, beta, and the negative-sampling count should be included in the paper itself.","section":"Appendix A.4"},{"comment":"Section 5.5 says MaskGAE 'lags behind ours in performance by nearly 10%' on Physics link prediction; Table 1 shows MaskGAE AUC 91.94 vs HQA-GAE 98.37 (about 6.4 percentage points) and AP 86.33 vs 98.22 (about 11.9 percentage points), so the claim should specify the metric.","section":"§5.5"},{"comment":"The negative-sampling edge reconstruction loss uses a set E− of negative edges, but the paper does not state how many negative edges are sampled per positive edge; this should be specified for reproducibility.","section":"Eq. (8)"}],"recommendation":"major_revision","confidential_remarks":"The practice of importing baseline numbers from prior papers is common in this area, but the combination of unmatched numbers with a strong abstract claim is risky. I recommend requiring either matched-environment re-runs or revised claims, since the node-classification margins are within one standard deviation on several datasets. The GitHub release should also be used to verify the exact code selection mechanism during revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read this as a solid empirical paper with one strong result and one overstated one.\n\nWhat's new: combining annealed softmax code selection with a second-layer codebook for vector-quantized graph autoencoders, applied to self-supervised node representation learning. The individual pieces are known (Gumbel-Softmax, VQ-VAE-2, prior VQ-for-graphs work), but the specific combination for graph SSL is new, and the authors ship code. The paper is well organized, the ablations are thorough (codebook size, temperature decay, encoder choice, loss weights), and the link prediction results are genuinely consistent: HQA-GAE ranks first on all seven datasets for both AUC and AP, often by large margins. I believe that result is real.\n\nThe node classification claim, as stated in the abstract, is not established. Table 2 mixes published numbers from at least three sources (S2GAE, Bandana, and the authors' own runs), and the paper's own footnote admits that these numbers are sensitive to software and seed. On Cora, PubMed, Photo, Computers, and Physics, HQA-GAE's margin over the runner-up is within one standard deviation, and it is outright second on CS and ogbn-arxiv. No paired significance test is reported. The abstract says \"outperforms 16 representative baseline methods in ... node classification,\" but the conclusion wisely retreats to \"comparable performance.\" That internal inconsistency matters.\n\nOther soft spots, in proportion: the code selection mechanism is under-specified. Is the model sampling from the annealed softmax, taking the expected code, or gradually switching to argmax? The text implies sampling, but the loss equations suggest a single selected code. This is fixable by looking at the code, but a paper should say it. The mechanism claim in Section 4.1 — that VQ forces the model to use structure when features are similar — is supported only by an MLP-encoder experiment with no significance testing, and the final model uses a GNN encoder, so the transfer is not demonstrated. The link prediction baselines are also from published numbers, but the uniform wins make that concern minor.\n\nOverall: this paper deserves a serious referee and, for graph SSL researchers, a careful read. If it were under review, I would ask for a matched-environment baseline rerun for node classification, an explicit statement of the code selection mechanism, and a toned-down abstract. As published, I'd cite the link prediction result but not the node classification superiority claim.","headline":"Link prediction wins are probably real, but the node classification SOTA claim is oversold by unmatched baselines and margins inside one standard deviation.","tokens_in":24474,"tokens_out":2951,"would_cite":true,"duration_ms":33151,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a graph autoencoder with a vector-quantization bottleneck, annealed code selection, and a second-layer clustering codebook beats 16 self-supervised baselines on link prediction and node classification.","keywords":["graph self-supervised learning","vector quantization","graph autoencoder","codebook underutilization","annealing-based code selection","hierarchical codebook","link prediction","node classification"],"falsifier":"A direct test is to train VQ-GAE and a plain GAE with an MLP encoder that sees only node features, as in the paper's preliminary experiment, but on a graph with planted community structure, and measure whether nodes sharing a code but having different neighborhoods are reconstructed more accurately than the plain GAE would predict. If the gap between VQ-GAE and GAE disappears when node features are made uninformative, the claimed structural mechanism is confirmed; if the gap persists, quantization is helping in some other way.","tokens_in":23475,"feed_emoji":"🕸️","tokens_out":10694,"duration_ms":92133,"temperature":0.7,"pith_summary":"Vector-quantized autoencoders, a staple of image and audio generation, have been slow to reach graphs. This paper tries to close that gap by showing that inserting a discrete codebook between a graph encoder and decoder helps the model capture graph topology, and by solving two obstacles that appear when quantization meets graph data: many codebook entries go unused, and the entries that are used are treated as isolated points even though graph nodes are related. The proposed model, HQA-GAE, replaces hard code selection with a temperature-annealed softmax, so training first explores many codes and later concentrates on the useful ones, and it stacks a second, smaller codebook on top of the first so similar codes are pulled together. On eight datasets the model reports the best link-prediction score on all of them and the best node-classification score on six of eight, against 16 contrastive and autoencoding baselines. If these results hold, perturbation-free graph self-supervision is a stronger route than the augmentation-heavy methods that dominate the area.","feed_headline":"16 baselines beaten by a vector-quantized graph autoencoder","feed_subtitle":"Annealed code selection and a two-layer codebook beat perturbation-heavy rivals on eight graph datasets.","key_machinery":"The load-bearing object is the two-layer vector-quantized bottleneck between the encoder and the decoders. A first-layer codebook of $M$ embeddings quantizes each encoder output $h_i$ to the nearest code $e_{1,i}$; the node decoder reconstructs features from $e_{1,i}$ while an edge decoder scores links from $h_i$. Because hard argmax selection starves most codes, selection is softened to a probability $p_{i,j} \\propto \\exp(\\mathrm{sim}(h_i, e_j)/T)$ with the temperature annealed as $T_k = \\max(\\gamma T_{k-1}, \\epsilon)$, so early training explores the codebook broadly and later training sharpens onto the effective codes. To fight codebook sparsity, a second-layer codebook of $C < M$ embeddings clusters the first-layer codes by maximizing $\\sum_j \\sum_{i \\in S_j} \\mathrm{sim}(e_{1,i}, e_{2,j})$, which pulls similar codes together and keeps dissimilar codes apart. The training loss combines scaled-cosine node reconstruction, negative-sampled edge reconstruction, and two vector-quantization losses, one per codebook layer.","core_discovery":"The central discovery is that vector quantization does useful work in graph autoencoders even though the quantized code is not the final node representation. Because the codebook is much smaller than the node set, nodes with similar raw features are forced into the same code; the paper argues that reconstruction then can only succeed if the decoder reads the structural differences between those nodes, and that this pressure feeds back through straight-through gradients to make the encoder's continuous embeddings carry more topology. On top of that mechanism, HQA-GAE addresses codebook underutilization by selecting codes with a softmax over code similarities at a temperature that starts broad and sharpens as training proceeds, and it addresses codebook sparsity by adding a second-layer codebook that effectively clusters the first-layer codes. The paper reports that this design outperforms 16 baselines in self-supervised link prediction and node classification across eight datasets, with average ranks of 1.00 and 1.25 respectively, and that the two-layer codebook improves clustering metrics over a single-layer codebook.","pith_inferences":["I infer the mechanism could be tested directly: track which nodes share first-layer codes and compare their neighborhoods; if same-code nodes with different structure are reconstructed accurately, the structural-differentiation story is supported, and if not, the gain must come from the discrete bottleneck acting as a regularizer rather than from topology capture.","The two-layer codebook resembles a learned cluster hierarchy over the first-layer codes, so a natural extension is to make the number of second-layer codes adaptive to the graph's class or community structure rather than fixed, which the sensitivity analysis already hints at.","Because the final embeddings are the continuous encoder outputs rather than the discrete codes, HQA-GAE is not yet a discrete graph representation; it behaves more like a regularized autoencoder, and a follow-up that uses the codes themselves as downstream features would test whether the discrete space is truly informative.","The annealing schedule is controlled by one decay factor, and datasets with very different sizes or feature distributions may need dataset-specific schedules; a self-tuning version that adapts temperature to codebook utilization is a concrete testable improvement."],"forward_implications":["If the reported results hold, self-supervised graph learning does not need hand-crafted augmentations or masking: a perturbation-free reconstruction objective with a discrete bottleneck matches or beats contrastive and masked-autoencoder baselines.","Annealing-based code selection is an alternative to Gumbel-Softmax for improving codebook utilization in graph vector-quantization models, and the experiments tie utilization and downstream accuracy to the decay factor, with performance peaking near $\\gamma = 0.9$.","The hierarchical second-layer codebook improves k-means clustering metrics on node representations, implying that relating codes to each other yields better representation geometry for downstream tasks.","The method's time and space complexity are linear in the number of nodes and edges when hyperparameters are fixed, which supports scaling to the 169k-node ogbn-arxiv graph.","Concatenating intermediate GNN layer outputs consistently beats using only the final layer as the node representation, across GCN, GAT, GraphSAGE, and GIN encoders."],"supporting_citations":[{"why":"Supplies the VQ-VAE discrete codebook framework, including the commitment and codebook losses, on which the graph autoencoder is built.","marker":"[38]"},{"why":"Provides the scaled cosine error for node reconstruction and is a masked-autoencoder baseline that HQA-GAE must beat.","marker":"[15]"},{"why":"Contributes the edge-decoder formulation with negative sampling and is a key masked-graph baseline.","marker":"[22]"},{"why":"Supplies the evaluation protocol and baseline results for link prediction and node classification, and is the strongest reported competitor.","marker":"[49]"},{"why":"Earlier vector-quantized graph work that this paper distinguishes by returning to the full VQ-VAE training scheme.","marker":"[6]"},{"why":"A vector-quantized graph model that uses labeled data, which establishes the self-supervised gap this paper addresses.","marker":"[45]"},{"why":"Prior VQ-VAE application restricted to molecular graphs, used to motivate the need for general graph tasks.","marker":"[43]"},{"why":"Provides the k-means clustering view behind the hierarchical second-layer codebook objective.","marker":"[2]"},{"why":"The GCN used as the encoder in the main experiments, with encoder choice shown to affect representation quality.","marker":"[19]"},{"why":"The GAT used as the node-feature decoder in the architecture.","marker":"[40]"}],"fun_headline_variants":["Vector quantization boosts graph autoencoders, beats 16 baselines","Annealed code selection and two-layer codebooks win on graphs","HQA-GAE: beating 16 baselines with smarter codebook design","Graph autoencoder with hierarchical codebook tops 16 baselines","Better topology capture via quantized codes and annealing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The explanation for the model's advantage rests on the premise that forcing similar-featured nodes into the same code makes the decoder rely on each node's structural context, and that this pressure carries back into the encoder; if that premise fails, the empirical gains would stand without the proposed mechanism.","fun_headline_variants_meta":{"raw":{"variants":["Vector quantization boosts graph autoencoders, beats 16 baselines","Annealed code selection and two-layer codebooks win on graphs","HQA-GAE: beating 16 baselines with smarter codebook design","Graph autoencoder with hierarchical codebook tops 16 baselines","Better topology capture via quantized codes and annealing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000211,"raw_usage":{"total_tokens":1425,"prompt_tokens":968,"completion_tokens":457,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":370}},"tokens_in":584,"tokens_out":457,"duration_ms":4663,"temperature":1.0,"reasoning_tokens":370,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:24:52.270821+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test is to train VQ-GAE and a plain GAE with an MLP encoder that sees only node features, as in the paper's preliminary experiment, but on a graph with planted community structure, and measure whether nodes sharing a code but having different neighborhoods are reconstructed more accurately than the plain GAE would predict. If the gap between VQ-GAE and GAE disappears when node features are made uninformative, the claimed structural mechanism is confirmed; if the gap persists, quantization is helping in some other way.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the evaluation protocol and baseline results for link prediction and node classification, and is the strongest reported competitor."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Earlier vector-quantized graph work that this paper distinguishes by returning to the full VQ-VAE training scheme."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"A vector-quantized graph model that uses labeled data, which establishes the self-supervised gap this paper addresses."}],"review_version":1}