{"id":"d7621196-c2c8-4644-a993-6ed6638b7a77","arxiv_id":"2508.16995","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A cross-attention context layer on top of graph neural networks yields small accuracy gains and uncertainty estimates for graph-level classification and regression.","lead":"The paper adds a context-aware prediction layer on top of graph neural networks, so the model looks at labeled example graphs before predicting a new one. It reports slight accuracy gains and aims to give cheaper uncertainty estimates than ensembles for graph-level tasks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1 never specifies that target and context sets are disjoint; independent sampling allows the context attention (Eq. 6) to read each target's own label during training, which would invalidate the reported E2E gains.","rationale":"The reader's weakest assumption (conditional independence in Eq. 3) is a modelling assumption common to neural processes; it could affect calibration but does not directly threaten the headline improvements. The variational overclaim is real but conceptual: even if the method is better described as an ANP-style conditional density estimator than as a true variational posterior predictive, the empirical tables could still support the method's usefulness. The overlap issue, by contrast, if present, undermines the very numbers that support the central claim. I therefore make it the primary concern. This is a good-faith reading of the pseudocode; I am not asserting the authors implemented it this way, only that the manuscript as written does not rule it out and no code is released to check. A conditional acceptance should require either a statement that T and C are sampled disjointly or the re-run described above. I keep the reader's CONDITIONAL verdict because the concern is unresolved rather than established: if the code samples C from L\\T, the published results may stand; if not, the empirical core of the paper is compromised.","tokens_in":23582,"tokens_out":7801,"duration_ms":88471,"concrete_test":"Run the training protocol described in Algorithm 1 on MUTAG, PTC, NCI1, and PROTEINS under two conditions: (a) independent T and C as literally written, allowing overlap; (b) C sampled from L\\T (disjoint). Keep all other hyperparameters from Table 9 fixed. Compare test accuracy, ECE, NLL, and selective-prediction curves. Also report the expected and observed |T∩C|. If condition (a) is materially better than (b), the published E2E results are explained by target-label leakage; if they match, the missing disjointness statement is innocent. Inspecting the released code for a disjoint-sampling implementation would settle it immediately.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Algorithm 1 (line 8) samples target indices T⊂L and context indices C⊂L 'randomly' with no disjointness constraint, and Section 4.1 says both 'are sampled uniformly from the training data.' In the cross-attention value computation (Eq. 6), each context entry contributes W_v[x_j∥y_j]; if a target index i also lies in C, the query x_i matches key x_i and the model can attend to its own ground-truth label y_i. The training loss in Eq. (9) can then be driven down by copying the label out of the context rather than by learning a predictive relationship. This is not a negligible edge case: on TU datasets with small L (MUTAG has 188 graphs, PTC 344, NCI1 4110, PROTEINS 1113), with context size 64 and a typical mini-batch target size, the expected intersection |T∩C| is large (e.g., MUTAG with |T|=128 gives E|T∩C|≈43). Hyperparameters in Table 9 were tuned in E2E mode under this regime, so the selected configuration and the reported Tables 1–5 and Figure 2 could reflect leak-assisted training. At inference the target is a test graph not present in the context, so the shortcut is unavailable. If the authors' code samples C from L\\T, the concern is void, but the paper should state this explicitly; as written, the empirical central claim is not shielded from this failure mode.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GraphPPD, an amortized module that, given a graph encoder, produces predictive distributions for graph-level classification and regression by cross-attending to a set of labeled context graphs. Training minimizes the negative conditional log-likelihood (Eq. 9) over randomly sampled target and context subsets; inference averages over P sampled context sets (Eq. 10). Experiments compare against the base GNN/transformer encoders, MC dropout, and ensembles on 7 OGB datasets, 10 TU datasets, and ZINC-12k, plus a selective-prediction task. The central claims are that this constitutes a novel variational framework for the posterior predictive distribution and that it yields uncertainty-aware predictions with improved accuracy and selective-prediction performance at lower parameter cost than ensembles.","tokens_in":23866,"tokens_out":5643,"duration_ms":61273,"significance":"If the empirical results are valid, the paper provides a flexible way to attach a context-aware predictive head to existing graph encoders, with a broad evaluation spanning 18 datasets and including parameter/time comparisons against MC dropout and ensembles. The use of a proper-scoring-rule objective (Eq. 9) is a sound starting point for amortized predictive modelling, and the modular design described in Section 4.5 makes the framework easy to extend. However, the contribution as stated is weakened by the unsupported 'variational PPD' label and, more importantly, by an unresolved train-time leakage risk in the sampling protocol of Algorithm 1; the uncertainty-calibration evidence in Appendix A.3 is also mixed and does not uniformly support the uncertainty-awareness claims.","major_comments":[{"comment":"The training protocol does not require the target set T and context set C to be disjoint: Algorithm 1 line 8 samples both 'randomly' from L, and Section 4.1 states only that both are sampled uniformly from the training data. If a target index i also lies in C, then in the cross-attention computation of Eq. (6) the query x_i can attend to its own associated label y_i via the value vector W_v[x_i || y_i]. The training loss in Eq. (9) can then be minimized by copying labels out of the context rather than by learning a predictive relationship. At inference, targets are test graphs not present in the context set, so the shortcut is unavailable; this asymmetry means the reported gains in Tables 1-5 and Figure 2 could be partly or wholly an artifact of this train-time leakage. The concern is not negligible on small datasets: for example, MUTAG has 188 graphs, and with a target batch of 128 and context size 64 (Table 9), the expected intersection size under uniform sampling without replacement is about 43. The authors must explicitly enforce and state C ∩ T = ∅ (or sample C from L \\ T) and rerun the experiments, or provide evidence from the released code that this disjointness is already enforced.","section":"Algorithm 1, Section 4.1, Eq. (6)"},{"comment":"The paper repeatedly describes the method as a 'variational modelling framework for the posterior predictive distribution', but no variational family, ELBO, KL divergence, or posterior over model parameters is ever defined. Eq. (1) gives the Bayesian posterior predictive as an integral over the parameter posterior p(γ | D_L), while the proposed objective Eq. (9) is an empirical conditional likelihood / proper-scoring-rule objective for the amortized model q_λ(y | G, D_C). No derivation shows that the quantity in Eq. (1) is approximated by Eq. (4). This is a load-bearing mismatch with the abstract and introduction: the contribution can be reframed as an amortized conditional predictive model in the style of neural processes, which is a legitimate contribution, but claiming 'variational' and 'posterior predictive' without a variational derivation overstates the theoretical basis and should be corrected.","section":"Section 2, Section 4.1, Section 4.3, Eq. (1) vs Eq. (9)"},{"comment":"The uncertainty-calibration evidence is mixed and does not consistently support the claim of improved uncertainty quantification. In Table 11, on the ECE criterion the E2E GraphPPD matches or loses to the base encoder in most cases (e.g., GIN ECE win/tie/loss 5/0/5; GMT ECE 3/0/7), and on NLL the results are similar (GIN NLL 5/0/5; GMT NLL 6/1/3). The 2-stage variant has large calibration degradations on several datasets (e.g., D&D ECE 0.175 vs GIN 0.128; PTC NLL 0.644 vs GIN 0.556). The selective-prediction results in Figure 2 compare only against the feature extractor, not against MC dropout or ensembles, and are not accompanied by calibration metrics. The main text's statements about 'effective uncertainty characterization' should be tempered, and the full ECE/NLL/Brier comparisons should be discussed honestly rather than deferred to the appendix.","section":"Table 11 (Appendix A.3), Section 6.6"}],"minor_comments":[{"comment":"The notation for the test set is confusing: the input line uses L for both the training set D_L and the test set {G_i}_{i ∈ L}, and line 15 says 'For each i ∈ L' where the context suggests the test set. Use distinct symbols, e.g., U for the unlabeled/test set.","section":"Algorithm 1"},{"comment":"The Brier Score rows appear to be identical to the NLL rows for every model and dataset (e.g., GIN Brier 0.416 equals GIN NLL 0.416). This is almost certainly a copy-paste error and must be fixed, since Brier scores for binary/multiclass classification should be computed separately.","section":"Table 11"},{"comment":"The GMT training-time block contains two rows both labeled 'Ours(2-stage)' (roughly 361.3 and 513.4 seconds for different datasets), which is confusing; one of these should presumably be 'Ours(E2E)'.","section":"Table 11"},{"comment":"Several hyperparameter entries are given as slash-separated values, e.g., '1/2', '2/1', '1/4'. It is unclear whether these represent multiple configurations, a range, or different settings for different folds/seeds; please clarify how these values were used.","section":"Table 9"},{"comment":"The conditional-independence assumption in Eq. (3) is stated without justification. For graph-level tasks with independently drawn test graphs it is plausible, but it should be explicitly identified as a modelling assumption, and the limitation should be discussed for settings where test graphs share latent structure beyond what the context captures.","section":"Section 4.1, Eq. (3)"}],"recommendation":"major_revision","confidential_remarks":"The train-time overlap concern in Major Comment 1 is the most important issue. If the authors' code already samples C from L \\ T, then the issue is resolved and the paper mainly needs a clarity fix; if not, the empirical central claim in Tables 1-5 and Figure 2 is not trustworthy. I recommend asking the authors to state the disjointness condition explicitly and to either provide the training code or rerun the experiments with an explicit disjoint-sampling protocol. Given that the fix is well-scoped, I do not recommend rejection at this stage, but the variational-framing overclaim and the mixed calibration results also need to be addressed before the paper can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Senior colleague, here's my take. GraphPPD is a useful application paper: it takes the attentive neural process cross-attention idea and puts it on top of a GNN encoder for graph-level classification and regression. The empirical coverage is unusually broad—18 datasets, MC dropout and ensemble comparisons, runtime and parameter counts, plus a selective-prediction experiment. The gains over the base encoders are modest but consistent, and the 2-stage mode (freeze encoder, train PPD module) is a practical selling point for people who already have a pretrained encoder.\n\nThe soft spots are about framing and a specification gap, not about the core math. First, the paper calls Eq. (4) a \"variational PPD,\" but there is no variational posterior and no ELBO. What they actually train is a conditional neural process: a cross-attention model that maps a context set to a predictive distribution for target embeddings. That is a legitimate thing, but the variational label is unearned and should be dropped or given a formal link to Eq. (1).\n\nSecond, and this is the one that matters: Algorithm 1 samples target and context indices \"randomly\" from L with no disjointness constraint. Section 4.1 says both are sampled uniformly from the training data. If that is literal, then in training the context can include the target's own graph and label, and the cross-attention in Eq. (6) can copy y_i from W_v[x_i || y_i]. On MUTAG with |T|=128, the expected intersection with a 64-size context is around 43. Even if the model does not always attend to itself, the learning problem is different from test time, where targets are never in the context. The authors might have intended C ~ L\\T; if so, they need to say that explicitly. Without a code release or a clarified statement, the reported numbers are not shielded from this failure mode. This is not a fabrication charge; it is an under-specification that a referee has to have resolved before trusting the empirical claims.\n\nThird, the uncertainty evidence is mixed. Table 11 shows ECE and NLL often worse than the plain GNN baseline on the TU datasets. The selective prediction curves in Figure 2 look nice but have no error bars and no competing uncertainty baselines. So the contribution to uncertainty quantification is not convincingly demonstrated.\n\nOverall: this is a paper for people building graph-level predictors who want a lightweight way to add a predictive distribution without paying the cost of ensembles. It deserves peer review, not desk rejection, but it needs revision: fix the variational terminology, specify the disjoint sampling (and ideally release code), and tighten the selective-prediction experiment with error bars and baselines. Set would_accept_peer_review=true.","headline":"A useful application of attentive neural process ideas to graph-level prediction, but the 'variational PPD' label is unearned and the training protocol has a target/context overlap ambiguity that needs to be resolved before the numbers are fully trustworthy.","tokens_in":24461,"tokens_out":4493,"would_cite":false,"duration_ms":46441,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A variational module that learns the posterior predictive distribution from labelled context graphs turns any graph encoder into an uncertainty-aware predictor for whole-graph classification and regression.","keywords":["posterior predictive distribution","graph-level inference","uncertainty quantification","graph neural networks","variational inference","cross-attention","selective prediction","amortized inference"],"falsifier":"Take a dataset where test graphs are generated in groups with a shared latent property (for example, molecules from the same synthesis batch with correlated solubility errors), train GraphPPD, and check whether adding the label of one test graph to the context set changes predictive probabilities for the other test graphs in the same group. If the conditional-independence model cannot use that within-group label information to sharpen its predictions, the central claim of a data-adaptive posterior predictive distribution fails on that setting.","tokens_in":23317,"feed_emoji":"🕸️","tokens_out":7154,"duration_ms":65327,"temperature":0.7,"pith_summary":"The paper sets out to give graph-level prediction tasks—classifying or regressing a property of an entire molecule, protein, or social network—the same kind of uncertainty quantification that node-level methods already enjoy. It proposes a variational framework, GraphPPD, that learns an approximation of the posterior predictive distribution directly from a training set of labelled graphs, in a data-adaptive way. The predictive distribution for a query graph is built by cross-attention over a sampled set of labelled context graphs, so the model can compare the query against similar training examples at inference time. If the framework works as claimed, any existing GNN or graph transformer can be upgraded to produce calibrated predictive distributions without redesigning the encoder, improving accuracy and enabling selective prediction.","feed_headline":"One attention module gives graph neural nets whole-graph uncertainty","feed_subtitle":"GraphPPD beats the base encoder and Monte Carlo dropout, matches ensembles with fewer parameters, and powers selective prediction.","key_machinery":"The machinery is the amortized posterior predictive module $q^{\\mathrm{emb}}_\\phi$, a cross-attention network that maps a target graph embedding and a set of labelled context graph embeddings to the parameters of a predictive distribution. It computes attention coefficients $\\alpha_{ij}$ between target query $x_i$ and context key $x_j$, aggregates context values $W_v[x_j \\Vert y_j]$ into a representation $r_i$, concatenates this with the target embedding, and passes the result through an MLP to produce either class probabilities or a Gaussian mean and covariance. This object carries the argument because it replaces the intractable integral over model parameters in the definition of the posterior predictive distribution with a single forward pass that is learned end-to-end or on top of a frozen encoder; the Monte Carlo average over context sets in Equation (10) is what turns context sampling into a source of predictive uncertainty.","core_discovery":"GraphPPD's central claim is that the posterior predictive distribution for graph-level labels can be learned by amortized variational inference rather than by Bayesian inference over GNN parameters. Conditioning on a context set of labelled graphs, the model factorizes the joint predictive distribution as a product over targets, $q_\\lambda(\\{y_i\\}_{i\\in T} \\mid \\{G_i\\}_{i\\in T}, D^{\\mathrm{graph}}_C) = \\prod_{i\\in T} q^{\\mathrm{emb}}_\\phi(y_i \\mid x_i, D^{\\mathrm{emb}}_C)$, where $x_i$ is the graph encoder's embedding. The factor $q^{\\mathrm{emb}}_\\phi$ is implemented with cross-attention: each target embedding attends to the context embeddings and labels, producing a context-aware representation that feeds a softmax head for classification or a Gaussian head with learned mean and covariance for regression. At inference, several random context sets are sampled and their predictive distributions are averaged by Monte Carlo. On 7 OGB molecular datasets, 10 TU datasets, and the ZINC-12k regression task, the paper reports that this module improves accuracy, ROC-AUC, or MAE over the underlying encoder (GIN, GINE, GMT, GraphGPS) in most settings, outperforms Monte Carlo dropout, and matches deep ensembles with fewer parameters and less training time.","pith_inferences":["The modular design implies the PPD module can be transplanted to any domain with a strong feature extractor—images, point clouds, or sequences—since the attention module consumes embeddings and labels only; the paper hints at this generality but runs no such experiment.","Relaxing the conditional-independence factorization in Eq. (3) to allow target-to-target attention or a shared latent variable would test whether correlated test labels carry information beyond the context set; this is a natural next step rather than a claim of the paper.","Decomposing the Monte Carlo average in Eq. (10) into context-sampling variance and within-context entropy would give practitioners a direct diagnostic for whether the module's uncertainty reflects genuine data adaptivity or irreducible label noise."],"forward_implications":["Adding GraphPPD's PPD module to GIN, GINE, GMT, or GraphGPS improves ROC-AUC, accuracy, or MAE on the majority of 18 graph-level benchmarks, with larger relative gains on the OGB molecular datasets (up to 3.93%).","The predictive uncertainty can drive selective prediction: when test graphs with the highest entropy are handed to an oracle reviewer, the collaborative accuracy and ROC-AUC curves of GraphPPD exceed those of the base encoder on datasets such as SIDER, TOXCAST, and TOX21.","The 2-stage variant, which freezes a pretrained encoder and trains only the PPD module on precomputed embeddings, retains most of the improvements while substantially cutting training time, so uncertainty awareness can be added post-hoc to existing GNN pipelines.","GraphPPD is statistically indistinguishable from deep ensembles in Wilcoxon signed-rank tests across the 17 classification datasets, while using fewer trainable parameters and less training time (for GIN on TU datasets, about 39.5% lower training time and 14.5% fewer parameters on average)."],"supporting_citations":[{"why":"Supplies the cross-attention construction that the PPD module is built on.","marker":"Kim et al., 2019"},{"why":"Establishes the neural-process-style target/context formulation that the framework adapts to graph-level supervised learning.","marker":"Garnelo et al., 2018b"},{"why":"Defines Monte Carlo dropout, the main uncertainty baseline the paper compares against.","marker":"Gal & Ghahramani, 2016"},{"why":"Defines deep ensembles, the accuracy and parameter-count baseline GraphPPD is measured against.","marker":"Lakshminarayanan et al., 2017"},{"why":"Provides the GIN encoder used for TU datasets and the base model whose accuracy the framework improves.","marker":"Xu et al., 2019"},{"why":"Provides the Graph Multiset Transformer encoder used as the second base model on TU datasets.","marker":"Baek et al., 2021"},{"why":"Provides the OGB molecular benchmarks on which the classification experiments are run.","marker":"Hu et al., 2021"},{"why":"Provides the ZINC-12k regression benchmark and its MAE protocol.","marker":"Dwivedi et al., 2020"},{"why":"Provides the GraphGPS graph transformer baseline for ZINC-12k regression.","marker":"Rampášek et al., 2022"},{"why":"Provides the selective prediction protocol used to evaluate the usefulness of the predictive uncertainty.","marker":"Kivlichan et al., 2021"}],"fun_headline_variants":["GraphPPD: Graph-level uncertainty with posterior predictive modelling","GraphPPD: Ensemble-level uncertainty with fewer parameters","GraphPPD: Uncertainty without Bayesian inference on parameters","GraphPPD: Whole-graph uncertainty via posterior predictive","GraphPPD: Cross-attention for graph-level predictive uncertainty"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that, once a batch of labelled context graphs is fixed, the labels of the test graphs are conditionally independent of one another; if test-graph labels carry shared structure beyond what the context set reveals, the learned 'posterior predictive' is misspecified.","fun_headline_variants_meta":{"raw":{"variants":["GraphPPD: Graph-level uncertainty with posterior predictive modelling","GraphPPD: Ensemble-level uncertainty with fewer parameters","GraphPPD: Uncertainty without Bayesian inference on parameters","GraphPPD: Whole-graph uncertainty via posterior predictive","GraphPPD: Cross-attention for graph-level predictive uncertainty"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001571,"raw_usage":{"total_tokens":6290,"prompt_tokens":983,"completion_tokens":5307,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":599,"completion_tokens_details":{"reasoning_tokens":5227}},"tokens_in":599,"tokens_out":5307,"duration_ms":39379,"temperature":1.0,"reasoning_tokens":5227,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:09:16.475739+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a dataset where test graphs are generated in groups with a shared latent property (for example, molecules from the same synthesis batch with correlated solubility errors), train GraphPPD, and check whether adding the label of one test graph to the context set changes predictive probabilities for the other test graphs in the same group. If the conditional-independence model cannot use that within-group label information to sharpen its predictions, the central claim of a data-adaptive posterior predictive distribution fails on that setting.","supporting_citations":[{"cited_title":"The task is to predict the constrained solubility (logP) of the molecule","cited_arxiv_id":null,"evidence_quote":"Provides the ZINC-12k regression benchmark and its MAE protocol."}],"review_version":1}