{"id":"29704db5-e3a5-4abd-9a2b-ade5804cb570","arxiv_id":"2605.22593","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Deep ensembles fail to capture meaningful epistemic uncertainty in message-passing GNNs due to epistemic collapse where independently trained networks converge to similar predictions.","lead":"Deep ensembles show little improvement over single models for uncertainty in graph neural networks, with gains mainly from averaging optimization noise rather than better uncertainty estimates. This questions the transfer of a standard technique from other domains to graph-structured data.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Epistemic collapse may be an artifact of the specific architectures, training regimes, and seven datasets rather than a general feature of message-passing GNNs.","rationale":"The reader's weakest_assumption already isolates the precise point at which the argument is least secure: whether the lack of diversity is caused by functional convexity that holds across message-passing GNNs or is an artifact of the experimental choices. My load-bearing concern is identical, so no adjustment to the UNVERDICTED verdict is warranted; the abstract alone cannot confirm generality.","tokens_in":1697,"tokens_out":425,"duration_ms":47509,"concrete_test":"Select one additional dataset outside the original seven (e.g., ogbn-arxiv or a heterophilic benchmark) and one non-standard message-passing architecture (e.g., GAT or GraphSAGE). Train an ensemble of 5–10 models under the paper's protocol, compute the mean pairwise predictive disagreement (KL divergence on softmax outputs for classification or variance for regression), and compare the resulting epistemic component and calibration metrics to the original tables. A substantial increase in disagreement would indicate the collapse is not general.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on the observation that independently trained GNNs produce overly similar predictions, which the authors attribute to functional (rather than weight-space) convexity and use to explain why ensembles yield only marginal gains from optimization stabilization instead of improved uncertainty. This interpretation is load-bearing because the paper's conclusion that 'deep ensemble success does not seamlessly transfer to graph machine learning' requires the collapse to be characteristic of message-passing GNNs in general. If the similarity arises only from the chosen models (e.g., standard GCN/GIN), optimization settings, or dataset properties (size, homophily, task type), then the aleatoric-epistemic decomposition and the dismissal of ensembles as an uncertainty method would not generalize. The abstract provides no indication that the authors tested alternative GNN layers, larger graphs, or heterophilic data to isolate this.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript benchmarks deep ensembles as an uncertainty quantification method for message-passing graph neural networks. Across seven datasets, it reports that ensembles yield only marginal gains over single models, which arise mainly from stabilizing optimization noise in point predictions rather than from meaningfully improved uncertainty estimates. Using an aleatoric-epistemic decomposition, the authors identify epistemic collapse: independently trained networks converge to overly similar predictions. They attribute this to functional (rather than weight-space) convexity and conclude that deep-ensemble success does not transfer to graph machine learning.","tokens_in":1899,"tokens_out":627,"duration_ms":38285,"significance":"If the central empirical findings hold, the work usefully challenges the default transfer of deep-ensemble uncertainty methods to GNNs and motivates GNN-specific alternatives. The aleatoric-epistemic decomposition and the explicit link between prediction similarity and the failure of disagreement-based epistemic uncertainty constitute a clear, falsifiable analysis. The paper's strength lies in its reproducible benchmarking protocol and the introduction of the epistemic-collapse observation as a concrete phenomenon to be explained or mitigated.","major_comments":[{"comment":"§4 (Experimental results): The claim that epistemic collapse is characteristic of message-passing GNNs in general rests on the seven chosen datasets and standard GCN/GIN architectures. Without ablations on alternative layers (GAT, GraphSAGE), heterophilic graphs, or larger-scale datasets, it remains possible that the observed prediction similarity is an artifact of the specific inductive biases, optimization settings, or dataset homophily rather than a general property; this directly affects the load-bearing conclusion that ensembles cannot capture epistemic uncertainty in GNNs.","section":"§4"},{"comment":"§3.3 (Aleatoric-epistemic decomposition): The decomposition treats disagreement across ensemble members as the primary source of epistemic uncertainty. The manuscript should explicitly verify that the chosen diversity metric remains valid under graph-structured dependencies (e.g., message-passing correlations) and report sensitivity to the number of ensemble members and training seeds; otherwise the quantitative attribution of marginal gains to optimization stabilization rather than uncertainty improvement is not fully isolated.","section":"§3.3"}],"minor_comments":[{"comment":"Abstract and §1: The seven datasets are described only as 'representing varied tasks and complexities'; listing their names, sizes, and task types (node classification, graph classification, etc.) would allow readers to assess coverage immediately.","section":"Abstract"},{"comment":"Figure captions and §4: Ensure all figures reporting ensemble vs. single-model metrics include error bars over multiple random seeds and clearly label the uncertainty decomposition components.","section":"§4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a good fit for a machine-learning venue focused on graphs or uncertainty. The citation list appears balanced, but the authors should confirm they have referenced the most recent GNN-uncertainty baselines that post-date the arXiv posting."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive and detailed feedback on our manuscript. The comments help clarify the scope and robustness of our findings on epistemic collapse in deep ensembles for message-passing GNNs. We address each major comment point-by-point below, indicating revisions made to the manuscript where appropriate.","responses":[{"response":"We appreciate the referee's emphasis on generalizability. Our experiments deliberately focused on canonical message-passing architectures (GCN and GIN) across seven datasets chosen to span different scales, tasks, and homophily levels, as these represent the most common inductive biases in the literature. We attribute epistemic collapse to functional convexity arising from the shared message-passing update rule rather than specific layer details. To address the concern, we have added new ablations in the revised Section 4 using GAT and GraphSAGE on both a homophilic dataset and a heterophilic one (Chameleon). These confirm similar levels of prediction similarity across ensemble members. For larger-scale datasets, we acknowledge practical compute limits prevented full replication but have expanded the discussion of scalability and potential limitations in the revised text. We believe these additions support the conclusion for standard message-passing GNNs while noting that future work could explore even broader settings.","revision_made":"yes","referee_comment":"[§4] §4 (Experimental results): The claim that epistemic collapse is characteristic of message-passing GNNs in general rests on the seven chosen datasets and standard GCN/GIN architectures. Without ablations on alternative layers (GAT, GraphSAGE), heterophilic graphs, or larger-scale datasets, it remains possible that the observed prediction similarity is an artifact of the specific inductive biases, optimization settings, or dataset homophily rather than a general property; this directly affects the load-bearing conclusion that ensembles cannot capture epistemic uncertainty in GNNs."},{"response":"Thank you for this methodological suggestion. The diversity metric (prediction variance across members) is computed on the final node or graph outputs after message passing, so graph-induced correlations are already reflected in the forward passes of each network. To explicitly verify robustness, we have added sensitivity analyses in the revised Section 3.3 and a new appendix subsection. These vary ensemble size (3 to 10 members) and training seeds, showing that the observed low disagreement and attribution of gains to optimization stabilization remain consistent. We also include a brief check correlating the metric with an alternative epistemic uncertainty proxy on a controlled synthetic graph task. These revisions better isolate the effects and strengthen the aleatoric-epistemic decomposition.","revision_made":"yes","referee_comment":"[§3.3] §3.3 (Aleatoric-epistemic decomposition): The decomposition treats disagreement across ensemble members as the primary source of epistemic uncertainty. The manuscript should explicitly verify that the chosen diversity metric remains valid under graph-structured dependencies (e.g., message-passing correlations) and report sensitivity to the number of ensemble members and training seeds; otherwise the quantitative attribution of marginal gains to optimization stabilization rather than uncertainty improvement is not fully isolated."}],"tokens_in":1399,"tokens_out":649,"duration_ms":51424,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main thing to know is that this paper shows standard deep ensembles do not transfer well to message-passing GNNs for uncertainty quantification. Across seven datasets the marginal gains come mostly from averaging out optimization noise in the point predictions, not from meaningfully better epistemic uncertainty estimates. They label the core problem epistemic collapse and link it to functional convexity rather than weight-space effects.","headline":"Deep ensembles add little to GNN uncertainty because independently trained models converge to nearly identical predictions.","tokens_in":2391,"tokens_out":137,"would_cite":false,"duration_ms":34528,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[],"headline":"Empirical study of epistemic collapse in GNN ensembles; no structural overlap with RS forcing chain","alignment":"orthogonal","rationale":"The paper's core machinery is an aleatoric-epistemic decomposition of ensemble uncertainty on message-passing GNNs, empirical benchmarking across seven datasets, and the hypothesis of functional (not weight-space) convexity causing prediction similarity. This has no connection to the RS foundation: the single-distinction forcing of J-cost, φ-ladder, 8-tick periodicity, or spacetime emergence. No RS theorem (e.g., reality_from_one_distinction, J-uniqueness, or Alexander-duality D=3) is paralleled or contradicted.","tokens_in":53753,"confidence":"high","tokens_out":159,"duration_ms":10032,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Deep ensembles provide only marginal gains over single graph neural networks because independently trained models converge to overly similar predictions.","keywords":["graph neural networks","deep ensembles","uncertainty quantification","epistemic uncertainty","epistemic collapse","message passing","functional convexity","aleatoric epistemic decomposition"],"falsifier":"Observing substantially higher prediction disagreement and correspondingly stronger uncertainty calibration from ensembles on a new collection of graph datasets or architectures would falsify the central claim.","tokens_in":2596,"feed_emoji":"📉","tokens_out":656,"duration_ms":36630,"temperature":0.7,"pith_summary":"The paper tests whether deep ensembles reliably quantify uncertainty when applied to graph neural networks that use message passing. It shows that ensembles add little value beyond a single model, and the small benefits mainly come from smoothing out random training variations in the point predictions themselves. An aleatoric-epistemic split of the uncertainty reveals that the networks agree too much on their outputs even though they start from different random initializations. This agreement removes the source of disagreement that ensembles normally use to measure epistemic uncertainty. Readers should care because graph data appears in many safety-critical settings where knowing when a prediction is uncertain matters for downstream decisions.","feed_headline":"Ensembles add little uncertainty value for graph neural networks","feed_subtitle":"Independently trained models converge to similar predictions, so disagreement fails to capture epistemic uncertainty.","key_machinery":"Epistemic collapse, the convergence of independently trained networks to nearly identical predictions on graph data despite different parameters, which eliminates the disagreement needed for epistemic uncertainty.","core_discovery":"Standard deep ensembles do not transfer their uncertainty-quantification success from other domains to message-passing graph neural networks. Across seven datasets the ensembles deliver only modest improvements over a lone model, and those gains arise chiefly from averaging optimization noise rather than from genuinely richer uncertainty estimates. The root cause is epistemic collapse: independently trained networks consistently produce nearly identical predictions because distinct parameter vectors map to almost the same function, a consequence of functional rather than weight-space convexity.","pith_inferences":["The same collapse may appear in other structured prediction settings where the input graph imposes strong functional constraints.","Single-model uncertainty techniques or explicit diversity-promoting regularizers could be tested as direct remedies.","If functional convexity is the driver, then architectural changes that increase the expressivity of the message-passing layers might restore ensemble diversity."],"forward_implications":["Ensembles cannot be treated as a default reliable method for epistemic uncertainty in graph neural networks.","Any observed performance lift from ensembles is explained by reduced training noise rather than better uncertainty.","New uncertainty methods tailored to the functional geometry of graph models are required.","The transfer of ensemble techniques from vision or tabular data to graphs must be re-examined rather than assumed."],"fun_headline_variants":["Ensembles fail to capture epistemic uncertainty in GNNs","Deep ensembles suffer from epistemic collapse on graphs","Minimal gains from GNN ensembles due to prediction similarity","Functional convexity drives similar outputs in ensemble GNNs"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The lack of prediction diversity is caused by functional convexity in the solution space and occurs generally for message-passing graph neural networks rather than only in the specific architectures and seven datasets examined.","fun_headline_variants_meta":{"raw":{"variants":["Ensembles fail to capture epistemic uncertainty in GNNs","Deep ensembles suffer from epistemic collapse on graphs","Minimal gains from GNN ensembles due to prediction similarity","Functional convexity drives similar outputs in ensemble GNNs"]},"model":"grok-4.3","cost_usd":0.007348,"raw_usage":{"total_tokens":3276,"prompt_tokens":620,"num_sources_used":0,"completion_tokens":60,"cost_in_usd_ticks":73478000,"prompt_tokens_details":{"text_tokens":620,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2596,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":620,"tokens_out":60,"duration_ms":36118,"temperature":1.0,"reasoning_tokens":2596,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-22T07:03:56.347843+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Observing substantially higher prediction disagreement and correspondingly stronger uncertainty calibration from ensembles on a new collection of graph datasets or architectures would falsify the central claim.","supporting_citations":[],"review_version":1}