{"id":"faade847-7b62-4ee9-bf82-89d24f99af61","arxiv_id":"2508.08551","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"UQGNN couples diffusion graph convolution, temporal convolution, and a multivariate Gaussian output head to jointly predict heterogeneous urban phenomena with uncertainty, and it outperforms twelve baselines across four city datasets.","lead":"Researchers built UQGNN, a graph neural network that forecasts several city phenomena at once, such as taxi, bike, subway, crime, and crash counts, and reports how uncertain each forecast is. Tests on data from Shenzhen, New York, and Chicago suggest it beats twelve existing models by roughly 2 to 5 percent on both accuracy and uncertainty metrics.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1 contradicts the headline claim: on Chicago UQGNN is worse than the best baseline on MAE, CRPS, and MPIW, and the stated 5% Shenzhen gain is not what the table shows.","rationale":"The reader's weakest assumption (unstated chronological split, Appendix B.3) is a legitimate experimental-validity concern and I partially agree with it. But it is not the most load-bearing problem: the paper's own Table 1 already disproves the central claim as stated. The Chicago row shows UQGNN losing to at least one baseline on MAE, CRPS, and MPIW, and the Shenzhen percentage improvements are arithmetically inconsistent with the table. This is an internal inconsistency, not a disagreement with external consensus, and it does not depend on unavailable code or on assumptions about the split. A revised version could correct the claims, add error bars, and clarify the split, but the current version's central empirical assertion is not supported. The verdict should therefore move from CONDITIONAL to REJECT for the current text.","tokens_in":28225,"tokens_out":9768,"duration_ms":96582,"concrete_test":"Take Table 1 as the source of truth. For each dataset and each of the six metrics, identify the best baseline value and compute (best_baseline - UQGNN)/best_baseline. Compare each recomputed percentage with the percentages in Section 5.2 and the abstract, specifically Chicago MAE (1.672 vs 1.680), Chicago CRPS (0.664 vs 0.680), and Shenzhen MAE (10.053 vs 9.717). If any stated improvement is negative or differs by more than one percentage point from the recomputed value, the 'consistently outperforms' and '5%' statements are not supported.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim (abstract; Section 5.2) is that UQGNN 'consistently outperforms' baselines in both accuracy and uncertainty quantification, with 'approximately 5%' gains on Shenzhen and 2% on Chicago. Table 1 does not support this. On Chicago, the best baseline MAE is SUMformer at 1.672 (DeepSTUQ is 1.673); UQGNN is 1.680, so it loses on MAE. On CRPS, STGCN achieves 0.664 and UQGNN 0.680, so UQGNN is not the best. On MPIW, CF-GNN achieves 2.324 and UQGNN 2.333. Thus UQGNN loses on three of six Chicago metrics, including both the headline accuracy metric (MAE) and the headline uncertainty metric (CRPS). On Shenzhen, the best baseline MAE is DiffSTG at 10.053; UQGNN at 9.717 is a 3.3% improvement, not 5%; the CRPS improvement relative to STZINB's 9.562 is 12.2%, not 5%. The claimed 'consistently outperforms' is therefore contradicted by the paper's own reported numbers, independent of the unstated chronological-split issue and of run-to-run variance. The claim could only hold under a different definition of 'best baseline' or different numbers than those printed in Table 1.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes UQGNN, a graph neural network framework for multivariate spatiotemporal prediction with uncertainty quantification. The framework has two main components: an Interaction-aware Spatiotemporal Embedding (ISTE) module, containing a Multivariate Diffusion Graph Convolutional Network (MDGCN) and an Interaction-aware Temporal Convolutional Network (ITCN), and a Multivariate Probabilistic Prediction (MPP) module that outputs distribution parameters (mean and covariance for a multivariate Gaussian) and is trained with a negative log-likelihood loss. The authors evaluate UQGNN on four public multivariate datasets from Shenzhen, NYC, and Chicago, comparing against 12 baselines with six metrics (MAE, RMSE, MAPE, KL, MPIW, CRPS). The central claim is that UQGNN consistently outperforms state-of-the-art baselines in both prediction accuracy and uncertainty quantification, with approximately 5% improvements on the Shenzhen dataset. The paper also includes ablations, an analysis of cross-phenomenon interactions, and a comparison of several multivariate distributions.","tokens_in":28481,"tokens_out":7022,"duration_ms":69495,"significance":"If the empirical claims were fully supported, the paper would be a useful contribution: it addresses a real gap by modeling uncertainty jointly across heterogeneous urban phenomena, releases code, evaluates on public datasets against twelve baselines, and provides ablation studies and distribution comparisons. The data-driven motivation in Section 2 is reasonable and clearly presented. However, the significance is currently limited by inconsistencies between the headline claims and the reported numbers, missing statistical rigor, an unspecified temporal split, and ambiguities in the equations that define the two claimed innovations. The conceptual framework and the released code are strengths, but the evidence as presented does not yet substantiate the central claim of consistent superiority.","major_comments":[{"comment":"The abstract and Section 5.2 state that UQGNN 'consistently outperforms' all baselines and achieves approximately 5% improvements, but Table 1 does not support this. On the Chicago dataset, UQGNN's MAE (1.680) is worse than SUMformer (1.672), its CRPS (0.680) is worse than STGCN (0.664), and its MPIW (2.333) is worse than CF-GNN (2.324). On Shenzhen, the MAE gain over the best baseline DiffSTG is 3.3% (9.717 vs. 10.053), not 5%, while the CRPS gain over STZINB is 12.2% (8.399 vs. 9.562), also not 5%. The headline claim needs to be corrected to a dataset-by-dataset statement or supported by a consistent definition of the reported percentages.","section":"Table 1; Section 5.2; Abstract"},{"comment":"All reported metrics are single-run point estimates with no error bars, confidence intervals, or significance tests. Since the claimed advantages over the best baselines are only 2-5% on most metrics, run-to-run variance could easily change the ranking; for example, the Chicago MAE gap between UQGNN and SUMformer is 0.008. The authors should report results over multiple random seeds with mean and standard deviation, or provide paired significance tests for the main comparisons.","section":"Tables 1-4; Section 5.1"},{"comment":"The data split is described only as 'training, validation, and testing subsets with a ratio of 8:1:1' but it is not stated that the split is chronological. With a 12-step input window and a horizon of 1, a random or shuffled split places test samples whose historical windows overlap training periods, leaking future information and inflating the reported gains. The chronological nature of the split must be stated explicitly, or the experiment must be rerun with a temporal split.","section":"Appendix B.3"},{"comment":"The two interaction-aware modules are not correctly specified as printed. In Eq. (5), the right-hand side uses H^l_m with the same index m as the left-hand side, and the summation over m=1 to M is therefore vacuous (or yields M copies of the same single-mode diffusion if Theta is shared); no cross-layer coupling between different phenomena appears in the equation. In Eq. (6), the sum over m of f(i) X^m is exactly a standard convolution over the M input channels, so the claimed interaction-aware design reduces to the default multi-channel TCN behavior. The equations need to be rewritten to show the cross-phenomenon mixing, for example by indexing Theta by the source phenomenon or by summing over H^l_{m'} with m' different from m.","section":"Section 4.2.1, Eq. (5); Section 4.2.2, Eq. (6)"},{"comment":"The probabilistic evaluation and the distribution-family claims rely on incorrect formulas. Eq. (16) is not the Kullback-Leibler divergence between two distributions; it is a pointwise ratio and is not even guaranteed to be nonnegative. Eq. (10) is the multivariate Gaussian log-likelihood with beta in place of Sigma, not a multivariate Laplace density. Eq. (11) adds a negative-binomial marginals term to a Gaussian quadratic form, which is not a proper joint multivariate negative binomial likelihood. Since RQ5 and the KL column in Tables 1-4 depend on these definitions, the formulas must be corrected and the affected experiments re-run.","section":"Appendix B.2, Eq. (16); Appendix A.2, Eqs. (10)-(11)"}],"minor_comments":[{"comment":"The architecture diagram labels the framework 'UGQNN'; the acronym should be UQGNN consistently.","section":"Figure 4"},{"comment":"The baseline CF-GNN is referred to as 'CF-CNN' in these sections; please use one name consistently throughout the paper.","section":"Section 5.1.2; Appendix B.1.2"},{"comment":"The abbreviation 'CPRS' appears in the CRPS definition; this should be 'CRPS'.","section":"Appendix B.2"},{"comment":"The text states that diagonal entries of the covariance matrix capture 'uncertainties caused by their interactions'; interactions between different phenomena are encoded by the off-diagonal entries, not the diagonal entries.","section":"Section 4.3"},{"comment":"The phrase 'almost all metrics' should be quantified; Table 1 shows that UQGNN is not the best on several Chicago metrics (MAE, CRPS, MPIW).","section":"Section 5.2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is in camera-ready form for SIGSPATIAL '25. Given the discrepancies between the headline claims and Table 1, the unstated chronological split, and the incorrect formulas in the appendices, I recommend that the editor require the authors to address the major comments before publication. The central method may be salvageable, but the current evidence does not support the paper's central empirical claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a coherent engineering contribution: it combines a multivariate diffusion GCN and an interaction-aware TCN with a probabilistic head that outputs a full covariance matrix, and it evaluates on four heterogeneous urban datasets with ablations, a selective regression check, and a distribution comparison. That combination is new relative to the cited literature, and the ablation that replaces the multivariate Gaussian with independent univariate Gaussians is a good check. The MPP eigenvalue-clamping trick is sensible. The authors clearly put real work into the experiments.\n\nThe problem is the central claim. The abstract and Section 5.2 say UQGNN \"consistently outperforms\" baselines and achieves \"5% improvement in both prediction accuracy and uncertainty quantification.\" Table 1 does not say that. On Chicago, UQGNN loses on MAE (1.680 vs 1.672 for SUMformer), on CRPS (0.680 vs 0.664 for STGCN), and on MPIW (2.333 vs 2.324 for CF-GNN). On Shenzhen, the actual MAE gain over the best baseline is about 3.3%, not 5%, and the CRPS gain is about 12%, not 5%. So the numbers in the text do not match the numbers in the table, and \"consistently outperforms\" is simply false when read literally. That needs to be fixed, and the claims should be rephrased metric-by-metric.\n\nThere are two other soft spots. Appendix B.3 says the split is 8:1:1 but does not say chronological. With 12-step input windows and a horizon of 1, a shuffled split would leak future information and inflate the reported gains. That is a real concern, not a nitpick. Also, Equations 5 and 6 are ambiguous as printed; a literal reading of the index sums does not give a clear channel-mixing rule, so independent re-implementation from the text would be hard. Single-run results without error bars or significance tests matter less because the differences are only 2-5% on several metrics, but still worth asking for.\n\nI would send this to peer review. The idea is useful and the evaluation breadth is above average for the subfield, but the authors need to correct the overstated claims, state the split protocol explicitly, and ideally add multiple seeds or at least variance information. The central mechanism, as described, is plausible enough to justify referee time.","headline":"A workable engineering contribution with a solid evaluation skeleton, but the headline claim is contradicted by the paper's own Table 1 and needs correction before it can be trusted.","tokens_in":29041,"tokens_out":1866,"would_cite":false,"duration_ms":21175,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"UQGNN claims that jointly modeling heterogeneous urban phenomena with a multivariate probabilistic head improves both point-forecast accuracy and uncertainty quantification, reporting 2–5% gains over twelve baselines on four real-world…","keywords":["uncertainty quantification","graph neural network","spatiotemporal prediction","multivariate prediction","heterogeneous urban phenomena","diffusion graph convolution","probabilistic forecasting"],"falsifier":"Re-run the experiments with a strictly chronological split—earliest 80% of timestamps for training, next 10% for validation, latest 10% for testing—and compare MAE and CRPS against the same twelve baselines; if UQGNN's margins shrink to noise or reverse, the central superiority claim fails.","tokens_in":27996,"feed_emoji":"🚕","tokens_out":7031,"duration_ms":71018,"temperature":0.7,"pith_summary":"This paper tries to establish that spatiotemporal forecasting improves when the model treats multiple urban phenomena—taxi, bus, subway, bike, and ride flows—as interacting variables and outputs a full predictive distribution rather than a single point estimate. Existing deterministic models give only expected values, while existing probabilistic models usually track one phenomenon and ignore cross-mode correlations. UQGNN couples a multivariate diffusion graph convolution and an interaction-aware temporal convolution to build shared embeddings, then a multivariate probabilistic head predicts a mean vector and covariance matrix. On Shenzhen, NYC, and Chicago datasets the authors report consistent gains of roughly 2–5% in MAE and CRPS over the best of twelve baselines.","feed_headline":"Multimode urban forecast with uncertainty beats 12 baselines","feed_subtitle":"Modeling how taxi, bus, subway, bike, and ride flows interact sharpens both point predictions and interval quality.","key_machinery":"The load-bearing components are the Multivariate Diffusion Graph Convolutional Network and the Interaction-aware Temporal Convolutional Network inside the Interaction-aware Spatiotemporal Embedding module, plus the Multivariate Probabilistic Prediction head. MDGCN replaces single-channel graph diffusion with a cross-layer diffusion convolution that mixes all phenomenon dimensions at each hidden layer; ITCN makes the dilated causal convolution filters sum over all phenomenon channels. The MPP head enforces symmetry and positive definiteness by reconstructing the covariance from its eigenvectors and a clamped eigenvalue matrix, then optimizes the negative log-likelihood $\\mathcal{L}=\\frac{1}{2}\\log|\\boldsymbol{\\Sigma}|+\\frac{1}{2}(\\boldsymbol{X}-\\boldsymbol{\\mu})^\\top\\boldsymbol{\\Sigma}^{-1}(\\boldsymbol{X}-\\boldsymbol{\\mu})$. The covariance is used not only for uncertainty output but to sharpen the mean prediction through the joint loss.","core_discovery":"The central claim is that explicitly modeling correlations among heterogeneous urban phenomena and quantifying their joint uncertainty yields better point forecasts and better-calibrated intervals than modeling each phenomenon separately. In the proposed architecture, the MDGCN applies diffusion convolution across phenomenon layers at every graph node, and the ITCN runs dilated causal convolutions whose filters mix channels from all phenomena; the fused embedding feeds a multivariate probabilistic head that outputs $\\boldsymbol{\\mu}$ and a positive-definite $\\boldsymbol{\\Sigma}$, trained by minimizing the multivariate Gaussian negative log-likelihood. The reported result is that UQGNN outperforms twelve baselines on six metrics across four datasets—for example, on Shenzhen an MAE of 9.717 versus 10.053 for the best baseline and a CRPS of 8.399 versus 9.562, which the authors describe as a 5% improvement in both accuracy and uncertainty quantification.","pith_inferences":["The learned covariance matrix could be read as a data-driven map of mode complementarity (e.g., high bike–subway covariance in the CBD); the paper only shows such correlations via Pearson coefficients, not from the network's output.","Because MPP models correlations across phenomena but not across regions or output timesteps, a natural extension is a low-rank or sparse cross-node covariance that would let uncertainty flow between neighborhoods without the $O(N^2)$ blow-up the paper avoids.","The per-mode gains are largest in CRPS for sparse phenomena (crime, crash), which suggests the framework may be most valuable in rare-event forecasting settings, a claim the paper does not develop.","A strict chronological re-split would settle whether the reported 2–5% edge survives when no test window overlaps a training window; the paper's appendix gives the split ratio but not its ordering."],"forward_implications":["Joint prediction lowers error for every individual mode: on Shenzhen each of the five mobility modes improves by 3–7% in MAE and 24–45% in CRPS compared to predicting that mode alone.","Removing the multivariate probabilistic head (w/o MPP) or replacing it with separate univariate Gaussians (w/ MPP-) degrades accuracy, so the uncertainty module is part of the point-forecast mechanism, not an add-on.","Replacing MDGCN with a standard diffusion GCN degrades Shenzhen MAE from 9.717 to 13.717, indicating that cross-phenomenon spatial mixing carries much of the gain.","Selective-regression analysis shows prediction error rises as coverage increases only when uncertainty scores are used, so the estimated uncertainties are informative for abstention decisions."],"supporting_citations":[{"why":"Supplies the diffusion-convolution formulation that MDGCN extends, and is a baseline UQGNN outperforms.","marker":"[30]"},{"why":"Provides the spatial-temporal graph convolution baseline STGCN used in the comparison.","marker":"[61]"},{"why":"Defines the single-phenomenon probabilistic baseline STZINB that motivates the multivariate extension.","marker":"[70]"},{"why":"Diffusion-based probabilistic spatiotemporal baseline DiffSTG against which UQGNN compares.","marker":"[56]"},{"why":"Uncertainty-quantification baseline DeepSTUQ and source of the KL metric convention.","marker":"[43]"},{"why":"Conformalized GNN baseline CF-GNN used in the comparison.","marker":"[19]"},{"why":"Baseline DSTAGNN and evidence for the Hadamard fusion of spatial and temporal embeddings.","marker":"[25]"},{"why":"Provides the temporal convolutional network design that ITCN makes interaction-aware.","marker":"[26]"},{"why":"Probabilistic time-series baseline TimeGrad included in the comparison.","marker":"[45]"}],"fun_headline_variants":["Uncertainty-aware graph net predicts multiple urban flows","Graph net quantifies uncertainty for city-scale flow forecasts","Joint uncertainty modeling improves multitask urban prediction","Uncertainty estimates for correlated urban flows beat separate models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported superiority depends on the 8:1:1 train/validation/test split being ordered in time; the paper never states that it is, and with 12-step input windows and a 1-step horizon, an unchronological split could leak future information into training and inflate the 2–5% gains.","fun_headline_variants_meta":{"raw":{"variants":["Uncertainty-aware graph net predicts multiple urban flows","Graph net quantifies uncertainty for city-scale flow forecasts","Joint uncertainty modeling improves multitask urban prediction","Uncertainty estimates for correlated urban flows beat separate models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000684,"raw_usage":{"total_tokens":3119,"prompt_tokens":979,"completion_tokens":2140,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":595,"completion_tokens_details":{"reasoning_tokens":2078}},"tokens_in":595,"tokens_out":2140,"duration_ms":14674,"temperature":1.0,"reasoning_tokens":2078,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:35:46.765355+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the experiments with a strictly chronological split—earliest 80% of timestamps for training, next 10% for validation, latest 10% for testing—and compare MAE and CRPS against the same twelve baselines; if UQGNN's margins shrink to noise or reverse, the central superiority claim fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Conformalized GNN baseline CF-GNN used in the comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the single-phenomenon probabilistic baseline STZINB that motivates the multivariate extension."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Uncertainty-quantification baseline DeepSTUQ and source of the KL metric convention."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Baseline DSTAGNN and evidence for the Hadamard fusion of spatial and temporal embeddings."}],"review_version":1}