{"id":"ac1149c8-cbbf-4adc-8b8c-b13f72e3f2de","arxiv_id":"2501.13274","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A flattened Transformer with temporal plus graph-structure encodings reports state-of-the-art RMSE and MAPE on traffic forecasting benchmarks.","lead":"T-Graphormer is a Transformer that flattens traffic sensor readings into one long sequence and uses graph and time encodings so every sensor can attend to every time step at once. On standard traffic benchmarks it reports lower prediction error than models that process space and time separately.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline SOTA margins in Table 1 rest on baseline numbers whose comparability is asserted, not demonstrated; a rerun under identical preprocessing is needed before the 24%/12% RMSE claims can be trusted.","rationale":"T-Graphormer is a simple, clearly specified extension of Graphormer, with ablations showing positional, centrality, and spatial encodings all help. The strongest claim, however, is empirical: it beats SOTA by large margins. That claim is only as strong as the baseline numbers. The paper's own numbers are internally inconsistent in several places, which lowers confidence that the table was assembled under a single protocol. The reproducible code and pretrained weights are a real asset, but they do not make the cited baselines reproducible. The proposed check is expensive but decisive: rerun the baselines. This does not change the reader's CONDITIONAL verdict; it sharpens the condition.","tokens_in":16382,"tokens_out":4733,"duration_ms":42270,"concrete_test":"Obtain the official implementations of STEP, PDFormer, STAEformer, Graph WaveNet, DCRNN, and GMAN and run them under the exact DCRNN preprocessing and split described in Section 5.1, with the same time-of-day one-hot feature and with predictions for horizons 3/6/12 extracted identically to T-Graphormer. Compare reproduced RMSE/MAPE/MAE to Table 1. If any reproduced baseline changes by more than ~3% relative on either dataset, the claimed 24%/12% margins are not established; if all reproduced values land within 1%, the central SOTA claim holds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is Section 5.3's 'reduces RMSE by 24% and 12% on PEMS-BAY and METR-LA.' These margins are measured against Table 1's cited numbers for STEP, PDFormer, STAEformer, Graph WaveNet, DCRNN, and GMAN. The paper states (Section 5.1) that it follows DCRNN's preprocessing for its own model, but it never states that any baseline was rerun under that exact pipeline. If those cited numbers come from the baselines' original papers, differences in normalization, time-of-day input, validation split (e.g., STSGCN's 6:2:2 vs. DCRNN's 7:2:1), or evaluation masking for short horizons can easily account for several percent of the reported gap. The claimed 24% advantage over STEP on PEMS-BAY is large enough that it could survive small protocol differences, but the 12% METR-LA margin and the 10% MAPE claim are not robust to a 3-5% shift in baseline preprocessing. Secondary inconsistencies (abstract says 'up to 20% and 10%' while Section 5.3 says 24%/12%; Table 4 swaps the std values; Section 5.2's baseline list omits DCRNN/STGCN/Graph WaveNet/ASTGCN even though Table 1 reports them) do not by themselves invalidate the architecture, but they make the unverified comparability the decisive uncertainty. The architecture claim remains plausible; the SOTA claim is conditional on baseline reproducibility.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes T-Graphormer, an extension of the Graphormer architecture for spatiotemporal forecasting. The input is flattened into a token sequence of length T' × N, and Graphormer-style structural encodings---centrality encoding, spatial encoding based on shortest-path distances, and learnable positional encodings---are added to the node features and attention scores. The architecture has no separate spatial and temporal learning modules, which is the paper's central architectural claim. The authors evaluate T-Graphormer on PEMS-BAY and METR-LA, reporting state-of-the-art results across all horizons in Table 1, and on three additional PEMS datasets in Table 6. They also provide ablation studies on the encoding components and special tokens, an attention-score analysis, and a discussion of model-size variants. The paper states that code, datasets, and pretrained weights will be released.","tokens_in":16707,"tokens_out":4273,"duration_ms":46431,"significance":"If the empirical claims are correct, the paper makes a useful point: flattening spatiotemporal data and applying a single Transformer with graph-structural and temporal encodings can be competitive with, or beat, specialized architectures that alternate spatial and temporal modules. This would be a meaningful contribution to the traffic-forecasting literature and to the broader question of how much inductive bias is needed for structured-sequence Transformers. The paper also ships several strengths: a clear architecture description, ablation experiments isolating each encoding component, an analysis of learned attention patterns, and a reproducibility statement with a promised code release. However, the novelty over Graphormer is incremental---the main new ingredient is the joint temporal positional encoding and the application domain---and the headline SOTA claim rests on a comparability assumption about baselines that is not demonstrated. The central architecture claim is plausible, but the quantitative SOTA claim is not yet established to the standard required for publication.","major_comments":[{"comment":"The central SOTA claim is stated in §5.3 as reducing RMSE by 24% and 12% on PEMS-BAY and METR-LA, but the comparability of baseline numbers in Table 1 is not established. Section 5.1 says the authors follow the DCRNN preprocessing pipeline for their own model, but it never states that any baseline was rerun under that same pipeline. The cited values for VAR, FC-LSTM, DCRNN, STGCN, Graph WaveNet, ASTGCN, GMAN, PDFormer, STAEformer, and STEP are likely taken from different original papers that used different data splits (e.g., DCRNN's 7:2:1 vs. STSGCN's 6:2:2), different inclusion of time-of-day features, different normalization, and possibly different evaluation details. Without a statement that each baseline was evaluated with identical preprocessing, splits, and evaluation code, the 24%/12% RMSE margins are not reliable. Please state explicitly which numbers were rerun and which were cited, and ideally release the evaluation code so that the comparison is reproducible.","section":"§5.1, Table 1"},{"comment":"Table 1 reports only point estimates with no error bars or multiple seeds. Given that model configurations are selected on validation performance (§5.2) and the reported margins over second-best baselines are sometimes small (e.g., METR-LA horizon 3, where T-Graphormer's MAE is 2.62 vs. STEP's 2.61), single runs are insufficient to establish that the differences are statistically meaningful. Please report means and standard deviations over at least three independent runs for T-Graphormer and for the baselines that are rerun, and state the number of seeds used.","section":"Table 1, §5.2"},{"comment":"The evaluation protocol for different horizons is asymmetric with respect to many baselines. Section 5.2 states that T-Graphormer is trained once with T=12 and that horizons 3 and 6 are evaluated by 'simply remov[ing] extra token values.' Many traffic-forecasting baselines are trained separately for each horizon. This protocol difference can change results nontrivially and should be disclosed in the table caption and controlled for in the comparison. At minimum, state which baselines were trained per horizon and which were trained for T=12 only, and discuss the impact on the reported numbers.","section":"§5.2, horizons 3/6"},{"comment":"The paragraph citing Kaplan et al.'s scaling law D ∝ N^0.74 and claiming that the experimental datasets are 'only optimal for training the mini models' is not supported by the evidence presented. Table 2 contains three model sizes on two datasets, with no compute-budget sweeps, no loss-vs-compute curves, and no controlled variation of dataset size; applying an LLM scaling law to traffic forecasting in this way is an unjustified extrapolation. The claim that the result is 'confirmed' by Table 2 should be removed or replaced with a more modest statement that larger models overfit on these datasets, which is already visible in Figure 10.","section":"§5.3, Table 2"}],"minor_comments":[{"comment":"The abstract states that T-Graphormer reduces RMSE and MAPE by 'up to 20% and 10%,' while Section 5.3 reports reductions of 24%/12% for RMSE and 13%/10% for MAPE. These numbers should be aligned.","section":"Abstract vs. §5.3"},{"comment":"The baseline list in Section 5.2 omits DCRNN, STGCN, Graph WaveNet, and ASTGCN even though these methods appear in Table 1. Please make the list consistent with the reported comparisons.","section":"§5.2"},{"comment":"Table 4 lists PEMS-BAY with std 19.49 and METR-LA with std 9.44, but Section 5.3 states that METR-LA has std 19.49 and PEMS-BAY has std 9.44. One of these is wrong; please correct the inconsistency.","section":"Table 4, §5.3"},{"comment":"In Equation (7), the second key term is written as (h_{t2,i} W_K)^T; it should be (h_{t2,j} W_K)^T to match the stated token indices (t2, j).","section":"Equation (7)"},{"comment":"The sentence 'This is not selectivity is not present in the first layer' contains a grammatical error and should be rewritten, e.g., 'This selectivity is not present in the first layer.'","section":"Appendix A.1.1"},{"comment":"The appendix contains typos such as 'weigh decay' in Tables 3 and 7; please correct to 'weight decay.'","section":"Appendix tables"}],"recommendation":"major_revision","confidential_remarks":"The main risk is the empirical comparability of baselines. The architecture itself is a straightforward extension of Graphormer, and the paper's contribution is primarily empirical; therefore the 24%/12% SOTA claim must be backed by a reproducible, same-protocol comparison. If the authors can provide a rerun of the key baselines under identical preprocessing and evaluation, or clearly qualify which numbers are cited, the paper could be acceptable. The use of Kaplan scaling-law language is an overreach and should be toned down. I would also encourage the editor to check the reproducibility statement against the actual code repository during revision, since the promise of code release is part of the paper's evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a straightforward extension of Graphormer to spatiotemporal data: flatten the graph sequence, add learnable temporal positional encodings alongside Graphormer's centrality and spatial encodings, and let attention do the rest. On the traffic benchmarks it reports large gains over a wide set of baselines, with ablations and attention heatmaps that support the qualitative claim that the model learns spatial and temporal structure without separate towers. That is a real contribution, though each ingredient is known.\n\nWhat the paper does well: the architecture is clearly described, the ablations isolate the contribution of each encoding, and the authors are transparent about memory limitations and the scaling-law speculation. The additional results on PEMS03/04/08 are a nice check on generalizability. The code and weights are promised, which would let others verify the numbers.\n\nThe soft spots are mostly about the evidence behind the headline margins. The 24% and 12% RMSE reductions in Section 5.3 are measured against Table 1 numbers that are cited from previous papers, not rerun under the authors' preprocessing. They say they follow DCRNN's preprocessing for their own model, but never state that the baselines were rerun under that same pipeline. Differences in normalization, time-of-day encoding, or validation split could shift baseline numbers by a few percent, which puts the smaller margins (METR-LA, MAPE) at risk. The abstract says \"up to 20% and 10%\" while Section 5.3 claims 24% and 12%; these inconsistencies don't help confidence. No error bars or multiple seeds are reported, so we can't tell whether the gains are stable. The practice of training one T=12 model and evaluating on horizons 3 and 6 by removing tokens is also unlike most baselines, which are trained per horizon. None of these are fatal to the architecture, but they are exactly the things that decide whether the SOTA claim holds.\n\nThe citation pattern is fine—Spacetimeformer is credited, and the limitations section is honest about the O(n^2) cost. The paper is not circular; it's an empirical benchmark comparison with standard model selection.\n\nWho is this for: someone working on graph-structured time series forecasting who wants a simple, attention-only baseline to compare against. It deserves a serious referee, but revision should require a rerun of the key baselines under identical preprocessing, multiple seeds, and a cleanup of the numeric inconsistencies. I'd want to see that before trusting the 24% claim.\n\nRecommendation: send to review, but insist on the baseline reproducibility check.","headline":"A simple, well-described Graphormer extension with plausible gains, but the headline SOTA margins rest on baseline comparability that is asserted, not demonstrated.","tokens_in":17214,"tokens_out":2158,"would_cite":false,"duration_ms":28184,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"T-Graphormer claims a single Transformer, with flattened graph tokens plus temporal and structural encodings, learns space and time together and beats separate spatial-temporal models on traffic benchmarks.","keywords":["spatiotemporal forecasting","traffic prediction","Transformers","Graphormer","graph structure encoding","attention mechanism","multivariate time series","PEMS-BAY"],"falsifier":"Re-run every baseline in Table 1 under the exact DCRNN preprocessing, same 70/20/10 chronological split, same horizon definitions, and same Z-score normalization; if the reported RMSE reductions (24% on PEMS-BAY and 12% on METR-LA) do not reproduce, the state-of-the-art claim fails.","tokens_in":16142,"feed_emoji":"🚦","tokens_out":7006,"duration_ms":60940,"temperature":0.7,"pith_summary":"T-Graphormer is an attempt to show that spatiotemporal forecasting does not need separate spatial and temporal learning modules. The paper flattens a graph-structured traffic sequence into one token stream and lets every node attend to every other node at every time step, adding temporal and structural encodings on top. On the PEMS-BAY and METR-LA benchmarks it reports large gains, cutting RMSE by 24% and 12% respectively against state-of-the-art baselines. If the comparison holds up, the result matters because it suggests a single unified Transformer with minimal inductive bias can learn time and space dependencies directly from the data.","feed_headline":"One flattened graph Transformer cuts traffic forecast error by 24%","feed_subtitle":"Unified attention across nodes and time beats separate spatial-temporal modules on traffic benchmarks.","key_machinery":"The central object is the flattened graph sequence as a Transformer token stream, with three Graphormer-style encodings: per-token learnable positional encoding, node-centrality embeddings indexed by in/out degree, and a learnable attention bias $b_{\\phi(i,j)}$ indexed by shortest path distance between nodes. A special cls token is prepended as a supernode. The attention score between tokens $(t_1,i)$ and $(t_2,j)$ is the standard query-key similarity plus $b_{\\phi(i,j)}$, so structure enters only through additive biases while the receptive field stays global across both time and space.","core_discovery":"On its own terms, the paper establishes that extending Graphormer's structural encodings into the temporal dimension produces a competitive spatiotemporal forecaster. After flattening the observed graph sequence, T-Graphormer adds centrality encoding and learnable positional encoding to each token, and injects a shortest-path-distance attention bias between node pairs. It reports the best MAE, RMSE, and MAPE among ten baselines at the 30-minute and 1-hour horizons on both traffic datasets, with the largest gains at horizon 12. Attention heatmaps and ablations are offered as evidence that the model learns spatial and temporal structure rather than receiving it as a hard architectural constraint.","pith_inferences":["The recipe is graph-agnostic: any multivariate series whose components sit on a fixed graph, such as power grids, environmental sensors, or supply chains, could use the same flatten-and-encode treatment; the paper only evaluates traffic.","A direct ablation of the flattening order itself would test whether the gains come from the unified attention or simply from the encodings; the paper does not run that comparison.","The attention analysis could be turned into a diagnostic tool for traffic management, pointing at the sensors and lagged time steps that most influence a forecast; the paper stops at visualization.","On larger networks the quadratic context cost would force sparse attention; whether the learned spatiotemporal patterns survive sparsification is an open question the paper does not answer."],"forward_implications":["On the two traffic benchmarks, a single unified attention module beats models that alternate spatial and temporal convolutions or attentions.","Using one model trained with horizon-12 loss and evaluated by trimming tokens gives strong results at horizons 3 and 6 as well.","Removing any of the three encodings degrades performance, so structural and temporal biases still matter even in a global-attention design.","The learned attention heatmaps show selective node-node and time-time patterns, indicating spatiotemporal structure is discoverable without explicit spatial or temporal modules.","The method's memory cost grows quadratically with the flattened sequence length, limiting it to small networks and short contexts without sparse attention."],"supporting_citations":[{"why":"Supplies the Graphormer structural encodings (centrality, spatial, edge) that T-Graphormer extends into the temporal dimension.","marker":"Ying et al., 2021"},{"why":"Provides the DCRNN preprocessing, graph construction, and baseline numbers used for PEMS-BAY and METR-LA evaluation.","marker":"Li et al., 2018"},{"why":"Supplies the node feature embedding recipe and the Graph WaveNet baseline that T-Graphormer compares against.","marker":"Wu et al., 2019"},{"why":"Defines the self-attention mechanism and positional encoding that form the Transformer core.","marker":"Vaswani et al., 2017"},{"why":"Supports flattening structured data into token sequences and the use of learned positional embeddings.","marker":"Dosovitskiy et al., 2020"},{"why":"Provides the STEP baseline and the PEMS-BAY/METR-LA evaluation convention.","marker":"Shao et al., 2022"},{"why":"Provides the PDFormer baseline that the paper claims to outperform.","marker":"Jiang et al., 2023"},{"why":"Provides the STAEformer baseline and the learnable spatiotemporal embedding idea the paper discusses.","marker":"Liu et al., 2023"}],"fun_headline_variants":["Transformer flattens graph sequences for simultaneous spacetime forecasting","T-Graphormer: one graph transformer for both space and time","Attention across nodes and timesteps cuts traffic forecast error","Unified Transformer learns spacetime patterns without separate modules","Graphormer extension models space and time together for traffic"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The state-of-the-art result assumes the ten baseline models were evaluated under the same preprocessing and data split as T-Graphormer; the paper cites DCRNN preprocessing but does not report re-running each baseline under it.","fun_headline_variants_meta":{"raw":{"variants":["Transformer flattens graph sequences for simultaneous spacetime forecasting","T-Graphormer: one graph transformer for both space and time","Attention across nodes and timesteps cuts traffic forecast error","Unified Transformer learns spacetime patterns without separate modules","Graphormer extension models space and time together for traffic"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000699,"raw_usage":{"total_tokens":3092,"prompt_tokens":814,"completion_tokens":2278,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":430,"completion_tokens_details":{"reasoning_tokens":2197}},"tokens_in":430,"tokens_out":2278,"duration_ms":16661,"temperature":1.0,"reasoning_tokens":2197,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T16:18:22.082203+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run every baseline in Table 1 under the exact DCRNN preprocessing, same 70/20/10 chronological split, same horizon definitions, and same Z-score normalization; if the reported RMSE reductions (24% on PEMS-BAY and 12% on METR-LA) do not reproduce, the state-of-the-art claim fails.","supporting_citations":[{"cited_title":"Do transformers really perform badly for graph representation? Advances in Neural Information Processing Systems, 34: 0 28877--28888, 2021","cited_arxiv_id":null,"evidence_quote":"Supplies the Graphormer structural encodings (centrality, spatial, edge) that T-Graphormer extends into the temporal dimension."},{"cited_title":"Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting","cited_arxiv_id":null,"evidence_quote":"Provides the STEP baseline and the PEMS-BAY/METR-LA evaluation convention."},{"cited_title":"Spatio-temporal adaptive embedding makes vanilla transformer sota for traffic forecasting","cited_arxiv_id":null,"evidence_quote":"Provides the STAEformer baseline and the learnable spatiotemporal embedding idea the paper discusses."}],"review_version":1}