{"id":"4706ab26-edd7-48e5-965c-a83891302730","arxiv_id":"2505.00307","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Gateformer combines temporal patching attention, variate-wise attention, and two gating mechanisms to achieve the best average forecasting error on most of 13 multivariate benchmarks, and it reports improved accuracy when plugged into several existing transformer and LLM forecasters.","lead":"This paper introduces Gateformer, a new neural network for multivariate time series forecasting that uses attention to capture both how each series changes over time and how different series influence each other. It reports the best average error on most of 13 standard benchmarks and claims its approach can be added to existing forecasting models to improve them.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Baseline comparability is unverified: Table 7 numbers are copied from Liu et al. (2024b) despite Appendix B.2 claiming local reproduction, so the SOTA claim may rest on a protocol mismatch.","rationale":"The reader's weakest assumption was that baseline numbers from Liu et al. (2024b) and the TimesNet repository are directly comparable to Gateformer's results under identical splits, look-back, and tuning budgets. My stress-test confirms and sharpens that concern: the paper contains a direct contradiction between the Table 7 caption ('Baseline results are from Liu et al. (2024b)') and Appendix B.2 ('Baseline models were reproduced using the TimesNet repository'), and spot-checking the iTransformer column shows it is verbatim from the published paper, implying the authors did not rerun those baselines under their own protocol. If this is correct, the reported SOTA gains could be an artifact of different training durations, hyperparameter search budgets, or seed handling. This is the single most load-bearing concern because it directly undermines the headline claim, whereas other issues (e.g., the exact 20.7% integration improvement not matching Table 3 arithmetic) are secondary and also addressable. Because the concern is concrete and testable, and because the paper could still be correct after a fair rerun, the reader's CONDITIONAL verdict remains appropriate. I do not see grounds to escalate to REJECT without running the proposed check, nor to accept without it.","tokens_in":17473,"tokens_out":11028,"duration_ms":99372,"concrete_test":"Run the official iTransformer, PatchTST, Crossformer, FEDformer, Autoformer, Stationary, TimesNet, SCINet, and DLinear code on the same 13 datasets with Gateformer's exact protocol: L=96, 10 epochs, early stopping patience 3, hidden-dim search {64,128,256,512,1024}, block search {1,2,3,4}, learning-rate search {1e-4,5e-4,1e-3}, and three seeds. Compare the reproduced baseline MSE/MAE against Table 7. If any reproduced baseline improves on Gateformer's reported value — especially on Electricity, Weather, or Traffic — the SOTA claim is not robust. As a minimal check, recompute the iTransformer column from the official code; if it differs from the Table 7 column by more than 1%, the table was copied and the protocol mismatch is confirmed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is state-of-the-art performance on 13 datasets. That claim depends on the baselines being fairly compared under the same protocol. The paper is internally inconsistent about this: Section 3.1 states 'Baseline results follow Liu et al. (2024b)', while Appendix B.2 says 'Baseline models were reproduced using the TimesNet repository, following configurations from their respective original papers or official code.' A spot-check confirms the inconsistency: the iTransformer column in Table 7 exactly matches Liu et al. (2024b)'s published values (e.g., Electricity 96/192/336/720: 0.148/0.162/0.178/0.225), indicating the numbers were copied from the paper rather than produced by fresh runs under Gateformer's protocol. If the baselines were not retrained with the same look-back length, the same 10-epoch budget, the same hyperparameter search ranges, and the same three-seed averaging, then the reported margins — often only 1–2% (Electricity 0.176 vs 0.178; Weather 0.253 vs 0.258) — could be within seed noise or tuning artifacts. The paper's own description does not rule this out, and the conflicting statements make the comparability assumption insecure.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Gateformer, a Transformer-based architecture for multivariate time series forecasting that first models cross-time dependencies via patch-based temporal attention and a global MLP path, fuses these with a gating mechanism, then models cross-variate dependencies via attention over variate embeddings with a second gate. The authors claim state-of-the-art results on 13 public benchmarks (top-1 in 91 of 130 settings) and report that the framework can be integrated into Autoformer, Flowformer, GPT4TS, and Moment, improving performance by up to 20.7%. The manuscript includes ablations, sensitivity analyses, efficiency comparisons, transfer learning experiments, and a code link.","tokens_in":17710,"tokens_out":7373,"duration_ms":66540,"significance":"If the empirical claims are verified, the contribution is a reproducible, simple architecture that improves average performance over strong baselines and a general integration mechanism for existing forecasters. The code is publicly available. However, the absence of error bars, the unresolved inconsistency in baseline provenance, and the underspecified integration protocol currently prevent the 'state-of-the-art' claim from being endorsed at face value.","major_comments":[{"comment":"The manuscript is internally inconsistent about the origin of baseline results. Section 3.1 states 'Baseline results follow Liu et al. (2024b)', Appendix B.2 states 'Baseline models were reproduced using the TimesNet repository', and Table 7's caption attributes baselines to Liu et al. (2024b). If the numbers are transcribed from published tables rather than produced under Gateformer's exact protocol (L=96, 10 epochs, early stopping, three-seed averaging, and the stated hyperparameter ranges), the comparison is not controlled, and the small margins (e.g., Electricity T=96 MSE 0.146 vs 0.148; Weather T=96 0.168 vs 0.174) could be within seed noise. Please report standard deviations for all results and either rerun the baselines under the same protocol or explicitly frame the comparison as against published numbers.","section":"Section 3.1 / Table 7 / Appendix B.2"},{"comment":"The integration experiments are not sufficiently specified to support the 'seamless integration' claim. The text does not describe how the framework is grafted onto Autoformer, Flowformer, GPT4TS, and Moment, nor what 'our framework' adds in each case; no code or configuration is provided for the integrated models. The 'original' numbers in Table 3 are not stated to be reproduced locally, and the reported improvements (e.g., Flowformer Electricity average MSE from 0.267 to 0.187) are large enough that a protocol mismatch (different look-back, epochs, or tuning budget) could explain them. Please detail the integration mechanism, document how each baseline was obtained, and ensure equal training budgets and seeds.","section":"Section 3.3 / Table 3"},{"comment":"Results are said to be averaged over three seeds, but no standard deviations or confidence intervals are reported in any table. Given that several claimed gains are around 1–2% relative (e.g., Electricity average MSE 0.176 vs 0.178; Weather 0.253 vs 0.258), the state-of-the-art claim is statistically unsupported. Please include per-seed standard deviations or a paired significance test (e.g., across horizons) for the main comparisons.","section":"All experimental tables / Appendix B.2"},{"comment":"The phrase 'consistently outperforms all baselines across 13 datasets' is contradicted by individual settings: in Table 7, ETTh2 at T=96 has Gateformer MSE 0.306 vs iTransformer 0.297; ETTm2 at T=96 has 0.176 vs PatchTST 0.175; and in Table 6, PEMS04 at T=12 and T=24 are worse than SCINet (0.083 vs 0.073 and 0.114 vs 0.084). If the claim refers to per-dataset average MSE, say so explicitly and reword 'consistently'; otherwise the claim is false as written.","section":"Section 3.1 / Table 6 / Table 7"}],"minor_comments":[{"comment":"The word 'propsed' should be 'proposed'.","section":"Contributions bullet list"},{"comment":"The gating in the temporal modeling phase is described as fusing temporal and global embeddings; the caption of Figure 1 says 'Horizontal Blending: Intra-series Correlations', which is ambiguous. Consider annotating the gates explicitly as 'gate' to match the text.","section":"Section 2.3 / Figure 1"},{"comment":"In the ETTm1 row for T=720, the Gateformer MSE is printed as '0459'; it should be '0.459'.","section":"Table 7"},{"comment":"The patch size and stride for the patching operation are not listed among the hyperparameters; these are essential for reproducibility and should be included.","section":"Appendix B.2"},{"comment":"The claimed improvement of 'up to 20.7%' is not directly traceable to a specific entry in Table 3; state which setting yields 20.7% and how the percentages are computed (e.g., relative to original MSE).","section":"Section 3.3"},{"comment":"The efficiency comparison says all models use the same hidden dimension and batch size, but it does not state whether each baseline uses its official hyperparameters; note this in the caption.","section":"Appendix H.2"},{"comment":"The transfer learning results would be more informative with error bars, given the small differences between fine-tuning and from-scratch training.","section":"Table 5 / Appendix G"}],"recommendation":"major_revision","confidential_remarks":"This is an empirical workshop-style paper with an incremental architecture; the main value is the claimed consistent improvements. The conflicting statements about baseline provenance (Section 3.1 vs Appendix B.2 vs Table 7) are a reproducibility red flag that must be resolved. If the authors cannot provide a controlled comparison with reruns and standard deviations, the state-of-the-art claim should be explicitly downgraded to 'competitive with published numbers'. The journal should verify the code and the integration protocol before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead Gateformer. The new thing is a gated two-stage attention design: each variate gets a patch-based temporal representation and a global MLP embedding, merged through a learned gate, then all variate embeddings go through a second attention with a gate that mixes in the unmodified embeddings. That combination is, as far as I can tell, not in PatchTST, iTransformer, or Crossformer. The ablations show all three pieces matter, and the integration experiments (Autoformer, Flowformer, GPT4TS, Moment) show consistent gains on Electricity and Weather. That part is worth taking seriously.\n\nThe soft spot is the baseline protocol, and it is not minor. Section 3.1 says \"Baseline results follow Liu et al. (2024b)\"; Appendix B.2 says baselines were reproduced locally via the TimesNet repository with their own configurations. Those two statements cannot both be true. The stress-test note is right: the iTransformer column in Table 7 exactly matches the published values in Liu et al. (2024b), for example Electricity 96/192/336/720 = 0.148/0.162/0.178/0.225. That means the numbers were copied, not rerun under Gateformer's protocol (10 epochs, three seeds, same look-back, same hyperparameter search). If the baselines were not retuned under identical settings, the reported margins—often 1–2%—are inside seed noise or tuning artifacts. The abstract's \"state-of-the-art on 13 real-world datasets\" is an overstatement on the paper's own numbers: Gateformer loses to a baseline in several settings (ETTm2 at 96, ETTh2 at 96, Weather at 96, Exchange at 336). No error bars are given anywhere despite the claim of three-seed averaging.\n\nThe integration experiments have the same problem in milder form: \"original\" model numbers in Table 3 appear to be taken from the original papers, and the appendix does not say how the framework was grafted in. The up-to-20.7% improvement claim is plausible but unverified without code or a precise recipe.\n\nWhat holds up: the architecture is clean, the ablations are consistent with the design story, and the efficiency-aware training trick (sampling variates per batch) is a sensible practical addition. The evaluation is broad—13 datasets, 10 baselines, plus transfer and scaling studies.\n\nVerdict: this deserves a serious referee, not a desk reject. The fix is straightforward: rerun every baseline locally under the same protocol, report error bars, and either add the newer strong baselines (TimeLLM, Timer, Moirai, etc.) or drop the \"state-of-the-art\" phrase. If the numbers hold after a fair rerun, it is a solid incremental contribution. If they do not, the architecture still stands on the integration results, which should be documented precisely.\n\nI'd bring it to a reading group only to discuss baseline reproducibility, and I would not cite the SOTA claim until the protocol is sorted.","headline":"Useful gated dual-attention architecture, but the SOTA claim rests on baseline numbers that appear lifted from prior papers rather than rerun under the same protocol.","tokens_in":18240,"tokens_out":3882,"would_cite":false,"duration_ms":36605,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Gateformer claims that multivariate forecasting improves when a Transformer models time with gated patch-level attention and variate interactions with gated cross-variate attention, and reports top results on 13 datasets.","keywords":["multivariate time series forecasting","Transformer","temporal attention","variate-wise attention","gating mechanism","patch tokenization","benchmark evaluation","model integration"],"falsifier":"Re-run Gateformer and all nine baselines from a single codebase with identical chronological splits, look-back length $L=96$, the same prediction horizons, the same seeds, and an equal hyperparameter search budget per model, then compare the per-dataset averages. If Gateformer no longer ranks first in the stated 91 of 130 settings, the central state-of-the-art claim is falsified.","tokens_in":17247,"feed_emoji":"📈","tokens_out":10216,"duration_ms":92025,"temperature":0.7,"pith_summary":"Gateformer is a Transformer architecture for multivariate time series forecasting that explicitly models both the dimension of time and the dimension across a dataset's individual series. The paper argues that existing models handle one of these well at the expense of the other: temporal-attention models such as PatchTST ignore cross-variate relations, while variate-attention models such as iTransformer lose fine-grained temporal structure. Gateformer fuses local patch-level temporal attention with a shared global temporal embedding through a gated combination, then applies attention across the resulting variate embeddings and gates that output with a non-interacting copy. With a fixed look-back of 96 steps, it reports lower prediction error than nine comparison models on 13 real-world datasets, ranking first in 91 of 130 measured settings. The same gated modules are also shown to improve the errors of existing Transformer-based, LLM-based, and foundation forecasters by up to 20.7%.","feed_headline":"Gated transformer tops 91 of 130 forecasting settings","feed_subtitle":"Fuses local and global temporal signals, gates cross-variate attention, and lifts existing forecasters by up to 20.7%.","key_machinery":"The load-bearing object is the two-stage gated variate representation. In the temporal stage, each variate's series is normalized, split into non-overlapping patches, and fed through self-attention to produce a local temporal embedding $v^{(i)}_T$; in parallel, the same raw series passes through a shared MLP to produce a global temporal embedding $v^{(i)}_G$. A sigmoid gate $\\text{Gate} = \\sigma(v^{(i)}_T W_{g1} + v^{(i)}_G W_{g2})$ fuses the two into the variate embedding $s^{(i)}$. In the variate stage, self-attention runs over the matrix of these embeddings and a second gate blends the attended output with the unattended $S$, so the effective cross-variate coupling can be weak for datasets with few variates. This second gate is what lets the model behave like a channel-independent forecaster when that helps.","core_discovery":"The paper's central claim is that the best way to repurpose the Transformer for multivariate forecasting is to make it variate-wise in representation but gated in both stages: each series is first encoded independently by a patch-attention path and a global-MLP path, with a sigmoid gate choosing the mix; then attention runs across these variate embeddings, and a second gate blends the interacted and non-interacted forms. The authors claim this arrangement keeps PatchTST-level temporal resolution while adding iTransformer-style cross-variate capacity, and that the second gate prevents the cross-variate signal from hurting low-dimensional datasets. They assert this produces consistent gains on 13 benchmarks and that the design transfers to other architectures, reporting that the framework improves Autoformer, Flowformer, GPT4TS, and Moment errors on Electricity and Weather by 6.9–20.7%. In their framing, the model works because gating lets the network decide, per dataset, how much of the cross-variate information to let through.","pith_inferences":["Beyond the paper: inspecting the learned gate values on a dataset-by-dataset basis would show whether the gates settle near zero for low-dimensional data and near one for high-dimensional data; the paper does not report such gate statistics.","Beyond the paper: the reported gains when attaching the module to Autoformer, Flowformer, GPT4TS, and Moment suggest it could also be attached to newer decoder-only and foundation forecasters, but that reach is an extrapolation, since those models were not tested.","Beyond the paper: an independent re-run that retrains all baselines under one codebase with an equal hyperparameter budget would settle how much of the 91-of-130 ranking comes from the architecture rather than from borrowed baseline numbers.","Beyond the paper: forcing the second gate open and closed on the same datasets would quantify how much cross-variate information contributes at each dataset size, giving a practical diagnostic for when channel-independent modeling is preferable."],"forward_implications":["If Gateformer's results hold, a single architecture can serve datasets with as few as 7 and as many as 883 channels, with the gating layers deciding how much cross-variate interaction to use.","The reported plug-in experiments imply that other forecasters can gain accuracy without changing their own attention machinery: Autoformer improves on Electricity by about 19.9%, Flowformer by 20.7%, GPT4TS by 10.8%, and Moment by 6.9% in the paper's Table 3.","The efficient variant, which swaps quadratic attention for Flowformer's linear attention and trains on 20% sampled variates, is claimed to keep performance while cutting memory (1.67GB vs 5.45GB on Traffic) and running as fast as iTransformer.","Because variate-wise attention operates on the variate dimension rather than the time dimension, models built this way can be trained jointly on datasets with different numbers of variates; the PEMS scaling experiment reports consistent error reductions as the training pool grows."],"supporting_citations":[{"why":"Defines the iTransformer baseline and is the source of the baseline numbers used in the main comparison table.","marker":"Liu et al. (2024b)"},{"why":"Supplies the benchmark protocol and the codebase used to reproduce all comparison baselines.","marker":"Wu et al. (2023)"},{"why":"Introduces patch-based temporal attention, which Gateformer's temporal path builds on as its local encoder.","marker":"Nie et al. (2023)"},{"why":"The main prior model that also combines cross-time and cross-variate attention; Gateformer positions itself against its noise problem.","marker":"Zhang & Yan (2023)"},{"why":"Baseline in the long-term forecasting comparison and one of the integration targets for the proposed framework.","marker":"Wu et al. (2021)"},{"why":"LLM-based forecaster used as an integration target in the generalizability experiments.","marker":"Zhou et al. (2023)"},{"why":"Foundation time-series model used as an integration target in the generalizability experiments.","marker":"Goswami et al. (2024)"},{"why":"Reversible instance normalization used to handle distribution shift; it is part of Gateformer's preprocessing.","marker":"Kim et al. (2022)"},{"why":"Linear-time attention used in the efficient variant of Gateformer for high-dimensional datasets.","marker":"Wu et al. (2022)"}],"fun_headline_variants":["Gateformer: gated attention fuses temporal and variate patterns for forecasting","Gated transformer boosts forecasters by up to 20.7% across 13 benchmarks","Gating cross-variate attention lifts time series forecasting accuracy","Gateformer integrates into LLM and transformer forecasters, lifting accuracy","New gated dual-attention model improves multivariate forecasting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The state-of-the-art claim rests on the assumption that the baseline numbers it compares against were produced under the same protocol as Gateformer's experiments—identical chronological splits, look-back length $L=96$, and comparable hyperparameter tuning budgets—so if the baselines were not retuned with equal effort, the reported lead could shrink or disappear.","fun_headline_variants_meta":{"raw":{"variants":["Gateformer: gated attention fuses temporal and variate patterns for forecasting","Gated transformer boosts forecasters by up to 20.7% across 13 benchmarks","Gating cross-variate attention lifts time series forecasting accuracy","Gateformer integrates into LLM and transformer forecasters, lifting accuracy","New gated dual-attention model improves multivariate forecasting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001082,"raw_usage":{"total_tokens":4541,"prompt_tokens":974,"completion_tokens":3567,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":590,"completion_tokens_details":{"reasoning_tokens":3474}},"tokens_in":590,"tokens_out":3567,"duration_ms":25801,"temperature":1.0,"reasoning_tokens":3474,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:45:26.852400+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run Gateformer and all nine baselines from a single codebase with identical chronological splits, look-back length $L=96$, the same prediction horizons, the same seeds, and an equal hyperparameter search budget per model, then compare the per-dataset averages. If Gateformer no longer ranks first in the stated 91 of 130 settings, the central state-of-the-art claim is falsified.","supporting_citations":[{"cited_title":"One fits all: Power general time series analysis by pretrained LM","cited_arxiv_id":null,"evidence_quote":"LLM-based forecaster used as an integration target in the generalizability experiments."}],"review_version":1}