{"id":"d39e9e02-8d33-438b-8d60-aa00458facc5","arxiv_id":"2507.00945","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Zero-shot time series foundation models outperform trained statistical and deep learning baselines for origin-destination crowd flow prediction on three real-world mobility datasets.","lead":"This study applies two pretrained time series foundation models, Moirai and TimesFM, to predict crowd flows between locations in New York, Beijing, and Spain without retraining on those cities. The models beat specialized baselines on error and similarity metrics, suggesting that generic forecasting models can handle flow prediction even with no spatial context or scarce labeled data.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Core comparison lacks a specified TSFM protocol (context, horizon, normalization, input form); reported margins may reflect protocol differences, not model superiority.","rationale":"The reader's conditional verdict is appropriate. I found a more specific version of the same concern: not just baseline tuning but the whole comparison protocol is unspecified. The TSFM input representation and context are undefined, which is more fundamental than hyperparameter tuning. Still, the claim is plausible and testable; a reproducible protocol check would resolve it. The internal percentage inconsistencies in Section 4 add weight but do not by themselves refute the Table 1 numbers. Therefore I would not move the verdict; I recommend keeping conditional acceptance. No ad hominem; all comments are about reproducibility and specification.","tokens_in":6750,"tokens_out":6635,"duration_ms":142510,"concrete_test":"Implement a reproducible evaluation: (1) use the exact splits of Cardia et al. [7]; (2) for all models, predict one step ahead on the test period using an expanding window whose context length is capped at the maximum available history, with per-series instance normalization computed only from train data; (3) tune all deep baselines by grid search on a validation split (e.g., last 15% of train) with 3 seeds; (4) run Moirai and TimesFM under both univariate-per-OD-flow and multivariate-per-origin input modes; (5) report scores with standard deviations. If TSFMs still exceed the best-tuned baselines by the claimed margins in both input modes, the paper's conclusion holds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that zero-shot Moirai/TimesFM outperform trained baselines on flow prediction (Table 1). The evidence rests on an underspecified evaluation protocol. Section 2.4 defines a multivariate per-origin series, yet the Abstract says 'only the temporal evolution of each OD flow', and Section 1 repeats this; the paper never states whether the TSFMs receive one univariate series per OD pair or an n-dimensional vector per origin. This matters because Moirai and TimesFM have different multivariate capabilities and normalization schemes. No context length, prediction horizon, or instance-normalization procedure is reported. The Taxi Beijing dataset is one week (Section 3.3); at hourly granularity this is 168 points, shorter than typical TSFM context windows (e.g., TimesFM uses 512), so the actual setup must be non-obvious (e.g., daily aggregation, shorter context, or multi-step zero-shot) but is left unspecified. If TSFMs use an expanding window that includes earlier test observations while baselines use a fixed pre-test window, the comparison is biased even though both are 'rolling'. Section 3.3 also reports no hyperparameters, validation protocol, or seeded runs for ST-ResNet, DMVSTNet, ACMF, CrowdNet, or MSAGGN, so under-training cannot be ruled out. Finally, Section 4's summary percentages are internally inconsistent with Table 1 (e.g., Bike NYC RMSE reduction should be ~24% vs 23.44; MAE ~15% vs 23.01), suggesting a mismatched computation. These gaps make the headline margins non-falsifiable from the manuscript alone.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a reformulation of crowd flow prediction as per-origin multivariate time series forecasting and applies two pretrained time series foundation models, Moirai and TimesFM, in a strict zero-shot setting. On three public datasets (Bike NYC, Taxi Beijing, Spanish national OD flows), the authors report that the TSFMs outperform statistical and deep learning baselines across RMSE, MAE, and CPC, despite not using explicit spatial information. The paper argues that TSFMs offer a scalable, generalizable alternative to specialized spatio-temporal deep learning models.","tokens_in":7076,"tokens_out":6714,"duration_ms":64644,"significance":"The finding is potentially significant: if confirmed, it would show that general-purpose TSFMs can outperform task-specific deep spatio-temporal models in flow prediction, which has practical implications for data-scarce scenarios. The paper's strengths are its clear problem formulation and the use of real-world datasets. However, the current evidence is undermined by an underspecified evaluation protocol and internal numerical inconsistencies, so the significance is not yet firmly established.","major_comments":[{"comment":"The input representation given to the TSFMs is ambiguous. Definition 2.2 defines a multivariate per-origin series s(i)_tau = [T_{tau,i,1}, ..., T_{tau,i,n}], but the Abstract and Section 1 state that the models receive 'only the temporal evolution of each OD flow' and 'individual time series.' The paper never states whether the TSFMs receive one univariate series per OD pair or an n-dimensional vector per origin. This distinction is critical because Moirai and TimesFM have different multivariate capabilities and normalization schemes, and because the per-origin vector contains destination information, which is arguably spatial structure. The evaluation protocol (context length, prediction horizon, instance normalization) is also not reported. Without these details, the 'strict zero-shot' scenario and the comparison to baselines are not reproducible as described.","section":"Section 2.4 and Abstract/Introduction"},{"comment":"The percentage improvements reported in Section 4 do not match Table 1. For example, comparing Moirai-L to the best baseline MSAGGN on Bike NYC, the RMSE reduction is (8.02 - 6.09)/8.02 = 24.1%, not the reported -23.44%, and the MAE reduction is (3.59 - 3.04)/3.59 = 15.3%, not -23.01%. On Taxi BJ, the RMSE reduction is (14.12 - 9.32)/14.12 = 34.0%, not -32.86%, and the MAE reduction is (11.95 - 7.34)/11.95 = 38.6%, not -39.86%. On Spain OD, the RMSE reduction is (28.05 - 21.74)/28.05 = 22.5%, not -20.83%, and the MAE reduction is (13.26 - 9.94)/13.26 = 25.0%, not -21.33%. These discrepancies affect the abstract's headline numbers (e.g., 'up to 39% lower MAE') and must be corrected with a clear statement of the comparison basis.","section":"Section 4 and Table 1"},{"comment":"The baseline training setup is severely underspecified. For ST-ResNet, DMVSTNet, ACMF, CrowdNet, and MSAGGN, the paper reports no hyperparameters, validation procedure, or number of runs; it merely says the data are split 'as proposed in [7].' Without this information, under-training or mis-tuning of the baselines cannot be ruled out, which would invalidate the claim that TSFMs outperform 'state-of-the-art competitors.' The authors should provide exact configurations or cite the original implementations, and they should report error bars or significance tests, as Table 1 appears to contain single-run results.","section":"Section 3.3"},{"comment":"The Taxi Beijing dataset is one week of data (168 hourly points), which is shorter than the typical context length of TSFMs (e.g., TimesFM uses a default context of 512). The paper does not state how the models are applied to this short series—whether the data are aggregated to a daily resolution, whether a shorter context is used, or whether the evaluation uses an expanding window that includes test observations. Without this information, the validity of the zero-shot evaluation on this dataset is unclear, and the reported gains could be an artifact of the chosen protocol.","section":"Section 3.3 (Taxi Beijing)"}],"minor_comments":[{"comment":"The CCS Concepts and Keywords sections still contain template placeholders ('Do Not Use This Code' and 'Do, Not, Us, This, Code') and must be replaced with correct terms.","section":"Title page"},{"comment":"The model name is written inconsistently as 'TimeFM' in Section 4 and 'TimesFM' elsewhere; please use one spelling consistently.","section":"Throughout"},{"comment":"There is a typo: 'Common Parts of Communters' should be 'Common Part of Commuters'.","section":"Section 1"},{"comment":"The caption and the figure itself are not fully described in the text; the reader is not told what the axes or colors represent, and the figure appears to be a schematic with insufficient explanation.","section":"Figure 1"},{"comment":"The manuscript does not specify which exact versions/checkpoints of Moirai and TimesFM are used (e.g., Moirai-Large, TimesFM-2.0), nor the patch size and forecast horizon settings. These details are necessary for reproducibility.","section":"Section 3.2"},{"comment":"There is no data availability or code availability statement. Given the reproducibility claims, at least the configuration scripts and evaluation code should be made available.","section":"End matter"}],"recommendation":"major_revision","confidential_remarks":"The paper appears to be an extended abstract with unfinished template fields. The central claim is plausible but currently rests on an underspecified evaluation protocol and internally inconsistent numbers. I recommend major revision rather than rejection, as the issues are fixable with a more detailed experimental section and corrected calculations. However, if the authors cannot provide the missing protocol details or reconcile the percentages, the paper should not be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe useful nugget: the paper reframes OD flow prediction as independent multivariate forecasting per origin, then runs Moirai and TimesFM zero-shot on three public datasets. That is a simple, sensible reframing, and the empirical claim—zero-shot TSFMs beat trained baselines without spatial information—is worth checking. The table supports the headline margins on its face, and the use of public datasets and pretrained checkpoints means the result is not an artifact of fitted parameters. Good.\n\nThe soft spots are real and concentrated in the evaluation protocol. The paper never states what context length, prediction horizon, normalization, or input representation the TSFMs receive. The abstract says only the temporal evolution of each OD flow; Section 2.4 instead defines a multivariate vector per origin. The distinction matters because Moirai and TimesFM handle multivariate series and normalization differently. Taxi Beijing has one week of data; at hourly granularity that is 168 points, shorter than typical TSFM context windows, so the actual setup must be non-obvious and is left unspecified. Without this, readers cannot reproduce or falsify the central comparison.\n\nBaselines are also under-specified: no hyperparameters, no validation procedure, no number of runs. The trained-versus-zero-shot comparison is therefore at risk of an uneven playing field. There is also an internal mismatch: the percentage summary in Section 4 does not match Table 1 in several places (e.g., Bike NYC RMSE reduction should be about 24%, not 23.44; MAE about 15%, not 23.01; Taxi Beijing shows similar discrepancies). That is not fatal, but it is the one place the authors interpret their own numbers, and it is wrong.\n\nI do not read this as dishonest or incoherent. The core idea is plausible and useful, and the paper is a legitimate empirical benchmark. But as written, the evidence is thin and the protocol is under-specified. It deserves a serious referee who can demand full evaluation details, corrected percentages, and at least one reproducibility appendix. I would not cite it until those are fixed, but I would bring it to a reading group as a useful example of how TSFMs should—and should not—be evaluated on mobility tasks.\n\nRecommendation: send to peer review, with the expectation of heavy revision.","headline":"Plausible zero-shot TSFM result on flow prediction, but the evaluation protocol is so under-specified that the headline margins are not yet reproducible.","tokens_in":7543,"tokens_out":1987,"would_cite":false,"duration_ms":23449,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that generic zero-shot time-series foundation models, given only the temporal evolution of origin–destination flows, outperform specialized spatiotemporal deep learning models on crowd-flow prediction across three…","keywords":["crowd flow prediction","time series foundation models","zero-shot forecasting","origin-destination flows","Moirai","TimesFM","human mobility","spatiotemporal prediction"],"falsifier":"Retrain CrowdNet and MSAGGN on the same Bike NYC and Taxi BJ splits with a documented hyperparameter search and repeated runs, then compare their best RMSE, MAE, and CPC to Moirai-L's values of 6.09/3.04/0.72 and 9.32/7.34/0.62; if the tuned baselines match or beat those numbers, the paper's central claim of zero-shot TSFM superiority on these benchmarks would be falsified.","tokens_in":6585,"feed_emoji":"🚇","tokens_out":9727,"duration_ms":94778,"temperature":0.7,"pith_summary":"This paper tries to establish that general-purpose time-series foundation models can serve as crowd-flow predictors without any training on the target city. The claim is that Moirai and TimesFM, used zero-shot and fed only the temporal history of each origin–destination flow, beat classical statistical models and specialized spatial deep networks on Bike NYC, Taxi Beijing, and Spanish nationwide origin–destination data. If the claim holds, accurate flow forecasting no longer requires annotated target-city data, a spatial graph, or a grid-based architecture; a pretrained forecaster can be pointed at the raw flow series directly. On the reported numbers the advantage is up to 33% lower RMSE, up to 39% lower MAE, and up to 49% higher common part of commuters (CPC) against the baselines.","feed_headline":"Zero-shot time-series models beat specialized flow predictors","feed_subtitle":"Moirai and TimesFM cut RMSE by up to 33% on three mobility datasets without retraining or spatial inputs.","key_machinery":"The central object is the per-origin time series reformulation (Definition 2.2): instead of modeling the OD tensor jointly, each origin tile $i$ yields a sequence of vectors $s^{(i)}_\\tau = [T_{\\tau,i,1}, \\dots, T_{\\tau,i,n}]$, and the task is to forecast $s^{(i)}_{t+1}$ from the history $s^{(i)}_1, \\dots, s^{(i)}_t$. This reduces one spatiotemporal tensor problem to $n$ independent multivariate forecasting tasks, which is exactly the input format that pretrained transformer forecasters such as Moirai and TimesFM consume. The mechanism doing the work is thus the decomposition plus zero-shot transfer: no fine-tuning, no spatial adjacency, and no convolution over a grid.","core_discovery":"The paper's central claim is that per-location decomposition turns crowd-flow prediction into a set of independent multivariate time-series tasks and that pretrained time-series foundation models solve those tasks better than the spatial baselines. For each origin tile the outgoing flows to all destinations form one series; Moirai and TimesFM forecast the next vector from that series alone. In Table 1, Moirai-L achieves the best score on every dataset and metric—RMSE 6.09, MAE 3.04, CPC 0.72 on Bike NYC; RMSE 9.32, MAE 7.34, CPC 0.62 on Taxi BJ; RMSE 21.74, MAE 9.94, CPC 0.67 on Spain OD—while TimesFM lands close behind. Because the deep-learning baselines have explicit spatial structure and the foundation models do not, the authors interpret this as evidence that the temporal regularities in origin–destination flows carry most of the predictive signal, and that zero-shot transfer to new cities and scales is feasible.","pith_inferences":["A consequence left implicit is that if the temporal signal dominates these flow series, adding spatial context to Moirai or TimesFM as extra input channels should give only small gains, and the size of that gain is directly measurable.","The comparison may partly reflect baseline tuning rather than TSFM superiority, so retraining the deep baselines with a documented hyperparameter search is the decisive check of the claimed advantage.","At finer temporal or spatial granularities the OD series become sparser and noisier, so the zero-shot advantage should be retested before extending the conclusion to real-time or street-level prediction."],"forward_implications":["A city or region with no labeled flow history and no spatial map can still get flow forecasts from a pretrained model, which lowers the data barrier for mobility analysis.","Decomposing OD tensors per origin makes prediction scalable: adding a region adds one more independent time series instead of requiring a retrained joint spatiotemporal model.","Because the winners ignore spatial adjacency, spatial structure appears less essential for these benchmarks than the surveyed deep-learning literature assumed.","The same zero-shot recipe becomes a natural baseline for future crowd-flow studies, and a candidate for other per-pair mobility prediction tasks such as demand and next-location forecasting."],"supporting_citations":[{"why":"Supplies Moirai, the transformer forecaster whose zero-shot predictions are the paper's main evidence.","marker":"[16]"},{"why":"Supplies TimesFM, the second zero-shot forecaster whose results confirm the pattern.","marker":"[9]"},{"why":"Provides the dataset processing and train/test splits used for the city benchmarks, and supplies the CrowdNet baseline.","marker":"[7]"},{"why":"Supplies the Taxi Beijing mobility data on which the TSFMs are tested.","marker":"[18]"},{"why":"Supplies the Spanish nationwide OD flow dataset used for the national-scale evaluation.","marker":"[14]"},{"why":"Defines the crowd-flow task and the RMSE, MAE, and CPC evaluation conventions used in the comparison.","marker":"[12]"},{"why":"Supplies the ST-ResNet baseline, a representative spatial deep-learning model the claim must beat.","marker":"[19]"},{"why":"Supplies MSAGGN, the strongest reported baseline on two of the three datasets.","marker":"[15]"}],"fun_headline_variants":["Zero-shot time-series models beat spatial flow predictors","No spatial info? Foundation models still win flow forecasting","Pretrained time-series models top mobility flow baselines","Zero-shot foundation models outdo spatial deep learning","Time-series foundation models predict flows without maps"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the baseline deep-learning models were trained well enough to represent their intended performance; the paper reports no hyperparameters, validation procedure, or number of runs for ST-ResNet, DMVSTNet, ACMF, CrowdNet, or MSAGGN.","fun_headline_variants_meta":{"raw":{"variants":["Zero-shot time-series models beat spatial flow predictors","No spatial info? Foundation models still win flow forecasting","Pretrained time-series models top mobility flow baselines","Zero-shot foundation models outdo spatial deep learning","Time-series foundation models predict flows without maps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000525,"raw_usage":{"total_tokens":2503,"prompt_tokens":879,"completion_tokens":1624,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":495,"completion_tokens_details":{"reasoning_tokens":1552}},"tokens_in":495,"tokens_out":1624,"duration_ms":13660,"temperature":1.0,"reasoning_tokens":1552,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:01:13.766423+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain CrowdNet and MSAGGN on the same Bike NYC and Taxi BJ splits with a documented hyperparameter search and repeated runs, then compare their best RMSE, MAE, and CPC to Moirai-L's values of 6.09/3.04/0.72 and 9.32/7.34/0.62; if the tuned baselines match or beat those numbers, the paper's central claim of zero-shot TSFM superiority on these benchmarks would be falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the dataset processing and train/test splits used for the city benchmarks, and supplies the CrowdNet baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Taxi Beijing mobility data on which the TSFMs are tested."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Spanish nationwide OD flow dataset used for the national-scale evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the crowd-flow task and the RMSE, MAE, and CPC evaluation conventions used in the comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the ST-ResNet baseline, a representative spatial deep-learning model the claim must beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies MSAGGN, the strongest reported baseline on two of the three datasets."}],"review_version":1}