Pith. sign in

REVIEW 4 major objections 5 minor 55 references

A Modular Multitask Reasoning Framework Integrating Spatio-temporal Models and LLMs

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a hybrid framework called STReason, which pairs large language models with modular spatio-temporal analysis tools, can answer complex queries about traffic and air quality with far higher factual accuracy than the…

desk verdict STReason is a promising modular framework with a useful new benchmark, but the headline numbers compare tool-using agents to tool-less LLMs, so the gains are not yet attributable to the planning itself. read the letter →

arxiv 2506.20073 v1 pith:PIB7JFA3 submitted 2025-06-25 cs.CL cs.AIcs.LG

Kethmi Hirushini Hettige , Jiahao Ji , Cheng Long , Shili Xiang , Gao Cong , Jingyuan Wang This is my paper
classification cs.CLcs.AIcs.LG
keywords spatio-temporalreasoninglargelanguagemodelsprogram-basedin-contextlearningbenchmarkdatasetlong-formquestionansweringanomalydetectiontrafficforecasting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a hybrid framework called STReason, which pairs a large language model with specialized spatio-temporal analysis modules, can answer complex natural-language questions about traffic and air quality far more accurately than the LLM alone. Without task-specific fine-tuning, STReason decomposes each query into an explicit program of modular steps, then executes those steps with dedicated modules for trend analysis, anomaly detection, forecasting, and constraint checking. On a new benchmark of 150 multi-task questions, STReason reaches 84.44% factual correctness versus 32.83% for the strongest LLM baseline, and human evaluators preferred its answers 74.1% of the time. The paper argues this shows that LLM planning plus modular execution, rather than direct question-answering by the LLM, is the right path for reliable, interpretable spatio-temporal reasoning.

What carries the argument

The load-bearing mechanism is the two-stage pipeline: a Command Generator—an LLM prompted with in-context query-program pairs plus a Function Pool, a curated dictionary of module signatures, parameters, and purposes—produces an ST Program, a sequence of executable commands with module names, arguments, and output variables; a Command Interpreter then executes each command as a Python class and merges the module summaries into a single coherent answer. The Function Pool is the grounding element: it lets the LLM pick the correct module even when the in-context examples do not match the query's wording or task type.

What would settle it

Run STReason on a set of spatio-temporal queries whose task types are absent from its in-context example pool—for example, queries about maritime traffic or epidemiological spread—and measure end-to-end factuality; if the score falls to roughly the 33% level of the best LLM baseline, the framework's claimed generalizable superiority fails.

Watch

Extended reading notes

Core claim

The central claim is that program-based modular execution, guided by in-context learning, lets a hybrid system outperform advanced LLM baselines on spatio-temporal reasoning without any task-specific fine-tuning. The key result is the factuality gap: STReason scores 84.44% against 32.83% for the best baseline, with perfect constraint adherence (100%) and coherence (100%), while producing long-form explanations. The paper attributes this to the ST Program: a structured sequence of commands (e.g., ANALYZE_TREND, DETECT_ANOMALY, FORECAST) that the LLM generates by matching the query to curated examples and a Function Pool, and that the Command Interpreter executes as 12 Python-class modules, compiling the outputs into a narrative answer. The paper also introduces a benchmark dataset and a three-metric evaluation framework designed for long-form spatio-temporal reasoning.

Load-bearing premise

The framework's advantage rests on the LLM command generator correctly mapping each new query to the right modules using manually curated in-context examples and a function pool; when those examples exclude queries similar to the test query, program precision falls from 0.98 to 0.61, so the reported gains may not transfer to unseen task types.

Editorial extensions

If this is right

  • STReason outperforms the best LLM baseline on factuality by a large margin (84.44% vs 32.83%) without task-specific fine-tuning, indicating that program-based decomposition is a viable alternative to end-to-end LLM generation for spatio-temporal questions.
  • The framework achieves perfect constraint adherence (100%), so generated answers reliably respect query-specific thresholds, time spans, and granularity requirements.
  • The modular design means new spatio-temporal tasks can be added by registering a new module class, without retraining the LLM or the interpreter.
  • The introduced benchmark of 150 multi-task instances provides a reusable evaluation target for any spatio-temporal reasoning system.
  • Program precision drops sharply when in-context examples exclude queries similar to the test query (to 0.61), which bounds the framework's current robustness to the coverage of its curated example pool.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The factuality gap suggests that LLM baselines fail not at producing fluent language but at executing exact analytical steps; if so, other tool-augmented agents with tighter numerical backends should also close part of the gap, which this paper does not test.
  • The same decompose-and-execute pattern could be transferred to other data-intensive reasoning domains (e.g., climate, finance, or mobility) by registering domain-specific modules, a testable extension the paper proposes only as future work.
  • The reliance on manually curated in-context examples might be reduced by automatically retrieving examples similar to the test query, which the paper's own ablation suggests would raise program precision from 0.61 toward 0.96.
  • The evaluation metrics use an LLM verifier against ground-truth components; an independent human-scored replication of the factuality and coherence scores on the same outputs would be a natural check on the reported numbers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes STReason, a framework that combines an LLM-based command generator with a pool of 12 spatio-temporal computational modules. A natural-language query is decomposed into an executable 'ST Program' via in-context learning, then executed by a command interpreter over real traffic and air-quality datasets to produce long-form responses. The authors construct a 150-instance benchmark spanning analysis, anomaly detection, and prediction/reasoning, and propose three LLM-based evaluation metrics (constraint adherence, factuality, coherence). They report that STReason achieves 84.44% factuality versus 32.83% for the best baseline and wins 74.1% of human-preference comparisons, and they perform ablations on the program generator.

Significance. If valid, STReason demonstrates a promising and practical architecture for end-to-end spatio-temporal question answering that requires no task-specific fine-tuning, provides interpretable intermediate rationale, and is evaluated on a new benchmark. The paper includes useful ablations showing the contribution of the function pool, and the human-preference results lend some external validation to the automatic scores. The main caveat is that the headline comparison does not yet isolate the effect of the framework's program decomposition because the baselines lack access to the data and tools given to STReason.

major comments (4)
  1. [§4.2, Table 1a] The main comparison is not apples-to-apples. STReason's command interpreter executes data-loading and forecasting modules over the real datasets, while the six baselines are prompted LLMs with no access to these data or functions (Appendix A.5 describes only model access and sampling parameters, not prompts or tools). The factuality margin (84.44% vs. 32.83%) and the forecasting results in Table 1b (where baseline predictions must be post-hoc zero-padded or forward-filled) therefore reflect unequal resources more than a difference in reasoning. Please include exact baseline prompts, add at least one tool-augmented LLM baseline that can access the same data and functions, and rephrase the claim accordingly.
  2. [§3.2 and Appendix A.3] The implementations of the 12 modules are not described beyond call signatures. The paper refers to 'state-of-the-art spatio-temporal prediction models' but never identifies which specific models or algorithms implement FORECAST, DETECT_ANOMALY_ST_DATA, ANALYZE_TREND, etc. Without this information, the experimental results cannot be reproduced or compared against other systems. Please specify the underlying models (with references and versions) and provide training details or code.
  3. [§4.1 and Appendix A.6] The proposed evaluation metrics rely on an LLM-based verifier, but the manuscript does not state which LLM is used, how its prompts were designed, or whether the verifier's outputs were validated beyond the 18-query human study. Since the ground truth for factuality consists of data-derived components, and since the verifier may itself be a GPT-family model, the evaluation could favor the system that emits fluent, well-structured text. Please report the verifier model, prompts, and a failure analysis, and consider releasing the benchmark to enable independent evaluation.
  4. [§4.4, Table 2 and §5] The ablation demonstrates that program generation accuracy degrades sharply when in-context examples exclude queries similar to the test query (precision falls from 0.9816 to 0.6091). The paper acknowledges this reliance on manual curation, but it directly limits the scope of the Introduction's claim that STReason generalizes 'across diverse domains' and to 'new tasks without significant retraining.' I encourage the authors to either present evidence on unseen task types or narrow the generalization claims in the abstract and conclusion.
minor comments (5)
  1. [§4.2] There are typos on this page: 'its’ ability' should be 'its ability', and 'stength' should be 'strength'.
  2. [Table 4] Table 4 lists DeepSeek-V3's access type as 'API (OpenAI)', but DeepSeek-V3 is not an OpenAI model; this should be corrected to the actual API provider.
  3. [§1, Figure 1] The caption 'Comparison between Human Expert and STReason Model workflows' is slightly awkward; consider rewording to 'workflow comparison'.
  4. [General] The paper alternates between 'spatio-temporal' and 'spatiotemporal'; please standardize the spelling throughout.
  5. [§1] The demo link (https://anon.to/T5lL94) is anonymous and may not be stable; please include a permanent repository link once available.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: STReason's results are empirical comparisons with external data, not self-referential derivations.

full rationale

I walked the derivation chain from the abstract through the methodology and experiments. The claim is that STReason, without task-specific fine-tuning, outperforms advanced LLM baselines on constraint adherence, factuality, and coherence. This claim is supported by direct measurements against a ground-truth benchmark built from external real-world data (PEMS-BAY, METR-LA, Beijing, Shenzhen), not by any equation that reduces to its own inputs. The Command Generator's in-context query-program pairs and Function Pool are inputs, and the generated ST programs are evaluated against separately annotated ground-truth programs; this is an empirical fit to a benchmark, not a fitted parameter renamed as a prediction. The authors cite their own prior work (AirPhyNet, STDEN) only as background examples of spatio-temporal deep learning, and no load-bearing argument rests on those citations. There is no imported 'uniqueness theorem' and no ansatz smuggled in through self-citation; the modular design is explicitly attributed to VISPROG, an external source. The paper's main weaknesses are fairness and external validity rather than circularity: baselines are unaided LLMs while STReason invokes data-loading and forecasting modules; the benchmark and metrics are author-constructed; and Section 5 explicitly admits that reliance on manually curated in-context examples may limit scalability to unseen task types, with Table 2 showing precision dropping to 0.6091 when query-similar examples are excluded. These are significant limitations, but they do not make any reported result equivalent to its input by construction. Therefore no specific circular step can be quoted and exhibited, and the appropriate score is low.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim rests on assumptions about the reliability of the LLM command generator, the accuracy of the specialized modules, the validity of LLM-based evaluation, and the quality of the author-constructed benchmark. None of these are verified with released artifacts or external validation, so the empirical headline should be treated as conditional.

assumptions (4)
  • domain assumption Manual in-context examples plus Function Pool are sufficient for accurate program generation on new queries.
    The whole method rests on the LLM choosing the right modules from curated examples; ablation in Table 2 shows performance degrades sharply when examples exclude similar queries (F1 0.7571), so this premise is load-bearing and only partially validated.
  • domain assumption The 12 modules, including the unnamed state-of-the-art ST prediction and anomaly models, are correct and suitably accurate for the underlying analytical tasks.
    Appendix A.3 describes module signatures but does not name or evaluate the forecasting and anomaly detection models; if these modules are weak, the final answers and factuality scores inherit that weakness.
  • domain assumption The LLM-based evaluators used for constraint adherence, factuality, and coherence produce valid scores.
    Appendix A.6 specifies prompts but not which LLM performs the evaluation, whether it was validated against human judgments, or its reliability; the paper claims human alignment but does not quantify evaluator agreement.
  • domain assumption The 150-instance benchmark's ground-truth programs and answer components are correct, complete, and unbiased.
    The dataset is constructed by the authors from four public data sources, with no release link and no inter-annotator agreement or third-party validation reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Modular Multitask Reasoning Framework Integrating Spatio-temporal Models and LLMs." pith.science (2026). https://pith.science/paper/PIB7JFA3

@misc{pith2026250620073,
  author       = {Pith},
  title        = {Pith review of: A Modular Multitask Reasoning Framework Integrating Spatio-temporal Models and LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PIB7JFA3}},
  note         = {Machine review of arXiv:2506.20073}
}
read the original abstract

Spatio-temporal data mining plays a pivotal role in informed decision making across diverse domains. However, existing models are often restricted to narrow tasks, lacking the capacity for multi-task inference and complex long-form reasoning that require generation of in-depth, explanatory outputs. These limitations restrict their applicability to real-world, multi-faceted decision scenarios. In this work, we introduce STReason, a novel framework that integrates the reasoning strengths of large language models (LLMs) with the analytical capabilities of spatio-temporal models for multi-task inference and execution. Without requiring task-specific finetuning, STReason leverages in-context learning to decompose complex natural language queries into modular, interpretable programs, which are then systematically executed to generate both solutions and detailed rationales. To facilitate rigorous evaluation, we construct a new benchmark dataset and propose a unified evaluation framework with metrics specifically designed for long-form spatio-temporal reasoning. Experimental results show that STReason significantly outperforms advanced LLM baselines across all metrics, particularly excelling in complex, reasoning-intensive spatio-temporal scenarios. Human evaluations further validate STReason's credibility and practical utility, demonstrating its potential to reduce expert workload and broaden the applicability to real-world spatio-temporal tasks. We believe STReason provides a promising direction for developing more capable and generalizable spatio-temporal reasoning systems.

Figures

Figures reproduced from arXiv: 2506.20073 by the authors.

Figure 1
Figure 1. Comparison between Human Expert and STReason Model workflows for answering a [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. STReason Command Generator uses a Func￾tion Pool and in-context query-program pairs to gener￾ate executable ST-Program for a given user query. The Command Generator is responsi￾ble for translating complex natural lan￾guage queries into executable ST-Programs. Specifically, it decomposes complex spatio￾temporal queries into manageable sub￾tasks, leveraging the in-context learning capabilities of LLMs. To enhance grou… view at source ↗
Figure 3
Figure 3. Command Interpreter Modules STReason features 12 specialized modules supporting three core spatio-temporal tasks: Analysis, Prediction and Reasoning, and Anomaly Detection (see [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: STReason vs. Baseline Preference Rate Anomaly Detection Analysis Prediction and Reasoning 0 20 40 60 80 100 Preference Rate (%) 59.88% 77.78% 84.57% [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 6
Figure 6. Figure 6: Human Evaluation: Qualitative Feedback Moreover, qualitative feedback from eval￾uators further supported these findings. As shown in [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 8
Figure 8. Figure 8: Effect of No: of Incontext Examples Effect of Task Type: We first assess performance across different task categories. As shown in [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: ST-program for Analysis Task [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: ST-program for Anomaly Detection Task 13 [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: ST-program for Prediction and Reasoning Task [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: Structure of Sample Function A.3 Command Interpreter Module Details We illustrate the details of the 12 modules within the STReason framework below to better understand their funcitionalities and specifications. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: Execution Rationale for Spatio-temporal Analysis Query [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]
Figure 14
Figure 14. Figure 14: Execution Rationale for Spatio-temporal Forecast Query [PITH_FULL_IMAGE:figures/full_fig_p017_14.png]
Figure 15
Figure 15. Figure 15: Prompt for assessing Constraint Adherence [PITH_FULL_IMAGE:figures/full_fig_p018_15.png]
Figure 16
Figure 16. Figure 16: Prompt for assessing Factual Correctness [PITH_FULL_IMAGE:figures/full_fig_p019_16.png]
Figure 17
Figure 17. Figure 17: Prompt for assessing Logical Coherence 19 [PITH_FULL_IMAGE:figures/full_fig_p019_17.png]
Figure 18
Figure 18. Figure 18: Evaluator Background and reasoning tasks. Your evaluations will play a pivotal role in identifying the most effective models and will guide the development of future enhancements. Task Overview: You will be presented with 18 questions, each accompanied by two answers.…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 38 canonical work pages

  1. [1]

    Airphynet: Harnessing physics- guided neural networks for air quality prediction,

    K. H. Hettige, J. Ji, S. Xiang, C. Long, G. Cong, and J. Wang, “Airphynet: Harnessing physics- guided neural networks for air quality prediction,” arXiv preprint arXiv:2402.03784, 2024

    arXiv 2024

  2. [2]

    Airformer: Predicting nationwide air quality in china with transformers,

    Y . Liang, Y . Xia, S. Ke, Y . Wang, Q. Wen, J. Zhang, Y . Zheng, and R. Zimmermann, “Airformer: Predicting nationwide air quality in china with transformers,” in Proceedings of the AAAI conference on artificial intelligence, vol. 37, pp. 14329–14337, 2023

    work page 2023

  3. [3]

    Urbangpt: Spatio-temporal large language models,

    Z. Li, L. Xia, J. Tang, Y . Xu, L. Shi, L. Xia, D. Yin, and C. Huang, “Urbangpt: Spatio-temporal large language models,” in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 5351–5362, 2024

    work page 2024

  4. [4]

    Stden: Towards physics-guided neural networks for traffic flow prediction,

    J. Ji, J. Wang, Z. Jiang, J. Jiang, and H. Zhang, “Stden: Towards physics-guided neural networks for traffic flow prediction,” in Proceedings of the AAAI conference on artificial intelligence , vol. 36, pp. 4048–4056, 2022

    work page 2022

  5. [5]

    Brain-jepa: Brain dynamics foundation model with gradient positioning and spatiotemporal masking,

    Z. Dong, R. Li, Y . Wu, T. T. Nguyen, J. Chong, F. Ji, N. Tong, C. Chen, and J. H. Zhou, “Brain-jepa: Brain dynamics foundation model with gradient positioning and spatiotemporal masking,” Advances in Neural Information Processing Systems , vol. 37, pp. 86048–86073, 2024

    work page 2024

  6. [6]

    Urban flow prediction from spatiotemporal data using machine learning: A survey,

    P. Xie, T. Li, J. Liu, S. Du, X. Yang, and J. Zhang, “Urban flow prediction from spatiotemporal data using machine learning: A survey,” Information Fusion, vol. 59, pp. 1–12, 2020

    work page 2020

  7. [7]

    Spatio-temporal graph neural networks for predictive learning in urban computing: A survey,

    G. Jin, Y . Liang, Y . Fang, Z. Shao, J. Huang, J. Zhang, and Y . Zheng, “Spatio-temporal graph neural networks for predictive learning in urban computing: A survey,”IEEE Transactions on Knowledge and Data Engineering, vol. 36, no. 10, pp. 5388–5408, 2023

    work page 2023

  8. [8]

    Deep learning for spatio-temporal data mining: A survey,

    S. Wang, J. Cao, and S. Y . Philip, “Deep learning for spatio-temporal data mining: A survey,” IEEE transactions on knowledge and data engineering, vol. 34, no. 8, pp. 3681–3700, 2020

    work page 2020

Show all 55 references
  1. [9]

    A survey of generative techniques for spatial-temporal data mining,

    Q. Zhang, H. Wang, C. Long, L. Su, X. He, J. Chang, T. Wu, H. Yin, S.-M. Yiu, Q. Tian, et al., “A survey of generative techniques for spatial-temporal data mining,” arXiv preprint arXiv:2405.09592, 2024

    2024 arXiv

  2. [10]

    Foundation models for time series analysis: A tutorial and survey,

    Y . Liang, H. Wen, Y . Nie, Y . Jiang, M. Jin, D. Song, S. Pan, and Q. Wen, “Foundation models for time series analysis: A tutorial and survey,” in Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, pp. 6555–6565, 2024

    2024

  3. [11]

    Foundation models for spatio-temporal data science: A tutorial and survey,

    Y . Liang, H. Wen, Y . Xia, M. Jin, B. Yang, F. Salim, Q. Wen, S. Pan, and G. Cong, “Foundation models for spatio-temporal data science: A tutorial and survey,” arXiv preprint arXiv:2503.13502, 2025

    2025 arXiv

  4. [12]

    TEMPO: Prompt-based generative pre-trained transformer for time series forecasting,

    D. Cao, F. Jia, S. O. Arik, T. Pfister, Y . Zheng, W. Ye, and Y . Liu, “TEMPO: Prompt-based generative pre-trained transformer for time series forecasting,” in ICLR, 2023

    2023

  5. [13]

    One Fits All: Power general time series analysis by pretrained lm,

    T. Zhou, P. Niu, X. Wang, L. Sun, and R. Jin, “One Fits All: Power general time series analysis by pretrained lm,” in NeurIPS, pp. 1–34, 2023

    2023

  6. [14]

    Can large language models be anomaly detectors for time series?,

    S. Alnegheimish, L. Nguyen, L. Berti-Équille, and K. Veeramachaneni, “Can large language models be anomaly detectors for time series?,” in DSAA, pp. 1–10, 2024

    2024

  7. [15]

    GATGPT: A pre-trained large language model with graph attention network for spatiotemporal imputation,

    Y . Chen, X. Wang, and G. Xu, “GATGPT: A pre-trained large language model with graph attention network for spatiotemporal imputation,” arXiv, 2023

    2023

  8. [16]

    Promptst: Prompt-enhanced spatio-temporal multi-attribute prediction,

    Z. Zhang, X. Zhao, Q. Liu, C. Zhang, Q. Ma, W. Wang, H. Zhao, Y . Wang, and Z. Liu, “Promptst: Prompt-enhanced spatio-temporal multi-attribute prediction,” in Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pp. 3195–3205, 2023

    2023

  9. [17]

    Unist: A prompt-empowered universal model for urban spatio-temporal prediction,

    Y . Yuan, J. Ding, J. Feng, D. Jin, and Y . Li, “Unist: A prompt-empowered universal model for urban spatio-temporal prediction,” in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 4095–4106, 2024. 10

    2024

  10. [18]

    Are language models actually useful for time series forecasting?,

    M. Tan, M. Merrill, V . Gupta, T. Althoff, and T. Hartvigsen, “Are language models actually useful for time series forecasting?,”Advances in Neural Information Processing Systems, vol. 37, pp. 60162–60191, 2024

    2024

  11. [19]

    Position: Llms can’t plan, but can help planning in llm-modulo frameworks,

    S. Kambhampati, K. Valmeekam, L. Guan, M. Verma, K. Stechly, S. Bhambri, L. P. Saldyt, and A. B. Murthy, “Position: Llms can’t plan, but can help planning in llm-modulo frameworks,” in Forty-first International Conference on Machine Learning, 2024

    2024

  12. [20]

    Spatial-temporal large language model for traffic prediction,

    C. Liu, S. Yang, Q. Xu, Z. Li, C. Long, Z. Li, and R. Zhao, “Spatial-temporal large language model for traffic prediction,” in 2024 25th IEEE International Conference on Mobile Data Management (MDM), pp. 31–40, IEEE, 2024

    2024

  13. [21]

    TimeCMA: Towards llm-empowered multivariate time series forecasting via cross-modality alignment,

    C. Liu, Q. Xu, H. Miao, S. Yang, L. Zhang, C. Long, Z. Li, and R. Zhao, “TimeCMA: Towards llm-empowered multivariate time series forecasting via cross-modality alignment,” in AAAI, 2025

    2025

  14. [22]

    Lc-llm: Explainable lane-change intention and trajectory predictions with large language models,

    M. Peng, X. Guo, X. Chen, K. Chen, M. Zhu, L. Chen, and F.-Y . Wang, “Lc-llm: Explainable lane-change intention and trajectory predictions with large language models,” Communications in Transportation Research, vol. 5, p. 100170, 2025

    2025

  15. [23]

    Genfollower: Enhancing car-following prediction with large language models,

    X. Chen, M. Peng, P. Tiu, Y . Wu, J. Chen, M. Zhu, and X. Zheng, “Genfollower: Enhancing car-following prediction with large language models,”IEEE Transactions on Intelligent Vehicles, 2024

    2024

  16. [24]

    Towards explainable traffic flow prediction with large language models,

    X. Guo, Q. Zhang, J. Jiang, M. Peng, M. Zhu, and H. F. Yang, “Towards explainable traffic flow prediction with large language models,” Communications in Transportation Research, vol. 4, p. 100150, 2024

    2024

  17. [25]

    Urbanllm: Autonomous urban activity planning and management with large language models,

    Y . Jiang, Q. Chao, Y . Chen, X. Li, S. Liu, and G. Cong, “Urbanllm: Autonomous urban activity planning and management with large language models,” arXiv preprint arXiv:2406.12360 , 2024

    2024 arXiv

  18. [26]

    Beyond forecasting: Composi- tional time series reasoning for end-to-end task execution,

    W. Ye, Y . Zhang, W. Yang, L. Tang, D. Cao, J. Cai, and Y . Liu, “Beyond forecasting: Composi- tional time series reasoning for end-to-end task execution,”arXiv preprint arXiv:2410.04047, 2024

    2024 arXiv

  19. [27]

    ELI5: Long form question answering,

    A. Fan, Y . Jernite, E. Perez, D. Grangier, J. Weston, and M. Auli, “ELI5: Long form question answering,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (A. Korhonen, D. Traum, and L. Màrquez, eds.), (Florence, Italy), pp. 3558–3567,...

    2019

  20. [28]

    GeoLLM: Extracting geospatial knowledge from large language models,

    R. Manvi, S. Khanna, G. Mai, M. Burke, D. B. Lobell, and S. Ermon, “GeoLLM: Extracting geospatial knowledge from large language models,” in The Twelfth International Conference on Learning Representations, 2024

    2024

  21. [29]

    Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face,

    Y . Shen, K. Song, X. Tan, D. Li, W. Lu, and Y . Zhuang, “Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face,” Advances in Neural Information Processing Systems, vol. 36, pp. 38154–38180, 2023

    2023

  22. [30]

    Visual programming: Compositional visual reasoning without train- ing,

    T. Gupta and A. Kembhavi, “Visual programming: Compositional visual reasoning without train- ing,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14953–14962, 2023

    2023

  23. [31]

    Large models for time series and spatio-temporal data: A survey and outlook,

    M. Jin, Q. Wen, Y . Liang, C. Zhang, S. Xue, X. Wang, J. Zhang, Y . Wang, H. Chen, X. Li,et al., “Large models for time series and spatio-temporal data: A survey and outlook,” arXiv preprint arXiv:2310.10196, 2023

    2023 arXiv

  24. [32]

    Unitime: A language- empowered unified model for cross-domain time series forecasting,

    X. Liu, J. Hu, Y . Li, S. Diao, Y . Liang, B. Hooi, and R. Zimmermann, “Unitime: A language- empowered unified model for cross-domain time series forecasting,” inProceedings of the ACM Web Conference 2024, pp. 4095–4106, 2024

    2024

  25. [33]

    Large language models are zero-shot time series forecasters,

    N. Gruver, M. Finzi, S. Qiu, and A. G. Wilson, “Large language models are zero-shot time series forecasters,” Advances in Neural Information Processing Systems, vol. 36, pp. 19622–19635, 2023. 11

    2023

  26. [34]

    Where would i go next? large language models as human mobility predictors,

    X. Wang, M. Fang, Z. Zeng, and T. Cheng, “Where would i go next? large language models as human mobility predictors,” arXiv preprint arXiv:2308.15197, 2023

    2023 arXiv

  27. [35]

    Gpt-4o system card,

    A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford, et al., “Gpt-4o system card,” arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  28. [36]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,

    D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi,et al., “Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,” arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  29. [37]

    Language models are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., “Language models are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020

    1901

  30. [38]

    Tree of thoughts: Deliberate problem solving with large language models,

    S. Yao, D. Yu, J. Zhao, I. Shafran, T. Griffiths, Y . Cao, and K. Narasimhan, “Tree of thoughts: Deliberate problem solving with large language models,” Advances in neural information processing systems, vol. 36, pp. 11809–11822, 2023

    2023

  31. [39]

    Self-consistency improves chain of thought reasoning in language models,

    X. Wang, J. Wei, D. Schuurmans, Q. V . Le, E. H. Chi, S. Narang, A. Chowdhery, and D. Zhou, “Self-consistency improves chain of thought reasoning in language models,” inThe Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023,...

    2023

  32. [40]

    Towards revealing the mystery behind chain of thought: a theoretical perspective,

    G. Feng, B. Zhang, Y . Gu, H. Ye, D. He, and L. Wang, “Towards revealing the mystery behind chain of thought: a theoretical perspective,”Advances in Neural Information Processing Systems, vol. 36, pp. 70757–70798, 2023

    2023

  33. [41]

    Chain-of-table: Evolving tables in the reasoning chain for table understanding,

    Z. Wang, H. Zhang, C.-L. Li, J. M. Eisenschlos, V . Perot, Z. Wang, L. Miculicich, Y . Fujii, J. Shang, C.-Y . Lee, and T. Pfister, “Chain-of-table: Evolving tables in the reasoning chain for table understanding,” ICLR, 2024

    2024

  34. [42]

    Stbench: Assessing the ability of large language models in spatio-temporal analysis,

    W. Li, D. Yao, R. Zhao, W. Chen, Z. Xu, C. Luo, C. Gong, Q. Jing, H. Tan, and J. Bi, “Stbench: Assessing the ability of large language models in spatio-temporal analysis,” arXiv preprint arXiv:2406.19065, 2024

    2024 arXiv

  35. [43]

    Situatedgen: Incorporating geographical and temporal contexts into generative commonsense reasoning,

    Y . Zhang and X. Wan, “Situatedgen: Incorporating geographical and temporal contexts into generative commonsense reasoning,” Advances in Neural Information Processing Systems , vol. 36, pp. 67355–67373, 2023

    2023

  36. [44]

    AutoGPT

    Significant Gravitas, “AutoGPT.”

  37. [45]

    Geogpt: Understanding and processing geospatial tasks through an autonomous gpt,

    Y . Zhang, C. Wei, S. Wu, Z. He, and W. Yu, “Geogpt: Understanding and processing geospatial tasks through an autonomous gpt,” arXiv preprint arXiv:2307.07930, 2023

    2023 arXiv

  38. [46]

    Large language models as urban residents: An llm agent framework for personal mobility generation,

    W. JIAWEI, R. Jiang, C. Yang, Z. Wu, R. Shibasaki, N. Koshizuka, C. Xiao, et al., “Large language models as urban residents: An llm agent framework for personal mobility generation,” Advances in Neural Information Processing Systems, vol. 37, pp. 124547–124574, 2024

    2024

  39. [47]

    Large language models empowered agent-based modeling and simulation: A survey and perspectives,

    C. Gao, X. Lan, N. Li, Y . Yuan, J. Ding, Z. Zhou, F. Xu, and Y . Li, “Large language models empowered agent-based modeling and simulation: A survey and perspectives,” Humanities and Social Sciences Communications, vol. 11, no. 1, pp. 1–24, 2024

    2024

  40. [48]

    Diffusion convolutional recurrent neural network: Data-driven traffic forecasting,

    Y . Li, R. Yu, C. Shahabi, and Y . Liu, “Diffusion convolutional recurrent neural network: Data-driven traffic forecasting,”arXiv preprint arXiv:1707.01926, 2017

    2017 arXiv

  41. [49]

    A comprehensive capability analysis of gpt-3 and gpt-3.5 series models,

    J. Ye, X. Chen, N. Xu, C. Zu, Z. Shao, S. Liu, Y . Cui, Z. Zhou, C. Gong, Y . Shen, et al., “A comprehensive capability analysis of gpt-3 and gpt-3.5 series models,” arXiv preprint arXiv:2303.10420, 2023

    2023 arXiv

  42. [50]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. , “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  43. [51]

    Deepseek-v3 technical report,

    A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan, et al., “Deepseek-v3 technical report,” arXiv preprint arXiv:2412.19437, 2024. 12

    2024 arXiv

  44. [52]

    A survey of reasoning with foundation models: Concepts, methodologies, and outlook,

    J. Sun, C. Zheng, E. Xie, Z. Liu, R. Chu, J. Qiu, J. Xu, M. Ding, H. Li, M. Geng, et al., “A survey of reasoning with foundation models: Concepts, methodologies, and outlook,” ACM Computing Surveys, 2023. A Appendix A.1 Task-wise Program Generation We showcase sample ST-progra...

    2023

  45. [53]

    • General understanding of spatio-temporal tasks such as analysis, anomaly detection, and forecasting

    Participant Selection: Evaluators were selected based on the following criteria to ensure technical competence: • Minimum of a Bachelor’s degree in Computing, Data Science, Statistics, Mathematics, or a related technical discipline. • General understanding of spatio-temporal t...

  46. [54]

    Each query was paired with two answers, one from STReason and one from a randomly selected baseline ensuring each baseline appeared an equal number of times

    Material Preparation: The study included 18 queries covering three task categories: Analysis, Anomaly Detection, and Prediction and Reasoning. Each query was paired with two answers, one from STReason and one from a randomly selected baseline ensuring each baseline appeared an...

  47. [55]

    They were also encouraged to provide open-ended feedback explaining their choices

    Evaluation Design: Participants were instructed to select the more effective answer based on clarity, completeness, reasoning, and overall helpfulness. They were also encouraged to provide open-ended feedback explaining their choices. Evaluator Background Human evaluation was ...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.