Pith. sign in

REVIEW 3 major objections 5 minor 5 cited by

No LLM reliably assembles multi-step time-series workflows

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-05 12:08 UTC pith:C7ZW6OBB

load-bearing objection Useful benchmark, but success thresholds are so lenient that the central capability claim is not cleanly supported—worth reviewing, not desk-rejecting. the 3 major comments →

arxiv 2509.01822 v1 pith:C7ZW6OBB submitted 2025-09-01 cs.LG cs.AI

When LLM Meets Time Series: Can LLMs Perform Multi-Step Time Series Reasoning and Inference

Wen Ye , Jinbo Liu , Defu Cao , Wei Yang , Yan Liu This is my paper
classification cs.LG cs.AI
keywords TSAIA benchmarktime series analysislarge language modelsmulti-step reasoningconstraint-aware forecastinganomaly detectioncausal discoveryfinancial decision-making
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces TSAIA, a benchmark meant to test whether large language models can act as general-purpose time series assistants, not just answer questions about trends but chain together forecasting, constraint checking, anomaly calibration, causal inference, and financial decisions. It contains 1,054 questions drawn from 33 real-world task formulations across energy, climate, finance, and healthcare. To pass, a model must produce executable code, follow operational constraints such as ramp-rate limits, incorporate supplied domain knowledge, and avoid trivial outputs. On this benchmark, eight current models all show the same pattern: they handle simple subtasks, but none reliably assembles the full workflow. The paper's conclusion is that current LLMs cannot yet serve as dependable time series inference agents and that domain-specific or hybrid methods are needed.

Core claim

The paper's central claim is that current large language models cannot reliably perform multi-step time series reasoning and inference when the task is posed as a complete analytical workflow. To test this, the authors build TSAIA, a benchmark of 33 task types and 1,054 questions derived from more than 20 application papers, spanning forecasting under operational constraints, anomaly detection with reference samples or known priors, causal discovery with domain knowledge, risk-return analysis, trading, and multiple-choice financial decisions. Each instance is generated dynamically from a dataset, a template, sampled parameters, and injected constraints, with ground truth either retrieved fro

What carries the argument

The carrying mechanism is the TSAIA question generator paired with task-specific success criteria. The generator runs five steps: task-type selection, data-source selection, context parameterization, complexity injection (operational constraints or domain knowledge), and ground-truth construction. The evaluator then checks structural correctness, checks constraint satisfaction and knowledge incorporation, and computes an inference-quality metric (MAPE, F1, absolute error, or trading metrics). Its work is to make heterogeneous workflows—forecasting, anomaly detection, causal discovery, financial analytics, trading, and multiple-choice decisions—comparable under one protocol while rejecting tr

Load-bearing premise

The load-bearing premise is that the benchmark's pass bars—MAPE below 1, F1 above 0, absolute error below 0.05, and a no-loss trading requirement—are fair proxies for real-world time-series competence; the paper sets these thresholds internally (Table 1) without external justification, and if they are not fair, the 'models cannot assemble workflows' conclusion would be an artifact of the chosen bars.

What would settle it

Run the same 1,054 TSAIA instances under the same protocol but with a model that explicitly invokes external time-series and calibration routines and is given more than six execution turns. If success rates on the reference-sample anomaly detection and financial trading groups rise above 90 percent, the paper's claim that current models cannot assemble these workflows would be shown to depend on the agent loop rather than on a fundamental reasoning limit.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the paper is right, no evaluated model can currently be trusted as a general-purpose time series assistant in energy, finance, or healthcare settings; deployment would require domain-specific guardrails.
  • Time-series LLM benchmarks should include multi-step workflows, injected operational constraints, and rejection of trivial outputs, rather than point forecasts or single QA items.
  • Code execution and feedback, while necessary, are not sufficient: models still fail workflow assembly, so future systems need explicit planning or symbolic components.
  • The dynamic generator means TSAIA can be extended with new datasets and task types, so the conclusion can be rechecked as models improve.
  • Task-specific success criteria allow different workflows to be compared under one protocol, giving concrete metric targets for future time-series agents.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the success thresholds are lenient (MAPE below 1, F1 above 0, absolute error below 0.05, no-loss trading), so observed failures are strong evidence of limitation, while passing would only certify meeting low bars, not real-world competence.
  • Beyond the paper: varying the thresholds and the six-turn execution cap would reveal how much of the reported gap is an artifact of the agent loop rather than of reasoning ability; this is a direct testable extension.
  • Beyond the paper: the code-execution protocol may reward verbose planning since more turns and more tokens correlate with higher success, meaning token-efficient models could be underestimated as assistants.
  • Beyond the paper: the same generator pattern could be applied to neighboring domains such as sensor-fault diagnosis, supply-chain demand planning, or clinical alarm prediction, where constraints and calibration are central.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces TSAIA, a benchmark for evaluating LLMs as time-series AI assistants. It contains 1054 question instances across 33 task formulations in four categories: predictive, diagnostic, analytical, and decision-making tasks. Task instances are generated programmatically from real-world datasets spanning energy, climate, finance, and healthcare, with task-specific success criteria and inference-quality metrics. The authors evaluate eight LLMs under a unified CodeAct agent protocol, allowing code generation and execution feedback. Their main finding is that current LLMs cannot reliably assemble multi-step time-series workflows, with common failure modes including trivial predictions, constraint violations, and execution errors. The benchmark and code are publicly released.

Significance. If the evaluation is valid, TSAIA would be a valuable resource: it is dynamic and extensible, covers a wider range of task types than existing temporal benchmarks, integrates execution feedback via CodeAct, and provides a systematic comparison of eight modern LLMs. The public release of the benchmark and code, and the analysis of failure modes across task categories, are concrete contributions. However, the central claim that LLMs 'cannot reliably perform multi-step time series reasoning' rests on Success Rate definitions in Table 1 whose thresholds are unusually permissive. Several reported 'successes' are indistinguishable from trivial baselines (all-positive anomaly detection, no-trade trading, persistence-like forecasts). This weakens the quantitative support for the paper's main conclusion and needs to be addressed before the benchmark can serve as a rigorous evaluation tool.

major comments (3)
  1. [§2.3, Table 1; §3.2, Tables 4–5] The 'non-trivial' thresholds in Table 1 are so permissive that Success Rate does not cleanly measure the intended capability. (i) Anomaly detection requires only F1>0, which an all-positive detector satisfies whenever any anomaly exists; Table 4 shows GPT-4o at 87% success on Energy Usage Anomaly with mean F1=0.08. (ii) Financial trading requires only 'no loss', which an all-cash/no-trade policy satisfies; Table 5 shows Llama-3.1 at 96% success with cumulative return 0.00 and maximum drawdown 0.00. (iii) Forecasting requires only MAPE<1, i.e., up to 100% error, far above the MAPEs actually reported in Table 3. Since Success Rate is the primary cross-task metric, the central claim is not cleanly supported. Please calibrate thresholds against simple baselines (persistence, all-anomaly, no-trade) and report baseline-relative success or full metric distributions.
  2. [§2.3 vs. §3.2, Figure 7] The text states that 'Trivial or degenerate outputs ... are flagged as failures', and Figure 7 lists 'Trivial Prediction/Inadequate Result' as a distinct failure category. However, the formal criteria in Table 1 count trivial outputs as successes whenever they satisfy the weak thresholds (e.g., F1>0, no loss, MAPE<1). This is an internal inconsistency, not merely an external calibration dispute. The paper should make the trivial-prediction rejection operational in the evaluator—for example, by explicitly excluding all-constant, all-anomaly, and no-trade outputs—and rerun all Success Rate numbers. As written, the benchmark counts as successes some outputs that the paper itself identifies as the dominant failure mode.
  3. [Appendix E; Table 4] The causal-discovery ground truth is generated synthetically using an LLM prompt that also defines the domain knowledge (the relation matrix) and the data-generation process. The reported success on 'Causal Discovery w/ Domain Knowledge' may therefore partly reflect a model recognizing patterns from LLM-generated data rather than performing causal discovery on real observational time series. This weakens the external validity of that task. Please validate causal discovery on an established causal benchmark or real dataset, or restrict the claims accordingly.
minor comments (5)
  1. [Abstract] Typo: 'temporal data application domain' should be 'temporal data application domains'. Also, 'a first attempt' appears twice in close proximity; consider rewording.
  2. [Appendix E] The heading reads 'Casual Discovery Data Generation Prompt'; 'Casual' should be 'Causal'.
  3. [Tables 3–5] No confidence intervals or significance tests are reported. Many success-rate cells are based on only 16–71 instances, so apparent differences between models may not be reliable. Reporting standard errors would strengthen the comparison.
  4. [§2.1, Reference [46]] The PSML dataset [46] is from the authors' research group, but this is not disclosed in the main text. Please add a statement of the potential conflict, as is common for self-cited datasets.
  5. [§2.4] The claim that TSAIA is 'first of its kind' is somewhat overstated given existing hybrid QA/analysis benchmarks such as MTBench and ChatTime, which are listed in Table 2. Qualifying the novelty would be more accurate.

Circularity Check

0 steps flagged

No material circularity: the evaluation is empirically grounded in external data and fixed success criteria; the only self-citation (PSML dataset) is not load-bearing.

full rationale

This paper is an empirical benchmark study rather than a derivation chain. Ground truths are obtained from held-out future values, anomaly labels, causal graphs, or backtests on external stock-price data (§2.3), and model outputs come from eight external LLMs executed through a CodeAct agent. No parameter is fitted to the benchmark and then renamed as a prediction, and no claimed result is defined by the model outputs themselves. The only author-overlap citation is the PSML dataset ([46], which includes Yan Liu as a co-author), used as one of several public data sources; the central claim about limitations in multi-step time series reasoning does not reduce to that dataset, so the self-citation is minor and non-load-bearing. I also weighed the reviewer concern about lenient success thresholds (e.g., F1>0 admitting near-trivial anomaly predictions, and 'no loss' admitting CR=0/MDD=0 no-trade behavior, which Figure 7 itself categorizes as 'trivial prediction/inadequate result'). This is a genuine threat to the external validity of the reported Success Rates, but it is not circularity: the thresholds are defined a priori and applied to independently computed metrics, so the successes/failures are not equivalent to the benchmark's construction. The paper's own characterization of trivial predictions as a dominant failure mode is an internal inconsistency in benchmark calibration, not a reduction of the conclusion to its inputs. Therefore no circular step is present; the low score reflects only the minor self-citation and the calibration caveat, not a circular derivation.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 0 invented entities

The benchmark relies on domain assumptions about task representativeness, metric validity, and the fairness of the CodeAct agent interface. The free parameters are the hand-chosen success thresholds in Table 1.

free parameters (4)
  • MAPE threshold for non-trivial forecasting = 1.0
    Table 1: prediction is non-trivial if MAPE < 1 (100% error). This is very lenient.
  • F1 threshold for anomaly detection = 0 (F1 > 0)
    Table 1: any nonzero F1 counts as success; a model flagging one true positive passes.
  • Absolute error threshold for financial analytics = 0.05
    Table 1: absolute error < 0.05 required.
  • Trading success criterion = no loss (cumulative return >= 0)
    Table 1: success if there is no loss in investment; market dependence not considered.
axioms (6)
  • domain assumption The 33 task formulations extracted from over 20 publications are representative of real-world multi-step time series analysis workflows.
    Section 2.1 describes manual curation by three graduate students; no systematic protocol ensures representativeness.
  • ad hoc to paper The task-specific success criteria (MAPE<1, F1>0, absolute error<0.05, no-loss trading) are meaningful measures of successful task completion.
    Table 1 sets these thresholds by hand; no external calibration or analysis justifies them.
  • domain assumption The CodeAct agent framework with Python execution is a fair and appropriate interface for all LLMs, so execution failures reflect model capability.
    Section 3.1 uses CodeAct for all models; different models may have varying familiarity with the instruction format.
  • standard math Financial formulas (Sharpe ratio, VaR, Jensen's alpha, beta, Calmar, Sortino) and backtesting on held-out stock data yield correct ground truth for financial tasks.
    Section 2.3 invokes standard definitions; the paper assumes these are implemented correctly.
  • domain assumption The LLM-generated synthetic causal data (Appendix E) preserves the known relation matrix sufficiently to evaluate causal discovery.
    Appendix E uses an LLM prompt to generate data from a relation matrix; noise and generation artifacts might distort the ground truth.
  • domain assumption Public datasets (ERA5, MIT-BIH, PSML, Yahoo Finance) are correctly preprocessed and split for task generation.
    Section 2.1 and Appendix A list sources; preprocessing details are not fully specified.

pith-pipeline@v1.4.0-alltime-deepseek-medium · 20680 in / 12125 out tokens · 106350 ms · 2026-08-05T12:08:03.168715+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of When LLM Meets Time Series: Can LLMs Perform Multi-Step Time Series Reasoning and Inference." pith.science (2026). https://pith.science/paper/C7ZW6OBB

@misc{pith2026250901822,
  author       = {Pith},
  title        = {Pith review of: When LLM Meets Time Series: Can LLMs Perform Multi-Step Time Series Reasoning and Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C7ZW6OBB}},
  note         = {Machine review of arXiv:2509.01822}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

The rapid advancement of Large Language Models (LLMs) has sparked growing interest in their application to time series analysis tasks. However, their ability to perform complex reasoning over temporal data in real-world application domains remains underexplored. To move toward this goal, a first step is to establish a rigorous benchmark dataset for evaluation. In this work, we introduce the TSAIA Benchmark, a first attempt to evaluate LLMs as time-series AI assistants. To ensure both scientific rigor and practical relevance, we surveyed over 20 academic publications and identified 33 real-world task formulations. The benchmark encompasses a broad spectrum of challenges, ranging from constraint-aware forecasting to anomaly detection with threshold calibration: tasks that require compositional reasoning and multi-step time series analysis. The question generator is designed to be dynamic and extensible, supporting continuous expansion as new datasets or task types are introduced. Given the heterogeneous nature of the tasks, we adopt task-specific success criteria and tailored inference-quality metrics to ensure meaningful evaluation for each task. We apply this benchmark to assess eight state-of-the-art LLMs under a unified evaluation protocol. Our analysis reveals limitations in current models' ability to assemble complex time series analysis workflows, underscoring the need for specialized methodologies for domain-specific adaptation. Our benchmark is available at https://huggingface.co/datasets/Melady/TSAIA, and the code is available at https://github.com/USC-Melady/TSAIA.

Figures

Figures reproduced from arXiv: 2509.01822 by Defu Cao, Jinbo Liu, Wei Yang, Wen Ye, Yan Liu.

Figure 1
Figure 1. Figure 1: Categorization of Tasks in TSAIA. Lighter colors denote tasks with less difficulty and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The proposed pipeline for multi-step time series inference task instance generation and [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Example Task Instance containing the task instruction, accompanied serialized dataset, and [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Average Success Rate of Models with respect to the Average Number of Tokens Used. We report model performance across four pri￾mary task categories: predictive, diagnostic, an￾alytical. and decision-making tasks. Predictive tasks involve forecasting under real-world oper￾ational constraints. As shown in table 3, models generally perform well on simpler constraints (maximum or minimum load) but struggle with… view at source ↗
Figure 5
Figure 5. Figure 5: Model Performance on Decision-Making and Analysis-Interpretation Tasks in Multiple [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Average Number of Turns Models Take to Reach a Solution Grouped by Difficulty Level [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Case Study on GPT-4o Error Distribution across Tasks Grouped by Difficulty Level [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Case Study on Claude 3.5 Error Distribution across Tasks Grouped by Difficulty Level [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Case Study on Qwen Error Distribution across Tasks Grouped by Difficulty Level [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Case Study on Codestral Error Distribution across Tasks Grouped by Difficulty Level [PITH_FULL_IMAGE:figures/full_fig_p016_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Case Study on Llama Error Distribution across Tasks Grouped by Difficulty Level [PITH_FULL_IMAGE:figures/full_fig_p016_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Case Study on Gemini Error Distribution across Tasks Grouped by Difficulty Level [PITH_FULL_IMAGE:figures/full_fig_p016_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Case Study on Deepseek-chat Error Distribution across Tasks Grouped by Difficulty Level [PITH_FULL_IMAGE:figures/full_fig_p017_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Case Study on Deepseek-R Error Distribution across Tasks Grouped by Difficulty Level [PITH_FULL_IMAGE:figures/full_fig_p017_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. TimeClaw: A Time-Series AI Agent with Exploratory Execution Learning

    cs.AI 2026-05 unverdicted novelty 7.0

    TimeClaw is an exploratory execution learning system that turns multiple valid tool-use paths into hierarchical distilled experience for improved time-series reasoning without test-time adaptation.

  2. LLaTiSA: Towards Difficulty-Stratified Time Series Reasoning from Visual Perception to Semantics

    cs.AI 2026-04 unverdicted novelty 7.0

    LLaTiSA is a vision-language model trained on a new 83k-sample hierarchical time series reasoning dataset that shows superior performance and out-of-distribution generalization on stratified TSR tasks.

  3. HEARTS: Benchmarking LLM Reasoning on Health Time Series

    cs.LG 2026-02 conditional novelty 7.0

    A 110-task benchmark across 20 health signal modalities shows current LLMs underperform specialized models and depend on simple heuristics rather than robust time-series reasoning.

  4. Harnessing Generalist Agents for Contextualized Time Series

    cs.AI 2026-06 unverdicted novelty 6.0

    TimeClaw is a framework that augments LLM agents with temporal tools, capability evolution, and episodic memory to enable contextualized time series reasoning, with reported gains on benchmarks across energy, finance,...

  5. Speaking Numbers to LLMs: Multi-Wavelet Number Embeddings for Time Series Forecasting

    cs.CL 2026-06 unverdicted novelty 5.0

    TempoWave maps scalar observations to multi-wavelet multi-scale digit embeddings that override standard LLM tokens and improve forecasting performance on five context-enriched benchmarks to a new state-of-the-art.

Reference graph

Works this paper leans on

75 extracted references · 44 canonical work pages · cited by 5 Pith papers · 1 internal anchor

  1. [1]

    Unified language model pre-training for natural language under- standing and generation

    Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. Unified language model pre-training for natural language under- standing and generation. Advances in neural information processing systems, 32, 2019

    work page 2019

  2. [2]

    A survey on large language models for code generation

    Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. A survey on large language models for code generation. arXiv preprint arXiv:2406.00515, 2024

    Pith/arXiv arXiv 2024

  3. [3]

    Galactica: A large language model for science

    Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. Galactica: A large language model for science. arXiv preprint arXiv:2211.09085, 2022

    Pith/arXiv arXiv 2022

  4. [4]

    Energy time series forecasting based on pattern sequence similarity

    Francisco Martinez Alvarez, Alicia Troncoso, Jose C Riquelme, and Jesus S Aguilar Ruiz. Energy time series forecasting based on pattern sequence similarity. IEEE Transactions on Knowledge and Data Engineering, 23(8):1230–1243, 2010

    work page 2010

  5. [5]

    Financial time series forecasting with deep learning: A systematic literature review: 2005–2019

    Omer Berat Sezer, Mehmet Ugur Gudelek, and Ahmet Murat Ozbayoglu. Financial time series forecasting with deep learning: A systematic literature review: 2005–2019. Applied soft computing, 90:106181, 2020

    work page 2005

  6. [6]

    Trend analysis of climate time series: A review of methods

    Manfred Mudelsee. Trend analysis of climate time series: A review of methods. Earth-science reviews, 190:310–322, 2019

    work page 2019

  7. [7]

    Time series analysis of variables associated with daily mean emergency department length of stay

    Niels K Rathlev, John Chessare, Jonathan Olshaker, Dan Obendorfer, Supriya D Mehta, Todd Rothenhaus, Steven Crespo, Brendan Magauran, Kathy Davidson, Richard Shemin, et al. Time series analysis of variables associated with daily mean emergency department length of stay. Annals of emergency medicine, 49(3):265–271, 2007

    work page 2007

  8. [8]

    Learning semantic context from normal samples for unsupervised anomaly detection

    Xudong Yan, Huaidong Zhang, Xuemiao Xu, Xiaowei Hu, and Pheng-Ann Heng. Learning semantic context from normal samples for unsupervised anomaly detection. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 3110–3118, 2021

    2021

  9. [9]

    Real-time load variability control using energy storage system for demand-side management in south korea

    Kyo Beom Han, Jaesung Jung, and Byung O Kang. Real-time load variability control using energy storage system for demand-side management in south korea. Energies, 14(19):6292, 2021

    2021

  10. [10]

    Complexity-based prompting for multi-step reasoning

    Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark, and Tushar Khot. Complexity-based prompting for multi-step reasoning. arXiv preprint arXiv:2210.00720, 2022

    Pith/arXiv arXiv 2022

  11. [11]

    The future is different: Large pre- trained language models fail in prediction tasks

    Kostadin Cvejoski, Ramsés J Sánchez, and César Ojeda. The future is different: Large pre- trained language models fail in prediction tasks. arXiv preprint arXiv:2211.00384, 2022

    Pith/arXiv arXiv 2022

  12. [12]

    Domain knowledge is all you need: A field deployment of llm-powered test case generation in fintech domain

    Zhiyi Xue, Liangguo Li, Senyue Tian, Xiaohong Chen, Pingping Li, Liangyu Chen, Tingting Jiang, and Min Zhang. Domain knowledge is all you need: A field deployment of llm-powered test case generation in fintech domain. InProceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings, pages 314–315, 2024

    work page 2024

  13. [13]

    220kv city power grid maximum loadability determination with static security-constraints

    Ke-qiu W ANG, Si-guang SUN, Hong-yi W ANG, Chang-xu JIANG, and Zhao-xia JING. 220kv city power grid maximum loadability determination with static security-constraints. Power, Energy Engineering and Management (PEEM2016), page 1, 2016

    2016

  14. [14]

    Language models still struggle to zero-shot reason about time series

    Mike A Merrill, Mingtian Tan, Vinayak Gupta, Thomas Hartvigsen, and Tim Althoff. Language models still struggle to zero-shot reason about time series. In EMNLP (Findings), 2024

    work page 2024

  15. [15]

    Tsi-bench: Benchmarking time series imputation

    Wenjie Du, Jun Wang, Linglong Qian, Yiyuan Yang, Zina Ibrahim, Fanxing Liu, Zepu Wang, Haoxin Liu, Zhiyuan Zhao, Yingjie Zhou, et al. Tsi-bench: Benchmarking time series imputation. arXiv preprint arXiv:2406.12747, 2024

    Pith/arXiv arXiv 2024

  16. [16]

    Informer: Beyond efficient transformer for long sequence time-series forecasting

    Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106–11115, 2021. 10

    work page 2021

  17. [17]

    Tram: Benchmarking temporal reasoning for large language models

    Yuqing Wang and Yun Zhao. Tram: Benchmarking temporal reasoning for large language models. In Findings of the Association for Computational Linguistics ACL 2024, pages 6389– 6415, 2024

    work page 2024

  18. [18]

    Understanding and patching compositional reasoning in llms

    Zhaoyi Li, Gangwei Jiang, Hong Xie, Linqi Song, Defu Lian, and Ying Wei. Understanding and patching compositional reasoning in llms. arXiv preprint arXiv:2402.14328, 2024

    Pith/arXiv arXiv 2024

  19. [19]

    Commonsense reasoning and commonsense knowledge in artificial intelligence

    Ernest Davis and Gary Marcus. Commonsense reasoning and commonsense knowledge in artificial intelligence. Communications of the ACM, 58(9):92–103, 2015

    2015

  20. [20]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    Pith/arXiv arXiv 2024

  21. [21]

    Qwen2.5-max: Exploring the intelligence of large-scale moe model, January 2025

    Qwen Team. Qwen2.5-max: Exploring the intelligence of large-scale moe model, January 2025. Accessed: 2025-05-14

    work page 2025

  22. [22]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    Pith/arXiv arXiv 2024

  23. [23]

    Claude 3.5 sonnet, June 2024

    Anthropic. Claude 3.5 sonnet, June 2024. Accessed: 2025-05-14

    work page 2024

  24. [24]

    Deepseek-v3 technical report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024

    Pith/arXiv arXiv 2024

  25. [25]

    Introducing gemini 2.0: Our new ai model for the agentic era, December

    Google DeepMind. Introducing gemini 2.0: Our new ai model for the agentic era, December

    work page

  26. [26]

    Codestral, May 2024

    Mistral AI. Codestral, May 2024. Accessed: 2025-05-14

    work page 2024

  27. [27]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    Pith/arXiv arXiv 2025

  28. [28]

    Executable code actions elicit better llm agents

    Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Executable code actions elicit better llm agents. In Forty-first International Conference on Machine Learning, 2024

    2024

  29. [29]

    Agentscope: A flexible yet robust multi-agent platform

    Dawei Gao, Zitao Li, Xuchen Pan, Weirui Kuang, Zhijian Ma, Bingchen Qian, Fei Wei, Wenhao Zhang, Yuexiang Xie, Daoyuan Chen, Liuyi Yao, Hongyi Peng, Ze Yu Zhang, Lin Zhu, Chen Cheng, Hongzhu Shi, Yaliang Li, Bolin Ding, and Jingren Zhou. Agentscope: A flexible yet robust multi-agent platform. CoRR, abs/2402.14034, 2024

    Pith/arXiv arXiv 2024

  30. [30]

    Hansel: Output length controlling framework for large language models

    Seoha Song, Junhyun Lee, and Hyeonmok Ko. Hansel: Output length controlling framework for large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 25146–25154, 2025

    work page 2025

  31. [31]

    The first step is the hardest: Pitfalls of representing and tokenizing temporal data for large language models

    Dimitris Spathis and Fahim Kawsar. The first step is the hardest: Pitfalls of representing and tokenizing temporal data for large language models. Journal of the American Medical Informatics Association, 31(9):2151–2158, 2024

    2024

  32. [32]

    Short-term scheduling of electric power systems under minimum load conditions

    Claudia Greif, Raymond B Johnson, Chao an Li, Alva J Svoboda, and K Andrijeski Uemura. Short-term scheduling of electric power systems under minimum load conditions. IEEE transactions on power systems, 14(1):280–286, 1999

    1999

  33. [33]

    Application of nowcasting to reduce the impact of irradiance ramps on pv power plants

    Jonas Schaible, Bijan Nouri, Lars Höpken, Tim Kotzab, Matthias Loevenich, Niklas Blum, Annette Hammer, Jonas Stührenberg, Klaus Jäger, Christiane Becker, et al. Application of nowcasting to reduce the impact of irradiance ramps on pv power plants. EPJ Photovoltaics, 15:15, 2024. 11

    work page 2024

  34. [34]

    Spyros I Gkavanoudis, Kyriaki-Nefeli D Malamaki, Eleftherios O Kontis, Charis S Demoulias, Aditya Shekhar, Umer Mushtaq, and Sagar Bandi Venu. Provision of ramp-rate limitation as ancillary service from distribution to transmission system: Definitions and methodologies for control and sizing of central battery energy storage system. Journal of Modern Powe...

    work page 2023

  35. [35]

    Unsupervised anomaly detection of industrial building energy consumption

    Yi Song, Sennan Kuang, Junling Huang, and Da Zhang. Unsupervised anomaly detection of industrial building energy consumption. Energy and Built Environment, 2024

    work page 2024

  36. [36]

    Lstm-based vae-gan for time-series anomaly detection

    Zijian Niu, Ke Yu, and Xiaofei Wu. Lstm-based vae-gan for time-series anomaly detection. Sensors, 20(13):3738, 2020

    work page 2020

  37. [37]

    Deep Learning Techniques in Extreme Weather Events: A Review

    Shikha Verma, Kuldeep Srivastava, Akhilesh Tiwari, and Shekhar Verma. Deep learning techniques in extreme weather events: A review. arXiv preprint arXiv:2308.10995, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  38. [38]

    Parkca: Causal inference with partially known causes

    Raquel Aoki and Martin Ester. Parkca: Causal inference with partially known causes. In BIO- COMPUTING 2021: Proceedings of the Pacific Symposium, pages 196–207. World Scientific, 2020

    2021

  39. [39]

    Maximum drawdown as predictor of mutual fund performance and flows

    Timothy Riley and Qing Yan. Maximum drawdown as predictor of mutual fund performance and flows. Financial Analysts Journal, 78(4):59–76, 2022

    work page 2022

  40. [40]

    Bitcoin, litecoin, and the euro: an annualized volatility analysis

    Cynthia Miglietti, Zdenka Kubosova, and Nicole Skulanova. Bitcoin, litecoin, and the euro: an annualized volatility analysis. Studies in Economics and Finance, 37(2):229–242, 2020

    work page 2020

  41. [41]

    Picking buy-sell signals: A practitioner’s perspective on key technical indicators for selected indian firms

    Talwar Shalini, Pranav Shah, and Shah Utkarsh. Picking buy-sell signals: A practitioner’s perspective on key technical indicators for selected indian firms. Studies in Business and Economics, 14(3):205–219, 2019

    work page 2019

  42. [42]

    Technical analysis to determine buying and selling signal in stock trade

    Rommy Pramudya. Technical analysis to determine buying and selling signal in stock trade. International Journal of Finance & Banking Studies (2147-4486), 9(1):58–67, 2020

    work page 2020

  43. [43]

    Common metrics for performance evaluation: Overview of popular performance measurement ratios

    Oliver Steinki and Ziad Mohammad. Common metrics for performance evaluation: Overview of popular performance measurement ratios. Available at SSRN 2662054, 2015

    work page 2015

  44. [44]

    The information ratio and performance

    Francis Gupta, Robertus Prajogi, and Eric Stubbs. The information ratio and performance. Journal of Portfolio Management, 26(1):33, 1999

    work page 1999

  45. [45]

    Focussing in reasoning and decision making

    Paolo Legrenzi, Vittorio Girotto, and Philip N Johnson-Laird. Focussing in reasoning and decision making. Cognition, 49(1-2):37–66, 1993

    work page 1993

  46. [46]

    Psml: a multi-scale time-series dataset for machine learning in decarbonized energy grids

    Xiangtian Zheng, Nan Xu, Loc Trinh, Dongqi Wu, Tong Huang, S Sivaranjani, Yan Liu, and Le Xie. Psml: a multi-scale time-series dataset for machine learning in decarbonized energy grids. arXiv preprint arXiv:2110.06324, 2021

    Pith/arXiv arXiv 2021

  47. [47]

    https://www.kaggle.com/competitions/ energy-anomaly-detection/data , 2022

    Large-scale energy anomaly detection (lead). https://www.kaggle.com/competitions/ energy-anomaly-detection/data , 2022. Kaggle Competition

    work page 2022

  48. [48]

    The era5 global reanalysis

    Hans Hersbach, Bill Bell, Paul Berrisford, Shoji Hirahara, András Horányi, Joaquín Muñoz- Sabater, Julien Nicolas, Carole Peubey, Raluca Radu, Dinand Schepers, et al. The era5 global reanalysis. Quarterly journal of the royal meteorological society, 146(730):1999–2049, 2020

    work page 1999

  49. [49]

    Physiobank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals

    Ary L Goldberger, Luis AN Amaral, Leon Glass, Jeffrey M Hausdorff, Plamen Ch Ivanov, Roger G Mark, Joseph E Mietus, George B Moody, Chung-Kang Peng, and H Eugene Stanley. Physiobank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals. circulation, 101(23):e215–e220, 2000

    2000

  50. [50]

    Yahoo Finance, n.d

    Yahoo Finance. Yahoo Finance, n.d. Accessed: 2025-05-14

    work page 2025

  51. [51]

    Minimum-variance portfolio composition

    Roger Clarke, Harindra De Silva, and Steven Thorley. Minimum-variance portfolio composition. Journal of Portfolio Management, 37(2):31, 2011

    work page 2011

  52. [52]

    Diversification returns and asset contributions

    David G Booth and Eugene F Fama. Diversification returns and asset contributions. Financial Analysts Journal, 48(3):26–32, 1992. 12

    work page 1992

  53. [53]

    Maximum drawdown

    Malik Magdon-Ismail and Amir F Atiya. Maximum drawdown. Risk Magazine, 17(10):99–102, 2004

    2004

  54. [54]

    The sharpe ratio

    William F Sharpe. The sharpe ratio. Journal of portfolio management, 21(1):49–58, 1994

    work page 1994

  55. [55]

    An overview of value at risk

    Darrell Duffie and Jun Pan. An overview of value at risk. Journal of derivatives, 4(3):7–49, 1997

    work page 1997

  56. [56]

    Risk adjusted performance attribution

    Jeffrey D Fisher and Joseph DAlessandro. Risk adjusted performance attribution. Available at SSRN 3232392, 2018

    work page 2018

  57. [57]

    Test of time: A benchmark for evaluating llms on temporal reasoning

    Bahare Fatemi, Mehran Kazemi, Anton Tsitsulin, Karishma Malkan, Jinyeong Yim, John Palowitch, Sungyong Seo, Jonathan Halcrow, and Bryan Perozzi. Test of time: A benchmark for evaluating llms on temporal reasoning. arXiv preprint arXiv:2406.09170, 2024

    Pith/arXiv arXiv 2024

  58. [58]

    The elephant in the room: Towards a reliable time-series anomaly detection benchmark

    Qinghua Liu and John Paparrizos. The elephant in the room: Towards a reliable time-series anomaly detection benchmark. In The Thirty-eight Conference on Neural Information Process- ing Systems Datasets and Benchmarks Track, 2024

    work page 2024

  59. [59]

    Gift-eval: A benchmark for general time series forecasting model evaluation

    Taha Aksu, Gerald Woo, Juncheng Liu, Xu Liu, Chenghao Liu, Silvio Savarese, Caiming Xiong, and Doyen Sahoo. Gift-eval: A benchmark for general time series forecasting model evaluation. In NeurIPS Workshop on Time Series in the Age of Large Models, 2024

    work page 2024

  60. [60]

    Jensen, Zhenli Sheng, and Bin Yang

    Xiangfei Qiu, Jilin Hu, Lekui Zhou, Xingjian Wu, Junyang Du, Buang Zhang, Chenjuan Guo, Aoying Zhou, Christian S. Jensen, Zhenli Sheng, and Bin Yang. Tfb: Towards comprehensive and fair benchmarking of time series forecasting methods. Proc. VLDB Endow., 17(9):2363– 2377, 2024

    work page 2024

  61. [61]

    Sasanur, Megha Sharma, Jiaming Cui, Qingsong Wen, Chao Zhang, and B

    Haoxin Liu, Shangqing Xu, Zhiyuan Zhao, Lingkai Kong, Harshavardhan Kamarthi, Aditya B. Sasanur, Megha Sharma, Jiaming Cui, Qingsong Wen, Chao Zhang, and B. Aditya Prakash. Time-MMD: Multi-domain multimodal dataset for time series analysis. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024

    work page 2024

  62. [62]

    Context is key: A benchmark for forecasting with essential textual information

    Andrew Robert Williams, Arjun Ashok, Étienne Marcotte, Valentina Zantedeschi, Jithendaraa Subramanian, Roland Riachi, James Requeima, Alexandre Lacoste, Irina Rish, Nicolas Cha- pados, et al. Context is key: A benchmark for forecasting with essential textual information. arXiv preprint arXiv:2410.18959, 2024

    Pith/arXiv arXiv 2024

  63. [63]

    Beyond trend and periodicity: Guiding time series forecasting with textual cues

    Zhijian Xu, Yuxuan Bian, Jianyuan Zhong, Xiangyu Wen, and Qiang Xu. Beyond trend and periodicity: Guiding time series forecasting with textual cues. arXiv preprint arXiv:2405.13522, 2024

    Pith/arXiv arXiv 2024

  64. [64]

    Mtbench: A multimodal time series benchmark for temporal reasoning and question answering

    Jialin Chen, Aosong Feng, Ziyu Zhao, Juan Garza, Gaukhar Nurbek, Cheng Qin, Ali Maatouk, Leandros Tassiulas, Yifeng Gao, and Rex Ying. Mtbench: A multimodal time series benchmark for temporal reasoning and question answering. arXiv preprint arXiv:2503.16858, 2025

    arXiv 2025

  65. [65]

    Chattime: A unified multimodal time series foundation model bridging numerical and textual data

    Chengsen Wang, Qi Qi, Jingyu Wang, Haifeng Sun, Zirui Zhuang, Jinming Wu, Lei Zhang, and Jianxin Liao. Chattime: A unified multimodal time series foundation model bridging numerical and textual data. In AAAI Conference on Artificial Intelligence, 2025

    work page 2025

  66. [66]

    Mistral ai, 2024

    Mistral AI. Mistral ai, 2024

    work page 2024

  67. [67]

    Faith and fate: Limits of transformers on compositionality

    Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Sean Welleck, Peter West, Chandra Bhagavatula, Ronan Le Bras, et al. Faith and fate: Limits of transformers on compositionality. Advances in Neural Information Processing Systems , 36:70293–70332, 2023

    work page 2023

  68. [68]

    */data.csv

    Chen Ling, Xujiang Zhao, Jiaying Lu, Chengyuan Deng, Can Zheng, Junxiang Wang, Tanmoy Chowdhury, Yun Li, Hejie Cui, Xuchao Zhang, et al. Domain specialization as the key to make large language models disruptive: A comprehensive survey. arXiv preprint arXiv:2305.18703, 2023. 13 A Dataset Statistics Dataset Number of Data Files Avg Total Timestamps Number o...

    Pith/arXiv arXiv 2023

  69. [70]

    data correlation: the multi variable should be correlated, sample: which A first influence B, then B have influence on C or D, there should be some time delay, as the influence on other staff needs time. 21

    work page

  70. [71]

    data trend: there should be some trend in the data, like the data is increasing or decreasing

  71. [72]

    data: seasonality there should be some seasonality in the data, like the data is periodic

  72. [73]

    data noise: the noise should be added to the data, as the real world data is not perfect

  73. [74]

    CoT Sample: Q: Approximate Relation Ratio: 0.5 Relation Matrix: A B C D A 1 1 0 1 B 0 1 0 1 C 0 1 1 1 D 0 0 0 1 • A influences B and D, and itself

    data background: the data should have some real world background, you should first think about different real world data, and provide a description for the variable and time series data, then generate the data using the code. CoT Sample: Q: Approximate Relation Ratio: 0.5 Relation Matrix: A B C D A 1 1 0 1 B 0 1 0 1 C 0 1 1 1 D 0 0 0 1 • A influences B an...

  74. [75]

    After a delay, this starts influencing sales

    Advertising (A): The level of advertising spend directly impacts the sales of each store. After a delay, this starts influencing sales. 2. Sales (B): The sales numbers for each store are influenced by both the advertising and local seasonal events. 3. Economic Factors (C): Broader economic trends, like GDP growth or unemployment rates, also impact sales. ...

    work page

  75. [2024]

    Accessed: 2025-05-14

    work page 2025