Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

TimeMaster: Training Time-Series Multimodal LLMs to Reason via Reinforcement Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read TimeMaster claims a 3B multimodal model can learn expert time-series reasoning from ~1k GPT-4o examples per task plus token-level GRPO, reaching 75.29% average accuracy on TIMERBED.

desk verdict Worth a serious look: the RL recipe is coherent and cheap, but the SOTA claim currently rests on an unstated train/test split for the SFT/RL data. read the letter →

arxiv 2506.13705 v1 pith:4AZXZF4L submitted 2025-06-16 cs.LG cs.AI

classification cs.LGcs.AI MSC 68T0768T50
keywords time-seriesreasoningmultimodallargelanguagemodelsreinforcementlearningGRPOstructuredoutputcompositerewardLLM-as-a-Judgeclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TimeMaster claims that reinforcement learning can turn a small multimodal language model into a time-series reasoner: given a line plot of a signal and a task prompt, the model must explain its analysis, output a class label, and optionally give domain advice, all in a fixed tagged format. Training uses a two-stage pipeline — supervised fine-tuning on roughly a thousand GPT-4o-generated examples per task, then token-level GRPO driven by a composite reward that checks format adherence, label correctness, and open-ended suggestion quality. On the six-task TIMERBED benchmark, the 3B-parameter model reaches 75.29% average accuracy, surpassing classical time-series models by 14.6% and few-shot GPT-4o by 7.3%. The paper's broader claim is that reward-driven RL, not massive curated datasets, is the scalable path to giving MLLMs genuine temporal understanding.

What carries the argument

The load-bearing mechanism is the coupling of a rigid three-tag output grammar with a three-term reward. The model must emit <think>…</think> (chain-of-thought over the plotted signal), <class>…</class> (the predicted label), and <extension>…</extension> (optional actionable advice); the format reward is a regex check on that structure, the hard reward is exact label match, and the soft reward delegates open-ended extension quality to a GPT-4o judge scoring four dimensions on [0,1]. GRPO (group size G=5) computes group-normalized advantages so tokens in better-scoring rollouts are pushed up without a separate value network, while the SFT warm-up supplies the temporal priors that keep RL exploration anchored. The same mechanism yields both the accuracy jump and the qualitative shift in reasoning behavior.

What would settle it

Re-run the full SFT+GRPO pipeline with strict split discipline: partition each TIMERBED dataset into train and test folds before sampling the GPT-4o warm-up examples and before collecting RL rollouts, holding the test fold out entirely. If the headline accuracy drops substantially below 75.29% or falls behind the VL-Time few-shot baseline, the result is an artifact of training/test overlap. A second check is to measure agreement between TimeMaster's <think> reasoning and its <class> label on the test set; the paper's own failure cases admit reasoning–label mismatches, so a high mismatch rate would undercut the 'expert-like reasoning' claim even if accuracy holds.

Watch

Extended reading notes

Core claim

The central claim is that a time-series multimodal LLM can acquire expert-style reasoning without large curated datasets: instead of hundreds of thousands of paired examples, TimeMaster warms a 3B vision-language model on about one thousand GPT-4o-generated reasoning/classification samples per task, then optimizes a composite reward with GRPO. The reward has three parts: a regex-based format reward that enforces the ordered <think>, <class>, and <extension> tags; a hard reward that gives 1 only on an exact label match; and a soft reward, active when the extension is enabled, in which a GPT-4o judge scores extension quality on specificity, appropriateness, relevance, and depth. On TIMERBED — whale-call detection, satellite lightning events, ECG arrhythmia, EMG neuromuscular diagnosis, activity recognition, and device-type inference — the full SFT+RL model scores 75.29% average accuracy, beating the strongest classical baselines and the VL-Time few-shot GPT-4o prompting pipeline, with qualitative evidence that RL-trained reasoning incorporates uncertainty, contrastive elimination of rival classes, and multi-scale pattern synthesis absent from the SFT-only model.

Load-bearing premise

The model is trained on data sampled from the same TIMERBED datasets it is then tested on, and the paper never states that the roughly 1,000 GPT-4o-generated SFT samples and the RL rollouts are confined to a training split disjoint from the test split; if any of them come from test instances, the reported 75.29% average accuracy is inflated and the state-of-the-art claim collapses.

Editorial extensions

If this is right

  • If the 75.29% result holds, roughly 1k warm-up examples per task substitutes for the ~200k curated pairs used by prior SFT-only time-series QA models, changing the data economics of building temporal reasoners.
  • The format-hard-soft reward stack is task-agnostic: the same recipe should transfer to forecasting-with-explanation, anomaly diagnosis, or time-series QA without architectural changes.
  • Because outputs are parseable tags, downstream systems can automatically audit structure, grade extension quality, and reject format-violating answers — a practical requirement for clinical or industrial deployment.
  • The paper's ablations show format and hard rewards are complementary (39% to 85% to 100% accuracy on EMG as rewards are added), implying reward composition rather than model scale explains most of the gain.
  • A 3B model beating frontier models at prompting time-series plots suggests the binding constraint in time-series MLLMs is task-specific temporal priors, not parameter count.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The soft reward's judge is the same model family that generated the SFT data, so the reward may optimize for what GPT-4o finds convincing rather than clinically true; swapping in an independent judge or human-rated extensions would reveal whether this bias exists.
  • Because inputs are rendered plots, the model's reasoning could be tracking rendering artifacts such as axis ranges, colors, or resolution rather than the underlying waveform; perturbing axes or occluding plot regions would test perceptual faithfulness.
  • The SFT-then-GRPO recipe with a composite reward is a general template for other visual reasoning domains — medical imaging, spectral analysis, industrial waveforms — where a small model plus structured rewards may rival large prompting-only models.
  • The train/test provenance question is resolvable by inspection or re-run; any downstream comparison should first verify that GPT-4o sampling and GRPO rollouts used only training-fold instances.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces TimeMaster, a two-stage training method that combines supervised fine-tuning (SFT) with Group Relative Policy Optimization (GRPO) to make a 3B-parameter multimodal LLM (Qwen2.5-VL-3B-Instruct) perform structured reasoning over visualized time-series inputs. The model is trained to output three XML-tagged segments—<think>, <class>, and <extension>—and is optimized with a composite reward comprising format, hard (classification accuracy), and soft (LLM-as-a-Judge extension quality) components. On the TIMERBED benchmark, the paper reports an average accuracy of 75.29%, claiming state-of-the-art results that surpass classical time-series models by 14.6% and few-shot GPT-4o/VL-Time by 7.3%. The paper also includes ablations of reward components and group size, plus qualitative case studies of the model's reasoning behavior.

Significance. If the empirical results are verified, the paper makes a valuable contribution: it suggests that a relatively small amount of SFT warm-up (about 1k examples per task) combined with token-level RL can elicit interpretable, expert-like reasoning from a 3B multimodal model, at a fraction of the data cost of prior approaches such as Time-MQA. The structured-output design and the decomposition into format, hard, and soft rewards are simple and potentially reusable across time-series reasoning tasks. The claimed gains over strong baselines are substantial, and the paper provides reproducible-looking hyperparameter settings. However, the central empirical claim is currently contingent on an unverified dataset-provenance assumption (train/test disjointness), and several secondary comparisons lack variance information. These issues do not affect the soundness of the RL formulation, but they do affect the credibility of the SOTA claim.

major comments (4)
  1. [Section 4.1 (Table 1), Appendices G.2 and I, Table 4] The central claim of 75.29% average accuracy rests on the unstated assumption that the TIMERBED samples used for GPT-4o SFT generation and GRPO rollout contexts are disjoint from the evaluation split. The paper never specifies a train/test split for these training samples, and for the small datasets (EMG: 205 samples, CTU: 500, TEE: 143, Table 4) the described class-balanced, ground-truth-filtered collection procedure in Appendix I could plausibly draw from the same pool used for evaluation. If any overlap exists, the hard reward in Eq. (3) and the GRPO objective in Eq. (7) directly optimize the reported test accuracy, inflating the SOTA claim. The authors must specify the per-dataset split used for SFT and RL, confirm disjointness from the evaluation set, and release the code and data needed to verify this.
  2. [Section 4.1 (Table 1), Table 3] Baseline accuracies are reported without standard deviations or repeated-seed runs, while Table 3 provides variance only for TimeMaster. Several reported gaps are small relative to plausible seed variation (e.g., TEE: 68.49 for TimeMaster vs. 64.29 for VL-Time few-shot; EMG: 100.00 for TimeMaster vs. 91.67 for VL-Time few-shot). The comparison should either include error bars or multiple seeds for the strongest baselines, or explicitly state that those numbers are single-run values inherited from prior work; otherwise the robustness of the margin over baselines is unclear.
  3. [Appendix G.2 and Table 2] The main-result configuration sets (λfmt, λhard, λsoft) = (0.1, 0.9, 0), so the reported 75.29% accuracy does not exercise the soft reward at all. The paper's abstract, introduction, and Takeaway III (Section 4.2) nevertheless present soft reward as a core component that improves extension quality; the only quantitative evidence for this is the EMG-only ablation in Table 2 (soft reward score improving from 0.45 to 0.72). This claim should be either demonstrated on at least one additional dataset or explicitly scoped as an EMG-only result, and the main text should clarify that the SOTA accuracy is achieved without the soft-reward term.
  4. [Appendix D.2 (Figure 5)] The group-size ablation on TEE and CTU reports accuracy curves without stating whether the evaluation set is disjoint from the RL training contexts. If the same provenance issue described in the first major comment applies here, the observed monotonic improvement with increasing G could reflect memorization of overlapping examples rather than better advantage estimation. This ablation should be re-run or re-reported on a confirmed disjoint split before it is used to justify the default choice of G=5.
minor comments (5)
  1. [Appendix G.1] The phrase 'Following the methodology of VL-Following Time [17]' appears to be a typo; it should read 'VL-Time [17]'.
  2. [Figure 14] In the VL-Time output, the text contains the string 'NoWhale/class>' which is missing the opening '<'; this is likely a typesetting error and should be corrected.
  3. [Table 4 vs. Appendix G.2] The datasets are listed in inconsistent orders across Table 4 and Appendix G.2; please unify the ordering (e.g., by reasoning type) for readability.
  4. [References] Several references contain spacing artifacts such as '[ 17]' and '[ 27]'; the bibliography should be formatted consistently.
  5. [Table 6] The hyperparameter table lists 'Lmax' but the text refers to 'max sequence length'; please rename the parameter for clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TimeMaster's RL training optimizes ground-truth accuracy directly, and the TIMERBED train/test provenance concern is a correctness issue, not a circular reduction.

full rationale

TimeMaster's derivation chain is self-contained with respect to the circularity patterns checked. The reported 75.29% is an accuracy on TIMERBED, and the training signal used to produce the model is the hard reward rhard = I[hat-c = c*] (Eq. 3), which is a direct function of ground-truth labels rather than of the evaluation numbers; RL optimizes this reward through the GRPO objective in Eq. (7). The SFT warm-up data are generated by GPT-4o and filtered by matching ground-truth labels (Appendix I), so no parameter is fitted to a 'prediction' that is then reported as independent; the evaluation metric is the same accuracy that the reward encourages, which is ordinary supervised/RL training rather than circularity. The soft reward uses GPT-4o as judge while GPT-4o also generated SFT data, but soft reward is disabled in the main experiments (lambda_soft = 0 in Appendix G.2), so it is not load-bearing for the headline accuracy. Self-citations ([2], [8]) appear in related-work context and do not carry any load-bearing premise; no uniqueness theorem or ansatz is imported from the authors' prior work. The one substantive risk, that SFT/RL contexts are sampled from TIMERBED without a stated disjoint train/test split (Appendix G.2, I), is a dataset-provenance and correctness concern, not a circularity reduction: even if leakage occurred, the reported numbers would be inflated by test-set fitting, not derived from the method's equations by construction. Since no specific Eq. X = Eq. Y or fitted-parameter-renamed-as-prediction step can be exhibited, the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the GRPO optimization framework, the line-plot input representation, the correctness of TIMERBED labels, and the unstated disjointness of train and test splits. The reward weights and per-task training budgets are hand-chosen hyperparameters rather than fitted scientific constants. No invented entities are introduced.

free parameters (4)
  • Reward weights (lambda_format, lambda_hard, lambda_soft) = (0.1, 0.9, 0) for main results; (0.1, 0.9, 1) for extension case studies
    Hand-chosen trade-offs; soft reward is disabled in the reported accuracy, so the main claim is not sensitive to it, but weights shape training.
  • GRPO group size G = 5
    Selected after an ablation on TEE and CTU (Appendix D.1); the paper notes larger G helps but costs compute.
  • Per-dataset RL epochs = 40 (RCW, ECG, HAR); 100 (EMG, CTU); 300 (TEE)
    Tuned per task without a described validation protocol; possible selection on test performance.
  • SFT data collection scale and filter = About 1,000 GPT-4o samples per dataset, temperature 1.0, only label-correct samples retained
    Data-generation hyperparameters; the correctness filter changes the SFT distribution and is a modeling choice, not a fitted scientific constant.
assumptions (4)
  • standard math GRPO clipped surrogate objective and KL regularization are valid optimizers for this policy
    Assumed from DeepSeekMath/DeepSeek-R1 and used in Section 3.4, Eq. 7.
  • domain assumption A line-plot image of a time series preserves enough temporal structure for classification and reasoning
    Input strategy inherited from VL-Time in Section 3.1; if plots lose information, all results are bounded.
  • domain assumption TIMERBED labels are correct and are the correct supervision for expert-like reasoning
    Section 4 treats ground truth as the sole hard-reward signal; label noise would weaken the SOTA claim.
  • ad hoc to paper The SFT and RL training samples from TIMERBED are drawn from a split disjoint from the evaluation split
    Never stated in Appendix G.2 or I; the accuracy claim collapses if this is false.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TimeMaster: Training Time-Series Multimodal LLMs to Reason via Reinforcement Learning." pith.science (2026). https://pith.science/paper/4AZXZF4L

@misc{pith2026250613705,
  author       = {Pith},
  title        = {Pith review of: TimeMaster: Training Time-Series Multimodal LLMs to Reason via Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4AZXZF4L}},
  note         = {Machine review of arXiv:2506.13705}
}
read the original abstract

Time-series reasoning remains a significant challenge in multimodal large language models (MLLMs) due to the dynamic temporal patterns, ambiguous semantics, and lack of temporal priors. In this work, we introduce TimeMaster, a reinforcement learning (RL)-based method that enables time-series MLLMs to perform structured, interpretable reasoning directly over visualized time-series inputs and task prompts. TimeMaster adopts a three-part structured output format, reasoning, classification, and domain-specific extension, and is optimized via a composite reward function that aligns format adherence, prediction accuracy, and open-ended insight quality. The model is trained using a two-stage pipeline: we first apply supervised fine-tuning (SFT) to establish a good initialization, followed by Group Relative Policy Optimization (GRPO) at the token level to enable stable and targeted reward-driven improvement in time-series reasoning. We evaluate TimeMaster on the TimerBed benchmark across six real-world classification tasks based on Qwen2.5-VL-3B-Instruct. TimeMaster achieves state-of-the-art performance, outperforming both classical time-series models and few-shot GPT-4o by over 14.6% and 7.3% performance gain, respectively. Notably, TimeMaster goes beyond time-series classification: it also exhibits expert-like reasoning behavior, generates context-aware explanations, and delivers domain-aligned insights. Our results highlight that reward-driven RL can be a scalable and promising path toward integrating temporal understanding into time-series MLLMs.

Figures

Figures reproduced from arXiv: 2506.13705 by the authors.

Figure 1
Figure 1. Example of TimeMaster. (a) A plot of a raw EMG signal (1500 time points), labeled [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed TimeMaster. Format reward r fmt . The format reward enforces strict adherence to the predefined output structure. A regex-based parser automatically checks if the generated response contains correctly nested and ordered <think>, <class>, and <extension> tags, with non-empty reasoning and classification spans. If the format is fully correct, a binary reward of r fmt(y) = 1 is assigned; otherw… view at source ↗
Figure 3
Figure 3. Comparison of various models’ reasoning on three time-series tasks. Shown are key [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Extension output of TimeMaster via <extension> tags on a neuropathic EMG signal. Takeaway III: Beyond Reasoning, RL Enables Flexible Extensions such as Insightful, Action￾able Suggestion Generation. As shown in Fig￾ure 4, TimeMaster not only excels at core tasks such a…
Figure 5
Figure 5. Figure 5: Effect of generation number G on accuracy (%) during GRPO training for TEE and CTU. Parameter analysis. We conducted an ablation study on the parameter G, which controls the num￾ber of generations per input during GRPO training [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Comparison of reasoning outputs on a standing-labeled HAR test instance across three [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Comparison of model reasoning on an ECG instance labeled as “other cardiac rhythms,” [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Comparison of reasoning outputs on a neuropathy-labeled EMG test instance across three [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: TimeMaster’s suggestions via <extension> tags on EMG signals for two cases: healthy (left) and myopathy (right). < t h i n k > To classify the physical activity, we need to examine the acceleration values along the x, y, and z axes. The plot shows three distinct curves…
Figure 10
Figure 10. Figure 10: Comparison of reasoning outputs on a walking-labeled HAR test instance under three [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: Comparison of reasoning outputs on a desktop-labeled CTU test instance under three [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: Comparison of reasoning outputs on a noise-labeled ECG test instance under three models: [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]
Figure 13
Figure 13. Figure 13: Comparison of reasoning outputs on a Gradual Intra-Cloud-labeled TEE test instance [PITH_FULL_IMAGE:figures/full_fig_p020_13.png]
Figure 14
Figure 14. Figure 14: Comparison of reasoning outputs on a RightWhale-labeled RCW test instance under three [PITH_FULL_IMAGE:figures/full_fig_p021_14.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Survey of Reasoning and Agentic Systems in Time Series with Large Language Models

    cs.AI 2025-09 conditional novelty 5.0 of 10

    The authors organize LLM-based time series reasoning into three exclusive topologies (direct, chain, branch) crossed with four objectives, and use them to label 125 papers, benchmarks, and resources.

Reference graph

Works this paper leans on

85 extracted references · 38 canonical work pages · cited by 1 Pith paper

  1. [1]

    Deep learning in human activity recognition with wearable sensors: A review on advances.Sensors, 22(4):1476, 2022

    Shibo Zhang, Yaxuan Li, Shen Zhang, Farzad Shahabi, Stephen Xia, Yu Deng, and Nabil Alshurafa. Deep learning in human activity recognition with wearable sensors: A review on advances.Sensors, 22(4):1476, 2022

  2. [2]

    Diverse intra-and inter-domain activity style fusion for cross-person generalization in activity recognition

    Junru Zhang, Lang Feng, Zhidan Liu, Yuhan Wu, Yang He, Yabo Dong, and Duanqing Xu. Diverse intra-and inter-domain activity style fusion for cross-person generalization in activity recognition. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4213–4222, 2024

  3. [3]

    Sensor alignment for multivariate time-series unsupervised domain adaptation

    Yucheng Wang, Yuecong Xu, Jianfei Yang, Zhenghua Chen, Min Wu, Xiaoli Li, and Lihua Xie. Sensor alignment for multivariate time-series unsupervised domain adaptation. InProceedings of the AAAI conference on artificial intelligence, volume 37, pages 10253–10261, 2023

  4. [4]

    Conditional contrastive domain generalization for fault diagnosis.IEEE Transactions on Instrumentation and Measurement, 71:1–12, 2022

    Mohamed Ragab, Zhenghua Chen, Wenyu Zhang, Emadeldeen Eldele, Min Wu, Chee-Keong Kwoh, and Xiaoli Li. Conditional contrastive domain generalization for fault diagnosis.IEEE Transactions on Instrumentation and Measurement, 71:1–12, 2022

  5. [5]

    Understanding electricity-theft behavior via multi-source data

    Wenjie Hu, Yang Yang, Jianbo Wang, Xuanwen Huang, and Ziqiang Cheng. Understanding electricity-theft behavior via multi-source data. InProceedings of The Web Conference 2020, pages 2264–2274, 2020

  6. [6]

    Tactis: Transformer-attentional copulas for time series

    Alexandre Drouin, Étienne Marcotte, and Nicolas Chapados. Tactis: Transformer-attentional copulas for time series. InInternational Conference on Machine Learning, pages 5447–5493. PMLR, 2022

  7. [7]

    Tslanet: Rethinking transformers for time series representation learning

    Emadeldeen Eldele, Mohamed Ragab, Zhenghua Chen, Min Wu, and Xiaoli Li. Tslanet: Rethinking transformers for time series representation learning. InInternational Conference on Machine Learning, pages 12409–12428. PMLR, 2024

  8. [8]

    Adacket: Adaptive convolutional kernel transform for multivariate time series classification

    Junru Zhang, Lang Feng, Haowen Zhang, Yuhan Wu, and Yabo Dong. Adacket: Adaptive convolutional kernel transform for multivariate time series classification. InJoint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 189–204. Springer, 2023

Show all 85 references
  1. [9]

    Times- net: Temporal 2d-variation modeling for general time series analysis.arXiv preprint arXiv:2210.02186, 2022

    Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Times- net: Temporal 2d-variation modeling for general time series analysis.arXiv preprint arXiv:2210.02186, 2022

  2. [10]

    Recurrent neural networks for time series classification

    Michael Hüsken and Peter Stagge. Recurrent neural networks for time series classification. Neurocomputing, 50:223–235, 2003

  3. [11]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

  4. [12]

    Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

  5. [13]

    Qwen technical report.arXiv preprint arXiv:2309.16609, 2023

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report.arXiv preprint arXiv:2309.16609, 2023

  6. [14]

    Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

  7. [15]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022. 10

  8. [16]

    Position: Empowering time series reasoning with multimodal llms.arXiv preprint arXiv:2502.01477, 2025

    Yaxuan Kong, Yiyuan Yang, Shiyu Wang, Chenghao Liu, Yuxuan Liang, Ming Jin, Stefan Zohren, Dan Pei, Yan Liu, and Qingsong Wen. Position: Empowering time series reasoning with multimodal llms.arXiv preprint arXiv:2502.01477, 2025

  9. [17]

    A picture is worth a thousand numbers: Enabling llms reason about time series via visualization.arXiv preprint arXiv:2411.06018, 2024

    Haoxin Liu, Chenghao Liu, and B Aditya Prakash. A picture is worth a thousand numbers: Enabling llms reason about time series via visualization.arXiv preprint arXiv:2411.06018, 2024

  10. [18]

    Explainable multi-modal time series prediction with llm-in-the-loop.arXiv preprint arXiv:2503.01013, 2025

    Yushan Jiang, Wenchao Yu, Geon Lee, Dongjin Song, Kijung Shin, Wei Cheng, Yanchi Liu, and Haifeng Chen. Explainable multi-modal time series prediction with llm-in-the-loop.arXiv preprint arXiv:2503.01013, 2025

  11. [19]

    Language models still struggle to zero-shot reason about time series.arXiv preprint arXiv:2404.11757, 2024

    Mike A Merrill, Mingtian Tan, Vinayak Gupta, Tom Hartvigsen, and Tim Althoff. Language models still struggle to zero-shot reason about time series.arXiv preprint arXiv:2404.11757, 2024

  12. [20]

    Gpt-4o, 2024

    OpenAI. Gpt-4o, 2024. Accessed: 2025-04-21

  13. [21]

    Chatts: Aligning time series with llms via synthetic data for enhanced understanding and reasoning.arXiv preprint arXiv:2412.03104, 2024

    Zhe Xie, Zeyan Li, Xiao He, Longlong Xu, Xidao Wen, Tieying Zhang, Jianjun Chen, Rui Shi, and Dan Pei. Chatts: Aligning time series with llms via synthetic data for enhanced understanding and reasoning.arXiv preprint arXiv:2412.03104, 2024

  14. [22]

    Timecap: Learning to contextualize, augment, and predict time series events with large language model agents

    Geon Lee, Wenchao Yu, Kijung Shin, Wei Cheng, and Haifeng Chen. Timecap: Learning to contextualize, augment, and predict time series events with large language model agents. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 18082–18090, 2025

  15. [23]

    From news to forecast: Integrating event analysis in llm-based time series forecasting with reflection.Advances in Neural Information Processing Systems, 37:58118–58153, 2024

    Xinlei Wang, Maike Feng, Jing Qiu, Jinjin Gu, and Junhua Zhao. From news to forecast: Integrating event analysis in llm-based time series forecasting with reflection.Advances in Neural Information Processing Systems, 37:58118–58153, 2024

  16. [24]

    Beyond trend and periodicity: Guiding time series forecasting with textual cues.arXiv preprint arXiv:2405.13522, 2024

    Zhijian Xu, Yuxuan Bian, Jianyuan Zhong, Xiangyu Wen, and Qiang Xu. Beyond trend and periodicity: Guiding time series forecasting with textual cues.arXiv preprint arXiv:2405.13522, 2024

  17. [25]

    Time-mmd: A new multi-domain multimodal dataset for time series analysis.arXiv preprint arXiv:2406.08627, 2024

    Haoxin Liu, Shangqing Xu, Zhiyuan Zhao, Lingkai Kong, Harshavardhan Kamarthi, Aditya B Sasanur, Megha Sharma, Jiaming Cui, Qingsong Wen, Chao Zhang, et al. Time-mmd: A new multi-domain multimodal dataset for time series analysis.arXiv preprint arXiv:2406.08627, 2024

  18. [26]

    Multi-modal forecaster: Jointly predicting time series and textual data.arXiv preprint arXiv:2411.06735, 2024

    Kai Kim, Howard Tsai, Rajat Sen, Abhimanyu Das, Zihao Zhou, Abhishek Tanpure, Mathew Luo, and Rose Yu. Multi-modal forecaster: Jointly predicting time series and textual data.arXiv preprint arXiv:2411.06735, 2024

  19. [27]

    Time-mqa: Time series multi-task question answering with context enhancement.arXiv preprint arXiv:2503.01875, 2025

    Yaxuan Kong, Yiyuan Yang, Yoontae Hwang, Wenjie Du, Stefan Zohren, Zhangyang Wang, Ming Jin, and Qingsong Wen. Time-mqa: Time series multi-task question answering with context enhancement.arXiv preprint arXiv:2503.01875, 2025

  20. [28]

    Gpt4mts: Prompt-based large language model for multimodal time-series forecasting

    Furong Jia, Kevin Wang, Yixiang Zheng, Defu Cao, and Yan Liu. Gpt4mts: Prompt-based large language model for multimodal time-series forecasting. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 23343–23351, 2024

  21. [29]

    MIT press, 2018

    Richard S Sutton and Andrew G Barto.Reinforcement learning: An introduction. MIT press, 2018

  22. [30]

    Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in Neural Information Processing Systems, 36:46595–46623, 2023

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in Neural Information Processing Systems, 36:46595–46623, 2023

  23. [31]

    Time-llm: Time series forecasting by reprogramming large language models.arXiv preprint arXiv:2310.01728, 2023

    Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, et al. Time-llm: Time series forecasting by reprogramming large language models.arXiv preprint arXiv:2310.01728, 2023. 11

  24. [32]

    Autotimes: Au- toregressive time series forecasters via large language models.Advances in Neural Information Processing Systems, 37:122154–122184, 2024

    Yong Liu, Guo Qin, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Autotimes: Au- toregressive time series forecasters via large language models.Advances in Neural Information Processing Systems, 37:122154–122184, 2024

  25. [33]

    Calf: Aligning llms for time series forecasting via cross-modal fine-tuning

    Peiyuan Liu, Hang Guo, Tao Dai, Naiqi Li, Jigang Bao, Xudong Ren, Yong Jiang, and Shu-Tao Xia. Calf: Aligning llms for time series forecasting via cross-modal fine-tuning. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 18915–18923, 2025

  26. [34]

    Test: Text prototype aligned embedding to activate llm’s ability for time series

    Chenxi Sun, Hongyan Li, Yaliang Li, and Shenda Hong. Test: Text prototype aligned embedding to activate llm’s ability for time series. InThe Twelfth International Conference on Learning Representations

  27. [35]

    Language models are unsupervised multitask learners.OpenAI blog, 1(8):9, 2019

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners.OpenAI blog, 1(8):9, 2019

  28. [36]

    How can time series analysis benefit from multiple modalities? a survey and outlook.arXiv preprint arXiv:2503.11835, 2025

    Haoxin Liu, Harshavardhan Kamarthi, Zhiyuan Zhao, Shangqing Xu, Shiyu Wang, Qingsong Wen, Tom Hartvigsen, Fei Wang, and B Aditya Prakash. How can time series analysis benefit from multiple modalities? a survey and outlook.arXiv preprint arXiv:2503.11835, 2025

  29. [37]

    Timecma: Towards llm-empowered multivariate time series forecasting via cross-modality alignment

    Chenxi Liu, Qianxiong Xu, Hao Miao, Sun Yang, Lingzheng Zhang, Cheng Long, Ziyue Li, and Rui Zhao. Timecma: Towards llm-empowered multivariate time series forecasting via cross-modality alignment. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, page...

  30. [38]

    Meit: Multi-modal electrocardiogram instruction tuning on large language models for report generation.arXiv preprint arXiv:2403.04945, 2024

    Zhongwei Wan, Che Liu, Xin Wang, Chaofan Tao, Hui Shen, Zhenwu Peng, Jie Fu, Rossella Arcucci, Huaxiu Yao, and Mi Zhang. Meit: Multi-modal electrocardiogram instruction tuning on large language models for report generation.arXiv preprint arXiv:2403.04945, 2024

  31. [39]

    Multi-modal deep learning for credit rating prediction using text and numerical data streams.Applied Soft Computing, page 112771, 2025

    Mahsa Tavakoli, Rohitash Chandra, Fengrui Tian, and Cristián Bravo. Multi-modal deep learning for credit rating prediction using text and numerical data streams.Applied Soft Computing, page 112771, 2025

  32. [40]

    Terra: A multimodal spatio-temporal dataset spanning the earth.Advances in Neural Information Processing Systems, 37:66329– 66356, 2024

    Wei Chen, Xixuan Hao, Yuankai Wu, and Yuxuan Liang. Terra: A multimodal spatio-temporal dataset spanning the earth.Advances in Neural Information Processing Systems, 37:66329– 66356, 2024

  33. [41]

    Bjtt: A large-scale multimodal dataset for traffic prediction.IEEE Transactions on Intelligent Transportation Systems, 2024

    Chengyang Zhang, Yong Zhang, Qitan Shao, Jiangtao Feng, Bo Li, Yisheng Lv, Xinglin Piao, and Baocai Yin. Bjtt: A large-scale multimodal dataset for traffic prediction.IEEE Transactions on Intelligent Transportation Systems, 2024

  34. [42]

    Event traffic forecasting with sparse multimodal data

    Xiao Han, Zhenduo Zhang, Yiling Wu, Xinfeng Zhang, and Zhe Wu. Event traffic forecasting with sparse multimodal data. InProceedings of the 32nd ACM International Conference on Multimedia, pages 8855–8864, 2024

  35. [43]

    Evaluating system 1 vs

    Haoxin Liu, Zhiyuan Zhao, Shiduo Li, and B Aditya Prakash. Evaluating system 1 vs. 2 reasoning approaches for zero-shot time-series forecasting: A benchmark and insights.arXiv preprint arXiv:2503.01895, 2025

  36. [44]

    Fine-tuning language models from human preferences

    Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019

  37. [45]

    Learning to summarize with human feedback

    Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea V oss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in neural information processing systems, 33:3008–3021, 2020

  38. [46]

    Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:27730–27744, 2022

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:27730...

  39. [47]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024. 12

  40. [48]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms.arXiv preprint arXiv:2501.12599, 2025

  41. [49]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

  42. [50]

    Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025

  43. [51]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

  44. [52]

    itransformer: Inverted transformers are effective for time series forecasting.arXiv preprint arXiv:2310.06625, 2023

    Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting.arXiv preprint arXiv:2310.06625, 2023

  45. [53]

    A time series is worth 64 words: Long-term forecasting with transformers.arXiv preprint arXiv:2211.14730, 2022

    Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers.arXiv preprint arXiv:2211.14730, 2022

  46. [54]

    Beyond numbers: A survey of time series analysis in the era of multimodal llms.Authorea Preprints, 2025

    Xiongxiao Xu, Yue Zhao, S Yu Philip, and Kai Shu. Beyond numbers: A survey of time series analysis in the era of multimodal llms.Authorea Preprints, 2025

  47. [55]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017

  48. [56]

    Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting.Advances in neural information processing systems, 34:22419–22430, 2021

    Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting.Advances in neural information processing systems, 34:22419–22430, 2021

  49. [57]

    Informer: Beyond efficient transformer for long sequence time-series forecasting

    Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106–11115, 2021

  50. [58]

    Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting

    Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. InInternational conference on machine learning, pages 27268–27286. PMLR, 2022

  51. [59]

    Are transformers effective for time series forecasting? InProceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121–11128, 2023

    Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? InProceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121–11128, 2023

  52. [60]

    STANDING

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025. 13 A Limitations Our work still has some limitations. We have only provided an initial ...

  53. [61]

    The x -axis should reflect forward motion , while the y and z axes can show lateral and vertical changes

    WALKING: walking involves significant changes in acceleration due to the forward and side movements . The x -axis should reflect forward motion , while the y and z axes can show lateral and vertical changes . Observing the trends in the chart , we see small fluctuations across...

  54. [62]

    The fluctuations would be larger and more pronounced in z -axis data due to the changes in vertical motion

    WALKING_UPSTAIRS/DOWNSTAIRS: Both activities involve rhythmic changes typically seen in walking , but they include vertical motion . The fluctuations would be larger and more pronounced in z -axis data due to the changes in vertical motion

  55. [63]

    SITTING/LAYING: These activities show very low variability , with minimal changes in all axes due to the lack of forward movement

  56. [64]

    LAYING” based on vague cues like “relatively small movements

    STANDING: This involves very low -paced changes due to stability and lack of dynamic activity . Typical accelerometer data for standing will show minuscule fluctuations and low variability on all axes . In the provided data , the acceleration values for x , y, and z axes range...

  57. [65]

    This is typical in neuropathy due to reinnervation and the presence of motor units with abnormal recruitment patterns

    Waveform Morphology : The waveform is polyphasic , meaning it has multiple peaks . This is typical in neuropathy due to reinnervation and the presence of motor units with abnormal recruitment patterns

  58. [66]

    The polyphasic nature of the waveform is indicative of reinnervation, where motor units are recruited in a different manner than in a healthy state

    Temporal Patterns : The waveform shows a consistent pattern of sharp onset and brief duration , which is characteristic of neuropathy . The polyphasic nature of the waveform is indicative of reinnervation, where motor units are recruited in a different manner than in a healthy...

  59. [67]

    </think> <class>Neuropathy</class> < t h i n k>1

    Conclusion: The waveform's characteristics—polyphasic morphology, increased amplitude, and long duration —are consistent with neuropathy. </think> <class>Neuropathy</class> < t h i n k>1. Amplitude Analysis : The waveform shows a range of amplitudes , with peaks reaching appro...

  60. [68]

    The waveform is polyphasic, meaning it has multiple peaks and troughs within the waveform

    Morphology Analysis : The waveform has sharp onset and brief duration , indicating a rapid rise and fall of the electrical activity. The waveform is polyphasic, meaning it has multiple peaks and troughs within the waveform

  61. [69]

    Temporal Patterns : The waveform shows consistent amplitude and morphology , with no indication of variability or changes in waveform morphology over time

  62. [70]

    - Myopathy: Typically shows small amplitude and short duration , indicating a loss or dysfunction of muscle fibers

    Comparison with Conditions : - Healthy: Typically shows normal recruitment and morphology of motor unit potentials, sharp onset , brief duration , and consistent amplitude. - Myopathy: Typically shows small amplitude and short duration , indicating a loss or dysfunction of mus...

  63. [71]

    </think> <class>Neuropathy</class> <think > 1

    Conclusion: The waveform shows sharp onset and brief duration, consistent amplitude , and polyphasic morphology, which aligns with the characteristics of neuropathy. </think> <class>Neuropathy</class> <think > 1. Amplitude Analysis: The waveform shows a range of amplitudes fro...

  64. [72]

    The waveform morphology is consistent with normal recruitment and morphology of motor unit potentials

    Morphology and Temporal Patterns : The waveform appears to have sharp onset and brief duration, which are typical of motor unit potentials . The waveform morphology is consistent with normal recruitment and morphology of motor unit potentials

  65. [73]

    WALKING_UPSTAIRS

    Conclusion: The waveform shows sharp onset and brief duration, consistent with normal recruitment and morphology of motor unit potentials . The amplitude range is consistent with normal motor unit potentials. </think> <class>Healthy</class> Figure 8: Comparison of reasoning ou...

  66. [74]

    walking up stairs

    Laying: Laying involves minimal acceleration , with all axes showing low values . Given the plot , the x and y axes show moderate values , while the z -axis shows significant values. This suggests that the activity involves significant vertical movement, which is characteristi...

  67. [75]

    atrial fibrillation

    Pattern Analysis : The pattern shows a sustained high energy usage, which aligns with desktop usage . Desktops often have consistent power usage due to their larger components and higher power requirements . 4. Conclusion: The sustained high energy usage pattern suggests the d...

  68. [76]

    IR Negative

    Approximate Value Ranges : The minimum value is approximately -1, and the maximum value is around 3. 4. Conclusion: The signal shows a gradual ramp -up followed by fluctuations, matching the characteristics of a Gradual Intra - Cloud event. </think> <class>Gradual Intra-Cloud<...

  69. [77]

    NOWHALE,

    Frequency Consideration: Right whale calls would manifest as regular up -calls with distinct peaks and valleys over approximately one second . 3. Signal Density : The waveform 's variations do not exhibit clear periodicity that matches the expected 1-second duration of right w...

  70. [78]

    Specificity – Is the extension clearly tailored to the predicted class or context, avoiding generic language?

  71. [79]

    Appropriateness – Is the recommendation suitable given the prediction and reasoning?

  72. [80]

    Relevance – Does the extension logically follow from the preceding reasoning and predicted class?

  73. [81]

    be careful

    Depth – Does the extension demonstrate domain knowledge, including detailed actions or next-step considerations? Instructions: - Penalize vague or generic outputs (e.g., "be careful") unless well-justified. - Reward informative, actionable, and context-aware suggestions. - Ret...

  74. [82]

    Since the sampling rate is 2kHz, any frequency components within this range should be detectable

    **Frequency Range**: Right whale up-calls typically range from 60Hz to 250Hz. Since the sampling rate is 2kHz, any frequency components within this range should be detectable

  75. [83]

    Given the 2-second duration of the waveform, any call should be visible if it exists

    **Duration**: Right whale calls usually last around 1 second. Given the 2-second duration of the waveform, any call should be visible if it exists

  76. [84]

    This would likely appear as a consistent pattern or peak within the correct frequency range over the duration of the call

    **Signal Characteristics**: The waveform should show a distinct modulation corresponding to the up-call frequency. This would likely appear as a consistent pattern or peak within the correct frequency range over the duration of the call

  77. [85]

    ] Generated Reasoning Sample (HAR) [

    **Noise Levels**: Low-frequency noise from other activities may obscure the call, making it less visible. In the provided waveform: - The waveform appears mostly consistent without distinct peaks that would stand out as a right whale up-call. - The signal amplitude ranges with...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.