Pith. sign in

REVIEW 5 major objections 6 minor 2 cited by

Text Reinforcement for Multimodal Time Series Forecasting

T0 review · 5 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Rewriting text with a task-guided LLM lowers multimodal forecasting error

desk verdict A new idea with an overclaimed headline: reward-driven LLM text augmentation helps a bit, but test-set hyperparameter selection and an unstated reward split keep the SOTA result from being credible as reported. read the letter →

arxiv 2509.00687 v1 pith:QYMUOP6W submitted 2025-08-31 cs.CL

Chen Su , Yuanhe Tian , Yan Song , Yongdong Zhang This is my paper
classification cs.CL
keywords multimodaltimeseriesforecastingtextreinforcementlargelanguagemodelslearningdirectpreferenceoptimizationdataaugmentationTime-MMDcross-modalfusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that multimodal time series forecasting fails not because text is unhelpful but because real-world text is often missing, stale, or misaligned with the numbers. It proposes TeR-TSF, a pipeline in which an LLM rewrites the available text into a 'reinforced' summary guided by the forecasting task itself. The LLM is trained with direct preference optimization, where candidate rewrites are scored by how much they lower the forecaster's mean squared error and how well they use task-relevant vocabulary. On the Time-MMD benchmark across eight domains, the method reports the lowest average error and best-or-second-best results per domain, beating raw-text multimodal models and a diffusion baseline that downweights text. The central message is that improving text quality, rather than just fusing text more cleverly, is what unlocks the value of the textual modality.

What carries the argument

The load-bearing mechanism is the reward-guided DPO loop around the text reinforcement model. A pre-trained LLM (Qwen3-1.7B) generates candidate reinforced texts from serialized time series, descriptive statistics, and raw text. Each candidate is scored by a reward generator with two terms: reward 1 is the negative MSE between the forecaster's output and ground truth, and reward 2 is the normalized frequency of task-relevant keywords. The highest- and lowest-scoring candidates become a preference pair for direct preference optimization, updating the LLM via LoRA and closing a self-improving loop over four rounds. The downstream multimodal forecaster, built from PatchTST time-series encoding

What would settle it

Inspect the released implementation to see whether the ground-truth values in Eq. 6 come from training, validation, or test windows; then rerun the DPO loop with the prediction-accuracy reward computed only on training/validation labels. If the test results remain essentially unchanged, the central claim survives; if they degrade sharply, the reported test numbers were inflated by label leakage.

Watch

Extended reading notes

Core claim

TeR-TSF claims that multimodal time series forecasting can be improved by generating reinforced text from the historical time series and any available raw text, then feeding that reinforced text into a multimodal forecaster instead of the original text. The text reinforcement model is an LLM that receives serialized numeric values, statistical summaries, metadata, the original text, and a task prompt; the reinforced text is fused with the time series through cross-attention to produce forecasts. The LLM is optimized by reinforcement learning: a dual reward combines prediction accuracy (negative mean squared error between forecast and ground truth) with task relevance (normalized count of pre

Load-bearing premise

The reported gains depend on the prediction-accuracy reward being computed without using test-set labels, and on the alternating LLM/forecaster retraining loop producing a stable signal; the paper never states which data split feeds that reward.

Editorial extensions

If this is right

  • Missing or misaligned text no longer needs to be repaired by hand: the method can generate task-aligned text that compensates for absent or noisy auxiliary reports.
  • Reinforcement learning with a downstream forecasting reward can optimize LLM-generated data augmentation without human annotations, since the reward comes from prediction error and keyword coverage.
  • Adding the task-relevance keyword reward on top of the accuracy reward further lowers error, suggesting that lexically precise, human-readable text contributes to forecasting gains beyond raw accuracy feedback.
  • The method's per-domain best-or-second-best results imply that text reinforcement transfers across diverse domains with different frequencies, text volumes, and noise levels.
  • The finding that a smaller 1.7B LLM outperforms larger ones indicates that model size is not the limiting factor; task-aligned training matters more than raw capacity for this augmentation role.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same reward-guided text reinforcement loop could be adapted to other multimodal prediction tasks, such as classification, anomaly detection, or nowcasting, wherever auxiliary text is sparse or misaligned with numeric signals.
  • The keyword-relevance reward is a coarse proxy for task relevance; replacing or augmenting it with a learned relevance scorer could sharpen the preference signal, though the paper does not test this.
  • Because k=2 (only two candidates per round) is optimal and larger k degrades performance, the method is sensitive to preference-pair quality; a margin-aware or rank-based DPO variant might make the loop more robust to candidate diversity.
  • The iterative alternation between LLM refinement and forecaster retraining resembles self-training; its stability likely depends on reward calibration, which future work could probe by monitoring reward variance across rounds.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes TeR-TSF, a framework that uses a Qwen3-1.7B LLM as a text reinforcement model. Given historical time series and optional raw text, TeR generates "reinforced text" that is fed together with the time series into a PatchTST+LLM multimodal forecast model. The LLM is trained with a DPO-style loop in which k candidate texts are generated per round, scored by a weighted reward combining negative MSE of the downstream forecaster (r1, Eq. 6) and keyword relevance (r2, Eq. 7), and the best/worst candidates form preference pairs. After m rounds, the final TeR is used at test time. Experiments on Time-MMD across eight domains compare against TSF-only, TFHTS, Time-LLM, MM-TSF, MCD-TSF and ablations, claiming consistent lowest/second-lowest errors and best average MSE/MAE.

Significance. If the results held under a clean evaluation protocol, the idea of optimizing generated text via downstream forecasting reward is a useful and underexplored direction in multimodal time series forecasting. The code is released, which supports reproducibility. However, the current support is weakened by test-set hyperparameter selection, an incorrectly specified DPO loss, unspecified reward components, and a per-domain claim contradicted by the paper's own table. These issues affect the validity of the SOTA claim.

major comments (5)
  1. [§5.3, Fig. 3; §5.2, Fig. 2] The RL rounds m are selected "on the test set of Energy domain" (Fig. 3 caption), and k is selected on the Energy domain (Fig. 2) before being fixed for all subsequent experiments. The reported averages in Tables 2–3 are then computed on the same test splits. This is test-set hyperparameter selection, which biases the reported numbers optimistically and undermines the out-of-sample validity of the SOTA claim. Please tune m and k on the validation split (or via nested cross-validation) and report the corrected test results, and state explicitly which split is used in Fig. 2.
  2. [§5.1, Table 3] The claim that "TeR-TSF consistently achieves the lowest or second-lowest prediction error across each domain" is contradicted by Table 3. On SocialGood, TeR-TSF MSE is 1.199, which is fourth behind MCD-TSF (1.035), Time-LLM (1.047), and MM-TSF (1.067). On Energy, MCD-TSF (0.153) also outperforms TeR-TSF (0.202). Please correct the claim and the surrounding discussion, or rerun experiments if these numbers are not the intended final results.
  3. [§3.3.2, Eq. (8)] Eq. (8) is not the standard DPO loss. Standard DPO optimizes a log-sigmoid of the difference between policy-to-reference log-ratios for preferred and dispreferred completions, with a KL penalty. Eq. (8) instead applies a softmax directly to rewards r+ and r− and contains no reference-model likelihood terms. As written, this is a reward-based classification loss, not the DPO objective cited in [40]. Please either provide a derivation showing that Eq. (8) is a valid preference-optimization objective for this setting, or replace it with the actual DPO loss.
  4. [§3.3.1, §3.3.2] The reward r1 in Eq. (6) uses ground-truth Y and the TSF model's predictions, but the paper never states which data split (training, validation, or test) is used to compute this reward during the DPO loop. If the test split is used for reward computation, the reported test results are invalid. In addition, the weights w1 and w2 in r = w1·r1 + w2·r2 are never reported, and the keyword set K in Eq. (7) is only loosely exemplified. These are free parameters that affect the generated text and the final results; please specify them and, ideally, provide a sensitivity analysis.
  5. [§5, Tables 2–3] All results are reported as single MSE/MAE values without standard deviations or significance tests. Many comparisons are close (e.g., Agriculture MSE 0.338 vs 0.222 for MCD-TSF, and Environment MSE 0.251 vs 0.275 for MCD-TSF). Without repeated-seed variance or statistical tests, the claimed consistent improvements cannot be assessed. Please report mean±std over multiple seeds and, where appropriate, significance tests or confidence intervals.
minor comments (6)
  1. [§3.3.1] The text says "The third reward term r2" but only two reward terms are defined; this should read "second reward term."
  2. [Fig. 1] Typography: "Rinforced Text" should be "Reinforced Text."
  3. [Fig. 5] Subcaptions list "(c) Economy (c) Health" — the second "(c)" should be "(d)."
  4. [§4.3] The default numbers are stated as "4 and 2" for rounds and generated texts, but Sections 5.2–5.3 describe selecting these values from experiments on the Energy domain. Please clarify the order of the tuning procedure and, if these are tuned values, report the validation split used for selection.
  5. [Fig. 1 / §3.1] The prompt template contains incomplete placeholders ("In , the value is , the report is ."). Please provide the exact prompt template in an appendix for reproducibility.
  6. [References] Reference [16] contains a placeholder "# PLACEHOLDER PARENTMETADATA VALUE#"; this needs to be completed.

Circularity Check

2 steps flagged · score 6.0 of 10

Partial circularity: RL-round count m is fitted on the Energy test set and then reported as a predictive result, and the task-relevance 'confirmation' is just the r2 objective restated.

  1. fitted input called prediction [Section 5.3, Fig. 3 caption; reported in Tables 2–3]
    "Forecasting errors with different numbers of RL round m = {1, 2, 3, 4, 5} on the test set of Energy domain. ... Given these findings, we set m = 4 for all subsequent experiments."

    The number of RL rounds m is chosen by inspecting MSE/MAE on the Energy test set, and the same Energy test set is then included in the per-domain and average results in Tables 2 and 3. The claim that TeR-TSF is 'lowest or second-lowest' on Energy is therefore not an out-of-sample prediction: the reported Energy error is the value used to select the configuration. A hyperparameter fitted to the test labels is presented as a model prediction.

  2. self definitional [Section 3.3.1 Eq. (7); Section 5.4 t-SNE analysis]
    "The reward r2 is calculated as the normalized count of such keywords. ... For instance, enhanced text frequently incorporates tokens such as 'trend' and 'seasonality' ... This evidence confirms that our proposed approach effectively filters out irrelevant tokens ..."

    r2 is defined as the normalized count of predefined task-relevant keywords in the generated text, and DPO ranks candidates by this reward. The t-SNE analysis then cites the abundance of exactly those keywords in reinforced text as independent 'evidence' that the method concentrates on temporally informative language. Because the keyword abundance is literally the training objective, this observation is the reward definition restated as a finding; it cannot provide confirmatory evidence beyond the constructed metric.

full rationale

The core TeR-TSF framework is not definitionally circular: r1 is an external supervised signal (negative MSE against ground truth), the baselines are external, and the benchmark is an external fixed split. The self-citations (e.g., MCD-TSF [29]) set experimental conventions rather than supply a uniqueness theorem or a load-bearing premise. The DPO/RL loop in which the TSF model is retrained on generated text is a feedback mechanism, not an equation that reduces the test result to the reward. Nevertheless, two specific passages reduce to their own inputs. First, m is explicitly selected on the Energy test set ('on the test set of Energy domain') and then the same test set contributes to the headline numbers, so the Energy portion of the SOTA claim is a fitted statistic rather than a prediction. Second, the Section 5.4 t-SNE 'confirmation' is self-verifying because reward r2 directly maximizes the keyword counts that the analysis then reports as evidence. A separate risk not counted as demonstrated circularity: the paper never states which split provides Y in Eq. (6); if test labels participated in DPO, the results would be invalid, but the text does not establish that. The k sweep (Section 5.2) is also performed on the Energy domain without stating the split, so I do not count it as proven test leakage. These issues are partial: the average result still involves external baselines and seven other domains, so the central claim is not fully forced by construction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several hand-chosen components (reward weights, keyword set, number of RL rounds, number of candidates) that are either unreported or tuned on a single domain's test set. The method also assumes that the Time-MMD benchmark and the chosen TSF architecture provide a valid reward signal, and that the non-standard DPO variant effectively optimizes generation. None of these assumptions is independently validated.

free parameters (4)
  • reward weights w1 and w2 = not reported
    The final reward r = w1*r1 + w2*r2 is a weighted sum, but the paper never specifies the values of w1 and w2. These weights directly control the optimization trade-off between forecasting accuracy and keyword relevance, and they are hand-chosen or tuned implicitly.
  • number of RL rounds m = 4
    Chosen based on experiments on the Energy domain test set (Section 5.3). This is a hyperparameter tuned on a test set, which risks overfitting the choice to one domain.
  • number of candidates k = 2
    Selected after evaluating k = 1,2,4,6,8,10 on the Energy domain test set (Section 5.2). This choice is also data-driven on a test set.
  • task-relevant keyword set K = unspecified
    The keyword set used in the relevance reward r2 is defined as examples like 'peak', 'fluctuation', 'seasonality', but the full set and its curation are not given. The composition directly affects the reward and the generated text.
assumptions (4)
  • domain assumption Time-MMD is a valid and representative benchmark for multimodal time series forecasting
    All experiments use this single dataset, and the paper assumes its text sparsity and misalignment reflect real-world conditions. No external validation is provided.
  • domain assumption The downstream TSF model (PatchTST encoder with LLM embedding and cross-attention) is an appropriate fusion architecture
    The method builds on TFHTS and assumes that this architecture is a reasonable proxy for multimodal TSF quality, which is used as the reward signal.
  • domain assumption The reward r1 based on the TSF model's MSE is a valid proxy for text quality
    The paper assumes that improving the TSF model's predictions on the reward set translates to better general forecasting. This is plausible but not proven, especially since the TSF model itself is retrained on the generated text.
  • ad hoc to paper The DPO-style loss (Eq. 8) optimizes the LLM effectively
    Eq. 8 uses a softmax over rewards directly, which is not the standard DPO loss; the paper does not justify why this objective improves generation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Text Reinforcement for Multimodal Time Series Forecasting." pith.science (2026). https://pith.science/paper/QYMUOP6W

@misc{pith2026250900687,
  author       = {Pith},
  title        = {Pith review of: Text Reinforcement for Multimodal Time Series Forecasting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QYMUOP6W}},
  note         = {Machine review of arXiv:2509.00687}
}
read the original abstract

Recent studies in time series forecasting (TSF) use multimodal inputs, such as text and historical time series data, to predict future values. These studies mainly focus on developing advanced techniques to integrate textual information with time series data to perform the task and achieve promising results. Meanwhile, these approaches rely on high-quality text and time series inputs, whereas in some cases, the text does not accurately or fully capture the information carried by the historical time series, which leads to unstable performance in multimodal TSF. Therefore, it is necessary to enhance the textual content to improve the performance of multimodal TSF. In this paper, we propose improving multimodal TSF by reinforcing the text modalities. We propose a text reinforcement model (TeR) to generate reinforced text that addresses potential weaknesses in the original text, then apply this reinforced text to support the multimodal TSF model's understanding of the time series, improving TSF performance. To guide the TeR toward producing higher-quality reinforced text, we design a reinforcement learning approach that assigns rewards based on the impact of each reinforced text on the performance of the multimodal TSF model and its relevance to the TSF task. We optimize the TeR accordingly, so as to improve the quality of the generated reinforced text and enhance TSF performance. Extensive experiments on a real-world benchmark dataset covering various domains demonstrate the effectiveness of our approach, which outperforms strong baselines and existing studies on the dataset.

Figures

Figures reproduced from arXiv: 2509.00687 by the authors.

Figure 1
Figure 1. An illustration of the TeR-TSF Model. The left-top panel depicts the multimodal TSF model, which separately [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 4
Figure 4. Forecasting errors with different LLMs on the Energy [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 3
Figure 3. Forecasting errors with different numbers of RL [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 6
Figure 6. Figure 6: Case study with two examples from Health (top panels) and SocialGood (bottom panels) domains. The left side [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Differencing the Diffusion Trajectory toward Uncertain Components for Time Series Forecasting

    cs.AI 2026-06 conditional novelty 6.0 of 10

    DiffDiff rewires diffusion forecasting so corruption gradually emphasizes second-order differences, concentrating generation on history-uncertain parts and improving forecasts on seven benchmarks.

  2. Heterogeneous Scientific Foundation Model Collaboration

    cs.AI 2026-04 unverdicted novelty 5.0 of 10

    Eywa enables language-based agentic AI systems to collaborate with specialized scientific foundation models for improved performance on structured data tasks.

Reference graph

Works this paper leans on

92 extracted references · 57 canonical work pages · cited by 2 Pith papers

  1. [40]

    Direct preference optimization: Your language model is secretly a reward model,

    R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn, “Direct preference optimization: Your language model is secretly a reward model,” Advances in Neural Information Processing Systems, vol. 36, pp. 53 728–53 741, 2023

    2023

  2. [1]

    A time series analysis-based stock price prediction using machine learning and deep learning models,

    S. Mehtab and J. Sen, “A time series analysis-based stock price prediction using machine learning and deep learning models,” In- ternational Journal of Business Forecasting and Marketing Intelligence , vol. 6, no. 4, pp. 272–335, 2020

    2020

  3. [2]

    An improved deep learning model for predicting stock market price time series,

    H. Liu and Z. Long, “An improved deep learning model for predicting stock market price time series,”Digital Signal Processing, vol. 102, p. 102741, 2020

    2020

  4. [3]

    Effective stock price prediction using time series forecast- ing,

    K. Prakhar, S. Sountharrajan, E. Suganya, M. Karthiga, and S. Ku- mar, “Effective stock price prediction using time series forecast- ing,” in 2022 6th International Conference on Trends in Electronics and Informatics (ICOEI). IEEE, 2022, pp. 1636–1640

    2022

  5. [4]

    A real-time weather forecasting and analysis,

    S. Kothapalli and S. Totad, “A real-time weather forecasting and analysis,” in 2017 IEEE International Conference on Power, Control, Signals and Instrumentation Engineering (ICPCSI) . IEEE, 2017, pp. 1567–1570

    2017

  6. [5]

    Transductive lstm for time-series prediction: An application to weather forecasting,

    Z. Karevan and J. A. Suykens, “Transductive lstm for time-series prediction: An application to weather forecasting,” Neural Net- works, vol. 125, pp. 1–9, 2020

    2020

  7. [6]

    Temporal convolutional neural (tcn) network for an effective weather forecasting using time-series data from the local weather station,

    P . Hewage, A. Behera, M. Trovati, E. Pereira, M. Ghahremani, F. Palmieri, and Y. Liu, “Temporal convolutional neural (tcn) network for an effective weather forecasting using time-series data from the local weather station,” Soft Computing, vol. 24, pp. 16 453– 16 482, 2020

    2020

  8. [7]

    Forecasting energy demand in china and india: Using single-linear, hybrid-linear, and non-linear time series forecast techniques,

    Q. Wang, S. Li, and R. Li, “Forecasting energy demand in china and india: Using single-linear, hybrid-linear, and non-linear time series forecast techniques,” Energy, vol. 161, pp. 821–831, 2018

    2018

Show all 92 references
  1. [8]

    Forecasting energy time series with profile neural networks,

    B. Heidrich, M. Turowski, N. Ludwig, R. Mikut, and V . Ha- genmeyer, “Forecasting energy time series with profile neural networks,” in Proceedings of the eleventh acm international conference on future energy systems, 2020, pp. 220–230

    2020

  2. [9]

    Temporal convolutional networks applied to energy- related time series forecasting,

    P . Lara-Ben ´ıtez, M. Carranza-Garc´ıa, J. M. Luna-Romera, and J. C. Riquelme, “Temporal convolutional networks applied to energy- related time series forecasting,” applied sciences , vol. 10, no. 7, p. 2322, 2020

    2020

  3. [10]

    Usage of time series fore- casting model in supply chain sales prediction,

    A. Raiyani, A. Lathigara, and H. Mehta, “Usage of time series fore- casting model in supply chain sales prediction,” in IOP Conference Series: Materials Science and Engineering , vol. 1042, no. 1. IOP Publishing, 2021, p. 012022

    2021

  4. [11]

    Fore- casting hierarchical time series in supply chains: an empirical investigation,

    D. Mircetic, B. Rostami-Tabar, S. Nikolicic, and M. Maslaric, “Fore- casting hierarchical time series in supply chains: an empirical investigation,” International Journal of Production Research , vol. 60, no. 8, pp. 2514–2533, 2022

    2022

  5. [12]

    Time series forecasting and mod- eling of food demand supply chain based on regressors analysis,

    S. K. Panda and S. N. Mohanty, “Time series forecasting and mod- eling of food demand supply chain based on regressors analysis,” IEEE Access, vol. 11, pp. 42 679–42 700, 2023

    2023

  6. [13]

    Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting,

    S. Li, X. Jin, Y. Xuan, X. Zhou, W. Chen, Y.-X. Wang, and X. Yan, “Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting,” Advances in neural infor- mation processing systems, vol. 32, 2019

    2019

  7. [14]

    Informer: Beyond efficient transformer for long se- quence time-series forecasting,

    H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, “Informer: Beyond efficient transformer for long se- quence time-series forecasting,” in Proceedings of the AAAI confer- ence on artificial intelligence, vol. 35, no. 12, 2021, pp. 11 106–11 115

    2021

  8. [15]

    Autoformer: Decomposition transformers with auto-correlation for long-term series forecast- ing,

    H. Wu, J. Xu, J. Wang, and M. Long, “Autoformer: Decomposition transformers with auto-correlation for long-term series forecast- ing,” Advances in neural information processing systems , vol. 34, pp. 22 419–22 430, 2021

    2021

  9. [16]

    Pyraformer: Low-complexity pyramidal attention for long- range time series modeling and forecasting,

    S. Liu, H. Yu, C. Liao, J. Li, W. Lin, A. X. Liu, and S. Dust- dar, “Pyraformer: Low-complexity pyramidal attention for long- range time series modeling and forecasting,” in # PLACE- HOLDER P ARENTMETADATA V ALUE#, 2022

    2022

  10. [17]

    Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting,

    T. Zhou, Z. Ma, Q. Wen, X. Wang, L. Sun, and R. Jin, “Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting,” in International conference on machine learning. PMLR, 2022, pp. 27 268–27 286

    2022

  11. [18]

    Etsformer: Exponential smoothing transformers for time-series forecasting,

    G. Woo, C. Liu, D. Sahoo, A. Kumar, and S. Hoi, “Etsformer: Exponential smoothing transformers for time-series forecasting,” arXiv preprint arXiv:2202.01381, 2022

    2022 arXiv

  12. [19]

    Diffu- sion models for time series forecasting: A survey,

    C. Su, Z. Cai, Y. Tian, Z. Chang, Z. Zheng, and Y. Song, “Diffu- sion models for time series forecasting: A survey,” arXiv preprint arXiv:2507.14507, 2025

    2025 arXiv

  13. [20]

    Time-mmd: A new multi-domain multimodal dataset for time series analysis,

    H. Liu, S. Xu, Z. Zhao, L. Kong, H. Kamarthi, A. B. Sasanur, M. Sharma, J. Cui, Q. Wen, C. Zhang et al. , “Time-mmd: A new multi-domain multimodal dataset for time series analysis,” arXiv preprint arXiv:2406.08627, 2024. 11

    2024 arXiv

  14. [21]

    Context matters: Leveraging contextual features for time series forecasting,

    S. Chattopadhyay, P . Paliwal, S. S. Narasimhan, S. Agarwal, and S. P . Chinchali, “Context matters: Leveraging contextual features for time series forecasting,” arXiv preprint arXiv:2410.12672, 2024

    2024 arXiv

  15. [22]

    Multi-modal forecaster: Jointly predicting time series and textual data,

    K. Kim, H. Tsai, R. Sen, A. Das, Z. Zhou, A. Tanpure, M. Luo, and R. Yu, “Multi-modal forecaster: Jointly predicting time series and textual data,” arXiv preprint arXiv:2411.06735, 2024

    2024 arXiv

  16. [23]

    Financial time series forecasting with multi-modality graph neural network,

    D. Cheng, F. Yang, S. Xiang, and J. Liu, “Financial time series forecasting with multi-modality graph neural network,” Pattern Recognition, vol. 121, p. 108218, 2022

    2022

  17. [24]

    Taxi demand forecasting based on the temporal multimodal information fusion graph neural network,

    W. Liao, B. Zeng, J. Liu, P . Wei, and X. Cheng, “Taxi demand forecasting based on the temporal multimodal information fusion graph neural network,” Applied Intelligence , vol. 52, no. 10, pp. 12 077–12 090, 2022

    2022

  18. [25]

    Unveiling the potential of text in high-dimensional time series forecasting,

    X. Zhou, W. Wang, S. Qu, Z. Zhang, and C. Bergmeir, “Unveiling the potential of text in high-dimensional time series forecasting,” arXiv preprint arXiv:2501.07048, 2025

    2025 arXiv

  19. [26]

    Time-llm: Time series forecast- ing by reprogramming large language models,

    M. Jin, S. Wang, L. Ma, Z. Chu, J. Y. Zhang, X. Shi, P .-Y. Chen, Y. Liang, Y.-F. Li, S. Pan et al. , “Time-llm: Time series forecast- ing by reprogramming large language models,” arXiv preprint arXiv:2310.01728, 2023

    2023 arXiv

  20. [27]

    Chattime: A unified multimodal time series foundation model bridging numerical and textual data,

    C. Wang, Q. Qi, J. Wang, H. Sun, Z. Zhuang, J. Wu, L. Zhang, and J. Liao, “Chattime: A unified multimodal time series foundation model bridging numerical and textual data,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 39, no. 12, 2025, pp. 12 694–12 702

    2025

  21. [28]

    Fusing large language models with temporal transformers for time series fore- casting,

    C. Su, Y. Tian, Q. Liu, J. Zhang, and Y. Song, “Fusing large language models with temporal transformers for time series fore- casting,” arXiv preprint arXiv:2507.10098, 2025

    2025 arXiv

  22. [29]

    Multimodal conditioned diffusive time series forecasting,

    C. Su, Y. Tian, and Y. Song, “Multimodal conditioned diffusive time series forecasting,” arXiv preprint arXiv:2504.19669, 2025

    2025 arXiv

  23. [30]

    Textual data for time series forecasting,

    D. Obst, B. Ghattas, S. Claudel, J. Cugliari, Y. Goude, and G. Op- penheim, “Textual data for time series forecasting,” arXiv preprint arXiv:1910.12618, 2019

    1910 arXiv

  24. [31]

    A survey on image data augmentation for deep learning,

    C. Shorten and T. M. Khoshgoftaar, “A survey on image data augmentation for deep learning,” Journal of big data , vol. 6, no. 1, pp. 1–48, 2019

    2019

  25. [32]

    A review: Data pre- processing and data augmentation techniques,

    K. Maharana, S. Mondal, and B. Nemade, “A review: Data pre- processing and data augmentation techniques,” Global Transitions Proceedings, vol. 3, no. 1, pp. 91–99, 2022

    2022

  26. [33]

    Data augmentation: A comprehen- sive survey of modern approaches,

    A. Mumuni and F. Mumuni, “Data augmentation: A comprehen- sive survey of modern approaches,” Array, vol. 16, p. 100258, 2022

    2022

  27. [34]

    Data augmentation can improve robustness,

    S.-A. Rebuffi, S. Gowal, D. A. Calian, F. Stimberg, O. Wiles, and T. A. Mann, “Data augmentation can improve robustness,” Advances in neural information processing systems, vol. 34, pp. 29 935– 29 948, 2021

    2021

  28. [35]

    A survey of data augmentation approaches for nlp,

    S. Y. Feng, V . Gangal, J. Wei, S. Chandar, S. Vosoughi, T. Mitamura, and E. Hovy, “A survey of data augmentation approaches for nlp,” arXiv preprint arXiv:2105.03075, 2021

    2021 arXiv

  29. [36]

    Time-series data augmentation based on interpolation,

    C. Oh, S. Han, and J. Jeong, “Time-series data augmentation based on interpolation,” Procedia Computer Science , vol. 175, pp. 64–71, 2020

    2020

  30. [37]

    A comprehensive survey on data augmenta- tion,

    Z. Wang, P . Wang, K. Liu, P . Wang, Y. Fu, C.-T. Lu, C. C. Aggarwal, J. Pei, and Y. Zhou, “A comprehensive survey on data augmenta- tion,” arXiv preprint arXiv:2405.09591, 2024

    2024

  31. [38]

    Exploring generative data augmentation in multivariate time series forecasting: opportunities and challenges,

    A. Debnath, G. Waghmare, H. Wadhwa, S. Asthana, and A. Arora, “Exploring generative data augmentation in multivariate time series forecasting: opportunities and challenges,”Solar-Energy, vol. 137, pp. 52–560, 2021

    2021

  32. [39]

    Timecap: Learning to contextualize, augment, and predict time series events with large language model agents,

    G. Lee, W. Yu, K. Shin, W. Cheng, and H. Chen, “Timecap: Learning to contextualize, augment, and predict time series events with large language model agents,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 17, 2025, pp. 18 082– 18 090

    2025

  33. [41]

    Stock movement prediction from tweets and historical prices,

    Y. Xu and S. B. Cohen, “Stock movement prediction from tweets and historical prices,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2018, pp. 1970–1979

    2018

  34. [42]

    From news to fore- cast: Integrating event analysis in llm-based time series forecasting with reflection,

    X. Wang, M. Feng, J. Qiu, J. Gu, and J. Zhao, “From news to fore- cast: Integrating event analysis in llm-based time series forecasting with reflection,” Advances in Neural Information Processing Systems , vol. 37, pp. 58 118–58 153, 2024

    2024

  35. [43]

    Timecma: Towards llm-empowered multivariate time series forecasting via cross-modality alignment,

    C. Liu, Q. Xu, H. Miao, S. Yang, L. Zhang, C. Long, Z. Li, and R. Zhao, “Timecma: Towards llm-empowered multivariate time series forecasting via cross-modality alignment,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 39, no. 18, 2025, pp. 18 780–18 788

    2025

  36. [44]

    How can large lan- guage models understand spatial-temporal data?

    L. Liu, S. Yu, R. Wang, Z. Ma, and Y. Shen, “How can large lan- guage models understand spatial-temporal data?” arXiv preprint arXiv:2401.14192, 2024

    2024 arXiv

  37. [45]

    A multi-modal time series intelli- gent prediction model,

    Q. Xian and W. Liang, “A multi-modal time series intelli- gent prediction model,” in INTERNATIONAL CONFERENCE ON WIRELESS COMMUNICATIONS, NETWORKING AND APPLI- CATIONS. Springer, 2021, pp. 1150–1157

    2021

  38. [46]

    Forecasting power demand in china with a cnn-lstm model including multimodal information,

    D. Wang, J. Gan, J. Mao, F. Chen, and L. Yu, “Forecasting power demand in china with a cnn-lstm model including multimodal information,” Energy, vol. 263, p. 126012, 2023

    2023

  39. [47]

    Gpt4mts: Prompt- based large language model for multimodal time-series forecast- ing,

    F. Jia, K. Wang, Y. Zheng, D. Cao, and Y. Liu, “Gpt4mts: Prompt- based large language model for multimodal time-series forecast- ing,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 21, 2024, pp. 23 343–23 351

    2024

  40. [48]

    Unitime: A language-empowered unified model for cross- domain time series forecasting,

    X. Liu, J. Hu, Y. Li, S. Diao, Y. Liang, B. Hooi, and R. Zimmer- mann, “Unitime: A language-empowered unified model for cross- domain time series forecasting,” in Proceedings of the ACM Web Conference 2024, 2024, pp. 4095–4106

    2024

  41. [49]

    Inspiration: A reinforcement learning-based human visual perception-driven image enhancement paradigm for underwater scenes,

    H. Wang, S. Sun, L. Chang, H. Li, W. Zhang, A. C. Frery, and P . Ren, “Inspiration: A reinforcement learning-based human visual perception-driven image enhancement paradigm for underwater scenes,” Engineering Applications of Artificial Intelligence, vol. 133, p. 108411, 2024

    2024

  42. [50]

    A review of research on reinforcement learning algorithms for multi-agents,

    K. Hu, M. Li, Z. Song, K. Xu, Q. Xia, N. Sun, P . Zhou, and M. Xia, “A review of research on reinforcement learning algorithms for multi-agents,” Neurocomputing, p. 128068, 2024

    2024

  43. [51]

    Deep reinforcement learning for robotics: A survey of real-world successes,

    C. Tang, B. Abbatematteo, J. Hu, R. Chandra, R. Mart ´ın-Mart´ın, and P . Stone, “Deep reinforcement learning for robotics: A survey of real-world successes,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 27, 2025, pp. 28 694–28 698

    2025

  44. [52]

    Learning multimodal contrast with cross-modal memory and reinforced contrast recognition,

    Y. Tian, F. Xia, and Y. Song, “Learning multimodal contrast with cross-modal memory and reinforced contrast recognition,” inFind- ings of the Association for Computational Linguistics ACL 2024 , 2024, pp. 6561–6573

    2024

  45. [53]

    Training language models to self-correct via reinforcement learning,

    A. Kumar, V . Zhuang, R. Agarwal, Y. Su, J. D. Co-Reyes, A. Singh, K. Baumli, S. Iqbal, C. Bishop, R. Roelofs et al., “Training language models to self-correct via reinforcement learning,” arXiv preprint arXiv:2409.12917, 2024

    2024 arXiv

  46. [54]

    Graph-enabled reinforcement learning for time series forecasting with adaptive intelligence,

    T. Shaik, X. Tao, H. Xie, L. Li, J. Yong, and Y. Li, “Graph-enabled reinforcement learning for time series forecasting with adaptive intelligence,” IEEE Transactions on Emerging Topics in Computational Intelligence, 2024

    2024

  47. [55]

    Langtime: A language-guided unified model for time series forecasting with proximal policy optimization,

    W. Niu, Z. Xie, Y. Sun, W. He, M. Xu, and C. Hao, “Langtime: A language-guided unified model for time series forecasting with proximal policy optimization,” arXiv preprint arXiv:2503.08271 , 2025

    2025 arXiv

  48. [56]

    Reinforcement learning based dynamic model combination for time series forecasting,

    Y. Fu, D. Wu, and B. Boulet, “Reinforcement learning based dynamic model combination for time series forecasting,” in Pro- ceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 6, 2022, pp. 6639–6647

    2022

  49. [57]

    Reinforcement learning based dynamic weighing of ensemble models for time series forecasting,

    S. K. Perepu, B. S. Balaji, H. K. Tanneru, S. Kathari, and V . S. Pinnamaraju, “Reinforcement learning based dynamic weighing of ensemble models for time series forecasting,” arXiv preprint arXiv:2008.08878, 2020

    2008 arXiv

  50. [58]

    Timehf: Billion-scale time series models guided by human feedback,

    Y. Qi, H. Hu, D. Lei, J. Zhang, Z. Shi, Y. Huang, Z. Chen, X. Lin, and Z.-J. M. Shen, “Timehf: Billion-scale time series models guided by human feedback,” arXiv preprint arXiv:2501.15942, 2025

    2025 arXiv

  51. [59]

    Synthetic data augmentation for deep reinforcement learning in financial trading,

    C. Liu, C. Ventre, and M. Polukarov, “Synthetic data augmentation for deep reinforcement learning in financial trading,” in Proceed- ings of the third ACM international conference on AI in finance , 2022, pp. 343–351

    2022

  52. [60]

    Automated data augmenta- tion for few-shot time series forecasting: A reinforcement learning approach guided by a model zoo,

    H. Yuan, X. Li, Y. Wang, and X. Yang, “Automated data augmenta- tion for few-shot time series forecasting: A reinforcement learning approach guided by a model zoo,” arXiv preprint arXiv:2409.06282, 2024

    2024 arXiv

  53. [61]

    Data augmentation techniques in time series domain: a survey and taxonomy,

    G. Iglesias, E. Talavera, ´A. Gonz ´alez-Prieto, A. Mozo, and S. G ´omez-Canaval, “Data augmentation techniques in time series domain: a survey and taxonomy,” Neural Computing and Applica- tions, vol. 35, no. 14, pp. 10 123–10 145, 2023

    2023

  54. [62]

    Time series data augmentation for deep learning: A survey,

    Q. Wen, L. Sun, F. Yang, X. Song, J. Gao, X. Wang, and H. Xu, “Time series data augmentation for deep learning: A survey,” arXiv preprint arXiv:2002.12478, 2020

    2002 arXiv

  55. [63]

    Data augmen- tation for time series classification using convolutional neural 12 networks,

    A. Le Guennec, S. Malinowski, and R. Tavenard, “Data augmen- tation for time series classification using convolutional neural 12 networks,” in ECML/PKDD workshop on advanced analytics and learning on temporal data, 2016

    2016

  56. [64]

    Multi-scale convolutional neural networks for time series classification,

    Z. Cui, W. Chen, and Y. Chen, “Multi-scale convolutional neural networks for time series classification,” arXiv preprint arXiv:1603.06995, 2016

    2016 arXiv

  57. [65]

    Data augmentation using synthetic data for time series classifica- tion with deep residual networks,

    H. I. Fawaz, G. Forestier, J. Weber, L. Idoumghar, and P .-A. Muller, “Data augmentation using synthetic data for time series classifica- tion with deep residual networks,”arXiv preprint arXiv:1808.02455, 2018

    2018 arXiv

  58. [66]

    Time series anomaly detection using con- volutional neural networks and transfer learning,

    T. Wen and R. Keyes, “Time series anomaly detection using con- volutional neural networks and transfer learning,” arXiv preprint arXiv:1905.13628, 2019

    1905 arXiv

  59. [67]

    Robusttad: Robust time series anomaly detection via decomposition and convolutional neural networks,

    J. Gao, X. Song, Q. Wen, P . Wang, L. Sun, and H. Xu, “Robusttad: Robust time series anomaly detection via decomposition and convolutional neural networks,” arXiv preprint arXiv:2002.09545 , 2020

    2002 arXiv

  60. [68]

    Feature representation and data augmentation for human activity classification based on wearable imu sensor data using a deep lstm neural network,

    O. Steven Eyobu and D. S. Han, “Feature representation and data augmentation for human activity classification based on wearable imu sensor data using a deep lstm neural network,” Sensors, vol. 18, no. 9, p. 2892, 2018

    2018

  61. [69]

    Specaugment: A simple data augmentation method for automatic speech recognition,

    D. S. Park, W. Chan, Y. Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and Q. V . Le, “Specaugment: A simple data augmentation method for automatic speech recognition,” arXiv preprint arXiv:1904.08779, 2019

    1904 arXiv

  62. [70]

    Generating synthetic time series to augment sparse datasets,

    G. Forestier, F. Petitjean, H. A. Dau, G. I. Webb, and E. Keogh, “Generating synthetic time series to augment sparse datasets,” in 2017 IEEE international conference on data mining (ICDM) . IEEE, 2017, pp. 865–870

    2017

  63. [71]

    Enhancing human activity recognition using deep learning and time series augmented data,

    L. Alawneh, T. Alsarhan, M. Al-Zinati, M. Al-Ayyoub, Y. Jararweh, and H. Lu, “Enhancing human activity recognition using deep learning and time series augmented data,” Journal of Ambient Intelligence and Humanized Computing, pp. 1–16, 2021

    2021

  64. [72]

    Intelligent random noise modeling by the improved variational autoencoding method and its applica- tion to data augmentation,

    Q. Feng, Y. Li, and H. Wang, “Intelligent random noise modeling by the improved variational autoencoding method and its applica- tion to data augmentation,” Geophysics, vol. 86, no. 1, pp. T19–T31, 2021

    2021

  65. [73]

    Improving clas- sification accuracy using data augmentation on small data sets,

    F. J. Moreno-Barea, J. M. Jerez, and L. Franco, “Improving clas- sification accuracy using data augmentation on small data sets,” Expert Systems with Applications, vol. 161, p. 113696, 2020

    2020

  66. [74]

    Using variational autoencoder to augment sparse time series datasets,

    M. Goubeaud, P . Joußen, N. Gmyrek, F. Ghorban, L. Schelkes, and A. Kummert, “Using variational autoencoder to augment sparse time series datasets,” in 2021 7th international conference on optimization and applications (ICOA). IEEE, 2021, pp. 1–6

    2021

  67. [75]

    Deep convolutional generative adversarial networks for traffic data imputation encod- ing time series as images,

    T. Huang, P . Chakraborty, and A. Sharma, “Deep convolutional generative adversarial networks for traffic data imputation encod- ing time series as images,” International journal of transportation science and technology, vol. 12, no. 1, pp. 1–18, 2023

    2023

  68. [76]

    Are language models actually useful for time series forecasting?

    M. Tan, M. Merrill, V . Gupta, T. Althoff, and T. Hartvigsen, “Are language models actually useful for time series forecasting?” Ad- vances in Neural Information Processing Systems , vol. 37, pp. 60 162– 60 191, 2024

    2024

  69. [77]

    Chronos: Learning the language of time series,

    A. F. Ansari, L. Stella, C. Turkmen, X. Zhang, P . Mercado, H. Shen, O. Shchur, S. S. Rangapuram, S. P . Arango, S. Kapoor et al. , “Chronos: Learning the language of time series,” arXiv preprint arXiv:2403.07815, 2024

    2024 arXiv

  70. [78]

    Non-stationary transform- ers: Exploring the stationarity in time series forecasting,

    Y. Liu, H. Wu, J. Wang, and M. Long, “Non-stationary transform- ers: Exploring the stationarity in time series forecasting,” Advances in neural information processing systems, vol. 35, pp. 9881–9893, 2022

    2022

  71. [79]

    Crossformer: Transformer utilizing cross- dimension dependency for multivariate time series forecasting,

    Y. Zhang and J. Yan, “Crossformer: Transformer utilizing cross- dimension dependency for multivariate time series forecasting,” in The eleventh international conference on learning representations , 2023

    2023

  72. [80]

    Glove: Global vec- tors for word representation,

    J. Pennington, R. Socher, and C. D. Manning, “Glove: Global vec- tors for word representation,” in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) , 2014, pp. 1532–1543

    2014

  73. [81]

    Complementary Learning of Word Em- beddings,

    Y. Song and S. Shi, “Complementary Learning of Word Em- beddings,” in Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, 7 2018, pp. 4368–4374

    2018

  74. [82]

    hyper- doc2vec: Distributed representations of hypertext documents,

    J. Han, Y. Song, W. X. Zhao, S. Shi, and H. Zhang, “hyper- doc2vec: Distributed representations of hypertext documents,” arXiv preprint arXiv:1805.03793, 2018

    2018 arXiv

  75. [83]

    Lan- guage models are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P . Dhari- wal, A. Neelakantan, P . Shyam, G. Sastry, A. Askell et al. , “Lan- guage models are few-shot learners,” Advances in neural informa- tion processing systems, vol. 33, pp. 1877–1901, 2020

    1901

  76. [84]

    The llama 3 herd of models,

    A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Let- man, A. Mathur, A. Schelten, A. Yang, A. Fan et al., “The llama 3 herd of models,” arXiv e-prints, pp. arXiv–2407, 2024

    2024

  77. [85]

    Qwen3 technical report,

    A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv et al. , “Qwen3 technical report,” arXiv preprint arXiv:2505.09388, 2025

    2025 arXiv

  78. [86]

    Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference,

    B. Warner, A. Chaffin, B. Clavi ´e, O. Weller, O. Hallstr ¨om, S. Taghadouini, A. Gallagher, R. Biswas, F. Ladhak, T. Aarsenet al., “Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference,” arXiv pr...

    2024 arXiv

  79. [87]

    Towards reasoning era: A survey of long chain-of-thought for reasoning large language models,

    Q. Chen, L. Qin, J. Liu, D. Peng, J. Guan, P . Wang, M. Hu, Y. Zhou, T. Gao, and W. Che, “Towards reasoning era: A survey of long chain-of-thought for reasoning large language models,”arXiv preprint arXiv:2503.09567, 2025

    2025 arXiv

  80. [88]

    Balanced training data aug- mentation for aspect-based sentiment analysis,

    J. Liu, Y. Tian, and Y. Song, “Balanced training data aug- mentation for aspect-based sentiment analysis,” arXiv preprint arXiv:2507.09485, 2025

    2025 arXiv

  81. [89]

    Large lan- guage models enhanced by plug and play syntactic knowledge for aspect-based sentiment analysis,

    Y. Tian, X. Li, W. Wang, G. Jin, P . Cheng, and Y. Song, “Large lan- guage models enhanced by plug and play syntactic knowledge for aspect-based sentiment analysis,” arXiv preprint arXiv:2506.12991 , 2025

    2025 arXiv

  82. [90]

    A time se- ries is worth 64 words: Long-term forecasting with transformers,

    Y. Nie, N. H. Nguyen, P . Sinthong, and J. Kalagnanam, “A time se- ries is worth 64 words: Long-term forecasting with transformers,” arXiv preprint arXiv:2211.14730, 2022

    2022 arXiv

  83. [91]

    Lora: Low-rank adaptation of large language models

    E. J. Hu, Y. Shen, P . Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen et al. , “Lora: Low-rank adaptation of large language models.” ICLR, vol. 1, no. 2, p. 3, 2022

    2022

  84. [92]

    Are transformers effective for time series forecasting?

    A. Zeng, M. Chen, L. Zhang, and Q. Xu, “Are transformers effective for time series forecasting?” in Proceedings of the AAAI conference on artificial intelligence , vol. 37, no. 9, 2023, pp. 11 121– 11 128

    2023

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.