Enhancing Channel-Independent Time Series Forecasting via Cross-Variate Patch Embedding

Donghwa Shin; Edwin Zhang

REVIEW 4 major objections 4 minor 30 references

Enhancing Channel-Independent Time Series Forecasting via Cross-Variate Patch Embedding

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a channel-independent time series forecaster can absorb cross-variate information by swapping only its patch-embedding layer for a version with a learnable position encoding and a router-attention block, improving…

desk verdict A clean, honest incremental idea whose central empirical claim is undercut by a non-original baseline and correlation-pruned datasets; worth reviewing, but not at face value. read the letter →

arxiv 2505.12761 v3 pith:7ST3Z7XY submitted 2025-05-19 cs.LG cs.AI

Donghwa Shin , Edwin Zhang This is my paper · ORCID

classification cs.LGcs.AI

keywords channelindependencecross-variatedependenciespatchembeddingrouterattentionlong-termtimeseriesforecastingTime-LLMtransformermultivariate

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that channel-independent (CI) forecasting models, which treat each variable as a separate univariate problem, can be given cross-variate awareness without giving up their robustness or redesigning the whole model. The proposed Cross-Variate Patch Embedding (CVPE) replaces the vanilla patch embedding with one that adds a learnable positional encoding and a lightweight router-attention block, so each patch carries information gathered from all variables before the frozen LLM sees it. Plugging CVPE into Time-LLM lowers average MSE by 4.6% on Weather and 6.7% on a ten-channel subset of Traffic, with no meaningful loss on ETTh1 or the reduced ECL set. On ETTh2 and ETTm2, where channel correlations are weak, the module loses up to 5.2% on average MSE, so the benefit is conditional on how much inter-variable signal actually exists. If the claim holds, CI models can be upgraded incrementally rather than replaced by fully channel-dependent architectures.

What carries the argument

The load-bearing object is the Cross-Variate Patch Embedding module: after linear projection of each channel's patches, the model adds a learnable position encoding $W_P \in \mathbb{R}^{P \times d_m}$ to the stacked embeddings $X_P \in \mathbb{R}^{N \times P \times d_m}$, then runs two multi-head attention steps with $c$ learnable router vectors $R \in \mathbb{R}^{N \times c \times d_m}$. The first step aggregates all variates into a compact router representation; the second redistributes that representation back to the patches, followed by a LayerNorm with skip connection and an MLP. Its stated complexity is $O(NP)$, which is what keeps the module 'lightweight'.

What would settle it

Re-run the comparison on the full Traffic and ECL datasets with the original Llama-7B backbone and a 512-step context: if the reported 6.7% and 4.6% average MSE gains do not reproduce or reverse, the benefit is an artifact of the reduced datasets or the weaker baseline.

Watch

Extended reading notes

Core claim

The central claim is that injecting cross-variate context at the patch-embedding stage is enough to let a channel-independent model like Time-LLM exploit inter-variable relationships. The authors show that adding a learnable position encoding plus a router-attention mechanism to the patch embeddings, and nothing else, yields consistent MSE and MAE improvements on Weather and a ten-feature Traffic subset while leaving weakly correlated ETT datasets essentially unchanged or slightly worse. This is presented as evidence that the channel-independent design, not the lack of a cross-channel mechanism, is the main thing separating CI models from richer multivariate models.

Load-bearing premise

The gains depend on comparing against a Time-LLM re-implementation that uses GPT-2 with a 256-step context, and on Traffic and ECL datasets reduced to their ten most correlated channels; if these do not match how practitioners actually run Time-LLM, the measured improvements could disappear.

Editorial extensions

If this is right

Channel-independent forecasters can be retrofitted with cross-channel information through a small, self-contained module; the rest of the pipeline, including the frozen LLM, stays untouched.
On datasets whose variables are strongly correlated, average MSE drops by 4.6% on Weather and 6.7% on the reduced Traffic set, and the improvement appears across all four tested horizons.
On weakly correlated datasets, the module does no consistent harm to ETTh1 or the reduced ECL set, suggesting the extra capacity can be absorbed without degrading temporal modeling.
On ETTh2 and ETTm2, gains reverse to losses of up to 5.2%, so the module's usefulness depends on the strength of real cross-variate signal.
Because CVPE is inserted before patches are split back into per-channel streams, the cross-variate information survives the channel-independent reprogramming and LLM layers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension would be to graft CVPE onto other patch-based CI forecasters, such as PatchTST-style encoders; if the router-attention transfer is the active ingredient, similar gains should appear there.
The evaluation's dependence on ten-feature subsets of Traffic and ECL means the headline gains could be partly an artifact of selecting highly correlated channels; testing on the full datasets would separate the module's effect from the selection effect.
The overfitting on ETTh2 and ETTm2 suggests a promising variant that restricts router attention to a subset of channels per patch, trading a bit of cross-variate capacity for lower noise.
If the $O(NP)$ complexity holds at larger channel counts, CVPE could offer a cheap way to keep CI models competitive with fully channel-dependent models without paying their full attention cost.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The one-line take: this is a modest, honest incremental paper that takes a known mechanism (Crossformer's router attention) and inserts it into the patch embedding of a channel-independent model (Time-LLM). That specific combination is new, and the authors are transparent about where it fails. But the headline "outperforms" outruns the evidence, mainly because the baseline is not the original Time-LLM and two datasets were pruned to the ten most correlated channels.

What it does well: the idea is clean and cheap. Add a learnable positional encoding and a router attention block at the patch embedding level, keep the rest of Time-LLM untouched, and you inject cross-variate information without converting the model to a full channel-dependent one. The complexity stays O(NP), which is genuinely lightweight. The authors also report per-horizon results and admit the losses on ETTh2 and ETTm2. Code is public.

The soft spots are real but not fatal for an exploratory paper. First, both arms of Table 1 use GPT-2 and context length 256, not Llama-7B and 512 as in the original Time-LLM. So the comparison is internally consistent but does not test "outperforms the original baseline" in the published sense. Second, Traffic and ECL are reduced to the top ten features by Pearson correlation. That selection is post hoc and picks exactly the regime where cross-variate reasoning should help; we don't get to see whether the benefit survives on the full datasets. Third, the numbers are single-run averages over four horizons with no error bars, and the method is worse on ETTh2/ETTm2 by up to 5.2%. So the 4.6% Weather and 6.7% modified-Traffic gains could be within run-to-run noise.

I don't think the stress-test note overreaches: the external validity of the central claim is genuinely in question. The authors themselves flag the memory constraints and call for full-dataset evaluation, which is honest but does not fix the current evidence.

Who should read it: anyone working on CI vs CD trade-offs in time series forecasting, or on lightweight ways to reintroduce cross-variate dependencies. For a referee, I'd send it out, because the idea is plausible and the execution is clear; but I'd ask for the full datasets, repeated runs, standard deviations, and a version of the comparison against the real original Time-LLM. As is, it's a useful incremental data point, not a mature claim.

I'd bring it to a reading group as a case study in evaluation choices. But I wouldn't cite it as support for a performance claim yet.

Referee Report

4 major / 4 minor

Summary. The paper proposes Cross-Variate Patch Embeddings (CVPE), a lightweight module that injects cross-variate information into channel-independent (CI) forecasting models by modifying only the patch embedding step. The module adds a learnable positional encoding and a router-attention block, inspired by Crossformer, to the vanilla patch embedding used in Time-LLM. The authors integrate CVPE into Time-LLM and compare against a re-implemented Time-LLM baseline on seven benchmarks (four ETT datasets, Weather, and correlation-reduced Traffic and ECL subsets) over four forecast horizons, reporting MSE and MAE. The main claim is that CVPE improves forecasting performance on datasets with rich cross-variate dependencies, with up to 4.6% average MSE gain on Weather and 6.7% on the modified Traffic subset, while acknowledging performance losses on ETTh2 and ETTm2.

Significance. If the empirical claims held, CVPE would be a simple and practical plug-in for augmenting CI models with cross-variate context, requiring no changes to the backbone and adding modest complexity. The paper's strengths are its minimal architectural intervention, clear attribution of the router-attention idea to Crossformer, and the public release of code. However, the current evidence is not sufficient to establish the central claim: the positive results are concentrated on datasets that the authors have modified by selecting the ten most correlated features, the baseline is not the original Time-LLM configuration, and all comparisons appear to come from a single run without error bars. The negative results on three of the seven benchmarks (up to 5.2% degradation) further complicate the advertised advantage. The contribution is potentially useful, but the empirical case needs substantially more support before the paper's conclusions can be accepted.

major comments (4)

[§4.1, Table 1] The blanket statement that the CVPE-enhanced model 'outperforms the original baseline' is not supported by the full table. Averaged over the four horizons, CVPE is worse on ETTh2 (MSE 0.385 vs 0.366, about 5.2% degradation), ETTm1 (0.390 vs 0.382), and ETTm2 (0.289 vs 0.275). The positive gains are concentrated in Weather and modified Traffic, with tiny gains on ETTh1 and modified ECL. The abstract and introduction should be reworded to describe selective gains rather than overall improvement, or the failures should be analyzed as part of the main claim.
[§4.1, Eq. (2); Appendix B.1] The Traffic and ECL datasets are reduced to the ten features with the highest Pearson correlation. This data-dependent selection creates a regime that is most favorable to a cross-variate module, and results on the full Traffic and ECL datasets are not reported. Consequently, the claim that CVPE improves 'key datasets with rich cross-variate dependencies' is not established for the actual benchmarks practitioners use. The authors should evaluate on the full datasets, or explicitly justify that the selected subsets are representative of the cross-variate structure in the original data.
[§4.1, Appendix B.3] The baseline labeled 'TIME-LLM (Original)' in Table 1 is not the original Time-LLM: the LLM backbone is changed from Llama-7B to GPT-2 and the context window T is reduced from 512 to 256. The comparison is internally consistent because both arms share this altered backbone, but the paper's wording 'outperforms the original baseline' is misleading and is never tested against the original configuration. The authors should either report results under the original Llama-7B/T=512 setting (even for a subset of datasets) or clearly restrict all claims to the modified re-implementation.
[§4.1, Table 1] No error bars, standard deviations, or significance tests are reported. Many of the observed differences are small (e.g., ETTh1 MSE 0.445 vs 0.453; ECL Modified 0.191 vs 0.192), and the averages are apparently computed from a single run. Given the mixed results across datasets, repeated runs with reported mean and standard deviation (or a significance test) are necessary to support the comparative performance claim.

minor comments (4)

[Eq. (2)] The Pearson correlation formula has a typo: the denominator repeats the sum of squared deviations for x in both factors; the second factor should involve (y_i - \bar{y})^2.
[§3.2] The router vectors are introduced as 'for each time step j', but the notation R \in \mathbb{R}^{N\times c\times d_m} has no time dimension and j never appears in Eq. (1). The description should say 'for each variate' or clarify the role of j.
[Table 3 and Appendix B.1] The configuration table lists 'Electricity' and 'Traffic', but the experiments use the modified ten-feature subsets. The labels should be consistent (e.g., 'ECL (Modified)' and 'Traffic (Modified)') throughout.
[Abstract] The abstract says 'seven real-world datasets,' but two of them are correlation-reduced subsets of Traffic and ECL, not the original benchmarks. This should be clarified.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the CVPE gain claim is an empirical comparison against a disclosed re-implementation; no prediction reduces by construction or via self-citation.

full rationale

The paper makes no mathematical derivation of the reported gains. CVPE is an architectural modification (a learnable positional encoding plus a router-attention block) applied to Time-LLM's patch embedding layer, and the central claim is evaluated by MSE/MAE on held-out test splits of seven datasets. The router-attention equations in Eq. (1) are explicitly attributed to Crossformer (Zhang & Yan, 2023) as an external prior work; they are not imported from the authors' own previous results, and no uniqueness or forced-choice theorem is invoked. The baseline is a re-implementation with GPT-2 and T=256 rather than Jin et al.'s Llama-7B/T=512, but this is disclosed in Section 4.1 and Appendix B.3 and is applied identically to both arms, so the relative comparison is internally consistent. The Traffic and ECL subsets are selected by top Pearson correlation (Eq. (2)), which may make the evaluation favorable to a cross-variate module; however, the forecasting errors are still measured on data not used for fitting, and the method does not fit its output to the selection criterion. This is a potential selection-bias or soundness concern, not a circular one. The paper also reports performance losses on ETTh2 and ETTm2 and acknowledges the limitation of not testing on full Traffic/ECL or Llama-7B, consistent with an honest empirical study. Hence no step in the paper's argument reduces to its own inputs by construction, and no self-citation is load-bearing.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The method depends on the validity of patch-based CI forecasting, the choice of Pearson correlation as the feature selection criterion, and the assumption that a frozen GPT-2 still provides useful sequence modeling after the embeddings are changed. The router count is a free parameter whose value is not reported.

free parameters (6)

router count c = not stated
The number of router vectors is introduced in Section 3.2 but its value or tuning procedure is never given.
embedding dimension dm = 32
Set in Table 3 following Time-LLM; affects all embeddings.
attention heads K = 8
Used in CVPE router attention and reprogramming, Table 3.
context window T = 256
Reduced from 512 in the original Time-LLM due to GPU memory, Section 4.1.
learning rate = 1e-2
From Table 3.
text prototypes V' = 1000
From Table 3.

assumptions (3)

domain assumption Patch-based channel-independent models are valid forecasting backbones.
The paper builds on PatchTST and Time-LLM without re-deriving their validity.
domain assumption Pearson correlation between features is a sufficient proxy for useful cross-variate dependency.
Used to select the ten features for the modified Traffic and ECL datasets in Section 4.1.
ad hoc to paper The frozen LLM reprogramming in Time-LLM remains effective when the input distribution shifts due to the new embedding layer.
The paper assumes GPT-2 with T=256 still benefits from cross-variate enriched embeddings without needing retraining of the LLM.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Enhancing Channel-Independent Time Series Forecasting via Cross-Variate Patch Embedding." pith.science (2026). https://pith.science/paper/7ST3Z7XY

@misc{pith2026250512761,
  author       = {Pith},
  title        = {Pith review of: Enhancing Channel-Independent Time Series Forecasting via Cross-Variate Patch Embedding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7ST3Z7XY}},
  note         = {Machine review of arXiv:2505.12761}
}

read the original abstract

Transformers have recently gained popularity in time series forecasting due to their ability to capture long-term dependencies. However, many existing models focus only on capturing temporal dependencies while omitting intricate relationships between variables. Recent models have tried tackling this by explicitly modeling both cross-time and cross-variate dependencies through a sequential or unified attention mechanism, but they are entirely channel dependent (CD) across all layers, making them potentially susceptible to overfitting. To address this, we propose Cross-Variate Patch Embeddings (CVPE), a lightweight CD module that injects cross-variate context into channel-independent (CI) models by simply modifying the patch embedding process. We achieve this by adding a learnable positional encoding and a lightweight router-attention block to the vanilla patch embedding layer. We then integrate CVPE into Time-LLM, a multimodal CI forecasting model, to demonstrate its effectiveness in capturing cross-variate dependencies and enhance the CI model's performance. Extensive experimental results on seven real-world datasets show that our enhanced Time-LLM outperforms the original baseline model simply by incorporating the CVPE module, with no other changes.

Figures

Figures reproduced from arXiv: 2505.12761 by the authors.

**Figure 1.** Model framework of Time-LLM with CVPE. Red dotted line represents original Time-LLM pipeline with the vanilla patch embedding; Green represents modified Time-LLM with CVPE module. As shown in the figure, no other modification is made to Time-LLM. 3.2. Cross-Variate Patch Embedding Overview Given a multivariate input time series with N variates X ∈ R N×T , we first divide each channel X(i) into P patches of length LP… view at source ↗

**Figure 2.** Visualization of router-attention mechanism, inspired by Zhang & Yan (2023). Given position-aware patch embedding XP , a small set of c routers aggregate information from all variates, then redistribute it back to create enhanced embeddings with crossvariate information. The overall complexity of the router-attention mechanism is O(NP) (Zhang & Yan, 2023), which aligns with the lightweight design of our CVPE module… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 6 canonical work pages

[1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
[2]

Time series forecasting of bed occupancy in mental health facilities in india using machine learning

Avinash, G., Pachori, H., Sharma, A., and Mishra, S. Time series forecasting of bed occupancy in mental health facilities in india using machine learning. Scientific Reports, 15 0 (2686), 2025. doi:10.1038/s41598-025-86418-9. URL https://www.nature.com/articles/s41598-025-86418-9

work page doi:10.1038/s41598-025-86418-9 2025
[3]

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, ...

arXiv 2020
[4]

End-to-end object detection with transformers, 2020

Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. End-to-end object detection with transformers, 2020. URL https://arxiv.org/abs/2005.12872

arXiv 2020
[5]

E., and Shah, K

Deb, C., Zhang, F., Yang, J., Lee, S. E., and Shah, K. W. A review on time series forecasting techniques for building energy consumption. Renewable and Sustainable Energy Reviews, 74: 0 902--924, 2017. doi:10.1016/j.rser.2017.02.085. URL https://daneshyari.com/article/preview/5483166.pdf

work page arXiv 2017
[6]

Bert: Pre-training of deep bidirectional transformers for language understanding, 2019

Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019. URL https://arxiv.org/abs/1810.04805

arXiv 2019
[7]

An image is worth 16x16 words: Transformers for image recognition at scale, 2021

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Transformers for image recognition at scale, 2021. URL https://arxiv.org/abs/2010.11929

arXiv 2021
[8]

Real-time Forecasting of Time Series in Financial Markets Using Sequentially Trained Many-to-one LSTMs

Gajamannage, K. and Park, Y. Real-time forecasting of time series in financial markets using sequentially trained many-to-one lstms, 2022. URL https://arxiv.org/abs/2205.04678

work page Pith review arXiv 2022

Show all 30 references

[9]

The capacity and robustness trade-off: Revisiting the channel independent strategy for multivariate time series forecasting, 2023

Han, L., Ye, H.-J., and Zhan, D.-C. The capacity and robustness trade-off: Revisiting the channel independent strategy for multivariate time series forecasting, 2023. URL https://arxiv.org/abs/2304.05206

2023 arXiv
[10]

Masked autoencoders are scalable vision learners, 2021

He, K., Chen, X., Xie, S., Li, Y., Dollár, P., and Girshick, R. Masked autoencoders are scalable vision learners, 2021. URL https://arxiv.org/abs/2111.06377

2021 arXiv
[11]

Y., Shi, X., Chen, P.-Y., Liang, Y., Li, Y.-F., Pan, S., and Wen, Q

Jin, M., Wang, S., Ma, L., Chu, Z., Zhang, J. Y., Shi, X., Chen, P.-Y., Liang, Y., Li, Y.-F., Pan, S., and Wen, Q. Time-llm: Time series forecasting by reprogramming large language models, 2024. URL https://arxiv.org/abs/2310.01728

2024 arXiv
[12]

Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization, 2017. URL https://arxiv.org/abs/1412.6980

2017 arXiv
[13]

Unitst: Effectively modeling inter-series and intra-series dependencies for multivariate time series forecasting, 2024 a

Liu, J., Liu, C., Woo, G., Wang, Y., Hooi, B., Xiong, C., and Sahoo, D. Unitst: Effectively modeling inter-series and intra-series dependencies for multivariate time series forecasting, 2024 a . URL https://arxiv.org/abs/2406.04975

2024 arXiv
[14]

X., and Dustdar, S

Liu, S., Yu, H., Liao, C., Li, J., Lin, W., Liu, A. X., and Dustdar, S. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=0EXmFzUn5I

2022
[15]

itransformer: Inverted transformers are effective for time series forecasting, 2024 b

Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., and Long, M. itransformer: Inverted transformers are effective for time series forecasting, 2024 b . URL https://arxiv.org/abs/2310.06625

2024 arXiv
[16]

Swin transformer: Hierarchical vision transformer using shifted windows, 2021

Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. Swin transformer: Hierarchical vision transformer using shifted windows, 2021. URL https://arxiv.org/abs/2103.14030

2021 arXiv
[17]

H., Sinthong, P., and Kalagnanam, J

Nie, Y., Nguyen, N. H., Sinthong, P., and Kalagnanam, J. A time series is worth 64 words: Long-term forecasting with transformers, 2023. URL https://arxiv.org/abs/2211.14730

2023 arXiv
[18]

A comprehensive survey of deep learning for multivariate time series forecasting: A channel strategy perspective, 2025

Qiu, X., Cheng, H., Wu, X., Hu, J., Guo, C., and Yang, B. A comprehensive survey of deep learning for multivariate time series forecasting: A channel strategy perspective, 2025. URL https://arxiv.org/abs/2502.10721

2025
[19]

Language models are unsupervised multitask learners

Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. Language models are unsupervised multitask learners. OpenAI, 2019. URL https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf

2019
[20]

Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer, 2023. URL https://arxiv.org/abs/1910.10683

2023 arXiv
[21]

Llama: Open and efficient foundation language models, 2023

Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., and Lample, G. Llama: Open and efficient foundation language models, 2023. URL https://arxiv.org/abs/2302.13971

2023 arXiv
[22]

Forecasting success: Achieving u.s

United States Senate Committee on Commerce, Science, and Transportation . Forecasting success: Achieving u.s. weather readiness for the long term. Hearing before the Subcommittee on Oceans, Atmosphere, Fisheries, and Coast Guard, December 2013. URL https://www.govinfo.gov/cont...

2013
[23]

N., Kaiser, L., and Polosukhin, I

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention is all you need, 2023. URL https://arxiv.org/abs/1706.03762

2023 arXiv
[24]

Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting, 2022

Wu, H., Xu, J., Wang, J., and Long, M. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting, 2022. URL https://arxiv.org/abs/2106.13008

2022 arXiv
[25]

Timesnet: Temporal 2d-variation modeling for general time series analysis, 2023

Wu, H., Hu, T., Liu, Y., Zhou, H., Wang, J., and Long, M. Timesnet: Temporal 2d-variation modeling for general time series analysis, 2023. URL https://arxiv.org/abs/2210.02186

2023 arXiv
[26]

Card: Channel aligned robust blend transformer for time series forecasting, 2024

Xue, W., Zhou, T., Wen, Q., Gao, J., Ding, B., and Jin, R. Card: Channel aligned robust blend transformer for time series forecasting, 2024. URL https://arxiv.org/abs/2305.12095

2024 arXiv
[27]

Are transformers effective for time series forecasting?, 2022

Zeng, A., Chen, M., Zhang, L., and Xu, Q. Are transformers effective for time series forecasting?, 2022. URL https://arxiv.org/abs/2205.13504

2022 arXiv
[28]

and Yan, J

Zhang, Y. and Yan, J. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=vSVLM2j9eie

2023
[29]

Informer: Beyond efficient transformer for long sequence time-series forecasting, 2021

Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W. Informer: Beyond efficient transformer for long sequence time-series forecasting, 2021. URL https://arxiv.org/abs/2012.07436

2021 arXiv
[30]

Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting, 2022

Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, L., and Jin, R. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting, 2022. URL https://arxiv.org/abs/2201.12740

2022 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

[2] [2]

Time series forecasting of bed occupancy in mental health facilities in india using machine learning

Avinash, G., Pachori, H., Sharma, A., and Mishra, S. Time series forecasting of bed occupancy in mental health facilities in india using machine learning. Scientific Reports, 15 0 (2686), 2025. doi:10.1038/s41598-025-86418-9. URL https://www.nature.com/articles/s41598-025-86418-9

work page doi:10.1038/s41598-025-86418-9 2025

[3] [3]

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, ...

arXiv 2020

[4] [4]

End-to-end object detection with transformers, 2020

Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. End-to-end object detection with transformers, 2020. URL https://arxiv.org/abs/2005.12872

arXiv 2020

[5] [5]

E., and Shah, K

Deb, C., Zhang, F., Yang, J., Lee, S. E., and Shah, K. W. A review on time series forecasting techniques for building energy consumption. Renewable and Sustainable Energy Reviews, 74: 0 902--924, 2017. doi:10.1016/j.rser.2017.02.085. URL https://daneshyari.com/article/preview/5483166.pdf

work page arXiv 2017

[6] [6]

Bert: Pre-training of deep bidirectional transformers for language understanding, 2019

Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019. URL https://arxiv.org/abs/1810.04805

arXiv 2019

[7] [7]

An image is worth 16x16 words: Transformers for image recognition at scale, 2021

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Transformers for image recognition at scale, 2021. URL https://arxiv.org/abs/2010.11929

arXiv 2021

[8] [8]

Real-time Forecasting of Time Series in Financial Markets Using Sequentially Trained Many-to-one LSTMs

Gajamannage, K. and Park, Y. Real-time forecasting of time series in financial markets using sequentially trained many-to-one lstms, 2022. URL https://arxiv.org/abs/2205.04678

work page Pith review arXiv 2022