LeMoLE: LLM-Enhanced Mixture of Linear Experts for Time Series Forecasting

Fugee Tsung; Lifeng Shen; Lingzheng Zhang; Shiyuan Piao; Yimin Zheng; Ziyue Li

REVIEW 4 major objections 6 minor 12 references

LeMoLE: LLM-Enhanced Mixture of Linear Experts for Time Series Forecasting

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that LeMoLE, a mixture of linear experts with varied lookback lengths conditioned on frozen GPT-2 text embeddings, achieves lower prediction error and higher computational efficiency than existing LLM-alignment forecasters.

desk verdict Sensible idea—frozen-LLM text conditioning for a mixture of linear experts—but the paper's numbers and architecture description are internally inconsistent, so the main accuracy claim is currently unverifiable. read the letter →

arxiv 2412.00053 v1 pith:VEJ5UP3Y submitted 2024-11-24 cs.LG cs.AIcs.CL

Lingzheng Zhang , Lifeng Shen , Yimin Zheng , Shiyuan Piao , Ziyue Li , Fugee Tsung This is my paper · ORCID

classification cs.LGcs.AIcs.CL

keywords timeseriesforecastingmixtureoflinearexpertslargelanguagemodelsmultimodalfusionFiLMconditioninglong-termfew-shotcomputationalefficiency

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a time-series forecaster can get the benefits of language knowledge without paying the cost of aligning time series into an LLM's semantic space. LeMoLE trains a small set of linear experts, each on a different lookback window, and fuses their forecasts with text features extracted by a frozen GPT-2 model from two prompts: a static dataset description and a dynamic list of timestamps. Across Electricity, Traffic, ETTh1 and ETTm1, the paper reports lower MSE than MoLE, TimeMixer, GPT4TS and AutoTimes in both long-range and few-shot settings, with far fewer parameters and faster inference. The implication, if true, is that lightweight text-conditioned linear ensembles can outperform alignment-based LLM forecasters on accuracy and efficiency at once.

What carries the argument

The load-bearing object is the mixture of linear experts with varying lookback lengths, Equation (3), where the $m$-th expert maps $X_{T-w_m:T}$ through a $W_m \in \mathbb{R}^{H \times w_m}$ matrix to a forecast. The second load-bearing object is the FiLM conditioning module: static prompt text $P_S$ and dynamic prompt text $P_D$ are encoded by a frozen GPT-2 into $Z_S, Z_D$, and two lightweight FiLM layers produce per-channel scale and shift, $\gamma \odot Y + \beta$, that modulate the ensemble before a CNN fuses the branches. FiLM, feature-wise linear modulation, computes these scale and shift parameters from a conditioning representation. This pair of mechanisms lets the model adapt expert weighting to the text while keeping the per-step computation linear in the series length, avoiding the self-attention cost of alignment-based LLM forecasters.

What would settle it

Remove the language model entirely and replace $Z_S$ and $Z_D$ with random vectors of the same shape, retraining the rest unchanged; if the MSE does not worsen, the text modality is not carrying the improvement. A second decisive check is to give MoLE the same number of experts and the same total parameter budget as LeMoLE but with identical lookback lengths; if LeMoLE still wins, the varying lookback lengths, not the multimodal fusion, explain the gains.

Watch

Extended reading notes

Core claim

The central claim is stated in Section 4.2: LeMoLE presents lower prediction errors and higher computational efficiency than existing LLM models. Concretely, the paper reports average MSE improvements of 23.17% over MoLE, 20.70% over TimeMixer, 11.76% over GPT4TS, and 29.85% over AutoTimes on four standard benchmarks, and in the few-shot 10%-training setting it again finishes first on average MSE. The efficiency table shows LeMoLE-T with about 0.5M parameters at H=96 and 3.85M at H=720, versus 3.9M-58M for LLM baselines, with inference times around 1-3 ms compared to 4-23 ms for GPT4TS and TimeLLM. The authors interpret this as evidence that aligning time series into LLM spaces is not necessary: a mixture of simple linear experts with different lookback lengths, modulated by text embeddings, can be both more accurate and much cheaper.

Load-bearing premise

The load-bearing premise is that the hand-written dataset descriptions and timestamp strings, once encoded by the frozen GPT-2, contain information that the FiLM layers can usefully inject; if those text features are uninformative, LeMoLE reduces to a mixture of linear experts with different lookback lengths, and the claimed advantage would rest on that architectural change alone.

Editorial extensions

If this is right

If the reported gains hold, forecasting practice can drop end-to-end time-series alignment with LLMs and instead use a frozen text encoder as a cheap conditioning signal.
Varying lookback lengths among experts is a simple way to make a linear ensemble capture both short- and long-term patterns, improving on a single-length mixture like MoLE.
Prompt content matters: the dynamic timestamp prompt contributes most on non-stationary datasets, while the static dataset description contributes more on the periodic Electricity data.
The small parameter count and millisecond-level inference of LeMoLE-T make the approach suitable for long-horizon and few-shot settings where LLM-alignment models are too slow or data-hungry.
The LeMoLE-F variant shows that replacing time-domain linear experts with frequency-domain FITS experts hurts accuracy, so in this architecture time-domain experts should be preferred.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural testable extension is to scramble or randomize the text prompts; if LeMoLE keeps its MSE, the gains would come from the variable-lookback mixture and FiLM conditioning rather than from language understanding.
Because the model only needs a dataset description and timestamps, it could be applied to domains with richer semantics, such as weather forecasting with textual forecasts as dynamic prompts, where the text channel might carry more signal than the four benchmarks show.
The reported comparison pits LeMoLE against full LLM-alignment models; a fairer assessment of the 'rethinking' claim would also match parameter budgets and training cost against a strong linear baseline with the same number of experts.
The four benchmark datasets are all electricity or traffic related; a broader benchmark with varied domains would clarify how much of the 23-30% MSE advantage comes from prompt informativeness versus the mixture architecture.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The idea here is worth a look: instead of aligning time series into LLM embedding space, LeMoLE keeps a frozen GPT-2 as a text encoder and uses FiLM conditioning to let static and dynamic prompts modulate a mixture of linear experts with different lookback lengths. That is a clean, cheap way to test whether text knowledge helps forecasting at all. The efficiency tables are the most convincing part — LeMoLE is orders of magnitude faster at inference than Time-LLM or AutoTimes, which is exactly what you'd expect since the LLM does no autoregressive generation.

The problem is that the paper's central accuracy claim is not supported by its own numbers. Section 4.2 reports average MSE improvements of 23.17% over MoLE and 20.70% over TimeMixer. Recomputing from the All Avg row in Table 2 gives 14.8% and 27.7% — neither matches. Same for the LLM baselines. That is not rounding. Then Table 4 lists LeMoLE at 0.0527 on ETTh1, H=336, while Table 2 shows 0.079 for the same cell. Table 10 shows ECL H=96 at 0.297 with std 0.087, while Table 2 has 0.197. At least one set of tables is describing a different evaluation run, and the paper does not explain why.

Worse, the method itself is described two ways. Equations (6)-(8) combine the M expert outputs with a CNN, apply FiLM to that combined representation, then pass through a final CNN. Algorithm 1 instead applies FiLM to each expert output separately and only then ensembles them via CNN. These are different models. Since the paper never says which one produced Table 2, the reported gains are not tied to a well-defined architecture. There is also no code, no error bars on the baselines, and no specification of the expert lookback lengths w_m.

None of this kills the underlying concept. The ablation logic is sensible, the frequency-domain extension adds useful perspective, and the efficiency advantage is structural and likely robust. But the accuracy claims need to survive a reconciliation pass: correct the percentages, make the tables agree, align the pseudocode with the equations, and release code.

My take: send it to review, but only with a referee who will check the numbers. I would not cite it as-is.

Referee Report

4 major / 6 minor

Summary. The paper proposes LeMoLE, a time-series forecasting architecture that combines a mixture of linear experts with varying lookback lengths and a multimodal fusion mechanism that conditions expert outputs on features extracted by a frozen GPT-2 from static and dynamic text prompts. The authors claim that LeMoLE achieves both lower prediction error and higher computational efficiency than existing alignment-based LLM forecasting models such as GPT4TS, AutoTimes, and Time-LLM. Experiments are reported on ETTh1, ETTm1, Electricity, and Traffic for long-term and few-shot forecasting, together with ablations, robustness runs, and an efficiency table.

Significance. If the reported results are correct, the paper makes a useful contribution: it provides a lightweight alternative to LLM-alignment forecasting that preserves accuracy while drastically cutting inference cost. The efficiency results in Table 5 are clear and are a genuine strength. The central qualitative claim that a text-conditioned mixture of linear experts can outperform heavier LLM-based forecasters is plausible and interesting. However, the paper's own tables contain multiple internal contradictions, and the architecture is not described consistently between the main text and the appendix. Because the reported average improvements in Section 4.2 cannot be reproduced from Table 2, and because Table 4 and Table 10 disagree with Table 2, the specific quantitative claims are not currently verifiable. The paper needs a careful correction of the reported numbers and a precise specification of the model before its claims can be accepted.

major comments (4)

[Section 4.2, Table 2] The reported average MSE improvements do not match Table 2. The text states average improvements of 23.17% over MoLE, 20.70% over TimeMixer, 11.76% over GPT4TS, and 29.85% over AutoTimes. Recomputing from the All Avg row of Table 2 (Ours 0.115, MoLE 0.135, TimeMixer 0.159, GPT4TS 0.137, AutoTimes 0.182) gives about 14.8%, 27.7%, 16.1%, and 36.8%, respectively. These discrepancies are far larger than rounding. The authors should either correct the percentages or explain the exact subset of settings over which the percentages were computed.
[Table 4 vs. Table 2] The ablation table reports numbers that conflict with the main results. For ETTh1 with horizon 336, Table 4 lists Ours = 0.0527, while Table 2 lists Ours = 0.079 for the same setting; the claimed degradations of 0.57%, 1.71%, and 2.09% are therefore computed against a value that appears nowhere else in the paper. Similarly, Table 10 reports ECL H=96 as 0.297 with std 0.087, while Table 2 reports 0.197. Since Table 4 is the main evidence that the text prompts matter, and Table 10 is the evidence for stability, these contradictions make both claims unverifiable as presented. The authors must clarify which evaluation produced each table.
[Appendix A, Algorithm 1 vs. Section 3.3] Algorithm 1 describes a different model from the one defined by Equations (6)-(8). In the main text, the CNN first combines the expert outputs into Y, and FiLM conditioning is applied once to this combined representation; the final prediction is then CNNfinal([Y; Y_S'; Y_D']). In Algorithm 1, FiLM conditioning is applied to each expert output individually and the final CNN combines the conditioned expert outputs. These are architecturally distinct. The paper does not state which implementation produced Table 2 and the other experimental tables, so the reported results are not attributable to a well-defined method. This must be fixed by aligning the pseudocode with the equations or by explicitly stating which variant was used.
[Section 4.2 and Table 4] The claim that multimodal text information is responsible for the gains rests on the ablation results in Table 4, but those results are internally inconsistent with Table 2, as noted above. If the ETTh1 value in Table 4 is correct, then the main-table value is wrong; if the main-table value is correct, then the ablation is not measuring the model whose results are reported in Table 2. Either way, the paper's central claim that text features improve forecasting is not currently supported by consistent evidence. The authors should rerun or correct both tables and confirm that the qualitative conclusions survive.

minor comments (6)

[Section 4.1, Datasets] The word 'conider' should be 'consider'.
[Section 3.3, last paragraph] The sentence 'The pseudocode for the training procedures of the backward denoising process can be found in Appendix A' refers to a 'backward denoising process' that does not exist in this paper; this should be rephrased to 'training procedure'.
[Section 4.1, Implementation details] The reference 'Appendix ??' is unresolved; the authors should cite the correct appendix for baseline configuration details.
[Table 6] Table 6 lists ETTm1 as '1 min', but the paper's own dataset description and the standard ETT convention state that ETTm1 is sampled every 15 minutes.
[Table 7 and Table 8 captions] The captions say 'univariate time series', but the experiments use multivariate datasets with channel independence; the wording should be clarified to avoid confusion.
[Table 4] The header of Table 4 is difficult to parse: the columns labeled 'Dataset ETTh Electricity ETT Electricity' mix dataset names with task names. The table should be reorganized so that each column clearly states the dataset and the task.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: LeMoLE is an end-to-end empirical architecture paper, and no claimed result reduces by construction to a fitted parameter or to a self-citation.

full rationale

The paper's derivation chain is a standard end-to-end training setup: Equations (1)-(9) define a forecasting model with trainable expert weights, FiLM conditioning parameters, and a convolutional fusion block, and Equation (9) minimizes MSE between predictions and ground truth on training data. The text features from the frozen GPT-2 encoder are inputs to the model, not quantities fitted to the evaluation targets, so the multimodal fusion claims are not predictions that reduce to their own inputs. The history length and number of experts are tuned on validation data, which is conventional model selection rather than circular reasoning. The paper contains no load-bearing self-citations: the cited baselines and techniques (MoLE, TimeMixer, AutoTimes, GPT4TS, FITS, FiLM) are external prior work by other authors, and there is no invoked uniqueness theorem or ansatz smuggled in through the authors' own prior publications. The reported discrepancies between Section 4.2's improvement percentages and Table 2, and the inconsistencies between Tables 2, 4, and 10, are serious correctness and reproducibility concerns, but they are not circularity: an inconsistent or misreported empirical result is not equivalent by definition to the model's inputs or fitted parameters. Likewise, the mismatch between Algorithm 1 and Equations (6)-(8) indicates an underspecified or inconsistent architecture description, not a self-referential derivation. No specific equation or result in the paper reduces to a fitted constant, a renamed empirical pattern, or a self-citation chain. Accordingly, the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

LeMoLE introduces no new physical or conceptual entities; it is a combination of standard linear layers, CNNs, FiLM conditioning, and a frozen LLM. The free parameters are architecture choices (history length, number of experts, lookback lengths) tuned on validation data. The key domain assumptions concern the usefulness of text features and the validity of univariate evaluation.

free parameters (3)

History length T = Searched from {96, 192, 336, 512, 672, 1024} per method
Selected by validation MSE for each method and dataset; differences in T can affect comparisons.
Number of experts M = 1-5 depending on dataset
Chosen per dataset based on validation performance, as shown in Figure 4 and Table 9.
Expert lookback lengths w_m = Not reported
The paper assumes w1 >= w2 >= ... >= wM but does not specify the actual lengths used, making them an implicit tuning choice.

assumptions (4)

domain assumption Linear projections over sufficient lookback windows can capture short-term and long-term temporal patterns in time series.
Invoked in Section 3.1, Eq. (3), where each expert is a linear map from a lookback window to the forecast horizon.
domain assumption FiLM (feature-wise linear modulation) can effectively inject text-derived features into time series prediction layers.
Used in Section 3.3, Eqs. (6)-(7), to scale and shift the CNN-aggregated prediction tensor using static and dynamic text representations.
domain assumption Frozen GPT-2 embeddings of hand-crafted static and dynamic prompts are informative for time series forecasting.
Assumed in Section 3.2, Eqs. (4)-(5), where the prompts are encoded and used as conditions; the ablation study attempts to validate this, but its numbers conflict with the main results.
domain assumption Single-channel (univariate) evaluation is a sufficient proxy for multivariate time series forecasting performance.
Stated in Section 4.1, following Jia et al. (2023); the paper evaluates on a single channel, which may not match how multivariate baselines are normally benchmarked.

how reviews work

0 comments

Cite this review

Pith. "Pith review of LeMoLE: LLM-Enhanced Mixture of Linear Experts for Time Series Forecasting." pith.science (2026). https://pith.science/paper/VEJ5UP3Y

@misc{pith2026241200053,
  author       = {Pith},
  title        = {Pith review of: LeMoLE: LLM-Enhanced Mixture of Linear Experts for Time Series Forecasting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VEJ5UP3Y}},
  note         = {Machine review of arXiv:2412.00053}
}

read the original abstract

Recent research has shown that large language models (LLMs) can be effectively used for real-world time series forecasting due to their strong natural language understanding capabilities. However, aligning time series into semantic spaces of LLMs comes with high computational costs and inference complexity, particularly for long-range time series generation. Building on recent advancements in using linear models for time series, this paper introduces an LLM-enhanced mixture of linear experts for precise and efficient time series forecasting. This approach involves developing a mixture of linear experts with multiple lookback lengths and a new multimodal fusion mechanism. The use of a mixture of linear experts is efficient due to its simplicity, while the multimodal fusion mechanism adaptively combines multiple linear experts based on the learned features of the text modality from pre-trained large language models. In experiments, we rethink the need to align time series to LLMs by existing time-series large language models and further discuss their efficiency and effectiveness in time series forecasting. Our experimental results show that the proposed LeMoLE model presents lower prediction errors and higher computational efficiency than existing LLM models.

Figures

Figures reproduced from arXiv: 2412.00053 by the authors.

**Figure 1.** Inference time on ETTh1. In addition, incorporating multimodal knowledge into predictive models is also a promising solution. Recently, there has been a significant surge of interest in multimodal time series forecasting. For example, TimeLLM (Jin et al., 2024) aims to align the modalities of time series data and natural language such that the capabilities of pretrained large language model (LLM) from natural lan… view at source ↗

**Figure 2.** The proposed LeMoLE is based on a mixture of linear experts with different look [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** MSE resulrs of time vs. frequency experts. 4.5 EFFECTS OF THE NUMBER OF EXPERTS (RQ4) In this experiment, we analyze the effects of the number of experts in the proposed LeMoLE. In [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Average MSE’s with varying numbers of experts in LeMoLE. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]

**Figure 5.** Figure 5: Text prompt examples on ETT dataset. C SUPPLEMENTARY OF DATASETS Dataset Sampling Frequency Total Observations Dimension Electricity 1 hour 26,304 321 Traffic 1 hour 17,544 862 ETTh1 1 hour 17,544 7 ETTm1 1 min 69,680 7 [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

**Figure 6.** Figure 6: Prediction results on Electricity. 0 50 100 150 200 250 300 350 400 1.2 1.0 0.8 0.6 0.4 History GroundTruth Prediction (a) SparseTSF 0 50 100 150 200 250 300 350 400 1.2 1.0 0.8 0.6 0.4 History GroundTruth Prediction (b) iTransformer 0 50 100 150 200 250 300 350 400 1.…

**Figure 7.** Figure 7: Prediction results on ETTh1. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 6 canonical work pages

[3]

Revisiting long-term time series forecasting: An investigation on linear mapping

Zhe Li, Shiyi Qi, Yiduo Li, and Zenglin Xu. Revisiting long-term time series forecasting: An investigation on linear mapping. Preprint arXiv:2305.10721,

arXiv
[4]

Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and fore- casting

Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and fore- casting. In ICLR, 2022a. Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. NeurIPS, 3...

arXiv
[5]

Llama: Open and efficient foundation language models

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth ´ee Lacroix, Baptiste Rozi `ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. Preprint arXiv:2302.13971,

arXiv
[9]

In contrast, the Traffic dataset shows consistent improvements up to three experts

In the Electricity dataset, LeMoLE improves up to three experts, but additional experts add com- plexity without accuracy gains. In contrast, the Traffic dataset shows consistent improvements up to three experts. For the ETTh1 and ETTm1 datasets, they are nonstationary and present highly nonlinear behaviors, suggesting these datasets require more experts ...

work page arXiv
[1970]

Language models are few-shot learners

Tom B Brown. Language models are few-shot learners. Preprint arXiv:2005.14165,

arXiv 2005
[2014]

For example, to reduce the time complexity and memory usage, Informer (Wen et al.,

D S UPPLEMENTARY OF BASELINES Recent transformer variants aim to improve the standard transformer structure for time series mod- eling (Wen et al., 2022; Zhou et al., 2021; Wu et al., 2023). For example, to reduce the time complexity and memory usage, Informer (Wen et al.,

work page 2022
[2016]

The Electricity 4 compiles hourly records of energy usage from a cohort of 321 individual clients, spanning a three-year time-frame between 2012 and

work page 2012
[2018]

The Traffic 3 provides insights into road congestion patterns by detail- ing occupancy rates along San Francisco’s freeway network, and encompasses hourly measurements spanning from 2015 through

work page 2015

Show all 12 references

[2021]

Both subsets span the period from July 2016 to July

encompasses a comprehensive collection of transformer operational data, consisting of two subsets:ETTh, featuring hourly record- ings, and ETTm, with data collected at a finer 15-minute interval. Both subsets span the period from July 2016 to July

2016
[2022]

designs the enhanced blocks with Fourier transform and wavelet transform, enabling the focus on capturing important structures in time series through frequency do- main mapping. Recently, PatchTST (Nie et al., 2023a) segments time series into patches that serve as input tokens...
[2023]

Less is more: Fast multivariate time series forecasting with light sampling-oriented mlp structures

Tianping Zhang, Yizhuo Zhang, Wei Cao, Jiang Bian, Xiaohan Yi, Shun Zheng, and Jian Li. Less is more: Fast multivariate time series forecasting with light sampling-oriented mlp structures. preprint arXiv:2207.01186,

arXiv
[2024]

Llm4ts: Two-stage fine-tuning for time-series forecasting with pre-trained llms

Ching Chang, Wen-Chih Peng, and Tien-Fu Chen. Llm4ts: Two-stage fine-tuning for time-series forecasting with pre-trained llms. Preprint arXiv:2308.08469,

arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [3]

Revisiting long-term time series forecasting: An investigation on linear mapping

Zhe Li, Shiyi Qi, Yiduo Li, and Zenglin Xu. Revisiting long-term time series forecasting: An investigation on linear mapping. Preprint arXiv:2305.10721,

arXiv

[2] [4]

Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and fore- casting

Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and fore- casting. In ICLR, 2022a. Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. NeurIPS, 3...

arXiv

[3] [5]

Llama: Open and efficient foundation language models

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth ´ee Lacroix, Baptiste Rozi `ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. Preprint arXiv:2302.13971,

arXiv

[4] [9]

In contrast, the Traffic dataset shows consistent improvements up to three experts

In the Electricity dataset, LeMoLE improves up to three experts, but additional experts add com- plexity without accuracy gains. In contrast, the Traffic dataset shows consistent improvements up to three experts. For the ETTh1 and ETTm1 datasets, they are nonstationary and present highly nonlinear behaviors, suggesting these datasets require more experts ...

work page arXiv

[5] [1970]

Language models are few-shot learners

Tom B Brown. Language models are few-shot learners. Preprint arXiv:2005.14165,

arXiv 2005

[6] [2014]

For example, to reduce the time complexity and memory usage, Informer (Wen et al.,

D S UPPLEMENTARY OF BASELINES Recent transformer variants aim to improve the standard transformer structure for time series mod- eling (Wen et al., 2022; Zhou et al., 2021; Wu et al., 2023). For example, to reduce the time complexity and memory usage, Informer (Wen et al.,

work page 2022

[7] [2016]

The Electricity 4 compiles hourly records of energy usage from a cohort of 321 individual clients, spanning a three-year time-frame between 2012 and

work page 2012

[8] [2018]

The Traffic 3 provides insights into road congestion patterns by detail- ing occupancy rates along San Francisco’s freeway network, and encompasses hourly measurements spanning from 2015 through

work page 2015