{"id":"4fc2ed3b-ac7c-454f-92e9-066e02b18aa2","arxiv_id":"2412.00053","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"LeMoLE, a mixture of linear experts with varying lookback lengths and text-conditioned fusion, outperforms LLM-based forecasters on four benchmarks with lower computational cost.","lead":"LeMoLE is a forecasting model that combines several simple linear predictors, each looking at a different length of history, with text descriptions fed through a frozen large language model. It reports lower error and faster inference than LLM-based forecasters on four benchmark datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central accuracy claim is not supported by the paper's own numbers: Section 4.2's percentages don't match Table 2, and Tables 4 and 10 contradict Table 2; Algorithm 1 also conflicts with Eqs. (6)-(8).","rationale":"The central claim is empirical: LeMoLE is more accurate and faster than LLM-alignment baselines. The accuracy half rests entirely on Table 2. That table cannot be cross-checked because the paper's own derived percentages (Section 4.2) are not consistent with it, and the ablation and stability tables report numbers for the same settings that differ substantially. If the table entries are unreliable, the claim of lower prediction errors is unverified, regardless of how sensible the architecture is. The efficiency half is supported by Table 5, but the reported speedups also depend on the exact model variant; the conflict between Algorithm 1 and Eqs. (6)-(8) means the implementation used for Table 5 is ambiguous. These are not minor stylistic issues: they are load-bearing because they determine whether the experimental evidence means anything. I am not claiming the results are fabricated; I am claiming the paper as written does not allow a reader to determine what was actually evaluated. The appropriate disposition is to require code and a corrected, internally consistent set of tables before the accuracy claim can be accepted. This matches the reader's CONDITIONAL verdict; the reader's identified weakest assumption (text features are informative) is plausible and worth testing with a prompt-ablation or permutation test, but the more immediate threat to the central claim is the unreliability of the quantitative evidence itself.","tokens_in":23911,"tokens_out":10952,"duration_ms":88539,"concrete_test":"Recompute the percentage improvements in Section 4.2 directly from Table 2. Then, under the exact protocol of Section 4.1 (same data split, lookback search, channel-independence), run the released implementation on ETTh1 H=336 and ECL H=96 and check whether the MSEs match Table 2 (0.079, 0.197), Table 4 (0.0527), or Table 10 (0.297, 0.087). Also implement both Algorithm 1 and the Eqs. (6)-(8) fusion order; if their test MSEs differ beyond seed noise, the paper must state which variant was used for Table 2.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To support the central claim of lower prediction errors, Section 4.2 reports average MSE improvements of 23.17% over MoLE and 20.70% over TimeMixer, and 11.76% and 29.85% over GPT4TS and AutoTimes. Recomputing from Table 2's All Avg row (Ours 0.115, MoLE 0.135, TimeMixer 0.159, GPT4TS 0.137, AutoTimes 0.182) gives improvements of 14.8%, 27.7%, 16.1%, and 36.8%, respectively; none match. The discrepancy is not a rounding issue. Additionally, Table 4 lists Ours=0.0527 for ETTh1 with horizon 336, while Table 2 lists 0.079 for the same horizon; Table 10's ECL H=96 entry is 0.297 with std 0.087, versus 0.197 in Table 2. At least one of these tables cannot describe the same evaluation. Finally, Algorithm 1 applies FiLM conditioning to each expert's output before ensembling, whereas Eqs. (6)-(8) first combine experts with a CNN and then apply FiLM to the combined representation; these are different models. Because the paper does not specify which architecture produced Table 2, the reported gains are not attributable to a well-defined method.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LeMoLE, a time-series forecasting architecture that combines a mixture of linear experts with varying lookback lengths and a multimodal fusion mechanism that conditions expert outputs on features extracted by a frozen GPT-2 from static and dynamic text prompts. The authors claim that LeMoLE achieves both lower prediction error and higher computational efficiency than existing alignment-based LLM forecasting models such as GPT4TS, AutoTimes, and Time-LLM. Experiments are reported on ETTh1, ETTm1, Electricity, and Traffic for long-term and few-shot forecasting, together with ablations, robustness runs, and an efficiency table.","tokens_in":24136,"tokens_out":3044,"duration_ms":26305,"significance":"If the reported results are correct, the paper makes a useful contribution: it provides a lightweight alternative to LLM-alignment forecasting that preserves accuracy while drastically cutting inference cost. The efficiency results in Table 5 are clear and are a genuine strength. The central qualitative claim that a text-conditioned mixture of linear experts can outperform heavier LLM-based forecasters is plausible and interesting. However, the paper's own tables contain multiple internal contradictions, and the architecture is not described consistently between the main text and the appendix. Because the reported average improvements in Section 4.2 cannot be reproduced from Table 2, and because Table 4 and Table 10 disagree with Table 2, the specific quantitative claims are not currently verifiable. The paper needs a careful correction of the reported numbers and a precise specification of the model before its claims can be accepted.","major_comments":[{"comment":"The reported average MSE improvements do not match Table 2. The text states average improvements of 23.17% over MoLE, 20.70% over TimeMixer, 11.76% over GPT4TS, and 29.85% over AutoTimes. Recomputing from the All Avg row of Table 2 (Ours 0.115, MoLE 0.135, TimeMixer 0.159, GPT4TS 0.137, AutoTimes 0.182) gives about 14.8%, 27.7%, 16.1%, and 36.8%, respectively. These discrepancies are far larger than rounding. The authors should either correct the percentages or explain the exact subset of settings over which the percentages were computed.","section":"Section 4.2, Table 2"},{"comment":"The ablation table reports numbers that conflict with the main results. For ETTh1 with horizon 336, Table 4 lists Ours = 0.0527, while Table 2 lists Ours = 0.079 for the same setting; the claimed degradations of 0.57%, 1.71%, and 2.09% are therefore computed against a value that appears nowhere else in the paper. Similarly, Table 10 reports ECL H=96 as 0.297 with std 0.087, while Table 2 reports 0.197. Since Table 4 is the main evidence that the text prompts matter, and Table 10 is the evidence for stability, these contradictions make both claims unverifiable as presented. The authors must clarify which evaluation produced each table.","section":"Table 4 vs. Table 2"},{"comment":"Algorithm 1 describes a different model from the one defined by Equations (6)-(8). In the main text, the CNN first combines the expert outputs into Y, and FiLM conditioning is applied once to this combined representation; the final prediction is then CNNfinal([Y; Y_S'; Y_D']). In Algorithm 1, FiLM conditioning is applied to each expert output individually and the final CNN combines the conditioned expert outputs. These are architecturally distinct. The paper does not state which implementation produced Table 2 and the other experimental tables, so the reported results are not attributable to a well-defined method. This must be fixed by aligning the pseudocode with the equations or by explicitly stating which variant was used.","section":"Appendix A, Algorithm 1 vs. Section 3.3"},{"comment":"The claim that multimodal text information is responsible for the gains rests on the ablation results in Table 4, but those results are internally inconsistent with Table 2, as noted above. If the ETTh1 value in Table 4 is correct, then the main-table value is wrong; if the main-table value is correct, then the ablation is not measuring the model whose results are reported in Table 2. Either way, the paper's central claim that text features improve forecasting is not currently supported by consistent evidence. The authors should rerun or correct both tables and confirm that the qualitative conclusions survive.","section":"Section 4.2 and Table 4"}],"minor_comments":[{"comment":"The word 'conider' should be 'consider'.","section":"Section 4.1, Datasets"},{"comment":"The sentence 'The pseudocode for the training procedures of the backward denoising process can be found in Appendix A' refers to a 'backward denoising process' that does not exist in this paper; this should be rephrased to 'training procedure'.","section":"Section 3.3, last paragraph"},{"comment":"The reference 'Appendix ??' is unresolved; the authors should cite the correct appendix for baseline configuration details.","section":"Section 4.1, Implementation details"},{"comment":"Table 6 lists ETTm1 as '1 min', but the paper's own dataset description and the standard ETT convention state that ETTm1 is sampled every 15 minutes.","section":"Table 6"},{"comment":"The captions say 'univariate time series', but the experiments use multivariate datasets with channel independence; the wording should be clarified to avoid confusion.","section":"Table 7 and Table 8 captions"},{"comment":"The header of Table 4 is difficult to parse: the columns labeled 'Dataset ETTh Electricity ETT Electricity' mix dataset names with task names. The table should be reorganized so that each column clearly states the dataset and the task.","section":"Table 4"}],"recommendation":"major_revision","confidential_remarks":"The internal inconsistencies are substantial enough that I would not recommend acceptance in the current form. The efficiency claim appears well supported, but the accuracy claims need a full re-check of the tables and a clarified architecture description. I would ask the authors to provide corrected tables and to specify exactly which implementation (Eqs. 6-8 or Algorithm 1) was used in each experiment."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe idea here is worth a look: instead of aligning time series into LLM embedding space, LeMoLE keeps a frozen GPT-2 as a text encoder and uses FiLM conditioning to let static and dynamic prompts modulate a mixture of linear experts with different lookback lengths. That is a clean, cheap way to test whether text knowledge helps forecasting at all. The efficiency tables are the most convincing part — LeMoLE is orders of magnitude faster at inference than Time-LLM or AutoTimes, which is exactly what you'd expect since the LLM does no autoregressive generation.\n\nThe problem is that the paper's central accuracy claim is not supported by its own numbers. Section 4.2 reports average MSE improvements of 23.17% over MoLE and 20.70% over TimeMixer. Recomputing from the All Avg row in Table 2 gives 14.8% and 27.7% — neither matches. Same for the LLM baselines. That is not rounding. Then Table 4 lists LeMoLE at 0.0527 on ETTh1, H=336, while Table 2 shows 0.079 for the same cell. Table 10 shows ECL H=96 at 0.297 with std 0.087, while Table 2 has 0.197. At least one set of tables is describing a different evaluation run, and the paper does not explain why.\n\nWorse, the method itself is described two ways. Equations (6)-(8) combine the M expert outputs with a CNN, apply FiLM to that combined representation, then pass through a final CNN. Algorithm 1 instead applies FiLM to each expert output separately and only then ensembles them via CNN. These are different models. Since the paper never says which one produced Table 2, the reported gains are not tied to a well-defined architecture. There is also no code, no error bars on the baselines, and no specification of the expert lookback lengths w_m.\n\nNone of this kills the underlying concept. The ablation logic is sensible, the frequency-domain extension adds useful perspective, and the efficiency advantage is structural and likely robust. But the accuracy claims need to survive a reconciliation pass: correct the percentages, make the tables agree, align the pseudocode with the equations, and release code.\n\nMy take: send it to review, but only with a referee who will check the numbers. I would not cite it as-is.","headline":"Sensible idea—frozen-LLM text conditioning for a mixture of linear experts—but the paper's numbers and architecture description are internally inconsistent, so the main accuracy claim is currently unverifiable.","tokens_in":24746,"tokens_out":3553,"would_cite":false,"duration_ms":28974,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that LeMoLE, a mixture of linear experts with varied lookback lengths conditioned on frozen GPT-2 text embeddings, achieves lower prediction error and higher computational efficiency than existing LLM-alignment forecasters.","keywords":["time series forecasting","mixture of linear experts","large language models","multimodal fusion","FiLM conditioning","long-term forecasting","few-shot forecasting","computational efficiency"],"falsifier":"Remove the language model entirely and replace $Z_S$ and $Z_D$ with random vectors of the same shape, retraining the rest unchanged; if the MSE does not worsen, the text modality is not carrying the improvement. A second decisive check is to give MoLE the same number of experts and the same total parameter budget as LeMoLE but with identical lookback lengths; if LeMoLE still wins, the varying lookback lengths, not the multimodal fusion, explain the gains.","tokens_in":23662,"feed_emoji":"📉","tokens_out":7815,"duration_ms":62500,"temperature":0.7,"pith_summary":"This paper tries to establish that a time-series forecaster can get the benefits of language knowledge without paying the cost of aligning time series into an LLM's semantic space. LeMoLE trains a small set of linear experts, each on a different lookback window, and fuses their forecasts with text features extracted by a frozen GPT-2 model from two prompts: a static dataset description and a dynamic list of timestamps. Across Electricity, Traffic, ETTh1 and ETTm1, the paper reports lower MSE than MoLE, TimeMixer, GPT4TS and AutoTimes in both long-range and few-shot settings, with far fewer parameters and faster inference. The implication, if true, is that lightweight text-conditioned linear ensembles can outperform alignment-based LLM forecasters on accuracy and efficiency at once.","feed_headline":"Text-prompted linear experts outforecast LLM-alignment models","feed_subtitle":"LeMoLE reports 11-30% lower MSE on four benchmarks while using far fewer parameters and faster inference.","key_machinery":"The load-bearing object is the mixture of linear experts with varying lookback lengths, Equation (3), where the $m$-th expert maps $X_{T-w_m:T}$ through a $W_m \\in \\mathbb{R}^{H \\times w_m}$ matrix to a forecast. The second load-bearing object is the FiLM conditioning module: static prompt text $P_S$ and dynamic prompt text $P_D$ are encoded by a frozen GPT-2 into $Z_S, Z_D$, and two lightweight FiLM layers produce per-channel scale and shift, $\\gamma \\odot Y + \\beta$, that modulate the ensemble before a CNN fuses the branches. FiLM, feature-wise linear modulation, computes these scale and shift parameters from a conditioning representation. This pair of mechanisms lets the model adapt expert weighting to the text while keeping the per-step computation linear in the series length, avoiding the self-attention cost of alignment-based LLM forecasters.","core_discovery":"The central claim is stated in Section 4.2: LeMoLE presents lower prediction errors and higher computational efficiency than existing LLM models. Concretely, the paper reports average MSE improvements of 23.17% over MoLE, 20.70% over TimeMixer, 11.76% over GPT4TS, and 29.85% over AutoTimes on four standard benchmarks, and in the few-shot 10%-training setting it again finishes first on average MSE. The efficiency table shows LeMoLE-T with about 0.5M parameters at H=96 and 3.85M at H=720, versus 3.9M-58M for LLM baselines, with inference times around 1-3 ms compared to 4-23 ms for GPT4TS and TimeLLM. The authors interpret this as evidence that aligning time series into LLM spaces is not necessary: a mixture of simple linear experts with different lookback lengths, modulated by text embeddings, can be both more accurate and much cheaper.","pith_inferences":["A natural testable extension is to scramble or randomize the text prompts; if LeMoLE keeps its MSE, the gains would come from the variable-lookback mixture and FiLM conditioning rather than from language understanding.","Because the model only needs a dataset description and timestamps, it could be applied to domains with richer semantics, such as weather forecasting with textual forecasts as dynamic prompts, where the text channel might carry more signal than the four benchmarks show.","The reported comparison pits LeMoLE against full LLM-alignment models; a fairer assessment of the 'rethinking' claim would also match parameter budgets and training cost against a strong linear baseline with the same number of experts.","The four benchmark datasets are all electricity or traffic related; a broader benchmark with varied domains would clarify how much of the 23-30% MSE advantage comes from prompt informativeness versus the mixture architecture."],"forward_implications":["If the reported gains hold, forecasting practice can drop end-to-end time-series alignment with LLMs and instead use a frozen text encoder as a cheap conditioning signal.","Varying lookback lengths among experts is a simple way to make a linear ensemble capture both short- and long-term patterns, improving on a single-length mixture like MoLE.","Prompt content matters: the dynamic timestamp prompt contributes most on non-stationary datasets, while the static dataset description contributes more on the periodic Electricity data.","The small parameter count and millisecond-level inference of LeMoLE-T make the approach suitable for long-horizon and few-shot settings where LLM-alignment models are too slow or data-hungry.","The LeMoLE-F variant shows that replacing time-domain linear experts with frequency-domain FITS experts hurts accuracy, so in this architecture time-domain experts should be preferred."],"supporting_citations":[{"why":"Defines MoLE, the mixture-of-linear-experts method LeMoLE extends and the main baseline for the claimed 23.17% average MSE gain.","marker":"(Ni et al., 2024)"},{"why":"TimeMixer, the decomposed multi-scale mixing baseline the paper compares against and claims a 20.70% average MSE advantage.","marker":"(Wang et al., 2024)"},{"why":"GPT4TS, the pretrained-LM baseline used for the claimed 11.76% average MSE improvement.","marker":"(Zhou et al., 2024)"},{"why":"AutoTimes, the autoregressive LLM forecaster whose timestamp prompting is reused for the dynamic prompt and whose performance is compared (29.85% average MSE improvement).","marker":"(Liu et al., 2024b)"},{"why":"FiLM conditioning layer, the mechanism (Eqs. 6-7) that injects text features into expert outputs.","marker":"(Perez et al., 2018)"},{"why":"GPT-2, the frozen language model that encodes the static and dynamic prompts.","marker":"(Radford et al., 2019)"},{"why":"FITS, the frequency-domain linear model that supplies the experts for the LeMoLE-F variant.","marker":"(Xu et al., 2024)"},{"why":"DLinear, the linear-baseline argument that motivates mixing linear experts.","marker":"(Zeng et al., 2023)"},{"why":"Time-LLM, the alignment-based LLM forecaster whose approach the paper contrasts, and a baseline in the efficiency comparison.","marker":"(Jin et al., 2024)"}],"fun_headline_variants":["LeMoLE: LLM text prompts make linear experts beat LLM-aligned models","Text-guided linear experts outperform LLM-aligned forecasting models","Efficient time-series: linear experts plus LLM text beat full LLM models","Linear experts with LLM text embeddings forecast faster and better","LeMoLE: 11-30% lower MSE with a fraction of LLM parameters"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the hand-written dataset descriptions and timestamp strings, once encoded by the frozen GPT-2, contain information that the FiLM layers can usefully inject; if those text features are uninformative, LeMoLE reduces to a mixture of linear experts with different lookback lengths, and the claimed advantage would rest on that architectural change alone.","fun_headline_variants_meta":{"raw":{"variants":["LeMoLE: LLM text prompts make linear experts beat LLM-aligned models","Text-guided linear experts outperform LLM-aligned forecasting models","Efficient time-series: linear experts plus LLM text beat full LLM models","Linear experts with LLM text embeddings forecast faster and better","LeMoLE: 11-30% lower MSE with a fraction of LLM parameters"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00128,"raw_usage":{"total_tokens":5233,"prompt_tokens":945,"completion_tokens":4288,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":561,"completion_tokens_details":{"reasoning_tokens":4189}},"tokens_in":561,"tokens_out":4288,"duration_ms":30775,"temperature":1.0,"reasoning_tokens":4189,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:51:45.451830+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Remove the language model entirely and replace $Z_S$ and $Z_D$ with random vectors of the same shape, retraining the rest unchanged; if the MSE does not worsen, the text modality is not carrying the improvement. A second decisive check is to give MoLE the same number of experts and the same total parameter budget as LeMoLE but with identical lookback lengths; if LeMoLE still wins, the varying lookback lengths, not the multimodal fusion, explain the gains.","supporting_citations":[],"review_version":1}