{"id":"810a1e2b-daba-4660-a595-3f50c542537c","arxiv_id":"2608.11623","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"FM-LLM uses Fourier token embeddings and a mixture-of-experts decoder to adapt frozen LLMs to time series forecasting, reporting gains over AutoTimes across many benchmarks.","lead":"FM-LLM adapts a frozen large language model to forecast time series by embedding each time segment into sine and cosine features and decoding through two kinds of experts, one for repeating patterns and one for irregular changes. The paper reports accuracy gains over many LLM-based forecasters, but the evaluation has several inconsistencies.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SOTA claim depends on unmatched evaluation protocols: FM-LLM uses L=672 while baselines are cited from original papers; a matched rerun is required.","rationale":"The central claim is that FM-LLM is state-of-the-art on 59/78 metrics with 5.3%/5.6% average improvement over the strongest autoregressive LLM baseline. For this to hold, FM-LLM's runs and the baseline numbers must be generated under comparable settings. The weakest point is evaluation comparability: the paper uses published baseline numbers while FM-LLM uses L=672, a setting shown in its own Fig. 4 to improve accuracy substantially between T=384 and T=768. No included baseline is rerun under this lookback, except possibly AutoTimes*, which is only marked as using Llama-3.2-1B. The PEMS section even keeps the better of original or L=672 results for baselines, a rule that can only help the comparison. The 59/78 count itself is internally consistent (51 long-term + 8 PEMS), so the issue is not arithmetic but protocol match. Missing recent baselines (TimeKAN, CVC, EV-STLLM, TimeMixer) are a separate concern: they weaken the 'state-of-the-art' wording but are secondary to the protocol question, which concerns even the included baselines. The proposed test is decisive: rerun included baselines under FM-LLM's protocol and recompute win counts. If the advantage persists, the central claim is supported; if not, the paper requires major revision. The reader's CONDITIONAL verdict is appropriate, and this analysis does not move it.","tokens_in":30293,"tokens_out":11091,"duration_ms":108052,"concrete_test":"Rerun every baseline in Tables A.12 and 5 (PatchTST, iTransformer, MICN, TimesNet, FEDformer, DLinear, AMD, AutoTimes, GPT4TS, PRADA, TQNet, CycleNet) under FM-LLM's exact protocol: lookback 672, token length 96, identical train/validation/test splits and seeds, Llama-3.2-1B backbone for LLM baselines and the same training budget. Recompute the 59/78 win count and the mean MSE/MAE improvement over AutoTimes. If the win count drops substantially (e.g., below 40/78) or the AutoTimes advantage shrinks to under 2%, the headline SOTA claim is a protocol artifact. Separately, recompute M4 MASE with the standard in-sample naive-error denominator and re-check Table 4's rankings.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline result—59/78 SOTA metrics and 5.3%/5.6% average gains over AutoTimes—requires that FM-LLM's numbers be comparable to the baselines'. That condition is not met: Section 4.1.4 states 'we primarily use the results reported in the original publications for all baseline models,' while FM-LLM is run with lookback L=672 and token length P=96 (Table 2). Many baselines (PatchTST, TimesNet, FEDformer, DLinear, AMD, etc.) report results with shorter lookbacks, commonly 96–512. The paper's own ablation (Fig. 4, observation 4) shows FM-LLM's MSE decreases substantially from T=384 to T=768, so lookback length is a known driver of performance; matching it is not a cosmetic detail. The PEMS protocol is even more asymmetric: 'If the forecasting performance at this length does not exceed the results reported in the original paper, we retain the results from the original publication,' which selects the better of two configurations for baselines only. AutoTimes* is the only baseline flagged as re-run under the Llama3.2-1B backbone, so the average improvement over AutoTimes is against a partially matched LLM baseline while the 59/78 count includes many unmatched non-LLM baselines. Secondary: Eq. (28)'s MASE denominator uses the forecast window rather than the in-sample naive errors, which can shift M4 numbers relative to published values. These issues do not disprove the method, but they make the SOTA claim unverifiable without a matched-protocol reproduction.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces FM-LLM, a framework for adapting frozen LLMs to multivariate time-series forecasting without textual prompts. The model consists of three main components: a Fourier Analysis Network (FAN)-based spectral token aligner that maps time-series patches into harmonic representations, an asymmetric Mixture-of-Experts (MoE) decoder in which shared experts with FAN layers model periodic structure and routed standard-FFN experts model non-periodic residuals, and a hybrid time-frequency loss that combines a signal-decay-weighted MSE with a frequency-domain L1 loss plus an auxiliary expert-balance loss. The authors evaluate FM-LLM on long-horizon benchmarks (ETT, Electricity, Weather, Traffic, PEMS), the M4 short-term benchmark, few-shot (10%) and zero-shot transfer settings. They claim state-of-the-art performance on 59/78 metrics in the abstract, with average MSE/MAE improvements of 5.3%/5.6% over the strongest autoregressive LLM baseline (AutoTimes), and support the design with ablations, expert-load analyses, spectral analyses, efficiency comparisons, and a distribution-shift diagnosis.","tokens_in":30676,"tokens_out":5323,"duration_ms":53039,"significance":"If the empirical claims hold, the contribution is likely valuable: a prompt-free, frequency-aware frozen-LLM forecaster with a structural role separation in the decoder is a plausible and reasonably novel design, and the reported efficiency gains over prompt-based LLM baselines are practically relevant. The paper contains several genuine strengths: full hyperparameter tables (Table 2), a useful decomposition of expert roles in the decoder, a detailed ablation set (Tables 8–9), expert-load histograms (Figure 5), a distribution-shift diagnosis for the non-monotonic data-scale effect (Appendix D), and explicit reporting of GPU memory, latency, and trainable-parameter counts (Table 10). These elements make the submission more reproducible and falsifiable than many empirical time-series papers. However, the central state-of-the-art claim is currently not verifiable because the evaluation protocol is asymmetric between FM-LLM and the baselines, the headline metric counts are inconsistent between the abstract and the main text, and the MASE formula appears nonstandard. These issues are fixable, but they are load-bearing for the paper's main conclusion.","major_comments":[{"comment":"The headline SOTA claim is internally inconsistent: the abstract states 59 out of 78 evaluation metrics, while §4.2 reports 51 best scores across 70 metrics and Table A.12 shows '1st Count 51'. Table 3's '1st Count' row (11 for FM-LLM) neither matches 51 nor 59. In addition, the average improvement over AutoTimes is given as 5.20% MSE and 5.32% MAE in §4.2 but as 5.3% and 5.6% in the abstract. Because the state-of-the-art claim is the central result, every count must be reconciled, and each number must be traceable to a specific table and metric set.","section":"Abstract vs. §4.2 and Table A.12"},{"comment":"The evaluation protocol is asymmetric and undermines the SOTA claim. FM-LLM is run with lookback L=672 and token length P=96, while baseline results are 'primarily' taken from original publications, where lookback lengths are commonly 96–512. The paper's own Figure 4, observation (4), shows that FM-LLM's MSE decreases substantially from T=384 to T=768, so lookback length is a known performance driver for this architecture; it cannot be treated as a cosmetic detail for baselines. For PEMS, §4.1.4 states that if the L=672 result does not exceed the original-paper result, the original result is retained, which gives baselines a 'best-of-two' advantage while FM-LLM is reported at a single setting. AutoTimes* is the only baseline explicitly re-run under the Llama3.2-1B backbone; other LLM baselines use their original backbones. A valid SOTA claim requires a matched-protocol rerun of all baselines under the same lookback, backbone, seeds, and preprocessing, or at least a demonstration that lookback length does not change the relative ordering.","section":"§4.1.4, Tables 2–3, and Figure 4"},{"comment":"The MASE formula in Eq. (28) is nonstandard. The denominator is written as (1/(N-s)) * sum_{j=s+1}^N |y_j - y_{j-s}|, with N described as 'the length of one token, which is equal to P'. Standard MASE scales the forecast error by the mean absolute error of the in-sample seasonal naive forecasts computed over the training set, not over the forecast window. Computing the scaling factor on the forecast window itself can shift M4 MASE values relative to published baselines, and the M4 comparison in Table 4 is therefore not on an equal footing. The authors should either adopt the standard definition or justify the alternative and validate it against published M4 numbers for a baseline such as Naive2.","section":"§4.1.2, Eq. (28)"},{"comment":"No error bars, confidence intervals, or multiple-seed results are reported anywhere. Several claimed improvements are extremely small: in Table A.12, on ETTh2-336 PatchTST achieves MSE/MAE 0.329/0.380 versus FM-LLM 0.361/0.395, and on ETTm1-336 PRADA achieves 0.350/0.384 versus FM-LLM 0.351/0.379 (essentially tied). Without variance estimates, the 'consistently outperforming' statement in §4.2 is not supported. I request mean and standard deviation over at least three seeds for FM-LLM and for the re-run baselines, and significance testing for close comparisons.","section":"All result tables (Tables 3–7, A.12)"},{"comment":"Several recent baselines are cited in the related work but absent from the experimental comparisons: TimeKAN [26], CVC [24], EV-STLLM [25], and TimeMixer [21]. Given the paper's claim to state-of-the-art performance, omitting all of these from the tables prevents the SOTA claim from being verified against the most current methods, including LLM-based (CVC, EV-STLLM) and KAN-based (TimeKAN) competitors. At least the LLM-based and KAN-based baselines should be added under the matched protocol.","section":"§2.1 and Tables 3–7"}],"minor_comments":[{"comment":"The sampling frequency of ETTh1 and ETTh2 is listed as '15 min', which contradicts the text in §4.1.1 where these datasets are described as sampled hourly. Correct this factual error.","section":"Table 1"},{"comment":"The frequency-domain loss term F(w_l · a_{t+l}) is not fully specified: it is unclear over what length the DFT is computed, whether the same window length P is used for every l, and how the normalization of F is chosen. Please specify the DFT length, normalization, and how the weighting w_l interacts with the frequency-domain term.","section":"Eq. (19)"},{"comment":"The notation for the gating variable is inconsistent: Eq. (13) uses g_{i,k}, the text says 'g_k^i indicates the normalized gating value' and then 'gi,k denotes the pre-normalized gating score'. Please unify the symbols and distinguish normalized from pre-normalized gating values throughout.","section":"Eq. (13) and surrounding text"},{"comment":"The problem-definition notation is inconsistent: the historical window is x_{1:T}, but the mapping is written as f: x_{1:T} -> \\hat{x}_{L+1:L+F}, using L instead of T. Use the same symbol for the lookback length throughout.","section":"§3.1, Eq. (1)"},{"comment":"There are numerous typos and formatting artifacts, including 'adpts' in the Figure 1 caption, 'M4-Weakly' instead of 'M4-Weekly' in Table 1, 'A verage' in Table 4, and stray LaTeX braces in Eq. (24). A careful proofread is needed.","section":"Throughout"},{"comment":"The '1st Count' row is confusing: it is not clear whether the numbers count per-model best scores across all 70 metrics or across some other subset. The count in the caption and the text (51 best) should be reconciled and the row removed or relabeled.","section":"Table 3"}],"recommendation":"major_revision","confidential_remarks":"The core architecture is interesting and the paper contains a substantial amount of experimental material, but the state-of-the-art claim currently rests on an asymmetric evaluation protocol and inconsistent headline counts. I would ask for a matched-protocol rerun of all baselines (or a convincing invariance argument), reconciliation of the SOTA counts, and a fix of the MASE formula before the paper can be accepted. The absence of the recent baselines cited in the related work is also concerning for a claim of state-of-the-art performance in 2026. I see no reason for outright rejection; the issues are within scope of a major revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Rentao Gu et al. propose FM-LLM, a prompt-free frozen-LLM forecaster that injects Fourier Analysis Network (FAN) spectral token aligners on the input side and an asymmetric MoE decoder on the output side, trained with a time-frequency hybrid loss. The combination is new relative to the cited literature: AutoTimes, PRADA, etc. all use patch/MLP alignment and shallow decoders. The ablations suggest each ingredient matters—removing FAN, MoE, LLM, or frequency loss degrades performance—and the efficiency comparison shows a real latency advantage over TimeLLM. That is a genuine engineering contribution, and the few-shot and zero-shot experiments are a thoughtful addition even if the story is mostly \"our recipe works.\"\n\nThe soft spot is evaluation hygiene. The headline SOTA count is inconsistent: abstract says 59/78, Section 4.2 reports 51 firsts across 70 metrics, and Table 3's \"1st Count\" row lists 11, which I cannot reconcile with the table. Eq. (28) defines MASE with the forecast-window length in the denominator, not the in-sample naive forecast errors; that shifts M4 numbers relative to published values. Table 1 lists ETTh1/ETTh2 frequency as 15 min, but they are hourly. Those are fixable errors, but they erode trust.\n\nBigger issue: baselines are cited from original papers while FM-LLM runs with L=672. Their own sensitivity analysis (Fig. 4, observation 4) shows MSE dropping substantially from T=384 to T=768, so lookback is a real driver. For PEMS they even select whichever of the two configurations is better, only for baselines. AutoTimes is re-run with the Llama3.2-1B backbone, so the AutoTimes comparison is partially matched, but the 59/78 count includes many unmatched non-LLM baselines. They also cite TimeKAN, CVC, EV-STLLM, and TimeMixer but do not include them in the comparison tables. No error bars and no code/data.\n\nNone of this disproves the method. The components are sensible, and the ablations are self-consistent. But the SOTA claim is not verifiable from the provided numbers; a matched-protocol re-run with released code is necessary. A serious referee should engage with this, not desk-reject it, and the revision should fix the count, the MASE formula, the frequency label, and ideally add error bars and a few matched baselines.\n\nI would bring it to reading group to discuss evaluation pitfalls; I would not cite it until code and matched numbers are out. Recommendation: send to peer review, conditional on the evaluation being substantiated.","headline":"A plausible, prompt-free frequency-MoE recipe for frozen-LLM forecasting whose SOTA claim is currently undercut by unmatched baseline protocols and small but telling evaluation errors.","tokens_in":31151,"tokens_out":1895,"would_cite":false,"duration_ms":18465,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A prompt-free, frequency-aware design lets a frozen large language model outperform prior methods on 59 of 78 time-series forecasting metrics.","keywords":["time series forecasting","large language models","frequency domain","mixture of experts","Fourier analysis","autoregressive forecasting","few-shot learning","zero-shot transfer"],"falsifier":"Run the strongest baselines (AutoTimes, PRADA, PatchTST, iTransformer) inside FM-LLM's exact protocol—672-step lookback, 96-length tokens, the same frozen backbone, seeds, and train/validation/test splits—and recompute M4 metrics with the standard MASE denominator instead of Eq. (28); if FM-LLM no longer wins a majority of the 78 metrics, the headline SOTA claim fails.","tokens_in":30124,"feed_emoji":"📈","tokens_out":10645,"duration_ms":96864,"temperature":0.7,"pith_summary":"This paper claims that the usual recipe for adapting large language models (LLMs) to time-series forecasting—text-like prompts plus a shallow linear decoder—is both inefficient and blind to the spectral structure of the data. It introduces FM-LLM, which sends each patch through a Fourier Analysis Network before it enters a frozen LLM, so harmonic information arrives without any prompt, and then reconstructs predictions with a mixture-of-experts decoder whose shared Fourier experts rebuild the periodic backbone while routed standard experts handle irregular residuals. On eleven public benchmarks the authors report state-of-the-art results on 59 of 78 metrics, with average improvements of 5.3% in mean squared error and 5.6% in mean absolute error over the strongest autoregressive LLM baseline. If true, this would mean prompt-free, frequency-aware adaptation can make frozen LLMs both more accurate and far cheaper than prompt-based forecasters, while preserving few-shot and zero-shot transfer. The cost is that the model is still slower and heavier than non-LLM baselines such as PatchTST, though markedly lighter than prompt-based LLM alternatives.","feed_headline":"Frequency-aware frozen LLM tops 59 of 78 forecasting metrics","feed_subtitle":"Fourier token aligner plus expert decoder beats prompt-based LLM forecasters on 11 benchmarks.","key_machinery":"The load-bearing mechanism is constrained asymmetric coupling: a Fourier Analysis Network (FAN) spectral token aligner on the encoding side and a heterogeneous mixture-of-experts (MoE) decoder on the decoding side. FAN injects harmonic structure by projecting each patch through sine-cosine basis functions combined with a nonlinear activation, giving the frozen LLM numerically compatible spectral tokens without any textual prompt. The decoder then adds always-on FAN-equipped Fourier experts to top-$K$ routed standard feed-forward networks, with a bias-based load-balancing scheme and a sequence-wise balance loss to prevent expert collapse. A time-frequency hybrid loss supervises both temporal accuracy and spectral consistency, and training computes the loss only on the token immediately following the context window, using signal-decay weights $w_l = 1/\\sqrt{l}$ to suppress error accumulation in autoregressive rollouts.","core_discovery":"The paper's central claim is that the modality gap between continuous time series and discrete LLM tokens is better bridged by explicit spectral structure than by textual prompts. FM-LLM tokenizes each channel into non-overlapping patches, projects each patch through a single-layer Fourier Analysis Network $FAN(x) = \\phi_L \\circ \\cdots \\circ \\phi_1(x)$, where interior layers output $[\\cos(W_p x)\\,\\Vert\\,\\sin(W_p x)\\,\\Vert\\,\\sigma(B_{\\bar p}+W_{\\bar p}x)]$, and feeds those harmonic embeddings into a frozen LLM. The LLM autoregressively predicts the next token embedding, and a constrained mixture-of-experts decoder reconstructs the time-domain token: shared Fourier experts with FAN layers rebuild the global periodic backbone, while top-$K$ routed standard feed-forward networks capture non-periodic residuals. Training minimizes a hybrid loss $\\alpha L_{\\mathrm{freq}} + \\beta L_{\\mathrm{time}} + \\lambda L_{\\mathrm{Bal}}$, with a discrete-Fourier-transform $\\ell^1$ discrepancy enforcing spectral consistency, signal-decay-weighted MSE promoting short-horizon fidelity, and a sequence-wise balance regularizer preventing routing collapse. On that basis the authors report state-of-the-art results on 59 of 78 metrics across eleven benchmarks, average MSE and MAE improvements of 5.3% and 5.6% over AutoTimes, and robust 10% few-shot and zero-shot transfer.","pith_inferences":["I would not treat the 59/78 count as settled until the leading baselines are re-run under FM-LLM's exact protocol; the paper compares against many numbers taken from original publications, so part of the margin could reflect lookback, backbone, or training differences.","The constrained-asymmetry recipe—spectral structure only in shared experts, residuals only in routed experts—could be lifted out and applied to other frozen encoders, with the hybrid time-frequency loss acting as a general regularizer.","A clean ablation that keeps the MoE decoder and hybrid loss but replaces the FAN aligner with a linear projection would isolate how much of the gain truly comes from spectral token alignment.","The non-monotonic training-data trend the paper reports on ETTm2 suggests standard fixed splits can misrepresent scaling behavior; reporting multiple training fractions, as FM-LLM does for two datasets, would be a more informative convention."],"forward_implications":["Prompt-based LLM forecasters pay a large inference penalty; FM-LLM runs at 143.48 ms/iter on the 720-step Traffic task versus 1056.0 ms/iter for Time-LLM, so prompt-free frequency alignment makes frozen-LLM forecasting practical at long horizons.","A single trained FM-LLM applies to multiple prediction lengths without retraining and adapts with about 8.68 million trainable parameters, fitting in roughly 6 GB of GPU memory.","With only 10% of training data, and in zero-shot dataset transfer, FM-LLM retains superior accuracy; the paper links few-shot difficulty to spectral divergence between the reduced and full training sets.","Expert specialization is structurally interpretable: the paper's visualizations show Fourier experts tracking the low-frequency backbone while routed experts handle high-frequency residuals, giving a mechanistic account of the accuracy gains."],"supporting_citations":[{"why":"AutoTimes supplies the autoregressive frozen-LLM forecasting paradigm FM-LLM extends and the strongest baseline it claims to beat.","marker":"[6]"},{"why":"Fourier Analysis Network is the module the spectral token aligner and Fourier experts are built on.","marker":"[13]"},{"why":"The bias-based load-balancing and sequence-wise balance loss stabilize the MoE decoder.","marker":"[36]"},{"why":"CARD provides the signal-decay weighting that shapes the time-domain loss.","marker":"[37]"},{"why":"FreDF justifies the frequency-domain loss used to enforce spectral consistency.","marker":"[39]"},{"why":"PRADA is the prompt-guided LLM baseline used for few-shot and zero-shot comparisons, especially ETT transfer.","marker":"[8]"},{"why":"PatchTST contributes the patch-token and channel-independence design FM-LLM adopts and is a strong non-LLM baseline.","marker":"[22]"},{"why":"iTransformer defines the PEMS benchmark protocol and provides a high-dimensional multivariate comparison point.","marker":"[23]"}],"fun_headline_variants":["Fourier-LLM forecaster wins 59 metrics without prompts","Frequency-aware MoE decoder boosts frozen LLM forecasts","Spectral token aligner lets frozen LLMs beat prompt rivals","LLM forecaster with Fourier experts: 5.3% better MSE"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim assumes the numbers quoted from other papers were produced under conditions close enough to FM-LLM's own runs that the differences reflect architecture rather than experimental setup; if that comparability fails, the 59-of-78 claim could dissolve.","fun_headline_variants_meta":{"raw":{"variants":["Fourier-LLM forecaster wins 59 metrics without prompts","Frequency-aware MoE decoder boosts frozen LLM forecasts","Spectral token aligner lets frozen LLMs beat prompt rivals","LLM forecaster with Fourier experts: 5.3% better MSE"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000683,"raw_usage":{"total_tokens":3195,"prompt_tokens":1137,"completion_tokens":2058,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":753,"completion_tokens_details":{"reasoning_tokens":1993}},"tokens_in":753,"tokens_out":2058,"duration_ms":16105,"temperature":1.0,"reasoning_tokens":1993,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:33:02.521961+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the strongest baselines (AutoTimes, PRADA, PatchTST, iTransformer) inside FM-LLM's exact protocol—672-step lookback, 96-length tokens, the same frozen backbone, seeds, and train/validation/test splits—and recompute M4 metrics with the standard MASE denominator instead of Eq. (28); if FM-LLM no longer wins a majority of the 78 metrics, the headline SOTA claim fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"AutoTimes supplies the autoregressive frozen-LLM forecasting paradigm FM-LLM extends and the strongest baseline it claims to beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CARD provides the signal-decay weighting that shapes the time-domain loss."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"FreDF justifies the frequency-domain loss used to enforce spectral consistency."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"PRADA is the prompt-guided LLM baseline used for few-shot and zero-shot comparisons, especially ETT transfer."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"PatchTST contributes the patch-token and channel-independence design FM-LLM adopts and is a strong non-LLM baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"iTransformer defines the PEMS benchmark protocol and provides a high-dimensional multivariate comparison point."}],"review_version":1}