REVIEW 4 major objections 5 minor 27 references
Coupling a pre-trained LLM, reconfigured only by attention-topology changes, with an interval-averaged flow-matching decoder yields one-step forecasts that the paper reports consistently outperform eight baselines across ten time-series ben
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 04:53 UTC pith:DK76EOK5
load-bearing objection CoGenCast is a plausible hybrid LLM+flow forecaster with broad experiments, but the 'consistently outperforms' claim overreaches and the pretrained-weight attribution is untested because the random-init control is missing. the 4 major comments →
CoGenCast: A Coupled Autoregressive-Flow Generative Framework for Time Series Forecasting
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that the conditional forecasting distribution is best modeled by jointly learning an autoregressive semantic representation and a continuous stochastic transport, and that pre-trained LLM weights can be repurposed for this by modifying only the attention topology: the encoder uses bidirectional self-attention to fuse look-back patches and textual context, the decoder keeps causal self-attention and adds cross-attention, and a denoising decoder predicts a velocity field averaged over a time interval rather than an instantaneous velocity. A Jacobian-vector-product-corrected MSE loss drives the trajectories straight, so a single Euler step suffices at inference. The paper r
What carries the argument
The load-bearing mechanism is the attention-topology reconfiguration of a pre-trained decoder-only LLM: replacing causal self-attention with bidirectional self-attention in the encoder and adding cross-attention in the causal decoder, initialized entirely from the LLM's pretrained weights. The second mechanism is interval-conditioned average-velocity flow matching: instead of predicting instantaneous velocity, the denoising decoder predicts the average velocity over an interval, with a JVP-based loss term that penalizes curvature; this keeps the transport path nearly straight, enabling one-step (1-NFE) sampling.
Load-bearing premise
The pretrained LLM weights remain useful after swapping causal self-attention for bidirectional attention and inserting cross-attention; if they do not, the reported advantage over a same-scale randomly initialized transformer disappears, and this is not tested against a randomly initialized control.
What would settle it
Train the same encoder–decoder–flow architecture from scratch (random initialization, same 0.6B scale, same data) and compare MSE/MAE on the ten benchmarks; statistical parity with CoGenCast would falsify the claim that pretraining, rather than architecture or training procedure, drives the gains.
If this is right
- If the reported gains hold, one-step forecasting with LLM-scale semantic conditioning becomes practical for latency-sensitive applications, since a single function evaluation replaces iterative denoising.
- Cross-domain training consistently lowers MSE versus in-domain training, implying that a single CoGenCast model can serve multiple domains without per-domain fine-tuning.
- Removing either the autoregressive LLM backbone or the flow decoder degrades accuracy, supporting the paper's dual-capability thesis.
- LLM backbones outperforming a vanilla transformer at equal scale suggests pretrained language model weights, not just scale, contribute to forecasting accuracy — provided the transferability assumption holds.
- Extending the look-back window from 96 to 336 steps yields progressive improvements, so the architecture can exploit longer history without structural changes.
Where Pith is reading between the lines
- A decisive control the paper does not run: same reconfigured architecture with randomly initialized weights. If that control matches CoGenCast, the advantage would come from the architecture and training recipe rather than from pretrained weights.
- The interval-averaging trick could transfer to other generative tasks — e.g., time-series imputation or multimodal generation — wherever a nearly straight probability path can be enforced by the JVP objective.
- The attention-topology reconfiguration recipe suggests a general way to adapt decoder-only pretrained models to encoder–decoder tasks, which could be tested on text summarization or translation with minimal fine-tuning.
- Because the paper's main table and appendix headers report different look-back lengths (L=96 vs L=336), a reader should confirm which setting produced the headline numbers before generalizing the comparison.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes CoGenCast, a generative time-series forecasting framework that (i) reconfigures a pretrained decoder-only LLM (Qwen3-0.6B) into an encoder-decoder by changing causal self-attention to bidirectional self-attention in the encoder and adding cross-attention in the decoder, and (ii) couples this backbone with a flow-matching denoising decoder that predicts an interval-conditioned average velocity field. Training uses an autoregressive patch loss and a JVP-corrected flow-matching objective (Eq. 7); inference generates patches autoregressively and, the authors claim, samples each patch in one function evaluation (Eq. 8). Experiments on ten benchmarks compare against LLM-based, generative, and transformer baselines, with ablations on architecture, AR-flow components, context features, backbone scale, NFE, patch size, and noise schedule.
Significance. If the empirical results are robust, the proposed framework is a meaningful step: it is one of the few systems that combine LLM semantic conditioning with continuous stochastic generation while avoiding iterative sampling. The evaluation spans ten datasets, per-horizon results are provided in the appendix, and the code is released, all of which are strengths. The empirical claim is not circular, since the flow objective is a training loss rather than a quantity fitted to benchmark results. However, the manuscript overstates the consistency of the wins, and the central attribution of gains to pretrained LLM weights is not yet supported by the experimental design. The methodological novelty of the JVP-corrected average-velocity objective also needs a derivation or explicit reference. These issues are fixable but require additional experiments and analysis.
major comments (4)
- [§4.2, Table 2; abstract and contribution 3] The paper claims CoGenCast "consistently outperforms previous compared baselines," but Table 2 shows the opposite on ETTh2: CoGenCast MSE 0.159 vs TimeDART 0.145 and PatchTST 0.142 (MAE 0.246 vs 0.253/0.251). No standard deviations, seeds, or significance tests are provided anywhere, and many between-method gaps in Table 2 are small (e.g., Solar 0.221 vs 0.232; ETTm2 0.109 vs 0.122). Please add multi-seed statistics or revise the claim to "competitive on most benchmarks."
- [§3.3 and Table 6] The load-bearing claim that reusing pretrained Qwen3-0.6B weights is beneficial is not tested. Reconfiguring causal self-attention to bidirectional self-attention and adding cross-attention may destroy the value of the pretrained weights, but the only backbone comparisons are against a vanilla Transformer (varying pretraining, architecture, and parameter count together) or against larger Qwen variants that also use pretrained weights. A same-architecture randomly initialized control (same encoder-decoder plus flow decoder, same patching/text embedding scheme) is required to attribute the observed gains to pretraining rather than scale or architecture. Appendix D.1's "full 0.6B > encoder-only 1.7B" comparison does not provide this control.
- [§3.4, Eq. (8)] The one-step/low-latency claim is not consistent with the written equation. Eq. (8) defines yout_j as y_j(0) + ∫_0^1 u(zτ,τ,zdec_out)dτ, an integral over a continuum; evaluating this is not a single function evaluation. If the denoising decoder outputs the average velocity û_j over [0,1], the one-step output should be y_j(0) + û_j. Please either rewrite Eq. (8) accordingly or state explicitly that 1-NFE refers to a one-step Euler approximation of the integral, and report measured latency/NFE.
- [§3.3, Eq. (7)] The JVP-corrected objective is insufficiently justified. The regression target v_j − (r−t) ∂u_j/∂t depends on the model's own time-derivative, so the loss is not a fixed supervised target; the text does not prove that minimizers learn the average velocity over [t,r]. This is the main methodological novelty, and the cited mean-flow papers (Geng et al. 2025a,b) are not connected to the derivation. Please provide a derivation or reference, and include a small synthetic check of the estimated velocity/straightness.
minor comments (5)
- [Appendix I, Table 8] The header states L=336, while Section 4.1 and Table 2 use L=96. Please clarify which look-back setting produced the main results and ensure the appendix tables match the main-text averages.
- [§3.3–§3.4, Eqs. (6) and (8)] Notation is inconsistent: Eq. (6) uses hat z_j but Eq. (8) uses z_τ for the noised latent. Please unify.
- [Figure 3] The figure contains a typo ("knowdge" for "knowledge") and reports only MSE without error bars or raw values in the main text.
- [§4.6, Figure 6] The text calls the predictive intervals "well-calibrated," but no quantitative coverage or calibration metrics are reported. Figure 6 alone is qualitative and does not support that wording.
- [§3.3, Related Work] Mean flows and improved mean flows are cited but not discussed. Please articulate the difference between the proposed interval-conditioned objective and those methods.
Circularity Check
No significant circularity: benchmark results are self-contained; missing random-init control and borrowed mean-flow mechanism are attribution/novelty concerns, not input-output loops.
full rationale
The paper's central empirical claim is not circular. CoGenCast is trained by minimizing the flow-matching MSE in Eq. (7) on fixed train/test splits, and the 1-NFE inference in Eq. (8) is the corresponding Euler step of the learned velocity field. No parameter is fitted to the reported benchmark numbers, and the comparisons in Tables 2, 5, and 6 are against external or controlled internal baselines. The only self-referential element is the JVP-corrected target in Eq. (7), which includes the model's own time derivative; this is a fixed-point regularizer used during training, not an equivalence between a fitted quantity and a claimed prediction. The 'average velocity' mechanism is closely related to the cited mean-flow literature (Geng et al., 2025a,b), but those works are cited, and the framework's claimed contribution is the coupling of an LLM backbone with a flow-matching decoder. The main weaknesses are attribution gaps rather than circularity: Table 6 lacks a same-architecture randomly initialized control, so the 'pretrained LLM' advantage is not cleanly isolated from scale or architecture, and several self-citations (e.g., Cheng et al., 2025a,b in Sec. 3.1) are background rather than load-bearing. These concerns justify a low non-zero score, but no step in the derivation reduces to its own inputs by construction.
Axiom & Free-Parameter Ledger
free parameters (4)
- Patch size P =
2-10; dataset-specific (e.g., 6 for Energy/ETTh1, 2 for Environ/Exchange)
- Learning rate =
5e-4, 1e-4, or 5e-5
- Training epochs =
10
- Look-back window L =
96
axioms (4)
- domain assumption Reconfiguring causal attention into bidirectional encoder attention plus cross-attention preserves the usefulness of Qwen3-0.6B pretrained weights.
- domain assumption Straight-line interpolation and the average-velocity objective can faithfully represent the conditional forecast distribution P(Xpred|Xhist,C).
- ad hoc to paper The JVP-corrected loss in Eq. 7 yields a valid mean-velocity estimator.
- domain assumption Official baseline implementations are evaluated fairly and comparably.
read the original abstract
Time series forecasting can be viewed as a generative problem that requires both semantic understanding over contextual conditions and stochastic modeling of continuous temporal dynamics. Existing approaches typically rely on either autoregressive large language models (LLMs) for semantic context modeling or diffusion-like models for continuous probabilistic generation. However, neither method alone can adequately model both aspects simultaneously. In this work, we propose CoGenCast, a hybrid generative framework that couples pre-trained LLMs with flow-matching mechanism for effective time series forecasting. Specifically, we reconfigure pre-trained decoder-only LLMs into a native forecasting encoder-decoder backbone by modifying only the attention topology, enabling bidirectional context encoding and causal representation generation. Building on this, a flow-matching mechanism is further integrated to model temporal evolution, capturing continuous stochastic dynamics conditioned on the autoregressively generated representation. Notably, CoGenCast naturally supports multimodal forecasting and cross-domain unified training. Extensive experiments on multiple benchmarks show that CoGenCast achieves competitive performance compared to previous baselines. Code is available at https://github.com/liuyaguo/_CoGenCast.
Figures
Reference graph
Works this paper leans on
-
[1]
Bian, Y ., Ju, X., Li, J., Xu, Z., Cheng, D., and Xu, Q. Multi-patch prediction: Adapting llms for time series representation learning.arXiv preprint arXiv:2402.04852,
-
[6]
Z., Wang, S., Zhang, J., Li, Z., and Chen, T
Hu, Y ., Wang, X., Ding, Z., Wu, L., Zhang, H., Li, S. Z., Wang, S., Zhang, J., Li, Z., and Chen, T. Flowts: Time series generation via rectified flow.arXiv preprint arXiv:2411.07506,
-
[7]
Fstllm: Spatio-temporal llm for few shot time series forecasting
Jiang, Y ., Chen, Y ., Li, X., Chao, Q., Liu, S., and Cong, G. Fstllm: Spatio-temporal llm for few shot time series forecasting. InForty-second International Conference on Machine Learning. Jin, M., Wang, S., Ma, L., Chu, Z., Zhang, J. Y ., Shi, X., Chen, P.-Y ., Liang, Y ., Li, Y .-F., Pan, S., et al. Time-llm: Time series forecasting by reprogramming la...
-
[8]
Kollovieh, M., Lienen, M., L ¨udke, D., Schwinn, L., and G¨unnemann, S. Flow matching with gaussian process priors for probabilistic time series forecasting.arXiv preprint arXiv:2410.03024,
-
[10]
Gener- ative pretrained hierarchical transformer for time series forecasting
Liu, Z., Yang, J., Cheng, M., Luo, Y ., and Li, Z. Gener- ative pretrained hierarchical transformer for time series forecasting. InProceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, pp. 2003–2013, 2024c. Lu, Y ., Sun, Q., Wang, X., Jiang, Z., Zhao, H., and He, K. Bidirectional normalizing flow: From data to noise and bac...
arXiv 2003
-
[11]
One-step latent-free im- age generation with pixel mean flows.arXiv preprint arXiv:2601.22158,
Lu, Y ., Lu, S., Sun, Q., Zhao, H., Jiang, Z., Wang, X., Li, T., Geng, Z., and He, K. One-step latent-free im- age generation with pixel mean flows.arXiv preprint arXiv:2601.22158,
-
[13]
M., Cornelis, B., and Munteanu, A
Moghadas, S. M., Cornelis, B., and Munteanu, A. Fr `eqflow: Long-term forecasting using lightweight flow matching. arXiv preprint arXiv:2511.16426,
-
[14]
H., Sinthong, P., and Kalagnanam, J
Nie, Y ., Nguyen, N. H., Sinthong, P., and Kalagnanam, J. A time series is worth 64 words: Long-term fore- casting with transformers. arxiv 2022.arXiv preprint arXiv:2211.14730,
Pith/arXiv arXiv 2022
-
[15]
Niu, W., Xie, Z., Sun, Y ., He, W., Xu, M., and Hao, C. Langtime: A language-guided unified model for time se- ries forecasting with proximal policy optimization.arXiv preprint arXiv:2503.08271,
-
[16]
Tao, X., Zhang, S., Cheng, M., Wang, D., Pan, T., Pan, B., Zhang, C., and Wang, S. From values to tokens: An llm-driven framework for context-aware time series forecasting via symbolic discretization.arXiv preprint arXiv:2508.09191,
-
[18]
Wang, R. and He, K. Diffuse and disperse: Image gener- ation with representation regularization.arXiv preprint arXiv:2506.09027,
-
[19]
Wu, X., Qiu, X., Gao, H., Hu, J., Yang, B., and Guo, C. k2 vae: A koopman-kalman enhanced variational autoen- coder for probabilistic time series forecasting.arXiv preprint arXiv:2505.23017,
-
[20]
Xie, Z., Li, Z., He, X., Xu, L., Wen, X., Zhang, T., Chen, J., Shi, R., and Pei, D. Chatts: Aligning time series with llms via synthetic data for enhanced understanding and reasoning.arXiv preprint arXiv:2412.03104,
-
[21]
Non-stationary diffusion for probabilistic time series forecasting.arXiv preprint arXiv:2505.04278,
Ye, W., Xu, Z., and Gui, N. Non-stationary diffusion for probabilistic time series forecasting.arXiv preprint arXiv:2505.04278,
-
[22]
Zhang, J., Cheng, M., Tao, X., Liu, Z., and Wang, D. Condi- tional denoising meets polynomial modeling: A flexible decoupled framework for time series forecasting.arXiv preprint arXiv:2410.13253, 2024a. Zhang, X. N., Pu, Y ., Kawamura, Y ., Loza, A., Bengio, Y ., Shung, D., and Tong, A. Trajectory flow matching with applications to clinical time series mo...
-
[23]
Samples are organized in (Train/Validation/Test)
Table 7.Full dataset descriptions. Samples are organized in (Train/Validation/Test). DATASETSLOOK-BACKPREDICTEDVARIABLESSAMPLESDOMAINFREQUENCY Energy 96{12,24,36,48}1 928/138/284 Energy 1 Week ETTh1 96{12,24,36,48}7 8293/2869/2869 Energy 1 Hour ETTh2 96{12,24,36,48}7 8293/2869/2869 Energy 1 Hour ETTm1 96{12,24,36,48}7 34417/11473/11473 Energy 15 Mins ETTm...
1993
-
[26]
Solar (Lai et al., 2018)This dataset records the solar power production of 137 PV plants in 2006, which are sampled every 10 minutes. B. Implementation Details B.1. Compared Baselines To more comprehensively evaluate the capabilities of our proposed method, we compare our method with eight representative baselines in our experiments. Specifically, we incl...
2018
-
[27]
consistently leads to performance degradation, most notably in the Health benchmark where the error significantly increases compared to the optimal setting. This deterioration likely stems from the smoothing effect of large patches, which obscures critical short-term dynamics required for accurate generation. Similarly, while small patches benefit specifi...
arXiv 1910
-
[2016]
Influenza-Like Illness (ILI) statistics spanning from 1954 to the present, recorded at a weekly frequency
Health (Liu et al., 2024a)This dataset comprises weekly U.S. Influenza-Like Illness (ILI) statistics spanning from 1954 to the present, recorded at a weekly frequency. The accompanying data is collected from the CDC Weekly U.S. Influenza Surveillance Report, released weekly. Wind (Li et al., 2022)This dataset comprises wind power measurements sampled ever...
1954
-
[2018]
Environment (Liu et al., 2024a)This dataset comprises daily air quality time series collected from the U.S
It is divided into four subsets, each with different recording intervals: ETTh1 and ETTh2 have hourly recordings, while ETTm1 and ETTm2 are recorded every 15 minutes. Environment (Liu et al., 2024a)This dataset comprises daily air quality time series collected from the U.S. EPA spanning the full available period, recorded at a daily frequency. Exchange (W...
2021
-
[2019]
Geng, Z., Deng, M., Bai, X., Kolter, J. Z., and He, K. Mean flows for one-step generative modeling.arXiv preprint arXiv:2505.13447, 2025a. Geng, Z., Lu, Y ., Wu, Z., Shechtman, E., Kolter, J. Z., and He, K. Improved mean flows: On the chal- lenges of fastforward generative models.arXiv preprint arXiv:2512.02012, 2025b. Gruver, N., Finzi, M., Qiu, S., and ...
-
[2021]
Wang, D., Cheng, M., Liu, Z., and Liu, Q. Timedart: A dif- fusion autoregressive transformer for self-supervised time series representation.arXiv preprint arXiv:2410.05711,
-
[2022]
Liu, H., Xu, S., Zhao, Z., Kong, L., Prabhakar Kamarthi, H., Sasanur, A., Sharma, M., Cui, J., Wen, Q., Zhang, 9 CoGenCast: A Coupled Autoregressive–Flow Generative Framework for Time Series Forecasting C., et al. Time-mmd: Multi-domain multimodal dataset for time series analysis.Advances in Neural Information Processing Systems, 37:77888–77933, 2024a. Li...
-
[2023]
Guo, X., Zhang, Y ., Chen, B., Xu, H., Wang, J., and Long, M. Dynamical diffusion: Learning temporal dynamics with diffusion models.arXiv preprint arXiv:2503.00951,
-
[2024]
Cheng, M., Liu, Z., Tao, X., Liu, Q., Zhang, J., Pan, T., Zhang, S., He, P., Zhang, X., Wang, D., et al. A com- prehensive survey of time series forecasting: Concepts, challenges, and future directions.Authorea Preprints, 2025a. Cheng, M., Tao, X., Liu, Q., Zhang, H., Chen, Y ., and Lian, D. Cross-domain pre-training with language models for transferable ...
-
[2025]
Deconstructing denoising diffusion models for self-supervised learning
Chen, X., Liu, Z., Xie, S., and He, K. Deconstructing denoising diffusion models for self-supervised learning. arXiv preprint arXiv:2401.14404,
-
[2026]
Luo, Y ., Zhou, Y ., Cheng, M., Wang, J., Wang, D., Pan, T., and Zhang, J. Time series forecasting as reasoning: A slow-thinking approach with reinforced llms.arXiv preprint arXiv:2506.10630,
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.