Pith. sign in

REVIEW 3 major objections 7 minor 2 cited by

Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Model

T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a non-causal, encoder-only time series transformer exhibits output scaling: extending the total forecast duration beyond the target horizon improves accuracy for the fixed target, an effect it names delayed…

desk verdict A genuinely new empirical effect—longer outputs improve fixed-horizon forecasts—buried under an unsupported 'delayed chain-of-thought' story; the finding deserves careful scrutiny, the mechanism does not. read the letter →

arxiv 2506.11029 v1 pith:BKC5MG6K submitted 2025-05-20 cs.LG cs.AI

Xue Wang , Tian Zhou , Jinyang Gao , Bolin Ding , Jingren Zhou This is my paper
classification cs.LGcs.AI
keywords timeseriesforecastingfoundationmodelsoutputscalingdelayedchain-of-thoughtnon-causalattentionmaskedtokenrecoveryzero-shotinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

YingLong is a pretrained time series forecasting model that treats forecasting more like language understanding than language generation: a bidirectional, encoder-only transformer trained by masked token recovery predicts all future points jointly rather than recursively or independently. The paper's central discovery is an output-scaling effect: for a fixed target horizon, the longer the total output sequence the model is asked to produce, the more accurate its predictions of that target become. The authors attribute this to delayed chain-of-thought (DCoT): tokens to the right of the target can influence it through non-causal attention, effectively acting as conditional anchors, which is useful because periodic and low-frequency structure makes some future points predictable. If the claim holds, forecasting systems can gain accuracy at inference time simply by requesting and generating extra future output, without retraining. On the 23-dataset GIFT-Eval benchmark, the 300M model with a 4096-length DCoT reportedly improves MASE by 10.5% and its average rank from 17.2 to 7.38.

What carries the argument

The mechanism that carries the argument is a masked-token joint forecasting architecture: an encoder-only transformer with fully dense (non-causal) bidirectional attention over the concatenation of input patches and masked output patches, trained by randomly masking 20% of patches and recovering them with a weighted quantile loss. At test time, the masked positions are extended to include the full requested output length, and because attention is non-causal, every output token—including those to the right of the target horizon—can influence the target. This is what enables delayed chain-of-thought. The architecture also includes a U-shaped transformer with token merging and long skip connections and a multi-input ensemble with temporal mirroring, but the paper's ablations indicate the joint forecasting and DCoT design, not the U-shaped backbone, is the main source of gains.

What would settle it

Take the 300M YingLong model and compare DCoT lengths 0 and 4096 on phase-randomized or shuffled versions of the GIFT-Eval datasets, which preserve the marginal distribution but destroy periodic and low-frequency structure; if the DCoT explanation is correct, the large and consistent MASE/CRPS improvement seen on the original data should largely disappear because future points no longer provide predictable anchors. A cleaner architectural check is to run the same joint forecasting model with a causal attention mask instead of bidirectional attention; if the mechanism is delayed chain-of-thought, output scaling should not occur under the causal mask.

Watch

Extended reading notes

Core claim

The core claim is that in joint forecasting, longer outputs improve predictions of shorter, fixed targets. Formally, the paper proposes the delayed chain-of-thought effect: with a non-causal bidirectional encoder, tokens placed after the target can serve as 'thoughts' that increase the conditional probability of the target, just as chain-of-thought tokens before an answer do in language models. The authors argue that time series contain periodic and low-frequency structures that make some future points more predictable, so those points can anchor the forecast of earlier horizons. In their evaluation, output lengths of 1024, 2048, and 4096 produce progressively larger gains over the no-DCoT baseline on the GIFT-Eval benchmark, with a reduction of up to 10.5% in MASE and 11.9% in CRPS, and the same pattern appears in vanilla transformer baselines, indicating the effect survives architectural changes. The paper also claims this effect is distinct from direct and recursive paradigms, where extending output does not affect earlier predictions.

Load-bearing premise

The load-bearing premise is that the extra future points generated beyond the target horizon are predictable enough and correlated with the target to act as informative anchors; if those points are essentially noise or unrelated to the target, the output-scaling gain should vanish or reverse.

Editorial extensions

If this is right

  • At inference, practitioners can improve a fixed target horizon by asking the model for a longer output window, with no retraining; in the reported experiments longer DCoT lengths consistently beat shorter ones.
  • The effect is not tied to the U-shaped architecture: vanilla transformers trained in the joint paradigm show output scaling too, and larger models benefit more, so output length acts like an additional scaling axis.
  • The error reduction comes mostly from trend components in an STL decomposition, implying the gain is about capturing low-frequency structure rather than seasonal or residual detail.
  • When combined with the multi-input ensemble, the gains stack: variable lookback lengths with sign-mirrored inputs buy an additional 1–4% accuracy.
  • Because direct and recursive paradigms cannot let later outputs influence earlier ones, they should not exhibit this scaling; this gives a clean way to distinguish the DCoT mechanism in practice.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If output scaling is general, then inference-time compute—the number of generated future tokens—becomes a tunable accuracy knob for time series foundation models, parallel to how more reasoning tokens can help in language models; this is an implied but not tested use of the paper's result.
  • The DCoT mechanism depends on future points being predictable anchors, so gains should shrink on datasets with weak autocorrelation or chaotic dynamics; a direct cross-dataset correlation between autocorrelation length and DCoT benefit would test this.
  • The mirrored-input ensemble suggests that any invertible transformation of the input (not only sign flip) could serve as a consistency constraint; one could extend the ensemble to phase shifts or scaling transforms and measure whether ensemble diversity further reduces variance.
  • The paper's framing implies that adding DCoT at pretraining time, rather than only at inference time, might teach the model to rely on future anchors more strongly; a training curriculum that always includes long outputs could amplify the effect.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes YingLong, a non-causal encoder-only transformer for time series forecasting trained with masked token recovery under a 'joint forecasting' paradigm that predicts all horizons in parallel. The central claim is a novel 'output scaling' effect: extending the output sequence beyond the target horizon improves accuracy for a fixed target horizon, which the authors attribute to 'delayed chain-of-thought' (DCoT) reasoning. The authors also propose a multi-input ensemble that averages forecasts over multiple lookback lengths with temporal mirroring. They release four models (6M-300M) and evaluate zero-shot on ETT/Weather and on the GIFT-Eval benchmark (23 datasets), reporting that the 300M model achieves the best rank among all baselines and that DCoT yields up to a 10.5% MASE reduction on GIFT-Eval. The paper includes structural ablations, an analysis attributing DCoT gains mainly to trend errors, and an appendix with extensive per-domain and per-frequency results.

Significance. If the output scaling effect is genuine and specifically attributable to future tokens acting as 'conditional anchors,' it would be a practically useful form of inference-time scaling for time series foundation models, analogous to chain-of-thought in LLMs. The paper's strengths include a broad benchmark evaluation on GIFT-Eval, the release of pretrained models, and ablation evidence that the scaling trend persists across model sizes and with a vanilla transformer architecture. However, the causal role of the delayed tokens is not established: the probabilistic justification in Section 5.1 is invalid as a general statement, and the DCoT ablation in Figure 4/Table 12 does not rule out masking or positional artifacts. The empirical effect is plausible and deserves further study, but the mechanistic interpretation currently outruns the evidence.

major comments (3)
  1. [Section 5.1, 'Probabilistic Interpretation of CoT'] The inequality P(target=truth | prompt, CoT) >= P(target=truth | prompt) is not valid for arbitrary CoT tokens; conditioning on additional random variables can decrease the probability of the target when those variables are uninformative or misleading. This inequality is the only theoretical justification for DCoT in the paper, and as stated it does not constitute a proof. The authors should either derive explicit conditions under which the inequality holds for delayed tokens (for example, when the CoT tokens are drawn from the true conditional distribution of the future given the past), or clearly present the argument as an intuition rather than a probabilistic theorem.
  2. [Section 6.5, Figure 4, Appendix Table 12] The DCoT ablation varies only the number of masked future tokens while holding the target length fixed, which confounds the number of masked positions, the total sequence length, the position of the target tokens within the sequence, and the information content of the extra tokens. The observed monotone improvement is also consistent with (a) better alignment with the training mask ratio rho=0.2, (b) positional-encoding effects from shifting target tokens earlier in the sequence, or (c) a general benefit of additional context tokens. To support the DCoT interpretation, the authors should add control experiments such as random placeholder tokens in the future positions, ground-truth future tokens (oracle upper bound), and extra masked tokens inserted before the prompt to control for sequence length and mask count without future information.
  3. [Section 6.5, GIFT-Eval reporting] The paper reports point estimates for MASE/CRPS/Rank without confidence intervals or significance tests, and the maximum DCoT length of 4096 appears to be selected because it gives the best GIFT-Eval results. Given the 23 datasets and the effect sizes (e.g., 6.4% MASE reduction at 1024 vs. 10.5% at 4096 for the 300M model), the authors should report variability across datasets (e.g., per-dataset win/loss counts) and ideally pre-register or hold out the DCoT-length choice. The monotonicity across model sizes and lengths makes pure noise unlikely, but selection on the test benchmark remains a concern for the magnitude of the claimed improvement.
minor comments (7)
  1. [Abstract] There is a missing space in 'respectively.The pretrained 300M model'.
  2. [Section 1] The benchmark name is misspelled: 'NLU METE Benchmark' should be 'NLU MTEB Benchmark'.
  3. [Section 3, Eq. (1)] The direct forecasting negative log-likelihood expression is not well-formed: the index j is not quantified, and the summation limits appear inconsistent with a direct multi-horizon objective.
  4. [Table 1] The 'Rank' row contains two numbers per model column (e.g., '5.45 5.00'), which is confusing; clarify that these are the ranks for MSE and MAE respectively.
  5. [Appendix Table 6] There are several typos in the table, such as '0.262 0305' and '5.857,9', which make the results hard to read.
  6. [Appendix Table 7] The domain entry 'Salses' should be 'Sales', and the frequency for 'Kaggle Web Traffic Weekly' is listed as 'D' but should be 'W' (weekly).
  7. [Appendix D] The sentence 'The addition results aggregated in prediction lengths...' should read 'The additional results...'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the output-scaling effect is measured on held-out benchmarks, and the DCoT label is an interpretive name, not a fitted input or self-citation.

full rationale

YingLong's central empirical claim is the output-scaling effect: fixing the target horizon, longer overall output length improves accuracy. This is established by the ablation in Figure 4 and Section 6.5 on the GIFT-Eval benchmark, which is external to the pretraining data and not used to fit any parameter in the model. The gains are reported for multiple model sizes and also for vanilla transformers (Appendix H), so the effect is not an artifact of a single fitted architecture. No load-bearing step reduces by construction to its inputs: the prediction head in Eq. (4) and the loss in Eq. (5) are standard quantile objectives, and the DCoT variable is simply the number of masked future tokens, whose effect is measured rather than fitted. The paper contains no load-bearing self-citation chain: its benchmark reference (Aksu et al. 2024) and architectural citations are external, and none of the cited works supplies a "uniqueness theorem" or ansatz that the present conclusions depend on. The main caveat is interpretive: Section 5.1 asserts P(target=truth | prompt, CoT) >= P(target=truth | prompt) as a premise, which is not a general probabilistic theorem, and Section 7 concedes that the delayed thought process remains hidden in latent space. This is an explanatory gap or a correctness risk, not a circular reduction, because the empirical output-scaling curve stands independently of the DCoT interpretation placed on it. No equation is equivalent to the claimed result by definition, and no fitted value is renamed as a prediction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The ledger records the hand-picked inference choices (DCoT length, ensemble lengths) and training choices (mask ratio, patch size) that the claims depend on, plus the domain assumptions behind the DCoT interpretation. The paper does not introduce a new physical entity, but it does postulate a latent reasoning process that is not independently observable, which is listed as an invented entity with independent_evidence false.

free parameters (4)
  • DCoT output length = 4096 tokens
    Selected as the best among 720, 1024, 2048, 3072, and 4096 in GIFT-Eval ablations (Figure 4, Section 6.5). The output-scaling claim is quantified at this chosen length.
  • Multi-input ensemble lookback lengths = 512, 1024, 2048, 4096
    Chosen in Section 5.2 to balance short and long lookback behavior; ensemble averaging contributes the reported 1 to 4 percent gain (Table 13).
  • Masking ratio rho = 0.2
    Set in Table 4 for all models; controls the masked-token pretraining objective and the inference masked-placeholder setup. No sensitivity analysis is reported.
  • Patch size = 32
    Set in Table 4; determines token granularity and interacts with all sequence-length choices. No sensitivity analysis is reported.
assumptions (4)
  • ad hoc to paper Conditioning on extra DCoT tokens increases target likelihood: P(target=truth|prompt,CoT) >= P(target=truth|prompt).
    Asserted in Section 5.1 without proof. This inequality is false for arbitrary CoT tokens and is load-bearing for the DCoT explanation.
  • domain assumption Future points beyond the target horizon are predictable enough to serve as anchors via periodic and low-frequency structure.
    Section 5.1 states that DCoT exploits periodic behavior, but no evidence is given that such points are easier to predict across all GIFT-Eval domains.
  • domain assumption Time series forecasting aligns more with natural language understanding than with natural language generation, so non-causal masked recovery is a suitable pretraining objective.
    Introduced in the Introduction and Related Work; motivates the architecture but is not isolated from other design choices except through the structural ablation.
  • domain assumption Masked token recovery pretraining with random masked patches transfers to inference where target and extra tokens are all masked placeholders.
    Section 4.1 and 4.3 rely on this transfer. If the train/inference masking mismatch were large, the DCoT effect could be an artifact of inference setup.
invented entities (1)
  • Delayed chain-of-thought (DCoT) latent reasoning process
    purpose: Proposed mechanism to explain why longer outputs improve fixed-horizon accuracy in the non-causal encoder-only model.
    No direct probe of intermediate reasoning tokens is provided. The only evidence is the output-length ablation used to define the effect, plus an indirect trend-error pattern, which is not a falsifiable handle on reasoning.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Model." pith.science (2026). https://pith.science/paper/BKC5MG6K

@misc{pith2026250611029,
  author       = {Pith},
  title        = {Pith review of: Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BKC5MG6K}},
  note         = {Machine review of arXiv:2506.11029}
}
read the original abstract

We present a joint forecasting framework for time series prediction that contrasts with traditional direct or recursive methods. This framework achieves state-of-the-art performance for our designed foundation model, YingLong, and reveals a novel scaling effect: longer outputs significantly enhance model accuracy due to delayed chain-of-thought reasoning in our non-causal approach. YingLong is a non-causal, bidirectional attention encoder-only transformer trained through masked token recovery, aligning more effectively with language understanding tasks than with generation tasks. Additionally, we boost performance by tackling output variance with a multi-input ensemble. We release four foundation models ranging from 6M to 300M parameters, demonstrating superior results in zero-shot tasks on the ETT and Weather datasets. YingLong achieves more than 60% best performance. To ensure generalizability, we assessed the models using the GIFT-Eval benchmark, which comprises 23 time series datasets across 7 domains. Yinglong significantly outperformed the best time-series foundation models, end-to-end trained models by 14% and 44% in rank respectively.The pretrained 300M model is available at https://huggingface.co/qcw1314/YingLong_300m

Figures

Figures reproduced from arXiv: 2506.11029 by the authors.

Figure 1
Figure 1. Joint Forecasting with Output Scaling via Delayed Chain of Thought (DCoT). In joint [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the architecture of YINGLONG. 4.2 Unet-Transformer For our transformer block, we adopt the standard architecture (Vaswani, 2017) with bidirectional attention. We use RMSNorm (Zhang & Sennrich, 2019) for pre-normalization, SwiGLU (Shazeer, 2020) as the activation function, and rotary positional embeddings (Su et al., 2024). U-shaped design Inspired by U-shaped generative models (e.g., Bao et al. 2023;… view at source ↗
Figure 3
Figure 3. Training with Masked Token Prediction. 5 Output Scaling 5.1 Delayed Chain of Thoughts Beyond the efficient direct prediction scheme described earlier, our approach also facilitates a new form of chain of thoughts (CoT) for time series forecasting. Chain of thoughts has been extensively studied in the field of natural language processing (NLP), where generating additional tokens (i.e., “thoughts”) can significantly i… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Ablation on DCoT. Four different output lengths up to 4096 are considered. In no CoT [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Comparison of relative MSE reduction in ETTm1 models. [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: Structure ablation:6M size Transformer, add Token-Merge, and uTransformer( [PITH_FULL_IMAGE:figures/full_fig_p020_6.png]
Figure 7
Figure 7. Figure 7: Output Scaling for 6M to 300M vanilla transformer model following joint forecasting [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CITRAS-FM: Tiny Time Series Foundation Model for Covariate-Informed Zero-Shot Forecasting

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    CITRAS-FM is a 7M-param decoder-only Transformer TSFM with Shifted Attention and CovSynth synthetic covariate pretraining that claims SOTA zero-shot accuracy among sub-10M models on fev-bench with sub-0.1s CPU inference.

  2. Beyond Accuracy: Are Time Series Foundation Models Well-Calibrated?

    cs.LG 2025-10 conditional novelty 6.0 of 10

    Time-series foundation models are better calibrated than ARIMA and N-BEATS baselines on the tested datasets, and their calibration does not show the systematic overconfidence seen in image and language models.

Reference graph

Works this paper leans on

27 extracted references · 6 canonical work pages · cited by 2 Pith papers

  1. [1]

    Gift-eval: A benchmark for general time series forecasting model evaluation

    Aksu, T., Woo, G., Liu, J., Liu, X., Liu, C., Savarese, S., Xiong, C., and Sahoo, D. Gift-eval: A benchmark for general time series forecasting model evaluation. arxiv preprint arxiv:2410.10393,

    arXiv

  2. [6]

    and Al-Onaizan, Y

    Freitag, M. and Al-Onaizan, Y . Beam search strategies for neural machine translation.arXiv preprint arXiv:1702.01806,

    arXiv

  3. [7]

    MASE = m − s n · Pm+n t=m+1 |ˆxt − xt| Pm−s t |xt − xt+s| , 14 where m in the lookback length, n is the forecasting length, and s is the seasonality parameter

    We consider mean absolute scaled error (MASE), continuous ranked probability score (CRPS) and Rank as metrics. MASE = m − s n · Pm+n t=m+1 |ˆxt − xt| Pm−s t |xt − xt+s| , 14 where m in the lookback length, n is the forecasting length, and s is the seasonality parameter. In this work, we follow the setting in Aksu et al. (2024) and use weighted quantile lo...

    work page 2024

  4. [9]

    B., Müller, S., Salinas, D., and Hutter, F

    10 Hoo, S. B., Müller, S., Salinas, D., and Hutter, F. The tabular foundation model tabpfn outperforms spe- cialized time series forecasting models based on simple features. arXiv preprint arXiv:2501.02945,

    arXiv

  5. [11]

    Towards general text embeddings with multi-stage contrastive learning

    Li, Z., Zhang, X., Zhang, Y ., Long, D., Xie, P., and Zhang, M. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281,

    arXiv

  6. [13]

    itransformer: Inverted transformers are effective for time series forecasting

    Liu, Y ., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., and Long, M. itransformer: Inverted transformers are effective for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024a. Liu, Y ., Qin, G., Huang, X., Wang, J., and Long, M. Autotimes: Autoregressive time series forecasters via large language models. arXi...

    arXiv

  7. [14]

    Mteb: Massive text embedding benchmark

    Muennighoff, N., Tazi, N., Magne, L., and Reimers, N. Mteb: Massive text embedding benchmark. arXiv preprint arXiv:2210.07316,

    arXiv

  8. [15]

    H., Constant, N., Ma, J., Hall, K

    Ni, J., Abrego, G. H., Constant, N., Ma, J., Hall, K. B., Cer, D., and Yang, Y . Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models. arXiv preprint arXiv:2108.08877,

    arXiv

Show all 27 references
  1. [16]

    H., Sinthong, P., and Kalagnanam, J

    Nie, Y ., Nguyen, N. H., Sinthong, P., and Kalagnanam, J. A time series is worth 64 words: Long- term forecasting with transformers. In The Eleventh International Conference on Learning Representations, 2023a. URL https://openreview.net/forum?id=Jbdc0vTOcol. Nie, Y ., Nguyen, ...

    arXiv

  2. [18]

    Statistical characterization of business-critical workloads hosted in cloud datacenters

    Shen, S., Van Beek, V ., and Iosup, A. Statistical characterization of business-critical workloads hosted in cloud datacenters. In 2015 15th IEEE/ACM international symposium on cluster, cloud and grid computing, pp. 465–474. IEEE,

    2015

  3. [19]

    Accelerating auto- regressive text-to-image generation with training-free speculative jacobi decoding

    Teng, Y ., Shi, H., Liu, X., Ning, X., Dai, G., Wang, Y ., Li, Z., and Liu, X. Accelerating auto- regressive text-to-image generation with training-free speculative jacobi decoding. arXiv preprint arXiv:2410.01699,

    arXiv

  4. [20]

    Attention is all you need

    Vaswani, A. Attention is all you need. arXiv preprint arXiv:1706.03762,

    arXiv

  5. [21]

    ISBN 9781450386647

    Association for Computing Machinery. ISBN 9781450386647. doi: 10.1145/3474717.3483923. URL https://doi.org/ 10.1145/3474717.3483923. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V ., Zhou, D., et al. Chain-of- thought prompting elicits reasoning in la...

  6. [22]

    Do large language models latently perform multi-hop reasoning? arXiv preprint arXiv:2402.16837,

    12 Yang, S., Gribovskaya, E., Kassner, N., Geva, M., and Riedel, S. Do large language models latently perform multi-hop reasoning? arXiv preprint arXiv:2402.16837,

    arXiv

  7. [23]

    Least-to-most prompting enables complex reasoning in large language models

    Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Cui, C., Bousquet, O., Le, Q., et al. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625, 2022a. Zhou, H., Zhang, S., Peng, J., Zhang, S., Li...

    arXiv

  8. [24]

    Film: Frequency improved legendre memory model for long-term time series forecasting

    Zhou, T., Ma, Z., Wen, Q., Sun, L., Yao, T., Yin, W., Jin, R., et al. Film: Frequency improved legendre memory model for long-term time series forecasting. Advances in neural information processing systems, 35:12677–12690, 2022b. Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, L., a...

    2022

  9. [26]

    2021 Energy 30T 1153584Bitcoin Godahewa et al

    Table 3: Pretraining Datasets Dataset Source domain Frequency # Length Australian Electricity Demand Godahewa et al. 2021 Energy 30T 1153584Bitcoin Godahewa et al. 2021 Econ/Fin D 68927Cif 2016 Godahewa et al. 2021 Econ/Fin M 7108Cif 2016 Godahewa et al. 2021 Econ/Fin M 7108Fr...

    2021

  10. [27]

    This presents a scalable ’free lunch’ approach, yielding post-training improvements through simple ensemble averaging

    Our input-ensemble strategy consistently enhances accuracy by 1% to 4% without necessitating additional model training. This presents a scalable ’free lunch’ approach, yielding post-training improvements through simple ensemble averaging. When compared to the single-worst comp...

  11. [2015]

    T., Rubinstein, M., et al

    Chang, H., Zhang, H., Barber, J., Maschinot, A., Lezama, J., Jiang, L., Yang, M.-H., Murphy, K., Freeman, W. T., Rubinstein, M., et al. Muse: Text-to-image generation via masked generative transformers. arXiv preprint arXiv:2301.00704,

    arXiv

  12. [2017]

    Retrieval-augmented generation for large language models: A survey

    Gao, Y ., Xiong, Y ., Gao, X., Jia, K., Pan, J., Bi, Y ., Dai, Y ., Sun, J., and Wang, H. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997,

    arXiv

  13. [2018]

    Nv-embed: Improved techniques for training llms as generalist embedding models

    Lee, C., Roy, R., Xu, M., Raiman, J., Shoeybi, M., Catanzaro, B., and Ping, W. Nv-embed: Improved techniques for training llms as generalist embedding models. arXiv preprint arXiv:2405.17428,

    arXiv

  14. [2019]

    Glu variants improve transformer

    Shazeer, N. Glu variants improve transformer. arXiv preprint arXiv:2002.05202,

    2002 arXiv

  15. [2020]

    Ansari, A

    URL http://jmlr.org/papers/v21/19-820.html. Ansari, A. F., Stella, L., Turkmen, C., Zhang, X., Mercado, P., Shen, H., Shchur, O., Rangapuram, S. S., Arango, S. P., Kapoor, S., et al. Chronos: Learning the language of time series. arXiv preprint arXiv:2403.07815,

    arXiv

  16. [2021]

    Moe-llava: Mixture of experts for large vision-language models

    Lin, B., Tang, Z., Ye, Y ., Cui, J., Zhu, B., Jin, P., Zhang, J., Ning, M., and Yuan, L. Moe-llava: Mixture of experts for large vision-language models. arXiv preprint arXiv:2401.15947,

    arXiv

  17. [2022]

    Y ., Rozière, B., Lopez-Paz, D., and Synnaeve, G

    Gloeckle, F., Idrissi, B. Y ., Rozière, B., Lopez-Paz, D., and Synnaeve, G. Better & faster large language models via multi-token prediction. arXiv preprint arXiv:2404.19737,

    arXiv

  18. [2023]

    J., Sun, J., and Liu, C

    Chen, M., Shen, L., Li, Z., Wang, X. J., Sun, J., and Liu, C. Visionts: Visual masked autoencoders are free-lunch zero-shot time series forecasters. arXiv preprint arXiv:2408.17253,

    arXiv

  19. [2024]

    Long-term forecasting with tide: Time-series dense encoder

    Das, A., Kong, W., Leach, A., Mathur, S., Sen, R., and Yu, R. Long-term forecasting with tide: Time-series dense encoder. arXiv preprint arXiv:2304.08424,

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.