Byte Pair Encoding for Efficient Time Series Forecasting

Leon G\"otz; Leo Schwinn; Marcel Kollovieh; Stephan G\"unnemann

REVIEW 3 major objections 5 minor 1 cited by

Byte Pair Encoding for Efficient Time Series Forecasting

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Time series can be tokenized as recurring variable-length motifs, and doing so makes zero-shot forecasting more accurate and far cheaper.

desk verdict A genuinely novel BPE-style tokenization for time series with a useful decoding trick, but the headline gains are best-of-selection and overstate the case. read the letter →

arxiv 2505.14411 v4 pith:ACTK24V7 submitted 2025-05-20 cs.LG

Leon Götz , Marcel Kollovieh , Stephan Günnemann , Leo Schwinn This is my paper

classification cs.LG

keywords bytepairencodingtimeseriestokenizationmotifvocabularyconditionaldecodingzero-shotforecastingdiscreteadaptivecompressionfoundationmodels

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the right tokenization for time series is pattern-centric: instead of one token per sample or fixed-length patches, the input should be compressed into variable-length tokens that correspond to frequently recurring motifs. It borrows byte pair encoding from NLP, quantizes each normalized series into discrete bins, then repeatedly merges the most frequent adjacent symbol pairs into a single motif token until no pair is common enough to be worth learning. On five standard datasets in a zero-shot setting, the authors report that this motif vocabulary lowers MSE by 36.1 percent and speeds up inference by 19.90 times on average compared with single-sample Chronos models, and that adding conditional decoding raises those gains to 43.2 percent and 26.73 times. The paper also introduces conditional decoding, a small first-order Markov lookup table that replaces bin-center decoding and removes much of the discretization error at no extra inference cost. If the claim holds, long-context forecasting becomes substantially cheaper and the bottleneck for foundation models shifts from token count to motif coverage.

What carries the argument

The load-bearing object is the motif vocabulary $\Psi$, built by byte pair merging on a quantized series: $q_\Omega$ maps each sample to one of $M$ equiprobable bins, and $m_\Psi$ replaces the most frequent adjacent pair with a new token until occurrences fall below $p_{\min}$. The compression ratio $\bar{c}=n/t$ relates input samples to motif tokens and is what turns token counts into wall-clock speedups. The second mechanism is conditional decoding, a table $\hat{\omega}_{j,k}$ of per-pair mean sample values computed analytically by minimizing squared error over the training data; it is a first-order Markov dequantizer with $M^2$ parameters (484 in the high-compression case) that removes up to 96.9 percent of the quantization error in a tokenizer-only setting.

What would settle it

Rerun the zero-shot benchmark with the 100,000-series vocabulary sample restricted to exclude all five evaluation datasets; if the MSE advantage over single-sample Chronos disappears or drops materially, the reported generalization depends on vocabulary overlap. A simpler version: train a tokenizer on synthetic series whose motifs deliberately exclude the flat and periodic shapes of one evaluation dataset, and check whether compression falls to $\bar{c}\approx 1$ and MSE returns to the Chronos baseline on that dataset.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that a discrete vocabulary of frequent temporal motifs is a better tokenization for time series forecasting than per-sample tokens or fixed-length patches. The construction normalizes each series, quantizes it into $M$ equiprobable bins, and then applies byte pair merging: the most frequent adjacent pair of tokens is replaced by a new token, iteratively, until no pair occurs at least $p_{\min}$ times; every merged pair is a motif, and motifs of different lengths end up as single tokens. Forecasting is next-token prediction over this vocabulary, so flat stretches compress into one token while complex stretches stay fine-grained. The reported consequence is that the motif representation Pareto-dominates sample-wise Chronos tokenization on all five zero-shot datasets and beats a patch-based Chronos variant by 21.3 percent in MSE (32.4 percent with conditional decoding), while conditional decoding—decoding each quantized symbol by the mean true value of its (previous-symbol, current-symbol) pair—cuts MSE by up to 44.3 percent on ETTm1 with only 484 parameters.

Load-bearing premise

The results stand on the premise that the motifs found in 100,000 randomly sampled pretraining series cover the recurring patterns of the five target datasets, and the paper does not state whether those target datasets were part of the sample; when a target pattern is absent, tokenization falls back to single-sample tokens and the reported gains shrink.

Editorial extensions

If this is right

A coarser 22-bin tokenizer with larger quantization error wins 15 of 25 model-dataset settings, so motif expressiveness can outweigh discretization loss; the paper argues this favors long, frequent motifs.
Longer motifs correlate with lower token-wise MSE on all five datasets, so vocabularies tuned to preserve long motifs should improve both compression and accuracy.
Input compression and generation compression are positively correlated, so efficiency gains compound when models are given longer context.
The tokenizer keeps working when trends, noise, and transients are injected, because unknown patterns fall back to single-sample tokens, so the representation cannot overlook individual samples by design.
Conditional decoding can be bolted onto any pretrained model with a discrete output vocabulary, so the accuracy gain is portable beyond the specific T5 backbone used in the paper.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension the paper does not pursue: build one shared motif vocabulary and use it across multiple sampling frequencies, since the tokenizer already compresses quarter-hourly ETTm1 more aggressively than hourly ETTh1 without retraining.
The conditional-decoding table could be estimated per dataset instead of globally, which would let the dequantizer adapt to local distribution shift; the paper fixes a single table from the training corpus.
Because parent-child motif embeddings are close (cosine similarity 0.475 vs 0.072 average), a tree-structured sampling rule that prefers a parent token before its children might improve autoregressive coherence beyond greedy decoding.
Vocabulary-building is cheap enough (3.8 hours on one CPU for 100 million samples) that a deployment could rebuild the tokenizer on a target corpus, turning the coverage assumption into a tunable design choice rather than a fixed pretrained artifact.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First thing to know: this paper applies BPE to time series tokenization, building a vocabulary of frequent motifs from a quantized corpus, and adds a cheap conditional decoding table that removes part of the discretization error. The core idea is sensible and the experiments are extensive. But the headline numbers (36% MSE improvement, 19.90x speedup) are best-of-15 selections per dataset, and the speedup average hides a 0.26x slowdown on Weather and is inflated by a 56.66x outlier on Traffic where the Chronos baseline diverged. The full Table 12 supports a real but more modest gain.

What's genuinely new: pattern-centric adaptive tokenization for time series, which no one else has done in this form. The conditional decoding trick is simple and effective, and the ablations (quantization granularity, p_min, dataset size, noise, transients, non-stationarity) are thorough. The token embedding analysis is a nice touch.

The soft spots: (1) The reported averages should not be computed by selecting the best among 15 configurations per dataset. A fixed model-selection rule would give a fairer headline, and the Traffic divergence should be reported with and without. (2) The arXiv metadata abstract says 40%/2314%/48% while the body says 36.1%/19.90x/44%. That needs reconciliation. (3) The paper does not state whether the five evaluation datasets were excluded from the 100k-series Chronos sample used to build the vocabulary. If they were included, the zero-shot claim is weakened; the authors need to say so explicitly. (4) No code released, and main results are single-run with no error bars; the appendix gives standard deviations for small models only. (5) Conditional decoding is fit on each target dataset's train split, which is fine but means 'zero-shot' applies only to the tokenizer, not the full pipeline.

Who should read this: anyone working on tokenization for time series foundation models. It deserves a serious referee; the idea is novel and the experiments are substantial, but the presentation of the headline results needs to be reworked before acceptance. I'd send it to review.

Referee Report

3 major / 5 minor

Summary. This paper proposes a byte-pair-encoding-inspired tokenization scheme for time series forecasting. A discrete vocabulary of frequent motifs is built from quantized time series in the Chronos corpus; input series are compressed into motif tokens, and a T5-style transformer is trained for next-token prediction. The paper also introduces conditional decoding, a closed-form post-hoc dequantization method that improves the mapping from tokens back to continuous values. Zero-shot evaluations on ETTh1, ETTm1, Weather, Electricity, and Traffic compare against Chronos single-sample tokenization, patch-based Chronos baselines, MOMENT, and Moirai. The headline claims are a 36.1% average MSE improvement, a 19.90x average inference speedup, and up to 44.3% MSE reduction from conditional decoding.

Significance. The direction is timely and potentially valuable: adaptive, pattern-based tokenization could reduce the compute of time series foundation models, and conditional decoding is a simple, elegant post-hoc correction with a closed-form optimum that appears applicable to any discrete-vocabulary forecasting model. The paper is unusually thorough in its empirical reporting: full per-configuration results in Table 12, seed standard deviations, computational cost estimates, and extensive ablations of quantization granularity, vocabulary size, training data size, noise robustness, non-stationarity, and token embedding structure. If the efficiency and accuracy claims are appropriately qualified, this is a solid empirical contribution.

major comments (3)

[Section 5.1, Table 2] The headline efficiency and accuracy gains are computed by per-dataset selection among 3 tokenizers × 5 model sizes ('best MSE' and 'fastest model that still beats the best Chronos reference'), then averaging the per-dataset ratios. This is an extreme statistic rather than a property of the tokenizer. The Weather row reports a 0.26x slowdown under the 'fastest better than reference' criterion, and Traffic's 56.66x speedup is measured against a Chronos reference whose MSE is 2.717, which the paper notes diverged. The average 19.90x speedup and 36.1% MSE improvement are therefore not representative. Please recompute aggregates under a fixed model-selection rule (e.g., same tokenizer and model size for all datasets) and report medians and per-dataset ranges; the full Table 12 shows real but more modest gains.
[Section 4 and Section 5.1] The paper does not state whether the five zero-shot evaluation datasets (ETTh1, ETTm1, Weather, Electricity, Traffic) are disjoint from the 100,000-series Chronos sample used to build the motif vocabulary. Because the evaluation is described as zero-shot and the paper claims generalization to unseen data, this exclusion must be explicit. If any of these datasets appear in the vocabulary-building sample, the compression and accuracy results could be optimistically biased. Please state the exclusion status; if they were not excluded, quantify the overlap or rebuild the vocabulary on a disjoint sample.
[Section 5.1, Figure 3] The text claims the tokenizer 'outperforms Chronos models with single-sample tokenization in forecasting quality and efficiency at the same time' and finds Pareto optimal points on all five datasets. This simultaneous-improvement claim is not supported on Weather, where the selected best-MSE point (0.251 MSE, 0.367 s) is slower than the best Chronos point (0.265 MSE, 0.094 s). The text should qualify that simultaneous improvement holds on the other datasets, while on Weather the method trades speed for accuracy.

minor comments (5)

[Abstract] The abstract in the arXiv metadata (40%, 2314%, 48%) does not match the abstract and results in the body (36%, 1990%, 44%). Please ensure the metadata matches the manuscript.
[Section 4] The paper should clarify that the motif-based T5 models are trained from scratch with the Chronos training recipe, not initialized from the released pretrained Chronos checkpoints, since the abstract says 'on recent time series foundation models.'
[Algorithm 1] In Algorithm 1, the update D' <- D' \ {pair} union {z'_new} is ambiguous for a multiset; it should specify that one occurrence of the pair is replaced by one occurrence of the new token, or describe the count updates.
[Algorithm 2] The caption of Algorithm 2 contains a typo: 'discrtized' should be 'discretized.'
[Section 5.3] In Section 5.3, 'the respective train set' for fitting conditional decoding should be explicitly identified as the training split of the target dataset, in contrast to the data- and model-independent variant in Appendix C.3 that uses the Chronos corpus.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central accuracy and efficiency claims are empirical measurements on held-out test data, not consequences of the tokenizer or decoding definitions.

full rationale

The paper's derivation chain is empirical rather than definitional. The motif vocabulary is built by byte-pair-encoding quantized samples from a large pretraining corpus (Section 4), the forecasting models are trained with cross-entropy on the resulting token sequences, and the headline MSE numbers are measured on standard zero-shot test splits against Chronos and patch baselines. The conditional decoding parameters are fit by the closed-form least-squares estimator in Eq. (6) on training data and evaluated on test data; this is a normal supervised post-hoc fit, not a prediction that is equivalent to its input. No equation defines a reported improvement in terms of a fitted parameter renamed as a prediction, and no load-bearing claim rests on a self-citation chain. The only self-citations are to earlier work used for baseline comparison numbers and for the motif inductive-bias idea; they are not used to forbid alternatives or to define the present method's outputs. Possible concerns about the headline aggregates (per-dataset selection of the best/fastest configuration, the Traffic reference divergence) and about potential corpus overlap with evaluation datasets are experimental-design or reporting issues, not circularity: the underlying tables report actual inference times and MSE values, and those numbers are not forced to equal the method's inputs by construction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method rests on standard time series preprocessing (normalization, quantization) and on the BPE assumption that frequent adjacent pairs capture meaningful structure. Key hyperparameters (M, quantization range, pmin) are chosen by hand; the conditional decoding table is fit to training data. No new physical or mathematical entities are introduced.

free parameters (4)

number of quantization bins M = 22, 26, 37
Controls granularity; three values chosen for low/medium/high compression tradeoff.
quantization range [omega_lb, omega_ub] = [-5, 5]
Truncated uniform range for normalized series; chosen by hand to span 5 standard deviations.
minimum motif occurrence pmin = 1000 (varied to 128000)
Stops BPE merging when pair count falls below threshold; default 1000 for main tokenizers.
conditional decoding table omega_{j,k} = M^2 parameters (e.g., 484 for M=22)
First-order Markov dequantization values fit to training data via equation (6).

assumptions (4)

domain assumption Most frequent adjacent pairs form semantically meaningful motifs
Core of BPE; assumes patterns repeat often enough that merging improves prediction. Used throughout Section 3.2.
domain assumption Normalized zero-mean unit-variance representation is appropriate
All series are normalized before quantization; results depend on this choice.
domain assumption First-order Markov dependence is sufficient for dequantization
Conditional decoding conditions only on the previous token (Section 3.3).
domain assumption The Chronos training corpus and the 5 evaluation datasets share temporal motifs
Vocabulary is learned from 100k Chronos series and applied zero-shot to ETTh1, ETTm1, Weather, Electricity, Traffic.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Byte Pair Encoding for Efficient Time Series Forecasting." pith.science (2026). https://pith.science/paper/ACTK24V7

@misc{pith2026250514411,
  author       = {Pith},
  title        = {Pith review of: Byte Pair Encoding for Efficient Time Series Forecasting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ACTK24V7}},
  note         = {Machine review of arXiv:2505.14411}
}

read the original abstract

Existing time series tokenization methods predominantly encode a constant number of samples into individual tokens. This inflexible approach can generate excessive tokens for even simple patterns like extended constant values, resulting in substantial computational overhead. Inspired by the success of byte pair encoding, we propose the first pattern-centric tokenization scheme for time series analysis. Based on a discrete vocabulary of frequent motifs, our method merges samples with underlying patterns into tokens, compressing time series adaptively. Exploiting our finite set of motifs and the continuous properties of time series, we further introduce conditional decoding as a lightweight yet powerful post-hoc optimization method, which requires no gradient computation and adds no computational overhead. On recent time series foundation models, our motif-based tokenization improves forecasting performance by 40% and boosts efficiency by 2314% on average. Conditional decoding further reduces MSE by up to 48%. In an extensive analysis, we demonstrate the adaptiveness of our tokenization to diverse temporal patterns, its generalization to unseen data, and its meaningful token representations capturing distinct time series properties, including statistical moments and trends.

Figures

Figures reproduced from arXiv: 2505.14411 by the authors.

**Figure 1.** Motif-based tokenization transforms time series data (gray) through a two-step process: 1) quantizing samples into discrete bins, 2) merging recurring patterns of variable length into representative motifs (equivalent motifs share the same color). Motif repetition is highlighted by their x-axis projection. inefficient representation, where every token captures little temporal information. This results in very long t… view at source ↗

**Figure 2.** (a) Our motif-based tokenization first quantizes a time series into symbols and finds recurring motifs as tokens, building a discrete vocabulary. (b) Based on the compressed motif sequence, a neural network forecasts the time series through a categorical distribution over our vocabulary. (c) Finally, we propose conditional decoding to reduce the discretization error when transforming tokens back to their continuous … view at source ↗

**Figure 3.** Zero-shot comparison between our motif-based and sample-wise tokenization (Chronos) on Electricity. large using 3 tokenizers (see experimental settings in section 4). Chronos and our models are based on the same architecture, training strategy, and dataset. They only differ in tokenization. In our zero-shot evaluation, our motif-based tokenization finds Pareto optimal points on all 5 datasets. We show in figure 3 t… view at source ↗

Figures from the paper (18 more)

**Figure 4.** Figure 4: Conditional decoding improves forecasting quality for 3 tokenizers in small models on 2 datasets. 5.4. Adaptive compression of diverse time series [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Our adaptive tokenizer (a) exploits periodically recurring motifs on the Traffic dataset and (b-d) compresses time series adaptively depending on pattern complexity on the Weather dataset. 5.5. Vocabulary complexity and generalization Longer motifs benefit the compres…

**Figure 7.** Figure 7: Principal component analysis of token embeddings of our medium tokenizer in a small model. 8 [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

**Figure 8.** Figure 8: Comparison of uniform discretization with other tokenization preprocessing strategies including derivatives, Gaussian kernels, window-based norms, and Gaussian and data distribution based discretization. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]

**Figure 9.** Figure 9: Comparison of our motif-based tokenization with and without conditional decoding with Chronos models tokenizing every sample during zero-shot evaluation on 5 datasets and 5 model sizes. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]

**Figure 10.** Figure 10: Conditional decoding improves forecasting quality for 3 tokenizers in small models on 5 datasets. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]

**Figure 11.** Figure 11: Histograms showing variable compressions of our medium tokenizer within 5 datasets. Input and generation compression We conduct additional experiments to explore relations between input compression and the model’s generations. The model can either predict long motifs …

**Figure 12.** Figure 12: Comparison of input and generation compression of small models and multiple tokenizers on 5 datasets. Please note that efficiency gains in table 2 and figures 3 and 9 relate to the more conservative generation compression. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]

**Figure 13.** Figure 13: Our medium tokenizer exploits periodically recurring motifs and compresses time series adaptively depending on pattern complexity on 5 datasets. Specifically, (o) highlights the noise rejection ability of discretization. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]

**Figure 14.** Figure 14: Varying token occurrence pmin influences forecasting quality for small models on 5 datasets [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]

**Figure 15.** Figure 15: Motif hierarchy for vocabularies of different complexity. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_15.png]

**Figure 16.** Figure 16: Influence of quantization bins M and token occurrence pmin on vocabulary size |Ψ| and compression c¯ for tokenizers on the Chronos dataset. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_16.png]

**Figure 17.** Figure 17: Resistance of Chronos models and our high compression tokenizer to Gaussian noise with standard deviation σ on 5 datasets. Generalization to non-stationary data In practice, trends on long non-stationary time series might hinder effective motif encoding. To explore th…

**Figure 18.** Figure 18: Generalization of Chronos models and our high compression tokenizer to non-stationary time series with linear and exponential trends on the Electricity dataset. Robustness to transients Extreme values might occur in real-world time series. Here, we explore the robustn…

**Figure 19.** Figure 19: Introducing a transient (yellow) in the center period leads to local changes in tokenization of time series samples (gray) compared to the other periods on the Electricity dataset. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_19.png]

**Figure 20.** Figure 20: Principal component analysis of token embeddings of our medium tokenizer in a small model. 5 0 5 Component 1 5 0 5 10 Component 2 [PITH_FULL_IMAGE:figures/full_fig_p028_20.png]

**Figure 21.** Figure 21: Parent-child token relations analyzed through principal component analysis of token embeddings from our medium tokenizer in a small model. Children and their first parents are connected. 28 [PITH_FULL_IMAGE:figures/full_fig_p028_21.png]

**Figure 22.** Figure 22: Visualization of motifs which our medium compression tokenizer uses to encode time series. Colors indicate motif length. 29 [PITH_FULL_IMAGE:figures/full_fig_p029_22.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Effective Data Pruning through Score Extrapolation
cs.LG 2025-06 conditional novelty 5.0 of 10

Data pruning importance scores can be extrapolated from a 10-25% random subset to the full dataset with kNN or GNN, saving time with a small accuracy loss.

Reference graph

Works this paper leans on

16 extracted references · 8 canonical work pages · cited by 1 Pith paper

[2]

Triformer: Triangular, variable-specific attentions for long sequence multivariate time series forecasting–full version

Cirstea, R.-G., Guo, C., Yang, B., Kieu, T., Dong, X., and Pan, S. Triformer: Triangular, variable-specific attentions for long sequence multivariate time series forecasting–full version. arXiv:2204.13767,

arXiv
[4]

A decoder-only foun- dation model for time-series forecasting.arXiv:2310.10688,

Das, A., Kong, W., Sen, R., and Zhou, Y . A decoder-only foun- dation model for time-series forecasting.arXiv:2310.10688,

arXiv
[5]

Multidimensional Byte Pair Encoding: Shortened Sequences for Improved Visual Data Generation

Elsner, T., Usinger, P., Nehring-Wirxel, J., Kobsik, G., Czech, V ., He, Y ., Lim, I., and Kobbelt, L. Multidimensional byte pair encoding: Shortened sequences for improved visual data generation.arXiv:2411.10281,

work page Pith review arXiv
[6]

and Mergenthaler-Canseco, M

Garza, A. and Mergenthaler-Canseco, M. Timegpt-1. arXiv:2310.03589,

arXiv
[8]

We utilize this method throughout our paper

Uniform discretization with different number of bins M is Pareto optimal in balancing the average tokenization error δavg and compression ¯c. We utilize this method throughout our paper. /uni00000013/uni00000011/uni00000013/uni00000013/uni00000011/uni00000014/uni00000013/uni00000011/uni00000015/uni00000013/uni00000011/uni00000016/uni00000013/uni00000011/u...

work page 2025
[10]

Unified training of universal time series forecasting transformers.arXiv:2402.02592,

Woo, G., Liu, C., Kumar, A., Xiong, C., Savarese, S., and Sa- hoo, D. Unified training of universal time series forecasting transformers.arXiv:2402.02592,

arXiv
[11]

An adaptive tokenization approach for time series We provide pseudocode for generating a vocabulary of motifs and utilizing the motifs to tokenize a time series

10 Byte Pair Encoding for Efficient Time Series Forecasting A. An adaptive tokenization approach for time series We provide pseudocode for generating a vocabulary of motifs and utilizing the motifs to tokenize a time series. Algorithm 1Motif vocabulary generation according to equation (3). Input:Dataset of discretized time seriesD ′, minimum motif occurre...

work page 2024
[12]

More complex input tokens generally benefit the prediction of longer motifs. /uni00000015/uni00000011/uni00000013/uni00000016/uni00000011/uni00000013/uni00000017/uni00000011/uni00000013 /uni00000026/uni00000052/uni00000050/uni00000053/uni00000055/uni00000048/uni00000056/uni00000056/uni0000004c/uni00000052/uni00000051/uni00000003¯cin /uni00000014/uni000000...

work page 2024

Show all 16 references

[14]

In19out of25cases, our foundation model in zero-shot setting outperforms the specifically trained models in forecasting quality

The authors forecast 96 time series samples from192context tokens. In19out of25cases, our foundation model in zero-shot setting outperforms the specifically trained models in forecasting quality. Table 11.Comparison of our motif-based tokenization with conditional decoding (cd...
[15]

Due to its hierarchical structure, motif length grows exponentially with vocabulary depth, enabling substantial compression. /uni00000013/uni00000015/uni00000017/uni00000019 /uni00000039 /uni00000052/uni00000046/uni00000044/uni00000045/uni00000058/uni0000004f/uni00000044/uni00...

2000
[1999]

J., and Günnemann, S

Sommer, J., Hetzel, L., Lüdke, D., Theis, F. J., and Günnemann, S. The power of motifs as inductive bias for learning molecular distributions. InICLR 2023 - Machine Learning for Drug Discovery workshop,

2023
[2003]

X., and Dustdar, S

Liu, S., Yu, H., Liao, C., Li, J., Lin, W., Liu, A. X., and Dustdar, S. Pyraformer: Low-complexity pyramidal attention for long- range time series modeling and forecasting. InInternational Conference on Learning Representations, 2022a. Liu, X., Liu, J., Woo, G., Aksu, T., Lian...

arXiv
[2015]

Reproducibility of measurementsIn our zero-shot evaluations, we use the same data splits as Wu et al. (2021). We evaluate once and report results on the test set. Regardingpredictive quality, we report MSE standard deviations for our most common experimental settings here, as ...

2021
[2020]

R., Ghonia, H., Bhagwatkar, R., Khorasani, A., Bayazi, M

Rasul, K., Ashok, A., Williams, A. R., Ghonia, H., Bhagwatkar, R., Khorasani, A., Bayazi, M. J. D., Adamopoulos, G., Riachi, R., Hassen, N., Biloš, M., Garg, S., Schneider, A., Chapados, N., Drouin, A., Zantedeschi, V ., Nevmyvaka, Y ., and Rish, I. Lag-llama: Towards foundati...

arXiv
[2022]

Toto: Time series optimized transformer for observability.arXiv:2407.07874,

Cohen, B., Khwaja, E., Wang, K., Masson, C., Ramé, E., Doubli, Y ., and Abou-Amal, O. Toto: Time series optimized transformer for observability.arXiv:2407.07874,

arXiv
[2024]

Tirex: Zero-shot forecasting across long and short horizons with enhanced in-context learning.arXiv:2505.23719,

Auer, A., Podest, P., Klotz, D., Böck, S., Klambauer, G., and Hochreiter, S. Tirex: Zero-shot forecasting across long and short horizons with enhanced in-context learning.arXiv:2505.23719,

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [2]

Triformer: Triangular, variable-specific attentions for long sequence multivariate time series forecasting–full version

Cirstea, R.-G., Guo, C., Yang, B., Kieu, T., Dong, X., and Pan, S. Triformer: Triangular, variable-specific attentions for long sequence multivariate time series forecasting–full version. arXiv:2204.13767,

arXiv

[2] [4]

A decoder-only foun- dation model for time-series forecasting.arXiv:2310.10688,

Das, A., Kong, W., Sen, R., and Zhou, Y . A decoder-only foun- dation model for time-series forecasting.arXiv:2310.10688,

arXiv

[3] [5]

Multidimensional Byte Pair Encoding: Shortened Sequences for Improved Visual Data Generation

Elsner, T., Usinger, P., Nehring-Wirxel, J., Kobsik, G., Czech, V ., He, Y ., Lim, I., and Kobbelt, L. Multidimensional byte pair encoding: Shortened sequences for improved visual data generation.arXiv:2411.10281,

work page Pith review arXiv

[4] [6]

and Mergenthaler-Canseco, M

Garza, A. and Mergenthaler-Canseco, M. Timegpt-1. arXiv:2310.03589,

arXiv

[5] [8]

We utilize this method throughout our paper

Uniform discretization with different number of bins M is Pareto optimal in balancing the average tokenization error δavg and compression ¯c. We utilize this method throughout our paper. /uni00000013/uni00000011/uni00000013/uni00000013/uni00000011/uni00000014/uni00000013/uni00000011/uni00000015/uni00000013/uni00000011/uni00000016/uni00000013/uni00000011/u...

work page 2025

[6] [10]

Unified training of universal time series forecasting transformers.arXiv:2402.02592,

Woo, G., Liu, C., Kumar, A., Xiong, C., Savarese, S., and Sa- hoo, D. Unified training of universal time series forecasting transformers.arXiv:2402.02592,

arXiv

[7] [11]

An adaptive tokenization approach for time series We provide pseudocode for generating a vocabulary of motifs and utilizing the motifs to tokenize a time series

10 Byte Pair Encoding for Efficient Time Series Forecasting A. An adaptive tokenization approach for time series We provide pseudocode for generating a vocabulary of motifs and utilizing the motifs to tokenize a time series. Algorithm 1Motif vocabulary generation according to equation (3). Input:Dataset of discretized time seriesD ′, minimum motif occurre...

work page 2024

[8] [12]

More complex input tokens generally benefit the prediction of longer motifs. /uni00000015/uni00000011/uni00000013/uni00000016/uni00000011/uni00000013/uni00000017/uni00000011/uni00000013 /uni00000026/uni00000052/uni00000050/uni00000053/uni00000055/uni00000048/uni00000056/uni00000056/uni0000004c/uni00000052/uni00000051/uni00000003¯cin /uni00000014/uni000000...

work page 2024