Measuring Pre-training Data Quality without Labels for Time Series Foundation Models

Jianfeng Zhang; Songkang Wen; Vasilii Feofanov

REVIEW 3 major objections 6 minor 24 references

Measuring Pre-training Data Quality without Labels for Time Series Foundation Models

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A label-free measure of embedding scatter predicts how well time-series pre-training data will improve a foundation model.

desk verdict Useful label-free proxy idea, but the paper's own improvement-prediction experiment is too mixed to carry the data-selection claim; needs revision, not rejection. read the letter →

arxiv 2412.06368 v1 pith:XOJNRAAN submitted 2024-12-09 cs.LG cs.AIstat.ML

Songkang Wen , Vasilii Feofanov , Jianfeng Zhang This is my paper · ORCID

classification cs.LGcs.AIstat.ML

keywords contrastiveaccuracypre-trainingdataselectiontimeseriesfoundationmodelslearninglabel-freeevaluationembeddinguniformityUCRarchive

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to answer a practical question: when building a time series foundation model by contrastive pre-training, how can you tell whether a candidate unlabeled dataset will improve the model, without running the full downstream evaluation? The authors propose contrastive accuracy (Acon), a label-free metric computed from the model's own embeddings: it measures how scattered the embeddings of the candidate data are, by checking whether two augmented views of the same example become mutual nearest neighbors within a batch. They report that Acon correlates strongly with the model's averaged accuracy on a collection of 127 UCR classification tasks when the pre-training set is subsampled, and that changes in Acon predict the performance change from adding a new dataset to pre-training. If these correlations hold in broader settings, Acon would give a cheap criterion for selecting and expanding pre-training corpora for time series foundation models without needing labels or repeated fine-tuning.

What carries the argument

The load-bearing object is the contrastive accuracy Acon, computed as the expectation over random augmentations of the fraction of samples whose augmented embedding has its own other augmented embedding as the nearest neighbor in the batch, using cosine similarity. It operationalizes the uniformity property of contrastive representations (following Wang and Isola, 2020): instead of estimating the distribution on the unit hypersphere, it counts self-nearest-neighbor matches, which is cheap to compute from batches. The metric is defined relative to a fixed pre-trained model and a fixed augmentation family (RandomCropResize with crop ratio 0.7 to 0.8), so it is a model-dependent, label-free proxy rather than a pure property of the data alone.

What would settle it

Take a diverse pool of candidate time-series datasets, compute contrastive accuracy for each under the fixed pre-trained model, and then measure the actual downstream accuracy after adding each dataset to pre-training; if the rank correlation between Acon and accuracy is close to zero or negative on this pool, the paper's central claim would be refuted. A sharper test would find two datasets with similarly high Acon where adding one improves downstream accuracy while adding the other degrades it, breaking the claimed monotonic relationship.

Watch

Extended reading notes

Core claim

The central claim is that the quality of a pre-training dataset for a time series foundation model can be evaluated without labels by measuring how well its embeddings satisfy the uniformity property of contrastive learning. Concretely, for a candidate dataset X', the contrastive accuracy Acon is the expected fraction of samples whose two random augmentations produce embeddings that are the nearest neighbor of each other within the batch; a high value means the representation space is well spread, which the paper argues preserves information and supports downstream discrimination. The paper shows that when a fraction r% of a large UCR dataset is used for pre-training, Acon computed on the subsample tracks both training and test accuracy on the remaining 127 datasets (correlations up to rho = 0.99 in the ElectricDevices case), and that the difference in Acon between two pre-training sets correlates with the difference in downstream performance (rho values around 0.74 to 0.75 for the shown AllGestureWiimoteX case, with a range across 12 datasets). The authors frame this as a way to guide dataset collection and selection for time series foundation models.

Load-bearing premise

The metric's predictive power rests on the assumption that a candidate dataset's embedding uniformity, measured with one fixed augmentation family, accurately reflects the model's downstream classification accuracy on other datasets across distribution shifts.

Editorial extensions

If this is right

Dataset curation for time series foundation models can be done by ranking candidate unlabeled datasets on contrastive accuracy and picking those with high values, without supervised evaluation.
The subsampling experiments suggest that Acon can indicate when enough examples have been included from a given dataset, guiding the scale of pre-training corpora.
The improvement-prediction results suggest that Acon differences can forecast whether adding a particular dataset to pre-training will help, enabling greedy data expansion.
Because Acon requires only the model and unlabeled candidate data, it can be used repeatedly during training as a cheap monitor of representation quality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The metric inherits the quality of the augmentation family; the authors flag the choice of augmentations for time series as an open question, so Acon's rankings may shift across augmentation choices and should be tested for stability.
Acon measures uniformity but not alignment or downstream class structure, so it may complement rather than replace other label-free performance estimators such as projection-norm or confidence-based methods from the out-of-distribution literature.
The correlations are reported on UCR; testing on larger and more heterogeneous pre-training pools, including forecasting or multivariate series, would calibrate how the relationship holds as domain diversity grows.
Because the uniformity principle is general, the same metric could be applied to contrastive foundation models in image or speech domains, offering a natural test bed beyond time series.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The new thing here is simple and worth knowing: contrastive accuracy, the mutual-nearest-neighbor rate of two augmented views, as a label-free proxy for downstream accuracy of a time series foundation model. The framing is good — connecting the alignment/uniformity view of contrastive learning to unlabeled performance estimation is a natural move that I don't think anyone has made for time series foundation models yet. The practical motivation is real: if this works, practitioners get a cheap way to screen candidate pre-training data without fine-tuning on labeled downstream tasks.

The paper does something genuinely useful in Section 4.1. Across four UCR datasets, subsampling the pre-training set and measuring CA gives correlations with downstream accuracy between 0.62 and 0.99. Those are strong, and the metric is cheap to compute. Credit where due: the idea is clear, the experiments are reproducible in principle, and the authors honestly flag in the conclusion that augmentation choice for time series contrastive learning is open.

The soft spots are concentrated in Section 4.2, which is the only experiment that actually tests the "quality" interpretation by swapping dataset identity rather than subsample size. In the fix-X0 panels (Figure 5), only 6 of 12 datasets show significant positive correlation; EOGVerticalSignal is negative (rho = -0.07), and five others are non-significant. The main text illustrates with AllGestureWiimoteX, one of the strongest panels, and hides the full distribution in the appendix. The vary-X0 panels (Figure 6) are stronger, but that direction is less practically relevant. Also, the Section 4.1 correlations are confounded with the subsample ratio r, so they don't establish ranking of datasets by quality. The p-value reporting is ambiguous — computed on 10 or 11 averaged points, not on the 50 individual runs — and there is no code, no comparison to simpler baselines like average pairwise distance or a pure uniformity score.

These are real but addressable problems. The central claim — that CA correlates with downstream accuracy — is plausible and partially supported; the abstract's stronger claim that CA "can serve as a criterion to search for time series datasets" is not established by the reported evidence. The paper deserves a serious referee, not a desk reject. For a full venue, it needs the complete 12-panel results in the main text, statistics over runs, code, and baseline comparisons. For a workshop paper, it's acceptable as a preliminary study.

My recommendation: send it to peer review, but expect revision or a transfer to a workshop track. I'd bring it to a reading group interested in time series foundation models or unsupervised performance estimation.

Referee Report

3 major / 6 minor

Summary. The paper asks whether the quality of pre-training data for a time-series foundation model can be evaluated without labels. It proposes contrastive accuracy (Acon), the expected fraction of samples whose two augmented views are mutual nearest neighbours in the embedding space of a contrastively pre-trained encoder, and claims that Acon correlates with downstream accuracy. The paper reports two experiments on UCR datasets: one varying the subsample ratio of a single pre-training dataset (Section 4.1) and one measuring the correlation between Acon improvement and downstream accuracy improvement when adding a candidate dataset to the pre-training set (Section 4.2). On this basis it suggests that Acon can serve as a criterion to search for pre-training datasets.

Significance. If the correlation held robustly across datasets and augmentation families, Acon would be a cheap, label-free tool for selecting and expanding pre-training corpora for time-series foundation models, which is a practically important problem. The paper is clearly written, the metric is simple, and the evaluation uses a large benchmark (UCR) with multiple seeds. The central idea is plausible and the Section 4.1 results are encouraging. However, the evidence for the dataset-selection claim is currently mixed, and the main experiment that varies dataset identity shows several non-significant and one negative correlation; this needs to be addressed before the central claim is accepted. The metric is model-dependent and augmentation-dependent, which is acknowledged in the paper but limits the current scope.

major comments (3)

[Sec. 4.2 / Figs. 5-6] The improvement-prediction experiment, which is the only direct test of the dataset-selection claim, does not consistently support it. In the 'Fix X0, Vary Xnew' panels of Figure 5, only 6 of 12 datasets reach p<0.05; EOGVerticalSignal has rho=-0.07 (p=0.83), and CricketY, MelbournePedestrian, Phoneme, ScreenType, and WordSynonyms are not significant at the 0.05 level. The text illustrates the claim with AllGestureWiimoteX (rho=0.74, p=0.0088), which is one of the strongest panels, without reporting the full distribution. Moreover, the reported p-values appear to be computed on 11 per-dataset averaged points (each the mean of 5 runs), so run-level variance and dataset clustering are ignored; the analysis should be repeated on the individual runs or with a mixed-effects model, and the full set of correlations should be reported.
[Sec. 4.1 / Fig. 1 and Fig. 4] The strong correlations between Acon and downstream accuracy are measured while varying the subsampling ratio r of the same pre-training dataset. In this design Acon, Ptrain, and Ptest all increase with r, so the correlation may be driven by the common dependence on dataset size rather than by data quality. This experiment therefore does not demonstrate that Acon ranks different datasets by quality, which is the claimed use case. A control experiment with fixed-size subsets of different datasets, or a partial correlation controlling for r, is needed to support the interpretation.
[Sec. 3.3, Eq. (3)] The definition of Acon is under-specified. The similarity vector si in Eq. (2) is defined for a batch B used in training, while Eq. (3) applies the argmax over all n examples in X' and states that for large n the data are split into disjoint batches. It is not specified how the indicator and the expectation over augmentations are aggregated over these batches, nor how the mutual-neighbour criterion is computed for examples at batch boundaries. This makes the metric difficult to reproduce exactly. In addition, Acon is defined with respect to the same augmentation family used in pre-training; the authors note in the Conclusion that the choice of augmentation for time-series contrastive learning is an open question, so transferability of the metric across augmentation distributions is an unresolved load-bearing assumption.

minor comments (6)

[Sec. 4.2] The sentence 'For each pair (X0, Xnew0' is missing a closing parenthesis; it should read 'For each pair (X0, Xnew0)'.
[Appendix B] The dataset name 'AllGestureWiimo-teX' is misspelled; it should be 'AllGestureWiimoteX', and the labels in Figures 5 and 6 should be checked for consistency (e.g., 'WormsTwoClass' vs 'WormsTwoClass').
[References] The Lag-Llama reference appears twice with identical content (Rasul et al., 2023a and 2023b); if these are the same paper they should be merged, and if they are different they should be distinguished by title or venue.
[Sec. 3.3 / Sec. 4.1] The notation for Acon is inconsistent: Eq. (3) writes A(X')con (X0), while Section 4.1 writes A(X0)con (X(r%)0). The superscript/subscript convention for the evaluation set and the pre-training set should be defined once and used consistently.
[Figures 5-6] The x-axis labels mix text and math ('Contrastive acc improvement, Acon (X0, X(new)0)'); use a defined symbol such as ΔAcon throughout for readability.
[General] The paper does not mention whether code or exact UCR data splits will be released; for a metric paper, releasing the pre-training and evaluation code would greatly improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the contrastive-accuracy proxy is measured against external downstream labels, not derived from them.

full rationale

The paper's central claim is an empirical correlation claim: contrastive accuracy Acon, defined in Section 3.3 as a nearest-neighbor retrieval rate on augmented embeddings, is compared with downstream classification accuracy Ptest/Ptrain measured on the UCR archive. No equation defines Acon in terms of P, and no parameter is fitted so that Acon reproduces P; the correlations in Figures 1-6 are reported outcomes, not identities. Section 4.1's correlation is partly confounded by the shared subsample ratio r, since both CA and accuracy grow with r, but this is a statistical weakness rather than a construction that forces the relationship. Section 4.2 is the experiment that varies dataset identity and it is validated against held-out downstream labels; the mixed and non-significant correlations in Figures 5-6 are evidence against the strength of the claim, not evidence of circularity. The only author-overlapping citation (Xie et al. 2024, MANO) appears in a related-work enumeration and is not load-bearing for the proposed measure. The conclusion's statement that the choice of time-series augmentations is an open question is a stated limitation. Thus, under the hard rules, no circular step can be quoted and exhibited, and the honest finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central claim depends on one new metric (contrastive accuracy), whose definition requires a pre-trained model and a fixed augmentation family. The metric has no free parameters fitted to downstream labels, but the augmentation scale and InfoNCE temperature are hand-chosen and affect the measured correlations. The paper relies on the UCR archive as both pre-training source and downstream benchmark, and on the standard assumption that alignment and uniformity of contrastive embeddings transfer to linear probe accuracy.

free parameters (3)

Augmentation distribution (RandomCropResize crop scale) = 0.7 to 0.8
Chosen by hand (Table 1); the contrastive accuracy metric is defined as an expectation over augmentations, so its values and correlations depend on this choice; the authors note in the Conclusion that augmentation selection is an open question.
InfoNCE temperature T = 0.1
Standard contrastive learning hyperparameter chosen by hand (Table 1); it affects the geometry of the embedding space and hence the contrastive accuracy values.
CA evaluation batch size = not specified
Section 3.3 says large evaluation sets are split into disjoint batches and similarity is computed within batches, but the batch size is not reported; increasing the number of in-batch distractors lowers the measured fraction, so this parameter affects all reported values.

assumptions (4)

domain assumption Uniformity and alignment of contrastive embeddings are indicative of downstream task performance (Wang and Isola, 2020).
Motivates the metric in Section 3.3; the paper assumes that a representation with good alignment and uniformity transfers to linear classifier accuracy.
domain assumption The UCR archive is a representative benchmark for time series classification generalization.
Used as the sole pre-training and downstream evaluation collection in Section 4; no justification that findings transfer to other time series domains.
domain assumption Reshaping all sequences to length 512 preserves discriminative information.
Section 3.2 and Table 1 fix sequence length to 512; many UCR datasets have other lengths, and the effect of resizing is not analyzed.
domain assumption Linear probing (frozen backbone plus fine-tuned linear head) measures representation quality.
Downstream accuracy in Section 3.1 is defined through fine-tuning a linear head; the metric CA is computed on the frozen backbone, and the correlation assumes these track each other.

invented entities (1)

Contrastive accuracy (Acon)
purpose: A label-free score measuring whether two augmented views of a time series are mutual nearest neighbors in the embedding space, used to rank pre-training data quality.
The metric is new to this paper; its only supporting evidence is the paper's own correlation experiments on the UCR archive, so there is no external falsifiable handle outside this work beyond re-running the same experiments.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Measuring Pre-training Data Quality without Labels for Time Series Foundation Models." pith.science (2026). https://pith.science/paper/XOJNRAAN

@misc{pith2026241206368,
  author       = {Pith},
  title        = {Pith review of: Measuring Pre-training Data Quality without Labels for Time Series Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XOJNRAAN}},
  note         = {Machine review of arXiv:2412.06368}
}

read the original abstract

Recently, there has been a growing interest in time series foundation models that generalize across different downstream tasks. A key to strong foundation models is a diverse pre-training dataset, which is particularly challenging to collect for time series classification. In this work, we explore the performance of a contrastive-learning-based foundation model as a function of the data used for pre-training. We introduce contrastive accuracy, a new measure to evaluate the quality of the representation space learned by the foundation model. Our experiments reveal the positive correlation between the proposed measure and the accuracy of the model on a collection of downstream tasks. This suggests that the contrastive accuracy can serve as a criterion to search for time series datasets that can enhance the pre-training and improve thereby the foundation model's generalization.

Figures

Figures reproduced from arXiv: 2412.06368 by the authors.

**Figure 1.** The correlation (ρ) between the contrastive accuracy and the foundation model’s performance when varying the ratio of examples used for pre-training. 4.2 Improvement Prediction In this experiment, in addition to the pre-training dataset X0, we consider having another one Xnew 0 and ask whether it is possible to predict the performance improvement from including X′ 0 to the pre-training data, i.e., ∆P(X0, Xnew 0 ) :… view at source ↗

**Figure 2.** The correlation (ρ) between the improvement in contrastive accuracy and the performance improvement on 116 UCR datasets when expanding the pre-training dataset X0 by including X (new) 0 . 5 Conclusion and Future Work In this paper, we studied the task of evaluating the effect of pre-training data on the foundation model’s performance. We proposed the contrastive accuracy and experimentally showed its promise as a cr… view at source ↗

**Figure 3.** The framework of the model. 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.62 0.64 0.66 0.68 0.70 0.72 0.74 Performance Train, train : =0.98, pval=5.4e-12 Test, test : =0.99, pval=1.1e-12 ElectricDevices 0.65 0.70 0.75 0.80 0.85 0.90 0.58 0.60 0.62 0.64 0.66 0.68 0.70 0.72 Train, train : =0.62, pval=0.01 Test, test : =0.68, pval=0.0037 Crop 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00 Contrastive Accuracy (CA, Acon) 0.62 0.64 0.66… view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: The correlation (ρ) between the contrastive accuracy and the foundation model’s performance when varying the ratio of examples used for pre-training. 7 [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: The correlation (ρ) between the improvement in contrastive accuracy and the performance improvement on 116 UCR datasets when expanding the pre-training dataset X0 by including X (new) 0 . To measure correlation, we fix X0 and vary X (new) 0 across 11 datasets. 9 [PITH…

**Figure 6.** Figure 6: The correlation (ρ) between the improvement in contrastive accuracy and the performance improvement on 116 UCR datasets when expanding the pre-training dataset X0 by including X (new) 0 . To measure correlation, we fix X (new) 0 and vary X0 across 11 datasets. 10 [PIT…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 14 canonical work pages

[1]

L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. (2023). Gpt-4 technical report. arXiv preprint arXiv:2303.08774

arXiv 2023
[2]

A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M

Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021). On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258

arXiv 2021
[3]

A., Bagnall, A., Kamgar, K., Yeh, C.-C

Dau, H. A., Bagnall, A., Kamgar, K., Yeh, C.-C. M., Zhu, Y ., Gharghabi, S., Ratanamahatana, C. A., and Keogh, E. (2019). The ucr time series archive. IEEE/CAA Journal of Automatica Sinica , 6(6):1293–1305

work page 2019
[4]

Donmez, P., Lebanon, G., and Balasubramanian, K. (2010). Unsupervised supervised learning i: Estimating classification and regression errors without labels. Journal of Machine Learning Research, 11(4)

work page 2010
[5]

Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. (2021). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929

arXiv 2021
[6]

K., Li, X., and Guan, C

Eldele, E., Ragab, M., Chen, Z., Wu, M., Kwoh, C. K., Li, X., and Guan, C. (2021). Time-series representation learning via temporal and contextual contrasting. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pages 2352–2359

work page 2021
[7]

He, K., Fan, H., Wu, Y ., Xie, S., and Girshick, R. (2020). Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729–9738

2020
[8]

and Gimpel, K

Hendrycks, D. and Gimpel, K. (2016). A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136

arXiv 2016

Show all 24 references

[9]

Ilbert, R., Odonnat, A., Feofanov, V ., Virmaux, A., Paolo, G., Palpanas, T., and Redko, I. (2024). SAMformer: Unlocking the potential of transformers in time series forecasting with sharpness- aware minimization and channel-wise attention. In Salakhutdinov, R., Kolter, Z., He...

2024
[10]

Y ., Shi, X., Chen, P.-Y ., Liang, Y ., Li, Y .-F., Pan, S., et al

Jin, M., Wang, S., Ma, L., Chu, Z., Zhang, J. Y ., Shi, X., Chen, P.-Y ., Liang, Y ., Li, Y .-F., Pan, S., et al. (2023). Time-llm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.01728

2023 arXiv
[11]

Lin, C., Wen, X., Cao, W., Huang, C., Bian, J., Lin, S., and Wu, Z. (2024). Nutime: Numerically multi-scaled embedding for large- scale time-series pretraining. Transactions on Machine Learning Research

2024
[12]

and Hutter, F

Loshchilov, I. and Hutter, F. (2016). Sgdr: Stochastic gradient descent with warm restarts. In International Conference on Learning Representations

2016
[13]

Nguyen, N., Sinthong, P., and Kalagnanam, J

Nie, Y ., H. Nguyen, N., Sinthong, P., and Kalagnanam, J. (2023). A time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representations

2023
[14]

Oord, A. v. d., Li, Y ., and Vinyals, O. (2018). Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748

2018 arXiv
[16]

V ., Schneider, A., et al

Ghonia, H., Hassen, N. V ., Schneider, A., et al. (2023b). Lag-llama: Towards foundation models for time series forecasting. arXiv preprint arXiv:2310.08278. 5

2023 arXiv
[17]

Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023). Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

2023 arXiv
[18]

and Isola, P

Wang, T. and Isola, P. (2020). Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International conference on machine learning , pages 9929–9939. PMLR

2020
[19]

Woo, G., Liu, C., Kumar, A., Xiong, C., Savarese, S., and Sahoo, D. (2024). Unified training of universal time series forecasting transformers. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F., editors, Proceedings of the 4...

2024
[20]

Xie, R., Odonnat, A., Feofanov, V ., Deng, W., Zhang, J., and An, B. (2024). Mano: Exploit- ing matrix norm for unsupervised accuracy estimation under distribution shifts. arXiv preprint arXiv:2405.18979

2024 arXiv
[21]

Yu, Y ., Yang, Z., Wei, A., Ma, Y ., and Steinhardt, J. (2022). Predicting out-of-distribution error with the projection norm. In International Conference on Machine Learning, pages 25721–25746. PMLR

2022
[22]

Yue, Z., Wang, Y ., Duan, J., Yang, T., Huang, C., Tong, Y ., and Xu, B. (2022). Ts2vec: Towards uni- versal representation of time series. Proceedings of the AAAI Conference on Artificial Intelligence, 36(8):8980–8987

2022
[23]

Zhang, D., Yuan, Z., Yang, Y ., Chen, J., Wang, J., and Li, Y . (2023). Brant: Foundation model for intracranial neural signal. In Thirty-seventh Conference on Neural Information Processing Systems

2023
[24]

Zhang, X., Zhao, Z., Tsiligkaridis, T., and Zitnik, M. (2022). Self-supervised contrastive pre-training for time series via time-frequency consistency. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A., editors, Advances in Neural Information Processin...

2022
[25]

Zhou, T., Niu, P., Wang, X., Sun, L., and Jin, R. (2023). One fits all: Power general time series analysis by pretrained lm. arXiv preprint arXiv:2302.11939. A Architecture and Implementation Details In this paper, similarly to Nie et al. (2023) and Lin et al. (2024), we emplo...

2023 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. (2023). Gpt-4 technical report. arXiv preprint arXiv:2303.08774

arXiv 2023

[2] [2]

A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M

Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021). On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258

arXiv 2021

[3] [3]

A., Bagnall, A., Kamgar, K., Yeh, C.-C

Dau, H. A., Bagnall, A., Kamgar, K., Yeh, C.-C. M., Zhu, Y ., Gharghabi, S., Ratanamahatana, C. A., and Keogh, E. (2019). The ucr time series archive. IEEE/CAA Journal of Automatica Sinica , 6(6):1293–1305

work page 2019

[4] [4]

Donmez, P., Lebanon, G., and Balasubramanian, K. (2010). Unsupervised supervised learning i: Estimating classification and regression errors without labels. Journal of Machine Learning Research, 11(4)

work page 2010

[5] [5]

Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. (2021). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929

arXiv 2021

[6] [6]

K., Li, X., and Guan, C

Eldele, E., Ragab, M., Chen, Z., Wu, M., Kwoh, C. K., Li, X., and Guan, C. (2021). Time-series representation learning via temporal and contextual contrasting. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pages 2352–2359

work page 2021

[7] [7]

He, K., Fan, H., Wu, Y ., Xie, S., and Girshick, R. (2020). Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729–9738

2020

[8] [8]

and Gimpel, K

Hendrycks, D. and Gimpel, K. (2016). A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136

arXiv 2016