{"id":"cb637243-db0a-4f43-8245-258903ec868d","arxiv_id":"2506.08641","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Frozen vision transformers, applied to image representations of time series, produce classification features that outperform or match time series foundation models on UCR and UEA benchmarks.","lead":"This paper converts time series into images and feeds them into frozen, pretrained vision transformers, then trains only a simple linear classifier on the hidden features. The approach matches or beats time-series-specific foundation models on standard benchmarks, and combining both kinds of features works even better.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported p=0.03 superiority over Mantis is computed for a single configuration selected from many backbones, layers, and hyperparameters; correcting for this selection could make the headline 'state-of-the-art' claim statistically unsupported.","rationale":"The reader identified Proposition 1's mismatch with the actual TiViT pipeline as the weakest assumption. I agree that the theoretical motivation is incomplete, but it is not the most load-bearing part of the paper: the empirical benchmark result can stand even if Proposition 1 collapses. The load-bearing risk is statistical. The paper makes a strong comparative claim based on p=0.03 for a configuration selected from many models, layers, and hyperparameters. The per-dataset appendix, fusion results, and robustness checks are credible, so this does not warrant rejection. It does warrant a conditional acceptance with the corrected inference reported or a pre-specified configuration. This is why I partially agree with the reader: the verdict stays CONDITIONAL, but for the empirical selection issue rather than primarily the theory gap.","tokens_in":27609,"tokens_out":9548,"duration_ms":125365,"concrete_test":"Recompute paired t-tests on UCR test accuracy for the full grid of configurations actually explored: each backbone (CLIP, DINOv2, SigLIP 2) by each hidden layer, both mean-token and CLS aggregation, with P=sqrt(T) and S=P/10, plus the per-dataset P* variant. Count how many configurations beat Mantis at p<0.05. Then recompute the p-value for the best configuration (CLIP layer 14) after Bonferroni or Benjamini-Hochberg correction over the total number of configurations considered, and separately over the three backbone families treating layer choice as nested. If the corrected p exceeds 0.05, the paper should downgrade 'significantly outperforms' to 'numerically higher for one selected configuration'.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central empirical claim is that frozen OpenCLIP ViT hidden features outperform Mantis on UCR with 81.3% vs 80.1%, p=0.03. But 'TiViT' is not a single pre-registered configuration: the paper evaluates CLIP, DINOv2, SigLIP 2, and MAE backbones; 30+ hidden layers per model; two token aggregation schemes (mean vs CLS); several patch overlaps and patch sizes, including a 20-value per-dataset P* search. Layer 14 for TiViT-CLIP is chosen as the best-performing layer on UCR validation, and TiViT-CLIP is described as the best-performing model. The reported paired t-test treats this selected configuration as if it were fixed in advance. Because p=0.03 is already marginal, multiple comparison correction over the effective number of independent layer/backbone choices could push it above 0.05. The theory gap in Proposition 1 is real but less load-bearing: a failure of the proposition would weaken the stated 'theoretical motivation' for 2D patching, whereas a failure of the corrected significance test would directly undermine the headline claim of state-of-the-art superiority over TSFMs.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TiViT, a framework that converts univariate time series into grayscale images by segmenting the series into patches, stacking them into a matrix, resizing to a ViT-compatible square resolution, and feeding the result through a frozen pretrained Vision Transformer (CLIP, DINOv2, or SigLIP 2). A linear classifier is trained on the mean of the hidden token representations from a selected layer. The authors provide a theoretical proposition claiming that 2D patching increases the number of label-relevant tokens and reduces sample complexity relative to 1D patching. Empirically, they report 81.3% mean accuracy on the 128-dataset UCR benchmark versus 80.1% for the Mantis TSFM (paired t-test p=0.03), 72.0% versus 72.4% on UEA, and 83.0% when concatenating TiViT with Mantis features. They also analyze layer-wise intrinsic dimension and representation alignment between ViTs and TSFMs, and report ablations over patch size, overlap, backbone size, and classifier choice.","tokens_in":27965,"tokens_out":10976,"duration_ms":123615,"significance":"If the empirical claims survive proper statistical treatment, this is a significant result: it demonstrates that large-scale vision pretraining can transfer to time series classification without any fine-tuning, and it quantifies a useful complementarity with dedicated time series foundation models. The paper's strengths include the use of public UCR/UEA splits, results averaged over three seeds, per-dataset tables in the appendix, a consistent linear-probing protocol, no fine-tuning of the ViT backbone, and released code. The theoretical contribution is currently a conditional statement that is not directly connected to the actual TiViT pipeline, and the headline statistical claim is computed after model selection. With those issues addressed, the paper would be a solid empirical contribution to the reuse of vision representations for time series.","major_comments":[{"comment":"The headline comparison against Mantis (81.3 vs 80.1, p=0.03) is computed for TiViT-CLIP after the model and its hyperparameters were selected on the same benchmarks. Table 3 reports the best test accuracy among three pretraining paradigms (CLIP 81.3, DINOv2 80.0, SigLIP 2 80.6); the best layer per backbone is chosen by validation, and the pipeline also selects the patch overlap (Table 2b, highest validation accuracy at 0.9) and the token aggregation (Table A3) on validation data. A paired t-test that treats this configuration as fixed ignores this selection. Because the reported p-value is close to the 0.05 threshold, it may not survive any reasonable correction for the number of configurations explored. The authors should either pre-specify a fixed configuration before seeing test labels, report corrected p-values (e.g., Bonferroni or Westfall-Young permutation over the 128 datasets), or add a nested validation scheme that accounts for model selection. Without this, the claim that TiViT significantly outperforms TSFMs is not supported by the reported test.","section":"4.3 (Benchmark), Tables 3-4"},{"comment":"Proposition 1 is not a theorem about the TiViT pipeline used in the experiments. It assumes T=k^2, p=k, non-overlapping sqrt(k) x sqrt(k) patches of a square matrix, and the conditional assumption that any 2D token containing at least sqrt(k) elements of the label-relevant pattern satisfies Eq. (1). The actual pipeline uses overlapping patches of size sqrt(T) with stride P/10, resizes the stacked matrix to the ViT resolution with nearest interpolation, applies the ViT's own patch embedding, and averages hidden-layer token representations; the proposition says nothing about these operations. Moreover, the cited sample-complexity bound from Li et al. concerns training a shallow ViT from scratch, while TiViT uses frozen pretrained features with linear probing. The proposition can remain as a motivating toy model, but the paper should not claim it explains the empirical gains of TiViT. At minimum, the authors should empirically test the label-relevance condition on real UCR data (e.g., compute the fraction of tokens satisfying Eq. (1) under 1D vs 2D patching for oracle pattern assignments) or clearly restrict the theoretical claim to the simplified setting.","section":"3.1, Eq. (1); Section 3.2"},{"comment":"The paper claims that intermediate layers with high intrinsic dimension are the most effective for time series classification, but this is supported only by visual inspection of the accuracy and intrinsic-dimension curves. No correlation coefficient, rank statistic, or hypothesis test is reported, and the intrinsic-dimension estimates depend on the subsampling ratio (mitigated only partially by the appendix stability check). In addition, the alignment analysis in Table 5 uses the mutual k-NN metric, but the relationship between alignment scores and the reported joint-accuracy gains is not quantified beyond the summary table. These analyses are secondary to the main benchmark claim, but they are presented as contributions and should be either substantiated with quantitative tests or softened.","section":"4.3 (Figure 5), Section 4.4"}],"minor_comments":[{"comment":"The statement that TiViT is 'statistically on par' with Mantis on UEA is asserted without reporting a test; please report the p-value or replace 'statistically' with a direct comparison.","section":"4.3, Table 4"},{"comment":"The claims that TiViT 'significantly outperforms' Mantis on the small-training and long-series subsets are made without p-values; add significance tests or soften the wording.","section":"4.5, Table 6"},{"comment":"The random forest results (TiViT 77.4 vs Mantis 77.5) show that the superiority of TiViT is classifier-dependent; this caveat should be mentioned in the main text, not only in the appendix.","section":"Appendix C.6, Table A6"},{"comment":"There are several typos: 'quention.bouniot' in the author email block should be 'quentin.bouniot'; reference [25] spells 'Pattern Recognitio' instead of 'Recognition'; and 'LBFGS' in Section 4 should be 'L-BFGS'.","section":"Author block and references"},{"comment":"The matrix X' is used both for the stacked real matrix and for the rendered grayscale image in R^{M x P x 3}; please use distinct notation for the three-channel image.","section":"3.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and the empirical study is extensive, but the statistical analysis of the headline claim needs to be corrected; the current p=0.03 is post-selection and likely not robust. The authors co-author the Mantis baseline, but using it as a comparison point is not circular. The theoretical motivation could be reframed as intuition without harming the paper's empirical value."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The gist is simple: take a univariate time series, cut it into sqrt(T)-length segments, stack them into an image, resize, push through a frozen CLIP/DINOv2/SigLIP2, average the hidden tokens at some middle layer, and train a logistic regression. That recipe beats Mantis on UCR (81.3 vs 80.1) and matches it on UEA, and concatenating with Mantis gives another ~3 points. I found the empirical core credible and worth knowing.\n\nWhat's genuinely new: the systematic evaluation of frozen vision backbones, not just the final layer but hidden layers, across 128+27 benchmark datasets with three seeds and per-dataset tables in the appendix. The finding that intermediate layers with high intrinsic dimension are the best is a real empirical observation. The complementarity between ViT and TSFM representations is shown both by alignment scores and by consistent concatenation gains. The controlled 1D-vs-2D comparison in Table 1 is a nice check, though it lacks error bars.\n\nThe soft spots are real but mostly on the framing. Proposition 1 assumes a toy setting where the label-relevant pattern covers whole 1D segments and the 'label-relevant' condition is assumed to hold per 2D token; the actual TiViT pipeline uses overlapping patches, resizing, and a frozen ViT with its own patch grid, so the proposition does not cover the method. It reads as an intuition, and the paper should say so. The bigger issue is the p=0.03 significance claim over Mantis. TiViT-CLIP is the best of many backbones/layers/overlap values, and the selected layer is chosen on validation; the paired t-test is computed as if the configuration were fixed in advance. A multiple-comparison correction would likely push that p above 0.05. The claim 'significantly outperforms' is not justified. Interestingly, the core practical result survives this: TiViT is at worst on par with the TSFMs across multiple backbones, and the concatenation gain is substantial and consistent. The paper would be more honest reporting 'competitive or better, with large gains when combined.'\n\nI also note the reader's weakest-assumption analysis matches mine: the theory is not load-bearing, but the empirical study doesn't rest on it.\n\nWho's this for: anyone doing time series classification who wants a cheap, strong baseline using existing vision models. It deserves a serious referee. The right review outcome is a revision that fixes the statistical over-claim, adds error bars to Table 1, and reframes the theory as motivation rather than proof.","headline":"Frozen vision transformers are a genuinely strong feature extractor for time series; the paper's main empirical result holds, but the headline significance test is over-claimed and the theory is not load-bearing.","tokens_in":28442,"tokens_out":2116,"would_cite":true,"duration_ms":23536,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that frozen image-pretrained vision transformers, fed images made by stacking time-series segments, classify time series better than specialized time-series foundation models without any fine-tuning.","keywords":["time series classification","vision transformers","frozen feature extraction","2D patching","representation transfer","foundation models","complementary representations","intrinsic dimension"],"falsifier":"Compute, on several real UCR datasets, the fraction of tokens that are closer to the true class-mean pattern in 1D versus 2D patching, using the paper's own definition of label-relevant tokens. If 2D patching does not increase this fraction whenever the number of label-relevant segments is not a multiple of sqrt(k), the theoretical mechanism would be refuted even if the empirical accuracy numbers stand.","tokens_in":27413,"feed_emoji":"🖼️","tokens_out":5161,"duration_ms":59397,"temperature":0.7,"pith_summary":"This paper claims that a time series can be treated as an image, and that frozen vision transformers pretrained on billions of natural images become state-of-the-art time-series classifiers. The proposed TiViT pipeline splits each univariate series into segments, stacks them into a 2D matrix, resizes it to a ViT's input resolution, and averages hidden-layer token representations before fitting a linear classifier. On the 128-dataset UCR benchmark this reaches 81.3% mean accuracy, surpassing the time-series foundation model Mantis at 80.1% (paired t-test p=0.03); on the 27 multivariate UEA datasets it matches Mantis, and concatenating TiViT features with Mantis features reaches 83.0% on UCR. The paper also argues theoretically that 2D patching increases the number of label-relevant tokens seen by a Transformer, which reduces sample complexity during training. If correct, this shows that large-scale vision pretraining transfers to a non-visual domain without any fine-tuning.","feed_headline":"Frozen vision transformers outclass time-series foundation models","feed_subtitle":"A frozen CLIP vision transformer reaches 81.3% on UCR; concatenating features with Mantis hits 83.0%.","key_machinery":"The load-bearing mechanism is the time-series-to-image transformation followed by token averaging at an intermediate layer. A univariate series is normalized by robust scaling, cut into M overlapping patches of length P = $\\sqrt$(T) with stride P/10, stacked into an M-by-P grayscale image, resized to the ViT's square resolution, and fed through the frozen ViT; the embedding is the mean of all K+1 tokens at a selected layer. The theoretical support is Proposition 1, which uses the label-relevant-token sample-complexity result of the paper's reference [31]: in a simplified T = $k^{2}$, p = k setting, 2D patching turns each label-relevant segment of length k into contributions to $\\sqrt$(k) patches, so the fraction of label-relevant tokens is at least as large as in 1D patching and strictly larger when the number of label-relevant segments is not a multiple of $\\sqrt$(k). The intermediate layers with highest intrinsic dimension are the ones that transfer best.","core_discovery":"On the paper's own terms, the central discovery is that the hidden representations of a frozen image-pretrained ViT, extracted at roughly 40 to 70 percent of the layer depth, encode enough temporal structure to classify time series competitively with, and on UCR better than, foundation models trained specifically on time-series data. The best configuration uses a contrastively pretrained CLIP ViT-H/14 at layer 14, giving 81.3% mean accuracy across 128 UCR datasets and 72.0% across UEA. The same representations are complementary to time-series foundation models: concatenating TiViT with Mantis raises UCR accuracy to 83.0% and UEA to 73.7%, and the mutual-kNN alignment of the two representation spaces is low, indicating that the two models capture different signal.","pith_inferences":["The theory in Proposition 1 is proven for a simplified non-overlapping square-patch setting; carrying the label-relevance argument through nearest-interpolation resizing and the frozen ViT's own patch embedding would turn the paper's motivation into a theorem for the actual pipeline.","The complementarity result suggests a cheap strategy for a new time-series benchmark: concatenate a frozen ViT's mid-layer embedding with a contrastively trained time-series foundation model's embedding rather than fine-tuning either model.","If the intrinsic-dimension pattern holds across domains, layer selection for frozen encoders could be done by measuring intrinsic dimension on unlabeled target data instead of by validation accuracy.","The same image-conversion recipe could be tested on other non-visual signals, such as audio spectrograms or sensor arrays, to see whether the transfer benefit extends beyond time series."],"forward_implications":["Frozen vision models can replace or match task-specific time-series foundation models in linear-probe classification, at least on the standard UCR and UEA benchmarks.","Hidden-layer features matter: using the mean token representation from a mid-depth layer outperforms both the final layer and the CLS token for time-series classification.","TiViT's gains concentrate where data is scarce: on the ten UCR datasets with the fewest training samples it reaches 89.8% versus 86.6% for Mantis, and on the longest series 75.0% versus 70.5%.","Combining vision-derived and time-series-derived features is a cheap accuracy boost: simple concatenation beats every single model on both benchmarks and also improves zero-shot nearest-centroid classification from about 71% to 73.4%.","The finding that intermediate layers with high intrinsic dimension are the most effective gives a principled way to select which frozen layer to use for a new dataset."],"supporting_citations":[{"why":"Supplies the label-relevant-token framework and the sample-complexity scaling in 1/alpha*^2 that Proposition 1 builds on.","marker":"[31]"},{"why":"Defines Mantis, the state-of-the-art time-series baseline that TiViT is compared against and combined with.","marker":"[18]"},{"why":"Provides the 128-dataset UCR archive used for the main univariate benchmark.","marker":"[12]"},{"why":"Provides the 30-dataset UEA multivariate archive from which 27 datasets are used.","marker":"[4]"},{"why":"Supplies the CLIP pretraining approach for the best-performing TiViT backbone.","marker":"[42]"},{"why":"Defines Moment, the second time-series foundation model baseline used in comparisons and fusions.","marker":"[22]"},{"why":"Supplies the large-scale image-text pretraining corpus that gives the frozen vision models their scale.","marker":"[44]"}],"fun_headline_variants":["Frozen image ViTs outperform specialized time-series models","Hidden ViT features beat time-series foundation models","Frozen ViTs beat and complement time-series foundation models","Reuse ViTs for time series: better than dedicated nets","Frozen CLIP ViT representations excel at time series classification"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument's load-bearing premise is the assumption behind Proposition 1: a 2D patch containing at least sqrt(k) elements of the label-relevant pattern automatically counts as label-relevant, because its dot product with the two class means falls on the right side of the decision threshold. The proof does not show that this condition still holds after the real pipeline resizes the stacked segments with nearest interpolation and re-patches them with the frozen ViT's own patch size.","fun_headline_variants_meta":{"raw":{"variants":["Frozen image ViTs outperform specialized time-series models","Hidden ViT features beat time-series foundation models","Frozen ViTs beat and complement time-series foundation models","Reuse ViTs for time series: better than dedicated nets","Frozen CLIP ViT representations excel at time series classification"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000828,"raw_usage":{"total_tokens":3616,"prompt_tokens":938,"completion_tokens":2678,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":554,"completion_tokens_details":{"reasoning_tokens":2598}},"tokens_in":554,"tokens_out":2678,"duration_ms":25539,"temperature":1.0,"reasoning_tokens":2598,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:07:40.575538+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute, on several real UCR datasets, the fraction of tokens that are closer to the true class-mean pattern in 1D versus 2D patching, using the paper's own definition of label-relevant tokens. If 2D patching does not increase this fraction whenever the number of label-relevant segments is not a multiple of sqrt(k), the theoretical mechanism would be refuted even if the empirical accuracy numbers stand.","supporting_citations":[{"cited_title":"A theoretical understanding of shallow vision transformers: Learning, generalization, and sample complexity","cited_arxiv_id":null,"evidence_quote":"Supplies the label-relevant-token framework and the sample-complexity scaling in 1/alpha*^2 that Proposition 1 builds on."},{"cited_title":"The ucr time series archive","cited_arxiv_id":null,"evidence_quote":"Provides the 128-dataset UCR archive used for the main univariate benchmark."},{"cited_title":"Learning transferable visual models from natural language supervision","cited_arxiv_id":null,"evidence_quote":"Supplies the CLIP pretraining approach for the best-performing TiViT backbone."},{"cited_title":"MOMENT: A family of open time-series foundation models","cited_arxiv_id":null,"evidence_quote":"Defines Moment, the second time-series foundation model baseline used in comparisons and fusions."},{"cited_title":"LAION-5B: An open large-scale dataset for training next generation image-text models","cited_arxiv_id":null,"evidence_quote":"Supplies the large-scale image-text pretraining corpus that gives the frozen vision models their scale."}],"review_version":1}