Pith. sign in

REVIEW 2 major objections 5 minor 73 references

On 124 time-series classification problems, six feature sets perform equivalently in 85% of pairwise comparisons, with the largest set, tsfresh, winning most often.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-05 00:33 UTC pith:63G45V6Z

load-bearing objection Useful, mostly careful benchmark, but the headline tie/win rates contradict the paper's own Fig. 3B and Eq. 3 misstates the corrected t-test; revise before trusting the numbers. the 2 major comments →

arxiv 2608.01586 v1 pith:63G45V6Z submitted 2026-08-03 stat.ME cs.LGstat.ML

Statistical comparisons of time-series feature sets on classification tasks

Trent Henderson , Ben D. Fulcher This is my paper
classification stat.ME cs.LGstat.ML
keywords time-series classificationtime-series featuresfeature setsbenchmarkingnormalized performance scoretsfreshcatch22ridge logistic regression
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks whether the choice among six open-source time-series feature libraries changes classification accuracy, and under what conditions. On 124 univariate problems, it finds the sets are broadly interchangeable: 85.3% of pairwise comparisons show no statistically significant difference, despite feature-set sizes ranging from 22 to 783 and computation times differing by orders of magnitude. The largest set, tsfresh, wins the most head-to-head comparisons overall (29.03%). Per-problem deviations are traced to specific feature composition—Kats lacks a linear-trend slope on SyntheticControl, while tsfresh keeps raw Fourier coefficients on Beef—and simple quantile or Fourier baselines can match or beat the specialist sets on some problems. The takeaway is that average performance masks where feature make-up actually matters.

Core claim

The paper benchmarks six open-source time-series feature sets and three simple baselines on 124 univariate classification problems. Despite the sets ranging from 22 to 783 features and differing by orders of magnitude in compute time, their classification accuracies were statistically indistinguishable on 85.3% of pairwise problem comparisons. The largest set, tsfresh, won the most comparisons overall (29.03%). Per-problem deviations were traceable to feature composition: Kats underperformed on SyntheticControl because it lacks a linear-trend slope feature, and tsfresh excelled on Beef because it retains raw fast-Fourier-transform coefficients. Simple baselines of 200 Fourier coefficients an

What carries the argument

The central object is the normalized performance score (NPS): for each problem, a feature set's mean accuracy is converted to a z-score relative to the mean and spread of accuracies across all feature sets on that problem. This makes problems of different difficulty comparable and exposes magnitude differences that average-rank methods hide. Pairwise significance is decided by a corrected resampled t-statistic that accounts for the non-independence of 30 shared train-test resamples. Together, these tools let the paper separate 'broadly similar' from 'different' both at the level of individual problems and in aggregate.

Load-bearing premise

The benchmark z-scores every time series before extracting features, so mean and variance cannot contribute to classification; if real problems separate classes by those moments, the conclusion that feature sets are broadly equivalent may not transfer to un-normalized practice.

What would settle it

Run the same 124-problem benchmark without z-scoring the raw series first, including problems where classes differ mainly by mean or variance (one such case is noted in the paper, where squared values alone separate classes). If pairwise win rates shift materially—for instance tsfresh wins more or catch22 loses more—then the reported equivalence is an artifact of removing distributional location and spread.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • On ordinary problems, the choice among the six feature sets will rarely change the outcome; problem difficulty, not feature-set size or composition, drives most accuracy variation.
  • tsfresh is the strongest overall default from this benchmark, but its 783 features do not guarantee a win on a given problem and come at a much higher compute cost.
  • Quantile and Fourier baselines should be standard comparators: they are fast, interpretable, and sometimes sufficient on their own, so a dedicated feature set should be required to beat them.
  • A feature set's unique features can create large per-problem swings, so aggregate rankings should be paired with per-problem normalized scores to reveal when composition matters.
  • Because 85.3% of pairwise comparisons tie, much of tsfresh's feature space is redundant for classification; a smaller set built from the union of unique features could approach its performance.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the z-score preprocessing is load-bearing, practitioners who do not standardize series before feature extraction may see different rankings; the equivalence result is safest for analyses of shape and dynamics, not raw scale.
  • The competitive baselines imply a cheap decision rule: compute quantiles and Fourier coefficients first, and only adopt a dedicated feature set if those baselines fall short on validation data.
  • The per-problem wins of small sets on specific structures suggest that a merged pool of a few dozen unique features across all six sets could match or beat tsfresh at a fraction of the cost; the paper proposes building such a set but does not test it.
  • Because the benchmark uses a single classifier (ridge logistic regression), the relative standing of feature sets could differ under nonlinear classifiers; the paper's claims are specific to this protocol.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. Using the UEA/UCR repository (124 univariate problems), the paper compares six open-source time-series feature sets (catch22, feasts, tsfeatures, Kats, TSFEL, tsfresh) and three simple baseline feature sets (FFT coefficients, quantiles, union). Series are z-scored before feature extraction; features are computed via the theft package; a ridge multinomial logistic regression is trained on 30 seeded train/test resamples with per-set regularization. The paper introduces a normalized performance score (NPS) and uses pairwise corrected resampled t-tests to classify each comparison as win/tie/loss. The headline findings are that feature sets perform similarly (85.3% ties), tsfresh is the strongest (29.03% wins), and that specific composition differences explain large per-problem deviations (e.g., Kats on SyntheticControl, tsfresh on Beef).

Significance. The paper addresses a timely and practically important question and has several strengths: a large external benchmark (UEA/UCR), a sensible resampling protocol (30 seeded splits), the use of a corrected resampled t-test, per-set regularization, public code, and explicit mechanism checks via feature ablation. If the quantitative claims are corrected, the work would be a valuable reference for practitioners: it suggests that feature-set choice matters less than often assumed, identifies simple baselines that are competitive, and demonstrates a problem-level magnitude-sensitive benchmarking approach. The specific feature-composition explanations are falsifiable and useful. However, as submitted the headline numerical results are not reproducible from the paper's own tables.

major comments (2)
  1. [Abstract; Sec. 3.3; Fig. 3B] The abstract and Discussion report 85.3% ties and a 29.03% win rate for tsfresh, but Fig. 3B does not support these numbers. Summing tsfresh's wins against the other five sets gives 20+18+42+35+48 = 163 of 620 comparisons (26.3%), not 29.03%. Summing the non-tied cells across the 15 pairwise blocks gives 433 of 1860 comparisons, i.e., 76.7% ties, not 85.3%. The discrepancy is far too large for rounding. The surrounding text also misstates specific cells: it says tsfeatures has the most wins against tsfresh (5), while the matrix assigns 5 wins to TSFEL and only 1 to tsfeatures; and it reports 6/9 wins/losses for tsfeatures–TSFEL, whereas the matrix shows 6/12. Please reconcile all numbers and update all derived rates.
  2. [Sec. 2.4.2, Eq. (3)] The displayed corrected resampled t-statistic is mis-specified. For the Nadeau–Bengio test, the denominator should contain the variance of the paired differences d_j, not 'the variance of all resampled classification accuracy values.' As printed, the numerator is a mean of differences while the denominator uses a variance of raw accuracies, so the statistic is not the one implemented by correctR and cannot reproduce the reported p-values. Please correct Eq. (3), define σ^2_d = Var(d_1,...,d_n), and clarify the roles of n1 and n2. If the code used correctR throughout, state this explicitly so readers can verify.
minor comments (5)
  1. [Sec. 2.4.2, Eq. (3)] The phrase 'n1 is the train test size' should read 'n1 is the training set size'.
  2. [Sec. 2.3] Please clarify whether the >10% missing-value feature removal is applied to training data only. If it is computed on the pooled training+test data, the feature filter leaks test-set information into the classifier.
  3. [Fig. 2 caption] The expression '−1≤NPS ij ≥1' is malformed; it should read '-1 ≤ NPS_ij ≤ 1'.
  4. [Sec. 2.1.1 and Discussion] The decision to z-score all series removes mean and variance as potential class-discriminative information. This is a defensible standardization, but the conclusions should be framed as conditional on this preprocessing. Please state this limitation explicitly and, if possible, include a sensitivity analysis on raw series.
  5. [References] Reference [26] is a GitHub repository with no version/date; please provide a stable archival reference or version number.

Circularity Check

0 steps flagged

No circular derivation; benchmark is externally anchored and self-citations are context/tool citations only.

full rationale

The paper's central claims are derived from an external benchmark (124 UEA/UCR problems) with pre-existing feature-set implementations, and the outcome (win/tie/loss rates) is produced by a fixed classification pipeline and corrected resampled t-tests. The authors' own tools and prior work appear only as implementation choices or supporting context: theft [17] and correctR [58] are software used to compute features and p-values, and prior results [27, 36] motivate z-scoring and redundancy expectations rather than define the accuracy comparisons. The normalized performance score (Eq. 1) is a descriptive z-score across the six feature sets for each problem, not a fitted parameter later renamed as a prediction; no equation in the paper reduces the reported rates to a definitional identity. The Beef ablation (removing tsfresh's 400 FFT features) is an empirical sensitivity check, and the Kats/SyntheticControl discussion is a post-hoc feature-composition explanation. Although there are several self-citations, none is load-bearing in the mathematical chain: the 85.3% tie rate and 29.03% win rate are asserted to come from pairwise tests of the accuracy distributions, not from the cited redundancy analysis. The reviewer-flagged internal inconsistency between the abstract's aggregate rates and Fig. 3B, and the apparent mis-specification of Eq. 3, are correctness/reproducibility concerns rather than circularity; they do not indicate that a conclusion was assumed in its own derivation.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The paper's central claims rest on the representativeness of the UEA/UCR subset, the z-score preprocessing that removes first two moments, the use of ridge logistic regression as a neutral probe, the validity of the corrected resampled t-test under the 30-split design, and the correctness of the `theft` implementations. The only hand-chosen quantities that materially affect the headline numbers are the significance threshold alpha=0.05, the ridge penalty C, and the sizes of the quantile/FFT baselines.

free parameters (4)
  • Ridge regularization penalty C = optimized per problem/feature set via 10-fold CV; median 0.36, range catch22 2.78 to tsfresh 0.05; fixed to 1.0 on small
    The classification results depend on this hyperparameter; the paper optimizes it separately per feature set, which affects the relative performance ranking.
  • Quantile grid size = 101 quantiles (0.00 to 1.00 in 0.01 steps)
    The baseline feature set's strength depends on how finely the distribution is quantized; a coarser or finer grid would change baseline performance.
  • FFT coefficient count = 200 coefficients (magnitude squared and angle at frequencies 0-99)
    The baseline spectral representation is truncated at 100 frequencies; this choice affects how well the baseline captures spectral class differences.
  • Statistical significance threshold alpha = 0.05
    The win/tie/loss classification and the headline '85.3% ties' depend directly on this threshold; no sensitivity analysis is reported.
axioms (5)
  • domain assumption The 124 retained UEA/UCR problems are a representative sample of univariate time-series classification tasks.
    The paper generalizes from this repository; 18 problems were excluded due to minimum-length failures (Sec. 2.1), which may skew the sample toward longer series.
  • domain assumption z-scoring each time series before feature extraction removes only trivial non-temporal signal, not meaningful class-discriminative information.
    Sec. 2.1.1 justifies this by citing the authors' prior work [36], but it is a strong preprocessing choice; if mean/variance are real class signal in many problems, the benchmark ranking could change.
  • domain assumption Ridge logistic regression is an adequate and fair probe of the information content of each feature set.
    The paper uses a single classifier for all feature sets (Sec. 2.3); if this classifier favors larger or more redundant feature sets, the relative comparison may not reflect feature-set quality independently of the classifier.
  • standard math The Nadeau-Bengio corrected resampled t-test, applied with 30 resamples and per-problem train/test sizes, is valid for the resampling scheme used.
    Eq. (3) applies the Nadeau-Bengio correction; the 30 resamples are random train/test splits shared across feature sets, and the correction's variance estimator is assumed appropriate.
  • domain assumption Feature sets are correctly implemented by the `theft` package at the specified versions.
    The entire benchmark depends on the correctness of the six feature-extraction implementations (Sec. 2.2); a bug in `theft` would propagate to all results.

pith-pipeline@v1.3.0-daily-deepseek · 17371 in / 28897 out tokens · 283882 ms · 2026-08-05T00:33:04.920761+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Statistical comparisons of time-series feature sets on classification tasks." pith.science (2026). https://pith.science/paper/63G45V6Z

@misc{pith2026260801586,
  author       = {Pith},
  title        = {Pith review of: Statistical comparisons of time-series feature sets on classification tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/63G45V6Z}},
  note         = {Machine review of arXiv:2608.01586}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

In recent years, numerous open-source software libraries have been developed for computing sets of features from univariate time series. The type and number of features vary across these feature sets, which have been constructed with varying disciplinary perspectives on quantifying structure in time-series data. To date, the relative strengths and weaknesses of these feature sets on time-series classification problems remains largely unexplored. Here we aimed to understand the relative performance of six open-source feature sets and three baseline feature sets (based on distributional and/or basic spectral structure) across 124 univariate time-series classification problems using a normalization-based approach to problem-level benchmarking that better indexes the relative strengths and weaknesses of different algorithms compared to prior rank-based approaches. Despite their dramatic differences in size, composition, and computation time, we found that feature sets performed relatively similarly overall (85.3% of pairwise comparisons resulted in ties), with the largest feature set, tsfresh, exhibiting the strongest overall performance (29.03% wins across all pairwise comparisons against other feature sets). We also highlighted specific problems on which the specific composition of a given feature set gave it a substantial performance advantage or disadvantage, and problems where simple baselines comprised of Fourier coefficients and quantiles were sufficient to achieve strong performance. Our results demonstrate the need to consider problem-level performance when benchmarking time-series feature sets, and highlight the importance of feature make-up in driving relative classification performance.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

73 extracted references · 57 canonical work pages · 1 internal anchor

  1. [1]

    Barbara, N.H., Bedding, T.R., Fulcher, B.D., Murphy, S.J., Van Reeth, T.: Clas- sifying Kepler light curves for 12,000 A and F stars using supervised feature-based machine learning (2022) https://doi.org/10.48550/arXiv.2205.03020

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2205.03020 2022

  2. [2]

    Sensors22(20), 8016 (2022)

    Wang, W.K., Chen, I., Hershkovich, L., Yang, J., Shetty, A., Singh, G., Jiang, Y., Kotla, A., Shang, J.Z., Yerrabelli, R.,et al.: A systematic review of time series classification techniques used in biomedical applications. Sensors22(20), 8016 (2022)

    work page 2022

  3. [3]

    Engineering Structures228, 111564 (2021) 18

    Arul, M., Kareem, A.: Applications of shapelet transform to time series classifi- cation of earthquake, wind and wave data. Engineering Structures228, 111564 (2021) 18

    work page 2021

  4. [4]

    Cleveland, R.B., Cleveland, W.S., McRae, J.E., Terpenning, I.,et al.: Stl: A seasonal-trend decomposition. J. off. Stat6(1), 3–73 (1990)

    work page 1990

  5. [5]

    Journal of econometrics31(3), 307–327 (1986)

    Bollerslev, T.: Generalized autoregressive conditional heteroskedasticity. Journal of econometrics31(3), 307–327 (1986)

    1986

  6. [6]

    Prentice-Hall, Inc., Englewood Cliffs, NJ (1988)

    Brigham, E.O.: The Fast Fourier Transform and Its Applications. Prentice-Hall, Inc., Englewood Cliffs, NJ (1988)

    work page 1988

  7. [7]

    In: Feature Engineering for Machine Learning and Data Analytics

    Fulcher, B.D.: Feature-Based Time-Series Analysis. In: Feature Engineering for Machine Learning and Data Analytics. CRC Press, Boca Raton, FL (2018)

    work page 2018

  8. [8]

    IEEE Transactions on Knowledge and Data Engineering26(12), 3026–3037 (2014) https://doi.org/10.1109/TKDE.2014.2316504 arXiv:1401.3531

    Fulcher, B.D., Jones, N.S.: Highly comparative feature-based time-series clas- sification. IEEE Transactions on Knowledge and Data Engineering26(12), 3026–3037 (2014) https://doi.org/10.1109/TKDE.2014.2316504 arXiv:1401.3531

    work page arXiv 2014

  9. [9]

    Data Mining and Knowledge Discovery35(3), 1032–1060 (2021) https: //doi.org/10.1007/s10618-021-00745-9

    Tan, C.W., Bergmeir, C., Petitjean, F., Webb, G.I.: Time series extrinsic regres- sion. Data Mining and Knowledge Discovery35(3), 1032–1060 (2021) https: //doi.org/10.1007/s10618-021-00745-9

    doi:10.1007/s10618-021-00745-9 2021

  10. [10]

    Data Mining and Knowledge Discovery38(4), 2141–2185 (2024) https://doi.org/10

    Guijo-Rubio, D., Middlehurst, M., Arcencio, G., Silva, D.F., Bagnall, A.: Unsu- pervised feature based algorithms for time series extrinsic regression. Data Mining and Knowledge Discovery38(4), 2141–2185 (2024) https://doi.org/10. 1007/s10618-024-01027-w

    work page 2024

  11. [11]

    Time Series Analysis-Recent Advances, New Perspectives and Applications, 298 (2024)

    Faouzi, J.: Time series classification: A review of algorithms and implementations. Time Series Analysis-Recent Advances, New Perspectives and Applications, 298 (2024)

    work page 2024

  12. [12]

    Journal of The Royal Society Interface10(83), 20130048 (2013) https://doi.org/10.1098/rsif.2013.0048

    Fulcher, B.D., Little, M.A., Jones, N.S.: Highly comparative time-series analysis: The empirical structure of time series and their methods. Journal of The Royal Society Interface10(83), 20130048 (2013) https://doi.org/10.1098/rsif.2013.0048

    arXiv 2013

  13. [13]

    stat1050, 14 (2016)

    Alain, G., Bengio, Y.: Understanding intermediate layers using linear classifier probes. stat1050, 14 (2016)

    work page 2016

  14. [14]

    Neural computing and applications32(24), 18069–18083 (2020)

    Vellido, A.: The importance of interpretability and visualization in machine learning for applications in medicine and health care. Neural computing and applications32(24), 18069–18083 (2020)

    work page 2020

  15. [15]

    In: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pp

    Bell, A., Solano-Kamaiko, I., Nov, O., Stoyanovich, J.: It’s just not that simple: an empirical study of the accuracy-explainability trade-off in machine learning for public policy. In: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pp. 248–266 (2022)

    work page 2022

  16. [16]

    Acta neurochirurgica166(1), 14 (2024)

    Ciobanu-Caraus, O., Aicher, A., Kernbach, J.M., Regli, L., Serra, C., Staartjes, V.E.: A critical moment in machine learning in medicine: on reproducible and 19 interpretable learning. Acta neurochirurgica166(1), 14 (2024)

    work page 2024

  17. [17]

    The R Journal17, 43–68 (2025) https://doi.org/10.32614/ RJ-2025-023

    Henderson, T., Fulcher, B.D.: Feature-based time-series analysis in r using the theft ecosystem. The R Journal17, 43–68 (2025) https://doi.org/10.32614/ RJ-2025-023 . https://doi.org/10.32614/RJ-2025-023

    work page doi:10.32614/rj-2025-023 2025

  18. [18]

    Cell Systems5(5), 527–5313 (2017) https://doi.org/10.1016/j.cels.2017.10.001

    Fulcher, B.D., Jones, N.S.: Hctsa: A Computational Framework for Automated Time-Series Phenotyping Using Massive Feature Extraction. Cell Systems5(5), 527–5313 (2017) https://doi.org/10.1016/j.cels.2017.10.001

    work page doi:10.1016/j.cels.2017.10.001 2017

  19. [19]

    Data Mining and Knowledge Discovery33(6), 1821–1852 (2019) https://doi.org/10.1007/s10618-019-00647-x

    Lubba, C.H., Sethi, S.S., Knaute, P., Schultz, S.R., Fulcher, B.D., Jones, N.S.: Catch22: CAnonical Time-series CHaracteristics. Data Mining and Knowledge Discovery33(6), 1821–1852 (2019) https://doi.org/10.1007/s10618-019-00647-x

    doi:10.1007/s10618-019-00647-x 2019

  20. [20]

    Hyndman, R., Kang, Y., Montero-Manso, P., Talagala, T., Wang, E., Yang, Y., O’Hara-Wild, M.: Tsfeatures: Time Series Feature Extraction. (2020). R package version 1.0.2. https://CRAN.R-project.org/package=tsfeatures

    work page 2020

  21. [21]

    Scientific Data7(1), 213 (2020) https://doi.org/10

    Fulcher, B.D., Lubba, C.H., Sethi, S.S., Jones, N.S.: A self-organizing, living library of time-series data. Scientific Data7(1), 213 (2020) https://doi.org/10. 1038/s41597-020-0553-0

    work page 2020

  22. [22]

    O’Hara-Wild, M., Hyndman, R., Wang, E.: Feasts: Feature Extraction and Statis- tics for Time Series. (2021). R package version 0.2.1. https://CRAN.R-project. org/package=feasts

    work page 2021

  23. [23]

    https://doi.org/ 10.48550/arXiv.1610.07717

    Christ, M., Kempa-Liehr, A.W., Feindt, M.: Distributed and Parallel Time Series Feature Extraction for Industrial Big Data Applications (2017). https://doi.org/ 10.48550/arXiv.1610.07717

    Pith/arXiv arXiv doi:10.48550/arxiv.1610.07717 2017

  24. [24]

    Neurocomputing307, 72–77 (2018) https://doi.org/10.1016/j.neucom.2018.03

    Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time Series FeatuRe Extraction on basis of Scalable Hypothesis tests (tsfresh – A Python package). Neurocomputing307, 72–77 (2018) https://doi.org/10.1016/j.neucom.2018.03. 067

    work page doi:10.1016/j.neucom.2018.03 2018

  25. [25]

    SoftwareX11, 100456 (2020) https://doi.org/10.1016/j.softx.2020.100456

    Barandas, M., Folgado, D., Fernandes, L., Santos, S., Abreu, M., Bota, P., Liu, H., Schultz, T., Gamboa, H.: TSFEL: Time Series Feature Extraction Library. SoftwareX11, 100456 (2020) https://doi.org/10.1016/j.softx.2020.100456

    arXiv 2020

  26. [26]

    https://github.com/facebookresearch/Kats 20

    Jiang, X., Srivastava, S., Chatterjee, S., Yu, Y., Handler, J., Zhang, P., Bopar- dikar, R., Li, D., Lin, Y., Thakore, U., Brundage, M., Holt, G., Komurlu, C., Nagalla, R., Wang, Z., Sun, H., Gao, P., Cheung, W., Gao, J., Wang, Q., Guer- ard, M., Kazemi, M., Chen, Y., Zhou, C., Lee, S., Laptev, N., Levendovszky, T., Taylor, J., Qian, H., Zhang, J., Shoydo...

    work page

  27. [27]

    In: 2021 International Conference on Data Mining Workshops (ICDMW), pp

    Henderson, T., Fulcher, B.D.: An Empirical Evaluation of Time-Series Feature Sets. In: 2021 International Conference on Data Mining Workshops (ICDMW), pp. 1032–1038 (2021). https://doi.org/10.1109/ICDMW53433.2021.00134

    work page arXiv 2021

  28. [28]

    In: International Work-conference on Artificial Neural Networks, pp

    Verleysen, M., Fran¸ cois, D.: The curse of dimensionality in data mining and time series prediction. In: International Work-conference on Artificial Neural Networks, pp. 758–770 (2005). Springer

    work page 2005

  29. [29]

    The Journal of Machine Learning Research7, 1–30 (2006)

    Demˇ sar, J.: Statistical Comparisons of Classifiers over Multiple Data Sets. The Journal of Machine Learning Research7, 1–30 (2006)

    work page 2006

  30. [30]

    Smart Materials and Structures29(7), 075005 (2020)

    Liu, G., Li, L., Zhang, L., Li, Q., Law, S.: Sensor faults classification for shm systems using deep learning-based method with tsfresh features. Smart Materials and Structures29(7), 075005 (2020)

    work page 2020

  31. [31]

    In: 2021 International Wireless Communications and Mobile Computing (IWCMC), pp

    Luqian, S., Yuyuan, Z.: Human activity recognition using time series pattern recognition model-based on tsfresh features. In: 2021 International Wireless Communications and Mobile Computing (IWCMC), pp. 1035–1040 (2021). IEEE

    work page 2021

  32. [32]

    Security and Communication Networks2021(1), 6677027 (2021)

    Yang, Z., Abbasi, I.A., Mustafa, E.E., Ali, S., Zhang, M.: An anomaly detection algorithm selection service for iot stream data based on tsfresh tool and genetic algorithm. Security and Communication Networks2021(1), 6677027 (2021)

    work page 2021

  33. [33]

    Sensors23(1), 12 (2022)

    Santis, R.B., Gontijo, T.S., Costa, M.A.: A data-driven framework for small hydroelectric plant prognosis using tsfresh and machine learning survival models. Sensors23(1), 12 (2022)

    work page 2022

  34. [34]

    Discover Artificial Intelligence5(1), 253 (2025)

    Gao, S.: English language intelligent expression evaluation based on multimodal interactive features. Discover Artificial Intelligence5(1), 253 (2025)

    work page 2025

  35. [35]

    Bagnall, A., Lines, J., Vickers, W., Keogh, E.: The uea & ucr time series classification repository, www.timeseriesclassification.com (2022)

    work page 2022

  36. [36]

    Henderson, T., Bryant, A.G., Fulcher, B.D.: Never a Dull Moment: Distributional Properties as a Baseline for Time-Series Classification (2023)

    work page 2023

  37. [37]

    Data Mining and Knowledge Discovery31(3), 606–660 (2017) https: //doi.org/10.1007/s10618-016-0483-9

    Bagnall, A., Lines, J., Bostrom, A., Large, J., Keogh, E.: The great time series classification bake off: A review and experimental evaluation of recent algorithmic advances. Data Mining and Knowledge Discovery31(3), 606–660 (2017) https: //doi.org/10.1007/s10618-016-0483-9

    doi:10.1007/s10618-016-0483-9 2017

  38. [38]

    Transactions of the American Institute of Electrical Engineers47(2), 617–644 (1928)

    Nyquist, H.: Certain topics in telegraph transmission theory. Transactions of the American Institute of Electrical Engineers47(2), 617–644 (1928)

    work page 1928

  39. [39]

    Proceedings of the IRE 37(1), 10–21 (1949)

    Shannon, C.E.: Communication in the presence of noise. Proceedings of the IRE 37(1), 10–21 (1949)

    work page 1949

  40. [40]

    IEEE Transactions on audio and electroacoustics15(2), 56–66 (1967)

    Bingham, C., Godfrey, M., Tukey, J.: Modern techniques of power spectrum 21 estimation. IEEE Transactions on audio and electroacoustics15(2), 56–66 (1967)

    work page 1967

  41. [41]

    The American Statistician50(4), 361–365 (1996)

    Hyndman, R.J., Fan, Y.: Sample quantiles in statistical packages. The American Statistician50(4), 361–365 (1996)

    work page 1996

  42. [42]

    Technometrics12(1), 55–67 (1970)

    Hoerl, A.E., Kennard, R.W.: Ridge regression: Biased estimation for nonorthog- onal problems. Technometrics12(1), 55–67 (1970)

    work page 1970

  43. [43]

    Wiley Interdisciplinary Reviews: Computa- tional Statistics1(1), 93–100 (2009)

    McDonald, G.C.: Ridge regression. Wiley Interdisciplinary Reviews: Computa- tional Statistics1(1), 93–100 (2009)

    work page 2009

  44. [44]

    Techno- metrics62(4), 426–433 (2020)

    Hastie, T.: Ridge regularization: An essential concept in data science. Techno- metrics62(4), 426–433 (2020)

    work page 2020

  45. [45]

    Journal of the Royal Statistical Society Series B: Statistical Methodology58(1), 267–288 (1996)

    Tibshirani, R.: Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology58(1), 267–288 (1996)

    1996

  46. [46]

    Journal of Machine Learning Research12, 2825–2830 (2011)

    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: Machine learning in Python. Journal of Machine Learning Research12, 2825–2830 (2011)

    2011

  47. [47]

    In: ECML PKDD Workshop: Languages for Data Mining and Machine Learning, pp

    Buitinck, L., Louppe, G., Blondel, M., Pedregosa, F., Mueller, A., Grisel, O., Niculae, V., Prettenhofer, P., Gramfort, A., Grobler, J., Layton, R., VanderPlas, J., Joly, A., Holt, B., Varoquaux, G.: API design for machine learning software: experiences from the scikit-learn project. In: ECML PKDD Workshop: Languages for Data Mining and Machine Learning, ...

    work page 2013

  48. [48]

    In: 2010 20th International Conference on Pattern Recognition, pp

    Brodersen, K.H., Ong, C.S., Stephan, K.E., Buhmann, J.M.: The balanced accu- racy and its posterior distribution. In: 2010 20th International Conference on Pattern Recognition, pp. 3121–3124 (2010). IEEE

    work page 2010

  49. [49]

    International Journal of Advanced Computer Science and Applications12(6), 599–606 (2021)

    Vujovi´ c,ˇZ.,et al.: Classification model evaluation metrics. International Journal of Advanced Computer Science and Applications12(6), 599–606 (2021)

    work page 2021

  50. [50]

    Data Mining and Knowledge Discovery38(4), 1958–2031 (2024)

    Middlehurst, M., Sch¨ afer, P., Bagnall, A.: Bake off redux: a review and experi- mental evaluation of recent time series classification algorithms. Data Mining and Knowledge Discovery38(4), 1958–2031 (2024)

    work page 1958

  51. [51]

    Data Mining and Knowledge Discovery35(2), 401–449 (2021) https://doi.org/10.1007/s10618-020-00727-3

    Ruiz, A.P., Flynn, M., Large, J., Middlehurst, M., Bagnall, A.: The great multi- variate time series classification bake off: A review and experimental evaluation of recent algorithmic advances. Data Mining and Knowledge Discovery35(2), 401–449 (2021) https://doi.org/10.1007/s10618-020-00727-3

    doi:10.1007/s10618-020-00727-3 2021

  52. [52]

    arXiv preprint arXiv:2407.13278 (2024)

    Wang, Y., Wu, H., Dong, J., Liu, Y., Long, M., Wang, J.: Deep time series models: A comprehensive survey and benchmark. arXiv preprint arXiv:2407.13278 (2024)

    Pith/arXiv arXiv 2024

  53. [53]

    In: Breakthroughs in 22 Statistics: Methodology and Distribution, pp

    Wilcoxon, F.: Individual comparisons by ranking methods. In: Breakthroughs in 22 Statistics: Methodology and Distribution, pp. 196–202. Springer, New York, NY (1992)

    work page 1992

  54. [54]

    Journal of machine learning research9(12) (2008)

    Garcia, S., Herrera, F.: An extension on” statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons. Journal of machine learning research9(12) (2008)

    work page 2008

  55. [55]

    https://arxiv.org/abs/2305.11921

    Ismail-Fawaz, A., Dempster, A., Tan, C.W., Herrmann, M., Miller, L., Schmidt, D.F., Berretti, S., Weber, J., Devanne, M., Forestier, G., Webb, G.I.: An Approach to Multiple Comparison Benchmark Evaluations that is Stable Under Manipulation of the Comparate Set (2023). https://arxiv.org/abs/2305.11921

    Pith/arXiv arXiv 2023

  56. [56]

    Neural Computation10(7), 1895–1923 (1998) https://doi.org/10.1162/089976698300017197

    Dietterich, T.G.: Approximate Statistical Tests for Comparing Supervised Clas- sification Learning Algorithms. Neural Computation10(7), 1895–1923 (1998) https://doi.org/10.1162/089976698300017197

    doi:10.1162/089976698300017197 1923

  57. [57]

    Machine Learning 52(3), 239–281 (2003) https://doi.org/10.1023/A:1024068626366

    Nadeau, C., Bengio, Y.: Inference for the Generalization Error. Machine Learning 52(3), 239–281 (2003) https://doi.org/10.1023/A:1024068626366

    doi:10.1023/a:1024068626366 2003

  58. [58]

    Henderson, T.: correctR: Corrected Test Statistics for Comparing Machine Learn- ing Models on Correlated Samples. (2022). R package version 0.1.2. https:// CRAN.R-project.org/package=correctR

    work page 2022

  59. [59]

    In: 7th Hellenic Conference on Informatics, pp

    Alcock, R.J., Manolopoulos, Y.,et al.: Time-series similarity queries employing a feature-based approach. In: 7th Hellenic Conference on Informatics, pp. 27–29 (1999)

    work page 1999

  60. [60]

    PhD thesis, Universite de Liege (Belgium) (2002)

    Geurts, P.: Contributions to decision tree induction: bias/variance tradeoff and time series classification. PhD thesis, Universite de Liege (Belgium) (2002)

    work page 2002

  61. [61]

    Journal of agricultural and food chemistry50(6), 1325–1329 (2002)

    Al-Jowder, O., Kemsley, E., Wilson, R.H.: Detection of adulteration in cooked meat products by mid-infrared spectroscopy. Journal of agricultural and food chemistry50(6), 1325–1329 (2002)

    work page 2002

  62. [62]

    Physica D: Nonlinear Phenom- ena142(3-4), 346–382 (2000)

    Schreiber, T., Schmitz, A.: Surrogate time series. Physica D: Nonlinear Phenom- ena142(3-4), 346–382 (2000)

    work page 2000

  63. [63]

    arXiv preprint arXiv:2511.15991 (2025)

    Nogare, T.D., Fulcher, B.D.: Identifying statistical indicators of temporal asym- metry using a data-driven approach. arXiv preprint arXiv:2511.15991 (2025)

    Pith/arXiv arXiv 2025

  64. [64]

    Acta mathematica55(1), 117–258 (1930)

    Wiener, N.: Generalized harmonic analysis. Acta mathematica55(1), 117–258 (1930)

    work page 1930

  65. [65]

    Henderson, T.: Rcatch22: Calculation of 22 CAnonical Time-Series CHaracteris- tics. (2021). R package version 0.1.12

    work page 2021

  66. [66]

    Ismail-Fawaz, A., Dempster, A., Tan, C.W., Herrmann, M., Miller, L., Schmidt, D.F., Berretti, S., Weber, J., Devanne, M., Forestier, G., Webb, G.I.: An 23 Approach to Multiple Comparison Benchmark Evaluations that is Stable Under Manipulation of the Comparate Set (2023)

    work page 2023

  67. [67]

    Nature Reviews Physics4(6), 413–420 (2022)

    Thiyagalingam, J., Shankar, M., Fox, G., Hey, T.: Scientific machine learning benchmarks. Nature Reviews Physics4(6), 413–420 (2022)

    work page 2022

  68. [68]

    In: Applied Machine Learning Using Mlr3 in R, pp

    Fischer, S., Lang, M., Becker, M.: Large-scale benchmarking. In: Applied Machine Learning Using Mlr3 in R, pp. 240–258. Chapman and Hall/CRC, New York, NY (2024)

    work page 2024

  69. [69]

    arXiv preprint arXiv:2006.10996 (2020)

    Tan, C.W., Bergmeir, C., Petitjean, F., Webb, G.I.: Monash university, uea, ucr time series extrinsic regression archive. arXiv preprint arXiv:2006.10996 (2020)

    Pith/arXiv arXiv 2006

  70. [70]

    Data Mining and Knowledge Discovery35(3), 1032–1060 (2021)

    Tan, C.W., Bergmeir, C., Petitjean, F., Webb, G.I.: Time series extrinsic regression: Predicting numeric values from time series data. Data Mining and Knowledge Discovery35(3), 1032–1060 (2021)

    work page 2021

  71. [71]

    Data Mining and Knowledge Discovery38(4), 2141–2185 (2024)

    Guijo-Rubio, D., Middlehurst, M., Arcencio, G., Silva, D.F., Bagnall, A.: Unsu- pervised feature based algorithms for time series extrinsic regression. Data Mining and Knowledge Discovery38(4), 2141–2185 (2024)

    work page 2024

  72. [72]

    Springer (2009)

    Hastie, T.: The elements of statistical learning: data mining, inference, and prediction. Springer (2009)

    work page 2009

  73. [73]

    bioRxiv, 2024–07 (2024) 24

    Alam, I., Harris, B., Cahill, P., Cliff, O., Markicevic, M., Zerbi, V., Fulcher, B.D.: Canonical time-series features for characterizing biologically informative dynamical patterns in fmri. bioRxiv, 2024–07 (2024) 24

    work page 2024