CSTS: A Benchmark for the Discovery of Correlation Structures in Time Series Clustering

Henry W J Reeve; Isabella Degen; Kate Robson Brown; Zahraa S Abdallah

REVIEW 4 major objections 5 minor 1 cited by

CSTS: A Benchmark for the Discovery of Correlation Structures in Time Series Clustering

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A benchmark with ground-truth correlation labels now lets researchers tell whether a clustering failure comes from the data, the algorithm, or the validation metric.

desk verdict A genuinely useful structure-first benchmark for correlation-based time series clustering, with one internal inconsistency in the recommended thresholds that should be fixed before publication. read the letter →

arxiv 2505.14596 v1 pith:YNJVD7SF submitted 2025-05-20 cs.LG stat.ML

Isabella Degen , Zahraa S Abdallah , Henry W J Reeve , Kate Robson Brown This is my paper · ORCID

classification cs.LGstat.ML

keywords timeseriesclusteringcorrelationstructuresyntheticbenchmarkgroundtruthlabelsvalidationmultivariatedownsamplingSpearman

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces CSTS, a synthetic benchmark for measuring how well multivariate time series clustering methods recover correlation structures. It provides ground truth labels for 23 distinct correlation patterns across 12 data variants, plus deliberately degraded clusterings and reference thresholds, so a poor result can be traced to one of three causes: the correlation structure itself has been distorted, the algorithm cannot find it, or the validation measure misreads it. The benchmark's validation experiments show that downsampling from one-second to one-minute resolution moderately distorts correlation structures, while distribution shifts and sparsification leave them nearly intact. A case study with a Toeplitz inverse covariance clustering method reveals a previously undocumented sensitivity to non-normal distributions. A sympathetic reader would care because it gives the field a way to answer the old question of whether clustering is more art than science with measurements rather than opinion.

What carries the argument

The load-bearing object is the relaxed canonical correlation pattern, a $3 \times 3$ correlation matrix whose off-diagonal coefficients are drawn from tolerance bands $\mathcal{B} = \{[-1,-0.7],\,[-0.2,0.2],\,[0.7,1]\}$, representing strong negative, negligible, and strong positive correlation. Each pattern is made positive semi-definite by eigendecomposition $P_\ell = \mathbf{U}\Lambda\mathbf{U}^T$ with negative eigenvalues clamped to zero, and the transformation matrix $\mathbf{W} = (\sqrt{\Lambda}\odot \mathbf{U})^T$ is applied to iid standard normal segments to embed the pattern into the data. This construction is what lets the paper treat any deviation between an empirical segment correlation and its target pattern as measurable error, and it supports the controlled-degradation labels and reference thresholds (silhouette width above $0.8$, Davies-Bouldin below $0.2$) used to interpret algorithm outputs.

What would settle it

Generate the same 23 relaxed patterns but with within-segment AR(1) autocorrelation at, say, $\phi=0.8$, and rerun the correlation-estimation comparisons; if the rank-based estimator's advantage over the linear one shrinks or segment-level MAE rises above $0.1$, then the iid-segment assumption is load-bearing and CSTS's claims about time series do not transfer to serially dependent data.

Watch

Extended reading notes

Core claim

CSTS is the first correlation-structure-specific benchmark for time series clustering. The paper argues that existing benchmarks built on classification datasets cannot validate discovery of correlation structure, because human class labels need not align with the statistical relationships an algorithm naturally finds. CSTS resolves this by modelling every valid correlation structure for three time series variates with strong positive, negligible, and strong negative coefficients, yielding 23 positive semi-definite "relaxed canonical" patterns, and by labelling each segment with the pattern that generated it. The paper then demonstrates that the generated structures survive distribution shifts and sparsification largely intact, that downsampling weakens strong correlations into moderate ones and hits negative correlations hardest, that rank-based correlation estimation recovers these structures more accurately than the alternatives, and that applying the benchmark to one established algorithm exposes a distributional sensitivity its original evaluation missed.

Load-bearing premise

The benchmark generates every segment as independent, identically distributed noise whose only structure is the correlation between variates, so it contains no autocorrelation, trends, or seasonality; if temporal dependencies matter to real-world correlation discovery, then CSTS results may not transfer.

Editorial extensions

If this is right

Researchers can compare an algorithm's output on CSTS against the provided degraded-clustering reference tables and say whether a failure comes from data distortion, algorithm limits, or validation choices.
Downsampling to low-frequency sampling should be avoided when correlation structure matters, because strong negative correlations can decay into moderate ones.
Rank-based correlation should be the default estimator for segment-level correlation structure, with at least 30 observations per segment for usable accuracy.
The case study shows that an algorithm validated only on normal data can fail on non-normal data, so benchmark results should be reported across data variants.
The generation framework extends to other numbers of variates, segment lengths, sparsity levels, and distribution families, providing a template for structure-specific benchmarks beyond correlation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An untested extension is whether changing the correlation estimator rescues algorithms that fail on non-normal data; CSTS's design makes that an easy ablation, but the paper only recommends the better estimator for validation, not for rescuing clustering algorithms.
A natural follow-up experiment would add within-segment autocorrelation or trends to the generator; if algorithm rankings shift, the iid-segment assumption is the limit of what CSTS can claim about time series.
The rank-deficient patterns such as $[1,1,1]$ may interact with covariance estimation in ways unrelated to correlation discovery; comparing algorithms on rank-deficient versus full-rank patterns with identical coefficients would isolate that effect.
The controlled-degradation calibration could be reused as a generic method for setting thresholds for new internal validity indices on other structural benchmarks.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: CSTS is worth engaging with. It is the first correlation-structure-specific benchmark I know of for time series clustering, and it ships the right things: explicit generator, 23 relaxed canonical patterns, 12 variants, degraded clusterings, code and data. The generation math is correct (eigen-decomposition transform W produces target correlation matrices), and the validation is systematic. The empirical findings—downsampling moderately distorts correlations, especially negative ones; Spearman beats Pearson and Kendall; segment length matters—are concrete and useful.

The soft spots are real but contained. The main one is internal to the evaluation protocol. Section 5.3 tells users to treat SWC > 0.8 and DBI < 0.2 as indicators of good structural clustering. But Appendix D Table 15 shows the ground-truth clustering of the downsampled variants at SWC ≈ 0.63 and DBI ≈ 0.50, with sparse at 0.67/0.44. A user following the protocol on downsampled data would therefore classify a perfect clustering as poor. The thresholds were calibrated on normal/non-normal variants only and are stated without that qualification. That is a genuine inconsistency, and it affects a third of the benchmark's variants. It does not sink the benchmark, but it needs fixing: either report thresholds separately per variant or present the downsampled GT values as the reference points.

Second, the TICC case study's headline finding—"previously undocumented sensitivity to non-normal distributions"—rests on untuned hyperparameters and a model that did not converge; the authors admit non-convergence. That weakens the claim. I would not call it fabricated, but "sensitivity" is too strong. Softening it or adding a converged run would help.

Third, the circularity burden: the thresholds and validation measures are calibrated on CSTS's own self-defined ground truth. That is a fair concern, and I agree it should be described as benchmark-calibrated reference values, not established universal thresholds. The absence of temporal dependencies in the data is disclosed in Section 7, so I would not count it heavily against the paper; it limits generalization but does not undermine the correlation-structure focus.

Who is this for: anyone evaluating or tuning correlation-based time series clustering algorithms or validation indices. It deserves a serious referee. I would accept it with major revision, mostly to fix the threshold inconsistency and soften the TICC claim.

Referee Report

4 major / 5 minor

Summary. The paper introduces CSTS, a synthetic benchmark for evaluating time series clustering algorithms that target correlation structures. The benchmark generates 23 valid correlation patterns for three variates, applies controlled data variations (distribution shifts, sparsification, downsampling), and provides ground-truth segment labels as well as simulated degraded clusterings. The authors validate structure preservation via MAE and tolerance-band exceedance rates, propose an evaluation protocol with recommended internal-index thresholds (SWC > 0.8, DBI < 0.2), and demonstrate the benchmark on TICC, reporting a sensitivity to non-normal distributions. The dataset and generation code are made publicly available, with an exploratory/confirmatory split for two-phase statistical validation.

Significance. CSTS addresses a genuine gap: existing time series clustering benchmarks are mostly built on classification labels rather than on structural properties, making it hard to attribute failures to data quality, algorithm limitations, or validation metrics. The strengths of the paper include a mathematically sound generation procedure based on eigen-decomposition, a systematic degradation analysis covering 12 data variants, reproducible code and data, and a carefully constructed independent confirmatory split. If the evaluation protocol is made internally consistent, the benchmark could become a useful resource for correlation-based clustering research. However, the current manuscript overstates the generality of its thresholds and its case-study conclusion, and the protocol is inconsistent for downsampled variants, which limits the benchmark's reliability as a diagnostic tool.

major comments (4)

[§5.2, §5.3, Appendix D Table 15] The paper states 'validated thresholds for strong correlation structures' (SWC > 0.8, DBI < 0.2) in §5.2 and instructs users in §5.3 to interpret SWC > 0.8 and DBI < 0.2 as indicators of good structural quality. However, Table 15 shows that the ground-truth clusterings of the downsampled variants achieve SWC 0.63–0.67 and DBI 0.44–0.50 across completeness levels. A perfect segmentation of these variants would therefore be classified as poor by the recommended fixed thresholds. Since downsampling is one of the three controlled data conditions, the diagnostic protocol is internally inconsistent for a third of the benchmark's variants. The thresholds must either be made variant-specific or the protocol must require comparison against the ground-truth baseline values in the reference tables rather than relying on fixed cut-offs.
[§6, Appendix E.2] The abstract and §6 claim that the case study identifies 'a previously undocumented sensitivity to non-normal distributions' of TICC. However, Appendix E.2 reports that TICC was run with max_iterations=10 and that it 'did not converge even with extended runs of 100 iterations'. The reported performance differences between normal and non-normal data are therefore obtained from a non-converged optimization, so the results could reflect premature termination or numerical breakdown rather than a genuine distribution sensitivity. The conclusion should be re-framed as a non-convergence finding, or the experiment should be repeated with converged TICC runs before making the distribution-sensitivity claim.
[§3, data generation] The generation procedure sets negative eigenvalues to zero (Section 3), and the transformation W = (sqrt(Lambda) ⊙ U)^T is applied to iid normal segments. For patterns with a zero eigenvalue, such as pattern 13 [1,1,1], the generated data are exactly rank-deficient, and the empirical correlation matrix can be singular. This may interact with algorithms that invert covariance matrices (e.g., TICC) in ways unrelated to the correlation structure itself. The paper should identify which of the 23 patterns are rank-deficient and either exclude them from covariance-based evaluations or recommend regularization, so that benchmark users do not mistake numerical artifacts for algorithmic failure.
[Abstract, §5.2] The terms 'established performance thresholds' and 'validated thresholds' overstate the status of the SWC/DBI cut-offs. These values are calibrated on CSTS's own 23-pattern, three-variate, 30-subject configuration with specific segment lengths and completeness levels; they are not universal constants for correlation-based clustering. The manuscript should present them as benchmark-specific calibration references and explicitly advise users to re-calibrate them when changing the number of variates, patterns, segment lengths, or sampling conditions.

minor comments (5)

[§3, Table 3, Table 17] There are several typographical errors: 'Patrial' in Section 3 should be 'Partial'; Table 3 uses '9,07' instead of '9.07'; Table 17 lists the pattern-discovery range for non-normal 100% as '21.7-739', which appears to be a typo for '21.7-73.9'; Table 13 uses 'SCW' instead of 'SWC'; Table 11 uses 'Kendal' instead of 'Kendall'. Please correct these.
[§5.3] In the 'Cluster-to-Ground-Truth Mapping' step, the paper specifies matching within a tolerance of ±0.1, while Section 3 defines tolerance bands B = {[−1,−0.7], [−0.2,0.2], [0.7,1]}. The relationship between the ±0.1 matching tolerance and the tolerance bands should be clarified, as the bands are used to define the ground-truth categories but the matching uses a different threshold.
[§2, Related Work] The claim that CSTS is 'to our knowledge the first correlation structure-specific evaluation framework' should be supported by a more comprehensive related-work discussion, including recent work on change-point detection benchmarks and covariance-based time series clustering, to ensure the novelty claim is accurate.
[§6, Appendix E.2] The choice of TICC hyperparameters (clusters=23, window=5, switch penalty=400, lambda=0.11, max iterations=10) is described as 'close to the original parameters', but no specific source is cited for these values. Please provide a reference or a table showing how the original TICC parameters were adapted.
[Table 1, §6] The DBI value '>19Mio' for the 10% non-normal variant is flagged in a footnote as an artifact of near-zero centroid distances. This explanation should appear in the main text, otherwise readers may misinterpret it as a meaningful performance value.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CSTS is a self-contained synthetic benchmark whose validations and thresholds are calibration artifacts, not derived predictions or self-citation chains.

full rationale

CSTS is a synthetic-benchmark paper rather than a derivation of predictions from first principles. The ground-truth segments are produced by applying the eigendecomposition-based transformation W=(sqrt(Lambda) circle U)^T to iid standard-normal segments, and the Section 4 MAE validation compares the empirical Spearman matrices of those same segments back to the relaxed target matrices P'_ell. This is a generator sanity check, not a hidden prediction: the target enters the construction, but the MAE is a measurement of finite-sample and degradation error, and it could in principle have been large. No output of the paper is claimed to follow from a fitted parameter without independent support. The SWC>0.8 and DBI<0.2 thresholds in Sections 5.2-5.3 are calibration values taken from CSTS's own reference tables; this is self-referential benchmarking, which is normal for a benchmark, and I do not count it as circular. I note as a correctness (not circularity) concern that Table 15 gives ground-truth SWC approximately 0.63 and DBI approximately 0.50 on downsampled variants, so the recommended thresholds are internally inconsistent for a third of the benchmark's variants; this does not make the MAE-based deterioration claim circular. The TICC case study is an empirical application with acknowledged confounds (untuned hyperparameters, non-convergence), not a derivation. There are no load-bearing self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation. Therefore the circularity score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The benchmark's ground truth is defined by the generator's own parameters (tolerance bands, relaxed coefficients, segment lengths, distribution choices), so validation results and thresholds are calibrated on the same design choices. No external benchmark is used to independently confirm that CSTS's thresholds generalize. The main free parameters are the hand-chosen tolerance bands and relaxed coefficients, plus thresholds derived from the benchmark itself.

free parameters (5)

Tolerance bands B = {[-1,-0.7],[-0.2,0.2],[0.7,1]} = [-1,-0.7],[-0.2,0.2],[0.7,1]
Hand-chosen thresholds defining strong negative, negligible, and strong positive correlation; used to relax ideal patterns and to define out-of-tolerance segments. No external justification beyond 'meaningful thresholds'.
Relaxed coefficient values (0.7 or 0.71) = 0.7 or 0.71 for non-ideal patterns
Chosen to lie within the tolerance bands while making correlation matrices positive semi-definite; the exact edge values are arbitrary within the band.
Minimum segment length threshold = 30 observations
Empirically derived from the MAE table (MAE < 0.1 at length 30); presented as a recommendation but not derived from a first-principles model.
Performance thresholds SWC > 0.8, DBI < 0.2 = 0.8 SWC, 0.2 DBI
Calibrated on CSTS's own ground-truth and degraded clusterings; not validated on external benchmarks.
Non-normal distribution parameters (IOB, COB, IG) = Extreme value shape [-0.52,0.07], negative binomial n=1, etc.
Fitted to real-world T1D data in prior work and used as inputs to mimic an application; not fitted to benchmark outcomes.

assumptions (5)

standard math Linear transformation of iid standard normal data by W = (sqrt(Lambda) ⊙ U)^T produces data with population correlation P'.
Used in Section 3 to encode correlation structures; correct when S columns are independent with unit variance, but for PSD matrices with zero eigenvalues the data is degenerate.
domain assumption Mean absolute error (MAE) and percentage of segments outside tolerance bands are adequate measures of correlation structure preservation.
Section 4; MAE aggregates three coefficients equally and does not capture which coefficient fails.
ad hoc to paper Tolerance bands B define meaningful strong/negligible correlation categories.
Section 3; no independent justification for [-0.2,0.2] as negligible or [0.7,1] as strong.
domain assumption Minute-mean aggregation is a representative model of downsampling.
Section 3; other downsampling strategies (decimation, last-observation) are not tested.
ad hoc to paper TICC hyperparameters (clusters=23, window=5, switch penalty=400, lambda=0.11, max iterations=10) provide a fair test of TICC's distribution sensitivity.
Appendix E.2; untuned and non-converged TICC may fail for reasons unrelated to non-normality.

how reviews work

0 comments

Cite this review

Pith. "Pith review of CSTS: A Benchmark for the Discovery of Correlation Structures in Time Series Clustering." pith.science (2026). https://pith.science/paper/YNJVD7SF

@misc{pith2026250514596,
  author       = {Pith},
  title        = {Pith review of: CSTS: A Benchmark for the Discovery of Correlation Structures in Time Series Clustering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YNJVD7SF}},
  note         = {Machine review of arXiv:2505.14596}
}

read the original abstract

Time series clustering promises to uncover hidden structural patterns in data with applications across healthcare, finance, industrial systems, and other critical domains. However, without validated ground truth information, researchers cannot objectively assess clustering quality or determine whether poor results stem from absent structures in the data, algorithmic limitations, or inappropriate validation methods, raising the question whether clustering is "more art than science" (Guyon et al., 2009). To address these challenges, we introduce CSTS (Correlation Structures in Time Series), a synthetic benchmark for evaluating the discovery of correlation structures in multivariate time series data. CSTS provides a clean benchmark that enables researchers to isolate and identify specific causes of clustering failures by differentiating between correlation structure deterioration and limitations of clustering algorithms and validation methods. Our contributions are: (1) a comprehensive benchmark for correlation structure discovery with distinct correlation structures, systematically varied data conditions, established performance thresholds, and recommended evaluation protocols; (2) empirical validation of correlation structure preservation showing moderate distortion from downsampling and minimal effects from distribution shifts and sparsification; and (3) an extensible data generation framework enabling structure-first clustering evaluation. A case study demonstrates CSTS's practical utility by identifying an algorithm's previously undocumented sensitivity to non-normal distributions, illustrating how the benchmark enables precise diagnosis of methodological limitations. CSTS advances rigorous evaluation standards for correlation-based time series clustering.

Figures

Figures reproduced from arXiv: 2505.14596 by the authors.

**Figure 2.** Effect of segment length on MAE between specified correlation structures and their estimation [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Empirical distributions (blue) with theoretical PDF/PMF (red) for the [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Empirical distributions (blue) with theoretical PDF/PMF (red) for the [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]

**Figure 5.** Figure 5: Empirical distributions (blue) with theoretical PDF/PMF (red) for the [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]

**Figure 6.** Figure 6: Histogram of time intervals between observations (in second) for the [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]

**Figure 7.** Figure 7: Correlation structures visualisation of Spearman estimates calculated using aggregated observations [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]

**Figure 8.** Figure 8: Correlation structures visualisation of Spearman estimates calculated using aggregated observations [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

RMISC: A Large-scale Real-world Multivariate Corpus for Time Series Foundation Models
cs.AI 2026-07 conditional novelty 6.0 of 10

A curated 142-billion-point real-world multivariate time series corpus improves zero-shot forecasting when combined with existing synthetic and univariate pretraining data across four foundation models.

Reference graph

Works this paper leans on

44 extracted references · 26 canonical work pages · cited by 1 Pith paper

[1]

Zomaya, Sebti Foufou, and Abdelaziz Bouras

Adil Fahad, Najlaa Alshatri, Zahir Tari, Abdullah Alamri, Ibrahim Khalil, Albert Y . Zomaya, Sebti Foufou, and Abdelaziz Bouras. A survey of clustering algorithms for big data: Taxonomy and empirical analysis. IEEE Transactions on Emerging Topics in Computing, 2:267–279, 9 2014. URL: https://ieeexplore.ieee.org/document/6832486, doi:10.1109/TETC.2014.2330519

work page arXiv 2014
[2]

Ezugwu, Abiodun M

Absalom E. Ezugwu, Abiodun M. Ikotun, Olaide O. Oyelade, Laith Abualigah, Jeffery O. Agushaka, Christopher I. Eke, and Andronicus A. Akinyelu. A comprehensive survey of clus- tering algorithms: State-of-the-art machine learning applications, taxonomy, challenges, and fu- ture research prospects. Engineering Applications of Artificial Intelligence , 110:10...

arXiv 2022
[3]

A comprehensive survey of clustering algorithms

Dongkuan Xu and Yingjie Tian. A comprehensive survey of clustering algorithms. Annals of Data Science 2015 2:2, 2:165–193, 8 2015. URL: https://link.springer.com/article/10.1007/ s40745-015-0040-1 , doi:10.1007/S40745-015-0040-1

doi:10.1007/s40745-015-0040-1 2015
[4]

Gao, Dominic Dwyer, Ye Zhu, Catherine L

Caroline X. Gao, Dominic Dwyer, Ye Zhu, Catherine L. Smith, Lan Du, Kate M. Filia, Johanna Bayer, Jana M. Menssink, Teresa Wang, Christoph Bergmeir, Stephen Wood, and Sue M. Cotton. An overview of clustering methods with guidelines for application in mental health research. Psychiatry Research, 327:115265, 9 2023. URL: https://www.sciencedirect.com/scienc...

work page arXiv 2023
[5]

Cluster analysis: A modern statistical review

Adam Jaeger and David Banks. Cluster analysis: A modern statistical review. Wiley Interdisciplinary Reviews: Computational Statistics, 15:e1597, 5 2023. URL: https://onlinelibrary.wiley.com/ doi/full/10.1002/wics.1597, doi:10.1002/WICS.1597

doi:10.1002/wics.1597 2023
[6]

Bridging the gap: A decade review of time-series clustering methods, 2024

John Paparrizos, Fan Yang, and Haojun Li. Bridging the gap: A decade review of time-series clustering methods, 2024. URL: https://arxiv.org/abs/2412.20582, arXiv:2412.20582

arXiv 2024
[7]

Lag penalized weighted correlation for time series cluster- ing

Thevaa Chandereng and Anthony Gitter. Lag penalized weighted correlation for time series cluster- ing. BMC Bioinformatics, 21, 1 2020. URL: https://www.ncbi.nlm.nih.gov/pmc/articles/ PMC6966853/, doi:10.1186/S12859-019-3324-1

work page doi:10.1186/s12859-019-3324-1 2020
[8]

A review of two decades of correlations, hierarchies, networks and clustering in financial markets

Gautier Marti, Frank Nielsen, Mikołaj Bi ´nkowski, and Philippe Donnat. A review of two decades of correlations, hierarchies, networks and clustering in financial markets. Signals and Communica- tion Technology, pages 245–274, 2021. URL: https://link.springer.com/chapter/10.1007/ 978-3-030-65459-7_10 , doi:10.1007/978-3-030-65459-7_10

work page doi:10.1007/978-3-030-65459-7_10 2021

Show all 44 references

[9]

Analysis of similarity measures in times series clustering for the discovery of building energy patterns

Félix Iglesias and Wolfgang Kastner. Analysis of similarity measures in times series clustering for the discovery of building energy patterns. Energies 2013, Vol. 6, Pages 579-597, 6:579–597, 1 2013. URL: https://www.mdpi.com/1996-1073/6/2/579/htm, doi:10.3390/EN6020579

2013 doi
[10]

Michael C. Thrun. Distance-based clustering challenges for unbiased benchmarking studies. Scientific Reports 2021 11:1 , 11:1–12, 9 2021. URL: https://www.nature.com/articles/ s41598-021-98126-1 , doi:10.1038/s41598-021-98126-1

2021 doi
[11]

Enhancing cluster analysis via topological manifold learning

Moritz Herrmann, Daniyal Kazempour, Fabian Scheipl, and Peer Kröger. Enhancing cluster analysis via topological manifold learning. Data Mining and Knowledge Discovery , 38:840– 887, 5 2024. URL: https://link.springer.com/article/10.1007/s10618-023-00980-2 , doi:10.1007/S10618-...

2024 doi
[12]

The ucr time series archive

Hoang Anh Dau, Anthony Bagnall, Kaveh Kamgar, Chin-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, Eamonn Keogh, Hoang Anh Dau, Anthony Bagnall, Kaveh Kamgar, Chin-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, and E...

2019
[13]

Ali Javed, Byung Suk Lee, and Donna M. Rizzo. A benchmark study on time series clustering. Machine Learning with Applications, 1:100001, 9 2020. URL: https://www.sciencedirect.com/ science/article/pii/S2666827020300013, doi:10.1016/J.MLWA.2020.100001

2020
[14]

Odyssey: An engine enabling the time-series clustering journey

John Paparrizos and Sai Prasanna Teja Reddy. Odyssey: An engine enabling the time-series clustering journey. Proceedings of the VLDB Endowment, 16:4066–4069, 8 2023. URL: https://dl.acm.org/ doi/10.14778/3611540.3611622, doi:10.14778/3611540.3611622. 10 Preprint: CSTS - August...

2023
[15]

Niyogi, S

P. Niyogi, S. Smale, and S. Weinberger. A topological view of unsupervised learning from noisy data. https://doi.org/10.1137/090762932, 40:646–663, 6 2011. URL: https://epubs.siam.org/doi/ 10.1137/090762932, doi:10.1137/090762932

2011 doi
[16]

Understanding and enhancement of internal clustering validation measures

Yanchi Liu, Zhongmou Li, Hui Xiong, Xuedong Gao, Junjie Wu, and Sen Wu. Understanding and enhancement of internal clustering validation measures. IEEE transactions on cybernetics, 43:982–994, 6 2013. URL: https://pubmed.ncbi.nlm.nih.gov/23193245/, doi:10.1109/TSMCB.2012. 2220543

2013
[17]

Time series clustering method with cluster validation to identify unknown local cell conditions in the aluminum reduction cell

Zhaoke Huang, Chunhua Yang, Xiaofang Chen, Xiaojun Zhou, and Weihua Gui. Time series clustering method with cluster validation to identify unknown local cell conditions in the aluminum reduction cell. Computers & Industrial Engineering, 174:108790, 12 2022. URL: https://www.sc...

2022
[18]

Clustering method for time-series images using quantum-inspired digital annealer technology

Tomoki Inoue, Koyo Kubota, Tsubasa Ikami, Yasuhiro Egami, Hiroki Nagai, Takahiro Kashikawa, Koichi Kimura, and Yu Matsuda. Clustering method for time-series images using quantum-inspired digital annealer technology. Communications Engineering 2024 3:1, 3:1–9, 1 2024. URL: http...

2024 doi
[19]

Williamson

Isabelle Guyon, Ulrike von Luxburg, and Robert C. Williamson. Clustering: Science or art? In NIPS 2009 Workshop on Clustering: Science or art? Towards principled approaches, Vancouver, Canada, December 2009. Position paper. URL: https://stanford.edu/~rezab/nips2009workshop/ op...

2009
[20]

Synthcity: a benchmark framework for diverse use cases of tabular synthetic data

Zhaozhi Qian, Rob Davis, and Mihaela van der Schaar. Synthcity: a benchmark framework for diverse use cases of tabular synthetic data. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36,...

2023
[21]

Scientific machine learning benchmarks

Jeyan Thiyagalingam, Mallikarjun Shankar, Geoffrey Fox, and Tony Hey. Scientific machine learning benchmarks. Nature Reviews Physics 2022 4:6, 4:413–420, 4 2022. URL: https://www.nature. com/articles/s42254-022-00441-7 , doi:10.1038/s42254-022-00441-7

2022 doi
[22]

Bake off redux: a review and ex- perimental evaluation of recent time series classification algorithms

Matthew Middlehurst, Patrick Schäfer, and Anthony Bagnall. Bake off redux: a review and ex- perimental evaluation of recent time series classification algorithms. Data Mining and Knowl- edge Discovery, 38:1958–2031, 7 2024. URL: https://link.springer.com/article/10.1007/ s1061...

1958 doi
[23]

Toeplitz inverse covariance-based clustering of multivariate time series data

David Hallac, Sagar Vare, Stephen Boyd, and Jure Leskovec. Toeplitz inverse covariance-based clustering of multivariate time series data. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’17, page 215–223, New York, NY , USA,
[24]

Campello, and Eduardo R

Lucas Vendramin, Ricardo J.G.B. Campello, and Eduardo R. Hruschka. Relative clustering validity criteria: A comparative overview. Statistical Analysis and Data Mining: The ASA Data Science Journal, 3:209–235, 8 2010. URL: https://onlinelibrary.wiley.com/doi/full/10.1002/ sam.1...

2010 doi
[25]

Pérez, and Iñigo Perona

Olatz Arbelaitz, Ibai Gurrutxaga, Javier Muguerza, Jesús M. Pérez, and Iñigo Perona. An ex- tensive comparative study of cluster validity indices. Pattern Recognition, 46:243–256, 1 2013. URL: https://www.sciencedirect.com/science/article/pii/S003132031200338X, doi: 10.1016/J....

2013 doi
[26]

Yerbury, Ricardo J

Luke W. Yerbury, Ricardo J. G. B. Campello, G. C. Livingston Jr, Mark Goldsworthy, and Lachlan O’Neil. On the use of relative validity indices for comparing clustering approaches, 2024. URL: https://arxiv.org/abs/2404.10351, arXiv:2404.10351

2024
[27]

Analytical comparison of clustering techniques for the recognition of communication patterns

Muhammed Fatih Kaya and Mareike Schoop. Analytical comparison of clustering techniques for the recognition of communication patterns. Group Decision and Negotiation , 31:555–589, 6 2022. URL: https://link.springer.com/article/10.1007/s10726-021-09758-7 , doi: 10.1007/S10726-02...

2022 doi
[28]

K-means properties on six clustering benchmark datasets

Pasi Fränti and Sami Sieranoja. K-means properties on six clustering benchmark datasets. Applied Intelligence, 48:4743–4759, 12 2018. URL: https://link.springer.com/article/10.1007/ s10489-018-1238-7 , doi:10.1007/S10489-018-1238-7 . 11 Preprint: CSTS - August 19, 2025

2018 doi
[29]

Thrun and Alfred Ultsch

Michael C. Thrun and Alfred Ultsch. Clustering benchmark datasets exploiting the fundamental clustering problems. Data in Brief, 30:105501, 6 2020. URL: https://www.sciencedirect.com/ science/article/pii/S2352340920303954, doi:10.1016/J.DIB.2020.105501

2020
[30]

van der Zant, and Mickael L

Maria El Abbassi, Jan Overbeck, Oliver Braun, Michel Calame, Herre S.J. van der Zant, and Mickael L. Perrin. Benchmark and application of unsupervised classification approaches for univariate data. Communications Physics 2021 4:1, 4:1–9, 3 2021. URL: https://www.nature.com/art...

2021 doi
[31]

Dance: a deep learning library and benchmark platform for single-cell analysis

Jiayuan Ding, Renming Liu, Hongzhi Wen, Wenzhuo Tang, Zhaoheng Li, Julian Vene- gas, Runze Su, Dylan Molho, Wei Jin, Yixin Wang, Qiaolin Lu, Lingxiao Li, Wangyang Zuo, Yi Chang, Yuying Xie, and Jiliang Tang. Dance: a deep learning library and benchmark platform for single-cell...

2024 doi
[32]

Rousseeuw

Peter J. Rousseeuw. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics , 20:53–65, 11 1987. URL: https://www.sciencedirect.com/science/article/pii/0377042787901257, doi: 10.1016/0377-0427(87)90125-7

1987
[33]

Davies and Donald W

David L. Davies and Donald W. Bouldin. A cluster separation measure. IEEE Transactions on Pattern Analysis and Machine Intelligence , PAMI-1:224–227, 1979. URL: https://ieeexplore.ieee. org/document/4766909, doi:10.1109/TPAMI.1979.4766909

1979
[34]

A metric for covariance matrices

Wolfgang Förstner and Boudewijn Moonen. A metric for covariance matrices. Geodesy-The Challenge of the 3rd Millennium, pages 299–309, 2003. URL: https://link.springer.com/chapter/10. 1007/978-3-662-05296-9_31 , doi:10.1007/978-3-662-05296-9_31

2003 doi
[35]

Time series classification with feature covariance matrices

Hamza Ergezer and Kemal Leblebicio˘glu. Time series classification with feature covariance matrices. Knowledge and Information Systems, 55:695–718, 6 2018. URL: https://link.springer.com/ article/10.1007/s10115-017-1098-1 , doi:10.1007/S10115-017-1098-1

2018 doi
[36]

An evaluation of change point detection algorithms

Gerrit J J Van Den Burg and Christopher K I Williams. An evaluation of change point detection algorithms. arXiv.org, 3 2020. URL: https://github.com/alan-turing-institute/TCPD

2020
[37]

André Gensler and B. Sick. Novel criteria to measure performance of time series segmentation techniques. LWA, 2014. 12 Preprint: CSTS - August 19, 2025 A Dataset Characteristics A.1 Key statistics This section provides descriptive statistics for both the exploratory and the co...

2014
[39]

The MAE for these patterns in the complete variant can move a correlation structure out of their negligible and strong correlation coefficients grouping into the moderate range

keep a low MAE through downsampling (0.03 − 0.07). The MAE for these patterns in the complete variant can move a correlation structure out of their negligible and strong correlation coefficients grouping into the moderate range. Interestingly, sparsification in the downsampled...
[40]

Selecting Data Variants We conducted the evaluation across six data variants, including the normal and non-normal distribution types with each three completeness levels (100%, 70%, 10%). This selection allowed us to assess TICC’s sensitivity to both distributional assumptions ...
[41]

Once these MRFs were learnt, we applied the trained models to the remaining 29 subjects without retraining

Generating Clustering Results For each data variant, we trained TICC on the exploratory subject ’unique-puddle-26’ to learn the Markov Random Fields (MRFs Θi) that in TICC represent each cluster’s relationship structure. Once these MRFs were learnt, we applied the trained mode...
[42]

This preserves the distinct correlation structure of each individual segment

Calculating Evaluation Measures For each data variant, we first mapped the clusters discovered by TICC to the ground truth correlation patterns of CSTS by grouping all segments assigned to each cluster identified by TICC and calculating the median value for each coefficient po...

2025
[43]

Interpretation To interpret and contextualise the results, we used the reference tables for CSTS (see Appendix D). These tables provide both ground truth baselines and systematically degraded results with known numbers of misclassified observations and misassigned segments for...
[44]

Statistical Validation Statistical validation was performed using Wilcoxon signed rank tests with a two-sided alternative hypothesis. We tested three hypotheses investigating whether the differences in SWC between the normal and non-normal data variants were significant, as we...

2025
[2017]

doi:10.1145/3097983.3098060

Association for Computing Machinery. doi:10.1145/3097983.3098060

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Zomaya, Sebti Foufou, and Abdelaziz Bouras

Adil Fahad, Najlaa Alshatri, Zahir Tari, Abdullah Alamri, Ibrahim Khalil, Albert Y . Zomaya, Sebti Foufou, and Abdelaziz Bouras. A survey of clustering algorithms for big data: Taxonomy and empirical analysis. IEEE Transactions on Emerging Topics in Computing, 2:267–279, 9 2014. URL: https://ieeexplore.ieee.org/document/6832486, doi:10.1109/TETC.2014.2330519

work page arXiv 2014

[2] [2]

Ezugwu, Abiodun M

Absalom E. Ezugwu, Abiodun M. Ikotun, Olaide O. Oyelade, Laith Abualigah, Jeffery O. Agushaka, Christopher I. Eke, and Andronicus A. Akinyelu. A comprehensive survey of clus- tering algorithms: State-of-the-art machine learning applications, taxonomy, challenges, and fu- ture research prospects. Engineering Applications of Artificial Intelligence , 110:10...

arXiv 2022

[3] [3]

A comprehensive survey of clustering algorithms

Dongkuan Xu and Yingjie Tian. A comprehensive survey of clustering algorithms. Annals of Data Science 2015 2:2, 2:165–193, 8 2015. URL: https://link.springer.com/article/10.1007/ s40745-015-0040-1 , doi:10.1007/S40745-015-0040-1

doi:10.1007/s40745-015-0040-1 2015

[4] [4]

Gao, Dominic Dwyer, Ye Zhu, Catherine L

Caroline X. Gao, Dominic Dwyer, Ye Zhu, Catherine L. Smith, Lan Du, Kate M. Filia, Johanna Bayer, Jana M. Menssink, Teresa Wang, Christoph Bergmeir, Stephen Wood, and Sue M. Cotton. An overview of clustering methods with guidelines for application in mental health research. Psychiatry Research, 327:115265, 9 2023. URL: https://www.sciencedirect.com/scienc...

work page arXiv 2023

[5] [5]

Cluster analysis: A modern statistical review

Adam Jaeger and David Banks. Cluster analysis: A modern statistical review. Wiley Interdisciplinary Reviews: Computational Statistics, 15:e1597, 5 2023. URL: https://onlinelibrary.wiley.com/ doi/full/10.1002/wics.1597, doi:10.1002/WICS.1597

doi:10.1002/wics.1597 2023

[6] [6]

Bridging the gap: A decade review of time-series clustering methods, 2024

John Paparrizos, Fan Yang, and Haojun Li. Bridging the gap: A decade review of time-series clustering methods, 2024. URL: https://arxiv.org/abs/2412.20582, arXiv:2412.20582

arXiv 2024

[7] [7]

Lag penalized weighted correlation for time series cluster- ing

Thevaa Chandereng and Anthony Gitter. Lag penalized weighted correlation for time series cluster- ing. BMC Bioinformatics, 21, 1 2020. URL: https://www.ncbi.nlm.nih.gov/pmc/articles/ PMC6966853/, doi:10.1186/S12859-019-3324-1

work page doi:10.1186/s12859-019-3324-1 2020

[8] [8]

A review of two decades of correlations, hierarchies, networks and clustering in financial markets

Gautier Marti, Frank Nielsen, Mikołaj Bi ´nkowski, and Philippe Donnat. A review of two decades of correlations, hierarchies, networks and clustering in financial markets. Signals and Communica- tion Technology, pages 245–274, 2021. URL: https://link.springer.com/chapter/10.1007/ 978-3-030-65459-7_10 , doi:10.1007/978-3-030-65459-7_10

work page doi:10.1007/978-3-030-65459-7_10 2021