Quantitative Evaluation of Motif Sets in Time Series

Aras Yurtman; Daan Van Wesenbeeck; Hendrik Blockeel; Wannes Meert

REVIEW 2 major objections 4 minor 32 references

Quantitative Evaluation of Motif Sets in Time Series

T0 review · 2 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read PROM is a metric that evaluates time series motif discovery by matching discovered motifs to ground-truth motifs by overlap, then optimally matching the motif sets, and finally computing precision, recall, and F1 from the resulting…

desk verdict PROM and TSMD-Bench are a solid, reproducible contribution to TSMD evaluation, but the 'broadly applicable' claim is narrower than advertised because of the non-overlap assumption. read the letter →

arxiv 2412.09346 v1 pith:VBFOXAUT submitted 2024-12-12 cs.LG cs.CV

Daan Van Wesenbeeck , Aras Yurtman , Wannes Meert , Hendrik Blockeel This is my paper · ORCID

classification cs.LGcs.CV

keywords timeseriesmotifdiscoveryquantitativeevaluationprecisionrecalloptimalmatchingbenchmarkconstructionvariable-lengthmotifsmultivariate

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that quantitative evaluation of time series motif discovery has been limited by metrics that assume fixed-length motifs, a single motif set, or motif pairs, and often ignore false discoveries. It introduces PROM, which compares any discovered collection of motif sets against a ground truth of motif sets in three steps: match individual motifs when their time overlap exceeds half, optimally match motif sets with an assignment algorithm, and count true and false positives and negatives to produce precision, recall, and F1. The paper also builds TSMD-Bench, fourteen benchmark datasets assembled from real time series classification data, which it claims are more representative and harder than the random-walk benchmarks used previously. Experiments with eleven methods are presented as evidence that older metrics skew toward either recall or precision, while PROM measures both, and that large-scale systematic comparison of motif discovery methods is now possible.

What carries the argument

The carrying mechanism is a two-level matching pipeline with a named identity at its core. For two segments the overlap rate is $\mathrm{OR}(\alpha, \beta) = \frac{|\alpha \cap \beta|}{|\alpha \cup \beta|}$, and segments are matchable when $\mathrm{OR} > 0.5$. Lemma 1 states that because ground-truth segments never overlap, each discovered segment is matchable with at most one ground-truth segment; this makes the per-motif matching order-independent and maximizes total overlap. The per-motif match counts form a contingency table, whose columns are then permuted by the Hungarian method to maximize the diagonal sum, giving an optimal matching of motif sets. Precision, recall, and F1 are micro-averaged over the matching matrix, with an option to ignore false-positive motif sets so that well-conserved but off-target patterns are not penalized.

What would settle it

Take a time series whose ground truth contains two motif sets that overlap in time, run PROM on a fixed discovered set, and see whether the reported precision and recall change when the ground-truth segments are processed in the reverse order; because Lemma 1 fails in that case, any order-dependence would falsify the claim that PROM is a broadly applicable, order-independent metric.

Watch

Extended reading notes

Core claim

The central claim is that a single criterion, PROM, can quantitatively evaluate any time series motif discovery output in the general setting of multiple motif sets, arbitrary cardinality, and variable-length motifs. PROM's definition reduces evaluation to a two-level matching: per-motif matching by an overlap rate threshold, then a per-set assignment solved as a linear sum assignment problem. From the resulting matching matrix it derives true positives, false negatives, and false positives at both the motif and the set level, and these yield micro-averaged precision, recall, and F1. The paper shows that on 30,800 results generated by eleven methods over fourteen datasets, PROM ranks results differently from existing correctness and score metrics, which the authors attribute to those metrics measuring mostly recall or mostly precision.

Load-bearing premise

The metric assumes the ground-truth motif sets are made of non-overlapping segments, so no segment of the time series belongs to two true motifs; if real ground truths contain overlapping or nested motifs, the matching guarantee behind PROM no longer holds.

Editorial extensions

If this is right

PROM produces a precision, recall, and F1 score for any time series motif discovery result, so methods that differ in the number of motif sets, motif length, or dimensionality can be compared on the same scale.
The benchmark construction converts any easily clusterable time-series classification dataset into a time series motif discovery benchmark with labeled ground truth, yielding fourteen datasets with separate validation and test splits.
Because the correctness metric mainly tracks recall and the score metric mainly tracks precision, a method comparison that relies on either one alone can be misleading; reporting PROM's precision and recall together avoids the bias.
A simple detector that finds random-walk regions achieves nearly perfect F1 on random-walk-based synthetic benchmarks, which supports the claim that those benchmarks are too easy to be representative.
With PROM and TSMD-Bench, model selection and hyperparameter tuning for time series motif discovery methods can be done automatically through F1 on validation sets, enabling large-scale systematic performance comparisons.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper: the two-level matching at the heart of PROM is generic enough to apply to other structured prediction tasks where the output is a set of sets with instance-level matching, such as event detection or pattern mining, since it only relies on overlap and optimal assignment.
Beyond the paper: the choice to ignore false-positive motif sets is a tuning decision that can change rankings; a fair comparison protocol would need a fixed default or a two-score report with and without penalizing off-target sets.
Beyond the paper: TSMD-Bench's construction from real instances may still create unlabeled higher-order patterns, since two classes can share a sub-shape; quantifying how often such off-target patterns appear would clarify how clean the ground truth actually is.
Beyond the paper: if overlapping or nested ground-truth motifs become important in an application, PROM would need to replace its non-overlap assumption with a one-to-many matching scheme, trading simplicity for broader applicability.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

PROM and TSMD-Bench are a solid, reproducible contribution to TSMD evaluation. The headline result is real: the field gets a formally defined precision/recall metric that handles variable-length, multivariate, multi-set cases, and a benchmark built from real UCR/UEA data instead of random-walk plus inserted instances.

What's actually new: the overlap criterion itself appeared in their prior LoCoMotif paper, but this paper is the first to formalize it as a metric—Lemma 1, the matching matrix, and the complexity analysis are original. TSMD-Bench is a genuinely new benchmark construction, and the 11-method comparison is the largest systematic evaluation I've seen in this area. The random-walk triviality result (a Dickey-Fuller-based method gets F1=0.98 on random-walk benchmarks) convincingly backs their claim that those benchmarks are too easy.

The math holds up. Lemma 1 is correct given the stated assumption that ground-truth segments from different motif sets cannot overlap, and the assignment step is standard. The experimental setup is careful: validation-based hyperparameter tuning, statistical tests via critical difference diagrams, and 30,800 results. They also ship code, which is good practice.

Soft spots, in proportion. The main one is the load-bearing assumption of non-overlapping GT motif sets. The abstract says 'broadly applicable,' but that's an overstatement. If GT motif sets can overlap or nest, a discovered segment can be matchable with two GT segments, and the greedy matching becomes order-dependent—exactly the example in the stress-test note. The authors state the assumption clearly in Section 2, so it's not an internal inconsistency, but the framing oversells the scope. The benchmark is constructed to satisfy the assumption, so the experiments never test how much this matters in practice. I'd like to see a discussion of when the assumption holds and possibly an extension or a principled restriction.

Second, the benchmark dataset selection uses ARI >= 0.5, which biases toward easily clusterable datasets. That doesn't invalidate the benchmark, but it's worth stating as a design choice with consequences. Third, the claim that PROM is 'more comprehensive' is supported by design arguments and examples, not an external validation. That's normal for a metric paper, so I wouldn't hold it against them. Fourth, the authors evaluate their own method with their own metric—disclosed, but the rankings should be read with that in mind.

Overall, this deserves a serious referee. The contributions are useful and the flaws are addressable. I'd recommend accepting it with revisions that clarify the scope and possibly add a discussion of overlapping ground truth.

Referee Report

2 major / 4 minor

Summary. The paper introduces PROM, a precision-recall-F1-style metric for evaluating time series motif discovery (TSMD) results against ground-truth motif sets, and TSMD-Bench, a set of synthetic benchmark time series built from UCR/UEA classification instances. PROM matches discovered motifs to ground-truth motifs by overlap rate, matches discovered motif sets to ground-truth sets via the Hungarian algorithm, and derives micro- or macro-averaged precision, recall, and F1 from the resulting matching matrix. The benchmark is constructed by concatenating class instances from easily clusterable classification datasets. Experiments with 11 TSMD methods on 14 benchmark datasets compare PROM with existing metrics, analyze method performance, and show that random-walk-based benchmarks are too easy.

Significance. If PROM is adopted, it would give the TSMD community a standard evaluation tool that handles multiple, variable-length, and multivariate motif sets while penalizing both motif-level and set-level false positives and negatives. The paper is careful: the matching procedure is formally defined with a proof of Lemma 1, hyperparameters are tuned on validation sets, and rankings are compared with Kendall's tau and statistical significance tests. The benchmark construction is a useful new resource. The main caveats are that the metric assumes non-overlapping ground-truth motif sets, which limits the 'broadly applicable' claim, and that a scalability claim is unsubstantiated.

major comments (2)

[Section 2, Section 4.1, Lemma 1] The non-overlap assumption on ground-truth motif sets (Section 2: 'segments from different motif sets cannot overlap') is load-bearing: Lemma 1 and the order-independence of the individual matching procedure both rely on it. If ground-truth segments from different sets are allowed to overlap or nest, a discovered segment can be matchable with two ground-truth segments (e.g., β1=[1:10], β2=[6:15], α=[1:15] gives OR(α,β1)=OR(α,β2)=10/15>0.5), making the greedy matching order-dependent and the contingency table change. The paper is internally consistent, but the abstract and Section 1 claim that PROM is 'broadly applicable' and 'overcomes those limitations' of existing metrics. Since real-world TSMD applications can involve overlapping or nested recurring patterns, I recommend that the authors either extend the matching to handle overlapping ground-truth segments, or add an explicit limitation statement and restrict the 'broadly applicable' claim to non-overlapping ground-truth settings.
[Section 6.1.1] The statement that PROM has time complexity O(km + m^3) in the illustrative case g=d=m with k motifs per set is not justified by the algorithm as described. The individual matching step in Section 4.1 requires, for each ground-truth segment, finding the discovered segment with highest overlap rate; without an additional data structure this is O((mk)^2) comparisons, yielding O(m^2 k^2 + m^3) overall. The authors should either describe an efficient implementation that achieves the claimed bound or correct the complexity claim, since the current text suggests PROM scales linearly in the time series length, which the described procedure does not guarantee.

minor comments (4)

[Section 5.2] The text refers to the 'UCR and UAE Time Series Classification Archive'; the archive is UCR/UEA. Please correct the typo (also in the same paragraph, 'UKAC' appears to be a misspelling).
[Section 4.1] The proof of Lemma 1 can be made more explicit: if β1 and β2 were disjoint, their intersections with α would be disjoint subsets of α, so |α∩β1|+|α∩β2| ≤ |α|, contradicting each being greater than 0.5|α|.
[Section 4.2] The text says 'The bottom right entry M*_{d+1,g+1} is undefined', but since M* is (g+1)×(d+1), the bottom-right entry should be M*_{g+1,d+1}. Please fix the index order.
[Figure 12] The critical difference diagrams would be more informative if they included the critical difference value and the exact post-hoc test used (e.g., Nemenyi), so that the statistical significance claims can be assessed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PROM is a self-contained metric and TSMD-Bench uses external ground truth; the non-overlap restriction is an explicit scope assumption, not a circular step.

full rationale

PROM is defined from explicit first-principles primitives: overlap rate OR, matchability threshold 0.5, greedy per-GT motif matching justified by Lemma 1, optimal set-level assignment solved as a linear sum assignment problem, and standard micro-averaged precision/recall/F1. The only load-bearing geometric premise is the stated non-overlap of ground-truth segments; given that premise, Lemma 1 and the order-independence of the matching are proved internally rather than assumed from prior work. The benchmark TSMD-Bench constructs ground-truth motif sets from class labels of external UCR/UEA archives, so the benchmark targets are not derived from PROM or from any TSMD method's output. The self-citation to Van Wesenbeeck et al. (2024) is a provenance disclosure for the criterion and an evaluated method; it is not used to justify PROM's correctness or uniqueness. Hyperparameters are selected on validation sets and fixed for the test set, so the experimental comparison is not a fitted-input-called-prediction loop. The skeptical concern about overlapping or nested ground-truth segments is a scope limitation or correctness risk for the 'broadly applicable' claim, not a circularity: the paper explicitly assumes non-overlap and all derivations are internally consistent under that assumption.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a small number of transparent design choices and standard background results. The main domain assumptions are the non-overlap of ground-truth motif sets and the clusterability of selected classification datasets. No new physical or algorithmic entities are postulated.

free parameters (4)

Overlap rate threshold = 0.5
Hand-chosen threshold above which a discovered and ground-truth motif are considered matchable (Section 4.1). The metric's scores depend on this value; the paper notes a stricter threshold can be used but does not test sensitivity.
ARI selection threshold = 0.5
Hand-chosen Adjusted Rand Index threshold for selecting classification datasets for TSMD-Bench (Section 5.2). Determines which datasets enter the benchmark and thus the results.
Validation/test set sizes = N=50 validation, N=200 test
Hand-chosen numbers of time series per benchmark dataset (Section 5.1). Affects statistical power and computational cost.
Significance level alpha = 0.05
Used for critical difference diagrams (Section 6.2.3) and the Dickey-Fuller test in the RQ3 demonstration (Section 6.3). Standard but hand-chosen.

assumptions (5)

domain assumption Ground-truth motif sets are non-overlapping, including across sets (Section 2).
Used in Lemma 1; if violated, a discovered motif could be matchable with two GT motifs, breaking the greedy matching.
domain assumption Instances of the same class in the selected UCR/UEA datasets are similar in shape (Section 5.1).
Justified only indirectly via the ARI >= 0.5 filter; the benchmark ground truth depends on this assumption.
standard math The first difference of a random walk is stationary (Section 6.3).
Underlies the Dickey-Fuller-based triviality detector in RQ3.
standard math The Dickey-Fuller test correctly distinguishes stationary from non-stationary windows at alpha=0.05 (Section 6.3).
Used in RQ3 to demonstrate that random-walk benchmarks are trivial.
standard math The Hungarian algorithm solves the linear sum assignment problem in polynomial time (Section 4.2).
Used for optimal motif-set matching in PROM.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Quantitative Evaluation of Motif Sets in Time Series." pith.science (2026). https://pith.science/paper/VBFOXAUT

@misc{pith2026241209346,
  author       = {Pith},
  title        = {Pith review of: Quantitative Evaluation of Motif Sets in Time Series},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VBFOXAUT}},
  note         = {Machine review of arXiv:2412.09346}
}

read the original abstract

Time Series Motif Discovery (TSMD), which aims at finding recurring patterns in time series, is an important task in numerous application domains, and many methods for this task exist. These methods are usually evaluated qualitatively. A few metrics for quantitative evaluation, where discovered motifs are compared to some ground truth, have been proposed, but they typically make implicit assumptions that limit their applicability. This paper introduces PROM, a broadly applicable metric that overcomes those limitations, and TSMD-Bench, a benchmark for quantitative evaluation of time series motif discovery. Experiments with PROM and TSMD-Bench show that PROM provides a more comprehensive evaluation than existing metrics, that TSMD-Bench is a more challenging benchmark than earlier ones, and that the combination can help understand the relative performance of TSMD methods. More generally, the proposed approach enables large-scale, systematic performance comparisons in this field.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 18 canonical work pages

[1]

Alaee, S., Kamgar, K., and Keogh, E. (2020). Matrix Profile XXII : Exact Discovery of Time Series Motifs Under DTW . In 2020 IEEE International Conference on Data Mining ( ICDM ) , pages 900--905, Sorrento, Italy. IEEE. https://doi.org/10.1109/ICDM50108.2020.00099

arXiv 2020
[2]

Bagnall, A., Hills, J., and Lines, J. (2014). Finding motif sets in time series. arXiv preprint arXiv:1407.3685. https://doi.org/10.48550/arXiv.1407.3685

arXiv doi:10.48550/arxiv.1407.3685 2014
[3]

Balasubramanian, A., Wang, J., and Prabhakaran, B. (2016). Discovering Multidimensional Motifs in Physiological Signals for Personalized Healthcare . IEEE Journal of Selected Topics in Signal Processing , 10(5):832--841. https://doi.org/10.1109/JSTSP.2016.2543679

work page arXiv 2016
[4]

Benavoli, A., Corani, G., and Mangili, F. (2016). Should we really use post-hoc tests based on mean-ranks? Journal of Machine Learning Research , 17(5):1--10

work page 2016
[5]

and Azevedo, P

Castro, N. and Azevedo, P. (2010). Multiresolution Motif Discovery in Time Series , pages 665--676. https://doi.org/10.1137/1.9781611972801.73

work page doi:10.1137/1.9781611972801.73 2010
[6]

M., Zhu, Y., Gharghabi, S., Ratanamahatana, C

Dau, A., Keogh, E., Kamgar, K., Yeh, C.-C. M., Zhu, Y., Gharghabi, S., Ratanamahatana, C. A., Yanping, Hu, B., Begum, N., Bagnall, A., Mueen, A., and Batista, G. (2018). The UCR time series classification archive. https://www.cs.ucr.edu/ eamonn/time\_series\_data\_2018/

work page 2018
[7]

Dau, H. A. and Keogh, E. (2017). Matrix profile V : A generic technique to incorporate domain knowledge into motif discovery. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , page 125–134. https://doi.org/10.1145/3097983.3097993

arXiv 2017
[8]

and Batista, G

Furtado Silva, D. and Batista, G. (2018). Elastic Time Series Motifs and Discords . In 2018 17th IEEE International Conference on Machine Learning and Applications ( ICMLA ) , pages 237--242. https://doi.org/10.1109/ICMLA.2018.00042

work page arXiv 2018

Show all 32 references

[9]

and Lin, J

Gao, Y. and Lin, J. (2018). Exploring variable-length time series motifs in one hundred million length scale. Data Mining and Knowledge Discovery , 32(5):1200--1228. https://doi.org/10.1007/s10618-018-0570-1

2018 doi
[10]

Gomes, E. F. and Batista, F. (2015). Using multiresolution time series motifs to classify urban sounds. International Journal of Software Engineering and Its Applications , 9(8):189--196. https://doi.org/10.14257/ijseia.2015.9.8.17

2015 doi
[11]

Grabocka, J., Schilling, N., and Schmidt-Thieme, L. (2016). Latent time-series motifs. ACM Transactions on Knowledge Discovery from Data , 11(1). https://doi.org/10.1145/2940329

2016 doi
[12]

Harris, R. (1992). Testing for unit roots using the augmented dickey-fuller test: Some issues relating to the size, power and the lag structure of the test. Economics Letters , 38(4):381--386. https://doi.org/10.1016/0165-1765(92)90022-Q

1992 doi
[13]

S., and Rizzo, D

Javed, A., Lee, B. S., and Rizzo, D. M. (2020). A benchmark study on time series clustering. Machine Learning with Applications , 1. https://doi.org/10.1016/j.mlwa.2020.100001

2020
[14]

and Volgenant, T

Jonker, R. and Volgenant, T. (1987). A shortest augmenting path algorithm for dense and sparse linear assignment problems. Computing , 38:325--340. https://doi.org/10.1007/BF02278710

1987 doi
[15]

Law, S. M. (2019). STUMPY: A Powerful and Scalable Python Library for Time Series Data Mining . The Journal of Open Source Software , 4(39):1504. https://doi.org/10.21105/joss.01504

2019 doi
[16]

Lin, J., Keogh, E., Lonardi, S., and Patel, P. (2002). Finding motifs in time series. Proceedings of the Second Workshop on Temporal Data Mining

2002
[17]

Linardi, M., Zhu, Y., Palpanas, T., and Keogh, E. (2018). Matrix Profile X : VALMOD - Scalable Discovery of Variable - Length Motifs in Data Series . In Proceedings of the 2018 International Conference on Management of Data , pages 1053--1066. https://doi.org/10.1145/3183713.3183744

2018
[18]

Liu, B., Li, J., Chen, C., Tan, W., Chen, Q., and Zhou, M. (2015). Efficient motif discovery for large-scale time series in healthcare. IEEE Transactions on Industrial Informatics , 11(3):583--590. https://doi.org/10.1109/TII.2015.2411226

2015
[19]

Minnen, D., Starner, T., Essa, I., and Jr, C. (2007). Improving activity discovery with automatic neighborhood estimation. In Proceedings of the 20th International Joint Conference on Artificial Intelligence , pages 2814--2819

2007
[20]

and Nishida, T

Mohammad, Y. and Nishida, T. (2014). Shift density estimation based approximately recurring motif discovery. Applied Intelligence , 42(1):112–134. http://doi.org/10.1007/s10489-014-0531-3

2014 doi
[21]

and Chavoshi, N

Mueen, A. and Chavoshi, N. (2015). Enumeration of Time Series Motifs of All Lengths . Knowledge and Information Systems , 45(1):105--132. https://doi.org/10.1109/ICDM.2013.27

2015 doi
[22]

Nunthanid, P., Niennattrakul, V., and Ratanamahatana, C. A. (2012). Parameter-free motif discovery for time series data. In 2012 9th International Conference on Electrical Engineering / Electronics , Computer , Telecommunications and Information Technology , pages 1--4. https:...

2012
[23]

and Leser, U

Schäfer, P. and Leser, U. (2023). Motiflets: Simple and accurate detection of motifs in time series. Proceedings of the VLDB Endowment , 16:725--737. https://doi.org/10.14778/3574245.3574257

2023
[24]

P., Chen, C., and Frankenstein, S

Senin, P., Lin, J., Wang, X., Oates, T., Gandhi, S., Boedihardjo, A. P., Chen, C., and Frankenstein, S. (2018). Grammarviz 3.0: Interactive discovery of variable-length time series patterns. ACM Trans. Knowl. Discov. Data , 12(1). https://doi.org/10.1145/3051126

2018 doi
[25]

A., Akhavan, Z., and Mueen, A

Siddiquee, M. A., Akhavan, Z., and Mueen, A. (2019). Seismo: Semi-supervised time series motif discovery for seismic signal detection. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management , CIKM '19, page 99–108, New York, NY, USA. As...

2019
[26]

v., and Postma, E

van Leeuwen, F., Bosma, B., den Born, A. v., and Postma, E. (2023). Normalization in Motif Discovery . In Machine Learning , Optimization , and Data Science , pages 314--325. Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-25891-6_24

2023 doi
[27]

Van Onsem, M., Ledoux, V., Mélange, W., Dreesen, D., and Van Hoecke, S. (2023). Variable length motif discovery in time series data. IEEE Access , 11:73754--73766. 10.1109/ACCESS.2023.3295995

2023
[28]

Van Wesenbeeck, D., Yurtman, A., Meert, W., and Blockeel, H. (2024). LoCoMotif : Discovering time-warped motifs in time series. Data Mining and Knowledge Discovery . https://doi.org/10.1007/s10618-024-01032-z

2024 doi
[29]

Williams, B. (2008). Character Trajectories . UCI Machine Learning Repository. https://doi.org/10.24432/C58G7V

2008 doi
[30]

M., Kavantzas, N., and Keogh, E

Yeh, C.-C. M., Kavantzas, N., and Keogh, E. (2017). Matrix profile VI : Meaningful multidimensional motif discovery. In 2017 IEEE International Conference on Data Mining (ICDM) , pages 565--574. https://doi.org/10.1109/ICDM.2017.66

2017 doi
[31]

Yingchareonthawornchai, S., Sivaraks, H., Rakthanmanon, T., and Ratanamahatana, C. A. (2013). Efficient proper length time series motif discovery. In 2013 IEEE 13th International Conference on Data Mining , pages 1265--1270. https://doi.org/10.1109/ICDM.2013.111

2013 doi
[32]

S., Yeh, C.-C

Zhu, Y., Zimmerman, Z., Senobari, N. S., Yeh, C.-C. M., Funning, G., Mueen, A., Brisk, P., and Keogh, E. (2016). Matrix profile II : Exploiting a novel algorithm and gpus to break the one hundred million barrier for time series motifs and joins. In 2016 IEEE 16th International...

2016

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Alaee, S., Kamgar, K., and Keogh, E. (2020). Matrix Profile XXII : Exact Discovery of Time Series Motifs Under DTW . In 2020 IEEE International Conference on Data Mining ( ICDM ) , pages 900--905, Sorrento, Italy. IEEE. https://doi.org/10.1109/ICDM50108.2020.00099

arXiv 2020

[2] [2]

Bagnall, A., Hills, J., and Lines, J. (2014). Finding motif sets in time series. arXiv preprint arXiv:1407.3685. https://doi.org/10.48550/arXiv.1407.3685

arXiv doi:10.48550/arxiv.1407.3685 2014

[3] [3]

Balasubramanian, A., Wang, J., and Prabhakaran, B. (2016). Discovering Multidimensional Motifs in Physiological Signals for Personalized Healthcare . IEEE Journal of Selected Topics in Signal Processing , 10(5):832--841. https://doi.org/10.1109/JSTSP.2016.2543679

work page arXiv 2016

[4] [4]

Benavoli, A., Corani, G., and Mangili, F. (2016). Should we really use post-hoc tests based on mean-ranks? Journal of Machine Learning Research , 17(5):1--10

work page 2016

[5] [5]

and Azevedo, P

Castro, N. and Azevedo, P. (2010). Multiresolution Motif Discovery in Time Series , pages 665--676. https://doi.org/10.1137/1.9781611972801.73

work page doi:10.1137/1.9781611972801.73 2010

[6] [6]

M., Zhu, Y., Gharghabi, S., Ratanamahatana, C

Dau, A., Keogh, E., Kamgar, K., Yeh, C.-C. M., Zhu, Y., Gharghabi, S., Ratanamahatana, C. A., Yanping, Hu, B., Begum, N., Bagnall, A., Mueen, A., and Batista, G. (2018). The UCR time series classification archive. https://www.cs.ucr.edu/ eamonn/time\_series\_data\_2018/

work page 2018

[7] [7]

Dau, H. A. and Keogh, E. (2017). Matrix profile V : A generic technique to incorporate domain knowledge into motif discovery. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , page 125–134. https://doi.org/10.1145/3097983.3097993

arXiv 2017

[8] [8]

and Batista, G

Furtado Silva, D. and Batista, G. (2018). Elastic Time Series Motifs and Discords . In 2018 17th IEEE International Conference on Machine Learning and Applications ( ICMLA ) , pages 237--242. https://doi.org/10.1109/ICMLA.2018.00042

work page arXiv 2018