Pith. sign in

REVIEW 3 major objections 4 minor 38 references

ConceptCF: Concept-based Counterfactuals for the Explainability of Time Series

T0 review · 3 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Pith's one-line read ConceptCF generates time-series counterfactuals by mutating high-level concepts rather than raw points, with validity guaranteed and top-tier scores on standard metrics.

desk verdict A promising concept-level counterfactual framework for time series, with a real but fixable evaluation flaw in the sparsity metric. read the letter →

arxiv 2607.18748 v1 pith:FGQ2GSFC submitted 2026-07-21 cs.LG cs.AI

classification cs.LGcs.AI
keywords concept-basedexplainabilitycounterfactualexplanationstimeseriesclassificationgeneticalgorithmdecompositionhuman-centeredconceptsFouriertransformwavelet
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that counterfactual explanations for time-series classifiers should be generated by perturbing high-level, human-interpretable concepts—such as the signal's scale, bias, trend, or frequency content—instead of individual timestamps or subsequences. It formalizes the concept-based counterfactual problem, solves it with a genetic algorithm that mutates concepts while keeping the construction invertible, and evaluates the method on five datasets against five existing approaches. ConceptCF always returns a valid counterfactual (guaranteed by construction) and ranks first on proximity and sparsity, second on confidence and plausibility. The point matters because in high-stakes settings an explanation that says 'increase the scale and the model would predict Sit instead of Walk' is more actionable and easier to audit than a list of changed timestamps.

What carries the argument

The concept set Sx = d(x) with an invertible construction d and reconstruction h. Three decompositions instantiate it: Fourier frequency bands (DFT), wavelet components (DWT), and a human-centered decomposition into trend, bias, scale, low-frequency, variance, and high-frequency components. The genetic algorithm mutates these concepts (scalars by Gaussian noise clipped to observed ranges; sequences by KD-tree-based replacement from training data), with the nearest-unlike-neighbour seeding and elitism guaranteeing validity, while sparsity is encouraged by a preset probability that returns concepts to their original values.

What would settle it

Give users original/counterfactual pairs from ConceptCF and ask them to name which concept changed; if identification is at chance level, the interpretability promise fails. Alternatively, train a classifier that provably depends only on, say, the high-frequency component and check whether flips are always accompanied by a change in that component.

Watch

Extended reading notes

Core claim

ConceptCF's central claim is that restricting counterfactual search to an invertible concept space—where concepts are scalars or sequences with semantic meaning—preserves the explanatory power of the mutation while remaining competitive with raw-signal methods. The paper proves (Proposition 1) that, because the nearest unlike neighbour is seeded into the initial population and elitism preserves it, every generation's best solution satisfies the validity constraint. Empirically, the results show ConceptCF at rank 1 for proximity and sparsity and rank 2 for confidence and plausibility across the five datasets, with validity always satisfied.

Load-bearing premise

The concept decomposition must capture properties that are both causally used by the classifier and intuitively meaningful to a human; if the model relies on signal content the concepts do not expose, the generated counterfactuals can be valid and sparse yet explain nothing real.

Editorial extensions

If this is right

  • If correct, counterfactual explanations for time series can be stated in terms of human-understandable properties (e.g., 'scale decreased') rather than opaque point changes.
  • The validity guarantee via nearest-unlike-neighbour seeding and elitism means users never receive an explanation that fails to flip the model's prediction, regardless of the optimizer's exploration.
  • Sparse counterfactuals (few concepts changed) are achievable without sacrificing validity, which point-based optimization methods cannot do.
  • The decomposition choice (Fourier, wavelet, human-centered) has limited effect on quantitative metrics, so the choice can be driven by end-user needs.
  • The same framework extends to targeted counterfactuals toward a specified class, widening its use to 'what would make the model predict class Y'.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's interpretability is asserted via concept design, not measured: a user study asking people to identify the changed concept from paired signals would directly test this claim.
  • Because metric-optimality is measured in point space while mutations occur in concept space, a counterfactual could be pointwise close yet semantically odd if the decomposition glosses over nonstationary behaviour; testing on nonstationary or noise-dominated series would stress this.
  • The invertibility requirement suggests a natural extension to other invertible representations (e.g., autocorrelation or spectral envelopes), which could be combined with the proposed genetic search without changing the algorithm.
  • If the classifier relies on high-frequency detail that the human-centered decomposition absorbs into the high-frequency component, the explanation 'scale changed' may be incomplete; targeted ablation of each concept's causal effect on the prediction would quantify which concepts actually drive flips.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces ConceptCF, a counterfactual explanation method for time series that operates on high-level 'concepts' (e.g., trend, scale, frequency bands) rather than on individual points or subsequences. The method extracts concepts via one of three invertible decompositions (DFT, DWT, human-centered), then uses a genetic algorithm to perturb the concept set subject to a validity constraint. A formal guarantee (Proposition 1) is provided: under a sufficiently large penalty and with NUN initialization and elitism, all generated counterfactuals are valid. The paper evaluates ConceptCF against five baselines on five datasets, reporting validity, confidence, proximity, sparsity, and plausibility, and claims top-tier performance, especially on sparsity.

Significance. The central idea—framing counterfactual generation as an optimization over interpretable concepts—is a meaningful contribution to time-series XAI. The formalization in Section 3 and the validity guarantee in Proposition 1 are clear and useful. The evaluation is broad: five baselines, five datasets, and a sensitivity analysis of the objective weights, which is commendable. However, the empirical claims are currently undermined by a non-comparable sparsity metric, a plausibility metric that is also an optimization objective, and an untested interpretability assertion. If these issues are addressed, the paper could be a solid contribution to the field.

major comments (3)
  1. [Section 5.6, Eq. (2), Eq. (3)] The sparsity metric is defined differently for different methods: for ConceptCF it counts the fraction of concepts changed (out of 6), for Native Guide and Sub-SpaCE the fraction of subsequences, and for Wachter, TSEvo, and Glacier the fraction of points. These units are not commensurable. A single concept mutation (e.g., scaling) can alter every time point of the reconstructed signal, yet it is counted as 1/6 of a unit. Consequently, the reported sparsity values (e.g., 0.288 for ConceptCF on MS-HAR vs. 1.000 for Wachter) are not like-for-like, and the statement in Section 6.2 that 'ConceptCF has the best sparsity across all datasets' is unsupported. Please report sparsity in a common unit (e.g., fraction of time points changed) or report both concept-level and point-level sparsity, and adjust the claims accordingly.
  2. [Section 5.6, Eq. (2), Eq. (3)] The plausibility metric used in the evaluation (IOS, Eq. (2)) is the same function that appears as the O_pl objective in the fitness function (Eq. (3)), and the autoencoder f_AE used in the metric is the same one used inside the optimization. Thus ConceptCF is directly optimizing the evaluation metric. This creates a circularity that inflates its plausibility scores relative to baselines that do not optimize the same objective. Please evaluate plausibility using an independent measure not used in the optimization, such as distance to the nearest training sample, density in a separately trained latent space, or likelihood under a probabilistic generative model.
  3. [Section 1, Section 7] The paper's central motivation is that the generated counterfactuals are 'human-interpretable' and 'meaningful' because they operate on concepts. However, the paper explicitly defers user evaluation to future work (Section 7), and the human-centered decomposition is taken from prior work without evidence that these concepts align with either end-user understanding or the features actually used by the classifier. Without such evidence, the explainability claim is asserted rather than demonstrated. Please add a user study or a task-based proxy (e.g., measuring whether users can predict the model's decisions from the concept explanations), or temper the claims from 'human-interpretable' to 'concept-level' throughout, including the title and abstract.
minor comments (4)
  1. [Section 4.1] The human-centered decomposition is only briefly described and the reader is referred to Jutte et al. (2025). Since this decomposition is central to one of the three concept variants, please provide a more self-contained summary, including the definitions of cLF, cVar, and cHF and any assumptions about their mutual independence.
  2. [Section 5.4] The hyperparameters of the genetic algorithm and the objective weights are said to be 'chosen based on initial experimentation' with no further detail. Please report the search range or the reasoning behind the chosen values to improve reproducibility.
  3. [Section 3.2, Eq. (2)] The definition of e(x, x_CF) with min(0, ...) means that if the counterfactual has lower reconstruction error than the original, the plausibility term becomes negative. The paper does not explain how negative plausibility values are interpreted or why the min is applied. A brief justification would help.
  4. [Section 6.2] Table 2 reports global ranks based on only five datasets; the authors note this should be interpreted cautiously. This is appreciated, but the large number of ties (e.g., rank 1 shared by several methods on proximity) makes the ranking difficult to interpret. Consider also reporting effect sizes or confidence intervals.

Circularity Check

2 steps flagged · score 6.0 of 10

Top-tier sparsity and confidence/plausibility scores largely reduce to ConceptCF's own optimized objective functions and to a self-defined sparsity unit.

  1. fitted input called prediction [Section 3.2 (Eq. 1) and Section 5.6 (Metrics)]
    "For the optimization, ... we formulate the problem in terms of four objectives: confidence, proximity, sparsity and plausibility. ... For the metrics we follow Refoyo and Luengo [2024] and consider metrics in line with the previously defined objectives for the optimization problem."

    The evaluation metrics named in the abstract are, for confidence, sparsity and plausibility, exactly the objective terms Oconf, Os and Opl from Eq. (3) that the genetic algorithm maximizes. Section 5.6 confirms that the metrics are 'in line with' the objectives. Consequently, ConceptCF's reported 'top-tier performance' on these metrics is a restatement of its own fitness components rather than an independent test. The sparsity metric is especially direct: Os is the fraction of concepts changed, and the reported sparsity is exactly that optimized fraction.

  2. self definitional [Section 5.6, Sparsity definition]
    "Sparsity: The fraction of 'meaningful' units changed from the original sample to the counterfactual. For Native Guide and Sub-SpaCE we consider the number of subsequences changed. For ConceptCF, we consider the number of concepts. For the other approaches, we consider the number of points."

    The sparsity claim 'ConceptCF has the best sparsity across all datasets' is produced by counting different units for ConceptCF than for the baselines. ConceptCF has only m=6 concepts, so one concept mutation such as a scale change can alter every timestamp yet counts as 1/6. Point-based methods count every changed point, yielding values near 1.0. Thus the rank-1 sparsity advantage is an artifact of the metric's unit choice, not an independently measured property.

full rationale

The paper does not claim a first-principles derivation; its central contribution is an empirical method plus benchmark. However, the benchmark's strongest claims are partially circular. The GA in Eq. (3) optimizes confidence, sparsity, and plausibility, and Section 5.6 explicitly adopts metrics 'in line with' those objectives, so the reported top-tier scores on those metrics largely restate the optimization target. The sparsity comparison is further compromised by the self-defined counting unit: ConceptCF is scored on concepts (6 per signal), while other methods are scored on points or subsequences. This makes the headline sparsity advantage a definitional artifact rather than a fair comparison. The human-centered decomposition is imported from the authors' own prior work (Jutte et al., 2025), but it is an input to the method rather than a claim reduced to that citation; user interpretability is explicitly deferred, so that self-citation is not load-bearing for the measured metrics. Proposition 1 is a definitional validity guarantee via the nearest unlike neighbour and is not a prediction. Overall, the empirical advantage is partially forced by construction, but the method still has independent content: the concept-based formulation, the three decompositions, and the proximity results are not purely self-referential.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper's central claims depend on a set of hand-chosen objective weights and GA hyperparameters (Section 5.4), on the validity of the human-centered decomposition from the authors' prior work, and on an invertibility requirement for the concept construction. No independent new entities are introduced.

free parameters (4)
  • Fitness weights alpha_conf=0.4, alpha_prox=0.6, alpha_s=0.4, alpha_pl=0.2, beta=100 = alpha_conf=0.4, alpha_prox=0.6, alpha_s=0.4, alpha_pl=0.2, beta=100
    Chosen based on initial experimentation (Section 5.4) without a held-out validation set; these directly control the counterfactual objectives.
  • GA hyperparameters (ngen=100, nindiv=50, nelite=2, nsaturate=15, nparents=10, pmutation=0.2, preset=0.05, nNN=50) = ngen=100, nindiv=50, nelite=2, nsaturate=15, nparents=10, pmutation=0.2, preset=0.05, nNN=50
    Set manually (Section 5.4); affect convergence and exploration of the genetic algorithm.
  • DWT wavelet (Daubechies 6) and number of components (ndetail=3) = Daubechies 6, ndetail=3
    Selected in Section 5.3; not fitted but nevertheless a modeling choice.
  • Human-centered sliding window size (27) = 27
    Taken from the authors' prior work Jutte et al. [2025] (Section 5.3); controls the variance component extraction.
assumptions (5)
  • domain assumption Concept construction d(·) is invertible via h(·) so that x = h(d(x))
    Section 3.1 requires invertibility to reconstruct a signal from perturbed concepts; if this fails, counterfactuals cannot be generated.
  • domain assumption For every test sample x, a nearest unlike neighbour xNUN exists in the training set
    Section 4.2.1 relies on the NUN for validity guarantee and initial population; datasets with a single class would break this.
  • ad hoc to paper The penalty beta can be set large enough that validity-violating solutions always have lower fitness than the NUN
    Proposition 1 (Section 4.2.4) depends on this; beta=100 is chosen empirically.
  • domain assumption The human-centered decomposition (Jutte et al. [2025]) produces concepts that are interpretable to end users and relevant to the classifier's decisions
    Section 4.1; this is the core interpretability claim, but no user study is performed (deferred to future work in Section 7).
  • domain assumption Autoencoder reconstruction error is a valid proxy for data plausibility
    Section 3.2; the plausibility objective and metric both rely on this assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ConceptCF: Concept-based Counterfactuals for the Explainability of Time Series." pith.science (2026). https://pith.science/paper/FGQ2GSFC

@misc{pith2026260718748,
  author       = {Pith},
  title        = {Pith review of: ConceptCF: Concept-based Counterfactuals for the Explainability of Time Series},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FGQ2GSFC}},
  note         = {Machine review of arXiv:2607.18748}
}
read the original abstract

This paper proposes ConceptCF, a method for counterfactual generation that operates on human-interpretable concepts. In high-stakes domains such as healthcare and predictive maintenance, artificial intelligence models can increase efficiency and safety. Explainability is key to ensure these models rely on causal relationships rather than spurious correlations. Counterfactual explanations identify minimal modifications that would change a model's predictions. Existing methods for time series operate on individual points or subsequences without ensuring interpretability of the mutations. ConceptCF instead modifies meaningful concepts. As a result we can provide explanations in terms of these concepts, for example ``the model's prediction would be `Sit' instead of `Walk' if you increase the scale of the movement''. In this paper, the concepts are constructed through time series decomposition, resulting in concepts such as scale, and frequency bands. Counterfactuals are generated using a genetic algorithm that optimizes the concept mutations. Evaluation against five state-of-the-art approaches demonstrates that ConceptCF consistently achieves top-tier performance across validity, confidence, proximity, sparsity and plausibility metrics.

Figures

Figures reproduced from arXiv: 2607.18748 by the authors.

Figure 1
Figure 1. ConceptCF for time series. Unlike point-wise optimization, ConceptCF operates on underlying concepts to generate [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Targeted generation was applied towards class ‘Sit’. The [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 2
Figure 2. Counterfactual explanations for a selected sample from the MS-HAR dataset of class ‘Walk’, counterfactuals are [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figures from the paper (1 more)
Figure 3
Figure 3. Figure 3: Sensitivity analysis of the objective weights [PITH_FULL_IMAGE:figures/full_fig_p007_3.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 7 canonical work pages

  1. [1]

    Counterfactual

    Wachter, Sandra and Mittelstadt, Brent and Russell, Chris , year =. Counterfactual. doi:10.2139/ssrn.3063289 , journal =

  2. [2]

    Machine Learning , author =

    Glacier: guided locally constrained counterfactual explanations for time series classification , volume =. Machine Learning , author =. 2024 , pages =. doi:10.1007/s10994-023-06502-x , number =

  3. [3]

    Dandl, Susanne and Molnar, Christoph and Binder, Martin and Bischl, Bernd , year =. Multi-. doi:10.1007/978-3-030-58112-1_31 , booktitle =

  4. [4]

    doi:10.1109/ICMLA55696.2022.00013 , booktitle =

    Höllig, Jacqueline and Kulbach, Cedric and Thoma, Steffen , year =. doi:10.1109/ICMLA55696.2022.00013 , booktitle =

  5. [5]

    Poeta, Eleonora and Ciravegna, Gabriele and Pastor, Eliana and Cerquitelli, Tania and Baralis, Elena , journal=

  6. [6]

    ACM Computing Surveys , author =

    Counterfactual. ACM Computing Surveys , author =. 2024 , pages =. doi:10.1145/3677119 , number =

  7. [7]

    Knowledge and Information Systems , author =

    Locally and globally explainable time series tweaking , volume =. Knowledge and Information Systems , author =. 2020 , pages =. doi:10.1007/s10115-019-01389-4 , number =

  8. [8]

    , year =

    Delaney, Eoin and Greene, Derek and Keane, Mark T. , year =. Instance-. doi:10.1007/978-3-030-86957-1_3 , booktitle =

Show all 38 references
  1. [9]

    and Coskun, Ayse K

    Ates, Emre and Aksar, Burak and Leung, Vitus J. and Coskun, Ayse K. , year =. Counterfactual. doi:10.1109/ICAPAI49758.2021.9462056 , booktitle =

  2. [10]

    Learning

    Wang, Zhendong and Samsten, Isak and Mochaourab, Rami and Papapetrou, Panagiotis , year =. Learning. doi:10.1007/978-3-030-88942-5_29 , booktitle =

  3. [11]

    Refoyo, Mario and Luengo, David , year =. Sub-. doi:10.1007/978-3-031-63800-8_1 , booktitle =

  4. [12]

    Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies , author =

    X-. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies , author =. 2023 , pages =. doi:10.1145/3580804 , number =

  5. [13]

    Interpretability

    Kim, Been and Wattenberg, Martin and Gilmer, Justin and Cai, Carrie and Wexler, James and Viegas, Fernanda and Sayres, Rory , year =. Interpretability. Proceedings of the 35th International Conference on Machine Learning , publisher =

  6. [14]

    Koh, Pang Wei and Nguyen, Thao and Tang, Yew Siang and Mussmann, Stephen and Pierson, Emma and Kim, Been and Liang, Percy , year =. Concept. Proceedings of the 37th

  7. [15]

    arXiv preprint arXiv:1907.07165 , author =

    Explaining. arXiv preprint arXiv:1907.07165 , author =. 2020 , doi =

  8. [16]

    Proceedings of the AAAI Conference on Artificial Intelligence , author =

    Deep. Proceedings of the AAAI Conference on Artificial Intelligence , author =. doi:10.1609/aaai.v32i1.11771 , number =

  9. [17]

    Chen, Chaofan and Li, Oscar and Tao, Daniel and Barnett, Alina and Rudin, Cynthia and Su, Jonathan K , year =. This. Advances in

  10. [18]

    arXiv preprint arXiv:2203.11103 , author =

    Diverse. arXiv preprint arXiv:2203.11103 , author =. doi:10.48550/arXiv.2203.11103 , year =

  11. [19]

    Comparative review of selection techniques in genetic algorithm , doi =

    Shukla, Anupriya and Pandey, Hari Mohan and Mehrotra, Deepti , year =. Comparative review of selection techniques in genetic algorithm , doi =. 2015

  12. [20]

    Time series classification from scratch with deep neural networks:

    Wang, Zhiguang and Yan, Weizhong and Oates, Tim , year =. Time series classification from scratch with deep neural networks:. doi:10.1109/IJCNN.2017.7966039 , booktitle =

  13. [21]

    arXiv preprint arXiv:2603.27792 , doi=

    What-If Explanations Over Time: Counterfactuals for Time Series Classification , author=. arXiv preprint arXiv:2603.27792 , doi=

  14. [22]

    Explanation in artificial intelligence:

    Miller, Tim , year =. Explanation in artificial intelligence:. doi:10.1016/j.artint.2018.07.007 , journal =

  15. [23]

    Shang, Ruoxi and Feng, K. J. Kevin and Shah, Chirag , year =. Why. doi:10.1145/3531146.3533189 , booktitle =

  16. [24]

    Explainable

    Theissler, Andreas and Spinnato, Francesco and Schlegel, Udo and Guidotti, Riccardo , year =. Explainable. doi:10.1109/ACCESS.2022.3207765 , journal =

  17. [25]

    , year =

    Geirhos, Robert and Jacobsen, Jörn-Henrik and Michaelis, Claudio and Zemel, Richard and Brendel, Wieland and Bethge, Matthias and Wichmann, Felix A. , year =. Shortcut learning in deep neural networks , volume =. Nature Machine Intelligence , publisher =. doi:10.1038/s42256-02...

  18. [26]

    Adadi, Amina and Berrada, Mohammed , year =. Peeking. doi:10.1109/ACCESS.2018.2870052 , journal =

  19. [27]

    Multimedia Tools and Applications , pages=

    Pygad: An intuitive genetic algorithm python library , author=. Multimedia Tools and Applications , pages=. 2023 , publisher=

  20. [28]

    Proceedings of the AAAI Conference on Artificial Intelligence , author =

    Counterfactual. Proceedings of the AAAI Conference on Artificial Intelligence , author =. 2026 , pages =. doi:10.1609/aaai.v40i21.38792 , number =

  21. [29]

    Data Mining and Knowledge Discovery , author =

    Counterfactual explanations and how to find them: literature review and benchmarking , volume =. Data Mining and Knowledge Discovery , author =. 2024 , pages =. doi:10.1007/s10618-022-00831-6 , number =

  22. [30]

    Journal of Intelligent Information Systems , author =

    Gaining insight through case-based explanation , volume =. Journal of Intelligent Information Systems , author =. 2009 , pages =. doi:10.1007/s10844-008-0069-0 , number =

  23. [31]

    Yeh, Chih-Kuan and Kim, Been and Arik, Sercan and Li, Chun-Liang and Pfister, Tomas and Ravikumar, Pradeep , year =. On. Advances in

  24. [32]

    Journal of Open Source Software , volume=

    H. Journal of Open Source Software , volume=

  25. [33]

    1989 , publisher=

    Heil, Christopher E and Walnut, David F , journal=. 1989 , publisher=

  26. [34]

    arXiv preprint arXiv:1811.00075 , author =

    The. arXiv preprint arXiv:1811.00075 , author =. 2018 , doi =

  27. [35]

    IEEE/CAA Journal of Automatica Sinica , author =

    The. IEEE/CAA Journal of Automatica Sinica , author =. 2019 , pages =. doi:10.1109/JAS.2019.1911747 , number =

  28. [36]

    1994 , publisher=

    Donoho, David L and Johnstone, Iain M , journal=. 1994 , publisher=

  29. [37]

    2019 , publisher=

    Lee, Gregory and Gommers, Ralf and Waselewski, Filip and Wohlfahrt, Kai and O'Leary, Aaron , journal=. 2019 , publisher=

  30. [38]

    Jutte, Annemarie and Ahmed, Faizan and Linssen, Jeroen and Keulen, Maurice van , year =. C-. doi:10.48550/arXiv.2504.11159 , publisher =

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.