REVIEW 3 major objections 4 minor 38 references
ConceptCF: Concept-based Counterfactuals for the Explainability of Time Series
T0 review · 3 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read ConceptCF generates time-series counterfactuals by mutating high-level concepts rather than raw points, with validity guaranteed and top-tier scores on standard metrics.
desk verdict A promising concept-level counterfactual framework for time series, with a real but fixable evaluation flaw in the sparsity metric. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The concept set Sx = d(x) with an invertible construction d and reconstruction h. Three decompositions instantiate it: Fourier frequency bands (DFT), wavelet components (DWT), and a human-centered decomposition into trend, bias, scale, low-frequency, variance, and high-frequency components. The genetic algorithm mutates these concepts (scalars by Gaussian noise clipped to observed ranges; sequences by KD-tree-based replacement from training data), with the nearest-unlike-neighbour seeding and elitism guaranteeing validity, while sparsity is encouraged by a preset probability that returns concepts to their original values.
What would settle it
Give users original/counterfactual pairs from ConceptCF and ask them to name which concept changed; if identification is at chance level, the interpretability promise fails. Alternatively, train a classifier that provably depends only on, say, the high-frequency component and check whether flips are always accompanied by a change in that component.
Extended reading notes
Core claim
ConceptCF's central claim is that restricting counterfactual search to an invertible concept space—where concepts are scalars or sequences with semantic meaning—preserves the explanatory power of the mutation while remaining competitive with raw-signal methods. The paper proves (Proposition 1) that, because the nearest unlike neighbour is seeded into the initial population and elitism preserves it, every generation's best solution satisfies the validity constraint. Empirically, the results show ConceptCF at rank 1 for proximity and sparsity and rank 2 for confidence and plausibility across the five datasets, with validity always satisfied.
Load-bearing premise
The concept decomposition must capture properties that are both causally used by the classifier and intuitively meaningful to a human; if the model relies on signal content the concepts do not expose, the generated counterfactuals can be valid and sparse yet explain nothing real.
Editorial extensions
If this is right
- If correct, counterfactual explanations for time series can be stated in terms of human-understandable properties (e.g., 'scale decreased') rather than opaque point changes.
- The validity guarantee via nearest-unlike-neighbour seeding and elitism means users never receive an explanation that fails to flip the model's prediction, regardless of the optimizer's exploration.
- Sparse counterfactuals (few concepts changed) are achievable without sacrificing validity, which point-based optimization methods cannot do.
- The decomposition choice (Fourier, wavelet, human-centered) has limited effect on quantitative metrics, so the choice can be driven by end-user needs.
- The same framework extends to targeted counterfactuals toward a specified class, widening its use to 'what would make the model predict class Y'.
Reading between the lines
- The paper's interpretability is asserted via concept design, not measured: a user study asking people to identify the changed concept from paired signals would directly test this claim.
- Because metric-optimality is measured in point space while mutations occur in concept space, a counterfactual could be pointwise close yet semantically odd if the decomposition glosses over nonstationary behaviour; testing on nonstationary or noise-dominated series would stress this.
- The invertibility requirement suggests a natural extension to other invertible representations (e.g., autocorrelation or spectral envelopes), which could be combined with the proposed genetic search without changing the algorithm.
- If the classifier relies on high-frequency detail that the human-centered decomposition absorbs into the high-frequency component, the explanation 'scale changed' may be incomplete; targeted ablation of each concept's causal effect on the prediction would quantify which concepts actually drive flips.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces ConceptCF, a counterfactual explanation method for time series that operates on high-level 'concepts' (e.g., trend, scale, frequency bands) rather than on individual points or subsequences. The method extracts concepts via one of three invertible decompositions (DFT, DWT, human-centered), then uses a genetic algorithm to perturb the concept set subject to a validity constraint. A formal guarantee (Proposition 1) is provided: under a sufficiently large penalty and with NUN initialization and elitism, all generated counterfactuals are valid. The paper evaluates ConceptCF against five baselines on five datasets, reporting validity, confidence, proximity, sparsity, and plausibility, and claims top-tier performance, especially on sparsity.
Significance. The central idea—framing counterfactual generation as an optimization over interpretable concepts—is a meaningful contribution to time-series XAI. The formalization in Section 3 and the validity guarantee in Proposition 1 are clear and useful. The evaluation is broad: five baselines, five datasets, and a sensitivity analysis of the objective weights, which is commendable. However, the empirical claims are currently undermined by a non-comparable sparsity metric, a plausibility metric that is also an optimization objective, and an untested interpretability assertion. If these issues are addressed, the paper could be a solid contribution to the field.
major comments (3)
- [Section 5.6, Eq. (2), Eq. (3)] The sparsity metric is defined differently for different methods: for ConceptCF it counts the fraction of concepts changed (out of 6), for Native Guide and Sub-SpaCE the fraction of subsequences, and for Wachter, TSEvo, and Glacier the fraction of points. These units are not commensurable. A single concept mutation (e.g., scaling) can alter every time point of the reconstructed signal, yet it is counted as 1/6 of a unit. Consequently, the reported sparsity values (e.g., 0.288 for ConceptCF on MS-HAR vs. 1.000 for Wachter) are not like-for-like, and the statement in Section 6.2 that 'ConceptCF has the best sparsity across all datasets' is unsupported. Please report sparsity in a common unit (e.g., fraction of time points changed) or report both concept-level and point-level sparsity, and adjust the claims accordingly.
- [Section 5.6, Eq. (2), Eq. (3)] The plausibility metric used in the evaluation (IOS, Eq. (2)) is the same function that appears as the O_pl objective in the fitness function (Eq. (3)), and the autoencoder f_AE used in the metric is the same one used inside the optimization. Thus ConceptCF is directly optimizing the evaluation metric. This creates a circularity that inflates its plausibility scores relative to baselines that do not optimize the same objective. Please evaluate plausibility using an independent measure not used in the optimization, such as distance to the nearest training sample, density in a separately trained latent space, or likelihood under a probabilistic generative model.
- [Section 1, Section 7] The paper's central motivation is that the generated counterfactuals are 'human-interpretable' and 'meaningful' because they operate on concepts. However, the paper explicitly defers user evaluation to future work (Section 7), and the human-centered decomposition is taken from prior work without evidence that these concepts align with either end-user understanding or the features actually used by the classifier. Without such evidence, the explainability claim is asserted rather than demonstrated. Please add a user study or a task-based proxy (e.g., measuring whether users can predict the model's decisions from the concept explanations), or temper the claims from 'human-interpretable' to 'concept-level' throughout, including the title and abstract.
minor comments (4)
- [Section 4.1] The human-centered decomposition is only briefly described and the reader is referred to Jutte et al. (2025). Since this decomposition is central to one of the three concept variants, please provide a more self-contained summary, including the definitions of cLF, cVar, and cHF and any assumptions about their mutual independence.
- [Section 5.4] The hyperparameters of the genetic algorithm and the objective weights are said to be 'chosen based on initial experimentation' with no further detail. Please report the search range or the reasoning behind the chosen values to improve reproducibility.
- [Section 3.2, Eq. (2)] The definition of e(x, x_CF) with min(0, ...) means that if the counterfactual has lower reconstruction error than the original, the plausibility term becomes negative. The paper does not explain how negative plausibility values are interpreted or why the min is applied. A brief justification would help.
- [Section 6.2] Table 2 reports global ranks based on only five datasets; the authors note this should be interpreted cautiously. This is appreciated, but the large number of ties (e.g., rank 1 shared by several methods on proximity) makes the ranking difficult to interpret. Consider also reporting effect sizes or confidence intervals.
Circularity Check
Top-tier sparsity and confidence/plausibility scores largely reduce to ConceptCF's own optimized objective functions and to a self-defined sparsity unit.
-
fitted input called prediction
[Section 3.2 (Eq. 1) and Section 5.6 (Metrics)]
"For the optimization, ... we formulate the problem in terms of four objectives: confidence, proximity, sparsity and plausibility. ... For the metrics we follow Refoyo and Luengo [2024] and consider metrics in line with the previously defined objectives for the optimization problem."
The evaluation metrics named in the abstract are, for confidence, sparsity and plausibility, exactly the objective terms Oconf, Os and Opl from Eq. (3) that the genetic algorithm maximizes. Section 5.6 confirms that the metrics are 'in line with' the objectives. Consequently, ConceptCF's reported 'top-tier performance' on these metrics is a restatement of its own fitness components rather than an independent test. The sparsity metric is especially direct: Os is the fraction of concepts changed, and the reported sparsity is exactly that optimized fraction.
-
self definitional
[Section 5.6, Sparsity definition]
"Sparsity: The fraction of 'meaningful' units changed from the original sample to the counterfactual. For Native Guide and Sub-SpaCE we consider the number of subsequences changed. For ConceptCF, we consider the number of concepts. For the other approaches, we consider the number of points."
The sparsity claim 'ConceptCF has the best sparsity across all datasets' is produced by counting different units for ConceptCF than for the baselines. ConceptCF has only m=6 concepts, so one concept mutation such as a scale change can alter every timestamp yet counts as 1/6. Point-based methods count every changed point, yielding values near 1.0. Thus the rank-1 sparsity advantage is an artifact of the metric's unit choice, not an independently measured property.
full rationale
The paper does not claim a first-principles derivation; its central contribution is an empirical method plus benchmark. However, the benchmark's strongest claims are partially circular. The GA in Eq. (3) optimizes confidence, sparsity, and plausibility, and Section 5.6 explicitly adopts metrics 'in line with' those objectives, so the reported top-tier scores on those metrics largely restate the optimization target. The sparsity comparison is further compromised by the self-defined counting unit: ConceptCF is scored on concepts (6 per signal), while other methods are scored on points or subsequences. This makes the headline sparsity advantage a definitional artifact rather than a fair comparison. The human-centered decomposition is imported from the authors' own prior work (Jutte et al., 2025), but it is an input to the method rather than a claim reduced to that citation; user interpretability is explicitly deferred, so that self-citation is not load-bearing for the measured metrics. Proposition 1 is a definitional validity guarantee via the nearest unlike neighbour and is not a prediction. Overall, the empirical advantage is partially forced by construction, but the method still has independent content: the concept-based formulation, the three decompositions, and the proximity results are not purely self-referential.
Assumptions & free parameters
free parameters (4)
- Fitness weights alpha_conf=0.4, alpha_prox=0.6, alpha_s=0.4, alpha_pl=0.2, beta=100 =
alpha_conf=0.4, alpha_prox=0.6, alpha_s=0.4, alpha_pl=0.2, beta=100
- GA hyperparameters (ngen=100, nindiv=50, nelite=2, nsaturate=15, nparents=10, pmutation=0.2, preset=0.05, nNN=50) =
ngen=100, nindiv=50, nelite=2, nsaturate=15, nparents=10, pmutation=0.2, preset=0.05, nNN=50
- DWT wavelet (Daubechies 6) and number of components (ndetail=3) =
Daubechies 6, ndetail=3
- Human-centered sliding window size (27) =
27
assumptions (5)
- domain assumption Concept construction d(·) is invertible via h(·) so that x = h(d(x))
- domain assumption For every test sample x, a nearest unlike neighbour xNUN exists in the training set
- ad hoc to paper The penalty beta can be set large enough that validity-violating solutions always have lower fitness than the NUN
- domain assumption The human-centered decomposition (Jutte et al. [2025]) produces concepts that are interpretable to end users and relevant to the classifier's decisions
- domain assumption Autoencoder reconstruction error is a valid proxy for data plausibility
Cite this review
Pith. "Pith review of ConceptCF: Concept-based Counterfactuals for the Explainability of Time Series." pith.science (2026). https://pith.science/paper/FGQ2GSFC
@misc{pith2026260718748,
author = {Pith},
title = {Pith review of: ConceptCF: Concept-based Counterfactuals for the Explainability of Time Series},
year = {2026},
howpublished = {\url{https://pith.science/paper/FGQ2GSFC}},
note = {Machine review of arXiv:2607.18748}
}
read the original abstract
This paper proposes ConceptCF, a method for counterfactual generation that operates on human-interpretable concepts. In high-stakes domains such as healthcare and predictive maintenance, artificial intelligence models can increase efficiency and safety. Explainability is key to ensure these models rely on causal relationships rather than spurious correlations. Counterfactual explanations identify minimal modifications that would change a model's predictions. Existing methods for time series operate on individual points or subsequences without ensuring interpretability of the mutations. ConceptCF instead modifies meaningful concepts. As a result we can provide explanations in terms of these concepts, for example ``the model's prediction would be `Sit' instead of `Walk' if you increase the scale of the movement''. In this paper, the concepts are constructed through time series decomposition, resulting in concepts such as scale, and frequency bands. Counterfactuals are generated using a genetic algorithm that optimizes the concept mutations. Evaluation against five state-of-the-art approaches demonstrates that ConceptCF consistently achieves top-tier performance across validity, confidence, proximity, sparsity and plausibility metrics.
Figures
Reference graph
Works this paper leans on
-
[1]
Wachter, Sandra and Mittelstadt, Brent and Russell, Chris , year =. Counterfactual. doi:10.2139/ssrn.3063289 , journal =
-
[2]
Glacier: guided locally constrained counterfactual explanations for time series classification , volume =. Machine Learning , author =. 2024 , pages =. doi:10.1007/s10994-023-06502-x , number =
-
[3]
Dandl, Susanne and Molnar, Christoph and Binder, Martin and Bischl, Bernd , year =. Multi-. doi:10.1007/978-3-030-58112-1_31 , booktitle =
-
[4]
doi:10.1109/ICMLA55696.2022.00013 , booktitle =
Höllig, Jacqueline and Kulbach, Cedric and Thoma, Steffen , year =. doi:10.1109/ICMLA55696.2022.00013 , booktitle =
arXiv 2022
-
[5]
Poeta, Eleonora and Ciravegna, Gabriele and Pastor, Eliana and Cerquitelli, Tania and Baralis, Elena , journal=
-
[6]
ACM Computing Surveys , author =
Counterfactual. ACM Computing Surveys , author =. 2024 , pages =. doi:10.1145/3677119 , number =
doi:10.1145/3677119 2024
-
[7]
Knowledge and Information Systems , author =
Locally and globally explainable time series tweaking , volume =. Knowledge and Information Systems , author =. 2020 , pages =. doi:10.1007/s10115-019-01389-4 , number =
-
[8]
Delaney, Eoin and Greene, Derek and Keane, Mark T. , year =. Instance-. doi:10.1007/978-3-030-86957-1_3 , booktitle =
Show all 38 references
-
[9]
and Coskun, Ayse K
Ates, Emre and Aksar, Burak and Leung, Vitus J. and Coskun, Ayse K. , year =. Counterfactual. doi:10.1109/ICAPAI49758.2021.9462056 , booktitle =
2021
-
[10]
Learning
Wang, Zhendong and Samsten, Isak and Mochaourab, Rami and Papapetrou, Panagiotis , year =. Learning. doi:10.1007/978-3-030-88942-5_29 , booktitle =
-
[11]
Refoyo, Mario and Luengo, David , year =. Sub-. doi:10.1007/978-3-031-63800-8_1 , booktitle =
-
[12]
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies , author =
X-. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies , author =. 2023 , pages =. doi:10.1145/3580804 , number =
2023 doi
-
[13]
Interpretability
Kim, Been and Wattenberg, Martin and Gilmer, Justin and Cai, Carrie and Wexler, James and Viegas, Fernanda and Sayres, Rory , year =. Interpretability. Proceedings of the 35th International Conference on Machine Learning , publisher =
-
[14]
Koh, Pang Wei and Nguyen, Thao and Tang, Yew Siang and Mussmann, Stephen and Pierson, Emma and Kim, Been and Liang, Percy , year =. Concept. Proceedings of the 37th
-
[15]
arXiv preprint arXiv:1907.07165 , author =
Explaining. arXiv preprint arXiv:1907.07165 , author =. 2020 , doi =
1907 arXiv
-
[16]
Proceedings of the AAAI Conference on Artificial Intelligence , author =
Deep. Proceedings of the AAAI Conference on Artificial Intelligence , author =. doi:10.1609/aaai.v32i1.11771 , number =
-
[17]
Chen, Chaofan and Li, Oscar and Tao, Daniel and Barnett, Alina and Rudin, Cynthia and Su, Jonathan K , year =. This. Advances in
- [18]
-
[19]
Comparative review of selection techniques in genetic algorithm , doi =
Shukla, Anupriya and Pandey, Hari Mohan and Mehrotra, Deepti , year =. Comparative review of selection techniques in genetic algorithm , doi =. 2015
2015
-
[20]
Time series classification from scratch with deep neural networks:
Wang, Zhiguang and Yan, Weizhong and Oates, Tim , year =. Time series classification from scratch with deep neural networks:. doi:10.1109/IJCNN.2017.7966039 , booktitle =
2017
-
[21]
arXiv preprint arXiv:2603.27792 , doi=
What-If Explanations Over Time: Counterfactuals for Time Series Classification , author=. arXiv preprint arXiv:2603.27792 , doi=
-
[22]
Explanation in artificial intelligence:
Miller, Tim , year =. Explanation in artificial intelligence:. doi:10.1016/j.artint.2018.07.007 , journal =
2018 doi
-
[23]
Shang, Ruoxi and Feng, K. J. Kevin and Shah, Chirag , year =. Why. doi:10.1145/3531146.3533189 , booktitle =
-
[24]
Explainable
Theissler, Andreas and Spinnato, Francesco and Schlegel, Udo and Guidotti, Riccardo , year =. Explainable. doi:10.1109/ACCESS.2022.3207765 , journal =
2022
-
[25]
, year =
Geirhos, Robert and Jacobsen, Jörn-Henrik and Michaelis, Claudio and Zemel, Richard and Brendel, Wieland and Bethge, Matthias and Wichmann, Felix A. , year =. Shortcut learning in deep neural networks , volume =. Nature Machine Intelligence , publisher =. doi:10.1038/s42256-02...
-
[26]
Adadi, Amina and Berrada, Mohammed , year =. Peeking. doi:10.1109/ACCESS.2018.2870052 , journal =
2018
-
[27]
Multimedia Tools and Applications , pages=
Pygad: An intuitive genetic algorithm python library , author=. Multimedia Tools and Applications , pages=. 2023 , publisher=
2023
-
[28]
Proceedings of the AAAI Conference on Artificial Intelligence , author =
Counterfactual. Proceedings of the AAAI Conference on Artificial Intelligence , author =. 2026 , pages =. doi:10.1609/aaai.v40i21.38792 , number =
2026 doi
-
[29]
Data Mining and Knowledge Discovery , author =
Counterfactual explanations and how to find them: literature review and benchmarking , volume =. Data Mining and Knowledge Discovery , author =. 2024 , pages =. doi:10.1007/s10618-022-00831-6 , number =
2024 doi
-
[30]
Journal of Intelligent Information Systems , author =
Gaining insight through case-based explanation , volume =. Journal of Intelligent Information Systems , author =. 2009 , pages =. doi:10.1007/s10844-008-0069-0 , number =
2009 doi
-
[31]
Yeh, Chih-Kuan and Kim, Been and Arik, Sercan and Li, Chun-Liang and Pfister, Tomas and Ravikumar, Pradeep , year =. On. Advances in
-
[32]
Journal of Open Source Software , volume=
H. Journal of Open Source Software , volume=
-
[33]
1989 , publisher=
Heil, Christopher E and Walnut, David F , journal=. 1989 , publisher=
1989
-
[34]
arXiv preprint arXiv:1811.00075 , author =
The. arXiv preprint arXiv:1811.00075 , author =. 2018 , doi =
2018 arXiv
-
[35]
IEEE/CAA Journal of Automatica Sinica , author =
The. IEEE/CAA Journal of Automatica Sinica , author =. 2019 , pages =. doi:10.1109/JAS.2019.1911747 , number =
2019
-
[36]
1994 , publisher=
Donoho, David L and Johnstone, Iain M , journal=. 1994 , publisher=
1994
-
[37]
2019 , publisher=
Lee, Gregory and Gommers, Ralf and Waselewski, Filip and Wohlfahrt, Kai and O'Leary, Aaron , journal=. 2019 , publisher=
2019
- [38]
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.