Pith. sign in

REVIEW 5 major objections 5 minor 26 references

drXAI claims that attribution scores from a fast explainer can reduce time series datasets by 80–90% while preserving or improving classification accuracy, letting heavy models like ConvTran run where they previously ran out of memory.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 22:21 UTC pith:TD3XPJWN

load-bearing objection Worth a serious referee, but the abstract overstates accuracy maintenance; several datasets show large drops. the 5 major comments →

arxiv 2607.15774 v2 pith:TD3XPJWN submitted 2026-07-17 cs.LG cs.AI

Scaling Time Series Classification via XAI-Driven Data Reduction

Davide Italo Serramazza , Thach Le Nguyen , Georgiana Ifrim This is my paper
classification cs.LG cs.AI
keywords drXAItime series classificationexplainable AIfeature attributionfeature selectiondata reductionchannel selectiontime point selection
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

drXAI repurposes explainable-AI attributions as a feature-selection signal for time series classification. The paper's central claim is that explaining a fast, linear-time classifier (Hydra) with lightweight attribution methods produces per-channel and per-time-point importance scores which, aggregated over samples and pruned with an automated elbow-cut, identify a small feature subset — typically 10–20% of the original data — that maintains or improves classifier accuracy. The practical payoff is that memory-hungry models such as ConvTran become trainable on datasets that previously caused out-of-memory errors. This matters because it turns XAI from an interpretability afterthought into a scaling tool, applicable to any explainer that outputs a relevance map, and it keeps data in the original feature space rather than projecting it away.

Core claim

On its own terms, the paper establishes that attribution values from a fast explainer can serve as a reliable proxy for globally informative features in time series. drXAI trains Hydra on the full data, explains a small per-class sample of the training set using Feature Ablation or Shapley-value sampling, aggregates attributions into two complementary global scores (signed average and mean absolute value), and selects the elbow-cut intersection of the two. Across synthetic and real-world multivariate (channel selection) and univariate (time-point selection) benchmarks, this yields 80–90% data reduction with accuracy matching or sometimes exceeding the full-data model. The headline result is

What carries the argument

The pipeline is a wrapper feature-selection method: (1) train Hydra, a fast GPU convolution-dictionary classifier, on the full data; (2) compute local attributions with Feature Ablation or Shapley-value sampling against a background set (Zeros or the proposed class-prototype average, 'Proto'); (3) aggregate attributions two complementary ways — aggr_avg averages signed attributions per feature to damp uncertain features, aggr_abs averages absolute values to catch the most active features; (4) apply an elbow-cut heuristic to each score vector and retain the intersection of the two selected sets. The intersection is designed to keep only features that are both magnitude- and sign-consistent, a

Load-bearing premise

The features that Hydra's attributions mark as important are also the features that ConvTran, InceptionTime, and MultiRocket-Hydra need for accurate predictions.

What would settle it

Compare drXAI-selected features against random features of the same size on a held-out set across many datasets; if random selection matches or beats drXAI on accuracy for the same reduction, the attribution signal is not the cause of the gains. A more targeted falsifier: find a dataset where Hydra's top-attributed channels are uninformative for a transformer, yet the transformer's accuracy collapses when those channels are removed.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Any TSC pipeline can be prefixed with a cheap selection stage, reducing training memory and time by roughly an order of magnitude while keeping accuracy.
  • Transformer-based classifiers become practical on long or multi-channel series that previously triggered out-of-memory errors, extending their reach to large real-world benchmarks.
  • Because the method is attribution-agnostic, improvements in XAI attribution quality translate directly into better feature selection, creating a closed loop between interpretability research and scalability.
  • Retaining original channels/time points (instead of projecting to latent space) means the selected features remain auditable by domain experts, which matters in medical and other high-stakes applications.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct test absent from the paper: compare selections from different explainer classifiers (e.g., train the explainer as InceptionTime or MRH instead of Hydra) to see whether the proxy assumption holds across model families; the paper only measures this indirectly via final accuracy.
  • The Proto background, a single average of class prototypes, could fail on classes with multimodal distributions; using per-class backgrounds or clusters of prototypes would be a natural extension and a likely source of stability gains.
  • The elbow-cut intersection rule assumes the two aggregation views agree; replacing the heuristic with a model-based size selector (e.g., validation accuracy on the reduced set) would add robustness at the cost of extra computation.
  • The accuracy drops observed on a few datasets (e.g., MosquitoSound with InceptionTime, 0.930 to 0.819) indicate that the method's guarantee is dataset-dependent; characterizing when the proxy breaks would make the trade-off actionable for practitioners.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces drXAI, a two-stage feature-selection wrapper for time series classification. In the first stage, a fast Hydra classifier is trained and local attributions are computed on a per-class sample subset using either Feature Ablation or Shapley Value Sampling, with either a zero background or a proposed class-prototype-average background (Proto). Attributions are aggregated over samples by averaging and by averaging absolute values, then an elbow-cut heuristic selects a feature set as the intersection of the two rankings. This selected set is used to reduce channels (MTSC) or time points (UTSC) before training SOTA classifiers (ConvTran, InceptionTime, MultiRocket-Hydra). The paper reports synthetic experiments showing good recovery of ground-truth informative features and real-world experiments claiming 80-90% data reduction with accuracy comparable to full-data training, including cases where ConvTran otherwise cannot run.

Significance. If the empirical claims were fully supported, drXAI would be a valuable practical contribution: it offers a way to use a cheap explainable model to reduce input dimensionality for expensive TSC models, and the proposed Proto background is a simple, plausible improvement over zeros. The synthetic recovery results are encouraging, and the open-source release is a concrete asset. However, as submitted the paper overstates its central trade-off. Several accuracy differences in the appendix are large and in the wrong direction, the reported reduction percentages vary widely across datasets and configurations, and the missing uncertainty quantification and underspecified elbow-cut make the claimed consistency difficult to evaluate. With a revised presentation that quantifies failure cases, adds significance testing, and specifies the selection heuristic, the contribution could meet the bar.

major comments (5)
  1. [§4.3, Appendix Table 4, Conclusion] The headline claim that drXAI 'maintains classification accuracy' is not supported by the reported results. For MosquitoSound, InceptionTime drops from .930 (all features) to .819 under FA-Proto and from .930 to .843 under SHAP-Proto; MRH drops from .885 to .771. For UrbanSound, InceptionTime drops from .729 to .627 (FA-Proto). For WhaleSounds, ConvTran drops from .726 to .666. These are substantial losses, not 'comparable.' Please present the full distribution of accuracy deltas, report the proportion of dataset/classifier pairs on which the reduced-data accuracy is within a tolerance (e.g., ±0.01) of the full-data accuracy, and revise the abstract and conclusion accordingly.
  2. [Abstract, Appendix Table 5] The data-reduction claim that drXAI 'achieves between 80% and 90% data reduction' is inconsistent with Appendix Table 5. For RightWhaleCalls, FA-Proto selects 40% of time points (60% reduction) and SHAP-Proto selects 15% (85%); for Cornell FA-Proto selects 30% (70%). The conclusion 'over 80-90%' is thus an overstatement. Please report reduction per configuration and qualify the claim to reflect the observed range (roughly 60-95% depending on dataset and explainer).
  3. [§3.3, Algorithm 1] The elbow-cut step is load-bearing but unspecified. The text gives no formula, stopping rule, or implementation detail for how the elbow is located, how it maps to a number of selected features, or how ties/edge cases are handled. Because the paper emphasizes 'automated ... without requiring manual thresholds,' this omission prevents replication. Provide a precise description or a reference to the code, and include a brief validation that the heuristic is stable across datasets.
  4. [§4, evaluation protocol] Only 3 runs are used and no error bars or significance tests are reported. Given the observed variance in deep classifiers, the comparison 'maintaining accuracy' is not statistically grounded. Report per-run values, confidence intervals, and a paired test (e.g., Wilcoxon signed-rank) across the dataset/classifier matrix. This is especially important because the synthetic UTSC results include both large gains (InceptionTime .508 to .813 under FA-Proto) and large losses (MRH .961 to .898), suggesting the method's effect is heterogeneous.
  5. [§2.2 / Algorithm 1] The selection is entirely determined by attributions of Hydra, and the transfer to other classifiers is only indirectly tested. The paper should state this proxy assumption explicitly in the main text and discuss the conditions under which it may fail (as in the MosquitoSound/InceptionTime case). At minimum, add a limitation paragraph and avoid language in §3.3 that implies the selected features are universally 'essential.'
minor comments (5)
  1. [Appendix Table 1, §4.2 Table 1] In the synthetic MTSC dataset, the row 'n. uninformative 200' appears to be a typo: there are only 20 uninformative channels. The column alignment of Table 1 is also confusing; please reformat.
  2. [§4.3] The statement 'Since UTSC datasets have more samples than the MTSC ones' is not universally true (UrbanSound has only 2,717 training samples, comparable to MTSC Military Press with 1,426). Please rephrase.
  3. [§3.3, §4.3] The number of windows (20) and the explanation sample size n_samp (50 or 100) are manually chosen. Since the paper claims to avoid manual thresholds, include a sensitivity analysis or at least justify these choices.
  4. [Figure 6] It is unclear whether the times shown are per-run or means over the 3 runs. Define this in the caption.
  5. [§4.4, Table 7] The statement 'total time is reduced by one order of magnitude' appears to be based on selected datasets/classifiers; for MRH on WhaleSounds the ratio is about 0.79, not an order of magnitude. Please qualify the claim.

Circularity Check

0 steps flagged

No material circularity: the feature-selection pipeline is evaluated by held-out accuracy, not derived from it; self-citations are implementation and baseline provenance, not load-bearing.

full rationale

The paper's derivation chain is: train Hydra on the training set, compute local attributions on a sampled explanation set, aggregate attributions in two ways, apply an elbow cut, intersect the two selected sets, then retrain SOTA classifiers on the reduced data and evaluate on a held-out test set. None of these steps fits a parameter to the accuracy that is later reported as a prediction; the reported accuracies are measured after retraining and are not defined in terms of the attributions or the selection heuristic. The final selected set is not constructed to match any target classifier's accuracy, and the paper does not invoke a uniqueness theorem or a forced ansatz from prior work to define its result. The self-citations (refs 14, 19, 20, 21) supply implementations, synthetic-data generators, and baselines from the same group, but these are concrete code/data artifacts and independent enough to serve as provenance; they do not by themselves make the central claim true. The paper's genuine weakness is empirical: the proxy assumption that Hydra-attribution-important features are also important to ConvTran, InceptionTime, and MRH fails in several Appendix results (e.g., MosquitoSound InceptionTime drops from .930 to .819). That is a correctness/robustness concern, not a circularity, because the evaluation is still against held-out labels. Therefore the derivation is self-contained with respect to circularity, and the score is 0.

Axiom & Free-Parameter Ledger

2 free parameters · 3 axioms · 1 invented entities

The method's validity depends on three assumptions: Hydra attributions transfer to other classifiers; the intersection of the two aggregations isolates essential features; and background-substitution (zeros or Proto) measures feature importance. The Proto background and the elbow cut are introduced ad hoc; neither is derived from first principles.

free parameters (2)
  • n_samp (explanation samples per class) = 50 (MTSC), 100 (UTSC)
    Set empirically per data type in Sections 4.2/4.3; no sensitivity analysis is provided, and the selected subset can change with this value.
  • Number of windows for time-point grouping = 20
    Section 4.3 groups time points into 20 equal-length windows for attribution; this choice affects the granularity of selection and is not justified or varied.
axioms (3)
  • domain assumption Features that are attribution-important to Hydra (the explainer classifier) are also important to the target classifiers (ConvTran, InceptionTime, MultiRocket-Hydra).
    Algorithm 1 trains Hydra and uses its attributions to select features; the paper validates only end accuracy, not transferability. Some results contradict this (e.g., MosquitoSound InceptionTime accuracy drops .930→.819 with FA-Proto, Appendix Table 4).
  • ad hoc to paper Intersecting the aggr_avg and aggr_abs selected sets retains the essential features.
    Section 3.3 states 'this set contains only features that are both magnitude- and sign-wise important,' but this is asserted; no evidence isolates each aggregation's contribution.
  • domain assumption Replacing a feature's values with a background sample (zeros or Proto) produces meaningful feature-importance estimates.
    Both FA and SVS define attribution as the change in output under background substitution (Section 2.3); this is the standard missingness simulation assumption, adopted without new justification.
invented entities (1)
  • Proto background (average of class prototypes) no independent evidence
    purpose: Single-sample background set for attribution methods to simulate data missingness.
    Proposed in Section 3.1; the paper shows it outperforms zeros on some datasets but not others (e.g., on MTSC real-world, FA-Proto is worse than FA-zeros, Section 4.2), and only internal benchmark evidence is provided.

pith-pipeline@v1.3.0-alltime-deepseek · 13427 in / 12624 out tokens · 94754 ms · 2026-08-01T22:21:54.163706+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Scaling Time Series Classification via XAI-Driven Data Reduction." pith.science (2026). https://pith.science/paper/TD3XPJWN

@misc{pith2026260715774,
  author       = {Pith},
  title        = {Pith review of: Scaling Time Series Classification via XAI-Driven Data Reduction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TD3XPJWN}},
  note         = {Machine review of arXiv:2607.15774}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Explainable AI (XAI) for time series has seen significant algorithmic growth, but its utility in providing measurable performance gains for downstream tasks remains under-explored. This paper bridges this gap by introducing drXAI, a novel methodology that repurposes XAI attribution methods for effective data reduction in Time Series Classification (TSC). The core challenge in modern TSC is scalability; state-of-the-art models, such as Transformers, exhibit quadratic complexity relative to sequence length and linear complexity relative to the number of channels. This renders them computationally prohibitive for massive datasets. drXAI addresses this by using a fast, GPU-accelerated classifier (Hydra) to generate local attributions. We aggregate these into global feature importance scores and employ an automated elbow-cut heuristic to select the most salient features without requiring manual thresholds. We evaluate our approach on both synthetic and real-world univariate and multivariate datasets. On synthetic benchmarks, drXAI successfully recovers ground-truth features where traditional baselines fail. On real-world data, drXAI achieves between 80% and 90% data reduction while maintaining classification accuracy comparable to models trained on the full dataset. Most importantly, we show that drXAI allows resource-intensive models like ConvTran to scale to datasets that were previously inaccessible due to memory constraints. Our results show the benefits of using XAI not just for interpretability, but as a robust tool for feature selection and scalability in time series analysis. All our code and data are openly available.

Figures

Figures reproduced from arXiv: 2607.15774 by Davide Italo Serramazza, Georgiana Ifrim, Thach Le Nguyen.

Figure 1
Figure 1. Figure 1: drXAI: Channel selection for MTSC using XAI scores computed from [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Mean accuracy of each selection (and All Features) for the 4 MTSC real [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Mean percentage of data saved by each selection for the 4 MTSC datasets. [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Mean accuracy of each selection (and All Features) for the 5 UTSC [PITH_FULL_IMAGE:figures/full_fig_p013_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Mean percentage of data saved of each selection for the 5 UTSC datasets. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Accuracy vs Time (minutes in log scale) for ConvTran on UTSC datasets. [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 3 linked inside Pith

  1. [1]

    Ashraf et al

    M. Ashraf et al. A survey on dimensionality reduction techniques for time-series data.IEEE Access, 11, 2023

    2023

  2. [2]

    A. J. Bagnall et al. The UEA multivariate time series classification archive, 2018. CoRR, abs/1811.00075, 2018

    Pith/arXiv arXiv 2018

  3. [3]

    T. M. Cox et al. Understanding the impacts of anthropogenic sound on beaked whales. 2006. 16 Serramazza et al

    2006

  4. [4]

    Dempster et al

    A. Dempster et al. Highly scalable time series classification for very large datasets. InInternational Workshop on Advanced Analytics and Learning on Temporal Data, pages 80–95. Springer, 2024

    2024

  5. [5]

    Dempster et al

    A. Dempster et al. Monster: Monash scalable time series evaluation repository, 2025

    2025

  6. [6]

    Dhariyal, T

    B. Dhariyal, T. Le Nguyen, and G. Ifrim. Scalable classifier-agnostic channel se- lection for multivariate time series classification.Data Mining and Knowledge Discovery, 37(2):1010–1054, 2023

    2023

  7. [7]

    N. M. Foumani et al. Improving position encoding of transformers for multivariate time series classification.Data mining and knowledge discovery, 38(1):22–48, 2024

    2024

  8. [8]

    Ismail-Fawaz et al

    A. Ismail-Fawaz et al. An approach to multiple comparison benchmark evalu- ations that is stable under manipulation of the comparate set.arXiv preprint arXiv:2305.11921, 2023

    Pith/arXiv arXiv 2023

  9. [9]

    Ismail Fawaz et al

    H. Ismail Fawaz et al. Inceptiontime: Finding alexnet for time series classification. Data mining and knowledge discovery, 34(6):1936–1962, 2020

    1936

  10. [10]

    Kokhlikyan et al

    N. Kokhlikyan et al. Captum: A unified and generic model interpretability library for pytorch.arXiv preprint arXiv:2009.07896, 2020

    Pith/arXiv arXiv 2009

  11. [11]

    Aunifiedapproachtointerpretingmodelpredictions

    S.M.LundbergandS.-I.Lee. Aunifiedapproachtointerpretingmodelpredictions. Advances in neural information processing systems, 30, 2017

    2017

  12. [12]

    Middlehurst, A

    M. Middlehurst, A. Ismail-Fawaz, A. Guillaume, C. Holder, D. Guijo-Rubio, G. Bu- latova, L. Tsaprounis, L. Mentel, M. Walter, P. Schäfer, and A. Bagnall. aeon: a pythontoolkitforlearningfromtimeseries.Journal of Machine Learning Research, 25(289):1–10, 2024

    2024

  13. [13]

    Middlehurst, P

    M. Middlehurst, P. Schäfer, and A. Bagnall. Bake off redux: a review and experi- mental evaluation of recent time series classification algorithms.Data Mining and Knowledge Discovery, 2024

    2024

  14. [14]

    T. L. Nguyen and G. Ifrim. Tshap: Fast and exact shap for explaining time series classification and regression. InJoint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 60–77. Springer, 2025

    2025

  15. [15]

    Nuyts et al

    L. Nuyts et al. Tselect: selecting relevant and non-redundant channels for mul- tivariate time series classification: L. nuyts et al.Data Mining and Knowledge Discovery, 39(6):76, 2025

    2025

  16. [16]

    Olivetti et al

    E. Olivetti et al. Meg decoding across subjects. In2014 international workshop on pattern recognition in neuroimaging, pages 1–4. IEEE, 2014

    2014

  17. [17]

    Pedregosa et al

    F. Pedregosa et al. Scikit-learn: Machine learning in Python. 12:2825–2830, 2011

    2011

  18. [18]

    Salamon et al

    J. Salamon et al. A dataset and taxonomy for urban sound research. InProceedings of the 22nd ACM international conference on Multimedia, pages 1041–1044, 2014

    2014

  19. [19]

    D. I. Serramazza et al. Improving the evaluation and actionability of explanation methods for multivariate time series classification. InJoint European Confer- ence on Machine Learning and Knowledge Discovery in Databases, pages 177–195. Springer, 2024

    2024

  20. [20]

    D. I. Serramazza et al. A short tutorial for multivariate time series explanation using tscaptum.Software Impacts, 22:100723, 2024

    2024

  21. [21]

    D. I. Serramazza, T. T. Nguyen, T. Le Nguyen, and G. Ifrim. Evaluating explana- tion methods for multivariate time series classification. InInternational Workshop on Advanced Analytics and Learning on Temporal Data, pages 159–175. Springer, 2023

    2023

  22. [22]

    Singh et al

    A. Singh et al. An examination of wearable sensors and video data capture for human exercise classification. InJoint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 312–329. Springer, 2023. XAI-Driven Data Reduction 17

    2023

  23. [23]

    C. W. Tan et al. Multirocket: multiple pooling operators and transformations for fast and effective time series classification: Cw tan.Data Mining and Knowledge Discovery, 36(5):1623–1646, 2022

    2022

  24. [24]

    Theissler et al

    A. Theissler et al. Explainable ai for time series classification: A review, taxonomy and research directions.IEEE Access, 10:100700–100724, 2022

    2022

  25. [25]

    Turbé, M

    H. Turbé, M. Bjelogrlic, C. Lovis, and G. Mengaldo. Evaluation of post-hoc in- terpretability methods in time-series classification.Nature Machine Intelligence, 5(3):250–260, 2023

    2023

  26. [26]

    Yousef et al

    I. Yousef et al. The arc loss dataset, Feb 2025. 1 Appendix drXAI Dataset Characteristics dataset name n. samples train n. samples test n. channels n. time points n. classes synthetic MTSC 5000 5000 40 1000 2 Arc Loss 2581 645 96 1101 2 FaceDetection 5890 3524 144 62 2 MP 1426 595 50 161 4 Rowing 1838 790 50 161 5 synthetic UTSC 1000 5000 1 20000 2 Cornel...

    2025