Pith. sign in

REVIEW 3 major objections 6 minor 14 references

Explainability signals — attention rollout and Grad-CAM — show that attention-guided replay improves accuracy on non-stationary groundwater forecasts and reveals how each model uses the past.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 09:13 UTC pith:YNYIE4MZ

load-bearing objection Exploratory XAI study overclaims the accuracy benefit; the qualitative observations are worth reading, but the central performance claim is unsupported by any reported number. the 3 major comments →

arxiv 2607.19382 v1 pith:YNYIE4MZ submitted 2026-07-01 cs.LG cs.AI

Challenges of Explainability in Continual Learning for Time Series Forecasting

Quentin Besnard (RFAI) , Emmanuel Doumard (BDTLN) , Nicolas Labroche (LIFAT , BDTLN) , Nicolas Ragot (RFAI) , Nicolas Ringuet (BDTLN) This is my paper
classification cs.LG cs.AI
keywords continual learningtime series forecastingexplainabilityattention rolloutGrad-CAMexperience replaynon-stationary datagroundwater level forecasting
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that explainability tools can open the black box of continual learning for time series forecasting. It studies three forecasting architectures adapted with replay and knowledge distillation, and shows that an attention-based sampling strategy improves predictive accuracy on real groundwater-level series with regime shifts. Using attention rollout and Grad-CAM, it finds that sampling decisions follow stable, model-agnostic patterns, while prediction attributions expose different inductive biases: transformer models concentrate on recent time steps, convolutional models spread attention more evenly. The authors position explainability not as a final solution but as a diagnostic lens for understanding why replay works and when it can mislead.

Core claim

The paper's central claim is that attention-based Experience Replay not only helps forecasting models adapt to non-stationary groundwater data, but does so in a way that is interpretable. Attention rollout scores used to select replay samples yield consistent patterns across PatchMixer and PatchTST, suggesting the sampler learns model-agnostic notions of informative temporal segments. In contrast, Grad-CAM attribution maps show strong architectural differences: PatchTST focuses predictions on recent time steps while PatchMixer distributes attribution broadly. The paper also shows that random sampling is opaque, maximum-loss sampling over-emphasizes errors, and minimum-loss sampling becomes r

What carries the argument

The explanatory engine is a pair of interpretability signals. Attention rollout, computed inside the attention-based replay sampler, assigns importance scores to temporal segments and directly drives which samples enter the replay buffer. Grad-CAM, applied to the final linear layer of the forecasting model, attributes each prediction back to input time steps. Together they connect the two levels of the system: why a sample is stored (sampling) and how the model uses stored patterns to forecast (attribution). The replay-plus-distillation continual learning setup (student trained from teacher on buffered samples) is the substrate on which these signals are measured.

Load-bearing premise

The analysis assumes the attention-rollout and Grad-CAM scores faithfully measure which input time steps are truly informative, and that the single piezometer case study represents the broader non-stationary setting.

What would settle it

Run the same continual-learning pipeline with replay samples chosen by the lowest attention-rollout scores instead of the highest; if predictive accuracy does not drop, attention is not the carrier of the reported gains. Likewise, mask the top Grad-CAM-attributed input steps and re-forecast: no error change would show the attributions are not causal.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If attention-guided replay indeed improves accuracy as reported, replay buffers for non-stationary forecasting can be built by scoring samples with attention rather than by loss or chance.
  • Grad-CAM's finding that PatchTST leans on recent observations and PatchMixer spreads attention predicts that each architecture will react differently to distribution shift; mixture or ensemble designs could compensate.
  • Stable, model-agnostic attention-rollout patterns imply an attention sampler trained with one forecasting architecture can likely be transferred to another without retraining.
  • The qualitative difference among sampling strategies suggests explainability should be an explicit criterion in replay-buffer design, not a post-hoc add-on.
  • Attribution maps could serve as early warnings of regime shifts in environmental monitoring if they change before forecast error jumps.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not run: counterfactual deletion of attention-selected time steps to verify that Grad-CAM and rollout importance are causal for forecast error; if deletion tests fail, the explanations are correlational.
  • If attention-rollout patterns are stable, they might encode physical recharge or low-water events; that hypothesis could be tested against piezometer metadata such as drought indices without new model training.
  • The attention sampler's stability hints at a reusable 'data worthiness' measure across stations; a single pretrained sampler could flag which monitoring stations or periods deserve more frequent adaptation, reducing compute in large sensor networks.
  • A testable extension: use the variance of Grad-CAM attributions over time as a drift detector and compare its lead time against forecasting error rises.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper investigates continual learning for long-term piezometric time-series forecasting under non-stationarity, comparing Experience Replay strategies (random, maximum-loss, minimum-loss, and attention-based) with PatchTST, PatchMixer, and DLinear. Explainability is used both to analyze model predictions (Grad-CAM) and to interpret replay-sample selection (attention rollout). The central claim is that attention-guided replay improves predictive accuracy during continual adaptation, while the qualitative analyses purport to show stable, model-agnostic sampling patterns and architecture-dependent temporal attribution. The manuscript is exploratory in tone, but it makes a direct empirical claim that is not currently supported by reported quantitative results.

Significance. If properly supported, the paper would provide a useful case study at the intersection of continual learning, time-series forecasting, and explainability for environmental monitoring. The qualitative observations about attention-based sampling and Grad-CAM attributions are plausible and could guide future work on interpretable replay selection. However, the main empirical claim—that attention-guided replay improves forecasting accuracy—rests on a single unquantified figure, and the explainability signals are used both as the mechanism and as the evidence for the mechanism's validity. The potential significance is therefore real but conditional on adding a rigorous quantitative evaluation and validating the attribution methods.

major comments (3)
  1. [§5, Figure 5; §6] The central accuracy claim is not supported by any reported number. The text states in §5 that 'the results confirm that attention-guided replay contributes to improved predictive accuracy under non-stationary conditions,' and §6 repeats this, but Figure 5 is the only evidence and its caption gives no metric, no axis description, and no error bars. No MSE/MAE/RMSE values, no comparison against random, max-loss, min-loss, or no-replay baselines, no multiple seeds, no confidence intervals, and no statistical test are provided. In addition, the attention strategy includes an extra sampling module with inner_loop=3 while the other strategies do not, so any observed gain would be confounded by additional computation/capacity. This claim currently cannot be verified or reproduced.
  2. [§5, Figures 8–9 and 11; §4] The argument for stable, model-agnostic attention-based sampling is partly circular. Attention rollout is computed from the same sampling module used to select replay samples, so observing that attention-based sampling emphasizes high-attention regions is true to a large extent by construction. The claim that the sampling mechanism 'learns stable, model-agnostic criteria' is drawn from only two transformer/convolutional models and is not validated against any alternative importance measure or counterfactual test. I would recommend comparing attention rollout scores against input perturbation or leave-one-out importance, or at minimum showing that the selected samples are more informative than random selection with matched computational budget.
  3. [§4, Grad-CAM applied to final linear layer; §5, Figures 6–7] The Grad-CAM attributions are load-bearing for the qualitative conclusions about architectural differences (e.g., convolutional models use uniform evidence, transformers focus on recent time steps), yet no faithfulness or sanity check is reported. For sequence-to-sequence forecasting with a final linear layer of dimension pred_len, it is not obvious how gradient attributions are mapped back to the input time axis. Without validation against input perturbation, occlusion, or known synthetic data, the attributions may reflect artifacts of the model architecture rather than true predictive evidence. At minimum, the exact formulation of the Grad-CAM computation for these models, including the reshaping/upsampling step, should be specified and the conclusions should be correspondingly tempered.
minor comments (6)
  1. [§4] The computation of attention rollout for PatchMixer is not described. Since PatchMixer uses convolution rather than a standard transformer, clarify whether attention rollout is applied only to attention layers or to some generalized mixing operation.
  2. [§5, Figure 5] The figure caption says 'comparison applied on piezometer 3' but does not specify what is being compared or what the plotted quantity is. Add axis labels, metric definition, and ideally numerical values for the test set.
  3. [§5, Figures 6–9] The notion of 'top 25% highest-weighted data points, aggregated over time' is ambiguous. Is the threshold computed per figure, per model, per sequence, or globally? A formal definition would improve reproducibility.
  4. [§4] The role of inner_loop=3 in the attention-based sampling module is never explained. Does it correspond to multiple sampling iterations, gradient steps, or something else? This matters for the confound raised above.
  5. [Throughout] There are minor language issues (e.g., 'Some example with XAI has already shown promising results', 'Künstliche' should be 'Künstliche Intelligenz'). These do not affect content but should be corrected for a clean final version.
  6. [§4] No code or data availability statement is provided, and the dataset link is only in a footnote. Given the paper's empirical nature, a reproducibility statement with code and exact data splits would be valuable.

Circularity Check

2 steps flagged

Attention rollout is both the replay-sampling criterion and the evidence for the paper's main sampling-behavior conclusions, making those conclusions partly true by construction.

specific steps
  1. self definitional [Section 5 (Analysis, Figure 11); sampling definition in Section 4]
    "In contrast, the attention-based sampling strategy (Figure 11) exhibits a more structured and heterogeneous selection pattern. By leveraging attention weights, this approach prioritizes samples that are deemed more influential by the model's internal mechanisms."

    Attention rollout is the selection criterion of this strategy: Section 4 states 'Attention rollout is used within the attention-based sampling strategy to record sample importance scores.' Therefore observing that the sampler produces a structured pattern and prioritizes samples with high attention-rollout importance is a restatement of the sampling rule, not an independent empirical discovery. The claimed link between sampling decisions and 'learned importance scores' holds by construction because those scores define the selection mechanism.

  2. self definitional [Section 5 (Analysis, Figures 8-9)]
    "The Attention Rollout results exhibit consistent selection patterns across PatchMixer and PatchTST. This consistency suggests that the attention-based sampling mechanism learns stable, model-agnostic criteria for identifying informative temporal segments, despite differences in the underlying forecasting architectures."

    Both configurations use the same attention-rollout scoring rule inside the attention-based sampling module, so consistency in the selected patterns is generated by the shared scoring function rather than independent confirmation that the mechanism has learned model-agnostic criteria. No external validation of sample informativeness is provided, so the 'stable, model-agnostic criteria' claim is at least partly true by construction.

full rationale

The paper contains one substantive circularity: attention rollout is used as the selection score for the attention-based replay strategy and then the same signal is presented as the observed 'sampling behavior' and as evidence of stable, model-agnostic sampling criteria. That part of the analysis is self-definitional. The Grad-CAM comparisons (Figures 6-7) are independent and are not circular. The headline accuracy claim ('attention-guided replay contributes to improved predictive accuracy') is not circular, but it is unverifiable as reported: Section 5 gives no error metric, statistical test, seed variance, or quantitative baseline comparison. That is a correctness/evidence problem, not a circularity. No self-citation chain or imported uniqueness theorem appears; references are external. Score 6 reflects partial circularity in the explainability-of-sampling conclusions, while the accuracy claim retains independent (if unquantified) content.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

The paper relies heavily on manually chosen hyperparameters and on the unverified validity of attention rollout and Grad-CAM as explanation signals. No new physical or mathematical entities are introduced.

free parameters (6)
  • seq_len = 180
    Input window length selected without a sensitivity analysis; directly affects the context available to attention and attribution.
  • pred_len = 360
    Forecast horizon chosen as approximately one year; impacts task difficulty and the meaning of attribution.
  • buffer_size = 1500
    Replay buffer capacity fixed without justification or ablation; controls how much past data is retained.
  • freq = 300
    Adaptation frequency (number of steps between updates) is set arbitrarily and is not varied.
  • inner_loop = 3
    Inner-loop parameter of the attention-based sampling module; no explanation or sensitivity check is provided.
  • top_25_percent_threshold = 25%
    Threshold used to define 'focus points' in Grad-CAM and attention rollout visualizations; arbitrary and may affect all qualitative conclusions.
axioms (4)
  • domain assumption The two selected piezometric stations exhibit distribution shifts and regime changes that make continual learning necessary.
    Motivates the setup; supported only by visual inspection of Figures 1–4, not by statistical change-point tests.
  • ad hoc to paper Attention rollout scores from the sampling module are a valid measure of sample informativeness for replay.
    The sampling strategy is built to select high-attention samples, and the analysis then treats the same scores as evidence of meaningful selection (Sections 4–5).
  • ad hoc to paper Grad-CAM applied to the final linear layer faithfully attributes model predictions to input time steps.
    No verification against ground-truth importance or ablation is provided (Section 4).
  • domain assumption Knowledge distillation from teacher to student on replay-buffer samples adequately preserves previously learned knowledge.
    This is a standard continual-learning assumption, but it is not benchmarked against alternative consolidation strategies (Section 4).

pith-pipeline@v1.3.0-alltime-deepseek · 6102 in / 10013 out tokens · 93470 ms · 2026-08-02T09:13:08.328882+00:00 · methodology

0 comments
read the original abstract

Deep learning models have shown strong potential for time series forecasting, yet their deployment in real-world environmental monitoring remains challenging due to non-stationary dynamics and limited explainability. In this work, we investigate explainability as a central tool for understanding continual learning in adaptive time series forecasting, with Experience Replay strategies. We study neural forecasting architectures such as PatchMixer, PatchTST and DLinear, augmented with attention-based sampling mechanisms to support model adaptation over time. Explainability is leveraged through attention rollout and gradient-based attribution methods (Grad-CAM) to analyze both predictive behavior and sampling strategies within a continual learning framework. Experiments conducted on real-world piezometric time series exhibiting heterogeneous patterns and regime shifts show that analyzing model and sampling behaviors provides valuable insights into the dynamics of the continual learning framework. Beyond predictive performance, our results highlight the challenges and opportunities of using explainability to understand continual learning behaviors, revealing how attribution patterns evolve over time and how they can inform data selection and adaptation strategies in non-stationary forecasting scenarios.

Figures

Figures reproduced from arXiv: 2607.19382 by BDTLN), Emmanuel Doumard (BDTLN), Nicolas Labroche (LIFAT, Nicolas Ragot (RFAI), Nicolas Ringuet (BDTLN), Quentin Besnard (RFAI).

Figure 1
Figure 1. Figure 1: Groundwater level time series and associated linear trend for Piezometer 3 (Beauce region). [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Groundwater level time series and associated linear trend for Piezometer 17 (Pyrénée-Orientales region). [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Heatmap of average monthly groundwater level over multiple decades (x-axis: months; y-axis: years). Example shown [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Heatmap of average monthly groundwater level over multiple decades (x-axis: months; y-axis: years). Example shown [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Continual learning framework with different forecasting models comparison applied on piezometer 3 with Attention [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: PatchMixer Grad-CAM focus points. In contrast, the attention-based sampling strategy ( [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: PatchTST Grad-CAM focus points [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: PatchMixer Attention Rollout focus points. [PITH_FULL_IMAGE:figures/full_fig_p007_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: PatchTST Attention Rollout focus points. [PITH_FULL_IMAGE:figures/full_fig_p008_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Random sampling strategy. 6 Conclusion This study investigates the interplay between continual learning, data sampling strategies, and explainability in the context of non-stationary time series forecasting for groundwater monitoring, addressing challenges highlighted in the introduction. By integrating attention-based sampling, adaptive replay mechanisms can effectively improve long-term forecasting perf… view at source ↗
Figure 11
Figure 11. Figure 11: Attention sampling strategy [PITH_FULL_IMAGE:figures/full_fig_p009_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Maximum loss sampling strategy. The comparative analysis of buffer sampling strategies further demonstrates the importance of approaches that strike a favorable balance between newly observed and previously learned data points. Purely loss-based or random sampling strategies, by contrast, either obscure the rationale behind data selection or risk reinforcing biased or redundant learning behaviors. These f… view at source ↗
Figure 13
Figure 13. Figure 13: Minimum loss sampling strategy. explainability as both a diagnostic and analytical tool for data-centric continual learning, directly addressing the interpretability challenges highlighted at the outset. Overall, this work contributes to understanding continual adaptation in time series forecasting and the challenges of implementing explainability in real-world applications, where explainability is crucia… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

14 extracted references · 1 canonical work pages

  1. [1]

    Yuliang Cai, Jesse Thomason, and Mohammad Rostami. 2023. Task-Attentive Transformer Architecture for Continual Learning of Vision-and-Language Tasks Using Knowledge Distillation. EMNLP (3 2023). https://arxiv.org/abs/2303.14423v1

    Pith/arXiv arXiv 2023

  2. [2]

    Hsin Yu Chen, Zoran Vojinovic, Weicheng Lo, and Jhe Wei Lee. 2023. Groundwater Level Prediction with Deep Learning Methods. Water (Switzerland) 15 (9 2023). Issue 17. https://doi.org/10.3390/w15173118

    work page doi:10.3390/w15173118 2023

  3. [3]

    Zeying Gong, Yujin Tang, and Junwei Liang. 2023. PatchMixer: A Patch-Mixing Architecture for Long-Term Time Series Forecasting. ICLR (10 2023). http://arxiv.org/abs/2310. 00655

    2023

  4. [4]

    Nuwan Gunasekara, Bernhard Pfahringer, Heitor Murilo Gomes, and Albert Bifet. 2023. Survey on Online Streaming Continual Learning. IJCAI International Joint Conference on Artificial Intelligence 2023-August (2023), 6628–6637. https://doi.org/10.24963/IJCAI.2023/743

    doi:10.24963/ijcai.2023/743 2023

  5. [5]

    Vibhor Gupta, Jyoti Narwariya, Pankaj Malhotra, Lovekesh Vig, and Gautam Shroff. 2022. Continual Learning for Multivariate Time Series Tasks with Variable Input Dimensions. ICDM (3 2022). http://arxiv.org/abs/2203.06852

    Pith/arXiv arXiv 2022

  6. [6]

    Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam

    Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. 2022. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers. ICLR (11 2022). http://arxiv.org/abs/2211.14730

    Pith/arXiv arXiv 2022

  7. [7]

    Michela Proietti, Alessio Ragno, and Roberto Capobianco. 2025. XAI-Guided Continual Learning: Rationale, Methods, and Future Directions. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 15, 4 (2025), e70046

    2025

  8. [8]

    Thomas Rojat, Raphaël Puget, David Filliat, Javier Del Ser, Rodolphe Gelin, and Natalia Díaz-Rodríguez. 2021. Explainable Artificial Intelligence (XAI) on TimeSeries Data: A Survey. (4 2021). http://arxiv.org/abs/2104.00950

    Pith/arXiv arXiv 2021

  9. [9]

    Dawid Rymarczyk, Joost van de Weijer, Bartosz Zieliński, and Bartlomiej Twardowski. 2023. ICICLE: Interpretable Class Incremental Continual Learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) . 1887–1898

    2023

  10. [10]

    Guido Schillaci, Uwe Schmidt, and Luis Miranda. 2021. Prediction Error-Driven Memory Consolidation for Continual Learning: On the Case of Adaptive Greenhouse Models. KI - Kunstliche Intelligenz 35 (3 2021), 71–80. Issue 1. https://doi.org/10.1007/s13218-020-00700-8

    doi:10.1007/s13218-020-00700-8 2021

  11. [11]

    James Seale Smith, Lazar Valkov, Shaunak Halbe, Vyshnavi Gutta, Rogerio Feris, Zsolt Kira, and Leonid Karlinsky. 2024. Adaptive Memory Replay for Continual Learning. (4 2024). http://arxiv.org/abs/2404.12526

    Pith/arXiv arXiv 2024

  12. [12]

    Ghada Sokar, Decebal Constantin Mocanu, and Mykola Pechenizkiy. 2021. Self-Attention Meta-Learner for Continual Learning. (1 2021). http://arxiv.org/abs/2101.12136

    Pith/arXiv arXiv 2021

  13. [13]

    Mengqi Xue, Haofei Zhang, Jie Song, and Mingli Song. 2022. Meta-attention for ViT-backed Continual Learning. CVPR (9 2022), 150–159. https://doi.org/10.1109/CVPR52688. 2022.00025

    arXiv 2022

  14. [14]

    Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. 2022. Are Transformers Effective for Time Series Forecasting? AAAI Press (8 2022). http://arxiv.org/abs/2205.13504

    Pith/arXiv arXiv 2022