REVIEW 4 major objections 5 minor 25 references
Attention-based Generative Latent Replay: A Continual Learning Approach for WSI Analysis
T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read This paper claims that an attention-filtered Gaussian mixture latent replay can match buffer-based continual learning for whole-slide image classification while storing no raw slides.
desk verdict Solid buffer-free generative replay for WSI continual learning, but the missing comparison to the authors' own prior MICCAI paper and absent error bars undercut the empirical and novelty claims. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the GMM family: for each past episode, a class-wise multivariate Gaussian mixture captures the distribution of patch embeddings and a one-dimensional Gaussian mixture captures the distribution of patch counts per slide. To synthesize a past-domain slide, the method samples a patch count from the count GMM and then independently draws that many embeddings from the embedding GMM, forming a feature sequence that is fed to the attention-based MIL aggregator together with current-domain data. The attention-based filtering step—retaining only the top $q\%$ of patches by attention score before fitting the mixtures—is what shapes the generative model toward diagnostically relevant tissue rather than background.
What would settle it
Construct a two-domain experiment where the first domain's class discrimination depends on a fixed inter-patch co-occurrence (for example, two patch types that must appear together in the same slide), and check whether the independent-draw GMM replay retains the first domain while the second is learned. A second decisive test: build a domain whose discriminative patches are systematically low-attention; if AGLR-CL forgets it while the unfiltered variant does not, the attention filter is discarding the signal it needs to replay.
Extended reading notes
Core claim
AGLR-CL's central claim is that a domain can be compressed into a Gaussian mixture family—per-class multivariate GMMs over patch embeddings plus a univariate GMM over slide patch counts—and that replaying synthetic slides sampled from these mixtures preserves enough of the past domain to prevent catastrophic forgetting. An attention filter keeps only the top 80% of patches by the MIL aggregator's attention scores before the GMMs are fit, so the generative model is built from the tissue that actually drives classification. The paper validates this on microsatellite instability, tumor mutational burden, HER2, and progesterone receptor tasks across sequences that mix organs and centers, reporting that the method surpasses buffer-free baselines and is on par with buffer-based replay while never storing original slides.
Load-bearing premise
Everything hinges on the assumption that sampling a patch count and then drawing patch embeddings independently from a class-wise Gaussian mixture faithfully reproduces the training signal of a past domain, which means any task-relevant structure living in correlations between patches or in low-attention tissue would be missing from replay.
Editorial extensions
If this is right
- Domain-incremental WSI classification can be done without a replay buffer, eliminating the main storage and privacy barrier to continual learning in clinical settings.
- Attention-guided filtering before GMM fitting improves retention compared to unfiltered GMM replay on four of the five tested domain sequences.
- The method retains knowledge across heterogeneous shifts (organ, center, mixed) and across four biomarker tasks, indicating the mechanism is not limited to one tissue or stain.
- Buffer-free generative replay reaches accuracy comparable to buffer-based replay, which the paper identifies as the practical upper bound for these continual learning sequences.
Reading between the lines
- The independence assumption in the GMM resampling is the natural stress point: if past-domain discriminative signal lives in inter-patch dependencies or spatial layout, replay quality would degrade; a variant that conditions each sampled embedding on its neighbors would test this directly.
- The attention filter trades off against low-attention informative tissue; the TMB sequence already shows a decline when filtering is applied, suggesting an adaptive retention rate per domain could recover that lost signal.
- The latent-replay scheme should transfer to other gigapixel multi-instance domains such as satellite imagery or full-slide cytology, where patch counts and patch features carry the label signal and raw data cannot be kept.
- A formal privacy analysis—for instance, bounding how much a GMM snapshot reveals about any individual slide—would turn the absence of raw storage into a quantifiable guarantee.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes AGLR-CL, a buffer-free continual learning method for domain-incremental whole slide image (WSI) classification. The method uses a frozen pathology foundation model to embed patches, an AB-MIL aggregator, and per-episode Gaussian mixture models fitted to attention-filtered patch embeddings and to patch counts. In later episodes, synthetic WSI representations are sampled from these GMMs and combined with current real data, avoiding storage of raw WSIs. The authors evaluate on five domain-incremental sequences spanning MSI, TMB, PR, and HER2 prediction using TCGA, CPTAC, PAIP, and BCNB data, comparing against naive, joint, cumulative, regularization-based, and rehearsal-based baselines. They report ACC, ILM, and BWT based on weighted F1, AUROC, and AUPRC, plus an ablation of the attention-based filtering step. The central claim is that AGLR-CL is the best buffer-free method on most sequences and matches or approaches buffer-based Replay and GDumb.
Significance. If the empirical claims hold, the paper makes a practically relevant contribution: it offers a privacy-preserving way to perform continual learning on WSIs without storing original slides, using a relatively simple generative model in latent space. The choice of clinically meaningful tasks (biomarker and molecular-status prediction), the use of public datasets, and the inclusion of standard continual learning baselines are strengths. The attention-filtering idea is well motivated by the observation that not all patches are informative for the MIL aggregator. The paper also provides an ablation study and attention heatmaps. However, the empirical evidence currently rests on single-run point estimates with no uncertainty quantification, and the core generative assumption of independent patch embeddings is not validated. These issues limit the strength of the claims until addressed.
major comments (4)
- [§4, Table 3] Every reported ACC, ILM, and BWT value is a single-run point estimate, with no seeds, error bars, confidence intervals, or significance tests. Some headline comparisons are close (e.g., a3 weighted-F1 ACC 67.97 for AGLR-CL vs 67.21 for GDumb; a1 AUPRC 38.38 for AGLR-CL vs 45.24 for SI), and the table itself contains cases where a buffer-free baseline beats AGLR-CL. Without variance estimates, the claims that AGLR-CL 'outperforms other buffer-free methods' and 'matches' buffer-based solutions are not statistically supported. Please report results over at least three random seeds per method and sequence and provide paired tests or confidence intervals.
- [§2.3–2.4, Eq. (1)–(3)] Synthetic WSI generation samples a patch count from GMM_t_count and then draws that many patch embeddings independently from a class-wise GMM_t_emb, replacing the joint distribution of a slide's patch multiset with a product of marginal densities. AB-MIL, as described in §2.1, computes an attention-weighted sum over the multiset, so within-slide composition (e.g., proportions of tissue types, correlations among patches from the same microenvironment or stain batch) can carry label information. The paper does not measure how faithfully synthetic slide-level statistics reproduce the real data, and it does not ablate the independence assumption. In addition, the covariance structure in Eq. (2) is not specified; if full covariances are used in the UNI embedding space (which is 1024-dimensional), the parameter count per component is large and stability is a concern, while diagonal covariances would make the independence assumption even more restrictive. Please add a fidelity analysis of generated versus real embeddings and, if needed, extend the generative model to capture inter-patch dependence.
- [§4, Table 4] The ablation shows that attention-based filtering is not uniformly beneficial. On sequence a5 (TMB), the filtered version is worse than the unfiltered version on weighted-F1 ACC (73.04 vs 73.58), AUROC (57.45 vs 58.60), and on most ILM/BWT entries, with only a small AUPRC improvement. The explanation that 'high variability of morphological alterations' makes discarded patches informative is post hoc and is not tested. Since the attention filter is presented as a central novelty ('ensuring high-quality synthetic samples'), the paper should characterize when filtering helps versus hurts and either adapt q per domain or temper the claim.
- [§1, Contribution (1)] The statement 'we introduce domain incremental CL for MIL for the first time' is difficult to reconcile with the closely related prior work cited in the paper, especially the authors' own [15] on continual domain incremental learning for privacy-aware digital pathology and existing WSI continual learning methods such as ConSlide [9] and lifelong histopathology retrieval [25]. Please clarify precisely how AGLR-CL differs from [15] in problem setting and method. If [15] already uses MIL and generative replay, the novelty should be positioned as the attention-filtered latent GMM replay rather than the problem setting.
minor comments (5)
- [§5 and throughout] There are several typographical errors: 'Wholeslideimage' in the abstract, 'datastes' in the Introduction, and 'subsequentially' in §2.4. These should be corrected.
- [§3, Implementation Details] The choice of q=80% and the candidate sets for K are not justified or analyzed. A sensitivity study over q and K would strengthen the paper, particularly because the ablation shows that q has non-negligible effects.
- [§4, Table 3] The BWT sign convention appears inconsistent with intuition. For example, Naive on sequence a2 shows positive BWT for AUROC (1.27) and AUPRC (6.78) despite being fine-tuned only on the current episode. Please state the exact BWT definition used and explain how positive values should be interpreted.
- [§4, Fig. 2] The attention heatmap analysis is qualitative. A quantitative measure of overlap between high-attention regions and annotated ground-truth regions across sessions would provide stronger evidence that past knowledge is preserved.
- [§3, Implementation Details] No code or reproducibility statement is provided. Releasing code (or at least the trained GMM parameters and evaluation pipeline) would significantly increase the reproducibility of the empirical claims.
Circularity Check
No significant circularity: the GMM replay pipeline is fitted on training episodes and evaluated on held-out test sets, with no load-bearing self-citation or construction-level reduction.
full rationale
AGLR-CL is an empirical continual-learning method rather than a formal derivation from an axiom. In Sec. 2.3, the class-wise GMMt_emb and count GMMt_count are estimated on patch embeddings and patch counts of the current episode after attention filtering; in Sec. 2.4, those GMMs are sampled to build synthetic replay features for training the MIL model. The resulting model is then evaluated on held-out test sets from all episodes, so the replayed data are not used to define the evaluation target. There is no equation in which the predicted quantity is constructed from itself, and no fitted parameter is renamed as a prediction: the GMM parameters are fit on training features and the reported ACC/ILM/BWT scores come from test data. The attention-filtering component is directly ablated in Table 4, including the negative result on sequence a5, so its contribution is empirically tested rather than assumed. Self-citations [13,15] provide background and a hyperparameter setting (alpha=1) and do not carry the central claim. The untested independence assumption in Sec. 2.4, where patch embeddings are drawn independently from the class-wise GMM, is a modeling risk but not a circularity, because AB-MIL's output is not defined in terms of that assumption and the paper does not claim to derive the replay distribution from the target metric. Overall, the derivation chain is self-contained against external benchmarks and no circular step is exhibited.
Assumptions & free parameters
free parameters (3)
- q =
80%
- K_emb =
selected from {8, 16, 24} by BIC
- K_count =
selected from {1, 2, 3, 4, 5} by BIC
assumptions (4)
- domain assumption Patch embeddings for each class can be approximated by a Gaussian mixture model, and sampling from the fitted model yields realistic feature sequences.
- domain assumption Top-q attention patches contain the information needed to preserve past domains, so discarding low-attention patches improves or maintains replay quality.
- domain assumption Independent sampling of patch count and patch embeddings is sufficient for MIL training; no inter-patch spatial or correlational structure is needed.
- standard math Standard EM convergence and BIC model selection are valid for the GMM fits.
Cite this review
Pith. "Pith review of Attention-based Generative Latent Replay: A Continual Learning Approach for WSI Analysis." pith.science (2026). https://pith.science/paper/QZ6X6TS6
@misc{pith2026250508524,
author = {Pith},
title = {Pith review of: Attention-based Generative Latent Replay: A Continual Learning Approach for WSI Analysis},
year = {2026},
howpublished = {\url{https://pith.science/paper/QZ6X6TS6}},
note = {Machine review of arXiv:2505.08524}
}
read the original abstract
Whole slide image (WSI) classification has emerged as a powerful tool in computational pathology, but remains constrained by domain shifts, e.g., due to different organs, diseases, or institution-specific variations. To address this challenge, we propose an Attention-based Generative Latent Replay Continual Learning framework (AGLR-CL), in a multiple instance learning (MIL) setup for domain incremental WSI classification. Our method employs Gaussian Mixture Models (GMMs) to synthesize WSI representations and patch count distributions, preserving knowledge of past domains without explicitly storing original data. A novel attention-based filtering step focuses on the most salient patch embeddings, ensuring high-quality synthetic samples. This privacy-aware strategy obviates the need for replay buffers and outperforms other buffer-free counterparts while matching the performance of buffer-based solutions. We validate AGLR-CL on clinically relevant biomarker detection and molecular status prediction across multiple public datasets with diverse centers, organs, and patient cohorts. Experimental results confirm its ability to retain prior knowledge and adapt to new domains, offering an effective, privacy-preserving avenue for domain incremental continual learning in WSI classification.
Figures
Reference graph
Works this paper leans on
-
[15]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Kumari, P., Reisenbüchler, D., Luttner, L., Schaadt, N.S., Feuerhake, F., Merhof, D.: Continual domain incremental learning for privacy-aware digital pathology. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 34–44. Springer (2024)
work page 2024
-
[9]
In: Proceedings of the IEEE/CVF International Conference on Computer Vision
Huang, Y., Zhao, W., Wang, S., Fu, Y., Jiang, Y., Yu, L.: Conslide: Asynchronous hierarchical interaction transformer with breakup-reorganize rehearsal for contin- ual whole slide image analysis. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 21349–21360 (2023) 10 Kumari et al
work page 2023
-
[25]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Zhu, X., Jiang, Z., Wu, K., Shi, J., Zheng, Y.: Lifelong histopathology whole slide image retrieval via distance consistency rehearsal. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 274–284. Springer (2024)
work page 2024
-
[1]
arXiv preprint arXiv:2407.00465 (2024)
Bhatt, R., Kumari, P., Mahapatra, D., Saddik, A.E., Saini, M.: Characteriz- ing continual learning scenarios and strategies for audio analysis. arXiv preprint arXiv:2407.00465 (2024)
arXiv 2024
-
[2]
Nature487(7407), 330–337 (Jul 2012)
Cancer Genome Atlas Network: Comprehensive molecular characterization of hu- man colon and rectal cancer. Nature487(7407), 330–337 (Jul 2012)
work page 2012
-
[3]
Nature Medicine30(3), 850–862 (2024)
Chen, R.J., Ding, T., Lu, M.Y., Williamson, D.F., Jaume, G., Song, A.H., Chen, B., Zhang, A., Shao, D., Shaban, M., et al.: Towards a general-purpose foundation model for computational pathology. Nature Medicine30(3), 850–862 (2024)
2024
-
[4]
Journal of the royal statistical society: series B (method- ological) 39(1), 1–22 (1977)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the em algorithm. Journal of the royal statistical society: series B (method- ological) 39(1), 1–22 (1977)
work page 1977
-
[5]
In: International Conference on Medical Image Computing and Computer- Assisted Intervention
Derakhshani, M.M., Najdenkoska, I., van Sonsbeek, T., Zhen, X., Mahapatra, D., Worring, M., Snoek, C.G.: Lifelonger: A benchmark for continual disease classifi- cation. In: International Conference on Medical Image Computing and Computer- Assisted Intervention. pp. 314–324. Springer (2022)
work page 2022
Show all 25 references
-
[6]
arXiv preprint arXiv:1810.13166 (2018)
Díaz-Rodríguez, N., Lomonaco, V., Filliat, D., Maltoni, D.: Don’t forget, there is more than forgetting: new metrics for continual learning. arXiv preprint arXiv:1810.13166 (2018)
2018 arXiv
-
[7]
Edwards, N.J., Oberti, M., Thangudu, R.R., Cai, S., McGarvey, P.B., Jacob, S., Madhavan, S., Ketchum, K.A.: The CPTAC data portal: A resource for cancer proteomics research. J. Proteome Res.14(6), 2707–2713 (Jun 2015)
2015
-
[8]
The computer journal41(8), 578–588 (1998)
Fraley, C., Raftery, A.E.: How many clusters? which clustering method? answers via model-based cluster analysis. The computer journal41(8), 578–588 (1998)
1998
-
[10]
In: International conference on machine learning
Ilse,M.,Tomczak,J.,Welling,M.:Attention-baseddeepmultipleinstancelearning. In: International conference on machine learning. pp. 2127–2136. PMLR (2018)
2018
-
[11]
Med- ical Image Analysis89, 102886 (Oct 2023).https://doi.org/10.1016/j.media
Kim, K., Lee, K., Cho, S., Kang, D.U., Park, S., Kang, Y., Kim, H., Choe, G., Moon, K.C., Lee, K.S., Park, J.H., Hong, C., Nateghi, R., Pourakpour, F., Wang, X., Yang, S., Jahromi, S.A.F., Khani, A., Kim, H.R., Choi, D.H., Han, C.H., Kwak, J.T., Zhang, F., Han, B., Ho, D.J., K...
2020
-
[12]
Proceedings of the national academy of sciences 114(13), 3521–3526 (2017)
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017)
2017
-
[13]
arXiv preprint arXiv:2312.17004 (2023)
Kumari, P., Chauhan, J., Bozorgpour, A., Azad, R., Merhof, D.: Continual learning in medical imaging analysis: A comprehensive review of recent advancements and future prospects. arXiv preprint arXiv:2312.17004 (2023)
2023 arXiv
-
[14]
Signal Processing: Image Communication p
Kumari, P., Choudhary, P., Kujur, V., Atrey, P.K., Saini, M.: Concept drift chal- lenge in multimedia anomaly detection: A case study with facial datasets. Signal Processing: Image Communication p. 117100 (2024)
2024
-
[16]
IEEE Transactions on Pattern Analysis and Machine Intelligence40(12), 2935–2947 (2018)
Li, Z., Hoiem, D.: Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence40(12), 2935–2947 (2018). https://doi.org/ 10.1109/TPAMI.2017.2773081
2018
-
[17]
Advances in neural information processing systems30 (2017)
Lopez-Paz, D., Ranzato, M.: Gradient episodic memory for continual learning. Advances in neural information processing systems30 (2017)
2017
-
[18]
Nature Biomedical Engineering5(6), 555–570 (2021)
Lu, M.Y., Williamson, D.F., Chen, T.Y., Chen, R.J., Barbieri, M., Mahmood, F.: Data-efficient and weakly supervised computational pathology on whole-slide images. Nature Biomedical Engineering5(6), 555–570 (2021)
2021
-
[19]
In: Computer Vision–ECCV 2020: 16th Euro- pean Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16
Prabhu, A., Torr, P.H., Dokania, P.K.: Gdumb: A simple approach that questions our progress in continual learning. In: Computer Vision–ECCV 2020: 16th Euro- pean Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16. pp. 524–540. Springer (2020)
2020
-
[20]
Advances in Neural Information Processing Systems32 (2019)
Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T., Wayne, G.: Experience replay for continual learning. Advances in Neural Information Processing Systems32 (2019)
2019
-
[21]
Medical image analysis83, 102645 (2023)
Wang, X., Du, Y., Yang, S., Zhang, J., Wang, M., Zhang, J., Yang, W., Huang, J., Han, X.: Retccl: Clustering-guided contrastive learning for whole-slide image retrieval. Medical image analysis83, 102645 (2023)
2023
-
[22]
Frontiers in Oncology p
Xu, F., Zhu, C., Tang, W., Wang, Y., Zhang, Y., Li, J., Jiang, H., Shi, Z., Liu, J., Jin, M.: Predicting axillary lymph node metastasis in early breast cancer using deep learning on primary tumor biopsy slides. Frontiers in Oncology p. 4133 (2021)
2021
-
[23]
Nature (2024)
Xu, H., Usuyama, N., Bagga, J., Zhang, S., Rao, R., Naumann, T., Wong, C., Gero, Z., González, J., Gu, Y., Xu, Y., Wei, M., Wang, W., Ma, S., Wei, F., Yang, J., Li, C., Gao, J., Rosemon, J., Bower, T., Lee, S., Weerasinghe, R., Wright, B.J., Robicsek, A., Piening, B., Bifulco,...
2024
-
[24]
In: International conference on machine learning
Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International conference on machine learning. pp. 3987–3995. PMLR (2017) AGLR-CL for WSI Analysis 11
2017
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.