Pith. sign in

REVIEW 3 major objections 6 minor 15 references

Test-time adaptive model merging retains prior cancer-subtyping knowledge on whole-slide images without storing old slides.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 14:01 UTC pith:X7FBMVUT

load-bearing objection Solid rehearsal-free WSI benchmark that maps TTA merging methods into continual pathology and shows real retention gains, with the order-sensitivity caveat already owned by the authors. the 3 major comments →

arxiv 2607.04755 v1 pith:X7FBMVUT submitted 2026-07-06 cs.CV

Continual Model Merging with Test-Time Adaptation for Whole-Slide Image Analysis

Duc-Thanh Le , Doanh C. Bui , Ma\"i K. Nguyen , Khang Nguyen This is my paper
classification cs.CV
keywords continual learningmodel mergingtest-time adaptationwhole-slide imagescomputational pathologycatastrophic forgettingTCGA cancer subtyping
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Pathology labs cannot keep every prior whole-slide image when a new cancer-subtyping task arrives, yet sequential fine-tuning forgets earlier cohorts. This benchmark study shows that modern model-merging techniques, which combine independently trained task models in parameter space and then adapt the merge coefficients, masks, or hierarchical layers on unlabeled current test slides, can deliver strong accuracy on the new task while preserving earlier knowledge far better than classic regularization methods and competitively with limited-replay baselines. Across six TCGA cohorts, both in-domain and out-of-domain sites, and two task orders, the approach works under both class-incremental and task-incremental protocols. The practical payoff is a rehearsal-free pipeline that never stores historical patient slides yet still reduces forgetting. The same experiments also reveal that the gains remain sensitive to the order of tasks and to how aggressively the current distribution is allowed to reshape the accumulated merge.

Core claim

When state-of-the-art test-time adaptive merging methods (AdaMerging, AdaRank, Hi-Vec) are transferred to a rehearsal-free continual stream of whole-slide cancer-subtyping tasks, they achieve strong current-task performance and substantially lower forgetting than regularization-based continual-learning baselines, without retaining any historical WSIs, while remaining competitive with small-buffer replay under both CLASS-IL and TASK-IL protocols.

What carries the argument

TTA-guided continual model merging: independently fine-tuned task vectors (or their SVD components or hierarchical heads) are combined with method-specific merge variables that are optimized by Shannon-entropy minimization on the current unlabeled test stream before the final merged weights are used for inference.

Load-bearing premise

That minimizing prediction entropy on only the newest unlabeled test slides is enough to adapt the merge without an explicit safeguard that protects subspaces already learned for earlier tasks.

What would settle it

A controlled run on the same six TCGA folds in which entropy-driven coefficient or mask updates are replaced by a fixed or history-aware merge rule; if forgetting and current-task accuracy then become no better than (or worse than) plain sequential fine-tuning, the claimed benefit of pure test-time adaptation disappears.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper presents a benchmark that transfers three test-time adaptive model-merging methods (AdaMerging, AdaRank, Hi-Vec) to rehearsal-free continual whole-slide image (WSI) classification. Independently fine-tuned task models are consolidated in parameter space, with merge variables (coefficients, singular-component masks, or hierarchical layers) adapted by Shannon entropy minimization on the current unlabeled test stream (Eq. 3). Evaluation uses six TCGA cancer-subtyping cohorts under CLASS-IL and TASK-IL, in-domain and site-held-out OOD splits, two task orders, and standard continual metrics (bACC, Macro/Weighted F1, FGT, BWT), with comparisons to LwF, EWC, DER++ (10/30 WSI buffers), naive fine-tuning, and a fully supervised upper reference. The authors report strong task-specific performance and improved retention relative to regularization and limited-replay baselines without storing historical WSIs, while documenting sensitivity to task order and to adaptation driven only by the current distribution.

Significance. Continual computational pathology is practically important: gigapixel WSIs and institutional constraints make rehearsal and centralized retraining difficult. Mapping multi-task/TTA merging methods into a strict rehearsal-free sequential protocol, with multi-cohort TCGA evaluation, IND/OOD site splits, dual task orders, and both CLASS-IL and TASK-IL, is a useful contribution to the field. The experimental design is relatively strong for a benchmark (10-fold CV, means±std, complementary F1 metrics, forgetting/BWT). The paper is appropriately cautious: it frames TTA-guided merging as promising rather than solved, and the Discussion explicitly identifies the stability–plasticity trade-off arising from entropy minimization on the current stream alone. If the reported retention advantages hold under broader orders and against static merging, this would motivate a concrete research line—adaptive merging with explicit historical-subspace protection—for lifelong pathology models.

major comments (3)
  1. [Introduction; Tables 2–4] The central claim that adapting merge variables at test time improves continual WSI performance is not isolated from static merging. MergeSlide is cited as the fixed-rule continual merging baseline that TTA is meant to extend (Introduction; §2), yet neither MergeSlide nor simple task-arithmetic / fixed-coefficient merging appears in Tables 2–4. Without that control, gains over LwF/EWC/DER++ cannot be attributed to TTA rather than to model merging itself. Please add at least one static merging baseline under the same protocol (same backbone, K-patch sampling, heads, and task orders).
  2. [Section 1; Section 3 Implementation Details] Section 1 poses whether repeated TTA update-and-reconstruct of merge variables justifies its computational cost, but the experiments report no wall-clock time, TTA steps, or memory overhead relative to EWC/LwF/DER++ or static merging. For a deployment-oriented pathology benchmark this is load-bearing: if TTA is expensive per test stream, the practical recommendation changes. Please report cost (e.g., seconds per task / per fold on the A100) alongside accuracy.
  3. [Tables 2–3; Section 4 Discussion] Only two task sequences are evaluated (B→R→N→E→T→C and its reverse; Tables 2–3). FGT for AdaMerging/Hi-Vec rises from ~2.2–2.7 to ~14 under reversal, which the paper correctly flags as order sensitivity, but two orders are a thin basis for that conclusion with six heterogeneous cohorts (common vs rare, Tab. 1). At least one additional order (e.g., rare-first or interleaved common/rare) would make the sensitivity claim more robust and better support the Discussion’s call for historical-subspace protection.
minor comments (6)
  1. [Running header] Page headers repeatedly show “ZeroSlide,” which does not match the title or content and looks like a leftover template string; please correct.
  2. [Abstract; Section 1] Several spacing/typo issues in the abstract and introduction (e.g., “Recentstate-of-the-art,” “test-timeadaptivemerging,” “withoutstoringhistoricalWSIs,” “Sequentialfine-tuningissusceptible”). A careful copy-edit pass is needed.
  3. [Fig. 1] Fig. 1 is dense; the three panels (regularization, rehearsal, TTA-guided merging) would be clearer with a short caption walkthrough of symbols (R_k, ξ_t, λ/B/ϕ*).
  4. [Section 2.2 Hi-Vec] Hi-Vec’s continual mapping (dynamic layer selection, δ_OOD, δ_sim, encoder updates) is compressed in §2.2; a short algorithm box or pseudocode for one continual step would improve reproducibility.
  5. [Section 3 Implementation Details] State the value of K (sampled patches) and the TTA optimizer/step budget explicitly in Implementation Details; they are free parameters of the protocol but not numerically listed in the main text.
  6. [Section 3 Implementation Details] Clarify whether task-specific linear heads are ever merged or only concatenated at CLASS-IL inference; a one-sentence statement would remove ambiguity for multi-head setups.

Circularity Check

1 steps flagged

Empirical benchmark with no derivation that reduces to its inputs; only mild non-load-bearing self-citation to prior MergeSlide work by overlapping authors.

specific steps
  1. self citation load bearing [Section 2.1 Preliminaries / WSI processing; citation [7]]
    "Inspired by MergeSlide [7], each WSI Xi ∈ Dt is first pre-processed via the CLAM pipeline [1] … Following MergeSlide, K patches are randomly sampled … MergeSlide [7] demonstrates the effectiveness of continual model merging for WSI classification."

    MergeSlide is prior work by overlapping authors and supplies the frozen TITAN + random-K sampling pipeline used throughout. The citation is not load-bearing for the paper's main claim (that TTA-guided coefficient/mask/layer adaptation improves retention relative to EWC/LwF/DER++), which is established by the new Tables 2-4 comparisons; it is ordinary methodological reuse rather than a circular premise.

full rationale

This paper is a benchmark study that maps existing external TTA-guided merging methods (AdaMerging, AdaRank, Hi-Vec) onto a rehearsal-free continual WSI protocol and compares them against standard CL baselines (EWC, LwF, DER++) on six TCGA cohorts under CLASS-IL/TASK-IL, IND/OOD, and two task orders. There is no first-principles derivation, uniqueness theorem, or fitted constant that is later re-presented as a prediction. The TTA objective (Eq. 3, Shannon entropy minimization on the unlabeled current test stream) is the standard unsupervised surrogate used by the cited methods; evaluation metrics (bACC, FGT, BWT, Macro/Weighted F1) are conventional and computed after adaptation. The sole self-reference is the use of MergeSlide [7] (overlapping authors) for the WSI preprocessing and sampling protocol and as a static-merging baseline; this is ordinary prior-work citation and is not invoked to force any uniqueness claim or to define the reported performance numbers. The paper itself flags the stability-plasticity limitation of pure current-stream adaptation (Discussion). Consequently the central empirical claims rest on new multi-cohort tables rather than on any circular reduction, yielding only a negligible self-citation score.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The central claim rests on standard continual-learning problem constraints, an entropy-minimization TTA surrogate, frozen TITAN features, and the assumption that independently fine-tuned task vectors can be merged without historical data. No new physical entities are introduced; free parameters are ordinary experimental knobs (epochs, buffer sizes, sampling K, agreement thresholds) rather than constants fitted to force a theoretical prediction.

free parameters (5)
  • epochs per task Ne
    Fixed to 10 for all methods; affects how strongly each task model is specialized before merging.
  • patch sample size K
    Number of randomly sampled patch embeddings per slide for continual/TTA methods; shared protocol but still a design choice that can change bag statistics.
  • DER++ replay buffer size
    Compared at 10 and 30 WSIs; directly controls the rehearsal baseline strength against which forgetting claims are judged.
  • Hi-Vec agreement/similarity thresholds delta_OOD and delta_sim
    Gate whether hierarchical adaptation is suppressed or shared; paper notes they create a stability-plasticity dilemma but does not fully characterize sensitivity.
  • TTA entropy-minimization optimization schedule for merge variables
    Coefficients, masks, or selected hierarchical layers are optimized on the current unlabeled stream; exact optimizer settings and stopping criteria are not exhaustively specified.
axioms (5)
  • domain assumption No access to previous task data or exemplar memory after a task completes.
    Core continual-learning constraint stated in Section 2.1; defines the rehearsal-free setting the methods must satisfy.
  • ad hoc to paper Shannon entropy of predictions on unlabeled current-test WSIs is a valid surrogate objective for adapting merge variables.
    Shared TTA objective in Eq. 3; inherited from the transferred methods but load-bearing for the claim that test-time adaptation improves continual merging.
  • domain assumption Frozen TITAN patch embeddings plus a shared slide aggregator initialization provide a sufficiently transferable representation across cancer-subtyping tasks.
    All methods start from the same TITAN-initialized backbone; performance comparisons assume this feature space is adequate for fair continual evaluation.
  • domain assumption Task vectors or hierarchical heads from independently fine-tuned models can be combined into a single inference model without revisiting old slides.
    Foundational model-merging premise used throughout Section 2.2 and Fig. 1.
  • standard math IND vs OOD site splits and reversed task orders are informative probes of distribution shift and order sensitivity.
    Standard experimental design assumptions for continual and domain-shift evaluation; not mathematically novel but required for the reported conclusions.

pith-pipeline@v1.1.0-grok45 · 15225 in / 3260 out tokens · 27090 ms · 2026-07-11T14:01:59.501752+00:00 · methodology

0 comments
read the original abstract

Model merging offers a practical alternative to conventional continual learning by integrating independently fine-tuned models without retaining previous training data. Recent state-of-the-art model merging methods employ test-time adaptation (TTA-guided merging) to address distribution shifts by adjusting merging-related variables using unlabeled target data. However, these methods have primarily been studied in multi-task or single-target settings, and their behavior under sequential continual learning remains insufficiently understood. We present a benchmark study that maps this family of methods to rehearsal-free continual Whole Slide Image classification and evaluates them against traditional continual-learning approaches. Experiments on six TCGA cancer-subtyping cohorts cover CLASS-IL and TASK-IL scenarios, in-domain and out-of-domain evaluation, and different task orders. The results show that adapting model merging at test time can provide strong task-specific performance and improve retention of previously acquired knowledge without storing historical WSIs. Nevertheless, performance remains sensitive to task order and to the interaction between adaptation on the current distribution and accumulated knowledge. This benchmark identifies model merging with test-time adaptation as a promising direction for continual computational pathology and motivates future methods that balance adaptation to domain shift with explicit preservation of historical knowledge.

Figures

Figures reproduced from arXiv: 2607.04755 by Doanh C. Bui, Duc-Thanh Le, Khang Nguyen, Ma\"i K. Nguyen.

Figure 1
Figure 1. Figure 1: Comparison of traditional continual learning and TTA-guided model [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Performance degradation of all methods under the (a) in-distribution [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

15 extracted references · 2 linked inside Pith

  1. [1]

    Data-efficient and weakly supervised computational pathol- ogy on whole-slide images,

    M. Y. Lu, D. F. Williamson, T. Y. Chen, R. J. Chen, M. Barbieri, and F. Mahmood, “Data-efficient and weakly supervised computational pathol- ogy on whole-slide images,”Nature biomedical engineering, vol. 5, no. 6, pp. 555–570, 2021

    2021

  2. [2]

    A federated learning system for histopathology image analysis with an orchestral stain- normalization gan,

    Y. Shen, A. Sowmya, Y. Luo, X. Liang, D. Shen, and J. Ke, “A federated learning system for histopathology image analysis with an orchestral stain- normalization gan,”IEEE Transactions on Medical Imaging, vol. 42, no. 7, pp. 1969–1981, 2022

    1969

  3. [3]

    Transfer without forgetting,

    M. Boschini et al., “Transfer without forgetting,” inEuropean conference on computer vision, Springer, 2022, pp. 692–709

    2022

  4. [4]

    Dark experience for general continual learning: A strong, simple baseline,

    P. Buzzega, M. Boschini, A. Porrello, D. Abati, and S. Calderara, “Dark experience for general continual learning: A strong, simple baseline,”Ad- vances in neural information processing systems, vol. 33, pp. 15920–15930, 2020

    2020

  5. [5]

    Editing models with task arithmetic,

    G. Ilharco et al., “Editing models with task arithmetic,”arXiv preprint arXiv:2212.04089, 2022

    Pith/arXiv arXiv 2022

  6. [6]

    Ties-merging: Resolving interference when merging models,

    P.Yadav, D. Tam,L.Choshen, C. A. Raffel, andM. Bansal, “Ties-merging: Resolving interference when merging models,”Advances in neural infor- mation processing systems, vol. 36, pp. 7093–7115, 2023

    2023

  7. [7]

    Mergeslide: Continual model merging and task-to-class prompt-aligned inference for lifelong learning on whole slide images,

    D. C. Bui, B. H. Ngo, H. L. Pham, K. Nguyen, M. K. Nguyen, and Y. Nakashima, “Mergeslide: Continual model merging and task-to-class prompt-aligned inference for lifelong learning on whole slide images,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Com- puter Vision, 2026, pp. 4859–4868

    2026

  8. [8]

    Adamerging: Adaptive model merging for multi-task learn- ing,

    E. Yang et al., “Adamerging: Adaptive model merging for multi-task learn- ing,” inInternational Conference on Learning Representations, vol. 2024, 2024, pp. 22743–22763

    2024

  9. [9]

    Adarank: Adaptive rank pruning for enhanced model merging,

    C. Lee, J. Choi, C. Lee, D. Kim, and S. Hong, “Adarank: Adaptive rank pruning for enhanced model merging,”arXiv preprint arXiv:2503.22178, 2025

    arXiv 2025

  10. [10]

    Hi- erarchical adaptive networks with task vectors for test-time adaptation,

    S. Ambekar, M. Hasny, L. A. Daza, D. M. Lang, and J. Schnabel, “Hi- erarchical adaptive networks with task vectors for test-time adaptation,” inProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2026, pp. 4661–4672

    2026

  11. [11]

    Overcoming catastrophic forgetting in neural net- works,

    J. Kirkpatrick et al., “Overcoming catastrophic forgetting in neural net- works,”Proceedings of the national academy of sciences, vol. 114, no. 13, pp. 3521–3526, 2017

    2017

  12. [12]

    Learning without forgetting,

    Z. Li and D. Hoiem, “Learning without forgetting,”IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 12, pp. 2935–2947, 2017

    2017

  13. [13]

    A multimodal whole-slide foundation model for pathology,

    T. Ding et al., “A multimodal whole-slide foundation model for pathology,” Nature medicine, pp. 1–13, 2025

    2025

  14. [14]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101, 2017. ZeroSlide 11

    Pith/arXiv arXiv 2017

  15. [15]

    Robust whole slide image analysis for cervical cancer screening using deep learning,

    S. Cheng et al., “Robust whole slide image analysis for cervical cancer screening using deep learning,”Nature communications, vol. 12, no. 1, p. 5639, 2021

    2021