Pith. sign in

REVIEW 3 major objections 5 minor 22 references

Merging independently fine-tuned pathology models yields a unified WSI survival predictor that forgets less than replay or regularization methods.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 14:09 UTC pith:Z2J44FJN

load-bearing objection Useful domain extension of model merging to WSI survival; the outperformance claim is real on this sequence but still order- and metric-fragile. the 3 major comments →

arxiv 2607.04747 v1 pith:Z2J44FJN submitted 2026-07-06 cs.CV

MergeSurv: Merging-Based Continual Learning for Survival Analysis on Whole-Slide Images

Vu Minh Tran , Doanh C. Bui , Ma\"i K. Nguyen , Khang Nguyen This is my paper
classification cs.CV
keywords continual learningmodel mergingwhole-slide imagessurvival analysiscomputational pathologycatastrophic forgettingCox regressionvision-language foundation model
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Cancer survival models built on whole-slide images are usually trained separately for each organ or cohort, which is expensive for gigapixel slides and blocks privacy-friendly reuse of past data. MergeSurv instead fine-tunes a pathology foundation model on each new survival task from the same starting weights, then sequentially merges the resulting parameter updates into one shared backbone without ever storing earlier slides. At test time a single averaged Cox head (One-for-All) or a set of retained task heads routed by text-prompt similarity (Voting-Expert Aggregation) produces risk scores for every cohort seen so far. On a four-cohort TCGA sequence the merged models match or beat joint training on average C-index while recording the lowest forgetting among the compared continual-learning baselines. The practical claim is that parameter merging can deliver scalable, privacy-preserving continual survival analysis in computational pathology.

Core claim

Independently fine-tuning a pathology vision-language foundation model on each successive WSI survival task and then orthogonally merging the task vectors produces a single unified model whose average ranking performance and forgetting metrics on four TCGA cohorts surpass naive fine-tuning as well as standard regularization-based and rehearsal-based continual-learning methods, without any storage of previous whole-slide images.

What carries the argument

OPCM-style sequential merging (Eq. 3): the new task vector is projected onto the orthogonal complement of the accumulated previous task vectors via SVD, then re-balanced by a norm-ratio scalar so that old and new survival knowledge coexist in one backbone; risk is later read out either by an averaged Cox head (OFA) or by prompt-similarity-weighted expert heads (VEA).

Load-bearing premise

That orthogonal projection of parameter updates plus either averaged or prompt-routed Cox heads is enough to keep relative risk ranking correct for censored survival outcomes across different cancer types, without joint re-training or knowing the task identity at test time.

What would settle it

Re-run the same BLCA o UCEC o LUAD o BRCA sequence (or a reverse order) with a larger buffer-free regularization baseline and a true multi-cohort joint model; if the merged VEA model no longer shows both higher average C-index and lower forgetting than those controls, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • New cancer cohorts can be added by fine-tuning only the latest model and merging once, avoiding full joint re-training on all prior gigapixel slides.
  • Hospitals can update a shared survival model without exchanging raw patient WSIs, satisfying privacy constraints that block rehearsal buffers.
  • A single merged backbone plus lightweight Cox heads can serve multiple organs at inference, reducing storage and deployment cost relative to one model per cohort.
  • Prompt-based expert routing (VEA) offers a practical way to keep task-specific risk heads without requiring task labels at test time.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same orthogonal-merge recipe may transfer to other continuous-valued WSI tasks (e.g., continuous biomarker regression) where rehearsal is equally costly.
  • If prompt routing remains accurate as the number of tasks grows beyond four, VEA could become a default multi-organ risk aggregator without an explicit task classifier.
  • Performance that matches or exceeds joint training on a short sequence raises the possibility that merging can regularize away cohort-specific noise that joint optimization retains.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. MergeSurv proposes a merging-based continual learning framework for WSI survival analysis. A pathology vision-language foundation model (TITAN) is independently fine-tuned on each cancer cohort with a Cox head (Eq. 1–2); task-specific WSI-aggregator parameters are then sequentially merged via OPCM-style orthogonal projection of task vectors (Eq. 3) without retaining prior WSIs. Two inference strategies are studied: One-for-All (averaged Cox head) and Voting-Expert Aggregation (prompt-similarity routing over retained task heads). On a fixed four-cohort TCGA sequence (BLCA o UCEC o LUAD o BRCA), 5-fold CV shows MergeSurv-VEA attaining the highest average C-Index and lowest forgetting among naive fine-tuning, EWC, LwF, ER, DER, and DER++, with competitive IPCW results and supportive Kaplan–Meier stratification.

Significance. The work addresses a practically important gap: continual survival modeling on gigapixel WSIs under privacy and storage constraints that make rehearsal unattractive. Extending model merging from classification (MergeSlide) to censored risk ranking, and providing both a unified-head and a prompt-routed multi-head inference path, is a concrete and useful contribution. Strengths include a clear experimental protocol (5-fold CV, C-Index/IPCW, forgetting/BWT), comparison against standard regularization- and rehearsal-based baselines, and qualitative KM curves with log-rank p-values. If the outperformance generalizes beyond the single reported order, MergeSurv would be a scalable, privacy-preserving alternative for multi-cohort computational pathology.

major comments (3)
  1. §3 and Table 1: The central claim that MergeSurv “outperforms … representative … continual learning methods” rests on a single fixed task order (BLCA o UCEC o LUAD o BRCA). Because OPCM (Eq. 3) projects each new task vector onto the orthogonal complement of the accumulated vectors, the residual subspace and λ_t schedule are path-dependent. Survival ranking under Cox censoring (Eq. 2) can be sensitive to that geometry. Without at least one additional order (or a multi-order average) the reported superiority cannot be treated as a method property rather than a sequence-specific outcome. This is load-bearing for the abstract claim.
  2. Table 1(a)–(b): Ranking is metric-dependent. MergeSurv-VEA leads average C-Index (0.6773) and forgetting, yet EWC leads average C-Index IPCW (0.6353 vs 0.6294). VEA also slightly exceeds Joint training on C-Index (0.6773 vs 0.6739) with overlapping standard deviations. The manuscript should either (i) report multi-seed / multi-order statistics that establish whether the edge is reliable, or (ii) qualify the claim to “competitive with lower forgetting and no replay,” rather than unqualified outperformance.
  3. §2.4 / Eq. (5)–(6): VEA’s routing relies on cosine similarity between the merged WSI embedding and task-level text prompts when task identity is unknown. No ablation of routing accuracy, temperature γ, or prompt wording is provided, nor is performance under oracle task-ID routing reported. Because VEA is the stronger of the two proposed strategies and the source of the best C-Index numbers, the contribution of the routing mechanism versus simply retaining multiple heads needs to be isolated.
minor comments (5)
  1. §3 Implementation: K=1000 randomly sampled patches and fixed learning rates / 20 epochs are reasonable but should be stated as fixed hyperparameters; a brief sensitivity note (or appendix) would strengthen reproducibility.
  2. Fig. 2: Kaplan–Meier panels are dense; the caption’s “Zooming 4×” instruction is awkward for print. Consider larger panels or a supplementary multi-page figure.
  3. Eq. (3): The projection operator G and the adaptive rule for λ_t are described only at a high level; a short algorithmic box or reference to the exact OPCM implementation would help readers reimplement.
  4. Related work: ConSurv (cited as [4]) is the closest multimodal continual survival baseline; a one-sentence contrast of setting (multimodal vs. WSI-only, rehearsal vs. merging) would clarify novelty.
  5. Typographical: “Ad hocfine-tuning” and several missing spaces in §2.1–2.2 headings; “Inthisstudy” in the conclusion.

Circularity Check

0 steps flagged

No significant circularity: empirical CL method whose survival metrics are measured on held-out folds, not derived by construction from its own inputs or self-citations.

full rationale

MergeSurv is an empirical systems paper. Task-specific models are fine-tuned independently with the Cox partial-likelihood loss (Eq. 2), parameters are merged via the external OPCM procedure (Eq. 3), and two inference heads (OFA averaging, VEA prompt-weighted experts) produce risk scores that are scored by standard C-Index / C-Index-IPCW, Forgetting and BWT on 5-fold held-out splits of four TCGA cohorts (Table 1). None of these quantities is defined in terms of the others; the reported outperformance is a measurement against external baselines (naive FT, EWC, LwF, ER, DER, DER++), not a tautology of a fitted parameter or a self-citation chain. Citations to MergeSlide (overlapping authors) and OPCM supply prior components for classification merging and the orthogonal-projection operator, respectively, but the survival-specific results (censored ranking, OFA/VEA heads, KM stratification) are new experimental outcomes and do not reduce to those citations by construction. No uniqueness theorem, ansatz smuggling, or renaming of a known empirical law appears. Consequently the derivation chain is self-contained against external benchmarks and circularity score is zero.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 3 invented entities

The claim rests on standard survival and MIL assumptions, a pretrained TITAN backbone treated as given, and the OPCM merge operator imported from prior work. Free knobs (λ_t, γ, patch count, epochs, LRs) and the design of OFA/VEA are paper-specific choices that affect reported rankings but are not derived from first principles.

free parameters (5)
  • λ_t (merge balance factor)
    Adaptive scalar in Eq. 3 controlling old vs new task-vector weight; computed from norm ratios but still a design-dependent stabilizer of the merge.
  • γ (routing temperature)
    Softmax temperature for VEA expert weights from prompt–slide cosine similarity; chosen for inference, not learned from a stated objective.
  • K=1000 random patches per WSI
    Bag sampling size under MIL; fixed hyperparameter that can change risk ranking.
  • Learning rates (1e-5 backbone, 1e-4 head) and E=20 epochs
    Training schedule for each ad hoc fine-tune; performance depends on these choices.
  • Replay buffer size 32 WSIs (baselines)
    Fixed memory for ER/DER/DER++; affects fairness of comparison to buffer-free MergeSurv.
axioms (5)
  • domain assumption Cox partial-likelihood ranking is an adequate training objective for relative survival risk on these cohorts.
    §2.2 uses L_Cox exclusively; proportional-hazards structure is assumed, not validated per cohort.
  • domain assumption A WSI can be treated as a bag of K patch embeddings (MIL) sufficient for patient-level risk.
    §2.2 and implementation details; standard but load-bearing for all methods compared.
  • ad hoc to paper OPCM orthogonal projection of task vectors preserves prior-task knowledge while admitting new tasks (⟨G(Δθ_S,t), Δẽθ_S,1:t−1⟩_F = 0).
    §2.3 imports OPCM [16] as the merge operator; correctness of continual survival hinges on this geometric assumption.
  • domain assumption TITAN pretrained weights are a suitable shared prior for all four cancer survival tasks.
    θ_prior initialization in §2.2; all task vectors are defined relative to this base.
  • ad hoc to paper Task-level text prompts yield cosine similarities that meaningfully route VEA experts when task ID is unknown.
    §2.4 VEA; routing quality is assumed rather than proven with an ablation on prompt design.
invented entities (3)
  • MergeSurv framework no independent evidence
    purpose: Package independent fine-tuning, OPCM merging, and survival-specific inference for continual WSI survival.
    Named system combining existing merge operators with Cox heads; no independent physical entity, only a method construct.
  • One-for-All (OFA) survival head no independent evidence
    purpose: Single averaged Cox head on the merged backbone for memory-efficient multi-task inference.
    Design choice in §2.4; evaluated only inside this paper.
  • Voting-Expert Aggregation (VEA) no independent evidence
    purpose: Retain task-specific Cox heads and weight them by prompt–slide similarity.
    Inference strategy introduced here; performance is the only evidence offered.

pith-pipeline@v1.1.0-grok45 · 13777 in / 3317 out tokens · 39294 ms · 2026-07-11T14:09:19.330430+00:00 · methodology

0 comments
read the original abstract

Survival analysis on Whole Slide Images (WSIs) is important in computational pathology for prognosis estimation and treatment planning. However, existing survival models are typically trained independently for each cancer cohort, making continual adaptation computationally expensive for gigapixel-scale WSIs. In this study, we propose MergeSurv, a merging-based continual learning framework for WSI survival analysis. A pathology vision-language foundation model is independently fine-tuned on each task, and the learned parameters are sequentially merged into a unified model without storing previous training data. We further investigate two inference strategies: One-for-All (OFA) and Voting-Expert Aggregation (VEA). Experiments on four TCGA cohorts demonstrate that MergeSurv outperforms naive fine-tuning as well as representative regularization-based and rehearsal-based continual learning methods, while effectively reducing catastrophic forgetting. The results suggest that model merging is a promising direction for scalable and privacy-preserving continual learning in computational pathology.

Figures

Figures reproduced from arXiv: 2607.04747 by Doanh C. Bui, Khang Nguyen, Ma\"i K. Nguyen, Vu Minh Tran.

Figure 1
Figure 1. Figure 1: Overall framework of MergeSurv. Each survival task is fine-tuned from [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Kaplan–Meier survival curves of selected continual learning methods [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references

  1. [1]

    Regression Models and Life-Tables

    David R. Cox. “Regression Models and Life-Tables”. In:Journal of the Royal Statistical Society: Series B (Methodological)34.2 (1972), pp. 187– 202. MergeSurv 9

    1972

  2. [2]

    Whole Slide Images Based Cancer Survival Prediction Using Attention Guided Deep Multiple Instance Learning Networks

    Jiawen Yao et al. “Whole Slide Images Based Cancer Survival Prediction Using Attention Guided Deep Multiple Instance Learning Networks”. In: Medical Image Analysis65 (2020), p. 101789

    2020

  3. [3]

    Data-efficient and weakly supervised computational pathology on whole-slide images

    Ming Y Lu et al. “Data-efficient and weakly supervised computational pathology on whole-slide images”. In:Nature Biomedical Engineering5.6 (2021), pp. 555–570

    2021

  4. [4]

    ConSurv: Multimodal Continual Learning for Survival Analysis

    Dianzhi Yu et al. “ConSurv: Multimodal Continual Learning for Survival Analysis”.In:Proceedings of the AAAI Conference on Artificial Intelligence 40.33 (Mar. 2026), pp. 27899–27907

    2026

  5. [5]

    Overcoming catastrophic forgetting in neural networks

    James Kirkpatrick et al. “Overcoming catastrophic forgetting in neural networks”. In:Proceedings of the National Academy of Sciences114.13 (2017), pp. 3521–3526

    2017

  6. [6]

    A Comprehensive Survey of Continual Learning: The- ory, Method and Application

    Liyuan Wang et al. “A Comprehensive Survey of Continual Learning: The- ory, Method and Application”. In:IEEE Transactions on Pattern Analysis and Machine Intelligence46.8 (2024), pp. 5362–5383

    2024

  7. [7]

    Learning without Forgetting

    Zhizhong Li and Derek Hoiem. “Learning without Forgetting”. In:IEEE Transactions on Pattern Analysis and Machine Intelligence40.12 (2018), pp. 2935–2947

    2018

  8. [8]

    Experience Replay for Continual Learning

    David Rolnick et al. “Experience Replay for Continual Learning”. In:Ad- vances in Neural Information Processing Systems. Vol. 32. Curran Asso- ciates, Inc., 2019

    2019

  9. [9]

    Dark experience for general continual learning: a strong, simple baseline

    Pietro Buzzega et al. “Dark experience for general continual learning: a strong, simple baseline”. In:Advances in Neural Information Processing Systems33 (2020), pp. 15920–15930

    2020

  10. [10]

    MergeSlide: Continual Model Merging and Task- to-Class Prompt-Aligned Inference for Lifelong Learning on Whole Slide Images

    Doanh C. Bui et al. “MergeSlide: Continual Model Merging and Task- to-Class Prompt-Aligned Inference for Lifelong Learning on Whole Slide Images”. In:Proceedings of the IEEE/CVF Winter Conference on Appli- cations of Computer Vision. 2026, pp. 4859–4868

    2026

  11. [11]

    Continual Multiple Instance Learning with En- hanced Localization for Histopathological Whole Slide Image Analysis

    Byung Hyun Lee et al. “Continual Multiple Instance Learning with En- hanced Localization for Histopathological Whole Slide Image Analysis”. In:Proceedings of the IEEE/CVF International Conference on Computer Vision. 2025, pp. 23232–23242

    2025

  12. [12]

    ConSlide: Asynchronous Hierarchical Interaction TransformerwithBreakup-ReorganizeRehearsalforContinualWholeSlide Image Analysis

    Yanyan Huang et al. “ConSlide: Asynchronous Hierarchical Interaction TransformerwithBreakup-ReorganizeRehearsalforContinualWholeSlide Image Analysis”. In:Proceedings of the IEEE/CVF International Confer- ence on Computer Vision. 2023, pp. 21349–21360

    2023

  13. [13]

    Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time

    Mitchell Wortsman et al. “Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time”. In:Proceedings of the 39th International Conference on Machine Learn- ing. Vol. 162. Proceedings of Machine Learning Research. PMLR, 2022, pp. 23965–23998

    2022

  14. [14]

    Editing Models with Task Arithmetic

    Gabriel Ilharco et al. “Editing Models with Task Arithmetic”. In:Interna- tional Conference on Learning Representations. 2023

    2023

  15. [15]

    AMultimodalWhole-SlideFoundationModelforPathol- ogy

    TongDingetal.“AMultimodalWhole-SlideFoundationModelforPathol- ogy”. In:Nature Medicine31 (2025), pp. 3749–3761. 10 Tran et al

    2025

  16. [16]

    Merging on the Fly Without Retraining: A Sequential Approach to Scalable Continual Model Merging

    Anke Tang et al. “Merging on the Fly Without Retraining: A Sequential Approach to Scalable Continual Model Merging”. In:Advances in Neural Information Processing Systems. Vol. 38. 2025

    2025

  17. [17]

    Multivariable Prog- nostic Models: Issues in Developing Models, Evaluating Assumptions and Adequacy, and Measuring and Reducing Errors

    Frank E. Harrell, Kerry L. Lee, and Daniel B. Mark. “Multivariable Prog- nostic Models: Issues in Developing Models, Evaluating Assumptions and Adequacy, and Measuring and Reducing Errors”. In:Statistics in Medicine 15.4 (1996), pp. 361–387

    1996

  18. [18]

    On the C-statistics for Evaluating Overall Adequacy of Risk Prediction Procedures with Censored Survival Data

    Hajime Uno et al. “On the C-statistics for Evaluating Overall Adequacy of Risk Prediction Procedures with Censored Survival Data”. In:Statistics in Medicine30.10 (2011), pp. 1105–1117

    2011

  19. [19]

    Riemannian Walk for Incremental Learning: Un- derstanding Forgetting and Intransigence

    Arslan Chaudhry et al. “Riemannian Walk for Incremental Learning: Un- derstanding Forgetting and Intransigence”. In:Proceedings of the European Conference on Computer Vision (ECCV). 2018, pp. 556–572

    2018

  20. [20]

    Gradient Episodic Memory for Continual Learning

    David Lopez-Paz and Marc’Aurelio Ranzato. “Gradient Episodic Memory for Continual Learning”. In:Advances in Neural Information Processing Systems. Vol. 30. 2017

    2017

  21. [21]

    Nonparametric Estimation from In- complete Observations

    Edward L. Kaplan and Paul Meier. “Nonparametric Estimation from In- complete Observations”. In:Journal of the American Statistical Associa- tion53.282 (1958), pp. 457–481

    1958

  22. [22]

    Evaluation of Survival Data and Two New Rank Order Statistics Arising in Its Consideration

    Nathan Mantel. “Evaluation of Survival Data and Two New Rank Order Statistics Arising in Its Consideration”. In:Cancer Chemotherapy Reports 50.3 (1966), pp. 163–170

    1966