Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

Sequential fine-tuning that refines a foundation model task by task while retaining pretraining knowledge beats per-task and joint fine-tuning for 3D medical image segmentation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-05 04:26 UTC pith:UPHTRV6N

load-bearing objection MedSeqFT's sequential fine-tuning idea is sensible and broadly evaluated, but the headline superiority over FFT is not fully established given unmatched compute and single-run results. the 5 major comments →

arxiv 2509.06096 v1 pith:UPHTRV6N submitted 2025-09-07 cs.CV

MedSeqFT: Sequential Fine-tuning Foundation Models for 3D Medical Image Segmentation

Yiwen Ye , Yicheng Wu , Xiangde Luo , He Zhang , Ziyang Chen , Ting Dang , Yanning Zhang , Yong Xia This is my paper
classification cs.CV
keywords sequential fine-tuning3D medical image segmentationfoundation modelscatastrophic forgettingknowledge distillationLoRAself-supervised learningtransfer learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that when a medical segmentation foundation model must handle many tasks arriving one at a time, the best way to adapt it is sequentially: finish one task, then start the next from the refined model rather than from the original pretrained weights. To keep the model from forgetting earlier knowledge, it keeps a buffer of samples most similar to the pretraining distribution and uses a LoRA-based distillation step that writes new task knowledge back into the encoder without large weight changes. On ten CT and MRI tasks, this recipe beats per-task full fine-tuning and joint multi-task training, with an average Dice gain of 3.0% and an HD95 improvement of 10 mm over full fine-tuning, and it transfers better to two held-out tasks. The cost is about 5.9 extra hours of training over five tasks, a modest overhead the paper treats as acceptable.

Core claim

MedSeqFT claims that adapting a self-supervised foundation model to a stream of segmentation tasks should be sequential: fully fine-tune on the first task, then initialize each next task from the previous refined model. To avoid forgetting, it keeps a buffer of the downstream samples whose self-supervised loss under the original model is lowest (MDS), and applies K&G RFT—an MSE distillation from the frozen previous encoder during fine-tuning, followed by LoRA adapters trained on the new task and merged as W' = W + BA. This yields task-specialized models with minimal parameter change. The paper reports a 3.0% average Dice gain and 10 mm HD95 gain over full fine-tuning on five CT tasks, simila

What carries the argument

The load-bearing mechanism is a two-part retention scheme inside a sequential fine-tuning loop. Maximum Data Similarity (MDS) scores each downstream training sample by its averaged SSL loss under the original foundation model and keeps the K lowest-loss samples in a buffer, assuming low pretext loss marks proximity to the pretraining distribution. Knowledge and Generalization Retention Fine-Tuning (K&G RFT) then uses that buffer twice: a KD-based FFT stage adds an MSE distillation loss from the frozen previous encoder to the actively trained encoder, and a LoRA-based KD stage freezes both old and new encoders, trains low-rank adapters on the new task data to capture the residual, and reparam

Load-bearing premise

That MedSeqFT's edge over full fine-tuning comes from its sequential and knowledge-retention design rather than from the extra 5.9 hours of training it receives; the full-fine-tuning baseline is not given matching compute, and all numbers are single runs without variance.

What would settle it

Train the standard FFT baseline on the same five CT tasks for the same total budget (about 27 hours, matching MedSeqFT's 27.0 h, or the same number of update and distillation steps) and compare average Dice and HD95. If FFT closes the 3.0% gap, the advantage is explained by compute rather than sequential retention. Rerun both with several seeds to check whether the 3.0% margin is within run-to-run noise.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the reported gains hold, hospitals can add a new segmentation task to an already refined model instead of retraining a separate model from the original pretrained weights, and expect better Dice and boundary accuracy than isolated fine-tuning on that task.
  • The framework's small parameter changes (max average variation 0.016 for VoCo, 0.012 for UniMiSS+) suggest a single refined model can serve multiple tasks without a larger architecture or parameter count.
  • The refined model transfers better to unseen tasks, with the largest gains on tumor segmentation, implying that sequential refinement emphasizes lesion-relevant features that carry over.
  • The overhead is modest—about 5.9 extra hours and negligible added GPU memory over five tasks—so the strategy is practical for clinical settings where datasets arrive incrementally.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper fixes one task order for each sequence; order effects are untested. A natural extension is to measure whether starting with the most general or hardest task changes the benefits, since MDS buffers are built from the original model and later tasks may not protect earlier task knowledge equally.
  • MDS is essentially a proxy for closeness to the pretraining distribution via SSL loss. Other distribution-matching scores, such as feature-space distances or uncertainty estimates, could replace or augment it; the paper does not compare them, so the mechanism's specificity is an open question.
  • The transfer experiment evaluates only CT-to-CT transfer after CT refinement. Whether a model refined across mixed modalities transfers similarly to other MRI tasks is implied by the MRI results but not tested directly.
  • Because all reported comparisons are single runs, the 3.0% margin might partly reflect run-to-run noise; repeated-seed evaluations would tell whether sequential fine-tuning's advantage is as stable as claimed.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes MedSeqFT, a sequential fine-tuning framework for adapting 3D medical image segmentation foundation models to a stream of tasks. The method has two main components: Maximum Data Similarity (MDS) selection, which buffers downstream samples with low SSL loss under the original pre-trained model, and Knowledge and Generalization Retention Fine-Tuning (K&G RFT), which combines KD-based full fine-tuning with LoRA-based distillation and reparameterization to retain pre-trained knowledge. The method is evaluated on five CT and five MRI datasets using VoCo and UniMiSS+ backbones, against parallel fine-tuning (FFT, PEFT variants) and multi-task baselines (MTLoRA, UniSeg). The paper reports an average Dice improvement of 3.0% and HD95 reduction of 10 mm over FFT on the CT benchmark, and additional experiments on two unseen CT tasks for transferability.

Significance. If the central claim is true, sequential fine-tuning is a practically useful alternative to per-task and multi-task fine-tuning for clinical deployment, where tasks arrive incrementally. The paper attacks a real problem and includes a broad set of baselines, two backbones, ten fine-tuning tasks, and two transfer tasks, which is a solid empirical scope. The method is novel in combining MDS with a two-step KD/refinement procedure, and the idea of refining a foundation model progressively is well motivated. The main strengths are the scale of the empirical study and the explicit treatment of catastrophic forgetting. However, the current evidence has serious internal inconsistencies and does not isolate the claimed mechanism from additional training compute or run-to-run noise.

major comments (5)
  1. [Section V-C, Table VII] The headline improvement over FFT is confounded by training budget. Table VII reports FFT at 21.1 h and MedSeqFT at 27.0 h (+5.9 h, ~28%). Although Section IV-D states that baselines are trained for an equal number of iterations, MedSeqFT adds KD-based FFT and LoRA-based KD stages. No FFT baseline is trained for the equivalent compute budget, so the 3.0% Dice / 10 mm HD95 gains in Tables II, IV, and V may reflect extra optimization rather than the sequential/retention mechanisms. Please add a compute-matched FFT baseline or otherwise isolate the mechanism (e.g., train FFT for the same total budget, or ablate the extra stages at matched total iterations).
  2. [Section IV-D and Tables II-VI] All reported metric values are single runs with no variance, confidence intervals, or significance tests. The statement that MedSeqFT 'consistently outperforms' is therefore unsupported statistically; e.g., Table IV shows UniSeg outperforms MedSeqFT on Liver (79.3 vs 77.6), and Table II shows the gain on Liver is zero by construction. Please report multiple seeds (at least 3) and significance tests for the main comparisons, particularly against FFT.
  3. [Section IV-E, Table III] The ablation table contradicts the accompanying text. The text states that integrating MDS with Seq. FT consistently improves performance and that MDS beats Random selection. In Table III, however, Seq. FT + MDS yields 65.8 average Dice / 56.4 HD95, which is worse than vanilla Seq. FT (66.1 / 56.0), and Random selection yields 67.1 / 54.8, better than MDS. This undermines the claimed benefit of the MDS component. Please correct the table, the text, or the checkmark encoding, and re-evaluate the conclusion.
  4. [Section V-A and Table VI] The transferability summary is inconsistent with the data. Section V-A claims average improvements of 3.5% Dice and 9.7 mm HD95 for FFT w/ MedSeqFT over FFT. From Table VI: COVID-19-20 +1.3 Dice / -8.3 HD95; Kidney -0.1 / +0.5; Kidney Tumor +3.5 / -9.7. The means are approximately +1.6 Dice and -5.8 HD95, not +3.5 and -9.7. Please recompute or clarify which subset is being averaged.
  5. [Section III-B and III-D] The method definition is internally inconsistent. Section III-B says that for task 1, M0 is fine-tuned and then 'followed by LoRA-based KD and Reparameterization'; Section III-D states K&G RFT is used 'for all tasks after the first'. The experimental tables show MedSeqFT and FFT are identical on the first task (e.g., Liver in Table II), which is consistent with no LoRA-based KD on task 1 but contradicts Section III-B. Please clarify whether task 1 receives the K&G RFT steps and adjust the time breakdown in Table VII accordingly.
minor comments (5)
  1. [Section III-C] The buffer size K is never specified. Please state K per task and how it was chosen, including whether it is a hyperparameter or fixed.
  2. [Section IV-D] The reference to 'Table IV-F' should be 'Table IV'; there is no Table IV-F.
  3. [Section IV-F / Table IV] Adapter hyperparameters are labeled 'r=2' in Table IV, but Table II uses 'γ=2' for the compression factor. Please make the notation consistent.
  4. [Section IV-B / III-C] Please clarify the computational cost of the MDS selection: is the SSL loss averaged over 1000 forward passes per sample? If so, this is non-negligible and should be reported in the time budget.
  5. [Section V-B, Fig. 4] The observation that VoCo's shallow linear layers show no variation, while UniMiSS+ does, is left unexplained. Please provide a brief architectural explanation or a reference.

Circularity Check

0 steps flagged

No significant circularity: MedSeqFT's superiority claims are empirical comparisons against external baselines, not consequences of its definitions.

full rationale

The central claim—that sequential fine-tuning with MDS selection and K&G RFT improves Dice/HD95 over FFT and other fine-tuning strategies—is established by direct empirical comparisons in Tables II–VI against external baselines (LoRA, Adapter, Conv-Adapter, MTLoRA, UniSeg, etc.) on fixed test splits. The two proposed components are not defined in terms of the reported target metrics: MDS selects buffer samples by low SSL loss under M0 (Sec. III-C), which is an operational heuristic, and K&G RFT uses KD losses (Sec. III-D) whose parameters are not fitted to the reported Dice/HD95. Table III validates MDS against random selection and K&G RFT against vanilla Seq. FT, so the improvements are not forced by construction. Self-citations appear in related work and baselines (e.g., UniSeg [38]), but those comparisons are run head-to-head rather than imported as evidence. The clearest weakness is Table VII, which shows MedSeqFT requires 5.9 h more training than FFT (27.0 h vs 21.1 h), and all reported numbers are single runs with no variance; this is a fairness/robustness concern, not circularity, because the gains are not an algebraic consequence of the method's definitions. No load-bearing circular step is present.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The paper's method is a combination of standard building blocks (data selection, KD, LoRA) plus an empirical claim; the main unstated dependencies are the MDS selection heuristic, the compute-matched comparison, and the single training order.

free parameters (3)
  • Buffer size K per task = not reported
    The MDS strategy selects the K samples with lowest SSL loss for each task (Section III-C), but K is never specified; the method's performance likely depends on it. Ablations compare MDS vs random 'of the same size' but never state the size.
  • LoRA rank r = 2
    Set to 2 'to constrain weight modifications' (Section IV-B); chosen by hand, not swept or justified.
  • SSL loss averaging runs = 1000
    The SSL loss per sample is averaged over 1000 runs (Section III-C); this is a computational budget choice, not an outcome of analysis.
axioms (5)
  • ad hoc to paper Samples with low SSL loss under M0 are representative of the pre-training distribution
    Core premise of MDS (Section III-C); no theoretical or empirical justification given beyond the ablation.
  • domain assumption MSE distillation to the previous encoder's outputs preserves prior knowledge without harming new-task learning
    Standard knowledge-distillation assumption, used in Section III-D; not proved for this setting.
  • domain assumption The additional training time of MedSeqFT does not account for the observed gains
    Implicit in the comparison against FFT (Tables II-VI); FFT is not trained for equivalent compute (Table VII shows +5.9h).
  • domain assumption The task order (Liver, HepaV, Pancreas, Colon, Lung) does not materially affect the conclusions
    Only one order is tested; sequential learning is order-sensitive in general, and no sensitivity analysis is provided.
  • domain assumption The SSL loss estimates are stable over 1000 stochastic runs
    The MDS selection depends on this estimator (Section III-C), but no variance of the estimator is reported.

pith-pipeline@v1.3.0-alltime-deepseek · 17862 in / 13241 out tokens · 126058 ms · 2026-08-05T04:26:49.552888+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of MedSeqFT: Sequential Fine-tuning Foundation Models for 3D Medical Image Segmentation." pith.science (2026). https://pith.science/paper/UPHTRV6N

@misc{pith2026250906096,
  author       = {Pith},
  title        = {Pith review of: MedSeqFT: Sequential Fine-tuning Foundation Models for 3D Medical Image Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UPHTRV6N}},
  note         = {Machine review of arXiv:2509.06096}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Foundation models have become a promising paradigm for advancing medical image analysis, particularly for segmentation tasks where downstream applications often emerge sequentially. Existing fine-tuning strategies, however, remain limited: parallel fine-tuning isolates tasks and fails to exploit shared knowledge, while multi-task fine-tuning requires simultaneous access to all datasets and struggles with incremental task integration. To address these challenges, we propose MedSeqFT, a sequential fine-tuning framework that progressively adapts pre-trained models to new tasks while refining their representational capacity. MedSeqFT introduces two core components: (1) Maximum Data Similarity (MDS) selection, which identifies downstream samples most representative of the original pre-training distribution to preserve general knowledge, and (2) Knowledge and Generalization Retention Fine-Tuning (K&G RFT), a LoRA-based knowledge distillation scheme that balances task-specific adaptation with the retention of pre-trained knowledge. Extensive experiments on two multi-task datasets covering ten 3D segmentation tasks demonstrate that MedSeqFT consistently outperforms state-of-the-art fine-tuning strategies, yielding substantial performance gains (e.g., an average Dice improvement of 3.0%). Furthermore, evaluations on two unseen tasks (COVID-19-20 and Kidney) verify that MedSeqFT enhances transferability, particularly for tumor segmentation. Visual analyses of loss landscapes and parameter variations further highlight the robustness of MedSeqFT. These results establish sequential fine-tuning as an effective, knowledge-retentive paradigm for adapting foundation models to evolving clinical tasks. Code will be released.

Figures

Figures reproduced from arXiv: 2509.06096 by He Zhang, Ting Dang, Xiangde Luo, Yanning Zhang, Yicheng Wu, Yiwen Ye, Yong Xia, Ziyang Chen.

Figure 1
Figure 1. Figure 1: Three strategies for fine-tuning self-supervised learning (SSL) models on multi-task datasets. (a) Parallel fine-tuning: The SSL model M0 is independently fine-tuned on each dataset from the dataset pool, with all or partial parameters (PP) updated. (b) Multi-task Fine-tuning: Multiple tasks are fine-tuned jointly, with M0 shared across tasks, while the PP can either be task-shared or task-independent. (c)… view at source ↗
Figure 2
Figure 2. Figure 2: Pipeline of the proposed MedSeqFT framework. MedSeqFT adopts a sequential fine-tuning strategy to progressively adapt a pre-trained model to multiple downstream segmentation tasks while continually refining its representations. To preserve strong generalization across tasks, we introduce two key components: the Maximum Data Similarity (MDS) strategy and the Knowledge and Generalization Retention Fine-Tunin… view at source ↗
Figure 3
Figure 3. Figure 3: Visualization of segmentation results obtained by SSL, Conv￾Adapter, UniSeg, MedSeqFT, and ground truths (GTs) on six datasets. The lesion and non-lesion regions are colored green and red, respec￾tively. The enlarged region is displayed in the right corner of each image. B. Implementations All experiments were implemented using two learning frameworks built upon the nnU-Net architecture [18]: (1) supervise… view at source ↗
Figure 4
Figure 4. Figure 4: The variation was computed by taking the absolute [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 4
Figure 4. Figure 4: Visualization of parameter variation for (a) VoCo and (b) UniMiSS+ after refinement on the CT multi-task fine-tuning dataset using MedSeqFT. Each point corresponds to a specific encoder layer (n denotes the n-th encoder layer). Parameter variation is computed as the average absolute change in parameter values before and after refinement. Full Fine-Tuning MedSeqFT [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Visualization of the loss landscapes [20] for FFT and MedSeqFT, both trained on the Lung dataset. Colors represent loss values, increas￾ing progressively from blue (low loss) to red (high loss). TABLE VII TIME COST OF FFT AND MEDSEQFT ON THE CT MULTI-TASK DATASET. VOCO IS THE BACKBONE. H: HOURS. Method Training Time (5 tasks) FFT ∼21.1 h MedSeqFT FFT (The first stage) KD-based FFT LoRA-based KD Sum ∼4.2h ∼… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. C^2GR: Coupled Comprehensive Generative Replay for a Continually Learnable Universal Segmentation Model

    cs.CV 2026-06 unverdicted novelty 7.0

    C^2GR uses Bayesian Joint Diffusion for coupled image-mask synthesis and Relation-aware Unified Prompt Synchronization to reduce forgetting in continual universal segmentation, showing a 2.44% performance drop versus ...

Reference graph

Works this paper leans on

47 extracted references · 43 canonical work pages · cited by 1 Pith paper · 4 internal anchors

  1. [1]

    Mtlora: Low-rank adaptation approach for efficient multi-task learning,

    A. Agiza, M. Neseem, and S. Reda, “Mtlora: Low-rank adaptation approach for efficient multi-task learning,” inIEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2024, pp. 16 196–16 205

    work page 2024

  2. [2]

    The medical segmentation decathlon,

    M. Antonelli, A. Reinke, S. Bakas, K. Farahani, A. Kopp-Schneider, B. A. Landman, G. Litjens, B. Menze, O. Ronneberger, R. M. Summers et al., “The medical segmentation decathlon,”Nature communications, vol. 13, no. 1, p. 4128, 2022

    2022

  3. [3]

    Medical image segmentation review: The success of u-net,

    R. Azad, E. K. Aghdam, A. Rauland, Y . Jia, A. H. Avval, A. Bozorgpour, S. Karimijafarbigloo, J. P. Cohen, E. Adeli, and D. Merhof, “Medical image segmentation review: The success of u-net,”IEEE Trans. Pattern Anal. Mach. Intell., 2024. 10 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. XX, NO. XX, XXXX 2020

    work page 2024

  4. [4]

    The liver tumor segmentation benchmark (lits),

    P. Bilic, P. Christ, H. B. Li, E. V orontsov, A. Ben-Cohen, G. Kaissis, A. Szeskin, C. Jacobs, G. E. H. Mamani, G. Chartrandet al., “The liver tumor segmentation benchmark (lits),”Med. Image Anal., vol. 84, p. 102680, 2023

    work page 2023

  5. [5]

    Efficient conditional pre-training for transfer learning,

    S. Chakraborty, B. Uzkent, K. Ayush, K. Tanmay, E. Sheehan, and S. Ermon, “Efficient conditional pre-training for transfer learning,” in IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2022, pp. 4241–4250

    work page 2022

  6. [6]

    Conv-adapter: Exploring parameter efficient transfer learning for convnets,

    H. Chen, R. Tao, H. Zhang, Y . Wang, X. Li, W. Ye, J. Wang, G. Hu, and M. Savvides, “Conv-adapter: Exploring parameter efficient transfer learning for convnets,” inIEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2024, pp. 1551–1561

    work page 2024

  7. [7]

    Expert tumor annotations and radiomic features for the ispy1/acrin 6657 trial data collection,

    R. Chitalia, S. Pati, M. Bhalerao, S. Thakur, N. Jahani, J. Belenky, E. McDonald, J. Gibbs, D. Newitt, and N. Hylton, “Expert tumor annotations and radiomic features for the ispy1/acrin 6657 trial data collection,”The Cancer Imaging Archive, 2022

    work page 2022

  8. [8]

    Efficient adaptation of large vision transformer via adapter re-composing,

    W. Dong, D. Yan, Z. Lin, and P. Wang, “Efficient adaptation of large vision transformer via adapter re-composing,”Adv. Neural Inf. Process. Syst., vol. 36, 2024

    work page 2024

  9. [9]

    Training like a medical resident: Context-prior learning toward universal medical image segmentation,

    Y . Gao, “Training like a medical resident: Context-prior learning toward universal medical image segmentation,” inIEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2024, pp. 11 194–11 204

    work page 2024

  10. [10]

    Dira: Discriminative, restorative, and adversarial learning for self-supervised medical image analysis,

    F. Haghighi, M. R. H. Taher, M. B. Gotway, and J. Liang, “Dira: Discriminative, restorative, and adversarial learning for self-supervised medical image analysis,” inIEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2022, pp. 20 824–20 834

    work page 2022

  11. [11]

    Masked autoencoders are scalable vision learners,

    K. He, X. Chen, S. Xie, Y . Li, P. Doll ´ar, and R. Girshick, “Masked autoencoders are scalable vision learners,” inIEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2022, pp. 16 000–16 009

    work page 2022

  12. [12]

    Geometric visual similarity learning in 3d medical image self-supervised pre-training,

    Y . He, G. Yang, R. Ge, Y . Chen, J.-L. Coatrieux, B. Wang, and S. Li, “Geometric visual similarity learning in 3d medical image self-supervised pre-training,” inIEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2023, pp. 9538–9547

    work page 2023

  13. [13]

    The state of the art in kidney and kidney tumor segmentation in contrast-enhanced ct imaging: Results of the kits19 challenge,

    N. Heller, F. Isensee, K. H. Maier-Hein, X. Hou, C. Xie, F. Li, Y . Nan, G. Mu, Z. Lin, M. Hanet al., “The state of the art in kidney and kidney tumor segmentation in contrast-enhanced ct imaging: Results of the kits19 challenge,”Med. Image Anal., vol. 67, p. 101821, 2021

    work page 2021

  14. [14]

    Isles 2022: A multi-center magnetic resonance imaging stroke lesion segmentation dataset,

    M. R. Hernandez Petzsche, E. de la Rosa, U. Hanning, R. Wiest, W. Valenzuela, M. Reyes, M. Meyer, S.-L. Liew, F. Kofler, I. Ezhov et al., “Isles 2022: A multi-center magnetic resonance imaging stroke lesion segmentation dataset,”Scientific data, vol. 9, no. 1, p. 762, 2022

    work page 2022

  15. [15]

    Parameter-efficient transfer learning for nlp,

    N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for nlp,” inInt. Conf. Mach. Learn.PMLR, 2019, pp. 2790– 2799

    work page 2019

  16. [16]

    LoRA: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” inInt. Conf. Learn. Represent., 2022

    work page 2022

  17. [17]

    Fine-grained prompt tuning: A parameter and memory efficient transfer learning method for high-resolution medical image classification,

    Y . Huang, P. Cheng, R. Tam, and X. Tang, “Fine-grained prompt tuning: A parameter and memory efficient transfer learning method for high-resolution medical image classification,” inMed. Image Comput. Comput.-Assist. Intervent.Springer, 2024, pp. 120–130

    work page 2024

  18. [18]

    nnu-net: a self-configuring method for deep learning-based biomedical image segmentation,

    F. Isensee, P. F. Jaeger, S. A. Kohl, J. Petersen, and K. H. Maier-Hein, “nnu-net: a self-configuring method for deep learning-based biomedical image segmentation,”Nature methods, vol. 18, no. 2, pp. 203–211, 2021

    2021

  19. [19]

    Standardized assessment of automatic segmentation of white matter hyperintensities and results of the wmh segmentation challenge,

    H. J. Kuijf, J. M. Biesbroek, J. De Bresser, R. Heinen, S. Andermatt, M. Bento, M. Berseth, M. Belyaev, M. J. Cardoso, A. Casamitjanaet al., “Standardized assessment of automatic segmentation of white matter hyperintensities and results of the wmh segmentation challenge,”IEEE Trans. Med. Imag., vol. 38, no. 11, pp. 2556–2568, 2019

    work page 2019

  20. [20]

    Visualizing the loss landscape of neural nets,

    H. Li, Z. Xu, G. Taylor, C. Studer, and T. Goldstein, “Visualizing the loss landscape of neural nets,”Adv. Neural Inf. Process. Syst., vol. 31, 2018

    work page 2018

  21. [21]

    Aligndet: Aligning pre-training and fine-tuning in object detection,

    M. Li, J. Wu, X. Wang, C. Chen, J. Qin, X. Xiao, R. Wang, M. Zheng, and X. Pan, “Aligndet: Aligning pre-training and fine-tuning in object detection,” inIEEE/CVF Int. Conf. Comput. Vis., 2023, pp. 6866–6876

    work page 2023

  22. [22]

    Ccq: cross-class query network for partially labeled organ segmentation,

    X. Liu, B. Wen, and S. Yang, “Ccq: cross-class query network for partially labeled organ segmentation,” inAAAI Conf. Artif. Intell., vol. 37, no. 2, 2023, pp. 1755–1763

    work page 2023

  23. [23]

    Ditask: Multi-task fine-tuning with diffeomorphic transformations,

    K. S. I. Mantri, C.-B. Sch ¨onlieb, B. Ribeiro, C. Baskin, and M. Eliasof, “Ditask: Multi-task fine-tuning with diffeomorphic transformations,” in IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2025

    work page 2025

  24. [24]

    Rapid artificial intelligence solutions in a pandemic—the covid-19-20 lung ct lesion segmentation challenge,

    H. R. Roth, Z. Xu, C. Tor-D ´ıez, R. S. Jacob, J. Zember, J. Molto, W. Li, S. Xu, B. Turkbey, E. Turkbeyet al., “Rapid artificial intelligence solutions in a pandemic—the covid-19-20 lung ct lesion segmentation challenge,”Med. Image Anal., vol. 82, p. 102605, 2022

    work page 2022

  25. [25]

    Med-tuning: A new parameter-efficient tuning framework for medical volumetric segmentation,

    J. Shen, W. Wang, C. Chen, J. Jiao, J. Liu, Y . Zhang, S. Song, and J. Li, “Med-tuning: A new parameter-efficient tuning framework for medical volumetric segmentation,” inMedical Imaging with Deep Learning

    work page

  26. [26]

    SG-MIM: Structured Knowledge Guided Efficient Pre-training for Dense Prediction

    S. Son, H. Choi, and D. Min, “Sg-mim: Structured knowledge guided efficient pre-training for dense prediction,”arXiv preprint arXiv:2409.02513, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  27. [27]

    Self-supervised pre-training of swin transformers for 3d medical image analysis,

    Y . Tang, D. Yang, W. Li, H. R. Roth, B. Landman, D. Xu, V . Nath, and A. Hatamizadeh, “Self-supervised pre-training of swin transformers for 3d medical image analysis,” inIEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2022, pp. 20 730–20 740

    work page 2022

  28. [28]

    Hydralora: An asymmetric lora architecture for efficient fine-tuning,

    C. Tian, Z. Shi, Z. Guo, L. Li, and C. Xu, “Hydralora: An asymmetric lora architecture for efficient fine-tuning,”Adv. Neural Inf. Process. Syst., 2024

    work page 2024

  29. [29]

    Training dataset for hntsmrg 2024 challenge,

    K. Wahid, C. Dede, M. Naser, and C. Fuller, “Training dataset for hntsmrg 2024 challenge,” May 2024. [Online]. Available: https://doi.org/10.5281/zenodo.11199559

    doi:10.5281/zenodo.11199559 2024

  30. [30]

    Bridging the gap between recognition-level pre-training and commonsensical vision- language tasks,

    Y . Wan, Y . Ma, H. You, Z. Wang, and S.-F. Chang, “Bridging the gap between recognition-level pre-training and commonsensical vision- language tasks,” inCSRR, 2022, pp. 23–35

    work page 2022

  31. [31]

    Task-robust pre-training for worst-case downstream adaptation,

    J. Wang, Y . Chen, X. Xie, C. Fang, and Z. Lin, “Task-robust pre-training for worst-case downstream adaptation,”Adv. Neural Inf. Process. Syst., vol. 36, 2024

    work page 2024

  32. [32]

    Large-scale 3d medical im- age pre-training with geometric context priors,

    L. Wu, J. Zhuang, and H. Chen, “Large-scale 3d medical im- age pre-training with geometric context priors,”arXiv preprint arXiv:2410.09890, 2024

    Pith/arXiv arXiv 2024

  33. [33]

    V oco: A simple-yet-effective volume contrastive learning frame- work for 3d medical image analysis,

    ——, “V oco: A simple-yet-effective volume contrastive learning frame- work for 3d medical image analysis,” inIEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2024, pp. 22 873–22 882

    work page 2024

  34. [34]

    Refs: A hybrid pre-training paradigm for 3d medical image segmentation,

    Y . Xie, J. Zhang, L. Liu, H. Wang, Y . Ye, J. Verjans, and Y . Xia, “Refs: A hybrid pre-training paradigm for 3d medical image segmentation,” Med. Image Anal., vol. 91, p. 103023, 2024

    work page 2024

  35. [35]

    Unimiss+: Universal medical self-supervised learning from cross-dimensional unpaired data,

    Y . Xie, J. Zhang, Y . Xia, and Q. Wu, “Unimiss+: Universal medical self-supervised learning from cross-dimensional unpaired data,”IEEE Trans. Pattern Anal. Mach. Intell., 2024

    work page 2024

  36. [36]

    MedUniSeg: 2D and 3D Medical Image Segmentation via a Prompt-driven Universal Model

    Y . Ye, Z. Chen, J. Zhang, Y . Xie, and Y . Xia, “Meduniseg: 2d and 3d medical image segmentation via a prompt-driven universal model,” arXiv preprint arXiv:2410.05905, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  37. [37]

    Continual self- supervised learning: Towards universal multi-modal medical data repre- sentation learning,

    Y . Ye, Y . Xie, J. Zhang, Z. Chen, Q. Wu, and Y . Xia, “Continual self- supervised learning: Towards universal multi-modal medical data repre- sentation learning,” inIEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2024, pp. 11 114–11 124

    work page 2024

  38. [38]

    Uniseg: A prompt-driven universal segmentation model as well as a strong representation learner,

    Y . Ye, Y . Xie, J. Zhang, Z. Chen, and Y . Xia, “Uniseg: A prompt-driven universal segmentation model as well as a strong representation learner,” inMed. Image Comput. Comput.-Assist. Intervent.Springer, 2023, pp. 508–518

    work page 2023

  39. [39]

    Desd: Self-supervised learning with deep self-distillation for 3d medical image segmentation,

    Y . Ye, J. Zhang, Z. Chen, and Y . Xia, “Desd: Self-supervised learning with deep self-distillation for 3d medical image segmentation,” inMed. Image Comput. Comput.-Assist. Intervent.Springer, 2022, pp. 545–555

    work page 2022

  40. [40]

    Cads: A self-supervised learner via cross-modal alignment and deep self-distillation for ct volume segmentation,

    ——, “Cads: A self-supervised learner via cross-modal alignment and deep self-distillation for ct volume segmentation,”IEEE Trans. Med. Imag., vol. 44, no. 1, pp. 118–129, 2025

    work page 2025

  41. [41]

    5%¿ 100%: Breaking performance shackles of full fine-tuning on visual recognition tasks,

    D. Yin, L. Hu, B. Li, Y . Zhang, and X. Yang, “5%¿ 100%: Breaking performance shackles of full fine-tuning on visual recognition tasks,” inIEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2025, pp. 20 071– 20 081

    work page 2025

  42. [42]

    AC-Norm: Effective Tuning for Medical Image Analysis via Affine Collaborative Normalization

    C. Zhang, Y . Yang, H. Zheng, and Y . Gu, “Ac-norm: Effective tuning for medical image analysis via affine collaborative normalization,”arXiv preprint arXiv:2307.15282, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  43. [43]

    Dodnet: Learning to segment multi-organ and tumors from multiple partially labeled datasets,

    J. Zhang, Y . Xie, Y . Xia, and C. Shen, “Dodnet: Learning to segment multi-organ and tumors from multiple partially labeled datasets,” in IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2021, pp. 1195–1204

    work page 2021

  44. [44]

    Unified multi-modal di- agnostic framework with reconstruction pre-training and heterogeneity- combat tuning,

    Y . Zhang, L. Pan, Q. Yang, T. Li, and Z. Chen, “Unified multi-modal di- agnostic framework with reconstruction pre-training and heterogeneity- combat tuning,”IEEE J. Biomed. Health Inform., 2024

    work page 2024

  45. [45]

    Large-Scale Multi-Center CT and MRI Segmentation of Pancreas with Deep Learning

    Z. Zhang, E. Keles, G. Durak, Y . Taktak, O. Susladkar, V . Gorade, D. Jha, A. C. Ormeci, A. Medetalibeyoglu, L. Yaoet al., “Large-scale multi-center ct and mri segmentation of pancreas with deep learning,” arXiv preprint arXiv:2405.12367, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  46. [46]

    Models genesis,

    Z. Zhou, V . Sodha, J. Pang, M. B. Gotway, and J. Liang, “Models genesis,”Med. Image Anal., vol. 67, p. 101840, 2021

    work page 2021

  47. [47]

    Uni- perceiver: Pre-training unified architecture for generic perception for zero-shot and few-shot tasks,

    X. Zhu, J. Zhu, H. Li, X. Wu, H. Li, X. Wang, and J. Dai, “Uni- perceiver: Pre-training unified architecture for generic perception for zero-shot and few-shot tasks,” inIEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2022, pp. 16 804–16 815

    work page 2022