Pith. sign in

REVIEW 4 major objections 4 minor 57 references

SAM+D claims that a pair of lightweight modules—depth-routed LoRA experts and a zero-parameter depth shift—can lift frozen SAM-family models by one dimension, giving 3D segmentation from SAM and end-to-end 3D+T segmentation from SAM2 while

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 14:55 UTC pith:K7OGO4G2

load-bearing objection Useful parameter-efficient lifting of SAM to 3D; the 3D story is solid, the 4D headline is broader than what the experiments actually support. the 4 major comments →

arxiv 2607.29033 v1 pith:K7OGO4G2 submitted 2026-07-31 cs.CV

SAM+D: Parameter-Efficient Dimensional Lifting of SAM-Family Models via Depth-Routed LoRA and Depth Shifting

Yu Song , Hao Sun , Shiyu Teng , Ikuko Nishikawa , Yen-wei Chen This is my paper
classification cs.CV
keywords SAMLoRAparameter-efficient fine-tuning3D segmentation4D segmentationdepth routingcell trackingmedical image segmentation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that a pre-trained 2D segmentation foundation model can be upgraded to volumetric and spatiotemporal understanding without retraining the whole network. It introduces two modules, DRLoRA (depth-routed low-rank experts that adapt features according to a slice's depth position) and DSM (a zero-parameter channel shift between neighboring slices), and inserts them into the frozen transformer blocks of SAM and SAM2. The claim is that this lifts SAM from 2D to 3D volumes and, for the first time via parameter-efficient fine-tuning, lifts SAM2 from 2D video to 3D+T spatiotemporal data, with only 2.8% and 3.7% trainable parameters respectively. If true, it means interactive volumetric segmentation and cell tracking can be built on top of frozen 2D foundation models with a tiny training budget, matching or beating fully-trained 3D networks.

Core claim

The central discovery is that dimensional lifting of SAM-family models reduces to two cheap, model-agnostic interventions in each frozen transformer block: routing low-rank expert updates by normalized depth position (DRLoRA) and exchanging a fraction of feature channels across adjacent slices before self-attention (DSM). The same design transfers unchanged from SAM (2D→3D) to SAM2 (2D+T→3D+T), where temporal propagation is delegated to SAM2's frozen memory architecture with an additional LoRA on its memory attention. On four CT tumor benchmarks and one cell-tracking challenge, this yields competitive or superior Dice/NSD and OPCTB scores compared with fully-supervised 3D networks, interacti

What carries the argument

DRLoRA is a mixture-of-experts LoRA where a tiny MLP router maps normalized depth z to a softmax over E=4 low-rank experts applied to the frozen Q and V projections; the router is roughly 0.14K parameters and needs no load-balancing loss. DSM re-purposes the Temporal Shift Module to shift α=25% of channels from slice i−1 and i+1 into each slice before self-attention, at zero added parameters and zero multiply-accumulate operations; over layers this accumulates cross-slice context. Together they provide depth-conditioned adaptation and inter-slice exchange inside otherwise frozen transformers.

Load-bearing premise

The 4D claim stands on the assumption that SAM2's video memory mechanism, trained on 2D+T, can carry cell identity through dividing and newly appearing objects in 3D+T volumes after only a LoRA adaptation of the memory attention, without explicit tracking supervision.

What would settle it

Run SAM2+D end-to-end from a single prompt on a 3D+T sequence where cells divide repeatedly and new cells enter; if track identity is lost at division events (one daughter inherits the mother's mask, or the track switches identity), the end-to-end 4D claim fails. A concrete check is reporting the CTC TRA metric on a test sequence with high mitosis density using only the initial prompt and no external detector.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • The same modules lift SAM to 3D and SAM2 to 3D+T without architectural redesign, implying one adapter recipe can serve the whole SAM family.
  • DRLoRA's depth-based routing outperforms content-based MoE-LoRA routers by +7–9 Dice on LiTS under a matched parameter budget and avoids the auxiliary balancing losses those routers require.
  • DSM with zero parameters beats parameter-matched cross-slice attention and a Conv3D adapter on LiTS, suggesting inter-slice exchange is better done by channel shifting than by learned attention when the budget is tiny.
  • On the 4D cell-tracking benchmark, SAM2+D raises the SEG score above the Cellpose ceiling (56.79 vs roughly 54), showing temporal propagation can refine per-frame detections beyond the detector's own quality.
  • In the single-point prompt setting, SAM+D matches or exceeds fully supervised 3D networks such as nnU-Net across four CT datasets while training only about 2.8% of parameters.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The depth router conditions on normalized position, so the method's gains may depend on consistent resampling of volumes to a fixed depth; a natural stress test is varying the depth-slice count and watching where the advantage vanishes.
  • Because DSM is a zero-parameter operator, it could be bolted onto any 2D transformer to inject cross-slice context, and in principle onto the time axis as a cheap way to make 2D image models handle video, as the paper itself suggests.
  • The 4D pipeline still needs an external detector for cell divisions and new entries; fully automatic end-to-end tracking from a single prompt remains untested, so the 'end-to-end' claim is scoped to single-object or externally detected multi-object settings.
  • On 3D+T data with very dense mitosis and frequent appearances, identity propagation through the LoRA-adapted memory attention could be the failure mode; a targeted evaluation on high-mitosis sequences would bound the claim.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes SAM+D, a parameter-efficient method to adapt SAM-family models to 3D volumetric and 3D+T spatiotemporal segmentation by inserting Depth-Routed LoRA (DRLoRA) experts and Depth Shift Modules (DSM) into frozen transformer blocks. For 3D, SAM is lifted from 2D to 3D; for 4D, SAM2 is lifted from 2D+T to 3D+T, with LoRA also injected into memory attention. The paper reports competitive results on four CT benchmarks (KiTS, Pancreas, LiTS, Colon) and on a CTC fluorescence dataset (Fluo-N3DH-SIM+), with only ~2.8% (SAM) and ~3.7% (SAM2) trainable parameters. The main claimed novelty is 'end-to-end 4D segmentation for the first time via PEFT.'

Significance. If substantiated, the paper would provide a simple, model-agnostic recipe for lifting SAM-family models to higher-dimensional data with very few trainable parameters, which is practically valuable. The 3D results are strong (e.g., KiTS Dice 84.74 vs nnU-Net 73.09 with a single point prompt). The release of code, detailed hyperparameters, and an extra contamination-free private dataset are commendable. However, the headline 'end-to-end 4D' claim is not supported by the experimental protocol as written, and the lack of statistical validation weakens the quantitative claims.

major comments (4)
  1. [§3.5, §4.1, §B.2] The 'end-to-end 4D' claim in the abstract/contributions is contradicted by the evaluation protocol. The Cellpose mode (Table 2) relies on an external detector, Hungarian association, volume filtering, and division detection (§B.2), and inference 'issues a new point prompt for each newly dividing or appeared cell' (§4.1). The paper's own Discussion (§5) concedes fully automatic tracking is difficult without an external detector. Thus the reported SEG/TRA numbers are not produced end-to-end. Please revise the claim or provide an evaluation that does not depend on external detections and re-prompting.
  2. [§4.2, Table 1] The default SAM prompt decoder is selected by highest average Dice/NSD on the four test sets. This is test-set selection and can bias the headline numbers upward. In addition, hyperparameters α, K, r are tuned on LiTS (Table 3) and the same LiTS results are reported as headline. Please select hyperparameters/decoder on a validation split or clearly state and justify the selection rule.
  3. [Tables 1 and 2] No error bars, confidence intervals, or significance tests are provided. Table 2 is based on a single test sequence (Seq02) and a single run; the SEG gain over the Cellpose baseline (56.79 vs 54.19) may be within noise. Multiple seeds or bootstrapped intervals are needed to support the claims of superiority.
  4. [Abstract, §4.1] The abstract says 'In both settings SAM+D achieves competitive or superior results under the single-point prompt setting'. The 4D setting is not single-point: each newly dividing or appeared cell receives a new prompt (§4.1). Please clarify that single-point refers to the 3D setting only, or specify the prompt budget for the 4D experiments.
minor comments (4)
  1. [§4.1 vs Table 4] The trainable parameter count is inconsistent: §4.1 says ~2.36M/83.78M (~2.8%), while Table 4 totals 2,571,042 parameters, which is ~3.1% of 83.78M. Please reconcile.
  2. [§3.1, §3.5] The claim that the 'same pair' of modules lifts SAM and SAM2 without redesign is not exact: the 4D setting also adds memory-attention LoRA and uses depth-pooled features (§3.5). Please describe this as the same core blocks plus a small per-setting interface.
  3. [Table 3] Notation is inconsistent: the number of experts is E in §3.2 but K in Table 3(c). Please unify.
  4. [Fig. 1 caption] The caption says 'prompt-free tracking at t > 0', but §4.1 describes re-prompting at divisions and new entries. Please reconcile the wording.

Circularity Check

0 steps flagged

No load-bearing circularity; the only flagged issue is a claim–protocol mismatch in the 4D end-to-end narrative, not a definitional reduction.

full rationale

Systematic walk of the claimed derivation chain found no step in which an output is defined from an input or a fitted parameter is renamed as a prediction. DRLoRA (Eqs. 1–4) is a learned depth-conditioned combination of standard LoRA experts; the router takes normalized depth z, not the target segmentation, as input. DSM (Eq. 5) is a zero-parameter channel shift borrowed from TSM, explicitly acknowledged as not a new operator. The 3D lifting (Sec. 3.4) and 4D lifting (Sec. 3.5) are architectural insertions trained with DiceCE against external benchmarks (KiTS, Pancreas, LiTS, Colon, Fluo-N3DH-SIM+), so the reported numbers are externally evaluated, not tautological. The few self-citations (refs [19], [43]) appear only as related-work context (LoRA in medical segmentation; patch-free 3D segmentation) and are not load-bearing for the main derivation; no uniqueness theorem or prior-work ansatz is invoked to force the design. One caveat deserves explicit flagging but is not circular: the abstract's unconditional "end-to-end 4D (3D+T) spatiotemporal segmentation" is contradicted by the paper's own methodology and limitations. Sec. 4.1 states "Our tuned SAM2 handles these events by issuing a new point prompt for each newly dividing or appeared cell," and Supp. B.2 describes a three-stage pipeline where Cellpose discovers cells/divisions and Hungarian association, volume filtering, division detection, and IoU/volume consistency checks are applied. Sec. 5 concedes: "frequent cell divisions and new entries require per-event prompting, increasing inference cost and making fully automatic tracking difficult without an external detector." This is an overclaim/scope mismatch rather than a derivation that reduces to its inputs; the 4D numbers are produced by a tracking-by-detection pipeline, not by a standalone end-to-end model. Because the central 3D results and the DRLoRA/DSM mechanism are self-contained against external benchmarks, the circularity score stays low.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 2 invented entities

The method has no mathematical derivation; its central claim is empirical. The main free parameters are model hyperparameters selected on LiTS and hand-set inference thresholds for the 4D tracking pipeline. The core assumptions are that LoRA can capture depth-dependent behavior, that depth-shifting is sufficient inter-slice communication, and that SAM2's memory generalizes to 3D+T.

free parameters (5)
  • DRLoRA rank r = 16
    Chosen by ablation on LiTS (Table 3d); values 4, 8, 16, 32 tested, 16 best.
  • DRLoRA expert count E = 4
    Chosen by ablation on LiTS (Table 3c); 1, 2, 4, 8 tested, 4 best.
  • DSM shift ratio α = 0.25
    Chosen by ablation on LiTS (Table 3b); 0.125, 0.25, 0.5 tested, 0.25 best.
  • Decoder selection (SAM prompt decoder) = SAM prompt decoder
    Selected as default after comparing three decoder designs on the test sets; it achieved highest average Dice/NSD. This is model selection on test data.
  • 4D inference thresholds = centroid 50 vox, min volume 100, dormant 3 frames, etc.
    Hand-specified association and consistency thresholds in the Cellpose+SAM2 pipeline (Table 5); tuned on the validation sequence.
axioms (4)
  • domain assumption Low-rank LoRA updates can capture volumetric-specific adaptation in frozen transformer blocks.
    Relies on the LoRA hypothesis (§3.2) and prior PEFT literature; not proven for 3D/4D in this paper.
  • domain assumption Shifting a fraction of channels between adjacent depth slices before attention provides useful cross-slice context at zero parameter cost.
    This is the core design assumption behind DSM (§3.3); the paper evaluates it empirically rather than deriving it.
  • domain assumption SAM2's video memory mechanism, with LoRA on memory attention, can propagate object identity in 3D+T data trained by per-frame DiceCE without explicit tracking supervision.
    Load-bearing for the 4D claim; stated in §3.5 as the temporal-propagation strategy.
  • domain assumption Baseline numbers from 3DSAM-adapter [15] and other cited papers are valid and comparable under the same splits.
    The main comparison table imports baseline numbers from prior publications rather than re-running all methods.
invented entities (2)
  • DRLoRA (Depth-Routed LoRA experts) independent evidence
    purpose: Provide depth-position-dependent low-rank updates to frozen Q and V projections.
    Evaluated through ablations and comparisons on four CT benchmarks and CTC; code release is promised.
  • DSM (Depth Shift Module) independent evidence
    purpose: Exchange feature channels between neighboring depth slices before self-attention at zero added parameters/FLOPs.
    Compared against cross-slice attention and Conv3D adapters in supplementary Table 8; contributes gains in the LiTS ablation.

pith-pipeline@v1.3.0-daily-deepseek · 18100 in / 9529 out tokens · 97840 ms · 2026-08-03T14:55:57.471263+00:00 · methodology

0 comments
read the original abstract

Existing methods for adapting 2D foundation models such as SAM to 3D volumes either process slices independently---ignoring inter-slice context---or require substantial architectural changes and retraining. In this paper, we present \textbf{SAM+D}, a parameter-efficient framework that lifts SAM-family models by one spatial dimension---enabling 3D volumetric segmentation from 2D SAM and, for the first time via parameter-efficient fine-tuning, end-to-end 4D (3D+T) spatiotemporal segmentation from video-based SAM2---while keeping the vast majority of pre-trained parameters frozen. SAM+D introduces two lightweight, model-agnostic modules into frozen transformer blocks: (1)~\textbf{Depth-Routed LoRA (DRLoRA)} experts with learned routing for spatially adaptive low-rank updates, and (2)~\textbf{Depth Shift Modules (DSM)} for cross-slice feature exchange at zero additional parameter cost. Together, they provide volume-level context while tuning only ${\sim}$2.8\% of parameters for SAM and ${\sim}$3.7\% for SAM2. We evaluate SAM+D in two distinct settings, each lifting the base model by one spatial dimension: 3D segmentation, where SAM(2D$\,\to\,$3D) is evaluated on four CT benchmarks (KiTS, Pancreas, LiTS, Colon), and 4D segmentation, where SAM2 (2D+T$\,\to\,$3D+T) is evaluated on a cell tracking challenge (CTC) dataset (Fluo-N3DH-SIM+). In both settings SAM+D achieves competitive or superior results under the single-point prompt setting while using fewer trainable parameters than existing methods, demonstrating that SAM+D generalizes across SAM-family architectures, target dimensionalities (3D, 4D), and domains spanning medical imaging and bio-scene understanding. Code is publicly available at https://github.com/JerrySongCST/SAM-Plus-D.

Figures

Figures reproduced from arXiv: 2607.29033 by Hao Sun, Ikuko Nishikawa, Shiyu Teng, Yen-Wei Chen, Yu Song.

Figure 1
Figure 1. Figure 1: Overview of SAM+D. (a) 3D setting: DRLoRA and DSM are inserted into SAM’s frozen ViT-B encoder; a LoRA-adapted decoder produces masks refined by a 3D head. (b) 4D setting: the same modules are inserted into SAM2’s frozen Hiera encoder; depth-pooled features interface with LoRA-adapted memory attention for prompt-free tracking at t > 0. Blue: frozen; orange: trainable; gray: zero-parameter operation. Depth-… view at source ↗
Figure 2
Figure 2. Figure 2: Depth-Routed LoRA (DRLoRA): a lightweight router maps the normalised depth position z to a softmax distribution over E LoRA experts. The weighted sum of expert outputs is added to the frozen Q and V projections, producing depth-dependent adaptations while leaving K unchanged. Concretely, DRLoRA maintains E parallel LoRA experts {E1, . . . , EE}, each a standard low-rank factorisation: \mathcal {E}_i(\mathb… view at source ↗
Figure 3
Figure 3. Figure 3: Depth Shift Module (DSM) applied along the depth axis. Before self-attention, a fraction α of channels are shifted forward from slice i−1 and backward from slice i+1, while the remaining (1−2α) channels stay in place. This gives each slice access to neighbouring features at zero extra parameters or FLOPs. Intuitively, adapting queries modifies the attention pattern—which tokens each position attends to—whi… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative 3D segmentation results. Each pair shows a full slice (top) and an enlarged crop (bottom) for four representative cases. Green: GT; Red: prediction. BGU-IL [2] Ultrack [4] TrackStra [14] SAM2 [9] Ours GT [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Visualization on Fluo-N3DH-SIM+. Top: Mode 1 (GT segmentation + propagation). Bottom: Mode 2 (Cellpose [38] + propagation). Green: correctly tracked; Purple: false positive; Blue: false negative. Best viewed when zoomed in [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Three decoder architectures for SAM+D. (a) Conv3D decoder: directly upsamples the 3D feature volume through transposed convolutions. (b) MLA decoder: aggregates intermediate features from encoder blocks {3, 6, 9, 12} through parallel path￾ways. (c) SAM prompt decoder (default): reuses SAM’s frozen prompt encoder and mask decoder with LoRA injection and a learnable depth embedding. Orange: train￾able; blue:… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

57 extracted references · 14 linked inside Pith

  1. [1]

    Nature communications13(1), 4128 (2022)

    Antonelli, M., Reinke, A., Bakas, S., Farahani, K., Kopp-Schneider, A., Landman, B.A., Litjens, G., Menze, B., Ronneberger, O., Summers, R.M., et al.: The medical segmentation decathlon. Nature communications13(1), 4128 (2022)

    2022

  2. [2]

    In: European Conference on Computer Vision

    Ben-Haim, T., Raviv, T.R.: Graph neural network for cell tracking in microscopy videos. In: European Conference on Computer Vision. pp. 610–626. Springer (2022)

    2022

  3. [3]

    Medical image analysis84, 102680 (2023)

    Bilic,P.,Christ,P.,Li,H.B.,Vorontsov,E.,Ben-Cohen,A.,Kaissis,G.,Szeskin,A., Jacobs, C., Mamani, G.E.H., Chartrand, G., et al.: The liver tumor segmentation benchmark (lits). Medical image analysis84, 102680 (2023)

    2023

  4. [4]

    bioRxiv pp

    Bragantini, J., Theodoro, I., Zhao, X., Huijben, T.A., Hirata-Miyasaki, E., Vi- jayKumar, S., Balasubramanian, A., Lao, T., Agrawal, R., Xiao, S., et al.: Ultrack: pushing the limits of cell tracking across biological scales. bioRxiv pp. 2024–09 (2024)

    2024

  5. [5]

    In: European Conference on Computer Vision

    Bragantini, J., Lange, M., Royer, L.: Large-scale multi-hypotheses cell tracking using ultrametric contours maps. In: European Conference on Computer Vision. pp. 36–54. Springer (2024)

    2024

  6. [6]

    arXiv preprint arXiv:2511.16719 (2025)

    Carion, N., Gustafson, L., Hu, Y.T., Debnath, S., Hu, R., Suris, D., Ryali, C., Alwala,K.V.,Khedr,H.,Huang,A.,etal.:Sam3:Segmentanythingwithconcepts. arXiv preprint arXiv:2511.16719 (2025)

    Pith/arXiv arXiv 2025

  7. [7]

    Medical Image Analysis98, 103310 (2024)

    Chen, C., Miao, J., Wu, D., Zhong, A., Yan, Z., Kim, S., Hu, J., Liu, Z., Sun, L., Li, X., et al.: Ma-sam: Modality-agnostic sam adaptation for 3d medical image segmentation. Medical Image Analysis98, 103310 (2024)

    2024

  8. [8]

    Advances in Neural Information Processing Systems35, 16664–16678 (2022)

    Chen, S., Ge, C., Tong, Z., Wang, J., Song, Y., Wang, J., Luo, P.: Adaptformer: Adapting vision transformers for scalable visual recognition. Advances in Neural Information Processing Systems35, 16664–16678 (2022)

    2022

  9. [9]

    In: International Workshop on Foundation Models for General Medical AI

    Chen, Z., Edgü, M., Jin, E., Stegmaier, J.: Segment anything for cell tracking. In: International Workshop on Foundation Models for General Medical AI. pp. 12–22. Springer (2025) 16 Y. Song et al

    2025

  10. [10]

    In: International conference on medical image computing and computer-assisted intervention

    Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3d u-net: learning dense volumetric segmentation from sparse annotation. In: International conference on medical image computing and computer-assisted intervention. pp. 424–432. Springer (2016)

    2016

  11. [11]

    Advances in neural information processing systems36, 10088–10115 (2023)

    Dettmers, T., Pagnoni, A., Holtzman, A., Zettlemoyer, L.: Qlora: Efficient fine- tuning of quantized llms. Advances in neural information processing systems36, 10088–10115 (2023)

    2023

  12. [12]

    arXiv preprint arXiv:2010.11929 (2020)

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

    Pith/arXiv arXiv 2010

  13. [13]

    Advances in Neural Information Processing Systems 37, 110746–110783 (2024)

    Du, Y., Bai, F., Huang, T., Zhao, B.: Segvol: Universal and interactive volumetric medical image segmentation. Advances in Neural Information Processing Systems 37, 110746–110783 (2024)

    2024

  14. [14]

    In: European conference on computer vision

    Gallusser, B., Weigert, M.: Trackastra: Transformer-based cell tracking for live-cell microscopy. In: European conference on computer vision. pp. 467–484. Springer (2024)

    2024

  15. [15]

    Medical Image Analysis98, 103324 (2024)

    Gong, S., Zhong, Y., Ma, W., Li, J., Wang, Z., Zhang, J., Heng, P.A., Dou, Q.: 3dsam-adapter: Holistic adaptation of sam from 2d to 3d for promptable tumor segmentation. Medical Image Analysis98, 103324 (2024)

    2024

  16. [16]

    Medical image analysis67, 101821 (2021)

    Heller, N., Isensee, F., Maier-Hein, K.H., Hou, X., Xie, C., Li, F., Nan, Y., Mu, G., Lin, Z., Han, M., et al.: The state of the art in kidney and kidney tumor segmentation in contrast-enhanced ct imaging: Results of the kits19 challenge. Medical image analysis67, 101821 (2021)

    2021

  17. [17]

    In: International conference on machine learning

    Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Ges- mundo, A., Attariyan, M., Gelly, S.: Parameter-efficient transfer learning for nlp. In: International conference on machine learning. pp. 2790–2799. PMLR (2019)

    2019

  18. [18]

    ICLR1(2), 3 (2022)

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al.: Lora: Low-rank adaptation of large language models. ICLR1(2), 3 (2022)

    2022

  19. [19]

    In: International Conference on Medical Image Computing and Computer-Assisted Intervention

    Hu, J., Li, Y., Sun, H., Song, Y., Zhang, C., Lin, L., Chen, Y.W.: Lga: A lan- guage guide adapter for advancing the sam model’s capabilities in medical im- age segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 610–620. Springer (2024)

    2024

  20. [20]

    Nature Methods18(2), 203–211 (2021)

    Isensee, F., Jaeger, P.F., Kohl, S.A.A., Petersen, J., Maier-Hein, K.H.: nnu-net: A self-configuring method for deep learning-based biomedical image segmentation. Nature Methods18(2), 203–211 (2021)

    2021

  21. [21]

    In: International Conference on Medical Image Computing and Computer-Assisted Intervention

    Jeon, Y.S., Yang, H., Fu, H., Kway, Y., Feng, M.: No more sliding window: effi- cient 3d medical image segmentation with differentiable top-k patch sampling. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 376–386. Springer (2025)

    2025

  22. [22]

    In: European conference on computer vision

    Jia, M., Tang, L., Chen, B.C., Cardie, C., Belongie, S., Hariharan, B., Lim, S.N.: Visual prompt tuning. In: European conference on computer vision. pp. 709–727. Springer (2022)

    2022

  23. [23]

    arXiv preprint arXiv:1312.6114 (2013)

    Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)

    Pith/arXiv arXiv 2013

  24. [24]

    In: IEEE Conf

    Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., et al.: Segment anything. In: IEEE Conf. Comput. Vis. Pattern Recog. pp. 4015–4026 (2023)

    2023

  25. [25]

    arXiv preprint arXiv:2209.15076 (2022) SAM+D 17

    Lee, H.H., Bao, S., Huo, Y., Landman, B.A.: 3d ux-net: A large kernel volumet- ric convnet modernizing hierarchical transformer for medical image segmentation. arXiv preprint arXiv:2209.15076 (2022) SAM+D 17

    Pith/arXiv arXiv 2022

  26. [26]

    In: Proceedings of the 2021 conference on empirical methods in natural language processing

    Lester, B., Al-Rfou, R., Constant, N.: The power of scale for parameter-efficient prompt tuning. In: Proceedings of the 2021 conference on empirical methods in natural language processing. pp. 3045–3059 (2021)

    2021

  27. [27]

    arXiv preprint arXiv:2404.15159 (2024)

    Li, D., Ma, Y., Wang, N., Ye, Z., Cheng, Z., Tang, Y., Zhang, Y., Duan, L., Zuo, J., Yang, C., et al.: Mixlora: Enhancing large language models fine-tuning with lora-based mixture of experts. arXiv preprint arXiv:2404.15159 (2024)

    Pith/arXiv arXiv 2024

  28. [28]

    Li, X.L., Liang, P.: Prefix-tuning: Optimizing continuous prompts for generation. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Pro- cessing (Volume 1: Long Papers). pp. 4582–4597 (2021)

    2021

  29. [29]

    In: IEEE Conf

    Lin, J., Gan, C., Han, S.: Tsm: Temporal shift module for efficient video under- standing. In: IEEE Conf. Comput. Vis. Pattern Recog. pp. 7083–7093 (2019)

    2019

  30. [30]

    In: Forty-first International Conference on Machine Learning (2024)

    Liu, S.Y., Wang, C.Y., Yin, H., Molchanov, P., Wang, Y.C.F., Cheng, K.T., Chen, M.H.: Dora: Weight-decomposed low-rank adaptation. In: Forty-first International Conference on Machine Learning (2024)

    2024

  31. [31]

    Medical image analysis72, 102102 (2021)

    Luo, X., Wang, G., Song, T., Zhang, J., Aertsen, M., Deprest, J., Ourselin, S., Vercauteren, T., Zhang, S.: Mideepseg: Minimally interactive segmentation of un- seen objects from medical images using deep learning. Medical image analysis72, 102102 (2021)

    2021

  32. [32]

    arXiv preprint arXiv:2504.03600 (2025)

    Ma, J., Yang, Z., Kim, S., Chen, B., Baharoon, M., Fallahpour, A., Asakereh, R., Lyu, H., Wang, B.: Medsam2: Segment anything in 3d medical images and videos. arXiv preprint arXiv:2504.03600 (2025)

    Pith/arXiv arXiv 2025

  33. [33]

    arXiv preprint arXiv:2304.07193 (2023)

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al.: Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 (2023)

    Pith/arXiv arXiv 2023

  34. [34]

    arXiv preprint arXiv:2408.00714 (2024)

    Ravi, N., Gabeur, V., Hu, Y.T., Hu, R., Ryali, C., Ma, T., Khedr, H., Rädle, R., Rolland, C., Gustafson, L., et al.: Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714 (2024)

    Pith/arXiv arXiv 2024

  35. [35]

    In: IEEE Conf

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: IEEE Conf. Comput. Vis. Pattern Recog. pp. 10684–10695 (2022)

    2022

  36. [36]

    Ryali, C., Hu, Y.T., Bolya, D., Wei, C., Fan, H., Huang, P.Y., Aggarwal, V., Chowdhury, A., Poursaeed, O., Hoffman, J., et al.: Hiera: A hierarchical vision transformerwithoutthebells-and-whistles.In:Internationalconferenceonmachine learning. pp. 29441–29454. PMLR (2023)

    2023

  37. [37]

    IEEE Transac- tions on Medical Imaging43(9), 3377–3390 (2024)

    Shaker, A., Maaz, M., Rasheed, H., Khan, S., Yang, M.H., Khan, F.S.: Unetr++: delving into efficient and accurate 3d medical image segmentation. IEEE Transac- tions on Medical Imaging43(9), 3377–3390 (2024)

    2024

  38. [38]

    Nature methods18(1), 100–106 (2021)

    Stringer, C., Wang, T., Michaelos, M., Pachitariu, M.: Cellpose: a generalist algo- rithm for cellular segmentation. Nature methods18(1), 100–106 (2021)

    2021

  39. [39]

    IEEE transactions on medical imaging36(1), 310–321 (2016)

    Svoboda, D., Ulman, V.: Mitogen: a framework for generating 3d synthetic time- lapse sequences of cell populations in fluorescence microscopy. IEEE transactions on medical imaging36(1), 310–321 (2016)

    2016

  40. [40]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Tang, Y., Yang, D., Li, W., Roth, H.R., Landman, B., Xu, D., Nath, V., Hatamizadeh, A.: Self-supervised pre-training of swin transformers for 3d med- ical image analysis. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 20730–20740 (2022)

    2022

  41. [41]

    Nature methods14(12), 1141–1152 (2017) 18 Y

    Ulman, V., Maška, M., Magnusson, K.E., Ronneberger, O., Haubold, C., Harder, N., Matula, P., Matula, P., Svoboda, D., Radojevic, M., et al.: An objective com- parison of cell-tracking algorithms. Nature methods14(12), 1141–1152 (2017) 18 Y. Song et al

    2017

  42. [42]

    IEEE Transactions on Neural Networks and Learning Systems (2025)

    Wang, H., Guo, S., Ye, J., Deng, Z., Cheng, J., Li, T., Chen, J., Su, Y., Huang, Z., Shen, Y., et al.: Sam-med3d: a vision foundation model for general-purpose seg- mentation on volumetric medical images. IEEE Transactions on Neural Networks and Learning Systems (2025)

    2025

  43. [43]

    In: International conference on medical image computing and computer-assisted intervention

    Wang, H., Lin, L., Hu, H., Chen, Q., Li, Y., Iwamoto, Y., Han, X.H., Chen, Y.W., Tong, R.: Patch-free 3d medical image segmentation driven by super-resolution technique and self-supervised guidance. In: International conference on medical image computing and computer-assisted intervention. pp. 131–141. Springer (2021)

    2021

  44. [44]

    In: International conference on medical image computing and computer-assisted intervention

    Wang, W., Chen, C., Ding, M., Yu, H., Zha, S., Li, J.: Transbts: Multimodal brain tumor segmentation using transformer. In: International conference on medical image computing and computer-assisted intervention. pp. 109–119. Springer (2021)

    2021

  45. [45]

    Medical image analysis102, 103547 (2025)

    Wu, J., Wang, Z., Hong, M., Ji, W., Fu, H., Xu, Y., Xu, M., Jin, Y.: Medical sam adapter: Adapting segment anything model for medical image segmentation. Medical image analysis102, 103547 (2025)

    2025

  46. [46]

    arXiv preprint arXiv:2404.13628 (2024)

    Wu, X., Huang, S., Wei, F.: Mixture of lora experts. arXiv preprint arXiv:2404.13628 (2024)

    Pith/arXiv arXiv 2024

  47. [47]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Xu, N., Price, B., Cohen, S., Yang, J., Huang, T.S.: Deep interactive object se- lection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 373–381 (2016)

    2016

  48. [48]

    arXiv preprint arXiv:2510.08967 (2025)

    Yang, Y., Xu, L., Tian, L.: Sam2-3dmed: Empowering sam2 for 3d medical image segmentation. arXiv preprint arXiv:2510.08967 (2025)

    arXiv 2025

  49. [49]

    arXiv preprint arXiv:2309.05444 (2023)

    Zadouri, T., Üstün, A., Ahmadian, A., Ermiş, B., Locatelli, A., Hooker, S.: Pushing mixture of experts to the limit: Extremely parameter efficient moe for instruction tuning. arXiv preprint arXiv:2309.05444 (2023)

    Pith/arXiv arXiv 2023

  50. [50]

    arXiv preprint arXiv:2304.13785 (2023)

    Zhang, K., Liu, D.: Customized segment anything model for medical image seg- mentation. arXiv preprint arXiv:2304.13785 (2023)

    Pith/arXiv arXiv 2023

  51. [51]

    arXiv preprint arXiv:2303.10512 (2023)

    Zhang, Q., Chen, M., Bukharin, A., Karampatziakis, N., He, P., Cheng, Y., Chen, W., Zhao, T.: Adalora: Adaptive budget allocation for parameter-efficient fine- tuning. arXiv preprint arXiv:2303.10512 (2023)

    Pith/arXiv arXiv 2023

  52. [52]

    arXiv preprint arXiv:2109.03201 (2021)

    Zhou, H.Y., Guo, J., Zhang, Y., Yu, L., Wang, L., Yu, Y.: nnformer: Interleaved transformer for volumetric segmentation. arXiv preprint arXiv:2109.03201 (2021)

    Pith/arXiv arXiv 2021

  53. [53]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Zhou, M., Wang, H., Zhao, Q., Li, Y., Huang, Y., Meng, D., Zheng, Y.: Interactive segmentation as gaussion process classification. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 19488–19497 (2023)

    2023

  54. [54]

    arXiv preprint arXiv:2408.00874 (2024)

    Zhu, J., Hamdi, A., Qi, Y., Jin, Y., Wu, J.: Medical sam 2: Segment medical images as video via segment anything model 2. arXiv preprint arXiv:2408.00874 (2024)

    Pith/arXiv arXiv 2024

  55. [55]

    In: International Conference on Medical Image Computing and Computer-Assisted Intervention

    Zhuang, J., Wu, L., Ni, X., Wang, X., Wang, L., Chen, H.: Bio2vol: Adapting 2d biomedical foundation models for volumetric medical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 24–34. Springer (2025)

    2025

  56. [56]

    Zou, X., Yang, J., Zhang, H., Li, F., Li, L., Wang, J., Wang, L., Gao, J., Lee, Y.J.: Segment everything everywhere all at once. Advances in neural information processing systems36, 19769–19782 (2023) SAM+D 19 Fig.6: Three decoder architectures for SAM+D.(a) Conv3D decoder: directly upsamples the 3D feature volume through transposed convolutions. (b) MLA ...

    2023

  57. [57]

    a computerized tomography of a kidney tumor

    followed by InstanceNorm3d, GELU, a Conv3d (3×3×3, padding1), Instan- ceNorm3d,andGELU.Thefirststagemaps256→128channelsat64 3 resolution; the second maps128→64channels at128 3 resolution. A final Conv3d (1×1×1) projects to2output classes, followed by trilinear interpolation to the target vol- ume size. This decoder adds 881,154 trainable parameters, bring...