REVIEW 2 major objections 6 minor 14 references
Depthwise-Dilated Convolutional Adapters for Medical Object Tracking and Segmentation Using the Segment Anything Model 2
T0 review · 2 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read DD-SAM2 adapts SAM2 to medical video tracking with lightweight depthwise-dilated adapters, beating MedSAM2 by 0.03–0.06 Dice on two benchmarks.
desk verdict A well-ablated adapter paper whose headline significance claims rest on frame-level Wilcoxon tests that likely overstate the evidence. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The DD-Adapter is the load-bearing component. It is a residual module placed after transformer blocks in SAM2's Hiera-based image encoder: a pointwise convolution squeezes channels, GELU activates, parallel depthwise-dilated convolution branches (dilation rates 1 and 3 by default) capture multi-scale local context, a second pointwise convolution restores channels, and a skip connection adds the input. Because depthwise-dilated convolutions are cheap, the six inserted adapters add about 0.54 million parameters and roughly 4 GMac, keeping inference near real time. Only the adapters and the mask decoder are updated during fine-tuning; SAM2's streaming memory, prompt encoder, and image encoder body stay frozen.
What would settle it
Run the public TrackRAD2025 challenge test set (the 58 held-out cases) through DD-SAM2 and MedSAM2 with identical first-frame box prompts; if the reported Dice gap of 0.03 does not persist, the claimed advantage depends on the 12-case evaluation subset used in the paper.
Extended reading notes
Core claim
The central claim is that a parameter-efficient adapter built from pointwise convolutions and parallel depthwise-dilated convolutions lets SAM2 track and segment medical objects across time better than the unmodified model or existing adapter designs. On TrackRAD2025, DD-SAM2 reaches Dice 0.93±0.04 versus 0.89±0.10 for SAM2 and 0.90±0.05 for MedSAM2, with Wilcoxon signed-rank tests (Bonferroni-corrected) below 0.005; on EchoNet-Dynamic, DD-MedSAM2 reaches 0.97±0.01 versus 0.91±0.06 for MedSAM2. The gains appear in boundary metrics too: HD95 drops and NSD rises on both datasets. The authors also show that the depthwise-dilated design beats standard-convolution and MLP adapters at a fraction of the parameters, and that models trained on MRI transfer to ultrasound (CAMUS) with about 0.02 Dice gain over baselines.
Load-bearing premise
The method assumes that leaving SAM2's streaming memory module frozen during fine-tuning does not cost accuracy; if temporal memory needs task-specific adaptation for medical videos, the reported gains may not be the best the architecture could achieve.
Editorial extensions
If this is right
- Adding six DD-Adapters plus mask-decoder fine-tuning is enough to lift SAM2 above a heavily fine-tuned MedSAM2 on both tested medical video benchmarks.
- Because the adapters add only ~0.54M parameters and ~4 GMac, the approach can run near real time (52–67 FPS on an RTX 4090).
- The gains hold on cross-domain transfer: models trained on MRI TrackRAD2025 improve Dice on ultrasound CAMUS by ~0.02 over SAM2 and MedSAM2 baselines.
- The adapter choice matters: MLP-based, standard-convolution, and LoRA adapters trail DD-Adapter on Dice and boundary metrics, indicating multi-scale local features drive the improvement.
Reading between the lines
- Inference: the same adapter could be dropped into other promptable video foundation models or into SAM2 with point prompts, and likely yield similar efficiency gains, though the paper only tests box prompts for tracking.
- Inference: the strong cross-domain CAMUS result suggests DD-Adapter partly compensates for domain shift in the image encoder; testing on modalities far from MRI and ultrasound, such as endoscopy or CT cine, would map where the benefit stops.
- Inference: since only the mask decoder and adapters train, the method is compatible with federated or privacy-preserving fine-tuning on institutional video data, where full retraining is infeasible.
- Inference: the ablation shows boundary metrics (NSD, HD95) are more sensitive to the dilated-depthwise branch than Dice is, so studies reporting only Dice may miss real degradation when the branch is removed.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes DD-SAM2, a parameter-efficient fine-tuning framework for SAM2 built around a Depthwise-Dilated Adapter (DD-Adapter) inserted after the first six transformer blocks of the image encoder. Only the adapters and the mask decoder are updated, while SAM2's image encoder, prompt encoder, and streaming memory are kept frozen. The method is evaluated on TrackRAD2025 cine-MRI tumor tracking, EchoNet-Dynamic left-ventricle tracking, and CAMUS cross-domain transfer, reporting Dice scores of 0.93±0.04 and 0.97±0.01 for the adapted models and claiming statistically significant (p<0.005) improvements over MedSAM2 and other adapter-based baselines. The manuscript also includes ablations on adapter count, dilation rates, prompt types, video lengths, and limited training data, as well as parameter and FLOP comparisons.
Significance. If the central empirical claim holds, the paper is a solid and practical contribution: the adapter design is simple and parameter-efficient (only 0.54M additional parameters), and the experimental campaign is unusually broad, covering registration-based and memory-based baselines, adapter and LoRA variants, cross-domain evaluation, prompt-sensitivity analysis, and limited-data experiments. The authors also explicitly discuss the limitation of freezing the streaming memory module rather than updating it. The main weakness is statistical: the headline significance claim rests on frame-level Wilcoxon tests that treat temporally correlated frames as independent observations, so the p<0.005 statement is not currently established. Because this is a fixable analysis issue rather than a contradiction in the method, a major revision with corrected video-level inference would make the claims trustworthy.
major comments (2)
- [§4.2, §5.1(1), Tables 3 and 5] The paper states in §4.2 that final metrics are averaged per test video, but the Wilcoxon signed-rank tests in §5.1(1) are run 'on all frames of the testing videos (frame-based, to improve the statistical power).' Frames within a cine-MRI or echocardiography sequence are temporally correlated, so a frame-level test treats non-independent observations as independent. On TrackRAD2025 this inflates the effective sample from 12 test videos to 1,096 frames, and on EchoNet-Dynamic from 1,277 test videos to many tens of thousands of frames. Because the central claim (DD-MedSAM2 outperforms MedSAM2, p<0.005) relies on this test, please report video-level Wilcoxon signed-rank tests or a clustered bootstrap that resamples whole videos, with exact p-values after Bonferroni correction. If the frame-level test is retained, the independence assumption needs an explicit justification beyond statistical power.
- [§5.1(2), Table 4] The contribution statement claims that DD-SAM2 'consistently outperforms existing adapter-based baselines in both segmentation performance and tracking accuracy,' but Table 4 reports no significance tests for the adapter-variant comparisons. With only 12 TrackRAD test videos and Dice differences of 0.01 to 0.02 between DD-MedSAM2 and MedSAM2-LoRA or MedSAM2-Adp-b, the reported standard deviations are large enough that these differences may not be reliable. Please report video-level confidence intervals or significance tests for the adapter comparisons in Table 4, or temper the 'consistently outperforms' claim to match the observed effect sizes.
minor comments (6)
- [§3.2] The paragraph beginning 'Specifically, the first PWConv layer...' appears twice, verbatim, after Figure 3; the duplicate should be removed.
- [References] The reference list is corrupted in the submitted text (for example, LeCun et al. is rendered as 'e un, Y .') and many entries contain missing letters; please regenerate and proofread all references.
- [§4.2, Tables 3–5] No tracking-specific metric such as J&F, contour accuracy, or temporal consistency is reported; the paper's 'tracking accuracy' claims currently rest on frame-wise segmentation overlap, which the authors should clarify.
- [§5.1(1)] The exact p-values for each pairwise comparison and the number of comparisons entered in the Bonferroni correction are not reported; a supplementary table with this information would allow readers to verify the corrected threshold.
- [§5.3(2), Table 8] The statement that the Stand-Conv-Adapter has 14× more parameters depends on the channel dimension C; please state the asymptotic condition or give the formula for general C instead of a single numerical example.
- [§5.1(2), Table 4] The adapter-variant experiments appear to be based on a single training run; given the small Dice differences, a statement about seed variance or confidence intervals would substantially improve reliability.
Circularity Check
No significant circularity: DD-SAM2's reported gains are empirical measurements on external test sets, not derivations that reduce to fitted inputs.
full rationale
The paper's central claims are empirical: the DD-Adapter equations (1)-(3) define an architectural forward pass, and Dice, NSD, HD95, and ASD are measured on held-out TrackRAD2025 and EchoNet-Dynamic test videos after supervised fine-tuning. No reported quantity is defined in terms of another by construction, and no fitted parameter is relabeled as a prediction; the improvements over SAM2, MedSAM2, and adapter baselines come from trained models evaluated on separate test sets. The only self-citation, SAMatch [50], appears in a future-work sentence in the Discussion and is not load-bearing. The acknowledged freezing of the streaming memory module is a stated limitation, not a circular step. Two validity concerns are present but are not circularity: the Wilcoxon tests in Section 5.1(1) are run on all frames of autocorrelated video sequences, which inflates the effective sample size, and several ablation and hyperparameter choices, such as dilation rate (1,3) in Table 10, are selected using the same test set, which risks selection bias. These are statistical and experimental-design risks, not circular derivation.
Assumptions & free parameters
free parameters (3)
- Number of DD-Adapters =
6
- Dilation rates =
[1, 3]
- Adapter bottleneck dimension =
unspecified
assumptions (3)
- domain assumption SAM2's pretrained weights and streaming memory are a suitable base for medical video adaptation
- domain assumption Freezing the streaming memory module during fine-tuning does not prevent effective temporal tracking
- ad hoc to paper Depthwise dilated convolutions with rates 1 and 3 capture the multi-scale local features relevant to tumors and cardiac structures
invented entities (1)
-
DD-Adapter
independent evidence
Cite this review
Pith. "Pith review of Depthwise-Dilated Convolutional Adapters for Medical Object Tracking and Segmentation Using the Segment Anything Model 2." pith.science (2026). https://pith.science/paper/DR3P35LF
@misc{pith2026250714613,
author = {Pith},
title = {Pith review of: Depthwise-Dilated Convolutional Adapters for Medical Object Tracking and Segmentation Using the Segment Anything Model 2},
year = {2026},
howpublished = {\url{https://pith.science/paper/DR3P35LF}},
note = {Machine review of arXiv:2507.14613}
}
read the original abstract
Recent advances in medical image segmentation have been driven by deep learning; however, most existing methods remain limited by modality-specific designs and exhibit poor adaptability to dynamic medical imaging scenarios. The Segment Anything Model 2 (SAM2) and its related variants, which introduce a streaming memory mechanism for real-time video segmentation, present new opportunities for prompt-based, generalizable solutions. Nevertheless, adapting these models to medical video scenarios typically requires large-scale datasets for retraining or transfer learning, leading to high computational costs and the risk of catastrophic forgetting. To address these challenges, we propose DD-SAM2, an efficient adaptation framework for SAM2 that incorporates a Depthwise-Dilated Adapter (DD-Adapter) to enhance multi-scale feature extraction with minimal parameter overhead. This design enables effective fine-tuning of SAM2 on medical videos with limited training data. Unlike existing adapter-based methods focused solely on static images, DD-SAM2 fully exploits SAM2's streaming memory for medical video object tracking and segmentation. Comprehensive evaluations on TrackRad2025 (tumor segmentation) and EchoNet-Dynamic (left ventricle tracking) datasets demonstrate superior performance, achieving Dice scores of 0.93 and 0.97, respectively. To the best of our knowledge, this work provides an initial attempt at systematically exploring adapter-based SAM2 fine-tuning for medical video segmentation and tracking. Code, datasets, and models will be publicly available at https://github.com/apple1986/DD-SAM2.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
The results are presented in Table 1
Impact of Different Prompt Types on DD-MedSAM2 We evaluated different prompt type s, including bounding box, ma sk, and sparse points (five randomly selected points per object), on the DD-MedSAM2 framework. The results are presented in Table 1. Among these results, using the ‘ground -truth’ mask from the first frame as the prompt achieved the best segment...
-
[2]
Related works 2.1 Segment Anything Model for Medical Imaging The Segment Anything Model (SAM) was originally developed for natural image segmentation and trained on over 1 billion masks across 11 million images using a prompt -based segmentation paradigm. Its architecture comprises three main components: an image encoder, a prompt encoder, and a mask deco...
-
[3]
Method 3.1 Preliminary of the SAM2 architecture SAM2 is a state -of-the-art interactive segmentation framework that extends the original SAM by introducing a streaming memory mechanism to effectively leverage historical context for both image and video segmentation tasks (see Figure 2). The architecture is composed of four key components: the image encode...
-
[4]
Experiments 4.1 Dataset To evaluate the effectiveness of DD -SAM2 in medical object tracking and segmentation, we use two publicly available datasets: TrackRAD20251 for tumor tracking and segm entation in MRI, and EchoNet - Dynamic [42] for left ventricle tracking and segmentation in ultrasound images. (1) TrackRAD2025 dataset The dataset used in this stu...
work page 2016
-
[5]
Experiments 5.1 Experiments for tumor tracking and segmentation on the TrackRAD2025 dataset (1) Comparison between different methods The learning-based registration approaches, VoxelMorph and TransMorph, perform worse than traditional intensity-based deformable registration (Table 3). One possible cause is that the traditional intensity-based methods use ...
-
[6]
Discussion In this study, we confirmed the efficacy of fine-tuning SAM2 with the proposed adapter s for object tracking and segmentation. The proposed method demonstrates strong clinical relevance, particularly in the context of radiotherapy, where precise and real-time tumor localization is essential for accurate dose delivery and treatment safety. The a...
-
[7]
Conclusion In this study, we propose d a depthwise-dilated convolutional adapter (DD-Adapter), a novel design that integrates multi-scale local features within the transformer blocks for effective fine-tuning of SAM2-based methods. The correspondingly adapted models, DD-SAM2 and DD-MedSAM2, achieve both high accuracy and efficiency, highlighting their pot...
arXiv 2020
-
[9]
The corresponding performance metrics are summarized in Table 2
Investigating DD-Adapter’s Model Sensitivity to Prompt Quality (1) Sensitivity of varying numbers of point prompts To assess the impact of the number of point prompts on segmentation performance, we conducted a n evaluation by randomly selecting 1 to 10 foreground points as input prompts for the DD-MedSAM2 model. The corresponding performance metrics are ...
Show all 14 references
-
[10]
The first -frame prompt was a bounding box, while the middle -frame prompt varied across three types: mask, bounding box, and 10 positive points
Combinations of Different Prompt Types To further investigate the effect of combining different prompt types, we conducted experiments using dual prompts: one derived from the annotated mask of the first frame, and the other from the middle frame of each video sequence. The fi...
-
[11]
Impacts of Video Lengths on Model Accuracy (1) Results by training with different lengths of video sub-sequences We investigated the impact of different training sub -sequence lengths on the performan ce of DD - MedSAM2 by retraining the model with 4, 8, and 12 consecutive fra...
-
[12]
The results, presented in Table 10, indicate that both models maintain competitive performance across all four evaluation metrics , even with as few as one or two training cases
Performance Under Limited Training Data To investigate model performance under limited annotations, we retrained DD-SAM2 and DD-MedSAM2 on the TrackRAD2025 dataset using only 1 to 6 training cases, respectively. The results, presented in Table 10, indicate that both models mai...
-
[13]
introduces a self -sorting memory bank mechanism that dynamically selects the most inform ative historical embeddings based on confidence and dissimilarity scores. Similarly, SurgicalSAM2 [30] implements a frame pruning strategy to retain only the most relevant frames in the m...
-
[14]
The results are summarized in Table 11
Comparison with Multi-Scale Standard Convolution Alternatives for the Adapter Structure To evaluate the use of standard convolution alternatives within the adapter structure, we replaced the depthwise-dilated convolution in the DD-Adapter with multi-scale variants (Stand-Conv)...
-
[15]
In these visualizations, the ground truth annotations are shown in red, whereas the model predictions are depicted in green
Sequence Prediction on TrackRAD2025 and EchoNet Dynamic Datasets We present the visualization results on the TrackRAD2025 dataset. In these visualizations, the ground truth annotations are shown in red, whereas the model predictions are depicted in green. This color scheme i s...
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.