Pith. sign in

REVIEW 4 major objections 5 minor 7 references

SAM2-Aug: Prior knowledge-based Augmentation for Target Volume Auto-Segmentation in Adaptive Radiation Therapy Using Segment Anything Model 2

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SAM2-Aug claims that feeding SAM2 the previous fraction's MRI and tumor contour, plus randomized prompt perturbations, lifts tumor segmentation to Dice 0.86–0.90 across liver, abdomen, and brain without retraining.

desk verdict A practical ART segmentation paper with a clever prior/prompt augmentation scheme, but the head-to-head comparison is undermined by unspecified test-time box provenance. read the letter →

arxiv 2507.19282 v1 pith:M75YZ7MP submitted 2025-07-25 eess.IV cs.CVphysics.med-ph

Guoping Xu , Yan Dai , Hengrui Zhao , Ying Zhang , Jie Deng , Weiguo Lu , You Zhang This is my paper · ORCID
classification eess.IVcs.CVphysics.med-ph
keywords SegmentAnythingModel2adaptiveradiationtherapytumorsegmentationpriorknowledgepromptaugmentationMRIMR-LINACgeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that SAM2, a generalist image-and-video segmentation model, can become accurate for tumor delineation in adaptive radiation therapy if it is given the right context and the right prompts. The proposed method, SAM2-Aug, feeds the model a three-channel input made of the current MRI, the previous fraction's MRI, and the previous fraction's tumor annotation, so the network can use the tumor's known location and shape as a scaffold. It also randomizes prompts during fine-tuning, expanding or contracting bounding boxes, eroding or dilating prior masks, and sometimes dropping them, so the model learns to tolerate imperfect user input. Fine-tuned on only 69 liver MRI scans, SAM2-Aug reports Dice scores of 0.86 for liver, 0.89 for abdominal, and 0.90 for brain targets, beating convolutional, transformer, and other SAM-based baselines without retraining on the abdominal and brain sets. The practical payoff is that re-contouring tumors at every treatment fraction, a time-consuming bottleneck in adaptive radiation therapy, could be automated using information the clinic already has.

What carries the argument

The load-bearing object is the three-channel prior-augmented input combined with prompt augmentation. In the adaptive radiation therapy workflow, fraction n's scan is rigidly registered to fraction (n-1)'s scan, so the previous tumor mask can be overlaid; the current MRI, prior MRI, and prior mask become a three-channel tensor fed to SAM2. On the prompt side, the current-frame bounding box is randomly expanded or contracted by 0 to 5 pixels and the prior mask is randomly eroded or dilated, with a stochastic selection that can drop either prompt; at test time only the box is perturbed and the prior mask is always used. The adaptation mechanism is joint fine-tuning of SAM2's image encoder, prompt encoder, and mask decoder on a small liver dataset, which then transfers to other sites.

What would settle it

A direct test would be to run SAM2-Aug on the same three datasets while systematically corrupting the prior: dropping the prior mask, adding synthetic misregistration shifts of 1 to 10 mm, or substituting a prior from a different MRI sequence. If Dice and HD95 degrade to the level of baseline SAM2 (0.78 liver, 0.84 abdomen, 0.88 brain) or below, the paper's causal claim that prior context drives the gain would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that prior knowledge from a patient's own earlier treatment fraction is sufficient to lift a promptable foundation model above specialist segmentation networks for tumor targets. Concretely, concatenating the current MR image, the prior MR image, and the prior tumor annotation into a three-channel input, combined with prompt augmentation (random bounding-box perturbation and prior-mask erosion/dilation with stochastic dropout), and fine-tuning SAM2's image encoder, prompt encoder, and mask decoder, yields mean Dice scores of 0.86 on liver ITV, 0.89 on abdominal ITV, and 0.90 on brain CTV. The generalization claim is that the same liver-trained weights transfer to different MRI sequences and tumor sites without retraining. The ablation evidence supports the mechanism: the full three-channel input adds 0.11 Dice over current-MR-only input, augmented box-plus-mask prompts add 0.1 over mask-only prompts, and joint fine-tuning of the three modules beats fine-tuning any subset or all modules.

Load-bearing premise

The method assumes that a physician-approved tumor contour from the previous treatment fraction or from simulation is always available, can be rigidly registered to the current scan, and still resembles the current tumor; if that prior is missing, heavily deformed, or from a different imaging protocol, the three-channel input and mask prompt can mislead the model rather than help it.

Editorial extensions

If this is right

  • If the claim holds, clinics can reuse the contour already approved for the previous fraction as the main guide, so each new fraction's tumor contour could be generated automatically rather than drawn from scratch.
  • The same liver-trained weights should transfer to other abdominal and brain sites, meaning one small fine-tuning dataset could support a cross-site auto-segmentation tool.
  • Prompt augmentation should make the pipeline robust to realistic sloppiness in user-drawn boxes, since the model is trained on boxes that are wrong by up to five pixels and on eroded or dilated masks.
  • The ablation results suggest prior-mask prompts are especially valuable for small tumors, so the approach should help in the low-contrast, small-target cases where manual contouring is most variable.
  • Because the inputs are simple three-channel concatenations, the method can ride on existing SAM2 infrastructure without architectural changes, making adoption comparatively cheap.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the prior annotation is unreliable, missing, from a different sequence, or misregistered, the contextual channels could actively mislead the model; a natural extension is measuring how Dice degrades as registration error or prior quality worsens.
  • The method's success suggests a broader design principle: for any sequential imaging task, a foundation model can be turned into an online segmenter by conditioning on the most recently approved label, which could extend to CBCT-guided radiation therapy and other longitudinal imaging.
  • A testable extension would be to make the prior-mask prompt itself a learnable parameter or to fuse the three channels with attention rather than concatenation, which might close the remaining gap to registration-only performance on near-stationary anatomy.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes SAM2-Aug, a SAM2-based tumor segmentation framework for adaptive radiation therapy (ART). The key ideas are (1) augmenting the input with the prior MR image and prior annotation as additional channels, and (2) augmenting prompts during fine-tuning via random bounding-box expansion/contraction, mask erosion/dilation, and stochastic prompt dropout. The model is fine-tuned on a One-Seq-Liver dataset (19 training patients) and then tested on a held-out liver test set plus two unseen datasets: Mix-Seq-Abdomen and Mix-Seq-Brain. The authors report mean Dice scores of 0.86, 0.89, and 0.90 on the three datasets, claiming superiority over convolutional, transformer, and SAM-based baselines, with ablations over input channels, prompt types, and fine-tuned modules.

Significance. If the reported results are robust, the work addresses a practically important and timely problem: leveraging the natural availability of prior-fraction annotations in ART. The held-out evaluation on unseen MR sequences and tumor sites is a notable strength, as is the ablation structure that isolates the contributions of prior input, prompt augmentation, and fine-tuning strategy. The paper is clearly written and the experimental design is mostly thoughtful. However, the central comparison with interactive SAM-based baselines hinges on how test-time prompts were generated, and this is not specified precisely enough to rule out an information leak that would make the headline gains optimistic.

major comments (4)
  1. [§4.3 and §3.2] The provenance of test-time bounding boxes for the prompt-based baselines is not specified. The text states in §4.3 that MedSAM and SAM-Med2D were given bounding boxes, SAM-Med3D/Turbo ten points, and SAM2 and SAM2-Aug boxes plus masks, but it does not state how these boxes were obtained. In §3.2, the testing stage for SAM2-Aug says all bounding boxes were 'expanded or contracted by 0 to 5 pixels', which suggests they were derived from the ground-truth current mask. If the same is true for MedSAM, SAM-Med2D, and SAM2, the reported Dice values are optimistic upper bounds and the comparison does not reflect a realistic ART workflow, where the current-fraction box must come from user interaction or propagation from the prior. Please state the box generation protocol explicitly and re-evaluate all interactive methods with a realistic prompt source (e.g., prior-mask propagation with registration, or simulated user boxes), reporting results for that protocol.
  2. [§5.1, Table 3] The 'SAM2 (without prompt augmentation)' baseline is not sufficiently defined. It is unclear whether this baseline receives the same three-channel input (current MR, prior MR, prior segmentation) and the same prior-mask prompt as SAM2-Aug, or only the current MR and a current box. If the baseline lacks the prior input, the 0.08 Dice gap in Table 3 conflates the proposed input augmentation with the prompt augmentation that the text credits for the improvement. Please specify the exact input and prompt configuration of every baseline, ideally adding this information as a column or footnote in the comparison tables.
  3. [§5, Tables 3–5] No statistical significance testing is reported. The margins over the closest competitor are small on some datasets (e.g., brain Dice 0.90 vs 0.88 for SAM2 in Table 5; liver Dice 0.86 vs 0.83 for MedSAM and registration-only in Table 3), and test scans are clustered by patient. Please add paired (per-case) significance tests, confidence intervals, or effect-size measures to support the claim that SAM2-Aug outperforms all compared methods.
  4. [§3.2 and §6] The robustness of the method to imperfect or missing prior information is not tested. The test protocol deliberately keeps the prior annotation unaugmented and performs no prompt dropout, 'assuming physician-approved prior information will always be available'. This assumption may fail in ART when the prior mask is outdated after large anatomical change or when registration is inaccurate. Since the entire approach depends on the prior, please add experiments with, e.g., simulated registration errors, prior masks from different protocols, or randomly dropped priors at test time, and report how performance degrades.
minor comments (5)
  1. [§5.2] There are two tables numbered Table 7: one for prompt-type ablations and one for fine-tuning ablations. Please renumber the second table.
  2. [§4.1] In the sentence about robustness and generalizability, the model is referred to as 'SAM-Aug2'; this should be 'SAM2-Aug'.
  3. [Table 3 caption] The caption repeats 'The best values are shown in bold.' twice; one instance should be removed.
  4. [§5.1] In Table 3, the baseline 'SAM2 (without prompt augmentation)' is listed without a footnote describing whether it uses the three-channel input from §3.3 and whether it receives the prior-mask prompt. A footnote would prevent misinterpretation.
  5. [§6, limitation (1)] The first limitation discusses partial prompts and SAM2's memory mechanism, but the experiments reported in the paper always use full slices with complete prompts. Clarify whether this limitation refers to experiments not reported or to an anticipated issue, and if the former, report the corresponding results.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found; the reported gains are grounded in held-out empirical evaluation.

full rationale

SAM2-Aug is an empirical fine-tuning study with no theoretical derivation chain that could reduce to its inputs. The claimed Dice improvements on One-Seq-Liver are evaluated on held-out test patients, and the Mix-Seq-Abdomen and Mix-Seq-Brain datasets were never used for training, so the generalization claims are genuinely external rather than fitted parameters renamed as predictions. The prior MR image and prior annotation are distinct inputs taken from the previous treatment fraction or simulation, not from the current target mask, so the method is not self-definitional. Comparison methods are described as receiving the same three-channel input, and the SAM2-Aug results are not justified by invoking the authors' own prior work; self-citations appear only in related-work and ablation-discussion contexts and are not load-bearing. The paper's unstated provenance of test-time bounding boxes is a potential evaluation-fairness concern, but it does not make the claimed derivation circular: even if the boxes were ground-truth-derived, SAM2-Aug would still be an empirically measured method, not a result equivalent to its inputs by construction. No circular step meeting the required evidence standard was identified.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim relies on standard machine learning hyperparameters and domain assumptions about the availability and alignment of prior annotations. No new physical entities or conserved quantities are introduced. The most notable assumption is the reliance on ground-truth-derived bounding boxes for evaluation, which is common in prompt-based segmentation benchmarks but may not reflect fully clinical use.

free parameters (3)
  • Bounding-box expansion and contraction range = 0-5 pixels
    Hand-chosen prompt augmentation range applied in training and testing; it controls how much the model is robustified to box placement variability.
  • Random bounding-box expansion range in robustness test = 1-10 pixels
    Used for the Figure 8 stress test; hand-chosen to simulate larger user input variations.
  • Mask erosion and dilation kernel size = not specified
    The morphological operation size is not reported, yet it is a hyperparameter of the proposed prompt augmentation and could affect the magnitude of the reported improvements.
assumptions (4)
  • domain assumption Rigid registration adequately aligns prior and current MR scans.
    Section 3.2 says current and prior scans are spatially aligned with rigid registration; the prior mask is then used as a prompt. Large deformations would break this assumption.
  • domain assumption Physician-approved prior annotations are available for every treatment fraction.
    Section 3.2 test protocol says prior annotation is assumed available without dropout; the method depends on this input for both the extra channels and the prior-mask prompt.
  • domain assumption SAM2-Tiny pretrained weights provide a transferable feature representation for medical images.
    Fine-tuning starts from SAM2-Tiny; if the natural-image features do not transfer to low-contrast MRI tumors, the approach would fail. This is an empirical assumption supported by prior SAM-based medical segmentation works.
  • domain assumption Ground-truth-derived bounding boxes are representative of clinical user prompts.
    The evaluation uses bounding boxes derived from the current ground-truth mask as prompts, which assumes such prompts approximate what a clinician would provide in practice. This assumption affects the reported absolute numbers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SAM2-Aug: Prior knowledge-based Augmentation for Target Volume Auto-Segmentation in Adaptive Radiation Therapy Using Segment Anything Model 2." pith.science (2026). https://pith.science/paper/M75YZ7MP

@misc{pith2026250719282,
  author       = {Pith},
  title        = {Pith review of: SAM2-Aug: Prior knowledge-based Augmentation for Target Volume Auto-Segmentation in Adaptive Radiation Therapy Using Segment Anything Model 2},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M75YZ7MP}},
  note         = {Machine review of arXiv:2507.19282}
}
read the original abstract

Purpose: Accurate tumor segmentation is vital for adaptive radiation therapy (ART) but remains time-consuming and user-dependent. Segment Anything Model 2 (SAM2) shows promise for prompt-based segmentation but struggles with tumor accuracy. We propose prior knowledge-based augmentation strategies to enhance SAM2 for ART. Methods: Two strategies were introduced to improve SAM2: (1) using prior MR images and annotations as contextual inputs, and (2) improving prompt robustness via random bounding box expansion and mask erosion/dilation. The resulting model, SAM2-Aug, was fine-tuned and tested on the One-Seq-Liver dataset (115 MRIs from 31 liver cancer patients), and evaluated without retraining on Mix-Seq-Abdomen (88 MRIs, 28 patients) and Mix-Seq-Brain (86 MRIs, 37 patients). Results: SAM2-Aug outperformed convolutional, transformer-based, and prompt-driven models across all datasets, achieving Dice scores of 0.86(liver), 0.89(abdomen), and 0.90(brain). It demonstrated strong generalization across tumor types and imaging sequences, with improved performance in boundary-sensitive metrics. Conclusions: Incorporating prior images and enhancing prompt diversity significantly boosts segmentation accuracy and generalizability. SAM2-Aug offers a robust, efficient solution for tumor segmentation in ART. Code and models will be released at https://github.com/apple1986/SAM2-Aug.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

7 extracted references · 6 canonical work pages

  1. [1]

    It is a highly effective treatment; however, its success relies on accurate tumor localization and delineation

    Introduction Radiation therapy aims to deliver a prescribed radiation dose to tumor targets while minimizing exposure to surrounding normal tissues [1, 2] . It is a highly effective treatment; however, its success relies on accurate tumor localization and delineation. Tumor delineation, mostly manual, is time -consuming, labor -intensive, and prone to int...

    work page

  2. [2]

    Data augmentation helps alleviate this by synthetically increasing data diversity [36]

    Related works 2.1 Data Augmentation Deep learning often requires large, annotated training datasets, which are limited due to cost, scarcity, and privacy concerns [35]. Data augmentation helps alleviate this by synthetically increasing data diversity [36]. Traditional techniques—e.g., rotation, flipping, scaling, and noise addition—improve generalizabilit...

    work page

  3. [3]

    The overall architecture of SAM2-Aug is illustrated in Figure 3

    Methods 7 In this work, we propose SAM2 -Aug, a comprehensive framework that enhances SAM2 through generalized data and prompt augmentation s, coupled with parameter fine-tuning for tumor segmentation in ART. The overall architecture of SAM2-Aug is illustrated in Figure 3. Built upon the SAM2 backbone, SAM2-Aug introduces three key components to enhance s...

    work page

  4. [4]

    Dataset and Implementation 4.1 Datasets To evaluate the e ffectiveness of SAM2 -Aug, we curated three MRI datasets derived from ART workflows. The first dataset, One-Seq-Liver, consists of 115 MRI scans from 31 patients diagnosed with liver cancer, all acquired using the same T2 -weighted MultiVane eXtended Dynamic sequence (T2-MVXD). There are 31 additio...

    work page

  5. [5]

    Experimental results 5.1 Comparison between SAM2-Aug and the other segmentation methods (1) Performance on the One-Seq-Liver dataset We evaluated SAM2 -Aug on the One-Seq-Liver test set and compared it with state -of-the-art segmentation methods (Table 3). Among all methods on One -Seq-Liver dataset , SAM2 -Aug achieved the best overall performance with a...

    work page

  6. [6]

    We found that simple input and prompt augmentations are effective strategies for improving tumor segmentation performance in ART

    Discussion In this study, we aimed to explore how to adapt SAM2 for ART by leveraging prior knowledge from previous treatments. We found that simple input and prompt augmentations are effective strategies for improving tumor segmentation performance in ART. However, there are still several important limitations to consider in the future: (1) Use of one/fe...

    work page

  7. [7]

    Conclusion In this study, we propose two novel and effective augmentation strategies for inputs and prompts in ART. Our results show that integrating the current MRI, pr ior MRI, and t he corresponding annotation of the prior MRI enhances tumor segmentation by providing essential prior knowledge, such as tumor location and shape, leading to more accurate ...

    arXiv 2017

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.