Pith. sign in

REVIEW 4 major objections 6 minor 38 references

TCSAFormer: Efficient Vision Transformer with Token Compression and Sparse Attention for Medical Image Segmentation

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that combining token pruning and merging before pixel-level top-k sparse attention, plus a dual-branch convolutional feed-forward network, gives medical image segmentation with better accuracy and lower FLOPs than the compa

desk verdict The architecture is a reasonable engineering combination, but the paper's own ablation contradicts its main result: full TCSAFormer scores 79.31 DSC in Tables 3/4 versus 83.16 in Table 1 on the same Synapse benchmark. read the letter →

arxiv 2508.04058 v1 pith:TEU6ZBRW submitted 2025-08-06 cs.CV

Zunhui Xia , Hongxing Li , Libin Lan This is my paper
classification cs.CV
keywords TCSAFormerMedicalimagesegmentationTokencompressionSparseattentionDual-branchfeed-forwardnetworkEfficientvisiontransformerTop-k
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes TCSAFormer, a U-shaped vision-transformer network for medical image segmentation that tries to fix two known transformer weaknesses at once: quadratic attention cost and the context-blind MLP feed-forward block. Its answer is to compress tokens before attention — pruning globally irrelevant ones and merging similar ones — then run pixel-level top-k sparse attention on the survivors, and finally decompress back to full resolution; meanwhile it replaces the MLP FFN with a dual-branch depthwise-convolution module that mixes 3x3 and 7x7 contexts. The authors claim this yields the best segmentation scores among the compared transformer methods on Synapse, ISIC-2018, and CVC-ClinicDB with lower FLOPs, i.e., a better accuracy-efficiency trade-off. A reader should care because the method suggests that token compression and sparse attention are complementary rather than competing speed-ups, and that the FFN is a worthwhile place to spend capacity in segmentation transformers.

What carries the argument

The Compressed Attention (CA) module: a Token Compression Pipeline (TCP) computes a global semantic token via global average pooling and a linear projection, scores each local token by dot-product, prunes below-threshold tokens, then merges the survivors using a bipartite-graph similarity matching (top-${r}$ edges); a Top-k Sparse Attention (TKSA) then keeps only the ${k}$ most relevant key-value pairs per query on the compressed set; a Token Decompression Pipeline (TDP) unmerges and restores pruned positions via the saved mask and indices, adding them back by a shortcut. The Dual-Branch Feed-Forward Network (DBFFN) uses a $1\times1$ convolution for channel interaction, two branches of $3\ti

What would settle it

Retrain all compared baselines under TCSAFormer's exact protocol (same resolution, optimizer, epochs, loss, and data splits) and recompute Dice/HD, mIoU/DSC, and FLOPs; if TCSAFormer no longer achieves the best scores or does not maintain lower FLOPs, the claim of a superior efficiency-accuracy trade-off fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that medical image segmentation can be made both more accurate and more efficient by (1) compressing tokens before attention — pruning globally irrelevant tokens and merging similar ones — and then (2) applying pixel-level top-k sparse attention to the compressed set, followed by decompression that restores spatial positions. It further claims that replacing the standard MLP feed-forward network with a dual-branch depthwise-convolution network (3x3 and 7x7 kernels) adds the local and multiscale context that segmentation needs. On Synapse, ISIC-2018, and CVC-ClinicDB, TCSAFormer reports the best Dice/mIoU among compared methods (83.16% average Dice on Synapse, 90.

Load-bearing premise

The reported superiority over the comparison methods assumes those baselines were evaluated under the same training protocol (resolution, pretraining, loss, schedule, data split) as TCSAFormer; the paper does not describe re-training them for the comparison.

Editorial extensions

If this is right

  • Token compression and sparse attention can be combined without sacrificing accuracy, so efficient transformers need not choose between pruning and merging.
  • Medical image segmentation can reach higher Dice scores — 83.16% on Synapse, 90.23% DSC on ISIC-2018, 94.90% DSC on CVC-ClinicDB — while using fewer FLOPs than the cited transformer baselines.
  • The dual-branch convolutional FFN is a drop-in replacement for the standard MLP FFN that improves segmentation accuracy while reducing computational cost relative to the baseline.
  • The per-stage pruning and merging ratios control the accuracy-efficiency trade-off, making the model's cost tunable via those hyperparameters.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the pruning and merging ratios are chosen per stage rather than learned, a testable extension is to make them input-dependent, letting the trade-off vary with image clutter.
  • The same prune-merge-attend pattern could transfer to other dense prediction tasks beyond medical segmentation, such as remote sensing or autonomous driving segmentation, where background tokens dominate.
  • The Grad-CAM visualizations suggest CA concentrates attention on target regions; if that holds broadly, token compression could double as an attention-regularization mechanism, predicting larger gains on cluttered images with large background areas.
  • The dual-branch FFN with $3\times3$ and $7\times7$ kernels raises the question of whether other kernel-size pairings, or learned kernel sizes, would shift the balance further.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. TCSAFormer is a U-shaped vision-transformer encoder-decoder for medical image segmentation. It introduces a Compressed Attention (CA) module that prunes and merges tokens, applies a content-aware Top-k sparse attention on the compressed set, and then decompresses to restore spatial positions; it also replaces the standard MLP FFN with a Dual-Branch Feed-Forward Network (DBFFN) using 3x3 and 7x7 depthwise convolutions to capture local and multiscale features. Experiments on Synapse, ISIC-2018, and CVC-ClinicDB report the best or second-best Dice/mIoU among the cited methods, with lower FLOPs claimed in Figure 1. The paper includes ablations comparing token-compression methods and FFN variants.

Significance. If the reported results are reproducible and the comparison is fair, the combination of token pruning, token merging, and per-query Top-k sparse attention would be a useful contribution to efficient medical image segmentation, and the DBFFN design is a sensible modification of the standard FFN. The paper's strengths include experiments on three public benchmarks, component-level ablations against several existing compression and FFN designs, and the explicit aim of reducing FLOPs while retaining accuracy. The authors also follow common practices such as ImageNet initialization and standard loss combinations. However, the current evidence contains a load-bearing internal inconsistency and several experimental-reporting gaps, so the claims cannot be verified as presented.

major comments (4)
  1. [Section 4.3.1 vs Section 4.4, Tables 1, 3, 4] The main Synapse result (Table 1) reports TCSAFormer at DSC 83.16% and HD 17.91 mm, using the Section 4.2 protocol (224x224, 400 epochs, 0.5 CE + 0.5 DICE, ImageNet init). The full model as reconstructed from the ablations — TCSAFormer with both TCP and DBFFN — is reported as DSC 79.31% and HD 23.51 mm in Table 3 (TCP row) and again in Table 4 (DBFFN row). The gaps of 3.85 DSC and 6.40 HD on the same dataset and the same architecture are too large to attribute to random seed variation. Either the ablation experiments were run under a different protocol that is not disclosed, or one set of numbers is erroneous. Because Table 1 is the central SOTA claim and Tables 3/4 are the only component-level evidence, this inconsistency is load-bearing and must be resolved.
  2. [Section 4.2, Tables 1 and 2] The training settings in Section 4.2 are given only for TCSAFormer. The paper does not state whether TransUNet, Swin-Unet, HiFormer, MISSFormer, BRAU-Net++, and other baselines were re-trained under the same resolution, image normalization, loss, scheduler, epochs, and data splits, or whether their metrics were copied from the original publications. If the baselines used different protocols, margins such as 83.16 vs 82.85 on Synapse may not reflect a fair comparison. In addition, the claimed lower FLOPs are only visualized in Figure 1 and not tabulated alongside the accuracy numbers with the source/conditions, making the efficiency claim difficult to verify.
  3. [Section 4.2, Tables 1 and 2] All quantitative results are single-run numbers with no standard deviation, no number of seeds, and no statistical significance test. The Synapse margin over MSSAANet (83.16 vs 82.85) is 0.31 DSC, and on several individual organs TCSAFormer is not the best; without variance estimates this margin is not meaningful. The same concern applies to the ablation conclusions: the differences between TCP and PITOME/ToMe (e.g., 79.31 vs 79.22/79.05) are within typical run-to-run variation for medical segmentation, so the claimed superiority of the proposed compression pipeline needs repeated-seed evidence.
  4. [Section 3.1.1, Eqs. (7)-(11)] The token-merging equations are internally inconsistent. Eq. (7) defines r = n - (rho_m*n) and the text says 'the top r edges with the highest weights are retained,' but the output X_m is in R^{(n-r)xC}. For rho_m=0.3 and n=100, r=70, which exceeds the maximum number of bipartite edges (n/2=50). The notation appears to conflate the number of removed tokens with the number of retained edges. A precise, correctly dimensional definition of rho_m and r is needed to make the method reproducible. Similarly, Eq. (4) defines X_p in R^{HW×C} as a masked version of X, while Eq. (20) uses X_p + Gather(X_d, M); the semantics of Gather and how pruned tokens are restored to their original spatial positions should be stated explicitly.
minor comments (6)
  1. [Abstract/Introduction] Typo: 'Top-k Sparse Sttention' should be 'Attention'.
  2. [Table 1] 'The best result is in Blod' should be 'Bold'.
  3. [Section 4.3.2/4.3.3, captions] The section headings say 'quantitative comparison' but the figure captions for Figs. 5 and 6 refer to 'qualitative comparison' of segmentation results; consider clarifying 'quantitative evaluation' vs 'visual comparison'.
  4. [Throughout] Minor typos: 'spacial' should be 'spatial'; 'qualitative comparison' in the CVC-ClinicDB section should be 'quantitative comparison'.
  5. [Section 3.1.2, Eq. (13)] The value of k is defined as lambda*(n-r), which must be an integer; please specify rounding (floor/ceil) and how the Top-k operation handles ties.
  6. [Section 1] The abstract and introduction state that the code is available on GitHub, but no repository URL is provided in the manuscript.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TCSAFormer's reported gains are measured against external benchmarks; no prediction reduces by construction to a fitted input or self-citation.

full rationale

TCSAFormer is an empirical architecture paper: its central claims are test-set Dice/mIoU numbers, not quantities derived from fitted parameters. The token-pruning/merging ratios and λ are chosen by grid search (Sec. 4.2), but no reported metric is identically equal to those hyperparameters by construction. The method composes prior token pruning/merging and top-k sparse attention components (cited to [19,2,24,18,34,28]); the combination is presented as a new architecture rather than a renaming. The only overlapping-author citations ([3] BRAU-Net, [13] BRAU-Net++) are used for related-work attribution and as comparative baselines; they are not invoked as a uniqueness theorem or as the sole justification for a design choice, so they are not load-bearing. I therefore find no circular step. Two non-circular weaknesses are flagged explicitly. (1) Internal inconsistency: Tables 3 and 4 both report the complete TCSAFormer at DSC 79.31 / HD 23.51 on Synapse (rows 'TCSAFormer TCP (Ours)' and 'TCSAFormer DBFFN (Ours)'), while Table 1 reports 'TCSAFormer 83.16 17.91' for the same architecture under the Sec. 4.2 protocol; this gap is too large for seed noise and makes the main comparison hard to verify, but it is a correctness/reproducibility issue, not a reduction of an output to an input. (2) The paper does not state that TransUNet, Swin-Unet, HiFormer, MISSFormer, BRAU-Net++, etc. were re-trained under identical protocols, so the SOTA margins may be affected by protocol differences; this is a comparability limitation. The conclusion also acknowledges that 'pruning thresholds and merging ratios at each stage are manually set as hyperparameters,' an adaptability limitation. None of these raise the circularity score.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or formal entities. Its proposed modules (TCP, TKSA, TDP, DBFFN) are compositions of existing neural network operations, so no independent falsifiable handles are required. The load-bearing assumptions are domain assumptions about token importance scoring, top-k relevance, decompression fidelity, baseline comparability, and pretraining transfer.

free parameters (6)
  • Token pruning ratio per encoder stage = [0.5, 0.4, 0.3, 0.1]
    Set by grid search in Section 4.2; controls how many tokens are discarded before attention and directly affects FLOPs and accuracy.
  • Token merging ratio per encoder stage = [0.3, 0.2, 0.1, 0.1]
    Set by grid search in Section 4.2; determines how many additional tokens are merged and affects information retention and compute.
  • Top-k attention ratio lambda = 1/8
    Set in Section 4.2; defines k = lambda * (n - r) keys retained per query in TKSA, a core factor in the efficiency-accuracy tradeoff.
  • Channel dimensions per encoder stage = [64, 128, 256, 512]
    Chosen by grid search in Section 4.2; sets model capacity at each stage.
  • Number of TCSAFormer blocks per stage = [2, 2, 8, 1, 1, 8, 2, 2]
    Manual architecture choice in Section 3.4; controls encoder and decoder depth.
  • Synapse loss weighting = 0.5 CE + 0.5 Dice
    Training protocol choice in Section 4.2; the loss composition affects reported performance and is not ablated against other weightings.
assumptions (5)
  • domain assumption Global average pooling plus linear projections in Eqs. 1-3 produces a reliable token-importance score for pruning.
    The entire TCP pruning step rests on this scalar score correlating with segmentation relevance; the paper gives no independent evidence beyond ablation results and a Grad-CAM figure.
  • domain assumption Top-k dot-product relevance is sufficient to select the key-value pairs needed for segmentation.
    TKSA assumes that ranking QK^T and keeping the top k entries preserves the information needed for segmentation; this is an empirical heuristic, not a proven guarantee.
  • domain assumption The token decompression pipeline in Eqs. 17-20 restores spatial topology without losing critical information.
    The mask-based shortcut and unmerge assume discarded or merged tokens are either recoverable or unnecessary; this is only tested indirectly through ablations.
  • domain assumption Baseline results in Tables 1-2 were produced under conditions comparable to TCSAFormer.
    Section 4.2 describes training settings only for TCSAFormer; the comparability of metrics taken from prior papers is assumed, not demonstrated.
  • domain assumption ImageNet-pretrained weights transfer productively to medical segmentation datasets.
    The encoder is initialized with ImageNet weights, and no pretraining-free ablation isolates the contribution of the proposed modules from the pretrained initialization.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TCSAFormer: Efficient Vision Transformer with Token Compression and Sparse Attention for Medical Image Segmentation." pith.science (2026). https://pith.science/paper/TEU6ZBRW

@misc{pith2026250804058,
  author       = {Pith},
  title        = {Pith review of: TCSAFormer: Efficient Vision Transformer with Token Compression and Sparse Attention for Medical Image Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TEU6ZBRW}},
  note         = {Machine review of arXiv:2508.04058}
}
read the original abstract

In recent years, transformer-based methods have achieved remarkable progress in medical image segmentation due to their superior ability to capture long-range dependencies. However, these methods typically suffer from two major limitations. First, their computational complexity scales quadratically with the input sequences. Second, the feed-forward network (FFN) modules in vanilla Transformers typically rely on fully connected layers, which limits models' ability to capture local contextual information and multiscale features critical for precise semantic segmentation. To address these issues, we propose an efficient medical image segmentation network, named TCSAFormer. The proposed TCSAFormer adopts two key ideas. First, it incorporates a Compressed Attention (CA) module, which combines token compression and pixel-level sparse attention to dynamically focus on the most relevant key-value pairs for each query. This is achieved by pruning globally irrelevant tokens and merging redundant ones, significantly reducing computational complexity while enhancing the model's ability to capture relationships between tokens. Second, it introduces a Dual-Branch Feed-Forward Network (DBFFN) module as a replacement for the standard FFN to capture local contextual features and multiscale information, thereby strengthening the model's feature representation capability. We conduct extensive experiments on three publicly available medical image segmentation datasets: ISIC-2018, CVC-ClinicDB, and Synapse, to evaluate the segmentation performance of TCSAFormer. Experimental results demonstrate that TCSAFormer achieves superior performance compared to existing state-of-the-art (SOTA) methods, while maintaining lower computational overhead, thus achieving an optimal trade-off between efficiency and accuracy.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 34 canonical work pages

  1. [1]

    Advances in medical image analysis with vision transformers: a comprehensive review

    Azad,R.,Kazerouni,A.,Heidari,M.,Aghdam,E.K.,Molaei,A.,Jia, Y., Jose, A., Roy, R., Merhof, D., 2024. Advances in medical image analysis with vision transformers: a comprehensive review. Medical Image Analysis 91, 103000

    work page 2024

  2. [2]

    Token merging: Your vit but faster, in: ICLR

    Bolya, D., Fu, C.Y., Dai, X., Zhang, P., Feichtenhofer, C., Hoffman, J., 2023. Token merging: Your vit but faster, in: ICLR

    work page 2023

  3. [3]

    Cai, P., Jiang, L., Li, Y., Liu, X., Lan, L., 2025. Pubic symphysis- fetalheadsegmentationnetworkusingbiformerattentionmechanism and multipath dilated convolution, in: Ide, I., Kompatsiaris, I., Xu, C., Yanai, K., Chu, W.T., Nitta, N., Riegler, M., Yamasaki, T. (Eds.), MultiMedia Modeling, Springer Nature Singapore, Singapore. pp. 243–256

    work page 2025

  4. [4]

    Ghostvit: Expediting vision transformers via cheap operations

    Cao, H., Qu, Z., Chen, G., Li, X., Thiele, L., Knoll, A., 2023. Ghostvit: Expediting vision transformers via cheap operations. IEEE Transactions on Artificial Intelligence 5, 2517–2525

    work page 2023

  5. [5]

    Swin-unet: Unet-like pure transformer for medical image segmentation,in:Europeanconferenceoncomputervision,Springer

    Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., Wang, M., 2022. Swin-unet: Unet-like pure transformer for medical image segmentation,in:Europeanconferenceoncomputervision,Springer. pp. 205–218

    work page 2022

  6. [6]

    Transunet:Transformersmakestrongencoders for medical image segmentation

    Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A.L.,Zhou,Y.,2021. Transunet:Transformersmakestrongencoders for medical image segmentation. arXiv preprint arXiv:2102.04306

    arXiv 2021

  7. [7]

    Animageisworth16x16words:Transformersforimage recognition at scale

    Dosovitskiy,A.,Beyer,L.,Kolesnikov,A.,Weissenborn,D.,Zhai,X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., etal.,2020. Animageisworth16x16words:Transformersforimage recognition at scale. arXiv preprint arXiv:2010.11929

    arXiv 2020

  8. [8]

    Cpfnet:Contextpyramidfusionnetworkfor medical image segmentation

    Feng, S., Zhao, H., Shi, F., Cheng, X., Wang, M., Ma, Y., Xiang, D., Zhu,W.,Chen,X.,2020. Cpfnet:Contextpyramidfusionnetworkfor medical image segmentation. IEEE transactions on medical imaging 39, 3008–3018

    work page 2020

Show all 38 references
  1. [9]

    Ce-net: Context encoder network for 2d medical image segmentation

    Gu, Z., Cheng, J., Fu, H., Zhou, K., Hao, H., Zhao, Y., Zhang, T., Gao, S., Liu, J., 2019. Ce-net: Context encoder network for 2d medical image segmentation. IEEE transactions on medical imaging 38, 2281–2292

    2019

  2. [10]

    6202–6212

    Heidari, M., Kazerouni, A., Soltany, M., Azad, R., Aghdam, E.K., Cohen-Adad,J.,Merhof,D.,2023.Hiformer:Hierarchicalmulti-scale representations using transformers for medical image segmentation, in: Proceedings of the IEEE/CVF winter conference on applications of computer visio...

    2023

  3. [11]

    Unet 3+: A full-scale connected unet for medical image segmentation, in: ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE

    Huang, H., Lin, L., Tong, R., Hu, H., Zhang, Q., Iwamoto, Y., Han, X., Chen, Y.W., Wu, J., 2020. Unet 3+: A full-scale connected unet for medical image segmentation, in: ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE. p...

    2020

  4. [12]

    Missformer: An effective transformer for 2d medical image segmentation

    Huang, X., Deng, Z., Li, D., Yuan, X., Fu, Y., 2022. Missformer: An effective transformer for 2d medical image segmentation. IEEE transactions on medical imaging 42, 1484–1494

    2022

  5. [13]

    Brau- net++: U-shaped hybrid cnn-transformer network for medical image segmentation

    Lan, L., Cai, P., Jiang, L., Liu, X., Li, Y., Zhang, Y., 2024. Brau- net++: U-shaped hybrid cnn-transformer network for medical image segmentation. arXiv preprint arXiv:2401.00722

    2024 arXiv

  6. [14]

    Dmsa-unet: Dual multi-scale attention makes unet more strong for medical image segmentation

    Li, X., Fu, C., Wang, Q., Zhang, W., Sham, C.W., Chen, J., 2024. Dmsa-unet: Dual multi-scale attention makes unet more strong for medical image segmentation. Knowledge-Based Systems 299, 112050

    2024

  7. [15]

    Maxformer: Enhanced transformer for medical image segmentation with multi-attention and multi-scale features fusion

    Liang, Z., Zhao, K., Liang, G., Li, S., Wu, Y., Zhou, Y., 2023. Maxformer: Enhanced transformer for medical image segmentation with multi-attention and multi-scale features fusion. Knowledge- Based Systems 280, 110987

    2023

  8. [16]

    Ds- transunet: Dual swin transformer u-net for medical image segmenta- tion

    Lin, A., Chen, B., Xu, J., Zhang, Z., Lu, G., Zhang, D., 2022. Ds- transunet: Dual swin transformer u-net for medical image segmenta- tion. IEEE Transactions on Instrumentation and Measurement 71, 1–15. doi:10.1109/TIM.2022.3178991

    2022

  9. [17]

    Adavit: Adaptive vision transformers for efficient image recognition, in: Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, pp

    Meng, L., Li, H., Chen, B.C., Lan, S., Wu, Z., Jiang, Y.G., Lim, S.N., 2022. Adavit: Adaptive vision transformers for efficient image recognition, in: Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, pp. 12309–12318

    2022

  10. [18]

    Norouzi, N., Orlova, S., De Geus, D., Dubbelman, G., 2024. Algm: Adaptive local-then-global token merging for efficient semantic seg- mentation with plain vision transformers, in: Proceedings of the IEEE/CVFConferenceonComputerVisionandPatternRecognition, pp. 15773–15782

    2024

  11. [19]

    Dynam- icvit:Efficientvisiontransformerswithdynamictokensparsification

    Rao,Y.,Zhao,W.,Liu,B.,Lu,J.,Zhou,J.,Hsieh,C.J.,2021. Dynam- icvit:Efficientvisiontransformerswithdynamictokensparsification. Advancesinneuralinformationprocessingsystems34,13937–13949

    2021

  12. [20]

    Ronneberger, O., Fischer, P., Brox, T., 2015. U-net: Convolutional networks for biomedical image segmentation, in: Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, pro- ceedings, part III...

    2015

  13. [21]

    Grad-cam:Visualexplanationsfromdeepnetworksvia gradient-basedlocalization,in:ProceedingsoftheIEEEinternational conference on computer vision, pp

    Selvaraju,R.R.,Cogswell,M.,Das,A.,Vedantam,R.,Parikh,D.,Ba- tra,D.,2017. Grad-cam:Visualexplanationsfromdeepnetworksvia gradient-basedlocalization,in:ProceedingsoftheIEEEinternational conference on computer vision, pp. 618–626

    2017

  14. [22]

    Smanet: Superpixel-guided multi-scale attention network for medical image segmentation

    Shen, Y., Guo, J., Liu, Y., Xu, C., Li, Q., Qi, F., 2025. Smanet: Superpixel-guided multi-scale attention network for medical image segmentation. Biomedical Signal Processing and Control 100, 107062

    2025

  15. [23]

    Msrf-net:amulti-scale residual fusion network for biomedical image segmentation

    Srivastava,A.,Jha,D.,Chanda,S.,Pal,U.,Johansen,H.D.,Johansen, D.,Riegler,M.A.,Ali,S.,Halvorsen,P.,2021. Msrf-net:amulti-scale residual fusion network for biomedical image segmentation. IEEE Journal of Biomedical and Health Informatics 26, 2252–2263

    2021

  16. [24]

    Patch slimming for efficient vision transformers, in: Proceedings of theIEEE/CVFConferenceonComputerVisionandPatternRecogni- tion, pp

    Tang, Y., Han, K., Wang, Y., Xu, C., Guo, J., Xu, C., Tao, D., 2022. Patch slimming for efficient vision transformers, in: Proceedings of theIEEE/CVFConferenceonComputerVisionandPatternRecogni- tion, pp. 12165–12174

    2022

  17. [25]

    Accelerating transformers with spectrum-preserving token merging

    Tran,C.,MHNguyen,D.,Nguyen,M.D.,Nguyen,T.,Le,N.,Xie,P., Sonntag, D., Zou, J.Y., Nguyen, B., Niepert, M., 2024. Accelerating transformers with spectrum-preserving token merging. Advances in Neural Information Processing Systems 37, 30772–30810

    2024

  18. [26]

    Valanarasu, J.M.J., Oza, P., Hacihaliloglu, I., Patel, V.M., 2021. Medical transformer: Gated axial-attention for medical image seg- mentation, in: Medical image computing and computer assisted intervention–MICCAI 2021: 24th international conference, Stras- bourg, France, Sept...

    2021

  19. [27]

    Mixed transformer u-net for medical image segmentation, Z

    Wang,H.,Xie,S.,Lin,L.,Iwamoto,Y.,Han,X.H.,Chen,Y.W.,Tong, R., 2022a. Mixed transformer u-net for medical image segmentation, Z. Xia et al.:Preprint submitted to Elsevier Page 10 of 11 in: ICASSP 2022-2022 IEEE international conference on acoustics, speech and signal processing...

    2022

  20. [28]

    Kvt: k-nn attention for boosting vision transformers, in: European conference on computer vision, Springer

    Wang, P., Wang, X., Wang, F., Lin, M., Chang, S., Li, H., Jin, R., 2022b. Kvt: k-nn attention for boosting vision transformers, in: European conference on computer vision, Springer. pp. 285–302

  21. [29]

    Pvt v2: Improved baselines with pyramid vision transformer

    Wang,W.,Xie,E.,Li,X.,Fan,D.P.,Song,K.,Liang,D.,Lu,T.,Luo, P., Shao, L., 2022c. Pvt v2: Improved baselines with pyramid vision transformer. Computational visual media 8, 415–424

  22. [30]

    Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J.M., Luo, P.,

  23. [31]

    Dcsau-net: A deeper and more compact split-attention u-net for medical image segmentation

    Xu, Q., Ma, Z., Duan, W., et al., 2023. Dcsau-net: A deeper and more compact split-attention u-net for medical image segmentation. Computers in Biology and Medicine 154, 106626

    2023

  24. [32]

    Msaanet: Multi-scale axial attention network for medical image segmentation, in: 2023 IEEE International Conference on Multimedia and Expo (ICME), IEEE

    Zeng, H., Shan, X., Feng, Y., Wen, Y., 2023. Msaanet: Multi-scale axial attention network for medical image segmentation, in: 2023 IEEE International Conference on Multimedia and Expo (ICME), IEEE. pp. 2291–2296

    2023

  25. [33]

    Zhang,Y.,Liu,H.,Hu,Q.,2021. Transfuse:Fusingtransformersand cnns for medical image segmentation, in: Medical image computing andcomputerassistedintervention–MICCAI2021:24thinternational conference, Strasbourg, France, September 27–October 1, 2021, pro- ceedings, Part I 24, Spr...

    2021

  26. [34]

    Explicit sparsetransformer:Concentratedattentionthroughexplicitselection

    Zhao, G., Lin, J., Zhang, Z., Ren, X., Su, Q., Sun, X., 2019. Explicit sparsetransformer:Concentratedattentionthroughexplicitselection. arXiv preprint arXiv:1912.11637

    2019 arXiv

  27. [35]

    2881–2890

    Zhao,H.,Shi,J.,Qi,X.,Wang,X.,Jia,J.,2017.Pyramidsceneparsing network, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2881–2890

    2017

  28. [36]

    Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J., 2018. Unet++:Anestedu-netarchitectureformedicalimagesegmentation, in:Deeplearninginmedicalimageanalysisandmultimodallearning for clinical decision support: 4th international workshop, DLMIA 2018, and 8th international...

    2018

  29. [37]

    Biformer: Vision transformer with bi-level routing attention, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recogni- tion, pp

    Zhu, L., Wang, X., Ke, Z., Zhang, W., Lau, R.W., 2023. Biformer: Vision transformer with bi-level routing attention, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recogni- tion, pp. 10323–10333. Z. Xia et al.:Preprint submitted to Elsevier Page 11 of 11

    2023

  30. [2021]

    Advances in neural information processing systems 34, 12077–12090

    Segformer: Simple and efficient design for semantic segmen- tation with transformers. Advances in neural information processing systems 34, 12077–12090

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.