Pith. sign in

REVIEW 4 major objections 6 minor 51 references

MedFormer: Hierarchical Medical Vision Transformer with Content-Aware Dual Sparse Selection Attention

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MedFormer claims a single transformer backbone with dual sparse selection attention that outperforms specialized medical models across classification, segmentation, and detection.

desk verdict DSSA is a genuinely new sparse-attention variant with solid empirical coverage, but the FLOPs proof in Sec. III-B is logically broken and the hyperparameters are tuned on the test sets. read the letter →

arxiv 2507.02488 v2 pith:6BEPCA72 submitted 2025-07-03 cs.CV

Zunhui Xia , Hongxing Li , Libin Lan This is my paper
classification cs.CV
keywords medicalimageclassificationsemanticsegmentationlesiondetectionvisiontransformersparseattentioncontent-awarehierarchicalbackbonenoiserobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MedFormer is a medical vision transformer designed as a single general-purpose backbone for classification, semantic segmentation, and lesion detection. The authors claim that its four-stage pyramid structure provides hierarchical features for dense prediction, and that its Dual Sparse Selection Attention (DSSA) makes the model faster and more noise-robust than full attention by selecting the most relevant regions and then the most relevant pixels. If this holds, medical imaging teams could replace task-specific transformer designs with one backbone that works across modalities such as dermoscopy, colonoscopy, MRI, and CT. Experiments across eight datasets support the claim that MedFormer matches or beats specialized models while using fewer parameters and FLOPs in several configurations. The authors also report a known weak spot: on multi-organ segmentation, boundary tokens tend to be dropped, showing up as worse Hausdorff distance.

What carries the argument

The central object is the Dual Sparse Selection Attention (DSSA) layer. It performs two explicit sparse selections: a region-level top-$k_1$ selection on region-averaged queries and keys, and a pixel-level top-$k_2$ selection within the surviving regions, so each query attends only to a small, content-chosen set of pixels. This mechanism carries the paper's efficiency and robustness claims: it cuts the token count entering the attention matrix, filters out noise tokens before computing attention, and is inserted into a four-stage pyramid (patch embedding, three patch-merging stages, and $\{2,2,8,2\}$ blocks in the Small variant) to produce multiscale features for dense prediction tasks.

What would settle it

Take a fixed configuration, e.g. $H=W=224$, $C=384$, $S=7$, $k_1=16$, $k_2=\lambda k_1 HW/S^2$ with $\lambda=1/8$, plug the actual operation counts from Eq. (11) into a script, and check whether the true FLOPs is below $3HWC^2+6C(k_1)^{2/3}(HW)^{4/3}$; the paper's complexity conclusion is false if the inequality is violated.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that a content-aware, twice-sparse attention mechanism can deliver the global modeling of a transformer without the quadratic cost or the noise sensitivity of full attention. DSSA first partitions the feature map into $S \times S$ regions, averages queries and keys inside each region, and keeps the $k_1$ most relevant regions per query; it then gathers pixel-level keys and values from those regions and keeps only the $k_2$ most relevant pixels per query. A full pixel-to-pixel attention is then computed on this small gathered set, followed by a $5 \times 5$ depth-wise convolution for local context. The authors argue that this makes DSSA's complexity less than $\mathcal{O}((HW)^{4/3})$, a substantial reduction from vanilla attention, and that it improves accuracy by excluding noise pixels. They further claim that a four-stage pyramid of such blocks, MedFormer, consistently enhances performance on medical image classification, segmentation, and detection across eight datasets.

Load-bearing premise

The efficiency claim stands on a single inequality step: the paper treats a lower bound on an intermediate expression as an upper bound on FLOPs, and if that step is invalid the claimed complexity advantage over full attention collapses.

Editorial extensions

If this is right

  • If DSSA's noise-filtering behavior is real, lesion and polyp models should need fewer tokens to reach the same accuracy, because attention is concentrated on clinically relevant content.
  • If the FLOPs bound holds, the complexity of DSSA scales sub-quadratically in image area, so MedFormer can be applied to larger medical images than full-attention transformers.
  • If MedFormer works as a general backbone, the same pretrained pyramid can be reused for classification, segmentation, and detection heads across different imaging modalities.
  • The authors' reported drop in Hausdorff distance on Synapse implies that tasks where organ boundaries are the focus may need a boundary-aware supplement to the sparse selection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the same region-then-pixel two-stage selection could be applied to video or volumetric data, where region-level pruning would cut most of the attention cost before pixel-level selection.
  • A corrected complexity analysis would need to bound FLOPs directly rather than through the intermediate expression; until then, the practical efficiency advantage should be judged from measured throughput and memory rather than from the stated bound.
  • The paper's boundary-token observation suggests a testable extension: adding a small local or boundary-aware branch could recover edge fidelity without sacrificing the global sparse attention, and should be evaluated on Hausdorff distance.
  • The ablation comparing DSSA against shifted-window, explicit sparse, deformable, and bi-level routing attention could be extended to modality-specific datasets to test whether content-aware selection matters most in noisy modalities such as ultrasound.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. MedFormer is a four-stage hierarchical vision transformer proposed as a general-purpose backbone for medical image classification, semantic segmentation, and lesion detection. Its core attention module, DSSA, first selects the k1 most relevant regions for each query and then selects the k2 most relevant pixel-level tokens within those regions before computing attention. The paper claims a theoretical FLOPs bound below O((HW)^(4/3)) and reports experiments on eight datasets, comparing against task-specific medical methods and general-purpose backbones. The empirical results show competitive accuracy, e.g., MedFormer-B* reaching 84.07% DSC on Synapse and 94.57% top-1 accuracy on ISIC-2018 classification, together with ablations of attention mechanisms, the LCE module, hyperparameters, model depth, and attention visualizations.

Significance. The breadth of experiments and the attempt at a general-purpose medical backbone are valuable; if the DSSA design and reported gains replicate, it could be a useful architecture for the medical imaging community. The manuscript compares against many relevant baselines and includes ablation studies, which is a strength. However, the central theoretical efficiency claim is invalid: the proof in Section III-B derives a lower bound and then treats it as an upper bound on FLOPs, and the actual token-attention term is O((HW)^2) under the fixed S and k1 settings in Table I. Because efficiency is a stated primary contribution, this is a load-bearing error. The selection of key hyperparameters on the evaluation datasets further weakens the generalizability claims, and the measured FLOPs do not consistently support a practical efficiency advantage over the compared backbones.

major comments (4)
  1. [III-B, Eqs. (12)-(14)] The complexity proof is logically invalid. Equation (12) establishes FLOPs < InEq, and Equation (13) establishes InEq > L, where L = 3HWC^2 + 6C(k1)^(2/3)(HW)^(4/3). From FLOPs < InEq and InEq > L, nothing follows about the relation between FLOPs and L, so the conclusion in Equation (14) is a non sequitur. Moreover, applying Equation (12) directly gives FLOPs = O((HW)^2) for fixed S, k1, and lambda, because the token-attention term in Equation (11) is (1+lambda)k1(C+1)(HW)^2/S^2. At Stage 4 in Table I, k1 = S^2 = 49, so the region-level selection retains all regions and the token-level attention is quadratic in HW. Thus the advertised sub-4/3 complexity is not established and, with the stated hyperparameters, is false.
  2. [III-B, Eq. (13)] Even setting aside the logical direction, the AM-GM evaluation is arithmetically incorrect. The cube root of the product 2(S^2)^2C * k1(HW)^2/S^2 C * k1(HW)^2/S^2 C is 2^(1/3) C (k1)^(2/3)(HW)^(4/3), so the coefficient should be 3*2^(1/3), not 6. More fundamentally, AM-GM provides a lower bound on InEq, and a lower bound on InEq cannot serve as an upper bound on FLOPs; this step cannot be repaired by correcting the constant.
  3. [IV-E3, Tables IX and X] The key hyperparameters S, the k1 schedule, lambda, and the network depth are selected by maximizing accuracy or DSC on ISIC-2018 Classification and CVC-ClinicDB, which are also used as evaluation datasets in the main comparisons. Because the same datasets are reused for tuning and evaluation, the reported performance gaps on those datasets are partly due to selection and cannot support the claim of 'potential generalizability across tasks and datasets' without a held-out validation protocol or an additional independent evaluation set.
  4. [Tables III and VI; Abstract] The claimed efficiency advantage is not supported by the measured FLOPs. In Table III, MedFormer-B reports 4.79G FLOPs versus 4.61G for ViT-S and 4.49G for BiFormer-S; in Table VI, MedFormer-B reports 13.34G FLOPs versus 13.32G for BiFormer-S. Thus, even if the asymptotic proof were repaired, the empirical results do not establish a systematic reduction in computational cost relative to full attention or routing attention, and the abstract's efficiency claim would remain unsupported.
minor comments (6)
  1. [Section I, Contributions] In the third contribution bullet, 'DDSA' should be 'DSSA' to match the notation used throughout the paper.
  2. [Figure 1 caption] The caption contains an incomplete sentence: 'represents the current query's position. represent the pixels or local windows that the query focuses on, respectively.' Please rewrite this as a complete sentence.
  3. [Table IV caption] The caption says 'THE BEST RESULT IS IN BLOD' rather than 'BOLD'; please correct the typo.
  4. [Table V header] The header row appears to repeat 'Params (M) FLOPs (G)' and the column grouping is confusing; please reformat so that each numeric column is labeled once and the metric groups are clear.
  5. [IV-B2] The text says pretrained MedFormer versions are 'not incorporated into the formal comparisons,' but the pretrained rows are printed in Table III; please clarify whether these rows are intended as part of the comparison or as a separate illustration.
  6. [III-B, Eq. (12)] The sentence 'we then substitute S2C and k1HW/S2 into Eq. 11' is unclear; the intended replacements are S^2 + k1 < S^2 C and k2 < k1HW/S^2, and the derivation should be stated accordingly.

Circularity Check

1 steps flagged · score 4.0 of 10

Hyperparameter selection on evaluation datasets inflates two reported benchmarks; core DSSA design remains independently supported.

  1. fitted input called prediction [Section IV-E3 (Table IX) and Section IV-E5 (Table X); final results in Tables III and V]
    "Ultimately, the k1 and λ hyperparameters we selected achieve the best performance on both datasets, demonstrating their potential generalizability across tasks and datasets. ... To balance best performance and computational efficiency, we set the depth to [2,2,8,2]."

    Table IX reports that the configuration k1=(1,4,16,49), λ=1/8 achieves 88.24% on ISIC-2018 and 88.23% DSC on CVC-ClinicDB, and the text states 'the k1 and λ hyperparameters we selected achieve the best performance on both datasets'. The final MedFormer-S results in Table III and Table V use that same configuration and report the same 88.24% and 88.23% numbers, which are therefore the maxima of the grid searched on the evaluation sets, not independent test-set predictions. Table X similarly selects depth [2,2,8,2] by maximizing ISIC-2018 accuracy, after which that same accuracy is reported as the final result.

full rationale

The paper's core contribution, the Dual Sparse Selection Attention (DSSA), is evaluated against external baselines (Shifted window, Explicit Sparse, Deformable, Bi-level Routing) in Table VII, showing a performance advantage that is independent of the hyperparameter selection. However, the final reported accuracy for ISIC-2018 classification (88.24%) and DSC for CVC-ClinicDB (88.23%) are the very numbers used to select k1, λ, and depth in Tables IX and X on the same datasets. Thus, part of the claimed improvement on those two benchmarks reduces to fitting hyperparameters to the evaluation set. Other datasets were not used in the selection, and the mechanism itself is not derived from those benchmark numbers, so the circularity is partial, not complete. The FLOPs bound in Section III-B is logically flawed (Eq. 12 and Eq. 13 do not jointly imply Eq. 14), but that is a proof error, not a circular reduction to inputs, and therefore does not contribute to the circularity score.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central claim depends on the hand-chosen hyperparameters S, k1, lambda, and depth, which were tuned on the evaluation benchmarks. The theoretical efficiency claim additionally relies on a misapplication of AM-GM, which is tracked as an axiom. No new physical entities are introduced.

free parameters (5)
  • S (region partition factor) = 7 (224x224), 8 (256x256)
    Hand-chosen; controls region granularity and hence the candidate pool for sparse selection.
  • k1 schedule per stage = [1,4,16,49] for MedFormer-S/B; variants [1,2,8,49] tested
    Number of regions retained at each stage; selected by ablation on ISIC-2018 and CVC-ClinicDB.
  • lambda (pixel selection ratio) = 1/8
    Fraction of gathered pixel tokens retained in the second sparse selection; chosen among 1, 1/4, 1/8, 1/16 by validation/test performance.
  • Network depth (N1..N4) = [2,2,8,2]
    Chosen via ablation Table X on ISIC-2018 classification.
  • MLP expansion ratio e = 3
    Fixed design choice, not ablated.
assumptions (3)
  • domain assumption Dot-product similarity in projected key/query space identifies the most semantically relevant tokens for medical tasks.
    DSSA's content-awareness relies on top-k by inner product reflecting task usefulness; no proof or calibration against medical priors (Section III-A).
  • standard math FLOPs can be counted by treating top-k selection cost as O(S^2(S^2+k1)) and O(HW(k1HW/S^2+k2)).
    The complexity analysis adopts a specific counting convention for top-k; real GPU top-k costs differ, so the FLOPs comparison is approximate (Section III-B).
  • ad hoc to paper AM-GM inequality provides a valid upper bound for the FLOPs expression.
    The paper applies AM-GM to produce a lower bound and then uses it as an upper bound; this is the specific invalid step in Eqs. 13 and 14.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MedFormer: Hierarchical Medical Vision Transformer with Content-Aware Dual Sparse Selection Attention." pith.science (2026). https://pith.science/paper/6BEPCA72

@misc{pith2026250702488,
  author       = {Pith},
  title        = {Pith review of: MedFormer: Hierarchical Medical Vision Transformer with Content-Aware Dual Sparse Selection Attention},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6BEPCA72}},
  note         = {Machine review of arXiv:2507.02488}
}
read the original abstract

Medical image recognition serves as a key way to aid in clinical diagnosis, enabling more accurate and timely identification of diseases and abnormalities. Vision transformer-based approaches have proven effective in handling various medical recognition tasks. However, these methods encounter two primary challenges. First, they are often task-specific and architecture-tailored, limiting their general applicability. Second, they usually either adopt full attention to model long-range dependencies, resulting in high computational costs, or rely on handcrafted sparse attention, potentially leading to suboptimal performance. To tackle these issues, we present MedFormer, an efficient medical vision transformer with two key ideas. First, it employs a pyramid scaling structure as a versatile backbone for various medical image recognition tasks, including image classification and dense prediction tasks such as semantic segmentation and lesion detection. This structure facilitates hierarchical feature representation while reducing the computation load of feature maps, highly beneficial for boosting performance. Second, it introduces a novel Dual Sparse Selection Attention (DSSA) with content awareness to improve computational efficiency and robustness against noise while maintaining high performance. As the core building technique of MedFormer, DSSA is designed to explicitly attend to the most relevant content. Theoretical analysis demonstrates that MedFormer outperforms existing medical vision transformers in terms of generality and efficiency. Extensive experiments across various imaging modality datasets show that MedFormer consistently enhances performance in all three medical image recognition tasks mentioned above. MedFormer provides an efficient and versatile solution for medical image recognition, with strong potential for clinical application.

Figures

Figures reproduced from arXiv: 2507.02488 by the authors.

Figure 1
Figure 1. Comparison of different transformer architectures and corresponding attention mechanisms, where “V-E”, “S-E”, “B-E”, and “M-E” represent Vision Transformer Encoder, Swin Transformer Encoder, BiFormer Encoder, and MedFormer Encoder, respectively. (a) Vision Transformer [16] uses full attention, where all layers output feature maps with the same resolution. (b-d) adopt various sparse attention and produce multiscale f… view at source ↗
Figure 2
Figure 2. Illustration of dual sparse selection attention (DSSA). First, region-level token sparse selection is performed by obtaining queries and key features through region averaging, selecting the top-k1 relevant regions. Then, pixel-level token sparse selection is conducted within these selected regions, choosing the top-k2 relevant pixel tokens. Finally, matrix multiplication is applied to obtain the output. The mechanis… view at source ↗
Figure 3
Figure 3. Top: The overall architecture of MedFormer is illustrated, presenting a four-stage hierarchical pyramid network. At each stage i, the input feature maps undergo patch embedding or patch merging layer to reduce spatial resolution while increasing the number of channels. Each stage contains Ni stacked MedFormer Blocks. This design helps to extract multiscale features while maintaining computational efficiency. Bottom:… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Visualization of attention maps from sample images in the ISIC 2018 Classification dataset (top) and the CVC-ClinicDB segmentation dataset (bottom), illustrating the effects of excluding noise tokens. The pink regions in the segmented samples serve as the ground truth.…
Figure 5
Figure 5. Figure 5: Visualization of attention maps on ISIC-2018 Classification dataset. For each input, we select two different query positions to observe the region-level tokens they attend to and the pixel-level tokens within those regions. The heatmap values of the pixel-level tokens …
Figure 6
Figure 6. Figure 6: On the ISIC-2018 Classification dataset, we visualize the regional attention maps at a fixed query position across network depths ranging from 1 to 12. The number of regions each query is determined by k1. aorta gallbladder left kidney right kidney liver pancreas splee…
Figure 7
Figure 7. Figure 7: Visualization of different methods on the Synapse dataset. MedFormer achieves finer segmentation results compared to other methods, showing a close alignment with the Ground Truth. In contrast, other approaches suffer from category confusion or fragmented segmentation …
Figure 8
Figure 8. Figure 8: Visualizations of Segmentation using various methods on the CVC-ClinicDB and ISIC-2018 Segmentation datasets. The lesion regions of GT are shown in red, while the segmentation results of methods are shown in pink. Kvasir_Seg Brain Tumor Detection GT PVT DAT BiFormer Sw…
Figure 9
Figure 9. Figure 9: Visualizations of object detection using various methods on the Kvasir-Seg and Brain Tumor Detection datasets. The Kvasir-Seg dataset consists of only one category, represented in red. The Brain Tumor Detection dataset contains three categories, represented respectivel…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

51 extracted references · 32 canonical work pages

  1. [1]

    Hematoma expansion prediction: still navigating the intersection of deep learning and radiomics,

    N. Q. K. Le, “Hematoma expansion prediction: still navigating the intersection of deep learning and radiomics,” European Radiology , vol. 34, no. 5, pp. 2905–2907, 2024

    work page 2024

  2. [2]

    Improved prediction model of protein and peptide toxicity by integrating channel attention into a convolutional neural network and gated recurrent units,

    Z. Zhao, J. Gui, A. Yao, N. Q. K. Le, and M. C. H. Chua, “Improved prediction model of protein and peptide toxicity by integrating channel attention into a convolutional neural network and gated recurrent units,” ACS omega, vol. 7, no. 44, pp. 40 569–40 577, 2022

    work page 2022

  3. [3]

    Transformers in medical imaging: A survey,

    F. Shamshad, S. Khan, S. W. Zamir, M. H. Khan, M. Hayat, F. S. Khan, and H. Fu, “Transformers in medical imaging: A survey,”Medical Image Analysis, vol. 88, p. 102802, 2023

    2023

  4. [4]

    Advances in medical image analysis with vision transformers: A comprehensive review,

    R. Azad, A. Kazerouni, M. Heidari, E. K. Aghdam, A. Molaei, Y . Jia, A. Jose, R. Roy, and D. Merhof, “Advances in medical image analysis with vision transformers: A comprehensive review,” Medical Image Analysis, vol. 91, p. 103000, 2024

    work page 2024

  5. [5]

    Is it time to replace cnns with transformers for medical images?

    C. Matsoukas, J. F. Haslum, M. S ¨oderberg, and K. Smith, “Is it time to replace cnns with transformers for medical images?” arXiv preprint arXiv:2108.09038, 2021

    arXiv 2021

  6. [6]

    Lesion-aware transformers for diabetic retinopathy grading,

    R. Sun, Y . Li, T. Zhang, Z. Mao, F. Wu, and Y . Zhang, “Lesion-aware transformers for diabetic retinopathy grading,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 10 938–10 947

    work page 2021

  7. [7]

    Transmil: Transformer based correlated multiple instance learning for whole slide image classification,

    Z. Shao, H. Bian, Y . Chen, Y . Wang, J. Zhang, X. Ji et al., “Transmil: Transformer based correlated multiple instance learning for whole slide image classification,” Advances in neural information processing XIA et al.: MEDFORMER: HIERARCHICAL MEDICAL VISION TRANSFORMER WITH CONTENT -AWARE DUAL SPARSE SELECTION ATTENTION 13 CVC_ClinicDB ISIC-2018 Segmen...

    work page 2018

  8. [8]

    A graph-transformer for whole slide image classification,

    Y . Zheng, R. H. Gindra, E. J. Green, E. J. Burks, M. Betke, J. E. Beane, and V . B. Kolachalama, “A graph-transformer for whole slide image classification,” IEEE transactions on medical imaging , vol. 41, no. 11, pp. 3003–3015, 2022

    work page 2022

Show all 51 references
  1. [9]

    Transunet: Transformers make strong encoders for medical image segmentation,

    J. Chen, Y . Lu, Q. Yu, X. Luo, E. Adeli, Y . Wang, L. Lu, A. L. Yuille, and Y . Zhou, “Transunet: Transformers make strong encoders for medical image segmentation,” arXiv preprint arXiv:2102.04306 , 2021

    2021 arXiv

  2. [10]

    Transfuse: Fusing transformers and cnns for medical image segmentation

    Y . Zhang, H. Liu, and Q. Hu, “Transfuse: Fusing transformers and cnns for medical image segmentation.” Springer, 2021, pp. 14–24

    2021

  3. [11]

    Swin-unet: Unet-like pure transformer for medical image segmenta- tion,

    H. Cao, Y . Wang, J. Chen, D. Jiang, X. Zhang, Q. Tian, and M. Wang, “Swin-unet: Unet-like pure transformer for medical image segmenta- tion,” in European conference on computer vision . Springer, 2022, pp. 205–218

    2022

  4. [12]

    Pubic symphysis-fetal head segmentation network using biformer attention mechanism and multipath dilated convolution,

    P. Cai, L. Jiang, Y . Li, X. Liu, and L. Lan, “Pubic symphysis-fetal head segmentation network using biformer attention mechanism and multipath dilated convolution,” in MultiMedia Modeling , I. Ide, I. Kompatsiaris, C. Xu, K. Yanai, W.-T. Chu, N. Nitta, M. Riegler, and T. Yama...

    2025

  5. [13]

    Brau-net++: U-shaped hybrid cnn-transformer network for medical image segmenta- tion,

    L. Lan, P. Cai, L. Jiang, X. Liu, Y . Li, and Y . Zhang, “Brau-net++: U-shaped hybrid cnn-transformer network for medical image segmenta- tion,” arXiv preprint arXiv:2401.00722 , 2024

    2024 arXiv

  6. [14]

    Spine-transformers: Vertebra detection and local- ization in arbitrary field-of-view spine ct with transformers

    R. Tao and G. Zheng, “Spine-transformers: Vertebra detection and local- ization in arbitrary field-of-view spine ct with transformers.” Springer, 2021, pp. 93–103

    2021

  7. [15]

    Multi-view vertebra localization and identification from ct images,

    H. Wu, J. Zhang, Y . Fang, Z. Liu, N. Wang, Z. Cui, and D. Shen, “Multi-view vertebra localization and identification from ct images,” in International Conference on Medical Image Computing and Computer- Assisted Intervention. Springer, 2023, pp. 136–145

    2023

  8. [16]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929 , 2020

    2010 arXiv

  9. [17]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 10 012–10 022

    2021

  10. [18]

    Biformer: Vision transformer with bi-level routing attention,

    L. Zhu, X. Wang, Z. Ke, W. Zhang, and R. W. Lau, “Biformer: Vision transformer with bi-level routing attention,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2023, pp. 10 323–10 333

    2023

  11. [19]

    Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers,

    S. Zheng, J. Lu, H. Zhao, X. Zhu, Z. Luo, Y . Wang, Y . Fu, J. Feng, T. Xiang, P. H. Torr et al. , “Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition...

    2021

  12. [20]

    Medical transformer: Gated axial-attention for medical image segmentation

    J. M. J. Valanarasu, P. Oza, I. Hacihaliloglu, and V . M. Patel, “Medical transformer: Gated axial-attention for medical image segmentation.” Springer, 2021, pp. 36–46

    2021

  13. [21]

    Pyramid vision transformer: A versatile backbone for dense prediction without convolutions,

    W. Wang, E. Xie, X. Li, D.-P. Fan, K. Song, D. Liang, T. Lu, P. Luo, and L. Shao, “Pyramid vision transformer: A versatile backbone for dense prediction without convolutions,” in Proceedings of the IEEE/CVF international conference on computer vision , 2021, pp. 568–578

    2021

  14. [22]

    Explicit sparse transformer: Concentrated attention through explicit selection,

    G. Zhao, J. Lin, Z. Zhang, X. Ren, Q. Su, and X. Sun, “Explicit sparse transformer: Concentrated attention through explicit selection,” arXiv preprint arXiv:1912.11637, 2019

    1912 arXiv

  15. [23]

    Mil-vt: Multiple instance learning enhanced vision transformer for fundus image classification

    S. Yu, K. Ma, Q. Bi, C. Bian, M. Ning, N. He, Y . Li, H. Liu, and Y . Zheng, “Mil-vt: Multiple instance learning enhanced vision transformer for fundus image classification.” Springer, 2021, pp. 45–54

    2021

  16. [24]

    Self-ensembling vision transformer (sevit) for robust medical image classification

    F. Almalik, M. Yaqub, and K. Nandakumar, “Self-ensembling vision transformer (sevit) for robust medical image classification.” Berlin, Heidelberg: Springer-Verlag, 2022, p. 376–386

    2022

  17. [25]

    Boundary- aware transformers for skin lesion segmentation

    J. Wang, L. Wei, L. Wang, Q. Zhou, L. Zhu, and J. Qin, “Boundary- aware transformers for skin lesion segmentation.” Springer, 2021, pp. 206–216

    2021

  18. [26]

    Transformer network for significant stenosis detection in ccta of coronary arteries,

    X. Ma, G. Luo, W. Wang, and K. Wang, “Transformer network for significant stenosis detection in ccta of coronary arteries,” in Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 . Cham: Springer International Publishing, 2021, pp. 516–525

    2021

  19. [27]

    Ida-net: Inheritable deformable attention network of structural mri for alzheimer’s disease diagnosis,

    Q. Zhao, G. Huang, P. Xu, Z. Chen, W. Li, X. Yuan, G. Zhong, C.-M. Pun, and Z. Huang, “Ida-net: Inheritable deformable attention network of structural mri for alzheimer’s disease diagnosis,” Biomedical Signal Processing and Control, vol. 84, p. 104787, 2023

    2023

  20. [28]

    Vision transformer with deformable attention,

    Z. Xia, X. Pan, S. Song, L. E. Li, and G. Huang, “Vision transformer with deformable attention,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 4794–4803

    2022

  21. [29]

    Hifuse: Hierarchical multi-scale feature fusion network for medical image classification,

    X. Huo, G. Sun, S. Tian, Y . Wang, L. Yu, J. Long, W. Zhang, and A. Li, “Hifuse: Hierarchical multi-scale feature fusion network for medical image classification,” Biomedical Signal Processing and Control, vol. 87, p. 105534, 2024

    2024

  22. [30]

    Pocformer: A lightweight transformer architecture for detection of covid-19 using point of care ultrasound,

    S. Perera, S. Adhikari, and A. Yilmaz, “Pocformer: A lightweight transformer architecture for detection of covid-19 using point of care ultrasound,” in 2021 IEEE international conference on image processing (ICIP). IEEE, 2021, pp. 195–199

    2021

  23. [31]

    Linformer: Self-attention with linear complexity,

    S. Wang, B. Z. Li, M. Khabsa, H. Fang, and H. Ma, “Linformer: Self-attention with linear complexity,” 2020. [Online]. Available: https://arxiv.org/abs/2006.04768

    2020 arXiv

  24. [32]

    Kernel attention transformer for histopathology whole slide image analysis and assistant cancer diagnosis,

    Y . Zheng, J. Li, J. Shi, F. Xie, J. Huai, M. Cao, and Z. Jiang, “Kernel attention transformer for histopathology whole slide image analysis and assistant cancer diagnosis,” IEEE Transactions on Medical Imaging , vol. 42, no. 9, pp. 2726–2739, 2023

    2023

  25. [33]

    Mg-trans: Multi-scale graph transformer with information bottleneck for whole slide image classification,

    J. Shi, L. Tang, Z. Gao, Y . Li, C. Wang, T. Gong, C. Li, and H. Fu, “Mg-trans: Multi-scale graph transformer with information bottleneck for whole slide image classification,” IEEE Transactions on Medical Imaging, vol. 42, no. 12, pp. 3871–3883, 2023

    2023

  26. [34]

    Msaanet: Multi-scale axial attention network for medical image segmentation,

    H. Zeng, X. Shan, Y . Feng, and Y . Wen, “Msaanet: Multi-scale axial attention network for medical image segmentation,” in 2023 IEEE International Conference on Multimedia and Expo (ICME) . IEEE, 2023, pp. 2291–2296

    2023

  27. [35]

    Hiformer: Hierarchical multi-scale representations using transformers for medical image segmentation,

    M. Heidari, A. Kazerouni, M. Soltany, R. Azad, E. K. Aghdam, J. Cohen-Adad, and D. Merhof, “Hiformer: Hierarchical multi-scale representations using transformers for medical image segmentation,” in Proceedings of the IEEE/CVF winter conference on applications of computer visio...

    2023

  28. [36]

    Maxformer: Enhanced transformer for medical image segmentation with multi- attention and multi-scale features fusion,

    Z. Liang, K. Zhao, G. Liang, S. Li, Y . Wu, and Y . Zhou, “Maxformer: Enhanced transformer for medical image segmentation with multi- attention and multi-scale features fusion,” Knowledge-Based Systems , vol. 280, p. 110987, 2023

    2023

  29. [37]

    Missformer: An effective transformer for 2d medical image segmentation,

    X. Huang, Z. Deng, D. Li, X. Yuan, and Y . Fu, “Missformer: An effective transformer for 2d medical image segmentation,” IEEE Transactions on Medical Imaging, vol. 42, no. 5, pp. 1484–1494, 2022

    2022

  30. [38]

    The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions,

    P. Tschandl, C. Rosendahl, and H. Kittler, “The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions,” Scientific data, vol. 5, no. 1, pp. 1–9, 2018

    2018

  31. [39]

    Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic),

    N. Codella, V . Rotemberg, P. Tschandl, M. E. Celebi, S. Dusza, D. Gutman, B. Helba, A. Kalloo, K. Liopyris, M. Marchetti et al., “Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic),” arXiv preprint ar...

    2018 arXiv

  32. [40]

    A real-world dataset and benchmark for foundation model adaptation in medical image classification,

    D. Wang, X. Wang, L. Wang, M. Li, Q. Da, X. Liu, X. Gao, J. Shen, J. He, T. Shen et al., “A real-world dataset and benchmark for foundation model adaptation in medical image classification,” Scientific Data , vol. 10, no. 1, p. 574, 2023

    2023

  33. [41]

    Brain tumor mri dataset,

    M. Nickparvar, “Brain tumor mri dataset,” 2021. [Online]. Available: https://www.kaggle.com/dsv/2645886

    2021

  34. [42]

    Wm-dova maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians,

    J. Bernal, F. J. S ´anchez, G. Fern ´andez-Esparrach, D. Gil, C. Rodr ´ıguez, and F. Vilari ˜no, “Wm-dova maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians,” Comput- erized medical imaging and graphics , vol. 43, pp. 99–111, 2015

    2015

  35. [43]

    Miccai multi-atlas labeling beyond the cranial vault-workshop and challenge (2015),

    B. Landman, Z. Xu, J. Igelsias, M. Styner, T. Langerak, and A. Klein, “Miccai multi-atlas labeling beyond the cranial vault-workshop and challenge (2015),” vol. 2, 2015

    2015

  36. [44]

    Kvasir-seg: A segmented polyp dataset,

    D. Jha, P. H. Smedsrud, M. A. Riegler, P. Halvorsen, T. de Lange, D. Jo- hansen, and H. D. Johansen, “Kvasir-seg: A segmented polyp dataset,” in International Conference on Multimedia Modeling . Springer, 2020, pp. 451–462

    2020

  37. [45]

    Brain tumor detection dataset,

    Y . Ghanem, “Brain tumor detection dataset,” jul 2022, visited on 2024-06-19. [Online]. Available: https://universe.roboflow.com/ yousef-ghanem-jzj4y/brain-tumor-detection-fpf1f

    2022

  38. [46]

    Focal- unet: Unet-like focal modulation for medical image segmentation,

    M. Naderi, M. Givkashi, F. Piri, N. Karimi, and S. Samavi, “Focal- unet: Unet-like focal modulation for medical image segmentation,”arXiv preprint arXiv:2212.09263, 2022

    2022 arXiv

  39. [47]

    Medical image segmentation via cascaded attention decoding,

    M. M. Rahman and R. Marculescu, “Medical image segmentation via cascaded attention decoding,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2023, pp. 6222–6231

    2023

  40. [48]

    Focal loss for dense object detection,

    T.-Y . Lin, P. Goyal, R. Girshick, K. He, and P. Doll ´ar, “Focal loss for dense object detection,” in Proceedings of the IEEE international conference on computer vision , 2017, pp. 2980–2988

    2017

  41. [49]

    Faster r-cnn: Towards real-time object detection with region proposal networks,

    S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 6, pp. 1137–1149, 2016. XIA et al.: MEDFORMER: HIERARCHICAL MEDICAL VISION TRA...

    2016

  42. [50]

    Deformable detr: Deformable transformers for end-to-end object detection,

    X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, “Deformable detr: Deformable transformers for end-to-end object detection,”arXiv preprint arXiv:2010.04159, 2020

    2010 arXiv

  43. [51]

    Unified perceptual parsing for scene understanding,

    T. Xiao, Y . Liu, B. Zhou, Y . Jiang, and J. Sun, “Unified perceptual parsing for scene understanding,” in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 418–434

    2018

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.