Pith. sign in

REVIEW 5 major objections 5 minor 57 references

Inter2Former: Dynamic Hybrid Attention for Efficient High-Precision Interactive

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Inter2Former claims that dense-token interactive segmentation can be made CPU-fast by allocating computation to boundaries instead of spreading it uniformly.

desk verdict Solid architecture paper with a real efficiency idea; the routing premise needs per-click evidence before the SOTA trade-off is fully convincing. read the letter →

arxiv 2507.09612 v1 pith:R4VAF4BA submitted 2025-07-13 cs.CV

You Huang , Lichao Chen , Jiayi Ji , Liujuan Cao , Shengchuan Zhang , Rongrong Ji This is my paper
classification cs.CV
keywords interactivesegmentationdynamichybridattentionbinarysphericalquantizationCPU-efficientinferencemixtureofexpertsdenseprompttokenshigh-precisionboundary-guidedrouting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Interactive segmentation lets an annotator refine a mask by clicking, and current models face a trade-off: dense-token decoders deliver precise boundaries but run too slowly on CPUs, while sparse-token decoders are fast but less accurate. Inter2Former tries to get both by treating computation as a budget to allocate, using the previous mask to find object boundaries and running full attention only on boundary tokens while everything else uses a linear-complexity quantized attention. It also crops prompt embedding and final upsampling to the detected region and restricts mixture-of-experts routing to boundary tokens. The paper reports a five-click mean IoU of 92.28 on HQSeg44K and 91.30 on DAVIS at 2048 resolution, with CPU time of 300 ms per 20 clicks, improving on the compared state-of-the-art methods. If correct, the central claim is that high-precision interactive segmentation no longer needs GPU-class compute.

What carries the argument

The load-bearing object is Dynamic Hybrid Attention (DHA), a routing mechanism that reads the previous mask, detects boundary tokens with a local-variance edge map, and sends boundary tokens through conventional full attention while non-boundary tokens go through BSQ Attention. BSQ Attention projects keys to a low-dimensional space, binarizes them on the unit hypersphere, and reconstructs quantized keys from learnable base vectors, so attention to the key-value sequence factorizes into a linear-time operation O(NS)=O(N). The same edge map also controls the Hybrid Mixture of Experts, which sends only edge tokens to the routed experts, and the Dynamic Prompt Embedding / Dynamic Local Upsampling pair crops computation to the object region, making the decoder's cost scale with boundary complexity rather than image area.

What would settle it

Measure per-click accuracy on a benchmark of objects with fine internal structure, such as thin filaments, holes, or textured interiors, and compare Inter2Former with a variant that gives every token full attention: if the BSQ-treated non-boundary tokens silently discard detail, the gap in average clicks to reach 95% IoU should widen beyond the ablation's reported differences.

Watch

Extended reading notes

Core claim

The paper's central claim is that dense-token interactive segmentation can be made CPU-efficient without sacrificing accuracy by allocating computation dynamically. Guided by the boundary of the previous mask, tokens are split into a minority of edge tokens, which receive full quadratic attention, and a majority of non-edge tokens, which receive the proposed BSQ Attention that quantizes keys into a small binary codebook and runs in linear time. The same boundary signal drives the Hybrid Mixture of Experts, where only edge tokens use routed experts while non-edge tokens pass through a shared expert, and the Dynamic Prompt Embedding and Dynamic Local Upsampling restrict prompt encoding and mask upsampling to the detected region. On high-precision benchmarks the model reports state-of-the-art five-click mean IoU and average-click counts to reach 90% and 95% IoU while keeping CPU latency near that of sparse-token models, meaning the accuracy-efficiency trade-off that motivated the work is, the authors argue, resolved.

Load-bearing premise

The routing assumption that tokens outside the previous mask's boundary can be processed with cheap quantized attention without losing the boundary detail that later clicks need.

Editorial extensions

If this is right

  • Dense-token decoders can run on CPU-only annotation workstations: Inter2Former reports 300 ms per 20 clicks at 2048 resolution, roughly a third of the dense-token baseline's 1020 ms.
  • Boundary-guided routing is what protects accuracy: replacing DHA with full attention changes five-click mean IoU by only 0.05-0.3 points, while using BSQ Attention everywhere drops it by about 1.6 points.
  • The token-rearrangement strategy makes mixture-of-experts practical on CPUs, cutting MoE latency by 56-85% as the expert count grows to 64.
  • Under distillation from a large sparse-token teacher, the model reaches 92.68 five-click mean IoU on HQSeg44K and 92.00 on DAVIS at the same CPU latency, so high-precision fine-tuning can be combined with foundation-model distillation.
  • If these numbers hold, interactive segmentation annotation can offer near-sparse-token speed without giving up dense-token boundary quality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same boundary-driven allocation could generalize to other dense prediction tasks where most of the image is stable across time steps, such as video object segmentation or medical slice annotation.
  • Because routing depends on the previous mask, efficiency is weakest at the first clicks when the mask is coarse; a per-click latency breakdown could show the CPU advantage emerging only after the mask stabilizes.
  • BSQ Attention's 8-bit codebook allows only 256 distinct quantized key representations; whether larger codebooks help on objects with many interior edges is a testable extension the paper does not run.
  • A user study measuring real annotation throughput would test the practical value of the CPU-time numbers, since human click time and waiting tolerance determine annotation cost as much as raw latency.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces Inter2Former, an interactive segmentation model that aims to combine the accuracy of dense-prompt-token methods with the CPU efficiency of sparse-token methods such as SAM. It proposes four components: Dynamic Prompt Embedding (DPE) to restrict prompt encoding to regions of interest; Dynamic Hybrid Attention (DHA) to route boundary tokens through full attention and non-boundary tokens through a quantized BSQ attention claimed to be linear; Hybrid Mixture of Experts (HMoE) to apply adaptive FFN computation with CPU-oriented parallelization; and Dynamic Local Upsampling (DLU) for selective mask upsampling. The authors report state-of-the-art results on HQSeg44K and DAVIS with competitive CPU latency, and provide ablations supporting each component.

Significance. If the reported results hold, the paper addresses a practical bottleneck in interactive segmentation: dense-token methods are accurate but slow on CPU, which limits large-scale annotation. The proposed adaptive computation allocation is a sensible direction, and the paper includes ablations (Table 2), complexity analysis (Eqs. 8-9), and a released code repository. The main empirical claim, that Inter2Former beats prior methods in accuracy-efficiency trade-off on CPU, is supported by Table 1 in the single-run measurements shown. However, the central efficiency argument rests on the BSQ attention factorization, whose derivation in Section 3.5 is not sound as written; this must be corrected or clarified before the claim can be accepted.

major comments (5)
  1. [Section 3.5, Eqs. (8) and (14)] The claimed linear-complexity factorization in Eq. (8) does not follow from the BSQA definition in Eq. (14). In VQ attention, \hat{K} = \Delta C with \Delta a one-hot assignment matrix, so exp(Q\hat{K}^T)V = exp(QC^T)(\Delta^T V). In BSQA, Eq. (14) defines \hat{K} = [I, 1-I][C_1; C_0], which is a sum of S selected base vectors, i.e., \Delta has multiple ones per row. For such a sum, exp(q_i^T \hat{k}_j) = \prod_s exp(q_i^T c_{j,s}), which cannot be represented as a single entry of exp(QC^T) with one codebook vector. Therefore the O(NS) complexity claimed for BSQA is not established; if the codebook is instead of size 2^S, the complexity becomes O(N2^S). This is a load-bearing issue because the efficiency advantage of DHA over full attention depends on the linear-complexity claim. Please correct the derivation, clarify the exact computation used at inference, and provide a valid complexity analysis.
  2. [Section 3.4, Eq. (4) and Table 2] The routing in DHA uses the edge map derived from the previous mask M_{k-1}, but the paper does not specify the initialization for the first click, when no previous mask exists. This matters because the ablation in Table 2 shows that sending all tokens through BSQ attention ('DHA → All BSQA') degrades 5-mIoU by 1.61 points at 1024×1024 (91.32 vs. 89.71). If the first click is processed mostly with BSQ attention, the reported averages could hide a systematic weakness in early interactions. Please specify how M_0 is initialized and, ideally, report first-click or per-click accuracy to demonstrate that edge-based routing does not penalize the initial prompt.
  3. [Section 4.2 and Table 1] The CPU-latency measurements are not reproducibility-defined. The paper reports CPU times (e.g., 300/131 ms for Inter2Former-ViT-B2048) but does not state the CPU model, number of threads, batch size, numerical precision (FP32/FP16/INT8), or whether values are means/medians over repeated runs. Since the paper's headline claim is 'high efficiency on CPU devices', these conditions are essential for a meaningful comparison with the baselines in Table 1.
  4. [Section 4.2 and Table 1] The results in Table 1 appear to come from a single evaluation run. The performance margins over HRSAM++ are small (0.78 points on HQSeg44K and 0.51 points on DAVIS at 2048×2048 resolution). Without multiple seeds or a measure of variance, the state-of-the-art claim is not statistically supported. Please report the number of runs and standard deviations, or at least confirm that repeated evaluations give stable results.
  5. [Section 4.1] The implementation details omit several hyperparameters needed to reproduce the experiments: the optimizer and its learning rate schedule, batch size, the number of routed experts M in HMoE, the BSQ bit width S used in the final model, and the exact setting for the CannyNet (the paper says 'CannetNet' in one place). These values are also necessary to interpret the HMoE efficiency analysis in Figure 3(c), which varies the number of experts.
minor comments (5)
  1. [Section 4.1] The text refers to 'CannetNet' in the implementation details; this should be 'CannyNet' to match the later usage.
  2. [Section 3.5 and Related Work] The reference [30] is called both 'VQ-Transformer' and 'Transformer-VQ' in different places; please unify the name.
  3. [Figure 2 caption] The caption contains a typo, 'tokzens' should be 'tokens'.
  4. [Section 3.4, Eq. (4)] Please clarify the downsampling operation in Eq. (4): the kernel size and stride of the max pooling that matches the edge map to the token grid should be stated explicitly.
  5. [Section 4.2, evaluation metrics] The definitions of 20-SPC and Online SPC would be clearer if the text stated whether 20-SPC divides the total preprocessing plus interaction time by 20 or reports the average over 20 clicks, and whether Online SPC excludes preprocessing entirely as currently implied.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; the SOTA and efficiency claims are benchmark-verified and the routing premise is empirically ablated rather than defined into existence.

full rationale

The paper's central claims are empirical: Table 1 compares Inter2Former against external baselines on HQSeg44K and DAVIS under standard metrics, and the per-module ablations in Table 2 and Figure 5 directly test DHA against All FA and All BSQA, DPE against Non-DPE, DLU against Non-DLU, and BSQA against VQA. No quantity in the derivation is fitted to the benchmark and then reported as a prediction: BSQA approximates key-value attention through binary spherical quantization, with the training-time full-attention replacement described in Section 3.8, and DHA's edge-map routing (Eq. 4) is an architectural allocation rule whose accuracy-efficiency trade-off is measured, not assumed. The paper does cite the authors' own InterFormer and HRSAM as architectural bases for dense prompt tokens and the encoder, but these citations supply implementation lineage rather than load-bearing proof; no uniqueness theorem or self-citation is invoked to forbid alternative routing or attention choices. The routing premise about non-boundary tokens is a genuine modeling assumption with possible correctness risk, but that is not circularity because its validity is checked by external benchmarks and ablations. Overall, no derivation step reduces to its own inputs, so the circularity score is at most 2 for minor, non-load-bearing self-citation.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical or conceptual entities are postulated; all proposed components are network modules trained on standard data. The free parameters listed are hand-chosen architectural or routing thresholds that directly affect the efficiency-accuracy balance and are not fully specified in the paper.

free parameters (4)
  • BSQ codebook bit width S = 8 (16 base vectors)
    Architectural capacity choice for quantized attention; trade-off between approximation error and linear-complexity cost. Stated in Implementation Details.
  • Edge detection threshold = variance > 0 in Eq. 4
    Hand-chosen criterion to classify tokens as boundary vs non-boundary; directly controls how many tokens take expensive full attention.
  • DPE/DLU region expansion padding = not reported
    Cropping margin around clicks and objects determines how much local region is processed; affects accuracy and latency but the exact value is omitted.
  • Number of routed experts in final model = not reported in main text
    HMoE efficiency and accuracy depend on M; the efficiency analysis sweeps 8 to 64 experts but the final model's M is not stated.
assumptions (5)
  • domain assumption Boundary regions from local variance of the previous mask are a sufficient routing signal; non-boundary tokens can be processed by quantized BSQ attention without sacrificing final segmentation quality.
    Invoked in Eqs. 4-5 and Figure 1; if false, the majority of tokens get an approximation that could lose detail needed for later clicks.
  • domain assumption Background tokens outside the object bounding box can be represented by a single learned background embedding during prompt fusion.
    Eq. 3 replaces out-of-box prompt features with e_bg; assumes prompt information outside the object region is not needed.
  • domain assumption BSQ quantization error on key vectors is small enough that training with quantized keys under full attention yields accurate inference with factorized attention.
    Sections 3.5 and 3.8; relies on the theoretical bound from [54] and empirical convergence, not proven for this architecture.
  • domain assumption Removing the SSM/Cycle-scan module from HRSAM++ while retaining the anchor map preserves encoder visual extrapolation for high-resolution images.
    Implementation Details; the authors re-train HRSAM++ without SSM for fair comparison, but this is a modification of the cited encoder.
  • standard math The algebraic factorization in Eqs. 8-9 is exact for the BSQA codebook, and inverse token permutation in Eqs. 19-20 is lossless.
    These are exact identities assuming correct implementation, so they are not a limitation by themselves.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Inter2Former: Dynamic Hybrid Attention for Efficient High-Precision Interactive." pith.science (2026). https://pith.science/paper/R4VAF4BA

@misc{pith2026250709612,
  author       = {Pith},
  title        = {Pith review of: Inter2Former: Dynamic Hybrid Attention for Efficient High-Precision Interactive},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R4VAF4BA}},
  note         = {Machine review of arXiv:2507.09612}
}
read the original abstract

Interactive segmentation (IS) improves annotation efficiency by segmenting target regions from user prompts, with widespread applications in real-world scenarios. Current approaches face a critical trade-off: dense-token methods achieve superior accuracy and detail preservation but suffer from prohibitively slow processing on CPU devices, while the Segment Anything Model (SAM) advances the field with sparse prompt tokens for fast inference but compromises segmentation quality. In this paper, we propose Inter2Former to address this challenge by optimizing computation allocation in dense-token processing, which introduces four key enhancements. First, we propose Dynamic Prompt Embedding (DPE) that adaptively processes only regions of interest while avoiding additional overhead from background tokens. Second, we introduce Dynamic Hybrid Attention (DHA), which leverages previous segmentation masks to route tokens through either full attention (O(N2)) for boundary regions or our proposed efficient BSQ attention (O(N)) for non-boundary regions. Third, we develop Hybrid Mixture of Experts (HMoE), which applies similar adaptive computation strategies in FFN modules with CPU-optimized parallel processing. Finally, we present Dynamic Local Upsampling (DLU), a reverse operation of DPE, which localizes objects with a lightweight MLP and performs fine-grained upsampling only in detected regions. Experimental results on high-precision IS benchmarks demonstrate that Inter2Former achieves SOTA performance with high efficiency on CPU devices.

Figures

Figures reproduced from arXiv: 2507.09612 by the authors.

Figure 1
Figure 1. Overview of Dynamic Hybrid Attention (DHA). Dur [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of Inter2Former. Given user clicks and previous mask as interactive prompts, our Inter2Former first employ DPE to [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Efficiency analysis. (a) Latency of DPE and Non-DPE (full prompt embedding). (b) DHA latency across different area ratios. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Qualitative results. Inter2Former produces precise segmentation maps for challenging line structures under 20-click prompts. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Performance-efficiency trade-off in the ablation study. We plot segmentation quality (NoC@90) against online inference latency [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

57 extracted references · 33 canonical work pages

  1. [1]

    Ef- ficient interactive annotation of segmentation datasets with polygon-rnn++

    David Acuna, Huan Ling, Amlan Kar, and Sanja Fidler. Ef- ficient interactive annotation of segmentation datasets with polygon-rnn++. In 2018 IEEE Conference on Computer Vi- sion and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 859–868. IEEE Computer Society, 2018. 2

    work page 2018

  2. [2]

    Moinul Hos- sain, Gianluca Marcelli, Marc Alemany-Fornes, and Anas- tasios D

    Saber Mirzaee Bafti, Chee Siang Ang, Md. Moinul Hos- sain, Gianluca Marcelli, Marc Alemany-Fornes, and Anas- tasios D. Tsaousis. A crowdsourcing semi-automatic image segmentation platform for cell biology. 2021. 1

    work page 2021

  3. [3]

    Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection

    Paul Bergmann, Michael Fauser, David Sattlegger, and Carsten Steger. Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9592–9600, 2019. 1

    work page 2019

  4. [4]

    nuscenes: A multi- modal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Gi- ancarlo Baldan, and Oscar Beijbom. nuscenes: A multi- modal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621–11631, 2020. 1

    2020

  5. [5]

    Focalclick: towards practical in- teractive image segmentation

    Xi Chen, Zhiyan Zhao, Yilei Zhang, Manni Duan, Donglian Qi, and Hengshuang Zhao. Focalclick: towards practical in- teractive image segmentation. pages 1300–1309, 2022. 1, 2, 7

    work page 2022

  6. [6]

    FlashAttention-2: Faster attention with better par- allelism and work partitioning

    Tri Dao. FlashAttention-2: Faster attention with better par- allelism and work partitioning. 2023. 3

    work page 2023

  7. [7]

    Fu, Stefano Ermon, Atri Rudra, and Christopher R´e

    Tri Dao, Daniel Y . Fu, Stefano Ermon, Atri Rudra, and Christopher R´e. FlashAttention: Fast and memory-efficient exact attention with IO-awareness. In Advances in Neural Information Processing Systems, 2022. 3

    work page 2022

  8. [8]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In 9th International Conference on Learning Rep- resentations, ICLR 20...

    2021

Show all 57 references
  1. [9]

    Image com- pression with product quantized masked image modeling

    Alaaeldin El-Nouby, Matthew J Muckley, Karen Ullrich, Ivan Laptev, Jakob Verbeek, and Herv´e J´egou. Image com- pression with product quantized masked image modeling. arXiv preprint arXiv:2212.07372, 2022. 3

    2022 arXiv

  2. [10]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021. 3

    2021

  3. [11]

    Eva: Exploring the limits of masked visual representa- tion learning at scale

    Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. Eva: Exploring the limits of masked visual representa- tion learning at scale. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition,...

    2023

  4. [12]

    Mamba: Linear-time sequence modeling with selective state spaces

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023. 4, 6

    2023 arXiv

  5. [13]

    Star-transformer

    Qipeng Guo, Xipeng Qiu, Pengfei Liu, Yunfan Shao, Xi- angyang Xue, and Zheng Zhang. Star-transformer. In Pro- ceedings of the 2019 Conference of the North, 2019. 3

    2019

  6. [14]

    Girshick

    Agrim Gupta, Piotr Doll ´ar, and Ross B. Girshick. LVIS: A dataset for large vocabulary instance segmentation. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 5356–5364. Computer Vision Foundation / IEEE, 2019. 6

    2019

  7. [15]

    Flatten transformer: Vision transformer using focused linear attention

    Dongchen Han, Xuran Pan, Yizeng Han, Shiji Song, and Gao Huang. Flatten transformer: Vision transformer using focused linear attention. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5961– 5971, 2023. 3

    2023

  8. [16]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 6

    2016

  9. [17]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017. 6

    2017

  10. [18]

    Interformer: Real-time interactive image segmentation

    You Huang, Hao Yang, Ke Sun, Shengchuan Zhang, Liu- juan Cao, Guannan Jiang, and Rongrong Ji. Interformer: Real-time interactive image segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 22301–22311, 2023. 1, 2, 3, 4, 7

    2023

  11. [19]

    Hrsam: Efficiently seg- ment anything in high-resolution images

    You Huang, Wenbin Lai, Jiayi Ji, Liujuan Cao, Shengchuan Zhang, and Rongrong Ji. Hrsam: Efficiently seg- ment anything in high-resolution images. arXiv preprint arXiv:2407.02109, 2024. 2, 3, 4, 6, 7, 12

    2024 arXiv

  12. [20]

    Interactive image seg- mentation via backpropagating refinement scheme

    Won-Dong Jang and Chang-Su Kim. Interactive image seg- mentation via backpropagating refinement scheme. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 5297–5306. Computer Vision Foundation / IEEE, 2019. 2

    2019

  13. [21]

    Segment anything in high quality

    Lei Ke, Mingqiao Ye, Martin Danelljan, Yifan Liu, Yu-Wing Tai, Chi-Keung Tang, and Fisher Yu. Segment anything in high quality. In NeurIPS, 2023. 2, 6, 7, 12

    2023

  14. [22]

    Transformers in vision: A survey

    Salman Khan, Muzammal Naseer, Munawar Hayat, Syed Waqas Zamir, Fahad Shahbaz Khan, and Mubarak Shah. Transformers in vision: A survey. ACM computing surveys (CSUR), 54(10s):1–41, 2022. 3

    2022

  15. [23]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 202...

    2023

  16. [24]

    Anton Konushin Konstantin Sofiiuk, Ilia A. Petrov. Reviv- ing iterative training with mask guidance for interactive seg- mentation. arXiv: Computer Vision and Pattern Recognition,

  17. [25]

    Lisa: Reasoning segmentation via large language model

    Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Reasoning segmentation via large language model. arXiv preprint arXiv:2308.00692,

    arXiv

  18. [26]

    Exploring plain vision transformer backbones for object de- tection

    Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer backbones for object de- tection. 3

  19. [27]

    Interactive image segmentation with latent diversity

    Zhuwen Li, Qifeng Chen, and Vladlen Koltun. Interactive image segmentation with latent diversity. In2018 IEEE Con- ference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 577–

    2018

  20. [28]

    Lawrence Zitnick

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C. Lawrence Zitnick. Microsoft coco: Common objects in context. Lec- ture Notes in Computer Science, 2014. 6

    2014

  21. [29]

    Interactive image segmentation with first click attention

    Zheng Lin, Zhao Zhang, Lin-Zhuo Chen, Ming-Ming Cheng, and Shao-Ping Lu. Interactive image segmentation with first click attention. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020 , pages 13336–13345. IEEE, 2020. 2

    2020

  22. [30]

    Transformer-vq: Linear-time transformers via vector quantization

    Lucas D Lingle. Transformer-vq: Linear-time transformers via vector quantization. arXiv preprint arXiv:2309.16354 ,

    arXiv

  23. [31]

    Deepseek-v3 technical report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024. 5

    2024 arXiv

  24. [32]

    Simpleclick: Interactive image segmentation with sim- ple vision transformers

    Qin Liu, Zhenlin Xu, Gedas Bertasius, and Marc Nietham- mer. Simpleclick: Interactive image segmentation with sim- ple vision transformers. arXiv preprint arXiv:2210.11006 ,

    arXiv

  25. [33]

    Pseudoclick: Interactive image segmentation with click imi- tation

    Qin Liu, Meng Zheng, Benjamin Planche, Srikrishna Karanam, Terrence Chen, Marc Niethammer, and Ziyan Wu. Pseudoclick: Interactive image segmentation with click imi- tation. pages 728–745, 2022. 2

    2022

  26. [34]

    Rethinking interactive image segmentation with low latency, high quality, and diverse prompts

    Qin Liu, Jaemin Cho, Mohit Bansal, and Marc Nietham- mer. Rethinking interactive image segmentation with low latency, high quality, and diverse prompts. arXiv preprint arXiv:2404.00741, 2024. 2, 6, 7

    2024 arXiv

  27. [35]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 3

    2021

  28. [36]

    Segment anything in medical images

    Jun Ma and Bo Wang. Segment anything in medical images. arXiv preprint arXiv:2304.12306, 2023. 2, 7

    2023 arXiv

  29. [37]

    Deep extreme cut: From extreme points to object segmentation

    Kevis-Kokitsi Maninis, Sergi Caelles, Jordi Pont-Tuset, and Luc Van Gool. Deep extreme cut: From extreme points to object segmentation. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 616–625. IEEE...

    2018

  30. [38]

    Segment anything model for medical image analysis: an experimental study

    Maciej A Mazurowski, Haoyu Dong, Hanxue Gu, Jichen Yang, Nicholas Konz, and Yixin Zhang. Segment anything model for medical image analysis: an experimental study. Medical Image Analysis, 89:102918, 2023. 2

    2023

  31. [39]

    Finite scalar quantization: Vq-vae made simple

    Fabian Mentzer, David Minnen, Eirikur Agustsson, and Michael Tschannen. Finite scalar quantization: Vq-vae made simple. arXiv preprint arXiv:2309.15505, 2023. 3

    2023 arXiv

  32. [40]

    Gross, and Alexander Sorkine- Hornung

    Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus H. Gross, and Alexander Sorkine- Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las...

    2016

  33. [41]

    Rabe and Charles Staats

    Markus N. Rabe and Charles Staats. Self-attention does not need o(n2) memory. CoRR, abs/2112.05682, 2021. 3

    2021 arXiv

  34. [42]

    Petrov, Olga Barinova, and Anton Konushin

    Konstantin Sofiiuk, Ilia A. Petrov, Olga Barinova, and Anton Konushin. F-BRS: rethinking backpropagating refinement for interactive segmentation. In 2020 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020 , pages 8620–

    2020

  35. [43]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063,

  36. [44]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information pro- cessing systems, 30, 2017. 2

    2017

  37. [45]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neu- ral Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017...

    2017

  38. [46]

    Order-aware interactive segmen- tation

    Bin Wang, Anwesa Choudhuri, Meng Zheng, Zhongpai Gao, Benjamin Planche, Andong Deng, Qin Liu, Terrence Chen, Ulas Bagci, and Ziyan Wu. Order-aware interactive segmen- tation. arXiv preprint arXiv:2410.12214, 2024. 2

    2024 arXiv

  39. [47]

    Internimage: Exploring large-scale vi- sion foundation models with deformable convolutions

    Wenhai Wang, Jifeng Dai, Zhe Chen, Zhenhang Huang, Zhiqi Li, Xizhou Zhu, Xiaowei Hu, Tong Lu, Lewei Lu, Hongsheng Li, et al. Internimage: Exploring large-scale vi- sion foundation models with deformable convolutions. In Proceedings of the IEEE/CVF Conference on Computer Vi- si...

    2023

  40. [48]

    Efficientsam: Leveraged masked image pretraining for efficient segment anything

    Yunyang Xiong, Bala Varadarajan, Lemeng Wu, Xiaoyu Xi- ang, Fanyi Xiao, Chenchen Zhu, Xiaoliang Dai, Dilin Wang, Fei Sun, Forrest Iandola, et al. Efficientsam: Leveraged masked image pretraining for efficient segment anything. arXiv preprint arXiv:2312.00863, 2023. 7

    2023 arXiv

  41. [49]

    Structured click control in transformer-based interactive segmentation

    Long Xu, Yongquan Chen, Rui Huang, Feng Wu, and Shiwu Lai. Structured click control in transformer-based interactive segmentation. arXiv preprint arXiv:2405.04009, 2024. 2

    2024 arXiv

  42. [50]

    Price, Scott Cohen, Jimei Yang, and Thomas S

    Ning Xu, Brian L. Price, Scott Cohen, Jimei Yang, and Thomas S. Huang. Deep interactive object selection. In2016 IEEE Conference on Computer Vision and Pattern Recogni- tion, CVPR 2016, Las Vegas, NV , USA, June 27-30, 2016 , pages 373–381. IEEE Computer Society, 2016. 2

    2016

  43. [51]

    Vector-quantized image modeling with improved vqgan

    Jiahui Yu, Xin Li, Jing Yu Koh, Han Zhang, Ruoming Pang, James Qin, Alexander Ku, Yuanzhong Xu, Jason Baldridge, and Yonghui Wu. Vector-quantized image modeling with improved vqgan. arXiv preprint arXiv:2110.04627, 2021. 3

    2021 arXiv

  44. [52]

    Faster segment anything: Towards lightweight sam for mo- bile applications

    Chaoning Zhang, Dongshen Han, Yu Qiao, Jung Uk Kim, Sung-Ho Bae, Seungkyu Lee, and Choong Seon Hong. Faster segment anything: Towards lightweight sam for mo- bile applications. arXiv preprint arXiv:2306.14289 , 2023. 7

    2023 arXiv

  45. [53]

    Leveraging ai predicted and expert re- vised annotations in interactive segmentation: Continual tun- ing or full training? arXiv preprint arXiv:2402.19423, 2024

    Tiezheng Zhang, Xiaoxi Chen, Chongyu Qu, Alan Yuille, and Zongwei Zhou. Leveraging ai predicted and expert re- vised annotations in interactive segmentation: Continual tun- ing or full training? arXiv preprint arXiv:2402.19423, 2024. 2

    2024 arXiv

  46. [54]

    Image and video tokenization with binary spherical quantization

    Yue Zhao, Yuanjun Xiong, and Philipp Kr ¨ahenb¨uhl. Image and video tokenization with binary spherical quantization. arXiv preprint arXiv:2406.07548, 2024. 2, 3, 5

    2024 arXiv

  47. [55]

    Online clustered code- book

    Chuanxia Zheng and Andrea Vedaldi. Online clustered code- book. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 22798–22807, 2023. 3 A. Ablation Study To validate the effectiveness of each key component in Inter2Former, we conduct extensive ...

    2023

  48. [585]

    IEEE Computer Society, 2018. 2

    2018

  49. [2021]

    OpenReview.net, 2021. 3

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.