Pith. sign in

REVIEW 2 major objections 7 minor 60 references

SAM-MT: Real-Time Interactive Multi-Target Video Segmentation

T0 review · 2 major / 7 minor · reviewed 2026-07-10 · glm-5.2

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read One query per target: 36 FPS video segmentation for 10+ objects

desk verdict SAM-MT achieves near-constant FPS across target counts by replacing per-object dense memory with shared global context plus per-target query tokens, but the synthetic benchmark where FPS is measured reports no accuracy metrics. read the letter →

arxiv 2607.08688 v1 pith:XZNRKGIK submitted 2026-07-09 cs.CV

Ruiqi Shen , Chang Liu , Henghui Ding This is my paper
classification cs.CV
keywords sam-mtsegmentationtargetsvideoindividualmulti-targetreal-timesingle-target
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The author argues that multi-target video object segmentation need not trade speed for target count. The prevailing approach — running the full single-target pipeline once per object — makes latency grow linearly with the number of targets. SAM-MT breaks this coupling by giving each target a single lightweight query token (instead of thousands of dense memory tokens), sharing one dense pixel-level memory across all targets, and using decoupled masked attention to prevent identity confusion between targets while still letting each target see the global scene context. An identity transformer retrieves each target's historical queries from sparse memory to maintain temporal consistency across frames. The result is a system that runs at near-constant 36+ FPS regardless of whether it tracks 1 or 20 targets, while matching or exceeding the segmentation accuracy of the single-target baseline it is built upon. The paper evaluates this on six standard VOS benchmarks and a synthetic multi-target benchmark, showing both the speed advantage and competitive accuracy.

What carries the argument

Three components carry the argument: (1) scalable target queries — one lightweight token per target, replacing the per-target dense memory of 4096 pixel tokens used in SAM2; (2) decoupled masked attention, which blocks attention between queries of different targets while allowing all targets to attend to shared global queries and image features; (3) a query-based sparse FIFO memory storing one token per target per frame (vs. HW dense tokens per target per frame), enabling a 16-frame temporal window at negligible cost. An identity transformer with per-target masking retrieves each target's own historical queries, preventing cross-target memory pollution.

What would settle it

If, in scenes with many simultaneously occluding targets or very long sequences, the shared dense memory produces lower-quality individual masks than per-target dense memory would, the core efficiency claim would hold but the accuracy parity claim would fail. The paper's synthetic benchmark has only one sequence per target count and 100 frames each, which may not stress this sufficiently.

Watch

Extended reading notes

Core claim

The multi-target bottleneck in modern video segmentation stems from replicating dense pixel-level memory — thousands of tokens per target per frame — for each object. By replacing this with a single shared dense memory for all targets combined, plus one lightweight query token per target stored in a sparse FIFO memory, the marginal cost of adding a target drops to a single cross-attention on a k-dimensional vector. Combined with decoupled masked attention that blocks cross-target interference while preserving shared access to global context, this allows the system to track and segment 10+ targets at 36+ FPS — roughly 6× faster than the baseline SAM2 model — with no degradation in accuracy on

Load-bearing premise

The paper assumes that a single shared dense memory representing the combined mask of all targets, plus per-target query tokens, is sufficient to maintain the same segmentation quality as having separate dense memory per target. If the shared representation cannot capture the pixel-level detail each individual target needs, accuracy would degrade in dense or complex scenes — and the 20-sequence synthetic benchmark may not be large enough to expose this.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. The paper presents SAM-MT, a framework extending SAM2 for real-time multi-target video object segmentation. The core architectural contribution is the replacement of per-target dense memory propagation (which scales linearly with target count) with a shared dense memory for global context plus per-target lightweight query tokens stored in a sparse FIFO memory. Decoupled masked attention prevents cross-target interference while allowing shared access to global queries. An identity transformer retrieves per-target historical queries for temporal consistency. The central claim—near-constant FPS regardless of target count—is well-supported by Tables 3–5, showing 36+ FPS at 10+ targets versus SAM2's sharp degradation. Accuracy is competitive across six VOS benchmarks (Tables 1–2). The efficiency contribution is clear and the architecture is technically sound.

Significance. The multi-target latency bottleneck in SAM-family models is a real practical limitation, and the paper addresses it at the framework level rather than through engineering tricks. The decoupled masked attention (§3.4, Eqs. 2–4) and identity-aware sparse memory (§3.6–3.7, Eqs. 6–8) are principled contributions. The FPS and VRAM scaling results (Tables 3–4) are the strongest part of the paper and are independently verifiable. Code is promised. The work would be of interest to practitioners needing real-time multi-object tracking.

major comments (2)
  1. §5.2, Tables 3–4: The synthetic multi-target benchmark (20 sequences, 1–20 targets) reports only FPS and VRAM, with no accuracy metrics (J&F). This is the only evaluation with genuinely dense target counts, yet the claim that segmentation quality is 'maintained' at scale rests entirely on benchmarks dominated by single-target sequences (79% single-target in MOSEv2-val per §5.2). The architectural change from per-target dense memory (4096 tokens/target in SAM2) to a single shared dense memory plus one C-dimensional query token per target (§3.6, Eq. 6) is exactly the kind of change whose accuracy impact should be measured at high target counts. Adding J&F (or at least J) to the synthetic benchmark would close this gap. Without it, the claim of maintained quality at 10–20 targets is supported only by qualitative figures (Figs. 6–7).
  2. §3.6, Eq. (6): The sparse memory stores one query token per target per frame (k×C for k targets). The identity transformer (§3.7, Eq. 7) cross-attends the previous frame's queries to this memory with an identity-aware mask. However, the paper does not discuss how re-identification works when a target is fully occluded for longer than the memory window T (set to 16 in §5.4, Table 8). SAM2 uses dense pixel-level memory that can match on appearance; a single C-dimensional token carries far less information for re-identification after extended disappearance. The LVOS results (Table 1) are encouraging, but the ablation in Table 8 shows only a 0.7-point gain from window size 8→32, suggesting diminishing returns. Can the authors provide per-target re-identification success rates on LVOS sequences with long disappearances, or discuss this limitation more concretely?
minor comments (7)
  1. §5.1: The term 'zero-shot' is used to describe evaluation on VOS benchmarks, but the model is trained on a filtered SA-V subset (§4.1). This is standard transfer evaluation, not zero-shot. The wording could be clarified.
  2. Table 1: Several baselines (R50-AOT, SwinB-AOT, R50-DeAOT) have missing entries (–) for LVOS benchmarks. The footnote says 'out-of-memory' but it would be more informative to note whether this is a fundamental limitation or a configuration issue.
  3. Figure 2: The diagram is dense and the distinction between 'from SAM2' and 'new in SAM-MT' components could be more visually prominent (e.g., color coding is mentioned but hard to distinguish in print).
  4. §4.1: The training data is filtered to sequences with ≥3 concurrent targets (≈35% of SA-V). The rationale is reasonable, but the threshold of 3 is arbitrary. Was this threshold validated, or could a lower threshold include more training data?
  5. Table 5: The 'All' column includes single-target sequences where multi-target efficiency is irrelevant. The ≥2, ≥3, ≥5 subsets are more informative, but the 'All' column could mislead readers into thinking the FPS advantage applies broadly when it matters most for dense scenes.
  6. §3.5, Eq. (5): The weighting head f_weight is described as 'MLP-based' but its architecture (number of layers, hidden dimensions) is not specified.
  7. References [2] and [27] are listed as 2025/2026 arXiv preprints or forthcoming; ensure final publication details are updated where applicable.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity detected: the central efficiency and accuracy claims are measured against external benchmarks and baselines, with no self-citation chain or definitional reduction.

full rationale

The paper's central claim is that SAM-MT decouples latency from target count while maintaining segmentation quality. This claim is evaluated against external benchmarks (MOSEv2, MOSEv1, LVOSv2, LVOSv1, SA-V val, SA-V test) and compared against external baselines (SAM2.1-B+, Cutie, DeAOT, XMem, etc.). The FPS measurements in Tables 3 and 5 are independently reproducible runtime measurements, not derived from any fitted parameter or self-citation. The accuracy metrics (J&F, J&˙F) in Tables 1–2 are computed on standard public datasets using standard metrics. The architecture builds on SAM2 (an external, independently published model) and introduces new components (decoupled masked attention in Eq. 2–4, query-based sparse memory in Eq. 6, identity transformer in Eq. 7–8) that are defined operationally, not circularly. No equation reduces to its own input by construction. The training data is a filtered subset of SA-V (an external dataset). The ablation studies (Tables 6–9) test each component against alternatives, showing independent contributions. While the skeptic correctly notes that multi-target accuracy is not reported on the synthetic benchmark (only FPS and VRAM are), this is a gap in evaluation coverage, not a circularity in the derivation chain. The paper does not claim a 'prediction' or 'first-principles result' that reduces to its inputs by definition. There are no self-citations that are load-bearing for the central claim. The derivation is self-contained against external benchmarks.

Assumptions & free parameters 5 free parameters · 4 assumptions · 3 invented entities

The free parameters are mostly empirically selected via ablation, which is standard practice but means the system is tuned rather than derived. The key ad-hoc axiom—that shared dense memory suffices for all targets—is the load-bearing assumption that could fail in extreme scenarios. The invented entities (target query, identity transformer, decoupled masked attention) all have falsifiable handles via ablation studies.

free parameters (5)
  • Sparse memory window size T = 16
    Chosen by ablation (Table 8) as a balance between accuracy and FPS. Not fitted to a theoretical prediction but selected empirically.
  • Identity transformer depth = 3
    Selected by ablation (Table 7) to balance accuracy and FPS. Could be 1 or 5 blocks.
  • Strided sampling span = 32 frames (8 frames sampled)
    Chosen to fit GPU memory constraints while capturing long-term dynamics. Stated in §4.2.
  • Point sampling range = 1-5 positive, 0-2 negative per target
    Chosen to simulate realistic user interactions. Stated in §4.2.
  • Loss weights for L_total = L_I + L_G = Omitted
    The paper states 'Loss weights are omitted for brevity' (§4.2). These are free parameters that affect training but are not reported.
assumptions (4)
  • domain assumption SAM2's dense pixel-level memory and mask decoder architecture provide a strong baseline for video object segmentation.
    The entire framework is built on top of SAM2's pretrained checkpoint and architecture (§3.1, §4.1). This is a standard domain assumption supported by SAM2's published results.
  • ad hoc to paper A single shared dense memory for the combined mask of all targets captures sufficient global context for individual mask prediction.
    This is the core design choice in §3.6 that enables the efficiency gain. The paper does not prove this is sufficient; it is assumed and then empirically validated on benchmarks.
  • ad hoc to paper Per-target query vectors stored in sparse memory contain enough information for robust re-identification across occlusions.
    The identity transformer (§3.7) relies on this. The paper provides empirical evidence via ablation (Table 7) but no theoretical guarantee.
  • domain assumption Strided frame sampling during training generalizes to adjacent-frame inference at test time.
    The model is trained with 4-frame strided sampling but tested on consecutive frames (§4.2). This is a standard training-inference gap assumption in video segmentation.
invented entities (3)
  • Target query independent evidence
    purpose: A single compact vector per target that represents the target's identity and appearance for propagation across frames.
    The target query is a learned representation whose effectiveness is measured by segmentation accuracy (J&F) and identity consistency in the ablation studies. It is falsifiable: if it fails to preserve identity, accuracy drops as shown in Table 7.
  • Identity transformer independent evidence
    purpose: A cross-attention module that updates target queries using their own historical queries from sparse memory, with an identity-aware mask preventing cross-target attention.
    Its contribution is measured via ablation (Table 7). Without the identity-aware mask, J&F drops by 3.9 points, providing falsifiable evidence of its necessity.
  • Decoupled masked attention independent evidence
    purpose: A self-attention masking strategy in the decoder that blocks cross-target query interactions while allowing all queries to access global context.
    Validated by ablation in Table 6, where full visibility causes a 5.5-point drop and full masking causes a 3.7-point drop.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SAM-MT: Real-Time Interactive Multi-Target Video Segmentation." pith.science (2026). https://pith.science/paper/XZNRKGIK

@misc{pith2026260708688,
  author       = {Pith},
  title        = {Pith review of: SAM-MT: Real-Time Interactive Multi-Target Video Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XZNRKGIK}},
  note         = {Machine review of arXiv:2607.08688}
}
read the original abstract

Modern Video Object Segmentation (VOS) involves tracking and segmenting user-specified targets. While recent approaches have achieved remarkable performance in single-target scenarios, extending them to multi-target settings typically involves replicating the single-target processing for each individual object, resulting in reduced frame rates (FPS) with unbounded latency as target count increases. Built upon Segment Anything 2 (SAM2), we propose SAM-MT, which addresses this by transforming the model into an interactive framework for real-time Multi-Target video segmentation. SAM-MT uses explicit queries to represent different individual targets, in parallel with a shared representation for global context. It employs decoupled masked attention to keep individual identities distinct from cross-target interference, and sparse memory for stable temporal evolution, along with specialized strategies for occlusion handling and overlap prevention. SAM-MT successfully decouples latency from the number of targets, achieving real-time speed on par with single-target baselines (>36 FPS for 10 targets) while maintaining SAM2's robust video segmentation performance.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

60 extracted references · 60 canonical work pages

  1. [1]

    MOSE: A new dataset for video object segmentation in complex scenes

    Henghui Ding, Chang Liu, Shuting He, Xudong Jiang, Philip HS Torr, and Song Bai. MOSE: A new dataset for video object segmentation in complex scenes. InICCV, pages 20224–20234, 2023

    work page 2023

  2. [2]

    MOSEv2: A more challenging dataset for video object segmentation in complex scenes.arXiv preprint arXiv:2508.05630, 2025

    Henghui Ding, Kaining Ying, Chang Liu, Shuting He, Xudong Jiang, Yu-Gang Jiang, Philip HS Torr, and Song Bai. MOSEv2: A more challenging dataset for video object segmentation in complex scenes.arXiv preprint arXiv:2508.05630, 2025

    work page arXiv 2025

  3. [3]

    Genie: A generalizable navigation system for in-the-wild environments.IEEE Robotics and Automation Letters, 2025

    Jiaming Wang, Diwen Liu, Jizhuo Chen, Jiaxuan Da, Nuowen Qian, Minh Man Tram, and Harold Soh. Genie: A generalizable navigation system for in-the-wild environments.IEEE Robotics and Automation Letters, 2025

    work page 2025

  4. [4]

    Video object segmentation-based visual servo control and object depth estimation on a mobile robot

    Brent Griffin, Victoria Florence, and Jason Corso. Video object segmentation-based visual servo control and object depth estimation on a mobile robot. InW ACV, pages 1647–1657, 2020

    work page 2020

  5. [5]

    Video object segmentation using space-time memory networks

    Seoung Wug Oh, Joon-Young Lee, Ning Xu, and Seon Joo Kim. Video object segmentation using space-time memory networks. InICCV, pages 9226–9235, 2019

    work page 2019

  6. [6]

    Rethinking space-time networks with improved memory coverage for efficient video object segmentation

    Ho Kei Cheng, Yu-Wing Tai, and Chi-Keung Tang. Rethinking space-time networks with improved memory coverage for efficient video object segmentation. InNeurIPS, volume 34, pages 11781–11794, 2021

    work page 2021

  7. [7]

    Xmem: Long-term video object segmentation with an atkinson-shiffrin memory model

    Ho Kei Cheng and Alexander G Schwing. Xmem: Long-term video object segmentation with an atkinson-shiffrin memory model. InECCV, pages 640–658. Springer, 2022

    work page 2022

  8. [8]

    Putting the object back into video object segmentation

    Ho Kei Cheng, Seoung Wug Oh, Brian Price, Joon-Young Lee, and Alexander Schwing. Putting the object back into video object segmentation. InCVPR, pages 3151–3161, 2024

    work page 2024

Show all 60 references
  1. [9]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. InICLR, pages 28085–28128, 2025

    2025

  2. [10]

    Sam2long: Enhancing sam 2 for long video segmentation with a training-free memory tree

    Shuangrui Ding, Rui Qian, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Yuwei Guo, Dahua Lin, and Jiaqi Wang. Sam2long: Enhancing sam 2 for long video segmentation with a training-free memory tree. InICCV, pages 13614–13624, 2025

    2025

  3. [11]

    MeViS: A large-scale benchmark for video segmentation with motion expressions

    Henghui Ding, Chang Liu, Shuting He, Xudong Jiang, and Chen Change Loy. MeViS: A large-scale benchmark for video segmentation with motion expressions. InICCV, pages 10156–10166, 2023

    2023

  4. [12]

    MeViS: A multi-modal dataset for referring motion expression video segmentation.IEEE TPAMI, 47(12):11400–11416, 2025

    Henghui Ding, Chang Liu, Shuting He, Kaining Ying, Xudong Jiang, Chen Change Loy, and Yu-Gang Jiang. MeViS: A multi-modal dataset for referring motion expression video segmentation.IEEE TPAMI, 47(12):11400–11416, 2025

    2025

  5. [13]

    Samurai: Adapting segment anything model for zero-shot visual tracking with motion-aware memory.arXiv preprint arXiv:2411.11922, 2024

    Cheng-Yen Yang, Hsiang-Wei Huang, Wenhao Chai, Zhongyu Jiang, and Jenq-Neng Hwang. Samurai: Adapting segment anything model for zero-shot visual tracking with motion-aware memory.arXiv preprint arXiv:2411.11922, 2024

    2024 arXiv

  6. [14]

    A distractor-aware memory for visual object tracking with sam2

    Jovana Videnovic, Alan Lukezic, and Matej Kristan. A distractor-aware memory for visual object tracking with sam2. InCVPR, pages 24255–24264, 2025

    2025

  7. [15]

    Modular interactive video object segmentation: Interaction- to-mask, propagation and difference-aware fusion

    Ho Kei Cheng, Yu-Wing Tai, and Chi-Keung Tang. Modular interactive video object segmentation: Interaction- to-mask, propagation and difference-aware fusion. InCVPR, pages 5559–5568, 2021

    2021

  8. [16]

    Learning position and target consistency for memory-based video object segmentation

    Li Hu, Peng Zhang, Bang Zhang, Pan Pan, Yinghui Xu, and Rong Jin. Learning position and target consistency for memory-based video object segmentation. InCVPR, pages 4144–4154, 2021

    2021

  9. [17]

    Efficient regional memory network for video object segmentation

    Haozhe Xie, Hongxun Yao, Shangchen Zhou, Shengping Zhang, and Wenxiu Sun. Efficient regional memory network for video object segmentation. InCVPR, pages 1286–1295, 2021

    2021

  10. [18]

    Swiftnet: Real-timevideoobjectsegmentation

    HaochenWang, XiaolongJiang, HaibingRen, YaoHu, andSongBai. Swiftnet: Real-timevideoobjectsegmentation. InCVPR, pages 1296–1305, 2021

    2021

  11. [19]

    Xmem++: Production-level video segmentation from few annotated frames

    Maksym Bekuzarov, Ariana Bermudez, Joon-Young Lee, and Hao Li. Xmem++: Production-level video segmentation from few annotated frames. InICCV, pages 635–644, 2023

    2023

  12. [20]

    Tracking anything with decoupled video segmentation

    Ho Kei Cheng, Seoung Wug Oh, Brian Price, Alexander Schwing, and Joon-Young Lee. Tracking anything with decoupled video segmentation. InICCV, pages 1316–1326, 2023. 12

    2023

  13. [21]

    Lvos: A benchmark for large-scale long-term video object segmentation.IEEE TPAMI, 2025

    Lingyi Hong, Zhongying Liu, Wenchao Chen, Chenzhi Tan, Yuang Feng, Xinyu Zhou, Pinxue Guo, Jinglun Li, Zhaoyu Chen, Shuyong Gao, et al. Lvos: A benchmark for large-scale long-term video object segmentation.IEEE TPAMI, 2025

    2025

  14. [22]

    Hierarchical memory matching network for video object segmentation

    Hongje Seong, Seoung Wug Oh, Joon-Young Lee, Seongwon Lee, Suhyeon Lee, and Euntai Kim. Hierarchical memory matching network for video object segmentation. InICCV, pages 12889–12898, 2021

    2021

  15. [23]

    Per-clip video object segmentation

    Kwanyong Park, Sanghyun Woo, Seoung Wug Oh, In So Kweon, and Joon-Young Lee. Per-clip video object segmentation. InCVPR, pages 1352–1361, 2022

    2022

  16. [24]

    Transformer-based visual segmentation: A survey.IEEE TPAMI, 2024

    Xiangtai Li, Henghui Ding, Wenwei Zhang, Haobo Yuan, Guangliang Cheng, Pang Jiangmiao, Kai Chen, Ziwei Liu, and Chen Change Loy. Transformer-based visual segmentation: A survey.IEEE TPAMI, 2024

    2024

  17. [25]

    Towards open vocabulary learning: A survey.IEEE TPAMI, 2024

    Jianzong Wu, Xiangtai Li, Shilin Xu, Haobo Yuan, Henghui Ding, Yibo Yang, Xia Li, Jiangning Zhang, Yunhai Tong, Xudong Jiang, Bernard Ghanem, and Dacheng Tao. Towards open vocabulary learning: A survey.IEEE TPAMI, 2024

    2024

  18. [26]

    VLT: Vision-language transformer and query generation for referring segmentation.IEEE TPAMI, 45(6):7900–7916, 2023

    Henghui Ding, Chang Liu, Suchen Wang, and Xudong Jiang. VLT: Vision-language transformer and query generation for referring segmentation.IEEE TPAMI, 45(6):7900–7916, 2023

    2023

  19. [27]

    Multimodal referring segmentation: A survey.IJCV, 2026

    Henghui Ding, Song Tang, Shuting He, Chang Liu, Zuxuan Wu, and Yu-Gang Jiang. Multimodal referring segmentation: A survey.IJCV, 2026

    2026

  20. [28]

    Per-pixel classification is not all you need for semantic segmentation.NeurIPS, 34:17864–17875, 2021

    Bowen Cheng, Alex Schwing, and Alexander Kirillov. Per-pixel classification is not all you need for semantic segmentation.NeurIPS, 34:17864–17875, 2021

    2021

  21. [29]

    Scaling open-vocabulary image segmentation with image-level labels

    Golnaz Ghiasi, Xiuye Gu, Yin Cui, and Tsung-Yi Lin. Scaling open-vocabulary image segmentation with image-level labels. InECCV, pages 540–557. Springer, 2022

    2022

  22. [30]

    Masked-attention mask transformer for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. InCVPR, pages 1290–1299, 2022

    2022

  23. [31]

    Fastinst: A simple query-based model for real-time instance segmentation

    Junjie He, Pengyu Li, Yifeng Geng, and Xuansong Xie. Fastinst: A simple query-based model for real-time instance segmentation. InCVPR, pages 23663–23672, 2023

    2023

  24. [32]

    A survey on 3d gaussian splatting in segmentation, editing and generation.IEEE TPAMI, 2026

    Shuting He, Peilin Ji, Yitong Yang, Changshuo Wang, Jiayi Ji, Yinglin Wang, and Henghui Ding. A survey on 3d gaussian splatting in segmentation, editing and generation.IEEE TPAMI, 2026

    2026

  25. [33]

    Mask dino: Towards a unified transformer-based framework for object detection and segmentation

    Feng Li, Hao Zhang, Huaizhe Xu, Shilong Liu, Lei Zhang, Lionel M Ni, and Heung-Yeung Shum. Mask dino: Towards a unified transformer-based framework for object detection and segmentation. InCVPR, pages 3041–3050, 2023

    2023

  26. [34]

    Oneformer: One transformer to rule universal image segmentation

    Jitesh Jain, Jiachen Li, Mang Tik Chiu, Ali Hassani, Nikita Orlov, and Humphrey Shi. Oneformer: One transformer to rule universal image segmentation. InCVPR, pages 2989–2998, 2023

    2023

  27. [35]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. InICCV, pages 4015–4026, 2023

    2023

  28. [36]

    Segment anything in high quality.NeurIPS, 36:29914–29934, 2023

    Lei Ke, Mingqiao Ye, Martin Danelljan, Yu-Wing Tai, Chi-Keung Tang, Fisher Yu, et al. Segment anything in high quality.NeurIPS, 36:29914–29934, 2023

    2023

  29. [37]

    Entitysam: Segment everything in video

    Mingqiao Ye, Seoung Wug Oh, Lei Ke, and Joon-Young Lee. Entitysam: Segment everything in video. InCVPR, pages 24234–24243, 2025

    2025

  30. [38]

    GRES: Generalized referring expression segmentation

    Chang Liu, Henghui Ding, and Xudong Jiang. GRES: Generalized referring expression segmentation. InCVPR, pages 23592–23601, 2023

    2023

  31. [39]

    Vision-language transformer and query generation for referring segmentation

    Henghui Ding, Chang Liu, Suchen Wang, and Xudong Jiang. Vision-language transformer and query generation for referring segmentation. InICCV, pages 16321–16330, 2021

    2021

  32. [40]

    Continual learning for image segmentation with dynamic query.IEEE Transactions on Circuits and Systems for Video Technology, 34 (6):4874–4886, 2023

    Weijia Wu, Yuzhong Zhao, Zhuang Li, Lianlei Shan, Hong Zhou, and Mike Zheng Shou. Continual learning for image segmentation with dynamic query.IEEE Transactions on Circuits and Systems for Video Technology, 34 (6):4874–4886, 2023

    2023

  33. [41]

    Rethinking query-based transformer for continual image segmentation

    Yuchen Zhu, Cheng Shi, Dingyou Wang, Jiajin Tang, Zhengxuan Wei, Yu Wu, Guanbin Li, and Sibei Yang. Rethinking query-based transformer for continual image segmentation. InCVPR, pages 4595–4606, 2025

    2025

  34. [42]

    Primitivenet: decomposing the global constraints for referring segmentation.Visual Intelligence, 2(1):16, 2024

    Chang Liu, Xudong Jiang, and Henghui Ding. Primitivenet: decomposing the global constraints for referring segmentation.Visual Intelligence, 2(1):16, 2024. 13

    2024

  35. [43]

    GREx: Generalized referring expression segmentation, comprehension, and generation.IJCV, 2026

    Henghui Ding, Chang Liu, Shuting He, Xudong Jiang, and Yu-Gang Jiang. GREx: Generalized referring expression segmentation, comprehension, and generation.IJCV, 2026

    2026

  36. [44]

    Fully convolutional networks for semantic segmentation

    Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In CVPR, pages 3431–3440, 2015

    2015

  37. [45]

    Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs.IEEE TPAMI, 40 (4):834–848, 2017

    Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs.IEEE TPAMI, 40 (4):834–848, 2017

    2017

  38. [46]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. InICCV, pages 2961–2969, 2017

    2017

  39. [47]

    Boundary-preserving mask r-cnn

    Tianheng Cheng, Xinggang Wang, Lichao Huang, and Wenyu Liu. Boundary-preserving mask r-cnn. InECCV, pages 660–676. Springer, 2020

    2020

  40. [48]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. InNeurIPS, volume 30, 2017

    2017

  41. [49]

    Mask2former for video instance segmentation.arXiv preprint arXiv:2112.10764, 2021

    Bowen Cheng, Anwesa Choudhuri, Ishan Misra, Alexander Kirillov, Rohit Girdhar, and Alexander G Schwing. Mask2former for video instance segmentation.arXiv preprint arXiv:2112.10764, 2021

    2021 arXiv

  42. [50]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770–778, 2016

    2016

  43. [51]

    Lvos: A benchmark for long-term video object segmentation

    Lingyi Hong, Wenchao Chen, Zhongying Liu, Wei Zhang, Pinxue Guo, Zhaoyu Chen, and Wenqiang Zhang. Lvos: A benchmark for long-term video object segmentation. InICCV, pages 13480–13492, 2023

    2023

  44. [52]

    Associating objects with transformers for video object segmentation

    Zongxin Yang, Yunchao Wei, and Yi Yang. Associating objects with transformers for video object segmentation. InNeurIPS, volume 34, pages 2491–2502, 2021

    2021

  45. [53]

    Decoupling features in hierarchical propagation for video object segmentation

    Zongxin Yang and Yi Yang. Decoupling features in hierarchical propagation for video object segmentation. In NeurIPS, volume 35, pages 36324–36336, 2022

    2022

  46. [54]

    Recurrent dynamic embedding for video object segmentation

    Mingxing Li, Li Hu, Zhiwei Xiong, Bang Zhang, Pan Pan, and Dong Liu. Recurrent dynamic embedding for video object segmentation. InCVPR, pages 1332–1341, 2022

    2022

  47. [55]

    Lisa: Reasoning segmentation via large language model

    Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Reasoning segmentation via large language model. InCVPR, pages 9579–9589, 2024

    2024

  48. [56]

    Adamcot: Rethinking cross-lingual factual reasoning through adaptive multilingual chain-of-thought

    Zheng Weihua, Xin Huang, Zhengyuan Liu, Tarun Kumar Vangani, Bowei Zou, Xiyan Tao, Yuhao Wu, AiTi Aw, Nancy F Chen, and Roy Ka-Wei Lee. Adamcot: Rethinking cross-lingual factual reasoning through adaptive multilingual chain-of-thought. InAAAI, pages 33863–33871, 2026

    2026

  49. [57]

    Ccl-xcot: An efficient cross-lingual knowledge transfer method for mitigating hallucination generation

    Zheng Weihua, Roy Ka-Wei Lee, Zhengyuan Liu, Wu Kui, Aiti Aw, and Bowei Zou. Ccl-xcot: An efficient cross-lingual knowledge transfer method for mitigating hallucination generation. InFindings of the Association for Computational Linguistics: EMNLP 2025, pages 1768–1788, 2025

    2025

  50. [58]

    One token to seg them all: Language instructed reasoning segmentation in videos

    Zechen Bai, Tong He, Haiyang Mei, Pichao Wang, Ziteng Gao, Joya Chen, Zheng Zhang, and Mike Zheng Shou. One token to seg them all: Language instructed reasoning segmentation in videos. InNeurIPS, volume 37, pages 6833–6859, 2024

    2024

  51. [59]

    Sa2va: Marrying sam2 with llava for dense grounded understanding of images and videos

    Haobo Yuan, Xiangtai Li, Tao Zhang, Yueyi Sun, Zilong Huang, Shilin Xu, Shunping Ji, Yunhai Tong, Lu Qi, Jiashi Feng, et al. Sa2va: Marrying sam2 with llava for dense grounded understanding of images and videos. arXiv preprint arXiv:2501.04001, 2025

    2025 arXiv

  52. [60]

    Mma-asia: A multilingual and multimodal alignment framework for culturally-grounded evaluation.arXiv preprint arXiv:2510.08608, 2025

    Weihua Zheng, Zhengyuan Liu, Tanmoy Chakraborty, Weiwen Xu, Xiaoxue Gao, Bryan Chen Zhengyu Tan, Bowei Zou, Chang Liu, Yujia Hu, Xing Xie, et al. Mma-asia: A multilingual and multimodal alignment framework for culturally-grounded evaluation.arXiv preprint arXiv:2510.08608, 2025. 14

    2025

Pith tools

Reviewed July 10, 2026 · model on record in the stance chip above.