Pith. sign in

REVIEW 4 major objections 4 minor 59 references

Latest Object Memory Management for Temporally Consistent Video Instance Segmentation

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that tracking each object's latest foreground-weighted state, and handling new objects separately from existing ones, gives state-of-the-art video instance segmentation, including 54.0 AP on YouTube-VIS 2022.

desk verdict Useful memory-update and decoupled-association ideas, but the 54.0 SOTA headline rides on an external offline refiner and needs reining in. read the letter →

arxiv 2507.19754 v1 pith:VB6PWIAX submitted 2025-07-26 cs.CV

Seunghun Lee , Jiwan Seo , Minwoo Choi , Kiljoon Han , Jaehoon Jeong , Zane Durante , Ehsan Adeli , Sang Hyun Park
show 1 more author
Sunghoon Im
This is my paper
classification cs.CV
keywords videoinstancesegmentationobjecttrackingtemporalconsistencymemory-basedassociationoccupancy-guidedmatchingYouTube-VISOVIS
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LOMM tackles the problem of keeping the same object identity across long videos in video instance segmentation. The paper proposes maintaining a Latest Object Memory that stores each object's most recent state, weighted by how confidently the object is present in the current frame, rather than aggregating features across many frames or freezing old ones. It also proposes Decoupled Object Association, which first matches objects already in memory and then assigns fresh indices to newly appearing objects using occupancy information. The paper reports that this design yields consistent identity tracking and sets new benchmark numbers, including 54.0 AP on YouTube-VIS 2022.

What carries the argument

The central object is the Latest Object Memory (LOM), an online per-object feature store updated each frame with the object's foreground probability as the update weight; this makes recent valid appearances dominate while preserving the last known state of objects that have disappeared. Around it sits Decoupled Object Association (DOA), a matching strategy in which an existing-object tracker aligns memory features with current-frame queries, then Occupancy-guided Hungarian Matching (OHM) first matches already-occupied indices and only afterwards assigns remaining indices to newly appeared objects. An adaptive anchor query, formed by cosine-similarity blending of aligned current features and stored memory, is fed to a second tracker that produces the final object features. The load-bearing idea is that presence, measured by foreground probability, should control how quickly memory changes.

What would settle it

Reproduce the ResNet-50 configuration on YouTube-VIS 2022 and replace the LOM update in Eq. (3) with a fixed-coefficient exponential moving average while keeping DOA fixed; the paper's ablations predict AP would fall from 41.1 toward the momentum-memory score near 35.2. If no such drop occurs, the foreground-probability weighting is not what carries the result.

Watch

Extended reading notes

Core claim

The central claim is that long-term identity consistency in video instance segmentation is best served by a memory that holds the latest foreground-weighted state of each tracked object, combined with an association step that separates existing objects from newly appearing ones. The authors argue that prior memory mechanisms either mix features from many frames, causing ambiguity when objects reappear, or freeze old information, so they fail to track appearance changes. LOMM updates its memory online as $M_t = (1-p_t)M_{t-1} + p_t \dot{Q}_t$, where $p_t$ is the foreground probability of each object, so present objects continuously refresh their stored features while absent ones keep their last valid state. Association then proceeds in two stages: an existing-object tracker aligns memory features with current-frame queries, and Occupancy-guided Hungarian Matching assigns new indices only to slots never used before. The authors report that this combination achieves state-of-the-art average precision on YouTube-VIS 2019, 2021, and 2022, and on OVIS.

Load-bearing premise

The load-bearing premise is that the frozen segmentation network detects every object that needs an identity; if it misses or misclassifies an object, neither the memory nor the association step can recover that object.

Editorial extensions

If this is right

  • On YouTube-VIS 2022, the reported offline ViT-L result of 54.0 AP would be a new high, exceeding DVIS-DAQ and DVIS++ by the margins shown in Table 1.
  • On OVIS, the reported ViT-H result of 52.9 AP would surpass listed prior methods including UNINEXT, indicating that the approach also helps under heavy occlusion.
  • The ablation numbers imply that most of the gain comes from the memory mechanism itself: adding LOM to DVIS raises AP by 5.3 points on YouTube-VIS 2022 with a ResNet-50 backbone.
  • The tracker adds little parameter overhead over the DVIS baseline (9.88M versus 9.68M parameters), so the reported gains are not explained by a large capacity increase.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the occupancy vector is a lightweight side output that could support explicit reasoning about object reappearance and lifetime, a use the paper does not develop.
  • Editorial inference: because LOM updates with foreground probability, it should transfer to any query-based segmentation backbone, so swapping the frozen segmenter would test how much of the gain is memory design rather than the specific backbone.
  • Editorial inference: the paper's stated limitation predicts that identity failures should concentrate exactly where the segmentation network misses objects, making corrupted-frame stress tests a natural way to map the method's boundary.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes LOMM, a tracking module for video instance segmentation built on a frozen Mask2Former segmentation network. It maintains a per-slot Latest Object Memory updated by foreground probability (Eq. 3) and uses Decoupled Object Association (Sec. 4.2) that first matches existing objects, then assigns new objects to unoccupied slots through Occupancy-guided Hungarian Matching, and finally aligns all objects with an adaptive anchor query. The paper reports state-of-the-art results on YouTube-VIS and OVIS, with a headline AP of 54.0 on YouTube-VIS 2022. In Table 1, that number appears only in the Ours* row, which uses an offline temporal refiner borrowed from DVIS; the online ViT-L model obtains 48.2 AP.

Significance. The proposed memory update and decoupled association are simple, cheap, and plausible components for long-term identity management; the ablation suggests gains over DVIS on YTVIS22 with only 9.88M tracker parameters. If the reported numbers are reproducible, the online variant is competitive and the offline variant may be state of the art. However, the current presentation does not establish the headline claim because the refiner contribution is not isolated and because the online result is not compared against TCOVIS, which appears in Table 1 with 51.0 AP on YTVIS22. The paper also omits variance estimates and contains inconsistent baseline numbers between the main table and the ablation table. The frozen-segmentation limitation in A.2.4 is acknowledged and is not, by itself, a flaw.

major comments (4)
  1. [Abstract; §5.2; Table 1; A.2.2] The abstract and Section 5.2 state that 'LOMM achieves state-of-the-art AP score of 54.0 on YouTube-VIS 2022' and that LOMM beats DVIS++ by '+3.1 AP' on YTVIS22, without disclosing that these numbers come from Ours*, the offline row in Table 1 that uses a temporal refiner from DVIS. The online Ours ViT-L row reports 48.2 AP, which is below TCOVIS Swin-L (51.0 AP) in the same table. Since the refiner is an external component and its contribution is not ablated, the paper's central claim overstates what LOMM's own components establish. The online and offline claims must be separated, and an ablation should isolate the refiner's effect.
  2. [Table 3 vs Table 1] The ablation baselines are inconsistent with the main results: Table 3-(b) reports MinVIS at 26.8 AP and DVIS at 33.9 AP on YTVIS22, while Table 1 reports 23.3 AP and 31.6 AP for the same configurations. The claimed LOM gains of +7.2 AP over MinVIS and +5.3 AP over DVIS are therefore computed against numbers that differ from the paper's own main table. In addition, Section 5.3 states that applying DOA alone improves DVIS by +3.3 AP, but Table 3-(a) shows 36.2 vs 33.9, a +2.3 AP difference. These numbers need to be reconciled or corrected.
  3. [§5.2] When the paper claims superiority over DVIS-DAQ under the same online ViT-L setting, the YTVIS22 comparison is 48.2 vs 42.0 AP, but TCOVIS, which achieves 51.0 AP on YTVIS22 with a Swin-L backbone, is not discussed anywhere in Section 5.2. Because the abstract's SOTA claim is for the offline variant, this omission obscures the fact that the online variant is not state of the art on YTVIS22; the online comparison should explicitly include TCOVIS and state the ranking.
  4. [§5.1, Tables 1–2] No error bars or multiple-seed results are reported for any benchmark number. Several claimed margins are small enough that variance information is needed to assess significance (e.g., +0.8 AP on YTVIS19 and +0.9 AP on OVIS against CTVIS). Please add at least the standard deviation over multiple seeds for the main comparisons, or justify why seed variation is negligible.
minor comments (4)
  1. [Eq. (5)] In Eq. (5), O_t is defined as O_{t-1} ∨ O_t, which is self-referential; define the current-frame presence indicator separately (e.g., o_t) before taking the OR.
  2. [A.2.4] The limitation passage says LOMM cannot handle objects that the segmentation network fails to detect, but the last sentence of the same passage claims that 'even when the segmentation network misses certain objects, our method maintains robust tracking.' This is contradictory and should be clarified.
  3. [Throughout] There are numerous typos and infelicities, including 'anDecoupled' in the Introduction, 'unltimately' and 'exisiting appoaches' in Section 3, and 'Y ellow' in the Fig. 1 caption. A careful proofread is needed.
  4. [Table 3] The formatting of Table 3 parts (e) and (f) is ambiguous: the checkmark columns do not identify which row is the ablation-without condition, and the 'Metric' column header is repeated. Please make each subtable self-explanatory.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: LOMM's memory update and DOA are constructive algorithms evaluated on benchmarks; the 54.0 SOTA figure's reliance on a borrowed offline refiner is a transparency issue, not a circular reduction.

full rationale

The paper's derivation chain is empirical and constructive rather than circular. LOM's memory update (Eq. 3) is a foreground-probability-weighted recurrence over segmentation-network features; DOA (Eqs. 4-6 and Algorithm 1) defines an association procedure; and the training losses (Eqs. 8-10) optimize tracking against ground-truth annotations. No quantity reported as a prediction is defined in terms of the target AP, and no parameter is fitted to the benchmark metric and then re-reported as a finding. The only self-citation is [23] in A.2.2 for contrastive pretraining, cited together with external works [38,49,51], and it is not load-bearing. The abstract's 54.0 YTVIS22 figure is an offline result obtained with a temporal refiner borrowed from DVIS [50] (Table 1 footnote: 'We employ a temporal refiner [50] for offline model'), while the online ViT-L model scores 48.2 AP and is below TCOVIS's 51.0 in Table 1; this is a reporting and attribution concern, not a circular reduction. A.2.4 honestly states the frozen-segmentation limitation. No step reduces to its own input by construction.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced. The free parameters are limited to two hand-set training choices. The main load-bearing assumptions are the frozen segmentation network and the reliability of foreground probabilities for memory updates.

free parameters (2)
  • lambda_Sim = 1.0
    Weight of the similarity loss in Eq. (10), set empirically in Sec. 5.1 and A.2.2.
  • early_training_ratio = 0.5 (first half of 160k iterations)
    Fraction of training that uses segmentation-network predictions for ground truth assignment, chosen by hand in Sec. 4.3.
assumptions (4)
  • domain assumption Tracking networks TE and TA are trained while the segmentation network S is frozen
    Sec. 5.1 and A.2.2. This makes LOMM depend entirely on S's detections; the paper acknowledges in A.2.4 that missed objects cannot be recovered.
  • domain assumption Foreground probability p_t, from S's class logits, reliably indicates object presence for memory weighting
    Eq. (3) scales the memory update by p_t; no independent evaluation of this reliability is provided.
  • domain assumption Contrastive-pretrained queries from prior methods are suitable for the new matching losses
    The paper inherits contrastive pretraining from [23,38,49,51] without ablating it under LOMM.
  • standard math Hungarian assignment provides the optimal matching for the decoupled stages
    Algorithm 1 and Eqs. (7)-(8) rely on the Hungarian algorithm; this is standard and not at issue.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Latest Object Memory Management for Temporally Consistent Video Instance Segmentation." pith.science (2026). https://pith.science/paper/VB6PWIAX

@misc{pith2026250719754,
  author       = {Pith},
  title        = {Pith review of: Latest Object Memory Management for Temporally Consistent Video Instance Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VB6PWIAX}},
  note         = {Machine review of arXiv:2507.19754}
}
read the original abstract

In this paper, we present Latest Object Memory Management (LOMM) for temporally consistent video instance segmentation that significantly improves long-term instance tracking. At the core of our method is Latest Object Memory (LOM), which robustly tracks and continuously updates the latest states of objects by explicitly modeling their presence in each frame. This enables consistent tracking and accurate identity management across frames, enhancing both performance and reliability through the VIS process. Moreover, we introduce Decoupled Object Association (DOA), a strategy that separately handles newly appearing and already existing objects. By leveraging our memory system, DOA accurately assigns object indices, improving matching accuracy and ensuring stable identity consistency, even in dynamic scenes where objects frequently appear and disappear. Extensive experiments and ablation studies demonstrate the superiority of our method over traditional approaches, setting a new benchmark in VIS. Notably, our LOMM achieves state-of-the-art AP score of 54.0 on YouTube-VIS 2022, a dataset known for its challenging long videos. Project page: https://seung-hun-lee.github.io/projects/LOMM/

Figures

Figures reproduced from arXiv: 2507.19754 by the authors.

Figure 1
Figure 1. Qualitative comparisons on a challenging video from YouTube-VIS 2022. Yellow text highlights objects that failed to be tracked. (First row) DVIS++ [51] often fails to track objects that reappear after a temporal gap, as it relies solely on the previous frame. (Second row) CTVIS [49] employs a memory mechanism that aggregates object features across frames based on similarity weights, potentially leading to ID switchi… view at source ↗
Figure 2
Figure 2. Limitations of existing trackers. When a new object appears, there is no reference for assigning its ID, so it must be mapped to an ID previously used for the background. This results in an unnatural learning process where foreground and background features become overly similar, leading to suboptimal performance. along the (K + 1) dimension. The principal challenge in VIS lies in maintaining consis￾tent object repr… view at source ↗
Figure 3
Figure 3. Overall pipeline of our LOMM. (a) For each frame, the existing object tracker TE utilizes Mt−1 from the Latest Object Memory and Q˜t from the segmentation network S to predict aligned features Qˆt for matched existing objects. The current frame’s existing objects Qˆt and all objects Q˜t are matched through Occupancy-guided Hungarian Matching (OHM), ensuring accurate ID assignment for potential new objects and genera… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: VIS results from our model on challenging videos in OVIS dataset. (First and Second row) In both videos, the objects are successfully tracked without any ID switch even after complete occlusion. (Third row) The model robustly tracks a newly appeared object throughout t…
Figure 5
Figure 5. Figure 5: Comparison of three different memory mechanisms [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Limitation. Our method cannot handle cases where the segmentation network fails to detect objects. However, for detected objects, it demonstrates consistent tracking performance. A.2.3. Additional Qualitative Results We provide additional comparisons with state-of-the-…
Figure 7
Figure 7. Figure 7: Qualitative comparison of LOMM with CTVIS, DVIS-DAQ, and DVIS++ on challenging scenarios in YTVIS22 dataset. [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

59 extracted references · 47 canonical work pages

  1. [1]

    Memot: Multi-object track- ing with memory

    Jiarui Cai, Mingze Xu, Wei Li, Yuanjun Xiong, Wei Xia, Zhuowen Tu, and Stefano Soatto. Memot: Multi-object track- ing with memory. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 8090–8100, 2022. 3

    work page 2022

  2. [2]

    Sipmask: Spatial information preservation for fast image and video in- stance segmentation

    Jiale Cao, Rao Muhammad Anwer, Hisham Cholakkal, Fa- had Shahbaz Khan, Yanwei Pang, and Ling Shao. Sipmask: Spatial information preservation for fast image and video in- stance segmentation. InComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIV 16, pages 1–18. Springer, 2020. 2

    work page 2020

  3. [3]

    End-to- end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. InEuropean confer- ence on computer vision, pages 213–229. Springer, 2020. 2, 3

    work page 2020

  4. [4]

    Vision transformer adapter for dense predictions.arXiv preprint arXiv:2205.08534, 2022

    Zhe Chen, Yuchen Duan, Wenhai Wang, Junjun He, Tong Lu, Jifeng Dai, and Yu Qiao. Vision transformer adapter for dense predictions.arXiv preprint arXiv:2205.08534, 2022. 13

    arXiv 2022

  5. [5]

    Mask2former for video instance segmentation.arXiv preprint arXiv:2112.10764, 2021

    Bowen Cheng, Anwesa Choudhuri, Ishan Misra, Alexan- der Kirillov, Rohit Girdhar, and Alexander G Schwing. Mask2former for video instance segmentation.arXiv preprint arXiv:2112.10764, 2021. 5, 6

    arXiv 2021

  6. [6]

    Per- pixel classification is not all you need for semantic segmenta- tion.Advances in neural information processing systems, 34: 17864–17875, 2021

    Bowen Cheng, Alex Schwing, and Alexander Kirillov. Per- pixel classification is not all you need for semantic segmenta- tion.Advances in neural information processing systems, 34: 17864–17875, 2021. 1

    work page 2021

  7. [7]

    Masked-attention mask trans- former for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar. Masked-attention mask trans- former for universal image segmentation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1290–1299, 2022. 1, 2, 3, 6, 13

    work page 2022

  8. [8]

    Xmem: Long-term video object segmentation with an atkinson-shiffrin memory model

    Ho Kei Cheng and Alexander G Schwing. Xmem: Long-term video object segmentation with an atkinson-shiffrin memory model. InEuropean Conference on Computer Vision, pages 640–658. Springer, 2022. 3

    work page 2022

Show all 59 references
  1. [9]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

    2021

  2. [10]

    Cap- sulevos: Semi-supervised video object segmentation using capsule routing

    Kevin Duarte, Yogesh S Rawat, and Mubarak Shah. Cap- sulevos: Semi-supervised video object segmentation using capsule routing. InProceedings of the IEEE/CVF interna- tional conference on computer vision, pages 8480–8489, 2019. 3

    2019

  3. [11]

    Stm- track: Template-free visual tracking with space-time memory networks

    Zhihong Fu, Qingjie Liu, Zehua Fu, and Yunhong Wang. Stm- track: Template-free visual tracking with space-time memory networks. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13774–13783,

  4. [12]

    Memotr: Long-term memory- augmented transformer for multi-object tracking

    Ruopeng Gao and Limin Wang. Memotr: Long-term memory- augmented transformer for multi-object tracking. InProceed- ings of the IEEE/CVF International Conference on Computer Vision, pages 9901–9910, 2023. 1, 3, 13

    2023

  5. [13]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 6, 13

    2016

  6. [14]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. InProceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017. 2

    2017

  7. [15]

    Vita: Video instance segmentation via object token association.Advances in Neural Information Processing Systems, 35:23109–23120, 2022

    Miran Heo, Sukjun Hwang, Seoung Wug Oh, Joon-Young Lee, and Seon Joo Kim. Vita: Video instance segmentation via object token association.Advances in Neural Information Processing Systems, 35:23109–23120, 2022. 1, 3, 6, 13

    2022

  8. [16]

    A generalized framework for video instance segmentation

    Miran Heo, Sukjun Hwang, Jeongseok Hyun, Hanjung Kim, Seoung Wug Oh, Joon-Young Lee, and Seon Joo Kim. A generalized framework for video instance segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 14623–14632, 2023. 3, 6, 13

    2023

  9. [17]

    Minvis: A minimal video instance segmentation framework without video-based training.Advances in Neural Information Pro- cessing Systems, 35:31265–31277, 2022

    De-An Huang, Zhiding Yu, and Anima Anandkumar. Minvis: A minimal video instance segmentation framework without video-based training.Advances in Neural Information Pro- cessing Systems, 35:31265–31277, 2022. 1, 2, 3, 6, 7, 8, 13

    2022

  10. [18]

    Fast video object segmentation with temporal aggregation network and dynamic template matching

    Xuhua Huang, Jiarui Xu, Yu-Wing Tai, and Chi-Keung Tang. Fast video object segmentation with temporal aggregation network and dynamic template matching. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8879–8889, 2020. 3

    2020

  11. [19]

    Video instance segmentation using inter-frame com- munication transformers.Advances in Neural Information Processing Systems, 34:13352–13363, 2021

    Sukjun Hwang, Miran Heo, Seoung Wug Oh, and Seon Joo Kim. Video instance segmentation using inter-frame com- munication transformers.Advances in Neural Information Processing Systems, 34:13352–13363, 2021. 2

    2021

  12. [20]

    Visage: Video instance segmentation with appearance-guided enhancement.arXiv preprint arXiv:2312.04885, 2023

    Hanjung Kim, Jaehyun Kang, Miran Heo, Sukjun Hwang, Seoung Wug Oh, and Seon Joo Kim. Visage: Video instance segmentation with appearance-guided enhancement.arXiv preprint arXiv:2312.04885, 2023. 1, 3

    2023 arXiv

  13. [21]

    Offline-to-online knowledge distillation for video instance segmentation

    Hojin Kim, Seunghun Lee, Hyeon Kang, and Sunghoon Im. Offline-to-online knowledge distillation for video instance segmentation. InProceedings of the IEEE/CVF Winter Con- ference on Applications of Computer Vision, pages 159–168,

  14. [22]

    The hungarian method for the assignment problem.Naval research logistics quarterly, 2(1-2):83–97,

    Harold W Kuhn. The hungarian method for the assignment problem.Naval research logistics quarterly, 2(1-2):83–97,

  15. [23]

    Context-aware video instance segmentation

    Seunghun Lee, Jiwan Seo, Kiljoon Han, Minwoo Choi, and Sunghoon Im. Context-aware video instance segmentation. arXiv preprint arXiv:2407.03010, 2024. 3, 13

    2024 arXiv

  16. [24]

    Tcovis: Temporally consistent online video instance segmentation

    Junlong Li, Bingyao Yu, Yongming Rao, Jie Zhou, and Jiwen Lu. Tcovis: Temporally consistent online video instance segmentation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 1097–1107, 2023. 1, 3, 5, 6, 13

    2023

  17. [25]

    Mdqe: Mining discriminative query embeddings to segment occluded instances on challenging videos

    Minghan Li, Shuai Li, Wangmeng Xiang, and Lei Zhang. Mdqe: Mining discriminative query embeddings to segment occluded instances on challenging videos. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10524–10533, 2023. 6, 13 9

    2023

  18. [26]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceeding...

    2014

  19. [27]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 6

    2021

  20. [28]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017. 13

    2017 arXiv

  21. [29]

    Trackformer: Multi-object tracking with transformers

    Tim Meinhardt, Alexander Kirillov, Laura Leal-Taixe, and Christoph Feichtenhofer. Trackformer: Multi-object tracking with transformers. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 8844–8854, 2022. 3

    2022

  22. [30]

    Novis: A case for end-to-end near-online video instance segmentation.arXiv preprint arXiv:2308.15266, 2023

    Tim Meinhardt, Matt Feiszli, Yuchen Fan, Laura Leal- Taixe, and Rakesh Ranjan. Novis: A case for end-to-end near-online video instance segmentation.arXiv preprint arXiv:2308.15266, 2023. 6

    2023 arXiv

  23. [31]

    Occluded video instance segmentation: A bench- mark.International Journal of Computer Vision, 130(8): 2022–2039, 2022

    Jiyang Qi, Yan Gao, Yao Hu, Xinggang Wang, Xiaoyu Liu, Xiang Bai, Serge Belongie, Alan Yuille, Philip HS Torr, and Song Bai. Occluded video instance segmentation: A bench- mark.International Journal of Computer Vision, 130(8): 2022–2039, 2022. 6, 13

    2022

  24. [32]

    Sam 2: Segment any- thing in images and videos.arXiv preprint arXiv:2408.00714,

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment any- thing in images and videos.arXiv preprint arXiv:2408.00714,

    arXiv

  25. [33]

    Learning video object segmentation with visual memory

    Pavel Tokmakov, Karteek Alahari, and Cordelia Schmid. Learning video object segmentation with visual memory. In Proceedings of the IEEE international conference on com- puter vision, pages 4481–4490, 2017. 3

    2017

  26. [34]

    Rvos: End- to-end recurrent network for video object segmentation

    Carles Ventura, Miriam Bellver, Andreu Girbau, Amaia Sal- vador, Ferran Marques, and Xavier Giro-i Nieto. Rvos: End- to-end recurrent network for video object segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5277–5286, 2019. 3

    2019

  27. [35]

    Omnitracker: Unifying object tracking by tracking-with-detection.arXiv preprint arXiv:2303.12079, 2023

    Junke Wang, Dongdong Chen, Zuxuan Wu, Chong Luo, Xiyang Dai, Lu Yuan, and Yu-Gang Jiang. Omnitracker: Unifying object tracking by tracking-with-detection.arXiv preprint arXiv:2303.12079, 2023. 3

    2023

  28. [36]

    End-to-end video instance segmentation with transformers

    Yuqing Wang, Zhaoliang Xu, Xinlong Wang, Chunhua Shen, Baoshan Cheng, Hao Shen, and Huaxia Xia. End-to-end video instance segmentation with transformers. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8741–8750, 2021. 2

    2021

  29. [37]

    Seqformer: Sequential transformer for video instance segmentation

    Junfeng Wu, Yi Jiang, Song Bai, Wenqing Zhang, and Xiang Bai. Seqformer: Sequential transformer for video instance segmentation. InEuropean Conference on Computer Vision, pages 553–569. Springer, 2022. 2, 13

    2022

  30. [38]

    In defense of online models for video instance segmentation

    Junfeng Wu, Qihao Liu, Yi Jiang, Song Bai, Alan Yuille, and Xiang Bai. In defense of online models for video instance segmentation. InEuropean Conference on Computer Vision, pages 588–605. Springer, 2022. 1, 2, 3, 6, 13

    2022

  31. [39]

    General object foundation model for images and videos at scale

    Junfeng Wu, Yi Jiang, Qihao Liu, Zehuan Yuan, Xiang Bai, and Song Bai. General object foundation model for images and videos at scale. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 3783–3795, 2024. 3

    2024

  32. [40]

    Youtube-vos: A large-scale video object segmentation benchmark.arXiv preprint arXiv:1809.03327, 2018

    Ning Xu, Linjie Yang, Yuchen Fan, Dingcheng Yue, Yuchen Liang, Jianchao Yang, and Thomas Huang. Youtube-vos: A large-scale video object segmentation benchmark.arXiv preprint arXiv:1809.03327, 2018. 3

    2018 arXiv

  33. [41]

    Learning spatio-temporal transformer for vi- sual tracking

    Bin Yan, Houwen Peng, Jianlong Fu, Dong Wang, and Huchuan Lu. Learning spatio-temporal transformer for vi- sual tracking. InProceedings of the IEEE/CVF international conference on computer vision, pages 10448–10457, 2021. 3

    2021

  34. [42]

    Towards grand unification of object tracking

    Bin Yan, Yi Jiang, Peize Sun, Dong Wang, Zehuan Yuan, Ping Luo, and Huchuan Lu. Towards grand unification of object tracking. InEuropean conference on computer vision, pages 733–751. Springer, 2022. 3

    2022

  35. [43]

    Universal instance percep- tion as object discovery and retrieval

    Bin Yan, Yi Jiang, Jiannan Wu, Dong Wang, Ping Luo, Ze- huan Yuan, and Huchuan Lu. Universal instance percep- tion as object discovery and retrieval. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15325–15336, 2023. 3, 6

    2023

  36. [44]

    Video instance seg- mentation

    Linjie Yang, Yuchen Fan, and Ning Xu. Video instance seg- mentation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 5188–5197, 2019. 1, 2, 3, 6, 13

    2019

  37. [45]

    The 3rd large-scale video object segmentation challenge - video in- stance segmentation track, 2021

    Linjie Yang, Yuchen Fan, Yang Fu, and Ning Xu. The 3rd large-scale video object segmentation challenge - video in- stance segmentation track, 2021. 13

    2021

  38. [46]

    Crossover learn- ing for fast online video instance segmentation

    Shusheng Yang, Yuxin Fang, Xinggang Wang, Yu Li, Chen Fang, Ying Shan, Bin Feng, and Wenyu Liu. Crossover learn- ing for fast online video instance segmentation. InProceed- ings of the IEEE/CVF International Conference on Computer Vision, pages 8043–8052, 2021. 2

    2021

  39. [47]

    Temporally efficient vision transformer for video instance segmentation

    Shusheng Yang, Xinggang Wang, Yu Li, Yuxin Fang, Jiemin Fang, Wenyu Liu, Xun Zhao, and Ying Shan. Temporally efficient vision transformer for video instance segmentation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2885–2895, 2022. 2

    2022

  40. [48]

    Learning dynamic memory networks for object tracking

    Tianyu Yang and Antoni B Chan. Learning dynamic memory networks for object tracking. InProceedings of the European conference on computer vision (ECCV), pages 152–167, 2018. 3

    2018

  41. [49]

    Ctvis: Consistent training for online video instance segmentation

    Kaining Ying, Qing Zhong, Weian Mao, Zhenhua Wang, Hao Chen, Lin Yuanbo Wu, Yifan Liu, Chengxiang Fan, Yunzhi Zhuge, and Chunhua Shen. Ctvis: Consistent training for online video instance segmentation. InProceedings of the IEEE/CVF International Conference on Computer Vision, ...

    2023

  42. [50]

    Dvis: Decoupled video instance segmentation framework.arXiv preprint arXiv:2306.03413, 2023

    Tao Zhang, Xingye Tian, Yu Wu, Shunping Ji, Xuebo Wang, Yuan Zhang, and Pengfei Wan. Dvis: Decoupled video instance segmentation framework.arXiv preprint arXiv:2306.03413, 2023. 1, 2, 3, 5, 6, 7, 8, 13

    2023 arXiv

  43. [51]

    Dvis++: Improved decoupled framework for uni- versal video segmentation.arXiv preprint arXiv:2312.13305,

    Tao Zhang, Xingye Tian, Yikang Zhou, Shunping Ji, Xuebo Wang, Xin Tao, Yuan Zhang, Pengfei Wan, Zhongyuan Wang, and Yu Wu. Dvis++: Improved decoupled framework for uni- versal video segmentation.arXiv preprint arXiv:2312.13305,

    arXiv

  44. [52]

    A transductive approach for video object segmentation

    Yizhuo Zhang, Zhirong Wu, Houwen Peng, and Stephen Lin. A transductive approach for video object segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6949–6958, 2020. 3

    2020

  45. [53]

    Object-centric multiple object tracking

    Zixu Zhao, Jiaze Wang, Max Horn, Yizhuo Ding, Tong He, Zechen Bai, Dominik Zietlow, Carl-Johann Simon-Gabriel, Bing Shuai, Zhuowen Tu, et al. Object-centric multiple object tracking. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 16601–16611, 2023. 3

    2023

  46. [54]

    Dvis-daq: Improving video segmentation via dynamic anchor queries.arXiv preprint arXiv:2404.00086,

    Yikang Zhou, Tao Zhang, Shunping Ji, Shuicheng Yan, and Xiangtai Li. Dvis-daq: Improving video segmentation via dynamic anchor queries.arXiv preprint arXiv:2404.00086,

    arXiv

  47. [55]

    Deformable detr: Deformable transformers for end-to-end object detection.arXiv preprint arXiv:2010.04159,

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection.arXiv preprint arXiv:2010.04159,

    2010 arXiv

  48. [59]

    The 2019 release includes 2,238 videos for training, 302 for validation, and 343 for testing

    alongside the pioneering study on the Video Instance Segmentation (VIS) task, the YouTube-VIS datasets con- sist of high-resolution YouTube videos across 40 categories. The 2019 release includes 2,238 videos for training, 302 for validation, and 343 for testing. In its 2021 up...

    2019

  49. [2020]

    Appendix Figure 5

    2 11 A. Appendix Figure 5. Comparison of three different memory mechanisms A.1. Motivation for Memory Design In this section, we outline the motivation behind the design of LOMM, comparing three different memory systems: (1) Latest Object Memory (LOM), (2) Similarity-based mem...

    2019

  50. [2022]

    Introduced by

    [44] and OVIS [31], as detailed below. Introduced by

  51. [2023]

    1, 2, 3, 6, 8, 13, 14 10

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.