Pith. sign in

REVIEW 3 major objections 4 minor 51 references

FluxGraph claims that SAM2's internal multi-mask disagreement is a sufficient and cheap trigger for detecting object transformations, enabling 3.3–10.7× speedups over exhaustive tracking while matching or improving quality.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

FluxGraph detects object transformations reactively via SAM2's multi-mask disagreement, cutting TubeletGraph's inference cost by 3.3–10.7x with comparable tracking and state-graph quality.

T0 review reviewed 2026-08-01 challenge →

load-bearing objection Legit efficiency upgrade with a genuinely new reactive trigger, but the speedup is measured per component-mask, not per video, so the headline numbers need a caveat or better data. the 3 major comments →

arxiv 2607.19743 v1 pith:VVDAJS2Q submitted 2026-07-22 cs.CV

Efficient Tracking and Understanding Object Transformations

Yihong Sun , Bharath Hariharan This is my paper
classification cs.CV
keywords object transformation trackingvideo object segmentationSAM2 multi-maskreactive triggerstate graphefficient video understandingreverse tracking
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FluxGraph sets out to show that tracking an object through splits, tears, and phase changes does not require building an exhaustive spatiotemporal partition of the whole video. The paper's key claim is that the tracker's own internal uncertainty—specifically the disagreement among SAM2's multiple mask hypotheses—already tells you when and where a transformation is producing a new object part, so all downstream computation can be skipped on frames and regions where the hypotheses agree. On four benchmarks, this reactive design is 3.3–10.7× faster than the exhaustive TubeletGraph pipeline while matching or improving tracking quality and preserving the quality of the constructed state graph. The practical payoff is that transformation-aware tracking, previously costing seconds per object-mask, moves into a range where online deployment or large-scale video analysis becomes feasible.

Core claim

The paper discovers that the expensive, eager spatiotemporal partition used by TubeletGraph is unnecessary for recovering object transformations. Instead, SAM2's internal multi-mask outputs—a primary mask and two alternative hypotheses—implicitly encode where related objects, including products of a transformation, are likely to emerge. FluxGraph turns this into a reactive trigger: when the alternative masks agree with the primary, no transformation is flagged and all expensive entity segmentation and tracking is skipped; when they disagree in a region not already covered by tracked masks, entity segmentation is run only on that region. Each candidate region is then propagated forward and ba

What carries the argument

The central mechanism is the multi-mask disagreement trigger: SAM2 produces a primary mask and two alternative hypotheses, and FluxGraph uses their agreement or disagreement as a per-frame, per-region signal. When both alternatives are nearly identical to the primary (IoU above τ_agree), no transformation is suggested; when they differ in a residual area not covered by already-tracked masks (above τ_res), entity segmentation is invoked only there. The complementary mechanism is reverse tracking: each candidate track is propagated backward in time and accepted only if its mean IoU with the prompt track through early frames exceeds τ_rev, providing a model-free geometric consistency filter tha

Load-bearing premise

The load-bearing premise is that SAM2's multi-mask disagreement is a reliable and sensitive signal for object transformations—frames where the candidate masks agree can safely be skipped without missing transformation products, and when they disagree, the disagreement points exactly where a new object appears.

What would settle it

Construct or select a video in which an object splits or tears across one or two frames with almost no change in shape or texture between consecutive frames, so that SAM2's three mask hypotheses remain mostly in agreement. If FluxGraph misses the transformation while an exhaustive per-frame entity-segmentation pipeline (TubeletGraph) detects it, the trigger's sensitivity assumption is refuted; the paper's own Limitations section predicts this failure mode.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Transformation-aware tracking becomes practical for online or resource-constrained deployment: per-object-mask cost on VOST drops from about 4.4 seconds to about 1.3 seconds, a 3.3× speedup.
  • Computational cost now scales with the number of detected transformation candidates rather than the total number of entities in the scene, so cluttered videos with a single transformation become dramatically cheaper.
  • On transformation-free videos, the method degrades gracefully: on DAVIS17 the cost is only 0.45 seconds per mask (1.5× over the base tracker), compared to 4.81 seconds for exhaustive partitioning.
  • The state graph construction benefits from tighter candidate proposals: the reactive trigger produces more clearly localized masks, and the paper reports improved semantic accuracy of resulting object descriptions (SO = 82.9 vs 72.3).
  • The pipeline is simplified by removing a separate semantic filtering model (FC-CLIP) entirely, replacing it with reverse tracking that reuses the existing tracker.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural next test is whether the disagreement trigger generalizes to trackers other than SAM2: any model that exposes multiple plausible mask hypotheses could power the same reactive strategy, but a tracker that only outputs a single mask would need a different uncertainty signal (the paper's own limitation acknowledges this dependency).
  • The principle behind reverse tracking—that a genuine transformation product, when propagated backward, must align with the original object—may serve as a general, model-free consistency check in other tracking or segmentation pipelines, beyond the transformation setting.
  • The speedups imply that 'skip-when-certain' reactive pipelines could apply to other sparse video understanding tasks, such as detecting occlusions, object interactions, or sudden appearance changes, where dense processing is currently the default.
  • A stress test the authors leave implicit: if a fast transformation produces almost no multi-mask disagreement, FluxGraph will miss it while exhaustive partitioning would not, suggesting that a hybrid that occasionally runs a cheap dense check could close that gap.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. FluxGraph is a reactive variant of TubeletGraph for the Track Any State task. Instead of eagerly partitioning the whole video into entity tracks, it uses disagreement among SAM2's primary and two alternative mask outputs as a per-frame trigger: when the masks agree, entity segmentation and new-track initialization are skipped; when they disagree, CropFormer segments only the untracked residual regions and candidate tracks are filtered by reverse tracking. The paper evaluates on VOST, VSCOS, M3-VOS and DAVIS17 and reports ~3.3x cost reduction per component-mask on VOST and 3.7-10.7x on the other datasets, with roughly matching tracking and state graph quality, plus ablations of the trigger and reverse tracking. Code is released.

Significance. If the efficiency measurement is sound, this is a useful contribution: it addresses a practical bottleneck (TubeletGraph's seconds-per-object-frame cost), introduces a simple and transferable trigger based on SAM2's internal uncertainty, removes the FC-CLIP dependency, and evaluates on four benchmarks with state-graph metrics on VOST-TAS. The strengths include released code, a clear ablation, and a reasonable qualitative analysis. The main caveat is that the headline speedups are normalized per component-mask rather than measured per video, so the actual wall-clock gain remains unverified.

major comments (3)
  1. [Sec. 4.2, Table 2] The central speedup claim is not supported by the reported numbers. Table 2 reports 'time per comp-mask' and MPS, i.e., cost normalized by the number of component masks each method itself produces. TubeletGraph builds an exhaustive spatiotemporal partition and hence produces many entity tracks, while FluxGraph only tracks candidates triggered by disagreement, so the two methods very plausibly produce different numbers of comp-masks per video. An MPS ratio of 3.3 does not equal a 3.3x wall-clock speedup; the conversion requires the mask-count ratio. The abstract and Sec. 4.2 phrase the result as 'inference cost' and 'faster,' which readers will read as per-video wall-clock. Please report end-to-end wall-clock time per video (or per object-frame) for both methods and the total number of comp-masks per video; also report the per-video mask-count ratio. Without this, the headline 3.3-10.7x s
  2. [Sec. 4.2 and Table 3] Tracking-quality statements overstate the evidence. The paper says FluxGraph 'improves tracking performance' and that VSCOS/M3-VOS differences are 'not statistically meaningful,' but no error bars, variance, or significance tests are reported on any J/J_tr/H metric. On VSCOS and M3-VOS, FluxGraph is numerically worse than TubeletGraph (75.3 vs 75.9, 73.7 vs 74.1), so the improvement claim is only true on VOST. Please add per-video standard errors or paired tests, and qualify the abstract claim accordingly. The state-graph numbers (H_ST=12.0 vs 12.0, H=9.3 vs 6.5) are based on only 108 transformations; without intervals this is not enough to support 'preserving state graph quality.'
  3. [Sec. 3.2 and Limitations] The entire method depends on the assumption that SAM2 multi-mask disagreement is a sensitive trigger for transformations. The Limitations section concedes that fast transformations with minimal disagreement can evade the trigger and would be missed by FluxGraph but not by TubeletGraph. This is not just an edge case: it is the failure mode of the core mechanism. The aggregate H_ST equality in Table 3 is reassuring but could fail to reveal a systematic miss pattern. Please report trigger-level statistics: number of transformation events on VOST-TAS actually missed at the trigger stage, number of CropFormer queries fired, and examples where the masks agree but a transformation occurs. This will let readers assess the reliability assumption.
minor comments (4)
  1. [Sec. 3.2] A large block of unrelated LaTeX template text (Sample LaTeX File, Binomial Theorem, Taylor Series, etc.) is inserted after Equation (1). This corrupts the paper and must be removed; the equations and numbering in this section should be restored.
  2. [Sec. 4.1] Four thresholds (tau_agree, tau_res, tau_new, tau_rev) are chosen by sweeping on VOST and transferred. Since these thresholds control the trigger and filtering, report sensitivity, or at least the sweep range and performance at neighboring values, so the reader can assess robustness.
  3. [Related Work] ReVOS is cited as [4], but [4] is the M3-VOS paper; the ReVOS entry appears to be missing or mismatched.
  4. [Abstract vs. Table 2] The abstract says TubeletGraph costs ~4.4 seconds per object-frame on VOST, while Table 2 labels the same magnitude as seconds per comp-mask. Clarify whether object-frame and comp-mask are the same quantity and define the normalization in the main text, not only in the table caption.

Circularity Check

0 steps flagged

No circular derivation: the reactive-trigger speedup is an empirical measurement against a same-group baseline, not an input fitted and renamed as a prediction.

full rationale

The paper's central derivation is a pipeline engineering claim: TubeletGraph is expensive because it densely runs entity segmentation and tracks every entity, and FluxGraph replaces this eager partition with a SAM2 multi-mask disagreement trigger plus reverse-track filtering. The trigger is introduced as an empirical observation ('SAM2 often captures it as an alternative mask hypothesis'), not defined in terms of the transformations it is meant to detect. Equations (1)-(4) define skip heuristics and candidate filtering rules; the thresholds are swept on VOST training and frozen for other datasets, so the benchmark numbers are not fitted by construction. The speedup figures in Table 2 are measured wall-clock per-component costs and MPS values for both systems, and the 98% cost attribution in Sec. 3.1 is computable directly from Table 2 (1.097+3.228 of 4.392). No equation reduces a reported tracking or state-graph number to an input of the method. The main self-citation burden is the reliance on TubeletGraph [34], which shares the first and senior authors; it serves as the baseline, task definition, and state-graph evaluation source. But this is a run-and-timed external comparison system rather than an unverified theorem or imported ansatz, and the paper's own ablation (Table 4) shows the components' independent contributions. The Limitations section explicitly concedes that fast transformations with minimal multi-mask disagreement may evade the trigger, which confirms the trigger is a falsifiable assumption rather than a circular definition. A separate concern raised by the skeptic is that MPS normalizes per component-mask rather than per-video wall-clock; that is an efficiency-measurement question, not a circularity, because it does not make the speedup true by definition. Overall, the derivation is self-contained against measured baselines and external trackers, with only minor self-citation that is not load-bearing in the circularity sense.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

FluxGraph introduces no new physical or mathematical entities; it is a software pipeline composed of existing models. The central claim rests on four tuned thresholds and on assumptions about SAM2's internal mask behavior, benchmark annotations, and the transferability of thresholds. The thresholds are reported and held fixed across datasets, but they are still free hyperparameters chosen by a VOST-training sweep.

free parameters (4)
  • tau_agree = 0.75
    IoU threshold above which SAM2 alternative masks are considered agreeing with the primary, so the frame is skipped. Selected by sweeping on the VOST training split (Sec 4.1).
  • tau_res = 0.1
    Residual-area fraction threshold below which no untracked region is considered meaningful. Selected on VOST training.
  • tau_new = 0.3
    Minimum coverage of an entity by SAM2 alternative masks in untracked regions for it to become a candidate. Selected on VOST training.
  • tau_rev = 0.1
    Mean reverse-track IoU threshold for accepting a candidate as a transformation product. Selected on VOST training.
axioms (5)
  • ad hoc to paper SAM2's alternative multi-mask heads produce meaningful hypotheses about where transformation products appear (Sec 3.2).
    This is the key insight of the paper, but it is supported only by qualitative examples and ablation; no independent quantification is provided. The authors themselves note it may fail for fast transformations.
  • domain assumption A genuine transformation product, when tracked backward to frame 1, overlaps the original prompt object (Eq 4, Sec 3.3).
    Used to accept or reject all candidates. It is a plausible geometric consistency assumption but unproven and can be violated under large post-transformation motion or occlusion.
  • domain assumption The public benchmark annotations (VOST, VSCOS, M3-VOS, DAVIS17, VOST-TAS) correctly capture transformations and state graphs.
    All quantitative claims are measured against these annotations; standard for the field but not independently verified here.
  • domain assumption Pretrained components (SAM2.1, CropFormer, GPT-4.1) behave as claimed in their source papers.
    FluxGraph is a pipeline over these models; any failure or undocumented behavior propagates into the results.
  • domain assumption Thresholds chosen on VOST training transfer to VSCOS, M3-VOS, and DAVIS17 without retuning.
    The paper tests this transfer, which is reasonable support, but it remains an assumption about distribution shift across datasets.

reviewed 2026-08-01 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Efficient Tracking and Understanding Object Transformations." pith.science (2026). https://pith.science/paper/VVDAJS2Q

@misc{pith2026260719743,
  author       = {Pith},
  title        = {Pith review of: Efficient Tracking and Understanding Object Transformations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VVDAJS2Q}},
  note         = {Machine review of arXiv:2607.19743}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Tracking objects through state transformations is essential for understanding real-world dynamics. However, existing methods are computationally expensive. TubeletGraph recently showed impressive capabilities, but its inference cost (~$4.4$ seconds per object-frame on VOST) precludes any real-time deployment possibilities. We observe that TubeletGraph's overhead arises from building a spatiotemporal partition of the input video: (1) entity segmentation is computed densely for every frame regardless of whether a transformation occurs, and (2) every entity in the scene is tracked, scaling cost with scene complexity rather than the number of transformations of interest. To address both, we propose FluxGraph, a reactive variant that uses SAM2's internal multi-mask disagreement as a lightweight trigger for transformation detection, and removes the need for tracking all entities in the given video. FluxGraph is ~$3.3\times$ faster than TubeletGraph on VOST while improving tracking performance and preserving state graph quality. Furthermore, we also observe consistent speedups of $3.7-10.7\times$ across VSCOS, M$^3$-VOS, and DAVIS17 while maintaining performance. Code is publicly available at https://github.com/YihongSun/FluxGraph.

Figures

Figures reproduced from arXiv: 2607.19743 by Bharath Hariharan, Yihong Sun.

Figure 1
Figure 1. Figure 1: Performance vs efficiency. Each line shows a dataset; [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Given a video and a prompt mask, the goal is to track the object and construct a state graph for each detected transformation and [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of FluxGraph. (a) Given an input video and a prompt object (aluminum foil box, magenta), FluxGraph first propagates the prompt forward through the entire video via SAM2 [30] to obtain the primary track P (in magenta) and two alternative mask hypotheses {m1 s, m2 s} (in cyan and yellow). Note that any overlapping region between the alternative masks and the primary track is omitted for clarity. Any… view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative results. We showcase FluxGraph’s tracking and state graph predictions on top, with comparisons against baselines at [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

51 extracted references · 9 linked inside Pith

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Pith/arXiv arXiv

  2. [2]

    Learning what to learn for video object segmenta- tion

    Goutam Bhat, Felix J ¨aremo Lawin, Martin Danelljan, An- dreas Robinson, Michael Felsberg, Luc Van Gool, and Radu Timofte. Learning what to learn for video object segmenta- tion. InComputer Vision–ECCV 2020: 16th European Con- ference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pages 777–794. Springer, 2020. 2

    2020

  3. [3]

    One- shot video object segmentation

    Sergi Caelles, Kevis-Kokitsi Maninis, Jordi Pont-Tuset, Laura Leal-Taix´e, Daniel Cremers, and Luc Van Gool. One- shot video object segmentation. InProceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 221–230, 2017. 2

    2017

  4. [4]

    M 3-vos: Multi-phase, multi-transition, and multi-scenery video object segmenta- tion.arXiv preprint arXiv:2412.13803, 2024

    Zixuan Chen, Jiaxin Li, Liming Tan, Yejie Guo, Junxuan Liang, Cewu Lu, and Yong-Lu Li. M 3-vos: Multi-phase, multi-transition, and multi-scenery video object segmenta- tion.arXiv preprint arXiv:2412.13803, 2024. 2, 5, 7

    Pith/arXiv arXiv 2024

  5. [5]

    Xmem: Long- term video object segmentation with an atkinson-shiffrin memory model

    Ho Kei Cheng and Alexander G Schwing. Xmem: Long- term video object segmentation with an atkinson-shiffrin memory model. InEuropean Conference on Computer Vi- sion, pages 640–658. Springer, 2022. 2

    2022

  6. [6]

    Putting the object back into video object segmentation

    Ho Kei Cheng, Seoung Wug Oh, Brian Price, Joon-Young Lee, and Alexander Schwing. Putting the object back into video object segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3151–3161, 2024. 2, 7

    2024

  7. [7]

    Rescaling egocentric vision: Collection, pipeline and chal- lenges for epic-kitchens-100.International Journal of Com- puter Vision, pages 1–23, 2022

    Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Antonino Furnari, Evangelos Kazakos, Jian Ma, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, et al. Rescaling egocentric vision: Collection, pipeline and chal- lenges for epic-kitchens-100.International Journal of Com- puter Vision, pages 1–23, 2022. 2, 5

    2022

  8. [8]

    Mevis: A large-scale benchmark for video segmentation with motion expressions

    Henghui Ding, Chang Liu, Shuting He, Xudong Jiang, and Chen Change Loy. Mevis: A large-scale benchmark for video segmentation with motion expressions. InProceed- ings of the IEEE/CVF international conference on computer vision, pages 2694–2703, 2023. 2

    2023

  9. [9]

    Mose: A new dataset for video object segmentation in complex scenes

    Henghui Ding, Chang Liu, Shuting He, Xudong Jiang, Philip HS Torr, and Song Bai. Mose: A new dataset for video object segmentation in complex scenes. InProceed- ings of the IEEE/CVF international conference on computer vision, pages 20224–20234, 2023. 2

    2023

  10. [10]

    Sam2long: Enhancing sam 2 for long video seg- mentation with a training-free memory tree.arXiv preprint arXiv:2410.16268, 2024

    Shuangrui Ding, Rui Qian, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Yuwei Guo, Dahua Lin, and Jiaqi Wang. Sam2long: Enhancing sam 2 for long video seg- mentation with a training-free memory tree.arXiv preprint arXiv:2410.16268, 2024. 2, 7

    Pith/arXiv arXiv 2024

  11. [11]

    Event neural net- works

    Matthew Dutson, Yin Li, and Mohit Gupta. Event neural net- works. InEuropean Conference on Computer Vision, pages 276–293. Springer, 2022. 2

    2022

  12. [12]

    Lasot: A high-quality benchmark for large-scale single ob- ject tracking

    Heng Fan, Liting Lin, Fan Yang, Peng Chu, Ge Deng, Sijia Yu, Hexin Bai, Yong Xu, Chunyuan Liao, and Haibin Ling. Lasot: A high-quality benchmark for large-scale single ob- ject tracking. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5374–5383,

  13. [13]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 18995–19012, 2022. 2, 5

    2022

  14. [14]

    Skip-convolutions for efficient video processing

    Amirhossein Habibian, Davide Abati, Taco S Cohen, and Babak Ehteshami Bejnordi. Skip-convolutions for efficient video processing. InProceedings of the IEEE/CVF Con- ference on computer vision and pattern recognition, pages 2695–2704, 2021. 2

    2021

  15. [15]

    Delta distillation for ef- ficient video processing

    Amirhossein Habibian, Haitam Ben Yahia, Davide Abati, Ef- stratios Gavves, and Fatih Porikli. Delta distillation for ef- ficient video processing. InEuropean Conference on Com- puter Vision, pages 213–229. Springer, 2022. 2

    2022

  16. [16]

    Lvos: A benchmark for long-term video object segmentation

    Lingyi Hong, Wenchao Chen, Zhongying Liu, Wei Zhang, Pinxue Guo, Zhaoyu Chen, and Wenqiang Zhang. Lvos: A benchmark for long-term video object segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 13480–13492, 2023. 2

    2023

  17. [17]

    Videomatch: Matching based video object segmentation

    Yuan-Ting Hu, Jia-Bin Huang, and Alexander G Schwing. Videomatch: Matching based video object segmentation. In Proceedings of the European conference on computer vision (ECCV), pages 54–70, 2018. 2

    2018

  18. [18]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. InProceedings of the IEEE/CVF international confer- ence on computer vision, pages 4015–4026, 2023. 2

    2023

  19. [19]

    Scsampler: Sampling salient clips from video for efficient action recog- nition

    Bruno Korbar, Du Tran, and Lorenzo Torresani. Scsampler: Sampling salient clips from video for efficient action recog- nition. InProceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 6232–6242, 2019. 2

    2019

  20. [20]

    Blade: Learning compositional behaviors from demonstration and language

    Weiyu Liu, Neil Nie, Ruohan Zhang, Jiayuan Mao, and Ji- ajun Wu. Blade: Learning compositional behaviors from demonstration and language. InConference on Robot Learn- ing (CoRL), 2024. 1

    2024

  21. [21]

    Mash, spread, slice! learning to manipulate object states via visual spatial progress.arXiv preprint arXiv:2509.24129,

    Priyanka Mandikal, Jiaheng Hu, Shivin Dass, Sagnik Ma- jumder, Roberto Mart ´ın-Mart´ın, and Kristen Grauman. Mash, spread, slice! learning to manipulate object states via visual spatial progress.arXiv preprint arXiv:2509.24129,

    arXiv

  22. [22]

    Spoc: Spatially-progressing ob- ject state change segmentation in video, 2025

    Priyanka Mandikal, Tushar Nagarajan, Alex Stoken, Zihui Xue, and Kristen Grauman. Spoc: Spatially-progressing ob- ject state change segmentation in video, 2025. 2

    2025

  23. [23]

    Video object segmentation without temporal information.IEEE transactions on pattern analysis and machine intelligence, 41(6):1515–1530, 2018

    K-K Maninis, Sergi Caelles, Yuhua Chen, Jordi Pont-Tuset, Laura Leal-Taix´e, Daniel Cremers, and Luc Van Gool. Video object segmentation without temporal information.IEEE transactions on pattern analysis and machine intelligence, 41(6):1515–1530, 2018. 2

    2018

  24. [24]

    A literature review of computer vision techniques in wildlife monitoring.IJSRP, 16:282–295, 2022

    Sangam B Neupane, Kazuhiko Sato, and Bishnu P Gautam. A literature review of computer vision techniques in wildlife monitoring.IJSRP, 16:282–295, 2022. 1 9

    2022

  25. [25]

    Fast video object segmentation by reference- guided mask propagation

    Seoung Wug Oh, Joon-Young Lee, Kalyan Sunkavalli, and Seon Joo Kim. Fast video object segmentation by reference- guided mask propagation. InProceedings of the IEEE con- ference on computer vision and pattern recognition, pages 7376–7385, 2018. 2

    2018

  26. [26]

    Video object segmentation using space-time memory networks

    Seoung Wug Oh, Joon-Young Lee, Ning Xu, and Seon Joo Kim. Video object segmentation using space-time memory networks. InProceedings of the IEEE/CVF international conference on computer vision, pages 9226–9235, 2019. 2

    2019

  27. [27]

    A benchmark dataset and evaluation methodology for video object segmentation

    Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus Gross, and Alexander Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 724–732,

  28. [28]

    The 2017 davis challenge on video object segmentation.arXiv preprint arXiv:1704.00675, 2017

    Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Ar- bel´aez, Alex Sorkine-Hornung, and Luc Van Gool. The 2017 davis challenge on video object segmentation.arXiv preprint arXiv:1704.00675, 2017. 2, 5

    Pith/arXiv arXiv 2017

  29. [29]

    High-quality entity segmentation.arXiv preprint arXiv:2211.05776, 2022

    Lu Qi, Jason Kuen, Weidong Guo, Tiancheng Shen, Jiuxiang Gu, Jiaya Jia, Zhe Lin, and Ming-Hsuan Yang. High-quality entity segmentation.arXiv preprint arXiv:2211.05776, 2022. 3, 4, 6

    Pith/arXiv arXiv 2022

  30. [30]

    Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024. 1, 2, 3, 4, 6, 7, 8

    Pith/arXiv arXiv 2024

  31. [31]

    The visual object track- ing vot2016 challenge results

    Giorgio Roffo, Simone Melzi, et al. The visual object track- ing vot2016 challenge results. InComputer Vision–ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8- 10 and 15-16, 2016, Proceedings, Part II, pages 777–823. Springer International Publishing, 2016. 2

    2016

  32. [32]

    Token turing machines

    Michael S Ryoo, Keerthana Gopalakrishnan, Kumara Kahat- apitiya, Ted Xiao, Kanishka Rao, Austin Stone, Yao Lu, Ju- lian Ibarz, and Anurag Arnab. Token turing machines. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 19070–19081, 2023. 2

    2023

  33. [33]

    xgen-mm-vid (blip-3-video): You only need 32 tokens to represent a video even in vlms

    Michael S Ryoo, Honglu Zhou, Shrikant Kendre, Can Qin, Le Xue, Manli Shu, Jongwoo Park, Kanchana Ranasinghe, Silvio Savarese, Ran Xu, et al. xgen-mm-vid (blip-3-video): You only need 32 tokens to represent a video even in vlms. arXiv preprint arXiv:2410.16267, 2024. 2

    Pith/arXiv arXiv 2024

  34. [34]

    Tracking and understanding object transformations

    Yihong Sun, Xinyu Yang, Jennifer J Sun, and Bharath Har- iharan. Tracking and understanding object transformations. arXiv preprint arXiv:2511.04678, 2025. 1, 2, 3, 4, 5, 6, 7, 8

    arXiv 2025

  35. [35]

    Breaking the” object” in video object segmentation

    Pavel Tokmakov, Jie Li, and Adrien Gaidon. Breaking the” object” in video object segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22836–22845, 2023. 2, 4, 5, 7

    2023

  36. [36]

    A distractor-aware memory for visual object tracking with sam2

    Jovana Videnovic, Alan Lukezic, and Matej Kristan. A distractor-aware memory for visual object tracking with sam2. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 24255–24264, 2025. 2, 7

    2025

  37. [37]

    Feelvos: Fast end-to-end embedding learning for video object seg- mentation

    Paul V oigtlaender, Yuning Chai, Florian Schroff, Hartwig Adam, Bastian Leibe, and Liang-Chieh Chen. Feelvos: Fast end-to-end embedding learning for video object seg- mentation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9481–9490,

  38. [38]

    Adaptive focus for efficient video recognition

    Yulin Wang, Zhaoxi Chen, Haojun Jiang, Shiji Song, Yizeng Han, and Gao Huang. Adaptive focus for efficient video recognition. Inproceedings of the IEEE/CVF international conference on computer vision, pages 16249–16258, 2021. 2

    2021

  39. [39]

    Tracking transforming objects: A benchmark

    You Wu, Yuelong Wang, Yaxin Liao, Fuliang Wu, Hengzhou Ye, and Shuiwang Li. Tracking transforming objects: A benchmark. InChinese Conference on Pattern Recogni- tion and Computer Vision (PRCV), pages 222–236. Springer,

  40. [40]

    Adaframe: Adaptive frame selection for fast video recognition

    Zuxuan Wu, Caiming Xiong, Chih-Yao Ma, Richard Socher, and Larry S Davis. Adaframe: Adaptive frame selection for fast video recognition. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1278–1287, 2019. 2

    2019

  41. [41]

    Efficient track anything

    Yunyang Xiong, Chong Zhou, Xiaoyu Xiang, Lemeng Wu, Chenchen Zhu, Zechun Liu, Saksham Suri, Balakrishnan Varadarajan, Ramya Akula, Forrest Iandola, et al. Efficient track anything. InProceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 11513–11524,

  42. [42]

    Youtube-vos: Sequence-to-sequence video object segmentation

    Ning Xu, Linjie Yang, Yuchen Fan, Jianchao Yang, Dingcheng Yue, Yuchen Liang, Brian Price, Scott Cohen, and Thomas Huang. Youtube-vos: Sequence-to-sequence video object segmentation. InProceedings of the Euro- pean conference on computer vision (ECCV), pages 585– 601, 2018. 2

    2018

  43. [43]

    Learn- ing object state changes in videos: An open-world perspec- tive, 2024

    Zihui Xue, Kumar Ashutosh, and Kristen Grauman. Learn- ing object state changes in videos: An open-world perspec- tive, 2024. 2

    2024

  44. [44]

    Samurai: Adapting segment anything model for zero-shot visual tracking with motion-aware memory.arXiv preprint arXiv:2411.11922,

    Cheng-Yen Yang, Hsiang-Wei Huang, Wenhao Chai, Zhongyu Jiang, and Jenq-Neng Hwang. Samurai: Adapting segment anything model for zero-shot visual tracking with motion-aware memory.arXiv preprint arXiv:2411.11922,

    Pith/arXiv arXiv

  45. [45]

    Efficient video object seg- mentation via network modulation

    Linjie Yang, Yanran Wang, Xuehan Xiong, Jianchao Yang, and Aggelos K Katsaggelos. Efficient video object seg- mentation via network modulation. InProceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 6499–6507, 2018. 2

    2018

  46. [46]

    Collaborative video object segmentation by foreground-background inte- gration

    Zongxin Yang, Yunchao Wei, and Yi Yang. Collaborative video object segmentation by foreground-background inte- gration. InEuropean Conference on Computer Vision, pages 332–348. Springer, 2020. 2

    2020

  47. [47]

    Associating ob- jects with transformers for video object segmentation.Ad- vances in Neural Information Processing Systems, 34:2491– 2502, 2021

    Zongxin Yang, Yunchao Wei, and Yi Yang. Associating ob- jects with transformers for video object segmentation.Ad- vances in Neural Information Processing Systems, 34:2491– 2502, 2021. 2

    2021

  48. [48]

    Video state-changing object segmentation

    Jiangwei Yu, Xiang Li, Xinran Zhao, Hongming Zhang, and Yu-Xiong Wang. Video state-changing object segmentation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 20439–20448, 2023. 2, 5, 7

    2023

  49. [49]

    Convolutions die hard: Open-vocabulary seg- mentation with single frozen convolutional clip.Advances in Neural Information Processing Systems, 36:32215–32234,

    Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, and Liang- Chieh Chen. Convolutions die hard: Open-vocabulary seg- mentation with single frozen convolutional clip.Advances in Neural Information Processing Systems, 36:32215–32234,

  50. [50]

    Trajtok: Learning trajectory to- kens enables better video understanding.arXiv preprint arXiv:2602.22779, 2026

    Chenhao Zheng, Jieyu Zhang, Jianing Zhang, Weikai Huang, Ashutosh Kumar, Quan Kong, Oncel Tuzel, Chun-Liang Li, and Ranjay Krishna. Trajtok: Learning trajectory to- kens enables better video understanding.arXiv preprint arXiv:2602.22779, 2026. 1, 2

    Pith/arXiv arXiv 2026

  51. [51]

    Edgetam: On-device track anything model

    Chong Zhou, Chenchen Zhu, Yunyang Xiong, Saksham Suri, Fanyi Xiao, Lemeng Wu, Raghuraman Krishnamoorthi, Bo Dai, Chen Change Loy, Vikas Chandra, et al. Edgetam: On-device track anything model. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 13832–13842, 2025. 3 11

    2025

This paper was first reviewed by deepseek-v4-flash on August 1, 2026.