Pith. sign in

REVIEW 4 major objections 4 minor 121 references

SAM2Auto: Auto Annotation Using FLASH

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SAM2Auto claims that open-vocabulary detection combined with memory-based mask propagation can annotate entire video datasets with zero human labels and no dataset-specific training.

desk verdict The paper overclaims 'no human intervention'—its own Algorithm 1 needs ground-truth labels—and the results don't support the headline, but the integration and bottleneck analysis deserve a serious referee. read the letter →

arxiv 2506.07850 v1 pith:VXNRBCWT submitted 2025-06-09 cs.CV

Arash Rocky , Q.M. Jonathan Wu This is my paper
classification cs.CV
keywords autoannotationvideoinstancesegmentationmulti-objecttrackingopen-vocabularydetectionSAM2YOLO-Worldvision-languagemodelsdatasetcreation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SAM2Auto claims to be the first fully automated annotation pipeline for video datasets that needs no human intervention and no dataset-specific training. The paper argues this removes the main bottleneck holding back vision-language models: the scarcity of cheap, large-scale annotated video. It combines an open-vocabulary detector with a memory-based segmentation tracker so that, once an object is spotted, it keeps its identity through the whole sequence. The reported experiments indicate that the pipeline produces useful instance-level annotations at a fraction of manual cost, while the authors acknowledge that detection quality, especially false-positive suppression, remains the limiting factor.

What carries the argument

The load-bearing mechanism is memory-based mask propagation: SAM2 stores object memories so that a mask seeded once by a bounding box continues to track that object across frames. FLASH converts this single-object video segmentation capability into multi-object video instance segmentation by feeding it bounding boxes from ByteTrack's online association module, which assigns consistent IDs, and by adding checkpointing, adaptive chunking, empty-mask removal, temporal smoothing, and overlap-region merging to handle long sequences. SMART-OD supplies the seeds by running SAM2 auto-mask generation, YOLO-World open-vocabulary detection, and SAHI-based verification with dynamic thresholding.

What would settle it

Run SAM2Auto on a long video in which two visually identical objects occlude each other and compare the output masks to manual identity labels; if the two tracks' identities swap after the occlusion, the claimed consistency of object IDs across frames is falsified.

Watch

Extended reading notes

Core claim

The paper's central discovery is that open-vocabulary detection and memory-based video segmentation can be joined into a zero-annotation annotation engine: SMART-OD supplies high-precision seed detections, and FLASH, through its SAM2ASH component, propagates those seeds' masks forward in time while preserving object IDs across detection gaps. On MOT17, MOT20, DanceTrack, and BDD100K, SAM2Auto reaches HOTA values of 40.5, 32.3, 37.4, and 56.6 with zero manual labels, whereas FLASH with curated detections reaches 43.4, 38.6, 62.0, and 58.8, showing that the tracking backbone is strong enough to outperform trained baselines on DanceTrack. The paper isolates detection quality, rather than tracking, as the main gap between fully automated annotation and semi-supervised methods.

Load-bearing premise

The framework assumes that parameters tuned on the single most crowded frame and checked on one random sequence, with quality assurance that uses manual annotations, transfer to the whole dataset without violating the claim of no human intervention or dataset-specific training.

Editorial extensions

If this is right

  • If the pipeline works as claimed, video datasets can be annotated without hiring annotators, removing a major labor-cost barrier to VLM training data.
  • Because each processed video yields annotations for every frame, the pipeline also covers single-image annotation for any frame of a video.
  • FLASH with curated detections can serve as a standalone multi-object tracker, with its top IDF1 score on DanceTrack indicating strong identity preservation.
  • Quality assurance can be limited to a small stratified sample of sequences, keeping human effort near zero rather than per-frame.
  • The paper's own analysis implies that improving false-positive suppression in the detector, not tracking, is the next lever for closing the gap to semi-supervised annotation methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not develop is applying SAM2Auto to unlabeled web video collections to generate training data for open-vocabulary video instance segmentation and VLM grounding.
  • If bidirectional tracking is added, the pipeline could annotate an object's full trajectory from a single detection, relaxing the current need to detect objects at scene entry.
  • The DanceTrack result suggests that for videos whose ground truth only marks visible objects, memory-based annotation may already rival fine-tuned trackers; a direct comparison on a large occlusion-light dataset would test this.
  • The paper notes that false-positive tracklets currently require manual removal, so automating that step is what would make the no-human-intervention claim fully operational at scale.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces SAM2Auto, an automatic video annotation pipeline that combines SMART-OD (SAM2 automatic mask generation, YOLO-World open-vocabulary detection, and SAHI verification) with FLASH, a memory-based multi-object tracker that propagates SAM2 masks over long sequences using ByteTrack association and chunked processing. The authors claim that SAM2Auto requires no human intervention or dataset-specific training, produces annotations comparable to manual ones, and works across MOT17, MOT20, DanceTrack, and BDD100K. The experimental sections compare SMART-OD and FLASH ablations, report detection precision/recall, and present SAM2Auto tracking metrics against SPAM and FLASH, with SAM2Auto generally trailing both.

Significance. If the paper's headline claim were true, a zero-cost, fully automatic video annotation pipeline would be a substantial contribution to VLM dataset creation. The paper's strengths include a modular architecture that separates detection, association, and memory-based segmentation, plus a chunking mechanism for long videos and an extensive benchmark evaluation. However, the central claim is contradicted by the method's own specification, and the quantitative evaluation contains internal inconsistencies. Consequently, the contribution as stated is not established, although the underlying components may be worth further study under a revised, semi-automatic framing.

major comments (4)
  1. [Section III-D, Algorithm 1; Eqs. (31)-(32), (36)] Algorithm 1 requires ground-truth annotations and manual QA, contradicting the abstract's claim of 'no human intervention or dataset-specific training.' Step 3 optimizes Θ by maximizing J(Θ)=α·Recall(Θ)+(1−α)·Precision(Θ) (Eq. 31), where recall and precision (Eq. 32) are computed from TP/FP/FN counts on the most crowded frame; these counts require ground-truth object boxes. Steps 11-15 compute IoU(V_i, M_i) (Eq. 36) against manual annotations M_i on a stratified sample, and sequences falling below τ_QA trigger parameter refinement. Moreover, Section V states that the pipeline 'requires manual removal of false positive tracklets.' Thus the central claim of fully automated annotation without human intervention is internally contradicted.
  2. [Section IV-D1/Appendix D, Table VI and Fig. 7] The detection-ablation numbers in Table VI and Fig. 7 are internally inconsistent. For SMART-OD, precision 0.728 with FP=2,516 implies TP≈6,734, while recall 0.224 with FN=24,206 implies TP≈6,987; these TP estimates do not agree. The same inconsistency appears for YOLO-World (TP≈8,575 from precision vs ≈11,897 from recall) and SAM2-YW (TP≈9,683 vs ≈11,890). Since this ablation is the main evidence for SMART-OD's design, its experimental support is unreliable.
  3. [Section IV-D1, Table III] Table III reports impossible and mutually contradictory values. IDF1=1.156 exceeds the theoretical maximum of 1, and HOTA=0.406 is labeled 'New metric' rather than being a standard HOTA score. The table also claims improvements in Precision and Recall (0.728→0.828 and 0.224→0.341) while simultaneously reporting increases in False Positives (2,516→3,084) and False Negatives (24,206→31,770), and it lists these error increases in the 'Improvement' column with positive signs. These trends cannot hold simultaneously on the same ground-truth set, so the table's quantitative claims are not credible.
  4. [Abstract, Section IV-D3, Table V] The abstract's claim that SAM2Auto 'achieves comparable accuracy to manual annotation' is not tested. All evaluations compare against SPAM (which uses 3.3% manual labels) and FLASH with curated detections, not against a manual-annotation baseline. In Table V, SAM2Auto trails SPAM on MOT17 (HOTA 40.5 vs 51.6), MOT20 (32.3 vs 47.9), and DanceTrack (37.4 vs 39.5-41.0), and it trails FLASH on every dataset. Thus the headline claim is empirically unsupported.
minor comments (4)
  1. [Section IV-C1 and Fig. 6] The discussion of τ_merge references Figure 6, but the figure's caption reads 'Key detection metrics for YOLO-World, SAM2-YW, and SMART-OD on MOT17 training set,' which describes detection metrics rather than segment merging. The figure and caption should be reconciled.
  2. [Section III-B, Eq. (13)] Equation (13) averages polygons as α·P_i^t+(1−α)·P_i^{t−1}; since polygon vertex sets generally have different cardinalities and orderings across frames, this operation is not defined unless a vertex correspondence and interpolation scheme are specified.
  3. [Appendix D] Appendix D fixes many parameters (e.g., stability score threshold 0.90, YOLO confidence 0.001, verification IoU 0.03) but it is unclear how these relate to the parameter optimization in Algorithm 1, which claims to select Θ using the ground-truth-based objective J(Θ). The paper should explain which parameters are optimized and which are set a priori.
  4. [Section III-A, Eq. (12)] Equation (12) has a malformed second case: it should state φ(M_i^t)=∅ for t>τ(o_i), but the empty-set symbol is missing, leaving an unreadable expression.

Circularity Check

2 steps flagged · score 6.0 of 10

Algorithm 1 tunes SMART-OD thresholds on ground-truth labels (Eq. 31) and refines via manual-annotation IoU (step 12), so the 'no human intervention' claim is contradicted by the method itself.

  1. fitted input called prediction [Algorithm 1, lines 1-6; Appendix C.2-C.3, Eqs. (31)-(32)]
    "3:Θ_opt ←argmax_Θ J(Θ,f_crowd )▷Optimize parameters ... J(Θ)=α·Recall(Θ)+(1−α)·Precision(Θ) (31) ... Precision= T P / T P+F P , Recall= T P / T P+F N (32)"

    The optimized objective J(Θ) is defined through Precision and Recall, which Eq. (32) computes from TP/FP/FN. Those counts exist only relative to ground-truth boxes for the selected crowded frame. Hence the SMART-OD thresholds behind every reported SAM2Auto result are selected on the target dataset's manual labels, while the abstract and contribution 1 call the pipeline fully automated and free of human intervention and dataset-specific training. The automatic detection output is therefore a fitted quantity, not a zero-shot prediction.

  2. fitted input called prediction [Algorithm 1, lines 11-15; Appendix C.7, Eq. (36)]
    "11:Perform Quality Assurance on stratified sample of sequences 12:Q_i ←IoU(V_i ,M_i ) for sampledS_i 13:foreachS_i whereQ_i <τ_Q A do 14:Refine parameters and reprocessS_i 15:end for ... we compute the Intersection over Union (IoU) between auto-generated annotations and a small set of manual annotations"

    M_i is the manual-annotation set described in Appendix C.7. The QA loop uses IoU against those manual labels to decide which sequences fail, then refines parameters and reprocesses S_i until the threshold is passed. Human labels therefore enter the deployment pipeline as an optimization target; the final annotations on sampled sequences are constructed to agree with manual annotations, so the headline claim of zero human intervention and the comparable-accuracy result are built in for those sequences rather than independently predicted.

full rationale

The circularity is concentrated in Algorithm 1's use of labeled data. Step 3 optimizes Θ with J(Θ)=α·Recall+(1−α)·Precision, and Appendix C defines Precision/Recall via TP/FP/FN, which are only computable against ground-truth boxes for the selected crowded frame. Step 4 then evaluates Precision and Recall on the representative sequence, and step 6 cross-validates against another sequence with the same ground-truth-derived quantities. This is not peripheral: the reported SMART-OD precision of 72.8% and all SAM2Auto tracking numbers are produced with thresholds selected on target-dataset labels. The QA loop (steps 11-15) then uses IoU against manual annotations M_i to refine parameters and reprocess failing sequences, so the final labels on the sampled sequences are fitted to human labels. These two steps make the no-human-intervention or dataset-specific-training claim false by the paper's own equations, and they mean the headline automatic outputs are in-sample fits rather than independent predictions. I do not see load-bearing self-citation: SAM2, YOLO-World, and SAHI are external components, and the authors' prior accident-detection review [13] is not used to justify the pipeline. Section V further concedes that the pipeline requires manual removal of false positive tracklets, an approach that cannot scale, which supports the verdict. Score 6: the central claim partially reduces to ground-truth parameter fitting, but the tracker and segmentation components still have independent content.

Assumptions & free parameters 9 free parameters · 4 assumptions · 0 invented entities

The central claims rest on a large number of hand-set thresholds, the pretrained behavior of four external models, and a deployment algorithm that uses ground truth to tune and refine parameters.

free parameters (9)
  • SAM2 auto-mask thresholds = stability 0.90, offset 0.7, box NMS 0.7
    Hand-set to balance mask recall and precision; controls which masks feed YOLO-World.
  • YOLO-World detection thresholds = confidence 0.001, IoU 0.1, NMS 0.1
    Hand-set; very low confidence keeps recall high before verification.
  • SAHI verification IoU threshold = 0.03
    Hand-set for the dual-criterion verification in Equation (5); affects which detections survive.
  • Area ratio bounds = 0.0008 to 0.20
    Hand-set filters for unreasonably sized boxes in Equation (23).
  • DBSCAN clustering parameters = epsilon=100, min_samples=1
    Hand-set for ROI construction; min_samples=1 effectively makes clustering trivial.
  • ByteTrack association thresholds = track threshold 0.6, match threshold 0.7, buffer 20
    Hand-set for online association; directly affects ID switches and track initialization.
  • ASH post-processing thresholds = tau_merge=0.3, alpha=0.2, epsilon=3
    Hand-set; Figure 6 shows tau_merge sensitivity, yet a single value is used across datasets.
  • Chunking parameters = chunk size 50, overlap 10, batch size 5
    Hand-set memory/speed trade-offs for long sequences.
  • Optimization objective weight alpha = not stated
    Equation (31) balances recall and precision in Algorithm 1's parameter search; the alpha value is never given.
assumptions (4)
  • domain assumption SAM2's pretrained checkpoints produce reliable masks and temporally consistent mask propagation across frames.
    The entire SMART-OD and ASH stages rely on SAM2 behavior that is not derived or independently verified in this paper.
  • domain assumption YOLO-World recognizes the target class prompts (person and vehicle classes) well enough in zero-shot mode.
    Detection quality is the acknowledged bottleneck, so this assumption is partially contradicted by the paper's own results.
  • domain assumption ByteTrack's IoU-based association preserves identity when detections are sparse and high-precision.
    FLASH's online association depends on this; the paper does not analyze its failure modes beyond SAM2's appearance confusion.
  • domain assumption The evaluation benchmarks' ground truth is complete and consistent, including annotations for occluded objects.
    The paper explains score gaps by claiming ground truth penalizes SAM2 for not labeling occluded objects, assuming the benchmark is correct.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SAM2Auto: Auto Annotation Using FLASH." pith.science (2026). https://pith.science/paper/VXNRBCWT

@misc{pith2026250607850,
  author       = {Pith},
  title        = {Pith review of: SAM2Auto: Auto Annotation Using FLASH},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VXNRBCWT}},
  note         = {Machine review of arXiv:2506.07850}
}
read the original abstract

Vision-Language Models (VLMs) lag behind Large Language Models due to the scarcity of annotated datasets, as creating paired visual-textual annotations is labor-intensive and expensive. To address this bottleneck, we introduce SAM2Auto, the first fully automated annotation pipeline for video datasets requiring no human intervention or dataset-specific training. Our approach consists of two key components: SMART-OD, a robust object detection system that combines automatic mask generation with open-world object detection capabilities, and FLASH (Frame-Level Annotation and Segmentation Handler), a multi-object real-time video instance segmentation (VIS) that maintains consistent object identification across video frames even with intermittent detection gaps. Unlike existing open-world detection methods that require frame-specific hyperparameter tuning and suffer from numerous false positives, our system employs statistical approaches to minimize detection errors while ensuring consistent object tracking throughout entire video sequences. Extensive experimental validation demonstrates that SAM2Auto achieves comparable accuracy to manual annotation while dramatically reducing annotation time and eliminating labor costs. The system successfully handles diverse datasets without requiring retraining or extensive parameter adjustments, making it a practical solution for large-scale dataset creation. Our work establishes a new baseline for automated video annotation and provides a pathway for accelerating VLM development by addressing the fundamental dataset bottleneck that has constrained progress in vision-language understanding.

Figures

Figures reproduced from arXiv: 2506.07850 by the authors.

Figure 1
Figure 1. An example of detailed annotation in an image[ [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. SMART-OD: Dataset-optimized pipeline for robust open [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. FLASH system architecture showing the three main modules: Initialization, Online Object Association, and Annotation and Segmentation [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: FLASH Integration in Sequence Manager: By leveraging Chunk Division, Inter-Chunk Consistency, and Chunk Sequence Checkpoints, FLASH [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Without Online Association module, when bounding boxes from standard MOT at occlusion times are directly fed to FLASH, artifact segments [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Key detection metrics for YOLO-World, SAM2-YW, and SMART-OD on MOT17 training set. SMART-OD achieves the highest precision (0.728) [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Key detection metrics for YOLO-World, SAM2-YW, and SMART-OD on MOT17 training set. SMART-OD achieves the highest precision (0.728) [PITH_FULL_IMAGE:figures/full_fig_p025_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

121 extracted references · 75 canonical work pages

  1. [1]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T . Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “ An image is worth 16x16 words: Transformers for image recognition at scale,” 2021

    2021

  2. [2]

    Language models are few-shot learners,

    T . B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P . Dhariwal, A. Neelakantan, P . Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert- Voss, G. Krueger, T . Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D....

    2020

  3. [3]

    Learning transferable visual models from natural language super- vision,

    A. Radford, J. W . Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P . Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language super- vision,” 2021

    2021

  4. [4]

    Llama: Open and efficient foundation language models,

    H. Touvron, T . Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T . Lacroix, B. Rozière, N. Goyal, E. Hambro, F . Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample, “Llama: Open and efficient foundation language models,” 2023

    2023

  5. [5]

    On the opportunities and risks of foundation models,

    R. Bommasaniet al., “On the opportunities and risks of foundation models,” 2022

    2022

  6. [6]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal,et al., “Gpt-4 technical report,” 2024

    2024

  7. [7]

    Coca: Contrastive captioners are image-text foundation models,

    J. Yu, Z. Wang, V . Vasudevan, L. Yeung, M. Seyedhosseini, and Y. Wu, “Coca: Contrastive captioners are image-text foundation models,” 2022

    2022

  8. [8]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “ Attention is all you need,” 2023

    2023

Show all 121 references
  1. [9]

    Laion-400m: Open dataset of clip-filtered 400 million image-text pairs,

    C. Schuhmann, R. Vencu, R. Beaumont, R. Kaczmarczyk, C. Mullis, A. Katta, T . Coombes, J. Jitsev, and A. Komatsuzaki, “Laion-400m: Open dataset of clip-filtered 400 million image-text pairs,” 2021

    2021

  2. [10]

    Emerging properties in self-supervised vision trans- formers,

    M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P . Bojanowski, and A. Joulin, “Emerging properties in self-supervised vision trans- formers,” 2021

    2021

  3. [11]

    I was surprised by the great response to my recent post on small object detection—thank you for all the dms and shares

    E. Ramírez, “I was surprised by the great response to my recent post on small object detection—thank you for all the dms and shares.” LinkedIn post, August 2024. Accessed: August 29, 2024

    2024

  4. [12]

    Training data-efficient image transformers & distillation through attention,

    H. Touvron, M. Cord, M. Douze, F . Massa, A. Sablayrolles, and H. Jégou, “Training data-efficient image transformers & distillation through attention,” 2021

    2021

  5. [13]

    Review of accident detection methods using dashcam videos for autonomous driving vehicles,

    A. Rocky, Q. J. Wu, and W . Zhang, “Review of accident detection methods using dashcam videos for autonomous driving vehicles,” IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 8, pp. 8356–8374, 2024

    2024

  6. [14]

    The pascal visual object classes (voc) challenge,

    M. Everingham, L. V . Gool, C. K. I. Williams, J. Winn, and A. Zisser- man, “The pascal visual object classes (voc) challenge,”International Journal of Computer Vision, vol. 88, no. 2, pp. 303–338, 2010

    2010

  7. [15]

    Fast r-cnn,

    R. Girshick, “Fast r-cnn,” 2015

    2015

  8. [16]

    Mobilenets: Efficient convolutional neural networks for mobile vision applications,

    A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W . Wang, T . Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” 2017

    2017

  9. [17]

    Vision meets robotics: The kitti dataset,

    A. Geiger, P . Lenz, C. Stiller, and R. Urtasun, “Vision meets robotics: The kitti dataset,”The International Journal of Robotics Research, vol. 32, no. 11, pp. 1231–1237, 2013

    2013

  10. [18]

    Masked generative distillation,

    Z. Yang, Z. Li, M. Shao, D. Shi, Z. Yuan, and C. Yuan, “Masked generative distillation,” 2022

    2022

  11. [19]

    How to train your vit? data, augmentation, and regular- ization in vision transformers,

    A. Steiner, A. Kolesnikov, X. Zhai, R. Wightman, J. Uszkoreit, and L. Beyer, “How to train your vit? data, augmentation, and regular- ization in vision transformers,” 2022

    2022

  12. [20]

    Afreeca: Annotation-free counting for all,

    A. D’Alessandro, A. Mahdavi-Amiri, and G. Hamarneh, “ Afreeca: Annotation-free counting for all,” 2024

    2024

  13. [21]

    Open-vocabulary point-cloud object detection without 3d annota- tion,

    Y. Lu, C. Xu, X. Wei, X. Xie, M. Tomizuka, K. Keutzer, and S. Zhang, “Open-vocabulary point-cloud object detection without 3d annota- tion,” 2023

    2023

  14. [22]

    Emernerf: Emergent spatial- temporal scene decomposition via self-supervision,

    J. Yang, B. Ivanovic, O. Litany, X. Weng, S. W . Kim, B. Li, T . Che, D. Xu, S. Fidler, M. Pavone, and Y. Wang, “Emernerf: Emergent spatial- temporal scene decomposition via self-supervision,” 2023

    2023

  15. [23]

    Stereo4d: Learning how things move in 3d from internet stereo videos,

    L. Jin, R. Tucker, Z. Li, D. Fouhey, N. Snavely, and A. Holynski, “Stereo4d: Learning how things move in 3d from internet stereo videos,” 2024

    2024

  16. [24]

    Open world object detection in the era of foundation models,

    O. Zohar, A. Lozano, S. Goel, S. Yeung, and K.-C. Wang, “Open world object detection in the era of foundation models,” inarXiv preprint arXiv:2312.05745, 2023

    2023 arXiv

  17. [25]

    Yolo-world: Real-time open-vocabulary object detection,

    T . Cheng, L. Song, Y. Ge, W . Liu, X. Wang, and Y. Shan, “Yolo-world: Real-time open-vocabulary object detection,” 2024

    2024

  18. [26]

    Sapiens: Foundation for human vision models,

    R. Khirodkar, T . Bagautdinov, J. Martinez, S. Zhaoen, A. James, P . Selednik, S. Anderson, and S. Saito, “Sapiens: Foundation for human vision models,” 2024

    2024

  19. [27]

    Detrs beat yolos on real-time object detection,

    Y. Zhao, W . Lv, S. Xu, J. Wei, G. Wang, Q. Dang, Y. Liu, and J. Chen, “Detrs beat yolos on real-time object detection,” 2024

    2024

  20. [28]

    Dino-x: A unified vision model for open-world object detection and understanding,

    T . Ren, Y. Chen, Q. Jiang, Z. Zeng, Y. Xiong, W . Liu, Z. Ma, J. Shen, Y. Gao, X. Jiang, X. Chen, Z. Song, Y. Zhang, H. Huang, H. Gao, S. Liu, H. Zhang, F . Li, K. Yu, and L. Zhang, “Dino-x: A unified vision model for open-world object detection and understanding,” 2024

    2024

  21. [29]

    Prompt-guided detr with roi-pruned masked attention for open-vocabulary object detection,

    H. Song and J. Bang, “Prompt-guided detr with roi-pruned masked attention for open-vocabulary object detection,”Pattern Recogni- tion, vol. 155, p. 110648, 2024

    2024

  22. [30]

    Detclip: Dictionary-enriched visual-concept paralleled pre- training for open-world detection,

    L. Yao, J. Han, Y. Wen, X. Liang, D. Xu, W . Zhang, Z. Li, C. Xu, and H. Xu, “Detclip: Dictionary-enriched visual-concept paralleled pre- training for open-world detection,” 2022

    2022

  23. [31]

    Aligning bag of regions for open-vocabulary object detection,

    S. Wu, W . Zhang, S. Jin, W . Liu, and C. C. Loy, “ Aligning bag of regions for open-vocabulary object detection,” 2023

    2023

  24. [32]

    Detclipv2: Scalable open-vocabulary object detection pre-training via word- region alignment,

    L. Yao, J. Han, X. Liang, D. Xu, W . Zhang, Z. Li, and H. Xu, “Detclipv2: Scalable open-vocabulary object detection pre-training via word- region alignment,” 2023

    2023

  25. [33]

    Fmgs: Founda- tion model embedded 3d gaussian splatting for holistic 3d scene understanding,

    X. Zuo, P . Samangouei, Y. Zhou, Y. Di, and M. Li, “Fmgs: Founda- tion model embedded 3d gaussian splatting for holistic 3d scene understanding,” 2024

    2024

  26. [34]

    General object foundation model for images and videos at scale,

    J. Wu, Y. Jiang, Q. Liu, Z. Yuan, X. Bai, and S. Bai, “General object foundation model for images and videos at scale,” 2023

    2023

  27. [35]

    Region-aware pretraining for open-vocabulary object detection with vision transformers,

    D. Kim, A. Angelova, and W . Kuo, “Region-aware pretraining for open-vocabulary object detection with vision transformers,” 2023

    2023

  28. [36]

    Open- emma: Open-source multimodal model for end-to-end autonomous driving,

    S. Xing, C. Qian, Y. Wang, H. Hua, K. Tian, Y. Zhou, and Z. Tu, “Open- emma: Open-source multimodal model for end-to-end autonomous driving,” 2024

    2024

  29. [37]

    Simple open-vocabulary object detection with vision transformers,

    M. Minderer, A. Gritsenko, A. Stone, M. Neumann, D. Weissenborn, A. Dosovitskiy, A. Mahendran, A. Arnab, M. Dehghani, Z. Shen, X. Wang, X. Zhai, T . Kipf, and N. Houlsby, “Simple open-vocabulary object detection with vision transformers,” 2022

    2022

  30. [38]

    Exploiting unlabeled data with vision and language models for object detection,

    S. Zhao, Z. Zhang, S. Schulter, L. Zhao, V . K. B. G, A. Stathopoulos, M. Chandraker, and D. Metaxas, “Exploiting unlabeled data with vision and language models for object detection,” 2022

    2022

  31. [39]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection,

    S. Liu, Z. Zeng, T . Ren, F . Li, H. Zhang, J. Yang, Q. Jiang, C. Li, J. Yang, H. Su, J. Zhu, and L. Zhang, “Grounding dino: Marrying dino with grounded pre-training for open-set object detection,” 2024

    2024

  32. [40]

    Sam 2: Segment anything in images and videos,

    N. Ravi, V . Gabeur, Y.-T . Hu, R. Hu, C. Ryali, T . Ma, H. Khedr, R. Rädle, C. Rolland, L. Gustafson, E. Mintun, J. Pan, K. V . Alwala, N. Carion, C.-Y. Wu, R. Girshick, P . Dollár, and C. Feichtenhofer, “Sam 2: Segment anything in images and videos,”arXiv preprint arXiv:2408...

    2024 arXiv

  33. [41]

    Slicing aided hyper inference and fine-tuning for small object detection,

    F . C. Akyon, S. O. Altinuc, and A. Temizel, “Slicing aided hyper inference and fine-tuning for small object detection,”2022 IEEE International Conference on Image Processing (ICIP), pp. 966–970, 2022

    2022

  34. [42]

    Bdd100k: A diverse driving dataset for heterogeneous multitask learning,

    F . Yu, H. Chen, X. Wang, W . Xian, Y. Chen, F . Liu, V . Madhavan, and T . Darrell, “Bdd100k: A diverse driving dataset for heterogeneous multitask learning,” 2020. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, JAN 2025 18

    2020

  35. [43]

    nuscenes: A multimodal dataset for autonomous driving,

    H. Caesar, V . Bankiti, A. H. Lang, S. Vora, V . E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and O. Beijbom, “nuscenes: A multimodal dataset for autonomous driving,” 2020

    2020

  36. [44]

    Multiple adverse weather conditions adaptation for object detection via causal intervention,

    H. Zhang, L. Xiao, X. Cao, and H. Foroosh, “Multiple adverse weather conditions adaptation for object detection via causal intervention,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 3, pp. 1742–1756, 2024

    2024

  37. [45]

    Learning enriched features for real image restoration and enhancement,

    S. W . Zamir, A. Arora, S. Khan, M. Hayat, F . S. Khan, M.-H. Yang, and L. Shao, “Learning enriched features for real image restoration and enhancement,” 2020

    2020

  38. [46]

    You only need 90k parameters to adapt light: A light weight transformer for image enhancement and exposure correction,

    Z. Cui, K. Li, L. Gu, S. Su, P . Gao, Z. Jiang, Y. Qiao, and T . Harada, “You only need 90k parameters to adapt light: A light weight transformer for image enhancement and exposure correction,” 2022

    2022

  39. [47]

    Learning multi-scale photo exposure correction,

    M. Afifi, K. G. Derpanis, B. Ommer, and M. S. Brown, “Learning multi-scale photo exposure correction,” 2021

    2021

  40. [48]

    Unpaired image-to-image translation using cycle-consistent adversarial networks,

    J.-Y. Zhu, T . Park, P . Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” 2020

    2020

  41. [49]

    Track- former: Multi-object tracking with transformers,

    T . Meinhardt, A. Kirillov, L. Leal-Taixe, and C. Feichtenhofer, “Track- former: Multi-object tracking with transformers,” inThe IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR), June 2022

    2022

  42. [50]

    Boosttrack: boosting the similarity measure and detection confidence for improved multiple object tracking,

    V . D. Stanojevic and B. T . Todorovic, “Boosttrack: boosting the similarity measure and detection confidence for improved multiple object tracking,”Machine Vision and Applications, vol. 35, p. 53, April 2024

    2024

  43. [51]

    Bytetrack: Multi-object tracking by associating every de- tection box,

    Y. Zhang, P . Sun, Y. Jiang, D. Yu, F . Weng, Z. Yuan, P . Luo, W . Liu, and X. Wang, “Bytetrack: Multi-object tracking by associating every de- tection box,” inComputer Vision – ECCV 2022(S. Avidan, G. Brostow, M. Cissé, G. M. Farinella, and T . Hassner, eds.), (Cham), pp. 1–...

    2022

  44. [52]

    Diffmot: A real-time diffusion-based multiple object tracker with non-linear prediction,

    W . Lv, Y. Huang, N. Zhang, R.-S. Lin, M. Han, and D. Zeng, “Diffmot: A real-time diffusion-based multiple object tracker with non-linear prediction,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19321–19330, 2024

    2024

  45. [53]

    Simple cues lead to a strong multi-object tracker,

    J. Seidenschwarz, G. Brasó, V . C. Serrano, I. Elezi, and L. Leal-Taixé, “Simple cues lead to a strong multi-object tracker,” in2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13813–13823, 2023

    2023

  46. [54]

    Romot: Referring-expression-comprehension open-set multi-object track- ing,

    W . Li, B. Li, J. Wang, W . Meng, J. Zhang, and X. Zhang, “Romot: Referring-expression-comprehension open-set multi-object track- ing,”The Visual Computer, June 2024

    2024

  47. [55]

    Samba: Synchronized set-of-sequences modeling for multiple object tracking,

    M. Segu, L. Piccinelli, S. Li, Y.-H. Yang, B. Schiele, and L. V . Gool, “Samba: Synchronized set-of-sequences modeling for multiple object tracking,” 2024

    2024

  48. [56]

    Sparsetrack: Multi-object tracking by performing scene decomposition based on pseudo-depth,

    Z. Liu, X. Wang, C. Wang, W . Liu, and X. Bai, “Sparsetrack: Multi-object tracking by performing scene decomposition based on pseudo-depth,”IEEE Transactions on Circuits and Systems for Video Technology, 2025

    2025

  49. [57]

    Unifying short and long- term tracking with graph hierarchies,

    O. Cetintas, G. Brasó, and L. Leal-Taixé, “Unifying short and long- term tracking with graph hierarchies,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 22877–22887, June 2023

    2023

  50. [58]

    Matching anything by segmenting anything,

    S. Li, L. Ke, M. Danelljan, L. Piccinelli, M. Segu, L. Van Gool, and F . Yu, “Matching anything by segmenting anything,”CVPR, 2024

    2024

  51. [59]

    Joint modeling of feature, correspondence, and a compressed memory for video object seg- mentation,

    J. Zhang, Y. Cui, G. Wu, and L. Wang, “Joint modeling of feature, correspondence, and a compressed memory for video object seg- mentation,” 2023

    2023

  52. [60]

    Putting the object back into video object segmentation,

    H. K. Cheng, S. W . Oh, B. Price, J.-Y. Lee, and A. Schwing, “Putting the object back into video object segmentation,” 2024

    2024

  53. [61]

    Tam-vt: Transformation- aware multi-scale video transformer for segmentation and tracking,

    R. Goyal, W .-C. Fan, M. Siam, and L. Sigal, “Tam-vt: Transformation- aware multi-scale video transformer for segmentation and tracking,” 2024

    2024

  54. [62]

    Videoclick: Video object segmentation with a single click,

    N. Homayounfar, J. Liang, W .-C. Ma, and R. Urtasun, “Videoclick: Video object segmentation with a single click,” 2021

    2021

  55. [63]

    Tracking anything with decoupled video segmentation,

    H. K. Cheng, S. W . Oh, B. Price, A. Schwing, and J.-Y. Lee, “Tracking anything with decoupled video segmentation,” inICCV, 2023

    2023

  56. [64]

    Segment anything meets point tracking,

    F . Rajiˇ c, L. Ke, Y.-W . Tai, C.-K. Tang, M. Danelljan, and F . Yu, “Segment anything meets point tracking,” 2023

    2023

  57. [65]

    Occluded video instance segmentation: A benchmark,

    J. Qi, Y. Gao, Y. Hu, X. Wang, X. Liu, X. Bai, S. Belongie, A. Yuille, P . H. S. Torr, and S. Bai, “Occluded video instance segmentation: A benchmark,” 2022

    2022

  58. [66]

    Towards open-vocabulary video instance segmentation,

    H. Wang, C. Yan, S. Wang, X. Jiang, X. Tang, Y. Hu, W . Xie, and E. Gavves, “Towards open-vocabulary video instance segmentation,” 2023

    2023

  59. [67]

    Dynomo: Online point tracking by dynamic online monoc- ular gaussian reconstruction,

    J. Seidenschwarz, Q. Zhou, B. Duisterhof, D. Ramanan, and L. Leal- Taixé, “Dynomo: Online point tracking by dynamic online monoc- ular gaussian reconstruction,” inInternational Conference on 3D Vision, 2025

    2025

  60. [68]

    Spatialtracker: Tracking any 2d pixels in 3d space,

    Y. Xiao, Q. Wang, S. Zhang, N. Xue, S. Peng, Y. Shen, and X. Zhou, “Spatialtracker: Tracking any 2d pixels in 3d space,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR), 2024

    2024

  61. [69]

    Track4gen: Teaching video diffusion models to track points im- proves video generation,

    H. Jeong, C.-H. P . Huang, J. C. Ye, N. Mitra, and D. Ceylan, “Track4gen: Teaching video diffusion models to track points im- proves video generation,” 2024

    2024

  62. [70]

    Cotracker3: Simpler and better point tracking by pseudo- labelling real videos,

    N. Karaev, I. Makarov, J. Wang, N. Neverova, A. Vedaldi, and C. Rup- precht, “Cotracker3: Simpler and better point tracking by pseudo- labelling real videos,” inProc. arXiv:2410.11831, 2024

    2024 arXiv

  63. [71]

    Omnitracker: Unifying visual object tracking by tracking- with-detection,

    J. Wang, Z. Wu, D. Chen, C. Luo, X. Dai, L. Yuan, and Y.-G. Jiang, “Omnitracker: Unifying visual object tracking by tracking- with-detection,”IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–15, 2025

    2025

  64. [72]

    Spam- ming labels: Efficient annotations for the trackers of tomorrow,

    O. Cetintas, T . Meinhardt, G. Brasó, and L. Leal-Taixé, “Spam- ming labels: Efficient annotations for the trackers of tomorrow,” in European Conference on Computer Vision (ECCV), 2024

    2024

  65. [73]

    Better call sal: Towards learning to segment anything in lidar,

    A. Osep, T . Meinhardt, F . Ferroni, N. Peri, D. Ramanan, and L. Leal- Taixé, “Better call sal: Towards learning to segment anything in lidar,” inEuropean Conference on Computer Vision (ECCV), 2024

    2024

  66. [74]

    Efficient track anything,

    Y. Xiong, C. Zhou, X. Xiang, L. Wu,et al., “Efficient track anything,” preprint arXiv:2411.18933, 2024

    2024 arXiv

  67. [75]

    Ref- ereverything: Towards segmenting everything we can speak of in videos,

    A. Bagchi, Z. Bao, Y.-X. Wang, P . Tokmakov, and M. Hebert, “Ref- ereverything: Towards segmenting everything we can speak of in videos,” 2024

    2024

  68. [76]

    Samurai: Adapting segment anything model for zero-shot visual tracking with motion-aware memory,

    C.-Y. Yang, H.-W . Huang, W . Chai, Z. Jiang, and J.-N. Hwang, “Samurai: Adapting segment anything model for zero-shot visual tracking with motion-aware memory,” 2024

    2024

  69. [77]

    Smite: Segment me in time,

    A. Alimohammadi, S. Nag, S. A. Taghanaki, A. Tagliasacchi, G. Hamarneh, and A. M. Amiri, “Smite: Segment me in time,” 2024

    2024

  70. [78]

    Deep multiple instance learning for image classification and auto-annotation,

    J. Wu, Y. Yu, C. Huang, and K. Yu, “Deep multiple instance learning for image classification and auto-annotation,” in2015 IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR), pp. 3460– 3469, 2015

    2015

  71. [79]

    Not all labels are equal: Rationalizing the labeling costs for training object detection,

    I. Elezi, Z. Yu, A. Anandkumar, L. Leal-Taixe, and J. M. Alvarez, “Not all labels are equal: Rationalizing the labeling costs for training object detection,” 2021

    2021

  72. [80]

    Scaling open- vocabulary object detection,

    M. Minderer, A. Gritsenko, and N. Houlsby, “Scaling open- vocabulary object detection,” 2024

    2024

  73. [81]

    Detclipv3: Towards versatile generative open-vocabulary object detection,

    L. Yao, R. Pi, J. Han, X. Liang, H. Xu, W . Zhang, Z. Li, and D. Xu, “Detclipv3: Towards versatile generative open-vocabulary object detection,” 2024

    2024

  74. [82]

    Apovis: Automated pixel-level open-vocabulary instance segmenta- tion through integration of pre-trained vision-language models and foundational segmentation models,

    Q. Ma, S. Yang, L. Zhang, Q. Lan, D. Yang, H. Chen, and Y. Tan, “ Apovis: Automated pixel-level open-vocabulary instance segmenta- tion through integration of pre-trained vision-language models and foundational segmentation models,”Image and Vision Computing, vol. 154, p. 105384, 2025

    2025

  75. [83]

    Towards real-time open-vocabulary video instance segmentation,

    B. Yan, M. Sundermeyer, D. J. Tan, H. Lu, and F . Tombari, “Towards real-time open-vocabulary video instance segmentation,” 2024

    2024

  76. [84]

    Aide: An automatic data engine for object detection in autonomous driving,

    M. Liang, J.-C. Su, S. Schulter, S. Garg, S. Zhao, Y. Wu, and M. Chandraker, “ Aide: An automatic data engine for object detection in autonomous driving,” 2024

    2024

  77. [85]

    Automatic labeling of objects from lidar point clouds via trajectory-level re- finement,

    A. J. Yang, S. C. Romero, M. Dvornik, S. Segal,et al., “ Automatic labeling of objects from lidar point clouds via trajectory-level re- finement,” December 2024

    2024

  78. [86]

    Cosmos world foundation model platform for physical ai,

    N. Agarwalet al., “Cosmos world foundation model platform for physical ai,” 2025

    2025

  79. [87]

    Launch: Auto label images with roboflow,

    J. Witt, “Launch: Auto label images with roboflow,”Roboflow Blog, March 2024

    2024

  80. [88]

    Semi-supervised open-world object detection,

    S. S. Mullappilly, A. S. Gehlot, R. M. Anwer, F . Shahbaz Khan, and H. Cholakkal, “Semi-supervised open-world object detection,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, p. 4305–4314, Mar. 2024

    2024

  81. [89]

    A density-based algorithm for discovering clusters in large spatial databases with noise,

    M. Ester, H.-P . Kriegel, J. Sander, and X. Xu, “ A density-based algorithm for discovering clusters in large spatial databases with noise,” inProceedings of the Second International Conference on Knowledge Discovery and Data Mining, KDD’96, p. 226–231, AAAI Press, 1996

    1996

  82. [90]

    Mot16: A benchmark for multi-object tracking,

    A. Milan, L. Leal-Taixe, I. Reid, S. Roth, and K. Schindler, “Mot16: A benchmark for multi-object tracking,” 2016

    2016

  83. [91]

    Simple unsupervised multi- object tracking,

    S. Karthik, A. Prabhu, and V . Gandhi, “Simple unsupervised multi- object tracking,”CoRR, vol. abs/2006.02609, 2020

    2006 arXiv

  84. [92]

    Probabilistic tracklet scoring and inpainting for multiple object tracking,

    F . S. Saleh, S. Aliakbarian, H. Rezatofighi, M. Salzmann, and S. Gould, “Probabilistic tracklet scoring and inpainting for multiple object tracking,”CoRR, vol. abs/2012.02337, 2020. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, JAN 2025 19

    2012 arXiv

  85. [93]

    Tracking objects as points,

    X. Zhou, V . Koltun, and P . Krähenbühl, “Tracking objects as points,” 2020

    2020

  86. [94]

    Mot20: A benchmark for multi object tracking in crowded scenes,

    P . Dendorfer, H. Rezatofighi, A. Milan, J. Shi, D. Cremers, I. Reid, S. Roth, K. Schindler, and L. Leal-Taixé, “Mot20: A benchmark for multi object tracking in crowded scenes,” 2020

    2020

  87. [95]

    Dance- track: Multi-object tracking in uniform appearance and diverse motion,

    P . Sun, J. Cao, Y. Jiang, Z. Yuan, S. Bai, K. Kitani, and P . Luo, “Dance- track: Multi-object tracking in uniform appearance and diverse motion,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

    2022

  88. [96]

    Framework for performance evaluation of face, text, and vehicle detection and tracking in video: Data, metrics, and protocol,

    R. Kasturi, D. B. Goldgof, P . Soundararajan, V . Manohar, J. S. Garofolo, R. Bowers, M. Boonstra, V . N. Korzhova, and J. Zhang, “Framework for performance evaluation of face, text, and vehicle detection and tracking in video: Data, metrics, and protocol,”IEEE Trans. Pattern ...

    2009

  89. [97]

    Per- formance measures and a data set for multi-target, multi-camera tracking,

    E. Ristani, F . Solera, R. Zou, R. Cucchiara, and C. Tomasi, “Per- formance measures and a data set for multi-target, multi-camera tracking,” inComputer Vision – ECCV 2016 Workshops(G. Hua and H. Jégou, eds.), (Cham), pp. 17–35, Springer International Publishing, 2016

    2016

  90. [98]

    Hota: A higher order metric for evaluating multi- object tracking,

    J. Luiten, A. Ošep, P . Dendorfer, P . Torr, A. Geiger, L. Leal-Taixé, and B. Leibe, “Hota: A higher order metric for evaluating multi- object tracking,”International Journal of Computer Vision, vol. 129, p. 548–578, Oct. 2020

    2020

  91. [99]

    Tracking without bells and whistles,

    P . Bergmann, T . Meinhardt, and L. Leal-Taixé, “Tracking without bells and whistles,” inThe IEEE International Conference on Computer Vision (ICCV), October 2019

    2019

  92. [100]

    Learning a neural solver for multiple object tracking,

    G. Brasó and L. Leal-Taixé, “Learning a neural solver for multiple object tracking,” inThe IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2020

    2020

  93. [101]

    YOLOX: exceeding YOLO series in 2021,

    Z. Ge, S. Liu, F . Wang, Z. Li, and J. Sun, “YOLOX: exceeding YOLO series in 2021,”CoRR, vol. abs/2107.08430, 2021

    2021 arXiv

  94. [102]

    Discriminative appearance modeling with multi-track pooling for real-time multi- object tracking,

    C. Kim, L. Fuxin, M. Alotaibi, and J. M. Rehg, “Discriminative appearance modeling with multi-track pooling for real-time multi- object tracking,” in2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9548–9557, 2021

    2021

  95. [103]

    Self-supervised multi-object track- ing with cross-input consistency,

    F . Bastani, S. He, and S. Madden, “Self-supervised multi-object track- ing with cross-input consistency,” inAdvances in Neural Information Processing Systems(M. Ranzato, A. Beygelzimer, Y. Dauphin, P . Liang, and J. W . Vaughan, eds.), vol. 34, pp. 13695–13706, Curran Associa...

    2021

  96. [104]

    Simple online and realtime tracking,

    A. Bewley, Z. Ge, L. Ott, F . Ramos, and B. Upcroft, “Simple online and realtime tracking,” in2016 IEEE International Conference on Image Processing (ICIP), IEEE, Sept. 2016

    2016

  97. [105]

    Online multi-object visual tracking using a gm-phd filter with deep appearance learning,

    N. L. Baisa, “Online multi-object visual tracking using a gm-phd filter with deep appearance learning,” in2019 22th International Conference on Information Fusion (FUSION), pp. 1–8, 2019

    2019

  98. [106]

    Motiontrack: Learning robust short-term and long-term motions for multi-object tracking,

    Z. Qin, S. Zhou, L. Wang, J. Duan, G. Hua, and W . Tang, “Motiontrack: Learning robust short-term and long-term motions for multi-object tracking,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 17939–17948, June 2023

    2023

  99. [107]

    Utm: A unified multiple object tracking model with identity-aware feature enhancement,

    S. You, H. Yao, B.-k. Bao, and C. Xu, “Utm: A unified multiple object tracking model with identity-aware feature enhancement,” in2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 21876–21886, 2023

    2023

  100. [108]

    Quasi-dense similarity learning for multiple object tracking,

    J. Pang, L. Qiu, X. Li, H. Chen, Q. Li, T . Darrell, and F . Yu, “Quasi-dense similarity learning for multiple object tracking,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 164–173, June 2021

    2021

  101. [109]

    Motr: End-to-end multiple-object tracking with transformer,

    F . Zeng, B. Dong, Y. Zhang, T . Wang, X. Zhang, and Y. Wei, “Motr: End-to-end multiple-object tracking with transformer,” inComputer Vision – ECCV 2022(S. Avidan, G. Brostow, M. Cissé, G. M. Farinella, and T . Hassner, eds.), (Cham), pp. 659–675, Springer Nature Switzer- land, 2022

    2022

  102. [110]

    Fairmot: On the fairness of detection and re-identification in multiple object tracking,

    Y. Zhang, C. Wang, X. Wang, W . Zeng, and W . Liu, “Fairmot: On the fairness of detection and re-identification in multiple object tracking,”International Journal of Computer Vision, vol. 129, no. 11, pp. 3069–3087, 2021

    2021

  103. [111]

    Memot: Multi-object tracking with memory,

    J. Cai, M. Xu, W . Li, Y. Xiong, W . Xia, Z. Tu, and S. Soatto, “Memot: Multi-object tracking with memory,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8090–8100, June 2022

    2022

  104. [112]

    Tracking every thing in the wild,

    S. Li, M. Danelljan, H. Ding, T . E. Huang, and F . Yu, “Tracking every thing in the wild,” inComputer Vision – ECCV 2022(S. Avidan, G. Brostow, M. Cissé, G. M. Farinella, and T . Hassner, eds.), (Cham), pp. 498–515, Springer Nature Switzerland, 2022. JOURNAL OF LATEX CLASS FI...

    2022

  105. [113]

    Segmentation Stage:The first stage employs SAM2 for automatic mask generation across all potential objects in the scene. The segmentation process is defined as: IMasked =AutoMask SAM2(I,θ s,θ o,θ n) (21) whereI Masked represents the generated instance masks, AutoMaskSAM2 is th...

  106. [114]

    Mask-guided Detection Stage:The second stage per- forms open-vocabulary object detection using YOLO-World on the masked representations. The detection process is formulated as: Dinit =OVOD(I Masked,C,θ c ,θ i ,θ n) (22) whereD init represents the initial detections, OVOD is th...

  107. [115]

    Initialization Module:The module implements a ro- bust three-phasecheckpoint managementto prevent data loss during save operations:

  108. [116]

    Write to temporary file:f t emp ←serialize(X)

  109. [117]

    Create backup of existing checkpoint:f backup ← fcheckpoi nt (if exists)

  110. [118]

    initial" max(v f r ames ) if checkpoint is “final

    Promote temporary file to checkpoint:f checkpoi nt ← ft emp whereXrepresents the data being checkpointed (the cur- rent state of the system), serialize(X) converts these data into a format that can be saved on disk. This three-phase approach ensures that there is always at lea...

    2025

  111. [119]

    In the first frame, all valid detections are initialized as new objects requiring segmentation

    Online Object Association Module: Temporal Context Processing::The system processes detections differently based on their temporal context. In the first frame, all valid detections are initialized as new objects requiring segmentation. In subsequent frames, de- tections are fi...

  112. [120]

    worst-case

    Annotation and Segmentation Handler (ASH): Memory-Efficient Processing::For effective real-time performance, ASH implements two key memory optimiza- tion strategies: Subset Frame Processing:For a video sequence with framesF={F 1,F 2, ...,FT }, ASH employs a subset of frames Sa...

    2025

  113. [121]

    training sets. As shown in Figure 7 and Table VI, we compare the performance of (i) YOLO-World baseline, (ii) SAM2 combined with YOLO-World (SAM2-YW), and (iii) the full SMART-OD pipeline with SAHI verification. It is important to note that at this detection stage, objects are...

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.