Pith. sign in

REVIEW 1 major objections 7 minor 34 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

T0 review · glm-5.2

Perfect detections, broken identities: long video breaks MOT trackers

2026-07-10 02:06 UTC pith:KWMHAC2Y

load-bearing objection Solid long-duration MOT benchmark; framing slightly overstates what's proven the 1 major comments →

arxiv 2607.08729 v1 pith:KWMHAC2Y submitted 2026-07-09 cs.CV

WaspMOT: A Benchmark for Long-Term Multi-Object Tracking of Trichogramma Wasps

classification cs.CV
keywords benchmarktrackingwaspmotlong-termidentityperformanceassociationbenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The authors introduce WaspMOT, a benchmark of 10 long-duration video sequences (approximately 12,000 frames each, over 8 minutes at 25 FPS) tracking Trichogramma wasps in a closed-set scenario where all individuals remain present throughout the entire video. By providing oracle detections — ground-truth bounding boxes stripped of identity labels — the benchmark isolates the association problem from detection errors. Five standard tracking-by-detection methods (ByteTrack, BoT-SORT, C-BIoU, OC-SORT, and McByte) all exhibit substantial identity fragmentation despite perfect detections, with identity-consistency scores (IDF1) ranging from 45.4 to 51.1. A simple post-hoc spatial tracklet stitching baseline consistently improves IDF1 by 6.6 to 9.0 points across all methods, indicating that a significant portion of fragmentation is recoverable through spatial proximity alone. The paper argues that conventional MOT benchmarks, whose average sequence durations range from 25 to 100 seconds, are too short to expose this failure mode, and that the field needs evaluation protocols that test identity preservation over thousands of frames.

Core claim

When tracking-by-detection methods are evaluated on sequences long enough to require identity preservation over thousands of frames rather than tens of seconds, they all produce severely fragmented trajectories — even when detections are perfect. The failure is in association, not detection: trackers lose identities because they rely on local motion-continuity assumptions that break under abrupt movements, occlusions, and visual similarity. A simple post-hoc stitching of trajectory fragments by spatial proximity recovers substantial performance, demonstrating that much of the fragmentation is not due to genuine identity ambiguity but to the limitations of local, frame-to-frame association.

What carries the argument

The closed-set tracking scenario with full-length trajectories evaluated under oracle detections. In this regime, every individual is present from the first frame to the last, so the only task is to assign and maintain consistent identities across thousands of frames. This setup separates the association problem — which identity does each detection belong to — from the detection problem — where are the objects — and from the track lifecycle problem — when do tracks start and end.

Load-bearing premise

The paper assumes that the trajectory fragmentation observed when tracking small, visually identical insects that perform abrupt jumps in a three-dimensional arena is representative of a general long-term tracking problem, rather than being specific to this ecological setting. It does not directly demonstrate that extending a conventional pedestrian dataset to comparable duration would produce similar fragmentation.

What would settle it

Run a conventional pedestrian or sports tracking dataset extended to comparable duration (thousands of frames) with oracle detections. If existing trackers maintain high identity consistency on those extended sequences, then the fragmentation observed here is domain-specific rather than a general long-term tracking failure.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If long-duration tracking fundamentally breaks local association mechanisms, then the dominant paradigm of frame-to-frame association with Kalman filtering needs augmentation with global or long-horizon reasoning to maintain identity consistency.
  • The consistent success of simple spatial stitching suggests that trajectory fragmentation is often a recoverable error — identity information is present in the data but current methods fail to exploit it — pointing toward hierarchical or two-stage tracking architectures.
  • The finding that appearance-based methods do not outperform simpler motion-based methods on visually similar targets suggests appearance features have a domain of applicability that degrades when inter-object visual differences are minimal.
  • The closed-set, full-length trajectory evaluation regime could be applied to other domains to test whether identity preservation failures generalize beyond insect tracking.
  • The gap between oracle-detection performance and perfect identity preservation quantifies how much room remains for association algorithms specifically, independent of further detection improvements.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 7 minor

Summary. The paper introduces WaspMOT, a multi-object tracking benchmark consisting of 10 long-duration video sequences (~12,000 frames each) of Trichogramma wasps in a controlled laboratory arena. The dataset is annotated in MOTChallenge format and provides oracle detections (ground-truth bounding boxes without identity labels) to isolate association performance from detection errors. Five tracking-by-detection methods (ByteTrack, BoT-SORT, C-BIoU, OC-SORT, McByte) are evaluated under a unified protocol, and a simple spatial tracklet stitching baseline is applied as post-processing. The central finding is that all evaluated trackers exhibit significant trajectory fragmentation (low IDF1) despite perfect detections, and that simple spatial stitching recovers substantial performance, indicating that current association strategies struggle with long-term identity preservation.

Significance. The dataset fills a genuine gap: existing MOT benchmarks average 25–100 seconds per sequence, while WaspMOT sequences exceed 480 seconds with full-length closed-set trajectories. The oracle-detection protocol is a clean experimental design that isolates association from detection, and the use of standard TrackEval metrics ensures comparability. The finding that simple spatial stitching yields +7–9 IDF1 across all methods is a concrete, falsifiable result that motivates future work. The public release of data and code is a positive step for reproducibility.

major comments (1)
  1. The central claim that WaspMOT exposes a failure mode 'not observable on conventional datasets' (Abstract, §I, §VI) is not empirically demonstrated. The paper shows that trackers fragment on WaspMOT, but it never isolates sequence duration as the causal factor. The fragmentation could be driven by domain-specific challenges — abrupt jumps (§III.C, Figs. 2–3), 3D floor/ceiling occlusions, and near-identical appearance — rather than by temporal length per se. Notably, the stitching results (+7–9 IDF1 from spatial reconnection, Table II) suggest that fragmentation arises from local association failures (post-jump, post-occlusion) that accumulate over long sequences, not from gradual long-range identity drift. To support the 'long-term' framing, the authors should add at least one of the following controls: (1) an analysis of IDF1 (or fragmentation rate) as a function of sequence length *e.g
minor comments (7)
  1. §IV.B: The paper states that BoT-SORT's appearance embeddings are 'trained following the procedure described in the original work, adapted to WaspMOT.' Please specify the training data (which sequences, how many identity labels) and whether training and evaluation sequences overlap, as this affects the fairness of the appearance-based comparison.
  2. §IV.C: The spatial stitching baseline is described qualitatively but lacks key parameters: the spatial and temporal thresholds for candidate association, and whether these were tuned on the evaluation set. Please specify or state that defaults were used.
  3. Table I: The 'Avg boxes / video' entry for DanceTrack and BEE24 is listed as '-', and the formatting of the WaspMOT row (described as 'in green') is not visible in the text. Please ensure the table is self-contained in the text version.
  4. §III.A: The camera resolution is 3840×2160, but the arena is only 3.5cm × 2cm. It would help to state the approximate pixel size of a wasp bounding box to contextualize the 'small object' challenge.
  5. Fig. 2–3: These figures are referenced but their axes and content are only briefly described. Please ensure axis labels, units, and legends are clear in the final version.
  6. §II.A: The reference to BEE24 [11] describes it as providing 'limited' trajectory coverage. A quantitative comparison (e.g., average trajectory length as a fraction of sequence length) would strengthen the contrast.
  7. Typos: §I, 'ofTrichogrammawasps' (missing space); §III.A, 'ofTrichogrammawasps' (repeated); §III.B, '2,569,527 annotated object instances' — please verify this count is consistent with 'Avg boxes / video' in Table I (256,952.70 × 10 = 2,569,527, which checks out, but the table should note this is per-sequence average).

Circularity Check

0 steps flagged

No significant circularity; one minor non-load-bearing self-citation (McByte)

full rationale

This is an empirical benchmark paper, not a derivation chain. The central claim — that all five tracking-by-detection methods suffer trajectory fragmentation on long-duration sequences even with oracle detections — is tested by running external algorithms (ByteTrack, BoT-SORT, C-BIoU, OC-SORT, McByte) on the dataset and measuring standard metrics (HOTA, IDF1, MOTA) via the independent TrackEval toolkit. No parameter is fitted to data and then presented as a prediction. The oracle detections are derived from ground-truth bounding boxes without identity labels, which removes detection as a confound by construction — but this is a deliberate experimental design choice, not a circular derivation. The spatial stitching baseline is a post-processing heuristic whose improvements are empirically measured, not predicted from a fitted model. The only self-citation is McByte [10], co-authored by the paper's first author, but it is one of five evaluated methods and the central claim (all methods fragment) does not depend on McByte's results specifically. This self-citation is minor and non-load-bearing. The paper is self-contained against external benchmarks and standard evaluation protocols.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

No new entities are introduced. The dataset, benchmark protocol, and stitching baseline use existing concepts and tools.

axioms (4)
  • domain assumption Oracle detections derived from ground truth bounding boxes without identity labels provide perfect detection accuracy and isolate association performance.
    §IV.A: The paper assumes that providing ground-truth bounding boxes as detections removes detection as a confound. This is standard practice but assumes the bounding boxes themselves are noise-free, which may not hold if annotation quality varies.
  • domain assumption Default tracker parameters are appropriate for evaluating inherent association capabilities without unfair bias.
    §IV.B: All methods are applied with default parameters without tuning. This enables consistent comparison but assumes defaults are not systematically disadvantageous for the insect-tracking domain.
  • domain assumption The closed-set tracking scenario (all individuals present throughout) is a meaningful evaluation regime for long-term identity preservation.
    §I, §III: The paper frames closed-set tracking as a feature enabling evaluation of full-length trajectories. This is a valid design choice but differs from open-set scenarios where objects enter and exit, limiting generalizability claims.
  • standard math Standard MOT metrics (HOTA, IDF1, MOTA) computed via TrackEval are appropriate for evaluating long-term tracking.
    §V.A: The paper uses established metrics from the MOT literature. These metrics were designed for shorter sequences but are applied here to much longer ones without modification.

pith-pipeline@v1.1.0-glm · 11968 in / 3637 out tokens · 252332 ms · 2026-07-10T02:06:10.089815+00:00 · methodology

0 comments
read the original abstract

Multi-object tracking (MOT) has achieved strong performance on benchmarks dominated by short video sequences. However, such datasets do not adequately evaluate long-term identity preservation, where objects must be tracked consistently over extended durations. We introduce WaspMOT, a benchmark designed to address this gap through long-duration tracking of Trichogramma wasps in controlled ecological experiments. The dataset contains 10 sequences of approximately 12,000 frames each (over 8 minutes at 25 FPS), with dense MOTChallenge annotations and oracle detections to isolate association performance. Unlike existing benchmarks, WaspMOT forms a closed-set tracking scenario where all individuals remain present throughout the sequence, requiring consistent identity assignment across thousands of frames despite abrupt jumps, occlusions, and highly similar appearance. We establish a benchmark by evaluating five tracking-by-detection methods, including ByteTrack, BoT-SORT, C-BIoU, OC-SORT, and McByte, under a unified protocol. Results show that all methods suffer from significant trajectory fragmentation, highlighting the difficulty of long-term identity preservation even with perfect detections. A simple spatial tracklet stitching baseline consistently improves performance, indicating that substantial gains remain possible. WaspMOT provides a new benchmark for studying long-term association and reveals limitations of current tracking approaches that are not observable on conventional datasets. The benchmark will be made publicly available at the project repository: https://github.com/tstanczyk95/WaspMOT/ .

Figures

Figures reproduced from arXiv: 2607.08729 by Francois Bremond, Hardik Agarwal, Seongroo Yoon, Tiantao Zhang, Tomasz Stanczyk, Vincent Calcagno, Yuan Gao.

Figure 1
Figure 1. Figure 1: Sample frame of a video with individuals inside an experimental [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Number of occlusion and jump events for each video. Original video [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Number of co-occurring jump events for each video. Original video [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: An example of an individual Trichogramma wasp performing an abrupt jump. The temporary disappearance results in trajectory fragmentation, which is recovered by spatial tracklet stitching. All trackers exhibit substantial identity fragmentation, re￾flected in relatively low IDF1 scores despite the use of oracle detections. This confirms that long-term identity preservation remains challenging even when dete… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 34 canonical work pages · 4 internal anchors

  1. [1]

    MOT16: A Benchmark for Multi-Object Tracking

    A. Milan, L. Leal-Taix ´e, I. Reid, S. Roth, and K. Schindler, “MOT16: A benchmark for multi-object tracking,”arXiv:1603.00831 [cs], Mar. 2016, arXiv: 1603.00831. [Online]. Available: http: //arxiv.org/abs/1603.00831

  2. [2]

    MOT20: A benchmark for multi object tracking in crowded scenes

    P. Dendorfer, H. Rezatofighi, A. Milan, J. Shi, D. Cremers, I. Reid, S. Roth, K. Schindler, and L. Leal-Taix ´e, “Mot20: A benchmark for multi object tracking in crowded scenes,” 2020. [Online]. Available: https://arxiv.org/abs/2003.09003

  3. [3]

    Sportsmot: A large multi-object tracking dataset in multiple sports scenes,

    Y . Cui, C. Zeng, X. Zhao, Y . Yang, G. Wu, and L. Wang, “Sportsmot: A large multi-object tracking dataset in multiple sports scenes,”Proceed- ings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023

  4. [4]

    Soccernet-tracking: Multiple object tracking dataset and benchmark in soccer videos,

    A. Cioppa, S. Giancola, A. Deliege, L. Kang, X. Zhou, Z. Cheng, B. Ghanem, and M. Van Droogenbroeck, “Soccernet-tracking: Multiple object tracking dataset and benchmark in soccer videos,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, 2022, pp. 3491–3502

  5. [5]

    Dancetrack: Multi-object tracking in uniform appearance and diverse motion,

    P. Sun, J. Cao, Y . Jiang, Z. Yuan, S. Bai, K. Kitani, and P. Luo, “Dancetrack: Multi-object tracking in uniform appearance and diverse motion,”Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021

  6. [6]

    Bytetrack: Multi-object tracking by associating every detection box,

    Y . Zhang, P. Sun, Y . Jiang, D. Yu, F. Weng, Z. Yuan, P. Luo, W. Liu, and X. Wang, “Bytetrack: Multi-object tracking by associating every detection box,” 2022

  7. [7]

    BoT-SORT: Robust Associations Multi-Pedestrian Tracking

    N. Aharon, R. Orfaig, and B.-Z. Bobrovsky, “Bot-sort: Robust associa- tions multi-pedestrian tracking,”arXiv preprint arXiv:2206.14651, 2022

  8. [8]

    Hard to track objects with irregular motions and similar appearances? make it easier by buffering the matching space,

    F. Yang, S. Odashima, S. Masui, and S. Jiang, “Hard to track objects with irregular motions and similar appearances? make it easier by buffering the matching space,” inProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), January 2023, pp. 4799– 4808

  9. [9]

    Observation- centric sort: Rethinking sort for robust multi-object tracking,

    J. Cao, J. Pang, X. Weng, R. Khirodkar, and K. Kitani, “Observation- centric sort: Rethinking sort for robust multi-object tracking,” inPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 9686–9696

  10. [10]

    No train yet gain: Towards generic multi-object tracking in sports and beyond,

    T. Stanczyk, S. Yoon, and F. Bremond, “No train yet gain: Towards generic multi-object tracking in sports and beyond,” inProceedings of the Computer Vision and Pattern Recognition Conference (CVPR) Workshops, June 2025, pp. 6039–6048

  11. [11]

    Topic: A parallel association paradigm for multi-object tracking under complex motions and diverse scenes,

    X. Cao, Y . Zheng, Y . Yao, H. Qin, X. Cao, and S. Guo, “Topic: A parallel association paradigm for multi-object tracking under complex motions and diverse scenes,”IEEE Transactions on Image Processing, vol. 34, pp. 743–758, 2025

  12. [12]

    A New Approach to Linear Filtering and Prediction Problems,

    R. E. Kalman, “A New Approach to Linear Filtering and Prediction Problems,”Journal of Basic Engineering, vol. 82, no. 1, pp. 35–45, 03

  13. [13]

    Available: https://doi.org/10.1115/1.3662552

    [Online]. Available: https://doi.org/10.1115/1.3662552

  14. [14]

    Putting the object back into video object segmentation,

    H. K. Cheng, S. W. Oh, B. Price, J.-Y . Lee, and A. Schwing, “Putting the object back into video object segmentation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

  15. [15]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, P. Doll ´ar, and R. Gir- shick, “Segment anything,”Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023

  16. [16]

    Scalable person re-identification: A benchmark,

    L. Zheng, L. Shen, L. Tian, S. Wang, J. Wang, and Q. Tian, “Scalable person re-identification: A benchmark,” inProceedings of the IEEE International Conference on Computer Vision (ICCV), December 2015

  17. [17]

    Towards re-identification for long-term tracking of group housed pigs,

    M. Wang, M. Larsen, D. Liu, J. Winters, J.-L. Rault, and T. Norton, “Towards re-identification for long-term tracking of group housed pigs,”Biosystems Engineering, vol. 222, p. 71–81, 10 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S1537511022001799

  18. [18]

    Individual honey bee tracking in a beehive environment using deep learning and kalman filter,

    P. Kongsilp, U. Taetragool, and O. Duangphakdee, “Individual honey bee tracking in a beehive environment using deep learning and kalman filter,”Scientific Reports, vol. 14, no. 11, p. 1061, 1 2024. [Online]. Available: https://www.nature.com/articles/s41598-023-44718-y

  19. [19]

    Wildlifedatasets: An open-source toolkit for animal re-identification,

    V . ˇCerm´ak, L. Picek, A. Luk ´aˇs, and K. Papafitsoros, “Wildlifedatasets: An open-source toolkit for animal re-identification,” inProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 1 2024, pp. 5953–5963

  20. [20]

    Facenet: A unified embedding for face recognition and clustering,

    F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,”2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2015. [Online]. Available: http://dx.doi.org/10.1109/CVPR.2015.7298682

  21. [21]

    FastReID: A Pytorch Toolbox for General Instance Re-identification

    L. He, X. Liao, W. Liu, X. Liu, P. Cheng, and T. Mei, “Fastreid: A pytorch toolbox for general instance re-identification,”arXiv preprint arXiv:2006.02631, 2020

  22. [22]

    Omni-scale feature learning for person re-identification,

    K. Zhou, Y . Yang, A. Cavallaro, and T. Xiang, “Omni-scale feature learning for person re-identification,” inICCV, 2019

  23. [23]

    Simple online and realtime tracking with a deep association metric,

    N. Wojke, A. Bewley, and D. Paulus, “Simple online and realtime tracking with a deep association metric,” in2017 IEEE International Conference on Image Processing (ICIP). IEEE, 2017, pp. 3645–3649

  24. [24]

    Strong- sort: Make deepsort great again,

    Y . Du, Z. Zhao, Y . Song, Y . Zhao, F. Su, T. Gong, and H. Meng, “Strong- sort: Make deepsort great again,”IEEE Transactions on Multimedia, 2023

  25. [25]

    Biological control with trichogramma: Advances, successes, and potential of their use,

    S. Smith, “Biological control with trichogramma: Advances, successes, and potential of their use,”Annual Review of Entomology, vol. 41, no. 1, p. 375–406, 1996. [Online]. Available: https://doi.org/10.1146/ annurev.en.41.010196.002111

  26. [26]

    Inundative release of trichogramma dendrolimi at different developmental stages enhances the control efficacy over ostrinia furnacalis,

    Y . Wang, Y . Hou, A. Iqbal, S. Wang, L. Monticelli, N. Desneux, and L. Zang, “Inundative release of trichogramma dendrolimi at different developmental stages enhances the control efficacy over ostrinia furnacalis,”Journal of Pest Science, vol. 97, no. 4, p. 1889–1898, 9

  27. [27]

    Available: https://doi.org/10.1007/s10340-023-01731-4

    [Online]. Available: https://doi.org/10.1007/s10340-023-01731-4

  28. [28]

    Intrinsic inter-and intraspecific competition in parasitoid wasps,

    J. Harvey, E. Poelman, and T. Tanaka, “Intrinsic inter-and intraspecific competition in parasitoid wasps,”Annual Review of Entomology, vol. 58, p. 333–351, 2013

  29. [29]

    Patch exploitation by non- aggressive parasitoids under intra- and interspecific competition,

    F.-A. Robert, J. Brodeur, and G. Boivin, “Patch exploitation by non- aggressive parasitoids under intra- and interspecific competition,”Ento- mologia Experimentalis et Applicata, vol. 159, no. 1, p. 92–101, 2016

  30. [30]

    Trackeval,

    A. H. Jonathon Luiten, “Trackeval,” https://github.com/JonathonLuiten/ TrackEval, 2020

  31. [31]

    The hungarian method for the assignment problem,

    H. W. Kuhn, “The hungarian method for the assignment problem,” Naval Research Logistics Quarterly, vol. 2, no. 1-2, pp. 83–97, 1955. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1002/nav. 3800020109

  32. [32]

    Hota: A higher order metric for evaluating multi-object tracking,

    J. Luiten, A. Osep, P. Dendorfer, P. Torr, A. Geiger, L. Leal-Taix ´e, and B. Leibe, “Hota: A higher order metric for evaluating multi-object tracking,”International Journal of Computer Vision, pp. 1–31, 2020

  33. [33]

    Performance measures and a data set for multi-target, multi-camera tracking,

    E. Ristani, F. Solera, R. Zou, R. Cucchiara, and C. Tomasi, “Performance measures and a data set for multi-target, multi-camera tracking,” in Computer Vision – ECCV 2016 Workshops, G. Hua and H. J ´egou, Eds. Cham: Springer International Publishing, 2016, pp. 17–35

  34. [34]

    Evaluating multiple object tracking performance: The clear mot metrics,

    K. Bernardin and R. Stiefelhagen, “Evaluating multiple object tracking performance: The clear mot metrics,”EURASIP Journal on Image and Video Processing, vol. 2008, 01 2008