REVIEW 1 major objections 7 minor 34 references
Reviewed by Pith at T0; open to challenge.
T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →
T0 review · glm-5.2
Perfect detections, broken identities: long video breaks MOT trackers
2026-07-10 02:06 UTC pith:KWMHAC2Y
load-bearing objection Solid long-duration MOT benchmark; framing slightly overstates what's proven the 1 major comments →
WaspMOT: A Benchmark for Long-Term Multi-Object Tracking of Trichogramma Wasps
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
When tracking-by-detection methods are evaluated on sequences long enough to require identity preservation over thousands of frames rather than tens of seconds, they all produce severely fragmented trajectories — even when detections are perfect. The failure is in association, not detection: trackers lose identities because they rely on local motion-continuity assumptions that break under abrupt movements, occlusions, and visual similarity. A simple post-hoc stitching of trajectory fragments by spatial proximity recovers substantial performance, demonstrating that much of the fragmentation is not due to genuine identity ambiguity but to the limitations of local, frame-to-frame association.
What carries the argument
The closed-set tracking scenario with full-length trajectories evaluated under oracle detections. In this regime, every individual is present from the first frame to the last, so the only task is to assign and maintain consistent identities across thousands of frames. This setup separates the association problem — which identity does each detection belong to — from the detection problem — where are the objects — and from the track lifecycle problem — when do tracks start and end.
Load-bearing premise
The paper assumes that the trajectory fragmentation observed when tracking small, visually identical insects that perform abrupt jumps in a three-dimensional arena is representative of a general long-term tracking problem, rather than being specific to this ecological setting. It does not directly demonstrate that extending a conventional pedestrian dataset to comparable duration would produce similar fragmentation.
What would settle it
Run a conventional pedestrian or sports tracking dataset extended to comparable duration (thousands of frames) with oracle detections. If existing trackers maintain high identity consistency on those extended sequences, then the fragmentation observed here is domain-specific rather than a general long-term tracking failure.
If this is right
- If long-duration tracking fundamentally breaks local association mechanisms, then the dominant paradigm of frame-to-frame association with Kalman filtering needs augmentation with global or long-horizon reasoning to maintain identity consistency.
- The consistent success of simple spatial stitching suggests that trajectory fragmentation is often a recoverable error — identity information is present in the data but current methods fail to exploit it — pointing toward hierarchical or two-stage tracking architectures.
- The finding that appearance-based methods do not outperform simpler motion-based methods on visually similar targets suggests appearance features have a domain of applicability that degrades when inter-object visual differences are minimal.
- The closed-set, full-length trajectory evaluation regime could be applied to other domains to test whether identity preservation failures generalize beyond insect tracking.
- The gap between oracle-detection performance and perfect identity preservation quantifies how much room remains for association algorithms specifically, independent of further detection improvements.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces WaspMOT, a multi-object tracking benchmark consisting of 10 long-duration video sequences (~12,000 frames each) of Trichogramma wasps in a controlled laboratory arena. The dataset is annotated in MOTChallenge format and provides oracle detections (ground-truth bounding boxes without identity labels) to isolate association performance from detection errors. Five tracking-by-detection methods (ByteTrack, BoT-SORT, C-BIoU, OC-SORT, McByte) are evaluated under a unified protocol, and a simple spatial tracklet stitching baseline is applied as post-processing. The central finding is that all evaluated trackers exhibit significant trajectory fragmentation (low IDF1) despite perfect detections, and that simple spatial stitching recovers substantial performance, indicating that current association strategies struggle with long-term identity preservation.
Significance. The dataset fills a genuine gap: existing MOT benchmarks average 25–100 seconds per sequence, while WaspMOT sequences exceed 480 seconds with full-length closed-set trajectories. The oracle-detection protocol is a clean experimental design that isolates association from detection, and the use of standard TrackEval metrics ensures comparability. The finding that simple spatial stitching yields +7–9 IDF1 across all methods is a concrete, falsifiable result that motivates future work. The public release of data and code is a positive step for reproducibility.
major comments (1)
- The central claim that WaspMOT exposes a failure mode 'not observable on conventional datasets' (Abstract, §I, §VI) is not empirically demonstrated. The paper shows that trackers fragment on WaspMOT, but it never isolates sequence duration as the causal factor. The fragmentation could be driven by domain-specific challenges — abrupt jumps (§III.C, Figs. 2–3), 3D floor/ceiling occlusions, and near-identical appearance — rather than by temporal length per se. Notably, the stitching results (+7–9 IDF1 from spatial reconnection, Table II) suggest that fragmentation arises from local association failures (post-jump, post-occlusion) that accumulate over long sequences, not from gradual long-range identity drift. To support the 'long-term' framing, the authors should add at least one of the following controls: (1) an analysis of IDF1 (or fragmentation rate) as a function of sequence length *e.g
minor comments (7)
- §IV.B: The paper states that BoT-SORT's appearance embeddings are 'trained following the procedure described in the original work, adapted to WaspMOT.' Please specify the training data (which sequences, how many identity labels) and whether training and evaluation sequences overlap, as this affects the fairness of the appearance-based comparison.
- §IV.C: The spatial stitching baseline is described qualitatively but lacks key parameters: the spatial and temporal thresholds for candidate association, and whether these were tuned on the evaluation set. Please specify or state that defaults were used.
- Table I: The 'Avg boxes / video' entry for DanceTrack and BEE24 is listed as '-', and the formatting of the WaspMOT row (described as 'in green') is not visible in the text. Please ensure the table is self-contained in the text version.
- §III.A: The camera resolution is 3840×2160, but the arena is only 3.5cm × 2cm. It would help to state the approximate pixel size of a wasp bounding box to contextualize the 'small object' challenge.
- Fig. 2–3: These figures are referenced but their axes and content are only briefly described. Please ensure axis labels, units, and legends are clear in the final version.
- §II.A: The reference to BEE24 [11] describes it as providing 'limited' trajectory coverage. A quantitative comparison (e.g., average trajectory length as a fraction of sequence length) would strengthen the contrast.
- Typos: §I, 'ofTrichogrammawasps' (missing space); §III.A, 'ofTrichogrammawasps' (repeated); §III.B, '2,569,527 annotated object instances' — please verify this count is consistent with 'Avg boxes / video' in Table I (256,952.70 × 10 = 2,569,527, which checks out, but the table should note this is per-sequence average).
Circularity Check
No significant circularity; one minor non-load-bearing self-citation (McByte)
full rationale
This is an empirical benchmark paper, not a derivation chain. The central claim — that all five tracking-by-detection methods suffer trajectory fragmentation on long-duration sequences even with oracle detections — is tested by running external algorithms (ByteTrack, BoT-SORT, C-BIoU, OC-SORT, McByte) on the dataset and measuring standard metrics (HOTA, IDF1, MOTA) via the independent TrackEval toolkit. No parameter is fitted to data and then presented as a prediction. The oracle detections are derived from ground-truth bounding boxes without identity labels, which removes detection as a confound by construction — but this is a deliberate experimental design choice, not a circular derivation. The spatial stitching baseline is a post-processing heuristic whose improvements are empirically measured, not predicted from a fitted model. The only self-citation is McByte [10], co-authored by the paper's first author, but it is one of five evaluated methods and the central claim (all methods fragment) does not depend on McByte's results specifically. This self-citation is minor and non-load-bearing. The paper is self-contained against external benchmarks and standard evaluation protocols.
Axiom & Free-Parameter Ledger
axioms (4)
- domain assumption Oracle detections derived from ground truth bounding boxes without identity labels provide perfect detection accuracy and isolate association performance.
- domain assumption Default tracker parameters are appropriate for evaluating inherent association capabilities without unfair bias.
- domain assumption The closed-set tracking scenario (all individuals present throughout) is a meaningful evaluation regime for long-term identity preservation.
- standard math Standard MOT metrics (HOTA, IDF1, MOTA) computed via TrackEval are appropriate for evaluating long-term tracking.
read the original abstract
Multi-object tracking (MOT) has achieved strong performance on benchmarks dominated by short video sequences. However, such datasets do not adequately evaluate long-term identity preservation, where objects must be tracked consistently over extended durations. We introduce WaspMOT, a benchmark designed to address this gap through long-duration tracking of Trichogramma wasps in controlled ecological experiments. The dataset contains 10 sequences of approximately 12,000 frames each (over 8 minutes at 25 FPS), with dense MOTChallenge annotations and oracle detections to isolate association performance. Unlike existing benchmarks, WaspMOT forms a closed-set tracking scenario where all individuals remain present throughout the sequence, requiring consistent identity assignment across thousands of frames despite abrupt jumps, occlusions, and highly similar appearance. We establish a benchmark by evaluating five tracking-by-detection methods, including ByteTrack, BoT-SORT, C-BIoU, OC-SORT, and McByte, under a unified protocol. Results show that all methods suffer from significant trajectory fragmentation, highlighting the difficulty of long-term identity preservation even with perfect detections. A simple spatial tracklet stitching baseline consistently improves performance, indicating that substantial gains remain possible. WaspMOT provides a new benchmark for studying long-term association and reveals limitations of current tracking approaches that are not observable on conventional datasets. The benchmark will be made publicly available at the project repository: https://github.com/tstanczyk95/WaspMOT/ .
Figures
Reference graph
Works this paper leans on
-
[1]
MOT16: A Benchmark for Multi-Object Tracking
A. Milan, L. Leal-Taix ´e, I. Reid, S. Roth, and K. Schindler, “MOT16: A benchmark for multi-object tracking,”arXiv:1603.00831 [cs], Mar. 2016, arXiv: 1603.00831. [Online]. Available: http: //arxiv.org/abs/1603.00831
work page internal anchor Pith review Pith/arXiv arXiv 2016
-
[2]
MOT20: A benchmark for multi object tracking in crowded scenes
P. Dendorfer, H. Rezatofighi, A. Milan, J. Shi, D. Cremers, I. Reid, S. Roth, K. Schindler, and L. Leal-Taix ´e, “Mot20: A benchmark for multi object tracking in crowded scenes,” 2020. [Online]. Available: https://arxiv.org/abs/2003.09003
work page internal anchor Pith review Pith/arXiv arXiv 2020
-
[3]
Sportsmot: A large multi-object tracking dataset in multiple sports scenes,
Y . Cui, C. Zeng, X. Zhao, Y . Yang, G. Wu, and L. Wang, “Sportsmot: A large multi-object tracking dataset in multiple sports scenes,”Proceed- ings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023
work page 2023
-
[4]
Soccernet-tracking: Multiple object tracking dataset and benchmark in soccer videos,
A. Cioppa, S. Giancola, A. Deliege, L. Kang, X. Zhou, Z. Cheng, B. Ghanem, and M. Van Droogenbroeck, “Soccernet-tracking: Multiple object tracking dataset and benchmark in soccer videos,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, 2022, pp. 3491–3502
work page 2022
-
[5]
Dancetrack: Multi-object tracking in uniform appearance and diverse motion,
P. Sun, J. Cao, Y . Jiang, Z. Yuan, S. Bai, K. Kitani, and P. Luo, “Dancetrack: Multi-object tracking in uniform appearance and diverse motion,”Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021
work page 2021
-
[6]
Bytetrack: Multi-object tracking by associating every detection box,
Y . Zhang, P. Sun, Y . Jiang, D. Yu, F. Weng, Z. Yuan, P. Luo, W. Liu, and X. Wang, “Bytetrack: Multi-object tracking by associating every detection box,” 2022
work page 2022
-
[7]
BoT-SORT: Robust Associations Multi-Pedestrian Tracking
N. Aharon, R. Orfaig, and B.-Z. Bobrovsky, “Bot-sort: Robust associa- tions multi-pedestrian tracking,”arXiv preprint arXiv:2206.14651, 2022
work page internal anchor Pith review Pith/arXiv arXiv 2022
-
[8]
F. Yang, S. Odashima, S. Masui, and S. Jiang, “Hard to track objects with irregular motions and similar appearances? make it easier by buffering the matching space,” inProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), January 2023, pp. 4799– 4808
work page 2023
-
[9]
Observation- centric sort: Rethinking sort for robust multi-object tracking,
J. Cao, J. Pang, X. Weng, R. Khirodkar, and K. Kitani, “Observation- centric sort: Rethinking sort for robust multi-object tracking,” inPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 9686–9696
work page 2023
-
[10]
No train yet gain: Towards generic multi-object tracking in sports and beyond,
T. Stanczyk, S. Yoon, and F. Bremond, “No train yet gain: Towards generic multi-object tracking in sports and beyond,” inProceedings of the Computer Vision and Pattern Recognition Conference (CVPR) Workshops, June 2025, pp. 6039–6048
work page 2025
-
[11]
X. Cao, Y . Zheng, Y . Yao, H. Qin, X. Cao, and S. Guo, “Topic: A parallel association paradigm for multi-object tracking under complex motions and diverse scenes,”IEEE Transactions on Image Processing, vol. 34, pp. 743–758, 2025
work page 2025
-
[12]
A New Approach to Linear Filtering and Prediction Problems,
R. E. Kalman, “A New Approach to Linear Filtering and Prediction Problems,”Journal of Basic Engineering, vol. 82, no. 1, pp. 35–45, 03
-
[13]
Available: https://doi.org/10.1115/1.3662552
[Online]. Available: https://doi.org/10.1115/1.3662552
-
[14]
Putting the object back into video object segmentation,
H. K. Cheng, S. W. Oh, B. Price, J.-Y . Lee, and A. Schwing, “Putting the object back into video object segmentation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024
work page 2024
-
[15]
A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, P. Doll ´ar, and R. Gir- shick, “Segment anything,”Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023
work page 2023
-
[16]
Scalable person re-identification: A benchmark,
L. Zheng, L. Shen, L. Tian, S. Wang, J. Wang, and Q. Tian, “Scalable person re-identification: A benchmark,” inProceedings of the IEEE International Conference on Computer Vision (ICCV), December 2015
work page 2015
-
[17]
Towards re-identification for long-term tracking of group housed pigs,
M. Wang, M. Larsen, D. Liu, J. Winters, J.-L. Rault, and T. Norton, “Towards re-identification for long-term tracking of group housed pigs,”Biosystems Engineering, vol. 222, p. 71–81, 10 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S1537511022001799
work page 2022
-
[18]
Individual honey bee tracking in a beehive environment using deep learning and kalman filter,
P. Kongsilp, U. Taetragool, and O. Duangphakdee, “Individual honey bee tracking in a beehive environment using deep learning and kalman filter,”Scientific Reports, vol. 14, no. 11, p. 1061, 1 2024. [Online]. Available: https://www.nature.com/articles/s41598-023-44718-y
work page 2024
-
[19]
Wildlifedatasets: An open-source toolkit for animal re-identification,
V . ˇCerm´ak, L. Picek, A. Luk ´aˇs, and K. Papafitsoros, “Wildlifedatasets: An open-source toolkit for animal re-identification,” inProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 1 2024, pp. 5953–5963
work page 2024
-
[20]
Facenet: A unified embedding for face recognition and clustering,
F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,”2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2015. [Online]. Available: http://dx.doi.org/10.1109/CVPR.2015.7298682
-
[21]
FastReID: A Pytorch Toolbox for General Instance Re-identification
L. He, X. Liao, W. Liu, X. Liu, P. Cheng, and T. Mei, “Fastreid: A pytorch toolbox for general instance re-identification,”arXiv preprint arXiv:2006.02631, 2020
work page internal anchor Pith review Pith/arXiv arXiv 2006
-
[22]
Omni-scale feature learning for person re-identification,
K. Zhou, Y . Yang, A. Cavallaro, and T. Xiang, “Omni-scale feature learning for person re-identification,” inICCV, 2019
work page 2019
-
[23]
Simple online and realtime tracking with a deep association metric,
N. Wojke, A. Bewley, and D. Paulus, “Simple online and realtime tracking with a deep association metric,” in2017 IEEE International Conference on Image Processing (ICIP). IEEE, 2017, pp. 3645–3649
work page 2017
-
[24]
Strong- sort: Make deepsort great again,
Y . Du, Z. Zhao, Y . Song, Y . Zhao, F. Su, T. Gong, and H. Meng, “Strong- sort: Make deepsort great again,”IEEE Transactions on Multimedia, 2023
work page 2023
-
[25]
Biological control with trichogramma: Advances, successes, and potential of their use,
S. Smith, “Biological control with trichogramma: Advances, successes, and potential of their use,”Annual Review of Entomology, vol. 41, no. 1, p. 375–406, 1996. [Online]. Available: https://doi.org/10.1146/ annurev.en.41.010196.002111
-
[26]
Y . Wang, Y . Hou, A. Iqbal, S. Wang, L. Monticelli, N. Desneux, and L. Zang, “Inundative release of trichogramma dendrolimi at different developmental stages enhances the control efficacy over ostrinia furnacalis,”Journal of Pest Science, vol. 97, no. 4, p. 1889–1898, 9
-
[27]
Available: https://doi.org/10.1007/s10340-023-01731-4
[Online]. Available: https://doi.org/10.1007/s10340-023-01731-4
-
[28]
Intrinsic inter-and intraspecific competition in parasitoid wasps,
J. Harvey, E. Poelman, and T. Tanaka, “Intrinsic inter-and intraspecific competition in parasitoid wasps,”Annual Review of Entomology, vol. 58, p. 333–351, 2013
work page 2013
-
[29]
Patch exploitation by non- aggressive parasitoids under intra- and interspecific competition,
F.-A. Robert, J. Brodeur, and G. Boivin, “Patch exploitation by non- aggressive parasitoids under intra- and interspecific competition,”Ento- mologia Experimentalis et Applicata, vol. 159, no. 1, p. 92–101, 2016
work page 2016
-
[30]
A. H. Jonathon Luiten, “Trackeval,” https://github.com/JonathonLuiten/ TrackEval, 2020
work page 2020
-
[31]
The hungarian method for the assignment problem,
H. W. Kuhn, “The hungarian method for the assignment problem,” Naval Research Logistics Quarterly, vol. 2, no. 1-2, pp. 83–97, 1955. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1002/nav. 3800020109
work page doi:10.1002/nav 1955
-
[32]
Hota: A higher order metric for evaluating multi-object tracking,
J. Luiten, A. Osep, P. Dendorfer, P. Torr, A. Geiger, L. Leal-Taix ´e, and B. Leibe, “Hota: A higher order metric for evaluating multi-object tracking,”International Journal of Computer Vision, pp. 1–31, 2020
work page 2020
-
[33]
Performance measures and a data set for multi-target, multi-camera tracking,
E. Ristani, F. Solera, R. Zou, R. Cucchiara, and C. Tomasi, “Performance measures and a data set for multi-target, multi-camera tracking,” in Computer Vision – ECCV 2016 Workshops, G. Hua and H. J ´egou, Eds. Cham: Springer International Publishing, 2016, pp. 17–35
work page 2016
-
[34]
Evaluating multiple object tracking performance: The clear mot metrics,
K. Bernardin and R. Stiefelhagen, “Evaluating multiple object tracking performance: The clear mot metrics,”EURASIP Journal on Image and Video Processing, vol. 2008, 01 2008
work page 2008
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.