REVIEW 4 major objections 7 minor 29 references
A Framework for Multi-View Multiple Object Tracking using Single-View Multi-Object Trackers on Fish Data
T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A stereo pair of off-the-shelf single-view trackers can yield 3D fish trajectories for ecological studies.
desk verdict A workmanlike stereo fish-tracking pipeline whose central accuracy claim is never tested; the real contribution is the dataset and honest evaluation, not the multi-view enhancement. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the post-track fusion pipeline that converts two independent 2D tracking outputs into a 3D trajectory set. Each view is processed by YOLOv8 detection with ByteTrack association; a hand-tuned post-track re-identification pass merges fragmented identities using a 50-pixel proximity threshold, a 100-frame disappearance window, and a 10-frame co-occurrence rule; then stereo correspondence is established through the fundamental matrix and epipolar-line distance thresholds, with the most common per-fish match selected across frames; finally, the calibrated camera parameters feed a triangulation step that produces x, y, z coordinates. The epipolar constraint—that a point in one camera lies on a specific line in the other—is the geometric carrier of the cross-view identity matching.
What would settle it
Run the pipeline on the short stereo videos without manual re-matching and compare the automatically matched fish-ID pairs to a hand-labelled ground-truth correspondence; if the automatic match rate stays at the reported 22–36% rather than near 100%, the framework's central identity-preservation premise fails for short recordings. A complementary check is to compare HOTA/MOTA on the fused multi-view output against the single-view output on the same videos: the claimed enhanced tracking accuracy would be falsified if the fused trajectories do not score higher on association metrics.
Extended reading notes
Core claim
The central claim is that single-view multi-object trackers can be dual-leveraged—run independently on each stereo view and then fused through geometry—to produce a new 3D output and enhanced tracking accuracy for fish in underwater videos. The framework detects fish with a relative accuracy of about 47% within the ground-truth margin, tracks a subset of fish for full video lengths, and uses the fundamental matrix and epipolar lines to find corresponding fish IDs between views; the most frequent match per fish across frames is chosen, and camera calibration parameters are used to triangulate x, y, z coordinates. In the paper's account, the 3D output enables more realistic speed, depth, and grouping analyses than 2D single-view tracks, and for a select set of videos the multi-view framework overcomes the challenges of small, visually similar fish with rapidly changing appearances. The authors conclude that this adapts off-the-shelf single-view trackers to ecological studies, a gap they identify in prior work.
Load-bearing premise
The whole framework stands on the assumption that the stereo-matching step pairs the same physical fish in the two camera views, so that every triangulated 3D point is the same real fish.
Editorial extensions
If this is right
- Ecologists can generate 3D fish trajectories (x, y, z per frame) from calibrated stereo cameras using off-the-shelf single-view trackers and the described fusion steps.
- The framework produces an enriched behavioral dataset: the paper reports speed over time, acceleration, path length, spatial distribution, density maps, temporal patterns, and depth over time from the triangulated coordinates.
- On long stereo videos (e.g., 3,117-frame recordings 129 and 406) the stereo-matching step reached 95–100% accuracy, indicating the approach is usable without manual correction on extended footage.
- Even with a detection recall near 48%, the post-track re-identification step reduced the number of distinct fish IDs by 43.3% on average, consolidating fragmented tracks before fusion.
Reading between the lines
- A direct consequence the authors do not spell out: if off-the-shelf single-view trackers plus epipolar fusion suffice for 3D fish tracking, then the expensive joint multi-view training used in models like DETR3D is unnecessary for this ecological scenario; the bottleneck becomes the per-view detector's recall, not the fusion strategy.
- The stereo-matching failure on short videos suggests the frequency-based 'most common match' selection needs temporal accumulation: instead of picking the modal match per fish from a 258-frame video, one could accumulate correspondences across the whole recording, or enforce one-to-one matching globally, which might recover the short-video cases that currently require manual re-matching.
- The claimed 'enhanced tracking accuracy' is asserted but never measured against the single-view baseline with the same MOT metrics; a reader could test it by computing HOTA/MOTA on the fused tracks versus the per-view tracks, which would isolate whether the 3D step improves association or merely adds coordinates.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a multi-view multiple-object tracking framework for underwater fish videos. The pipeline runs YOLOv8 with ByteTrack independently on each stereo view, applies a post-hoc ID reconciliation heuristic, matches fish identities across views via epipolar geometry, triangulates matched bounding-box centers to 3D coordinates, and produces 3D trajectory visualizations. The authors report single-view MOT metrics, detection-margin statistics, ID-count changes after re-identification, stereo-matching accuracy, and example 3D coordinates. The central claim is that this framework enhances fish tracking accuracy for ecological studies by leveraging stereo inputs and generating a new 3D output.
Significance. If the enhancement claim were supported, the paper would offer ecologists a practical recipe for obtaining 3D fish trajectories from calibrated stereo cameras using off-the-shelf single-view trackers. The authors document a complete pipeline, make code available, and test on a real underwater fish dataset, which are useful engineering contributions. However, the load-bearing assertion that multi-view processing improves tracking accuracy is never tested: the pipeline adds no cross-view feedback to the single-view tracks, and no multi-view versus single-view comparison is reported. The reported metrics are also weak (recall 0.48, HOTA 0.100, DetA 0.023), and the stereo-correspondence step succeeds on only 22-36% of frames for short videos, requiring manual re-matching. As it stands, the contribution reduces to appending triangulated 3D coordinates to independently computed 2D tracks, so the significance of the claimed enhancement is not established.
major comments (4)
- [Section 6, Section 3.1.4] The central claim that the multi-view framework enhances tracking accuracy is never tested. In the pipeline of Section 3.1.4, YOLOv8+ByteTrack is run independently on each view (step 1), and the stereo matching (step 4) and triangulation (step 6) operate only on the resulting track files; no cross-view information is fed back to correct detections, associations, or missed tracks. Consequently, the 2D tracking metrics (HOTA, MOTA, IDF1) are identical with and without the multi-view stages. The paper reports no multi-view versus single-view comparison and no metric that measures the claimed enhancement. The statement in Section 6 that the framework 'thereby enhancing fish tracking accuracy' is therefore unsupported by the experimental design.
- [Section 4.5, Table 6] The stereo-correspondence step, which is essential for the 3D output, is correct in only 22% and 36% of frames for the short videos, and the text states that those matches 'had to be re-matched manually' to be usable. Thus, for the short-video subset the 3D coordinates are not produced by the proposed framework but by manual annotation. The paper does not report how many of the videos and frames used in the 3D evaluation were manually re-matched, so the claimed automatic generation of 3D trajectories is established only for the two long videos.
- [Section 3.2] The evaluation relies on an ID mapping function asserted to have '100% accuracy' in mapping tracking IDs to ground-truth IDs, but the paper gives no description of how this mapping is obtained or validated. If this mapping is imperfect, every association-sensitive metric (HOTA, AssA, IDF1, MOTA) and the re-identification evaluation in Section 4.3 are affected. This assertion is load-bearing and must be substantiated.
- [Section 4.3, Table 4] The post-track re-identification heuristic (Section 3.1.4 step 2) is not validated. Table 4 shows that after re-identification the number of IDs drops below the ground-truth number for several videos (e.g., video 13_1: GT 13, before re-ID 17, after re-ID 6; video 8_1: GT 9, after re-ID 6), indicating that distinct fish are being merged. The thresholds (50 pixels, 100 frames, 10 frames, 30 frames) are introduced without ablation or sensitivity analysis, so the claimed improvement from re-identification is not established.
minor comments (7)
- [Abstract] The phrase 'novel 3D output' overstates standard triangulation from calibrated stereo cameras; suggest rephrasing to '3D output' or similar.
- [Section 3.1.4, step 4] The text alternates between 'keypoints' (e.g., corners) and fish bounding-box centers; clarify what features are actually matched and whether fish centers are used or arbitrary corner features.
- [Section 4.1, Table 1] The 'Average*' row is not defined (which videos are included, how many), and no standard deviations are reported; add the full protocol and variance.
- [Section 4.2, Table 3] The margin criterion (half the average bounding box size and twice the average size) needs a precise definition of which average is used and how the two tiers are combined; currently the reported percentages are difficult to interpret.
- [Section 4.3 and Section 3.1.4 step 2] The post-hoc heuristic is called 're-identification,' which may be confused with the learned re-ID branch of MOT models; rename it (e.g., 'ID reconciliation') to avoid ambiguity.
- [Section 3.1.3] The model trained by Dr. Nejadasl is described only as having better performance; provide training data, hyperparameters, and validation metrics so the experiments are reproducible.
- [Appendix C] Several figure captions in Appendix C appear mis-numbered (multiple figures labeled 'Figure 12'), which should be corrected.
Circularity Check
No circularity found: the multi-view pipeline uses external single-view trackers and standard stereo geometry; the unsupported accuracy-enhancement claim is a validation gap, not a circular reduction.
full rationale
No significant circularity. The derivation chain is: YOLOv8+ByteTrack (external, cited to [15,26]) produces per-view tracks; epipolar matching and triangulation use standard multi-view geometry [6,10,11,18,23]; and the 3D coordinates are outputs of those operations, not inputs to the claimed conclusion. The post-track re-ID thresholds (50 pixels, 100 frames, 10 frames, 30 frames, Section 3.1.4(2)) are hand-chosen heuristics evaluated against external ground-truth ID mappings, so they are not a fitted quantity disguised as a prediction. The self-citations [2,8,12] appear only as related-work context and are not load-bearing. Section 4.5 admits stereo matching failed on short videos and required manual re-matching, and Section 6 claims 'enhancing fish tracking accuracy' without any multi-view versus single-view MOT comparison; these are correctness and validation limitations, not circular reductions. Similarly, calling the standard triangulation output 'novel' is an overclaim, but no quoted equation or construction makes the conclusion equivalent to its inputs.
Assumptions & free parameters
free parameters (5)
- reid_distance_threshold =
50 pixels
- reid_frame_window =
100 frames
- reid_overlap_frames =
10 frames
- false_positive_min_frames =
30 frames
- detection_margin =
0.5 average GT box size (and 2x for near miss)
assumptions (4)
- standard math Epipolar geometry, fundamental matrix estimation, and triangulation from calibrated stereo pairs are valid.
- domain assumption The ground-truth MOT files and stereo calibration .mat parameters are correct.
- ad hoc to paper The pre-trained fish detection model supplied by Dr. Nejadasl, replacing locally trained models, is trained on comparable data and is representative.
- ad hoc to paper The ID mapping function used for evaluation achieves 100% accuracy in mapping tracking IDs to ground truth IDs.
Cite this review
Pith. "Pith review of A Framework for Multi-View Multiple Object Tracking using Single-View Multi-Object Trackers on Fish Data." pith.science (2026). https://pith.science/paper/5X453RAN
@misc{pith2026250517201,
author = {Pith},
title = {Pith review of: A Framework for Multi-View Multiple Object Tracking using Single-View Multi-Object Trackers on Fish Data},
year = {2026},
howpublished = {\url{https://pith.science/paper/5X453RAN}},
note = {Machine review of arXiv:2505.17201}
}
read the original abstract
Multi-object tracking (MOT) in computer vision has made significant advancements, yet tracking small fish in underwater environments presents unique challenges due to complex 3D motions and data noise. Traditional single-view MOT models often fall short in these settings. This thesis addresses these challenges by adapting state-of-the-art single-view MOT models, FairMOT and YOLOv8, for underwater fish detecting and tracking in ecological studies. The core contribution of this research is the development of a multi-view framework that utilizes stereo video inputs to enhance tracking accuracy and fish behavior pattern recognition. By integrating and evaluating these models on underwater fish video datasets, the study aims to demonstrate significant improvements in precision and reliability compared to single-view approaches. The proposed framework detects fish entities with a relative accuracy of 47% and employs stereo-matching techniques to produce a novel 3D output, providing a more comprehensive understanding of fish movements and interactions
Figures
Figures from the paper (16 more)
Reference graph
Works this paper leans on
-
[1]
Derya Akkaynak and Tali Treibitz. 2019. Sea-thru: A method for removing water from underwater images. (2019), 1682–1691
work page 2019
-
[2]
Karim Anwar and Seyed Sahand Mohammadi Ziabari. 2025. Attention to the Branches: A Comparative Analysis of FairMOT with Transformers on Fish Dataset. InMulti-disciplinary Trends in Artificial Intelligence, Chattrakul Sombat- theera, Paul Weng, and Jun Pang (Eds.). Springer Nature Singapore, Singapore, 64–76. https://doi.org/10.1007/978-981-96-0692-4_6
-
[3]
Keni Bernardin and Rainer Stiefelhagen. 2008. Evaluating multiple object track- ing performance: the clear mot metrics.EURASIP Journal on Image and Video Processing2008 (2008), 1–10
work page 2008
-
[4]
Akash D. 2023. yolov8_in_depth. https://github.com/akashAD98/yolov8_in_ depth Accessed: 2024-06-10
work page 2023
-
[5]
Patrick Dendorfer, Hamid Rezatofighi, Anton Milan, Javen Shi, Daniel Cremers, Ian Reid, Stefan Roth, Konrad Schindler, and Laura Leal-Taixé. 2020. Mot20: A benchmark for multi object tracking in crowded scenes.arXiv preprint arXiv:2003.09003(2020)
arXiv 2020
-
[6]
Victor H Diaz-Ramirez, Martin Gonzalez-Ruiz, Vitaly Kober, and Rigoberto Juarez- Salazar. 2022. Stereo image matching using adaptive morphological correlation. Sensors22, 23 (2022), 9050
work page 2022
-
[7]
Meng Joo Er, Jie Chen, Yani Zhang, and Wenxiao Gao. 2023. Research Challenges, Recent Advances, and Popular Datasets in Deep Learning-Based Underwater Marine Object Detection: A Review.Sensors23, 4 (Feb. 2023), 1990. https: //doi.org/10.3390/s23041990
-
[8]
Faber, Seyed Sahand Mohammadi Ziabari, and Fatemeh Karimi Nejadasl
Niels G. Faber, Seyed Sahand Mohammadi Ziabari, and Fatemeh Karimi Nejadasl
Show all 29 references
-
[9]
Rui Gao and Lu Wang. 2023. MeMOTR: Long-Term Memory-Augmented Trans- former for Multi-Object Tracking.arXiv(2023). arXiv:2307.15700 [cs.CV]
2023 arXiv
-
[10]
2003.Multiple view geometry in computer vision
Richard Hartley and Andrew Zisserman. 2003.Multiple view geometry in computer vision. Cambridge university press
2003
-
[11]
Heiko Hirschmuller. 2007. Stereo processing by semiglobal matching and mutual information.IEEE Transactions on pattern analysis and machine intelligence30, 2 (2007), 328–341
2007
-
[12]
Katona, S.S
Z. Katona, S.S. Mohammadi Ziabari, and F. Karimi Nejadasl. 2025. MARINE: A Computer Vision Model for Detecting Rare Predator-Prey Interactions in Animal Videos. InBig Data and Artificial Intelligence. BDA 2024 (Lecture Notes in Computer Science, Vol. 15526), A. Dasgupta, R.U. ...
2025 doi
-
[13]
Haitong Lou, Xuehu Duan, Junmei Guo, Haiying Liu, Jason Gu, Lingyun Bi, and Haonan Chen. 2023. DC-YOLOv8: small-size object detection algorithm based on camera sensor.Electronics12, 10 (2023), 2323
2023
-
[14]
= 2020, month = oct, pages = 548–578
Jonathon Luiten, Aljoša Ošep, Patrick Dendorfer, Philip Torr, Andreas Geiger, Laura Leal-Taixé, and Bastian Leibe. = 2020, month = oct, pages = 548–578. HOTA: A Higher Order Metric for Evaluating Multi-object Tracking.International Journal of Computer Vision129, 2 (= 2020, mon...
2020 doi
-
[15]
Muhammad Rizwan Munawar, Glenn Jocher, and Ayush Chaurasia. 2023. Home — docs.ultralytics.com. https://docs.ultralytics.com/. [Accessed 24-05-2024]
2023
-
[16]
David MW Powers. 2020. Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation.arXiv preprint arXiv:2010.16061 (2020)
2020 arXiv
-
[17]
Zou, Rita Cucchiara, and Carlo Tomasi
Ergys Ristani, Francesco Solera, Roger S. Zou, Rita Cucchiara, and Carlo Tomasi
-
[18]
Olgierd Stankiewicz, Gauthier Lafruit, and Marek Domański. 2018. Multiview video: Acquisition, processing, compression, and virtual view rendering. (2018), 3–74
2018
-
[19]
Ultralytics. 2023. Are class and box losses calculated the same in YoloV8 and YoloV5? https://github.com/ultralytics/ultralytics/issues/2789 Accessed: 2024- 06-10
2023
-
[20]
Ultralytics. 2023. yolo8 loss function for classification task. https://github.com/ ultralytics/ultralytics/issues/4684 Accessed: 2024-06-10
2023
-
[21]
Yue Wang, Vitor Campagnolo Guizilini, Tianyuan Zhang, Yilun Wang, Hang Zhao, and Justin Solomon. 2022. Detr3d: 3d object detection from multi-view images via 3d-to-2d queries. InConference on Robot Learning. PMLR, 180–191
2022
-
[22]
Xin Xiao and Xinlong Feng. 2023. Multi-object pedestrian tracking using im- proved YOLOv8 and OC-SORT.Sensors23, 20 (2023), 8439
2023
-
[23]
Atsushi Yamashita, Toru Kaneko, Shinya Matsushita, Kenjiro T Miura, and Sue- kichi Isogai. 2003. Camera Calibration and 3-D Measurement with an Active Stereo Vision System for Handling Moving Objects.Journal of Robotics and Mechatronics15, 3 (2003), 304–313
2003
-
[24]
En Yu et al. 2023. MOTRv3: Release-Fetch Supervision for End-to-End Multi- Object Tracking.arXiv preprint arXiv:2305.14298(2023)
2023 arXiv
-
[25]
Fangao Zeng et al. 2022. Motr: End-to-end multiple-object tracking with trans- former. InEuropean Conference on Computer Vision. Springer Nature Switzerland
2022
-
[26]
Yifu Zhang, Peize Sun, Yi Jiang, Dongdong Yu, Fucheng Weng, Zehuan Yuan, Ping Luo, Wenyu Liu, and Xinggang Wang. 2022. ByteTrack: Multi-Object Tracking by Associating Every Detection Box. arXiv:2110.06864 [cs.CV]
2022 arXiv
-
[27]
Yifu Zhang, Chunyu Wang, Xinggang Wang, Wenjun Zeng, and Wenyu Liu. 2021. FairMOT: On the Fairness of Detection and Re-identification in Multiple Object Tracking.International Journal of Computer Vision129, 11 (Sept. 2021), 3069–3087. https://doi.org/10.1007/s11263-021-01513-4...
2021 doi
-
[2016]
arXiv:1609.01775 [cs.CV]
Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking. arXiv:1609.01775 [cs.CV]
-
[2024]
Leveraging Foundation Models via Knowledge Distillation in Multi-Object Tracking: Distilling DINOv2 Features to FairMOT. arXiv:2407.18288 [cs.CV] https://arxiv.org/abs/2407.18288 A Framework for Multi-View Multiple Object Tracking using Single-View Multi-Object Trackers on Fis...
2018 arXiv
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.