REVIEW 4 major objections 4 minor 1 cited by
Cross-View Referring Multi-Object Tracking
T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read The paper proposes that referring multi-object tracking should operate on synchronized cross-view videos instead of a single view, and reports that its CRTracker method outperforms single-view baselines by 31.45% in CVRIDF1 on the new…
desk verdict A new cross-view referring MOT task and benchmark that are worth having, wrapped around a method whose SOTA claim rests on manually tuned thresholds without a validation split. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the fusion of cross-view tracking with language–attribute matching, capped by a prediction module. CRTracker reuses CrossMOT's detection and Re-ID heads (single-view, cross-view, full), merges APTM image encodings with the full Re-ID feature, and computes both text scores and attribute scores through APTM's cross encoder; the two scores are fused with a weight and fed, together with frame-to-frame association tracks, into a prediction module. That module treats tracks as detections and fusion scores as confidences, then uses three thresholds (average fusion score, single-view fusion score, hit score) to keep or drop each track. The same pipeline works for cross-domain evaluation without adaptation.
What would settle it
Re-run the comparison with a single-view referring tracker that uses a more powerful vision-language backbone (e.g., grounded-language models) plus the same MvMHAT association, or re-tune the prediction module thresholds on a held-out validation split instead of the test set; if CVRIDF1 margin drops below the claimed 31.45% or the ranking changes, the central claim would be weakened.
Extended reading notes
Core claim
The central claim is that cross-view observations solve the failure mode where a person's appearance in one camera does not show all attributes described in the language query. The authors argue that by associating the same person across overlapping views, the network can assemble a complete appearance, judge whether the description matches, and keep one identity across views. As evidence, they construct CRTrack from DIVOTrack and CAMPUS, annotate objects with invariant attributes (headwear, coat, trousers, shoes, held items, transportation), generate descriptions with GPT-4o, and define CVRIDF1 and CVRMA metrics that average per-language cross-view IDF1 and matching accuracy. On the in-domain test set, CRTracker reaches 54.88% CVRIDF1, far above the 23.30% of TransRMOT and 22.18% of TempRMOT; cross-domain, it still leads with 12.52% vs 3.78%. The proposed prediction module, which converts frame-to-frame association results and fused text–attribute scores into output trajectories, accounts for a 7.34% CVRIDF1 gain in the ablation.
Load-bearing premise
The reported edge assumes that adding MvMHAT to the two single-view trackers makes them fair, strong baselines, and that the fixed thresholds in the prediction module were not chosen by looking at the test set; if a stronger baseline or a validation-based threshold selection were used, the margin could shrink.
Editorial extensions
If this is right
- CRMOT becomes a new evaluation task: any future method that wants to score on CRTrack must handle synchronized multi-view input and per-view identity consistency, not just single-view referring.
- The observed margins imply that single-view referring trackers, when evaluated on occluded scenes, are penalized by invisible attributes; adding cross-view association to them (as done with MvMHAT) is not enough—the architecture needs to integrate cross-view appearance encoding.
- The prediction module's threshold-based filtering suggests that trajectory confidence can be derived from per-view fusion scores, which could be transplanted to other multi-view tracking or retrieval pipelines.
- The benchmark's long sequences (up to thousands of frames per description) stress temporal consistency, so future work can focus on long-range identity retention.
Reading between the lines
- Because the prediction module's thresholds are fixed constants (Tas=0.5, Tss=0.75, Ths=30), a natural extension is to learn them from a validation split; if the thresholds are overfit to the test scenes, the real cross-domain margin may be smaller than reported.
- The same cross-view aggregation idea could be applied to other vision-language grounding tasks such as referring expression segmentation or visual question answering under occlusion, not just tracking.
- The paper evaluates only two single-view baselines; the claim of 'state of the art' would be stronger with baselines from the cross-view MOT literature (e.g., ReST or other multi-camera trackers) adapted to the referring task, which the authors did not include.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a new task, Cross-view Referring Multi-Object Tracking (CRMOT), which extends single-view referring multi-object tracking to synchronized multi-view sequences, and introduces CRTrack, a benchmark built from DIVOTrack and CAMPUS with 13 scenes and 221 language descriptions. The authors also present CRTracker, an end-to-end method that combines CrossMOT and APTM with a hand-crafted prediction module that converts frame-to-frame association results and fusion scores into final trajectories. The paper reports that CRTracker outperforms two single-view RMOT baselines, TransRMOT and TempRMOT, adapted with MvMHAT, by large margins on in-domain and cross-domain test sets.
Significance. If the claims are supported, the paper makes a useful contribution by identifying a new task formulation, releasing a benchmark with 82K frames and 221 descriptions, and providing a working baseline and open code. The motivation that multi-view observation can resolve appearance ambiguity in referring tasks is plausible, and the dataset annotation pipeline using invariant attributes plus GPT-4o is a practical method. The reported gains are large, but the evaluation protocol needs strengthening regarding hyperparameter validation, baseline selection, and metric construction before the state-of-the-art claim is fully convincing.
major comments (4)
- [Experiments / Settings] The central SOTA claim rests on hyperparameters that are not validated. In Section 4 (Settings) and Algorithm 1, the fusion weights α=0.01, β=0.1 and thresholds Tas=0.5, Tss=0.75, Ths=30, s1=3, s2=3, s3=1 are fixed without a validation split or sensitivity analysis. Because the reported margins (31.45% CVRIDF1 and 25.83% CVRMA in-domain; 8.74% and 1.92% cross-domain) are the principal claim, this is load-bearing. Please add a validation split, report sensitivities over plausible ranges, and state how the constants were chosen.
- [Experiments / Quantitative Results (Table 2)] The comparison in Table 2 is limited to two single-view RMOT methods (TransRMOT and TempRMOT) each adapted with MvMHAT. The abstract and conclusion claim "state-of-the-art", but no cross-view MOT baseline (e.g., CrossMOT itself) or two-stage RMOT method (e.g., iKUN or LaMOT) is evaluated. If one of these were competitive, the claimed superiority could fail. Please add at least one strong cross-view baseline and, where feasible, a two-stage RMOT baseline; report results over multiple runs with error bars to establish statistical significance.
- [Evaluation Metrics (Eq. 4)] Equation (4) truncates each CVMA at zero before averaging, with max(CVMA, 0). This discards negative values that indicate a large number of false positives, so CVRMA can overstate performance and is not strictly comparable across methods with different false-positive behavior. Please report the untruncated average as a secondary metric, and state how many language descriptions are affected by the truncation in the reported numbers.
- [Benchmark / Dataset Annotation] The benchmark contribution is a core part of the paper, but the annotation section provides no measure of agreement or ambiguity. The authors state that descriptions were "carefully" manually checked, yet no inter-annotator agreement, description-level statistics, or examples of ambiguous descriptions are given. Please add a small human-evaluation study or at least per-description statistics to support the validity of the 221 language descriptions as ground truth.
minor comments (4)
- [Table 1] The column header "A verage Number of frames" contains a spacing typo and should be "Average Number of Frames per View" for clarity.
- [Section 4] The text lists several hyperparameters in running prose; a compact notation table or list would improve reproducibility, especially since Algorithm 1 references many threshold symbols.
- [Figure 4] The pipeline diagram is dense; labeling the components and adding a legend for the frozen, merging, and extraction symbols would aid readability.
- [Supplementary Material / Questions and Replies] The "Questions and Replies" section reads like reviewer responses embedded in the paper; integrating this content into a limitations or discussion section would make the paper more self-contained.
Circularity Check
No significant circularity: the paper's central claim rests on an independently testable benchmark and external baselines, not on a derivation that reduces to its own inputs.
full rationale
The paper contains no derivation whose conclusion is built into its assumptions. CRTracker combines existing components (CrossMOT and APTM) and the only closed-form equations are feature fusion (Eq. 5), score fusion (Eq. 9), and standard loss definitions (Eqs. 6-8); none of these equations defines the reported metrics or output trajectories in terms of themselves. The CVRIDF1/CVRMA metrics are new but are explicitly defined as averages of existing CVIDF1 and clamped CVMA, so they are transparent evaluation tools rather than circular constructions. The state-of-the-art claim is assessed on held-out in-domain and cross-domain test scenes against external baselines (TransRMOT and TempRMOT), which gives the performance claim independent empirical content. The manually fixed thresholds in Algorithm 1 and the fusion weights alpha and beta are hyperparameters; the absence of a validation split is a legitimate benchmark-design and robustness concern, but it is not circular in the specific sense of a fitted parameter being renamed as a prediction. The few self-citations in the related-work list are not load-bearing for the central claim. Overall, no circular step meeting the quoted-reduction standard was found.
Assumptions & free parameters
free parameters (3)
- feature fusion weight alpha =
0.01
- score fusion weight beta =
0.1
- prediction module thresholds and hit scores =
Tas=0.5, Tss=0.75, Ths=30, s1=3, s2=3, s3=1
assumptions (5)
- domain assumption The single-view appearance of objects is often invisible, causing incorrect language-object matching; cross-view observation resolves this.
- domain assumption Existing DIVOTrack and CAMPUS training annotations and synchronization are reliable enough to build a referring benchmark on top of them.
- domain assumption APTM text-image matching features transfer to cross-view person tracks and can be fused with CrossMOT re-ID features.
- domain assumption Frame-to-frame association by MvMHAT provides correct tracklets as inputs to the prediction module.
- domain assumption The CRTrack evaluation metrics CVRIDF1 and CVRMA are valid measures of the task.
Cite this review
Pith. "Pith review of Cross-View Referring Multi-Object Tracking." pith.science (2026). https://pith.science/paper/YVE6VSIH
@misc{pith2026241217807,
author = {Pith},
title = {Pith review of: Cross-View Referring Multi-Object Tracking},
year = {2026},
howpublished = {\url{https://pith.science/paper/YVE6VSIH}},
note = {Machine review of arXiv:2412.17807}
}
read the original abstract
Referring Multi-Object Tracking (RMOT) is an important topic in the current tracking field. Its task form is to guide the tracker to track objects that match the language description. Current research mainly focuses on referring multi-object tracking under single-view, which refers to a view sequence or multiple unrelated view sequences. However, in the single-view, some appearances of objects are easily invisible, resulting in incorrect matching of objects with the language description. In this work, we propose a new task, called Cross-view Referring Multi-Object Tracking (CRMOT). It introduces the cross-view to obtain the appearances of objects from multiple views, avoiding the problem of the invisible appearances of objects in RMOT task. CRMOT is a more challenging task of accurately tracking the objects that match the language description and maintaining the identity consistency of objects in each cross-view. To advance CRMOT task, we construct a cross-view referring multi-object tracking benchmark based on CAMPUS and DIVOTrack datasets, named CRTrack. Specifically, it provides 13 different scenes and 221 language descriptions. Furthermore, we propose an end-to-end cross-view referring multi-object tracking method, named CRTracker. Extensive experiments on the CRTrack benchmark verify the effectiveness of our method. The dataset and code are available at https://github.com/chen-si-jia/CRMOT.
Figures
Figures from the paper (13 more)
Forward citations
Cited by 1 Pith paper
-
CrowdTrack: A Benchmark for Difficult Multiple Pedestrian Tracking in Real Scenarios
CrowdTrack is a dense, first-person-view pedestrian tracking benchmark that exposes large performance drops in existing multi-object trackers.
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Bashar, M.; Islam, S.; Hussain, K. K.; Hasan, M. B.; Rahman, A.; and Kabir, M. H. 2022. Multiple object tracking in recent times: A literature review. arXiv preprint arXiv:2209.04796
arXiv 2022
-
[4]
Chavdarova, T.; Baqu \'e , P.; Bouquet, S.; Maksai, A.; Jose, C.; Bagautdinov, T.; Lettry, L.; Fua, P.; Van Gool, L.; and Fleuret, F. 2018. Wildtrack: A multi-camera hd dataset for dense unscripted pedestrian detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, 5030--5039
work page 2018
-
[5]
Chen, S.; Yu, E.; Li, J.; and Tao, W. 2024. Delving into the Trajectory Long-tail Distribution for Muti-object Tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19341--19351
work page 2024
-
[6]
Cheng, C.-C.; Qiu, M.-X.; Chiang, C.-K.; and Lai, S.-H. 2023. Rest: A reconfigurable spatial-temporal graph model for multi-camera multi-object tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 10051--10060
work page 2023
-
[7]
Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805
arXiv 2018
-
[8]
Du, Y.; Lei, C.; Zhao, Z.; and Su, F. 2024. ikun: Speak to trackers without retraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19135--19144
work page 2024
Show all 30 references
-
[9]
Fleuret, F.; Berclaz, J.; Lengagne, R.; and Fua, P. 2007. Multicamera people tracking with a probabilistic occupancy map. IEEE transactions on pattern analysis and machine intelligence, 30(2): 267--282
2007
-
[10]
Gan, Y.; Han, R.; Yin, L.; Feng, W.; and Wang, S. 2021. Self-supervised multi-view multi-human association and tracking. In Proceedings of the 29th ACM international conference on multimedia, 282--290
2021
-
[11]
Gao, R.; Zhang, Y.; and Wang, L. 2024. Multiple Object Tracking as ID Prediction. arXiv preprint arXiv:2403.16848
2024 arXiv
-
[12]
Hao, S.; Liu, P.; Zhan, Y.; Jin, K.; Liu, Z.; Song, M.; Hwang, J.-N.; and Wang, G. 2024. Divotrack: A novel dataset and baseline method for cross-view multi-object tracking in diverse open scenes. International Journal of Computer Vision, 132(4): 1075--1090
2024
-
[13]
P.; and Ba, J
Kingma, D. P.; and Ba, J. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980
2014 arXiv
-
[14]
Li, S.; Ke, L.; Danelljan, M.; Piccinelli, L.; Segu, M.; Van Gool, L.; and Yu, F. 2024 a . Matching Anything by Segmenting Anything. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18963--18973
2024
-
[15]
Li, X.; Liu, D.; Zhao, L.; Wu, Y.; Wu, X.; and Gao, J. 2024 b . Fast-Poly: A Fast Polyhedral Framework For 3D Multi-Object Tracking. arXiv preprint arXiv:2403.13443
2024 arXiv
-
[16]
Li, Y.; Liu, X.; Liu, L.; Fan, H.; and Zhang, L. 2024 c . LaMOT: Language-Guided Multi-Object Tracking. arXiv preprint arXiv:2406.08324
2024 arXiv
-
[17]
Liu, Z.; Lin, Y.; Cao, Y.; Hu, H.; Wei, Y.; Zhang, Z.; Lin, S.; and Guo, B. 2021. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, 10012--10022
2021
-
[18]
OpenAI. 2024. GPT-4o: OpenAI's language model. https://openai.com/blog/hello-gpt-4o/
2024
-
[19]
Wu, D.; Han, W.; Wang, T.; Dong, X.; Zhang, X.; and Shen, J. 2023. Referring multi-object tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 14633--14642
2023
-
[20]
Xu, Y.; Liu, X.; Liu, Y.; and Zhu, S.-C. 2016. Multi-view people tracking via hierarchical trajectory composition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 4256--4265
2016
-
[21]
Yang, S.; Zhou, Y.; Zheng, Z.; Wang, Y.; Zhu, L.; and Wu, Y. 2023. Towards unified text-based person retrieval: A large-scale multi-attribute and language search benchmark. In Proceedings of the 31st ACM International Conference on Multimedia, 4492--4501
2023
-
[22]
Yi, K.; Luo, K.; Luo, X.; Huang, J.; Wu, H.; Hu, R.; and Hao, W. 2024. Ucmctrack: Multi-object tracking with uniform camera motion compensation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 6702--6710
2024
-
[23]
Yu, E.; Li, Z.; Han, S.; and Wang, H. 2022. Relationtrack: Relation-aware multiple object tracking with decoupled representation. IEEE Transactions on Multimedia, 25: 2686--2697
2022
-
[24]
Yu, E.; Liu, S.; Li, Z.; Yang, J.; Li, Z.; Han, S.; and Tao, W. 2023 a . Generalizing multiple object tracking to unseen domains by introducing natural language representation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 3304--3312
2023
-
[25]
Yu, E.; Wang, T.; Li, Z.; Zhang, Y.; Zhang, X.; and Tao, W. 2023 b . Motrv3: Release-fetch supervision for end-to-end multi-object tracking. arXiv preprint arXiv:2305.14298
2023 arXiv
-
[26]
Zeng, F.; Dong, B.; Zhang, Y.; Wang, T.; Zhang, X.; and Wei, Y. 2022. Motr: End-to-end multiple-object tracking with transformer. In European Conference on Computer Vision, 659--675. Springer
2022
-
[27]
Zhang, Y.; Wang, C.; Wang, X.; Zeng, W.; and Liu, W. 2021. Fairmot: On the fairness of detection and re-identification in multiple object tracking. International journal of computer vision, 129: 3069--3087
2021
-
[28]
Zhang, Y.; Wu, D.; Han, W.; and Dong, X. 2024. Bootstrapping Referring Multi-Object Tracking. arXiv preprint arXiv:2406.05039
2024
-
[29]
Zhang, Z.; and Sabuncu, M. 2018. Generalized cross entropy loss for training deep neural networks with noisy labels. Advances in neural information processing systems, 31
2018
-
[30]
a henb \
Zhou, X.; Wang, D.; and Kr \"a henb \"u hl, P. 2019. Objects as points. arXiv preprint arXiv:1904.07850
2019 arXiv
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.