{"id":"107ffbcb-a5c9-4ff5-b15b-a86051879bd5","arxiv_id":"2501.00843","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"A systematic comparison of four cost-fusion methods for data association in online multi-object tracking shows that fusion choice materially affects MOTA, IDF1, and HOTA on MOT17, MOT20, and DanceTrack validation sets.","lead":"This paper compares four ways of combining motion, appearance, and weak cues when matching detected objects to tracked objects in video. It finds that the choice of fusion method changes tracking accuracy, with weighted sum and Kalman gating using weak cues better than minimum or Hadamard fusion.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fusion comparison is confounded (different motion metric, thresholds, and weight scales), and the 'weighted sum favorable with weak cues' claim is contradicted by KF gating on MOT17/MOT20.","rationale":"The paper's stated goal is to show that the fusion method matters and, more specifically, that weighted-sum IoU fusion is preferable when weak cues are added. For that conclusion to hold, the comparison must vary only the fusion operator. The paper instead varies several inseparable components: KF gating uses Mahalanobis distance while the other methods use IoU; KF gating has no threshold masks while minimum/Hadamard do; and the combination weights are on completely different scales (0.98 vs 0.1-1.0). Even within the reported numbers, the key qualitative claim is inconsistent: KF gating with all four cues outperforms weighted-sum on both MOT17 and MOT20 by HOTA, so 'weighted sum is more favourable' is only true on average because of DanceTrack. This is a load-bearing problem because it directly undermines the abstract/conclusion recommendation, not merely the exact magnitude of the gains. The reader's concern about hand-set weights is valid and related, but the deeper issue is that the experimental protocol does not isolate the fusion method. I still do not recommend rejection: the source code is available, the validation-only evaluation is transparently reported, and a controlled ablation could settle the question. The conditional verdict should stand; no further verdict change is needed.","tokens_in":13414,"tokens_out":7341,"duration_ms":64625,"concrete_test":"Run an ablation with the released code in which all four fusion operators receive identical cost matrices (same IoU motion cost, same cosine appearance cost, same hIoU and confidence costs), identical threshold masks, and identical normalized weights; if the ranking changes or KF gating no longer beats weighted sum with weak cues on MOT17/MOT20, the reported preference is an artifact of the confounded setup.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Tables I-III do not isolate the fusion operator. KF gating (Eq. 19) uses Mahalanobis motion and C = 0.98(Ca + 0.2Ch + 0.2Cc) + 0.02Cm, while the other methods use IoU motion with C = 1.0Cm + 0.1Ca + 0.1Ch + 0.1Cc (Eq. 18). Minimum and Hadamard apply the threshold masks of Eqs. (10)-(12); weighted sum applies only the appearance threshold; KF gating applies no thresholding or 0.5 multiplication. Observed gaps can therefore be caused by the motion metric, threshold policy, or relative weight scale, not by the fusion rule itself. The specific recommendation that weighted sum is more favourable with weak cues is also not consistently supported by the paper's own tables: with all cues, KF gating beats weighted sum on MOT17 (HOTA 69.366 vs 68.812) and MOT20 (58.695 vs 58.368), while weighted sum wins only on DanceTrack. Thus the headline ranking is not established by the reported experiments.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FusionSORT, an online tracking-by-detection framework, and uses it to compare four fusion methods for combining strong cues (motion, appearance) with weak cues (height-IoU, tracklet confidence) in the data-association step. The four methods are minimum fusion, weighted sum based on IoU, Kalman-filter (KF) gating with Mahalanobis distance, and Hadamard product of cost matrices. Experiments are conducted on the validation sets of MOT17, MOT20, and DanceTrack, reporting MOTA, IDF1, and HOTA for different cue combinations. The central claim is that the choice of fusion method matters and that the weighted sum based on IoU is more favorable when weak cues are added to strong cues. The paper also introduces a confidence state into the Kalman-filter state vector and releases source code.","tokens_in":13734,"tokens_out":3978,"duration_ms":34631,"significance":"If the comparison were clean, this study would offer useful practical guidance to the tracking community, since many trackers adopt different fusion rules without a systematic comparison. The paper's strengths include its breadth across three benchmarks, the use of publicly available detectors and re-identification models, the absence of tracklet interpolation, and a supplementary comparison of IoU versus Mahalanobis distance for the second association stage (Table IV). The code is publicly available, which supports reproducibility. However, the empirical comparison does not isolate the fusion operator from other design choices, and the headline conclusion is not consistently supported by the paper's own tables. These issues substantially weaken the central claims in their current form.","major_comments":[{"comment":"The four fusion methods are not compared under controlled conditions, so observed differences cannot be attributed to the fusion operator alone. KF gating (Eq. 19) uses Mahalanobis distance as the motion cost, whereas the other three methods use IoU (Eq. 18, Eq. 20, and the minimum method). The threshold policy also differs: minimum and Hadamard apply the masks of Eqs. (10)-(12), weighted sum applies only the appearance threshold, and KF gating applies no thresholding or 0.5 multiplication. The weight scales differ as well (λ1=1.0, λ2=0.1 for weighted sum versus λ=0.98, λh=λc=0.2 for KF gating). To substantiate the claim that fusion method is key, the paper needs a controlled ablation that keeps the motion metric, threshold policy, and weight scales fixed while changing only the fusion rule, or at least disentangles these effects through a sensitivity analysis.","section":"§III.C, Tables I-III"},{"comment":"The conclusion that 'the weighted sum based on IoU is more favourable when using weak cues along with the strong cues' is contradicted by the reported results. On MOT17 (Table I), the weighted-sum HOTA with all cues is 68.812, which is lower than the 69.379 obtained without weak cues, and KF gating achieves a higher HOTA (69.366) than weighted sum (68.812) when all cues are used. On MOT20 (Table II), KF gating with all cues reaches 58.695 HOTA against 58.368 for weighted sum. Weak cues also degrade KF gating on DanceTrack (Table III: 49.08 with mot+app versus 48.695 with all cues). The per-dataset behavior should be reported honestly, and the global conclusion should be reformulated accordingly or supported by additional evidence.","section":"§V Conclusion, Tables I and II"},{"comment":"All reported numbers are from single runs without error bars, significance tests, or sensitivity analyses, while some of the key differences are very small (for example, 69.366 versus 68.812 HOTA on MOT17). The fusion weights (λ1...λ4), thresholds (θiou, θemb), and detection score thresholds (τ1, τ2) were selected empirically on the same validation sets used to compare the methods, so the ranking could shift under different parameter choices. The paper should include a sensitivity analysis over these parameters and, where feasible, multiple runs or per-sequence statistics to assess whether the observed differences are robust.","section":"§IV.B, Tables I-III"},{"comment":"The Hadamard-product fusion is described as treating all cues with equal importance implicitly, but this is not fully accurate in the implementation because the masks of Eqs. (10)-(12) are applied to the appearance, height-IoU, and confidence costs before multiplication. This means the Hadamard method also includes a thresholding policy that the weighted-sum and KF-gating methods do not share. The paper should clarify how the threshold masks affect the scale and meaning of the Hadamard cost, and whether the observed degradation of Hadamard is due to the multiplicative combination itself or to the interaction with the threshold masks.","section":"§III.C.4, Eq. (20)"}],"minor_comments":[{"comment":"The confidence cost is defined twice, in Eq. (9) and Eq. (17); the earlier definition could be removed to avoid redundancy.","section":"§III.B, Eq. (9) and Eq. (17)"},{"comment":"The sentence 'we did not apply any thresholding and 0.5 multiplication as in (10), (11), and (12) in this fusion method' would benefit from a brief explanation of why the 0.5 multiplication and the IoU/appearance threshold masks are omitted for KF gating, since the reader may otherwise suspect an unfair comparison.","section":"§III.C.3, Eq. (19)"},{"comment":"The claim that 'the choice of a fusion method is key' is vague; it would be more informative to state the conditions (cue combinations, dataset characteristics) under which each fusion method is preferred, given the per-dataset variability reported in the tables.","section":"Abstract and Introduction"},{"comment":"In the description of Table I, the statement that weak cues improve KF gating is correct for HOTA and IDF1; however, the corresponding MOTA decreases from 78.035 to 78.003, so the description could note the metric-dependent nature of the improvement.","section":"§IV.B.1, Table I"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant and practical question, and the experimental setup is largely reproducible. However, the central comparison is confounded by differences beyond the fusion operator, and the headline conclusion is contradicted by parts of the paper's own tables. I would like to see a redesigned experiment that isolates the fusion rule (while controlling motion metric, thresholds, and weights), a sensitivity analysis for the hand-set parameters, and a more nuanced conclusion that reports per-dataset behavior. With those changes, the study could make a useful contribution; in the current form, the evidence does not support the stated claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a useful, transparently-written empirical study, but the stress-test note is correct—the four fusion methods are not compared on equal terms, and the paper's preferred conclusion ('weighted sum is more favourable with weak cues') is not supported by its own tables.\n\nWhat's actually new: the fusion operators themselves are all borrowed from prior work (minimum in BoT-SORT, weighted sum in ByteTrack/Hybrid-SORT, KF gating in StrongSORT, Hadamard in POI/GM-PHD), and the author says so. The contribution is the head-to-head comparison of these four operators in one SORT-style tracker with strong cues (motion, appearance) and weak cues (height-IoU, confidence), across three datasets. That comparison doesn't exist in the cited literature as far as I can tell. The paper also ships code, uses public detectors/re-ID models, does no tracklet interpolation, and clearly labels the evaluation as validation-only. Credit where due: the experiment is easy to reproduce in spirit.\n\nWhat's soft: the comparison is confounded. KF gating uses Mahalanobis distance for motion and a different weight scale (C = 0.98(Ca + 0.2Ch + 0.2Cc) + 0.02Cm), while the other three use IoU motion and C = 1.0Cm + 0.1Ca + 0.1Ch + 0.1Cc. Minimum and Hadamard apply threshold masks (Eqs. 10-12); weighted sum only the appearance threshold; KF gating applies none. So the observed gaps between 'fusion methods' are really gaps between the whole association pipelines. The paper's own numbers undercut the abstract: with all cues, KF gating beats weighted sum on MOT17 (HOTA 69.366 vs 68.812) and MOT20 (58.695 vs 58.368); weighted sum wins only on DanceTrack. So the specific recommendation is not established.\n\nAlso, single runs with no error bars or significance tests, and the fusion weights and thresholds were chosen on the same validation sets used to compare methods, without a sensitivity analysis. That makes the exact ranking provisional, though not worthless.\n\nNet: the broader claim—that the choice of fusion operator matters and that min/Hadamard degrade when weak cues are added while weighted-sum/KF-gating can gain—is consistently supported across the three datasets and is a useful practical lead. But the paper would need a deconfounded design (same motion metric, same threshold policy, per-method weight sensitivity) before I'd trust the specific ordering.\n\nRecommendation: send it to peer review. The question is legitimate, the code and protocol are clear, and the flaws are fixable in revision rather than fatal. I wouldn't cite the ranking as-is, but I'd consider bringing it to reading group as a case study in evaluation confounds.","headline":"An honest but confounded empirical comparison of fusion operators for SORT-style tracking; the headline ranking overreaches the data.","tokens_in":14246,"tokens_out":2466,"would_cite":false,"duration_ms":20648,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper shows that in online multi-object visual tracking, the choice of fusion method for combining motion, appearance, height-IoU, and confidence costs determines whether weak cues improve or degrade association performance.","keywords":["multi-object tracking","data association","fusion methods","motion and appearance cues","height-IoU","tracklet confidence","Kalman filter gating","SORT"],"falsifier":"Re-run the same four fusion methods on the same validation splits while sweeping the fusion weights (for example, varying the appearance weight from 0.05 to 0.5 and the height-IoU and confidence weights from 0 to 0.3) and the matching thresholds; if the weighted-sum advantage over minimum disappears under a different weight setting, the conclusion that the fusion method is the key factor would be shown to be parameter-dependent rather than intrinsic.","tokens_in":13223,"feed_emoji":"🎯","tokens_out":7268,"duration_ms":61424,"temperature":0.7,"pith_summary":"This paper argues that the way detection-to-tracklet costs are fused, the fusion method, matters as much as which cues are used in online multi-object visual tracking. It builds a SORT-style tracker and compares four fusion recipes: taking the minimum of per-cue costs, a weighted sum with IoU-based motion, Kalman-filter gating with Mahalanobis motion, and an element-wise (Hadamard) product of costs. On validation splits of MOT17, MOT20, and DanceTrack, the paper finds that the weighted sum based on IoU is the most favorable when weak cues (height-IoU and tracklet confidence) are combined with strong cues, while minimum fusion works for motion plus appearance but degrades once weak cues are added. The intended takeaway is practical guidance for choosing the right fusion method for a given cue set in data association.","feed_headline":"Fusion method choice decides multi-object tracker accuracy","feed_subtitle":"Weighted IoU sum wins with weak cues; minimum fusion only suits motion plus appearance","key_machinery":"The load-bearing object is the fused cost matrix $C$ whose $(i,j)$ entry is the combined association cost between tracklet $i$ and detection $j$. The paper builds four versions of $C$ from four per-cue cost matrices: appearance cosine distance, IoU distance, height-IoU distance (one-dimensional overlap of the boxes' vertical extent), and confidence distance (absolute difference between the tracklet confidence state and the detection score). The fusion operators are $\\min(C_a,C_m,C_h,C_c)$, the weighted sum $\\lambda_1 C_m + \\lambda_2 C_a + \\lambda_3 C_h + \\lambda_4 C_c$, KF gating $C = \\lambda(C_a + \\lambda_h C_h + \\lambda_c C_c) + (1-\\lambda)C_m$ using Mahalanobis motion subject to gating, and the element-wise product $C_a \\odot C_m \\odot C_h \\odot C_c$. The same Hungarian solver performs the assignment in all cases, so performance differences are attributed to the fusion operator itself.","core_discovery":"The paper's central claim is that the fusion method is at least as important as the cues themselves for data association in multi-object visual tracking. Using a tracking-by-detection pipeline with a Kalman filter whose state vector is extended with tracklet confidence and its velocity, the author compares four fusion operators on the validation sets of MOT17, MOT20, and DanceTrack: minimum, weighted sum based on IoU, Kalman-filter gating with Mahalanobis distance, and Hadamard product. The consistent pattern is that minimum and Hadamard fusion benefit from adding appearance to motion but lose performance when height-IoU and confidence cues are added, whereas weighted-sum based on IoU and KF gating can exploit those weak cues. On DanceTrack, KF gating trails the other methods overall, so the author recommends weighted sum based on IoU as the favourable choice when weak cues accompany strong cues.","pith_inferences":["A natural extension the paper does not run is a weight-sensitivity sweep: the recommendation would be stronger if the weighted-sum advantage persisted across a range of $\\lambda$ values, since the current weights are fixed on the same validation sets used for comparison.","The conclusion may transfer to other tracking-by-detection systems only insofar as their detectors and appearance embeddings have similar error profiles; testing with a different detector or re-identification model would show whether fusion choice interacts with cue quality.","Because height-IoU and confidence costs are nearly free to compute, the weighted-sum result suggests a practical recipe: keep the strong motion term dominant and use weak cues as small tie-breakers rather than as equal partners.","One could test the fusion methods on held-out test sets or on other benchmarks with nonlinear motion to see whether the DanceTrack ranking generalizes beyond the three validation sets reported."],"forward_implications":["Trackers that currently fuse many cues with minimum or Hadamard product can expect an association-quality gain by switching to an IoU-based weighted sum, since weak cues then contribute instead of hurting.","The two-stage association scheme can keep plain IoU in the low-confidence second stage; fusion is only needed where appearance is available, so practitioners do not pay fusion cost on low-confidence boxes.","KF gating's weak-cue gains on crowded scenes and its lower overall results on the diverse-motion benchmark indicate that the choice between IoU-based and Mahalanobis-based motion fusion should depend on the motion pattern of the scene.","Extending the Kalman state with confidence and its velocity is a cheap addition that lets a tracker use detection score as a smoothly predicted association cue rather than a per-frame threshold.","On uniform-appearance, diverse-motion scenes, minimum fusion with motion plus appearance remains competitive, so the simple minimum rule is still a reasonable default when only strong cues are used."],"supporting_citations":[{"why":"Establishes the simple online realtime tracking-by-detection baseline with Kalman-filter motion and IoU association that the paper extends.","marker":"[1]"},{"why":"Supplies the deep appearance-association lineage used for the appearance cost and the Kalman-gating association style.","marker":"[2]"},{"why":"Supplies the Kalman-gating fusion variant and the noise-scale-adaptive Kalman idea that the paper tests and does not adopt.","marker":"[3]"},{"why":"Contributes the two-stage association of high- and low-confidence detections and the detector weights used for the MOT benchmarks.","marker":"[4]"},{"why":"Provides the minimum fusion rule for motion and appearance, camera-motion compensation, and the appearance embedding baseline used in the experiments.","marker":"[5]"},{"why":"Introduces the weak cues (height and confidence) and supplies the detector and appearance models used for the DanceTrack experiments.","marker":"[18]"},{"why":"Provides the strong appearance-embedding baseline used for computing the cosine appearance cost.","marker":"[30]"},{"why":"Defines the MOT16/MOT17 benchmark whose training split is halved to form the MOT17 validation set.","marker":"[32]"},{"why":"Defines the MOT20 crowded-scene benchmark whose training split is halved to form the MOT20 validation set.","marker":"[33]"},{"why":"Defines the DanceTrack benchmark and its separate validation set used for the diverse-motion evaluation.","marker":"[34]"}],"fun_headline_variants":["Fusion method outranks cue choice in tracking","Weighted IoU fusion exploits weak cues best","Minimum fusion only suits strong cues","Fusion choice beats cue selection in trackers","Pick fusion by cue strength in multi-object tracking"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported ranking assumes the hand-set fusion weights, matching rejection thresholds, and detection score thresholds, chosen on the same validation sets used for comparison, are equally fair to all four fusion methods; the paper does not test how sensitive the ranking is to those parameters.","fun_headline_variants_meta":{"raw":{"variants":["Fusion method outranks cue choice in tracking","Weighted IoU fusion exploits weak cues best","Minimum fusion only suits strong cues","Fusion choice beats cue selection in trackers","Pick fusion by cue strength in multi-object tracking"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000219,"raw_usage":{"total_tokens":1398,"prompt_tokens":858,"completion_tokens":540,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":474,"completion_tokens_details":{"reasoning_tokens":471}},"tokens_in":474,"tokens_out":540,"duration_ms":5491,"temperature":1.0,"reasoning_tokens":471,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:40:28.020496+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same four fusion methods on the same validation splits while sweeping the fusion weights (for example, varying the appearance weight from 0.05 to 0.5 and the height-IoU and confidence weights from 0 to 0.3) and the matching thresholds; if the weighted-sum advantage over minimum disappears under a different weight setting, the conclusion that the fusion method is the key factor would be shown to be parameter-dependent rather than intrinsic.","supporting_citations":[{"cited_title":"Simple online and rea ltime tracking with a deep association metric,","cited_arxiv_id":null,"evidence_quote":"Supplies the deep appearance-association lineage used for the appearance cost and the Kalman-gating association style."},{"cited_title":"ByteTrack: Multi-object tracking by associat ing every detection box,","cited_arxiv_id":null,"evidence_quote":"Contributes the two-stage association of high- and low-confidence detections and the detector weights used for the MOT benchmarks."},{"cited_title":"Hybrid-SORT: Weak cues matter for online multi-object tra cking,","cited_arxiv_id":null,"evidence_quote":"Introduces the weak cues (height and confidence) and supplies the detector and appearance models used for the DanceTrack experiments."},{"cited_title":"Bag of tricks and a strong baseline for deep person re-identification,","cited_arxiv_id":null,"evidence_quote":"Provides the strong appearance-embedding baseline used for computing the cosine appearance cost."},{"cited_title":"DanceTrack: Multi-object tracking in uniform appearance and diverse motion,","cited_arxiv_id":null,"evidence_quote":"Defines the DanceTrack benchmark and its separate validation set used for the diverse-motion evaluation."}],"review_version":1}