{"id":"8407ec69-ea50-4d51-8517-8e4175aff845","arxiv_id":"2505.01257","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"CAMELTrack is an online tracker whose association step is learned end to end from multiple cues, reaching state-of-the-art HOTA on DanceTrack, SportsMOT, PoseTrack21 and BEE24, and competitive results on MOT17.","lead":"This paper introduces CAMEL, a trainable module that replaces the hand-written rules used to link detections into tracks in online multi-object tracking. It reports state-of-the-art results on several tracking benchmarks while training in about one hour on a single GPU.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central transfer assumption behind CAMEL's learned association is untested: the synthetic cross-video training scenarios of Sec. 3.3 may not match real occlusion, re-entry, and detector-error distributions, and the reported gains are not isolated from architecture or training-scheme effects.","rationale":"The reader's weakest_assumption is the same concern I identified: ACT builds image-free, cross-video synthetic samples and assumes they are representative enough for the learned context-aware fusion to transfer to real online tracking. This is load-bearing because the central claim is that a learned, cue-agnostic association module outperforms heuristic association; if the training distribution is the source of the gains, the architecture itself is not validated. The paper's ablations support the contribution of each component, and the code and weights are publicly available, which is credit-worthy. However, no multi-seed variance is reported, and the BEE24 experiment, while encouraging, does not isolate transfer from dataset-specific adaptation. The proposed test directly separates architecture benefits from training-distribution benefits. A REJECT verdict would be too strong, since the concern is addressable and the empirical evidence is otherwise credible; CONDITIONAL remains the right verdict.","tokens_in":26794,"tokens_out":7257,"duration_ms":83631,"concrete_test":"Train CAMEL with identical architecture, cues, and hyperparameters on DanceTrack and SportsMOT under two protocols: (i) the paper's ACT (Sec 3.3) and (ii) a recursive real-frame protocol that samples consecutive frames from the same training sequences with the same detections, ground-truth identities, and InfoNCE objective. Evaluate both on each validation set using identical thresholds. If ACT's HOTA advantage over recursive training is within 1 point, transfer is not the source of the gains; if ACT is worse or the margin exceeds 3 points, the reported results are confounded by the synthetic training distribution. Additionally, train on DanceTrack and evaluate zero-shot on SportsMOT validation (and vice versa) to quantify cross-domain transfer.","verdict_should_be":"UNCHANGED","load_bearing_attack":"CAMEL's state-of-the-art claims rest on the assumption that association-centric training (ACT), which builds each sample by mixing detections and tracklets from random frames and distinct videos, produces contexts representative enough of real online tracking that the learned context-aware fusion transfers. The paper does not test this directly. The ablations in Table 5 compare ACT-trained CAMEL against heuristic baselines, but never against the same architecture trained with a recursive real-frame protocol or against an ACT model evaluated on a different dataset without fine-tuning. The BEE24 result is not a controlled transfer test: it uses only bounding-box cues and is adapted to that dataset. If ACT's synthetic identity-swap, dropout, and cross-video contexts differ systematically from real occlusions, re-entries, and detector failures, the learned association strategy could be overfitting to the training distribution, and the reported +3 to +7 HOTA gains on DanceTrack, SportsMOT, and PoseTrack21 would not generalize. This is a correctness risk, not a novelty dispute.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CAMEL, a transformer-based association module for online multi-object tracking that replaces three classic heuristics (tracklet representation, feature fusion, and multi-stage matching) with a single trainable architecture. CAMELTrack, the full tracking-by-detection pipeline, extracts multiple cues (bounding box, ReID appearance, pose keypoints) via off-the-shelf models, encodes each cue temporally, then fuses them in a group-aware self-attention encoder to produce a shared embedding space for Hungarian matching. A novel Association-Centric Training (ACT) scheme trains the module on pre-computed, image-free scenarios that mix detections and tracklets across frames and videos, with identity-swap, dropout, and perturbation augmentations. The paper reports state-of-the-art results on DanceTrack, SportsMOT, PoseTrack21, and BEE24, and competitive results on MOT17, with training under one hour on a single GPU and code/weights released.","tokens_in":27020,"tokens_out":6788,"duration_ms":62523,"significance":"If the results hold, this is a valuable step: it shows that learned association can match or beat hand-crafted association rules while preserving the modularity of tracking-by-detection, and the low training cost and cue-agnostic design make the approach broadly applicable. The paper ships released code, model weights, detections, and detailed configuration files, which is a strong positive for reproducibility. The ablation study in Table 5 is carefully designed and cleanly supports the role of the temporal encoders, group-aware fusion, and data augmentation. The oracle-based analysis provides an intuitive upper-bound framing.","major_comments":[{"comment":"The claim that CAMELTrack 'outperforms both E2E and SORT-based methods across five challenging benchmarks' is not supported by Table 4: on MOT17, the SORT-based ByteTrack and GHOST report HOTA 62.8, above CAMELTrack's 62.4. The abstract's 'state-of-the-art performance on multiple tracking benchmarks' should be qualified, e.g., by naming the benchmarks where SOTA is achieved, and the bullet should be corrected to avoid a false claim.","section":"Section 1, contribution bullet 3, Table 4"},{"comment":"The central transfer assumption of ACT is that synthetic scenarios built from random frames and cross-video mixing, with identity-swap and dropout augmentations, are representative of real online contexts. This is not tested directly. Please add a controlled experiment, such as training on one dataset (e.g., DanceTrack) and evaluating on another (e.g., SportsMOT or BEE24) without fine-tuning, or compare ACT-trained CAMEL against the same architecture trained with a recursive real-frame protocol. Without such a test, the generalization claim of ACT beyond the training distribution remains an assumption.","section":"Section 3.3, Appendix E.2"},{"comment":"All reported results are single-run. The main gains (+3.2 HOTA on DanceTrack, +3.2 HOTA on SportsMOT) should be accompanied by mean and standard deviation over at least three random seeds, so the reader can judge whether the improvements are within noise.","section":"Section 4.4, Table 5"},{"comment":"The statement that 'Exp. 10-11 achieve close performances' is not accurate: the gap is 4.7 HOTA on DanceTrack and 2.2 HOTA on SportsMOT. Please rephrase to 'narrow the gap' without overstating closeness.","section":"Section 4.4, 'Complete Architecture and Training' paragraph"}],"minor_comments":[{"comment":"'+7.6% HOTA on PoseTrack21' should be '+7.7%' per Table 6 (66.0 vs 58.3).","section":"Section 4.3"},{"comment":"'ExpLoitation' is likely a spelling error for 'Exploitation'; please confirm and correct throughout the manuscript.","section":"Title and abstract"},{"comment":"Figure 2 is dense; consider splitting it into two figures (pipeline overview and model detail) to improve readability.","section":"Figure 2"},{"comment":"'eEach' should be 'Each'.","section":"Appendix F.3"},{"comment":"'CAMEL build upon' should be 'CAMEL builds upon'.","section":"Section 3.2"},{"comment":"The delta rows in Table 5 are ambiguous; specify the baseline for each delta (e.g., relative to Exp. 1).","section":"Table 5"},{"comment":"The statement about DiffMOT's interpolation is supported by a repository link, but the version/date of the release is not given; please add it for reproducibility.","section":"Appendix C.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical contribution with a clear central claim, but the overclaim about outpacing SORT-based methods across all five benchmarks must be fixed before acceptance. I would welcome a revision that adds the cross-dataset transfer experiment; if the authors cannot run it, the claim should be softened accordingly. The code release is a genuine strength."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Vladimir et al. have written one of the cleaner learned-association papers for tracking-by-detection. The core idea – replace all three SORT-style heuristics (tracklet representation, feature fusion, multi-stage matching) with a trainable module – is genuinely new relative to TransMOT, TADN, STRN, and BUSCA, which all retain heuristic pieces. The architecture is simple: per-cue temporal encoders plus a group-aware fusion encoder, trained with a contrastive loss on precomputed features. The association-centric training (ACT) is the most interesting part: it builds synthetic scenarios from random frames, sometimes across videos, and with identity-swap/dropout augmentations, so training takes one hour on a single GPU. That is an order-of-magnitude cheaper than E2E methods.\n\nThe ablations in Table 5 are carefully done and support the design choices. Results on DanceTrack, SportsMOT, PoseTrack21, and BEE24 are strong, and releasing code and weights is a plus. I would send this to review.\n\nTwo soft spots worth naming. First, the MOT17 bullet overclaims. The paper says it outperforms SORT-based and E2E methods across five benchmarks, but on MOT17 ByteTrack's HOTA is 62.8 versus CAMELTrack's 62.4. The paper's own discussion of MOT17's bias toward heuristics is reasonable, but the contribution bullet should be softened to 'competitive' or 'state-of-the-art on four of five.' Minor.\n\nSecond, the ACT transfer assumption is undertested. Training samples pair detections from a frame with tracklets built from previous frames, sometimes from different videos. This distribution is synthetic relative to real online tracking, where tracklets and detections come from the same video and history is causally coherent. The paper never compares ACT against a recursive real-frame training protocol with the same architecture, nor runs a zero-shot cross-dataset evaluation. BEE24 is adapted with only bounding-box cues, so it is not a transfer test. I don't think this invalidates the method – the held-out test results are real – but it is a genuine gap for the paper's claim that the learned strategies capture real occlusions and re-entries.\n\nAlso, no multi-seed variance is reported, and per-dataset thresholds are tuned on validation sets. Those are common omissions in MOT and not fatal.\n\nNet: this is a solid empirical paper that deserves a serious referee. My recommendation: accept with minor revisions – ask for one transfer experiment or a recursive-training ablation, and fix the MOT17 phrasing.","headline":"A clean, well-ablated learned-association paper for TbD tracking; the ACT transfer assumption is undertested and the MOT17 claim is slightly overclaimed, but the central contribution holds up.","tokens_in":27585,"tokens_out":3991,"would_cite":true,"duration_ms":37788,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"One trainable cue-agnostic transformer replaces the hand-crafted association rules of tracking-by-detection, with state-of-the-art results on five benchmarks.","keywords":["multi-object tracking","tracking-by-detection","learned association","transformer","feature fusion","re-identification","pose estimation","association-centric training"],"falsifier":"Run CAMELTrack without retraining on a held-out set of sequences whose detections come from a detector of very different quality than the YOLOX ones used in training (for instance, a much weaker detector with frequent missed detections), giving both CAMELTrack and the strongest heuristic-fusion baseline the same motion, appearance, and pose cues; if CAMELTrack fails to match or beat that baseline on association accuracy (AssA) and identity F1 (IDF1), the context-aware strategy learned from synthetic scenarios did not transfer to a real distribution shift.","tokens_in":2217,"feed_emoji":"🐫","tokens_out":2332,"duration_ms":153038,"temperature":0.7,"pith_summary":"Online multi-object tracking is dominated by tracking-by-detection pipelines in which the three steps of association — summarizing a tracklet's history, blending motion and appearance cues, and staging the matches — are engineered by hand. This paper asserts that all three can be replaced by one compact trainable module, CAMEL, composed of two transformer encoders, while keeping the modularity that lets a tracker use off-the-shelf detectors, re-identification models, and pose estimators. It further asserts that its association-centric training, which learns from image-free synthetic scenarios assembled from pre-computed cues rather than from whole videos, transfers to real benchmarks and completes in under an hour on a single GPU. An oracle study supports the premise: with cues and detections held fixed, an optimal association rule improves HOTA by about 15.5 percent on DanceTrack and 8.3 percent on SportsMOT over heuristic baselines, so the headroom the paper targets is real. If these claims hold, learned association becomes a cheap, modular alternative to both heuristic SORT-style trackers and costly end-to-end DETR trackers.","feed_headline":"Tracking by learned rules, not heuristics, tops five benchmarks","feed_subtitle":"Learned from data in one hour on one GPU, it blends motion, appearance, and pose to keep identities across occlusions.","key_machinery":"The load-bearing machinery is a two-stage transformer stack plus an image-free training scheme. First, one Temporal Encoder (TE) per cue type runs intra-object self-attention over the feature bank — the W most recent detections of a tracklet — after adding sinusoidal positional encodings of each detection's age, and returns a learned [CLS] token as the tracklet-level representation; this replaces the mean/EMA/Kalman heuristics for summarizing history. Second, the Group-Aware Feature Fusion Encoder (GAFFE) linearly projects each cue token, sums the projections into one token per active object, then applies inter-object self-attention across all tracklets and detections at once, producing disentangled embeddings in which same-identity pairs are close and different-identity pairs are far; this replaces both the hand-tuned fusion weights and the staged matching cascade. The training scheme makes the design practical: a pre-generated, image-free dataset of detections, labels, and cues is sampled into synthetic association scenarios (a random frame's detections paired with tracklets from earlier frames, sometimes stitched from different videos), augmented with identity swaps, detection dropout, and noise, and optimized with the InfoNCE loss, so the module learns how much to trust each cue in each context from data rather than from rules.","core_discovery":"The central claim is that CAMEL is, to the authors' knowledge, the first fully-learned and cue-agnostic association module for online tracking-by-detection: it consumes any number of tracking cues (bounding boxes, re-identification embeddings, pose keypoints) and replaces the three classic heuristics — tracklet representation, feature fusion, and multi-stage matching — with a single trainable forward pass that outputs one disentangled embedding per tracklet and detection, matched by Euclidean distance and the Hungarian algorithm. Two transformer components do the work: a Temporal Encoder per cue aggregates each tracklet's recent detections into a tracklet-level token, and the Group-Aware Feature Fusion Encoder (GAFFE) projects every cue into a shared space, sums the projections per object, and runs inter-object self-attention to separate identities contextually. Training is association-centric: each sample is built from a random frame plus that frame's detections and tracklets from preceding frames, drawn across videos, with augmentations that simulate identity switches, dropped detections, and noisy cues, optimized by the InfoNCE contrastive loss. The resulting CAMELTrack pipeline, a compact 42.6-million-parameter model, reports state-of-the-art HOTA on SportsMOT (80.4), gains of roughly 7.6 HOTA points on PoseTrack21 and 3.7 on BEE24, surpasses all end-to-end methods on MOT17, and, with keypoints added, surpasses all end-to-end methods on DanceTrack (69.3 HOTA) — while training in one hour on one GPU and running at 13 frames per second.","pith_inferences":["The paper reports that cue dropout had no measurable effect during training, which suggests the model is not actually learning a fallback behavior for absent cues; a structured dropout that deletes cues exactly when occlusions or detector failures occur might close that robustness gap, but that is a testable extension, not a claim of the paper.","Because every cue enters through its own Temporal Encoder before summation, the same trained pipeline could plausibly ingest qualitatively different modalities — audio, radar, text descriptions, or vehicle license plates — but the paper mentions plates only as a possible extension, not a demonstrated one.","The gap between the paper's Feature Fusion Oracle and Association Oracle on DanceTrack implies that even a perfect fusion of the current cues is not enough there, so future gains on that benchmark likely come from designing stronger cues rather than from better fusion architectures.","The disentangled per-object embeddings CAMEL produces are, in effect, learned tracklet-level descriptors, so they could be reused for downstream tasks such as tracklet re-identification or clustering; the paper does not explore these uses."],"forward_implications":["Adding a new tracking cue becomes a matter of training one Temporal Encoder and plugging it into CAMEL, with no redesign of the matching logic; the paper demonstrates this with pose keypoints, which add 3.2 HOTA on DanceTrack.","Learned association no longer requires the multi-day, multi-GPU budgets of end-to-end DETR trackers: CAMEL trains in under an hour on a single consumer GPU while outperforming those methods on several benchmarks.","The oracle study implies that current SORT-based trackers fail to exploit the cues they already have, so replacing association heuristics offers more headroom than improving detectors or cues within the tracking-by-detection paradigm.","Long-term identity recovery after occlusions and scene re-entries improves, because the feature bank and long training windows let the tracker re-associate objects that heuristic trackers would re-initialize as new identities.","The same architecture and training recipe transfer across domains with different cue reliability: position-only cues on BEE24, pose-augmented cues on PoseTrack21, and the full motion-plus-appearance-plus-pose stack on DanceTrack and SportsMOT."],"supporting_citations":[{"why":"Establishes the tracking-by-detection paradigm whose three association heuristics CAMEL replaces.","marker":"[5]"},{"why":"The canonical heuristic association pipeline (EMA tracklet representation, cascade matching) that serves as the baseline CAMEL must beat.","marker":"[61]"},{"why":"The closest prior learned-association module in a TbD pipeline, still dependent on hand-crafted multi-stage matching; the reference for the 'first fully-learned' claim.","marker":"[15]"},{"why":"The end-to-end DETR tracker defining the competing heuristic-free paradigm that CAMEL compares against on compute and performance.","marker":"[69]"},{"why":"Supplies the detections for all benchmarks, matching the detection setup of SORT-style baselines such as DiffMOT and ByteTrack.","marker":"[28]"},{"why":"The part-based re-identification model whose appearance embeddings are a core CAMEL cue.","marker":"[51]"},{"why":"The pose estimator whose keypoint cue adds 3.2 HOTA on DanceTrack and none on SportsMOT, supporting the context-aware multi-cue claim.","marker":"[34]"},{"why":"The benchmark where the oracle study shows a 15.5 percent HOTA gap for heuristic association and where keypoints prove valuable.","marker":"[56]"},{"why":"The benchmark where SORT-based methods previously dominated; CAMELTrack reports a 3.2-point HOTA gain over the prior state of the art.","marker":"[18]"},{"why":"The InfoNCE contrastive objective used in association-centric training to shape the embedding geometry.","marker":"[44]"}],"fun_headline_variants":["CAMELTrack: no hand-crafted rules, just learned cues, tops five benchmarks","Learned tracker replaces heuristics, trains in one hour, hits SOTA","One-hour training, no heuristics, five benchmarks topped: CAMELTrack","Learned association, no rules: CAMELTrack hits SOTA in 60 minutes","No heuristics: learned association, SOTA in an hour"],"cache_read_input_tokens":29696,"weakest_assumption_plain":"The module is trained only on image-free synthetic association scenarios — a random frame's detections paired with tracklets from earlier frames, sometimes stitched from different videos — so the reported gains rest on the assumption that these artificial contexts faithfully represent real occlusions, scene re-entries, and detector noise well enough that the learned fusion transfers to genuine online tracking.","fun_headline_variants_meta":{"raw":{"variants":["CAMELTrack: no hand-crafted rules, just learned cues, tops five benchmarks","Learned tracker replaces heuristics, trains in one hour, hits SOTA","One-hour training, no heuristics, five benchmarks topped: CAMELTrack","Learned association, no rules: CAMELTrack hits SOTA in 60 minutes","No heuristics: learned association, SOTA in an hour"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001753,"raw_usage":{"total_tokens":7007,"prompt_tokens":1119,"completion_tokens":5888,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":735,"completion_tokens_details":{"reasoning_tokens":5786}},"tokens_in":735,"tokens_out":5888,"duration_ms":38914,"temperature":1.0,"reasoning_tokens":5786,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:22:41.909890+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run CAMELTrack without retraining on a held-out set of sequences whose detections come from a detector of very different quality than the YOLOX ones used in training (for instance, a much weaker detector with frequent missed detections), giving both CAMELTrack and the strongest heuristic-fusion baseline the same motion, appearance, and pose cues; if CAMELTrack fails to match or beat that baseline on association accuracy (AssA) and identity F1 (IDF1), the context-aware strategy learned from synthetic scenarios did not transfer to a real distribution shift.","supporting_citations":[{"cited_title":"Simple online and realtime tracking with a deep association metric","cited_arxiv_id":null,"evidence_quote":"The canonical heuristic association pipeline (EMA tracklet representation, cascade matching) that serves as the baseline CAMEL must beat."},{"cited_title":"Motr: End-to-end multiple- object tracking with transformer","cited_arxiv_id":null,"evidence_quote":"The end-to-end DETR tracker defining the competing heuristic-free paradigm that CAMEL compares against on compute and performance."},{"cited_title":"Body part-based representation learn- ing for occluded person re-identification","cited_arxiv_id":null,"evidence_quote":"The part-based re-identification model whose appearance embeddings are a core CAMEL cue."},{"cited_title":"DanceTrack: Multi-object track- ing in uniform appearance and diverse motion","cited_arxiv_id":null,"evidence_quote":"The benchmark where the oracle study shows a 15.5 percent HOTA gap for heuristic association and where keypoints prove valuable."}],"review_version":1}