{"id":"66e537a7-eeae-4e88-84e0-9674623f1c31","arxiv_id":"2411.16466","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"People trackers can learn 2D motion offsets from detection heatmaps alone by enforcing consistency between a warped heatmap and the next frame's heatmap.","lead":"This paper presents a people-tracking method that learns frame-to-frame motion using only detection labels, not identity labels. It warps one detection heatmap by a predicted 2D offset and matches it to the next heatmap, creating a self-supervisory signal for motion.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Single-view low-FPS gains may depend on OSNet re-ID features pretrained with identity labels, which are used as 512-channel input to the motion predictor (Appendix A.4); the claimed 'no identity, no problem' result is therefore not tested without identity-supervised representation.","rationale":"The reader's degeneracy concern is valid and acknowledged by the authors; their nearest-neighbor and bipartite baselines are a reasonable check, though they do not fully rule out learned degenerate solutions in novel scenes. My concern is different and, in my view, more directly tied to the advertised contribution. The 'no identity' claim is the paper's headline, and Appendix A.4 shows the single-view system actually consumes OSNet re-ID features pretrained with identity labels. Since Table A.1 demonstrates a clear positive effect from these features, the single-view low-FPS improvements cannot be attributed to the self-supervised motion signal without an ablation. I would not reject the paper: the multi-view WILDTRACK and MultiviewX results, plus the motion-offset evaluation, do not use OSNet and provide independent support for the core idea. But the single-view evidence needs a control, and the text should disclose the OSNet dependence. That keeps the verdict CONDITIONAL, which is what the reader already recommended.","tokens_in":18834,"tokens_out":9638,"duration_ms":90338,"concrete_test":"Retrain the single-view model with the OSNet branch removed: change the first ResNet layer to accept the 1-channel Gaussian mask only, keep YOLOX frozen, and use the same training schedule and frame-interval protocol as Table 2 (e.g., intervals [8-13[ for the 2 FPS row, [1-2[ for the 30 FPS row). Report MOTA and IDF1 at 0.75, 2, and 30 FPS on the same validation split. If the low-FPS MOTA advantage over ByteTrack shrinks to the no-feature levels shown in Table A.1 or disappears, then the single-view claim depends on identity-supervised features; if a substantial gap survives, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Appendix A.4 reveals that the single-view motion predictor is not trained from detection maps alone. The main-text implementation (Section 4.1.2) states that detections are converted to 'gaussian heatmaps of 512 channels, first channel is a gaussian mask, while the remaining 512 channels are use to provide crop features of each detection'; Appendix A.4 identifies these remaining 512 channels as features extracted by OSNet, a person re-identification network pretrained with identity annotations. This means the single-view experiments in Table 2 and Fig. 5 use an input representation that carries identity-supervised appearance information. Table A.1 shows that adding these features improves MOTA and IDF1 at both 2 and 30 FPS, so the reported low-FPS gain over ByteTrack may be attributable to re-ID features rather than to the self-supervised motion consistency loss. The paper's central claim that motion is learned 'without identity or motion annotations' is therefore not established for the single-view setting. The multi-view WILDTRACK results do not use OSNet and remain supportive, but the single-view headline comparison is confounded unless results without these features are reported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a tracking method in which a 2D motion offset map is learned without motion or identity annotations. The key idea is to predict detection heatmaps at two times together with an offset field, then warp the earlier heatmap by the offsets (Eq. 1-3) and require consistency with the later heatmap. The total loss (Eq. 4) combines this motion-consistency term with a detection loss, a forward-backward consistency loss (Eq. 6), and a spatial-extent regularization (Eq. 7). Two instantiations are presented: a single-view model built around YOLOX and a ResNet motion predictor, and a multi-view ground-plane model. Tracking is performed by modifying ByteTrack with learned motion and by using muSSP with a motion-based edge weight (Eq. 8). Experiments cover MOT17 (validation and test), WILDTRACK, MOT20, and MultiviewX, with ablations on loss components, frame intervals, and detection features.","tokens_in":38,"tokens_out":5230,"duration_ms":109369,"significance":"If the claims hold, the method would be a practically valuable step toward reducing annotation cost in multi-object tracking: motion supervision emerges from detection heatmaps alone, and the multi-view results on WILDTRACK (MOTA 91.7 vs 91.3 for MVFlow) are competitive. The motion offsets are validated against identity-derived ground truth (Table 3: L1 0.58 vs 0.55 for fully supervised), the ablations cleanly separate the loss components (Table 4), and the nearest-neighbor and bipartite-matching baselines in Appendix A.2 show that the method is not merely performing trivial detection matching. However, the single-view 'no identity' claim is weakened by the use of OSNet re-ID features pretrained with identity labels as input to the motion predictor (Appendix A.4), and the low-FPS gains are obtained with training intervals matched to the test frame rate (Appendix A.5). These issues are addressable with additional experiments, so the contribution remains potentially sound.","major_comments":[{"comment":"The single-view motion predictor receives 512-dimensional OSNet features per detection (Appendix A.4), and OSNet is pretrained with identity annotations. The main-text claim that motion is learned 'without identity or motion annotations' is therefore not established for the single-view experiments in Table 2 and Fig. 5: Table A.1 shows that adding these features improves MOTA/IDF1 at both 2 and 30 FPS, so the low-FPS improvement over ByteTrack may be partly due to the identity-pretrained representation rather than to the motion-consistency loss. Please report single-view results without these features (or with features from a detector-only backbone) and adjust the claim accordingly.","section":"Appendix A.4 and Section 4.1.2"},{"comment":"The paper acknowledges in Section 4.4 that L_mot has multiple global minima: any offset map sending each detection at t to some detection at t+1 achieves zero loss. The nearest-neighbor and bipartite-matching baselines in Appendix A.2 are designed to realize such solutions, but they are evaluated only through downstream tracking MOTA/IDF1, not against the ground-truth motion metrics of Table 3. Because the claim that the network learns true motion (Table 3: L1 0.58 vs supervised 0.55) is load-bearing, please report L1/angle/norm errors for these two baselines as well, or otherwise demonstrate quantitatively that the learned offsets track physical motion rather than a valid matching.","section":"Section 3.3, Eq. (3); Section 4.4; Appendix A.2"},{"comment":"The training frame interval is selected to match the target test frame rate ('for all our experiments, we select the training sampling interval based on the desired target frame rate'). This makes the low-FPS gains in Fig. 5 and Table 2 partly a result of train/test interval matching, not of a single motion model that generalizes across frame rates. Please report results for a model trained once with a fixed interval and evaluated across all FPS values, or explicitly frame the per-FPS retraining as part of the method and discuss the associated compute cost.","section":"Appendix A.5, Table A.1; Section 4.1.2"}],"minor_comments":[{"comment":"The text says detections are converted to 'gaussian heatmaps of 512 channels, first channel is a gaussian mask, while the remaining 512 channels...' which implies 513 channels in total; the next sentence says the first layer is modified to 513 input channels. Please correct this inconsistency.","section":"Section 4.1.2"},{"comment":"The table heading says 'MOT17 val dataset' but the caption and content refer to the test set; fix the label.","section":"Table A.4"},{"comment":"The y-axis begins at 0.5, which visually inflates the gap between the curves; consider starting at 0 or indicating an axis break.","section":"Fig. 5"},{"comment":"The single-view results are reported without variance or number of runs; given the small MOTA differences at 30 FPS (76.6 vs 77.0), please report multiple seeds or otherwise characterize uncertainty.","section":"Table 2 and Fig. 5"},{"comment":"Minor typos: 'Detetection features' in Table A.1; 'Bytetrack' in several captions; 'the remaining 512 channels are use to provide' (grammar) in Section 4.1.2.","section":"Throughout"},{"comment":"The caption says a location 2 pixels away has weight 1 when lambda_r = 0.8, but inserting lambda_r = 0.8 and l = 2 into W(l) = 1/(1 + e^{4*lambda_r*l - 10}) gives approximately 0.97, not exactly 1; rephrase to 'close to 1'.","section":"Fig. 3 caption"}],"recommendation":"major_revision","confidential_remarks":"The multi-view results are strong and the core idea is attractive, but the single-view 'no identity' claim needs to be re-examined in light of the OSNet features, and the low-FPS claim needs to be separated from train/test interval matching. Both issues are fixable with additional ablations, so I do not see grounds for rejection at this stage."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The multi-view story is solid, and the differentiable heatmap-reconstruction loss is a genuinely useful self-supervised signal. But the 'no identity, no problem' headline is not actually tested in the single-view experiments: the motion predictor is fed 512-dimensional OSNet features that were trained with identity labels (Appendix A.4). The Table A.1 ablation shows those features are worth several MOTA points at 2 FPS, so the headline low-FPS gain over ByteTrack is partly a re-ID gain, not purely a motion-supervision gain.\n\nWhat the paper does well: the warping-based consistency loss (Eq. 3) is a clean way to supervise 2D offsets without motion annotations, and it generalizes the earlier weakly-supervised motion work to unconstrained displacements. The WILDTRACK multi-view results are strong (Table 1), the motion-offset quality is close to a fully supervised upper bound (Table 3), and the appendix baselines (nearest-neighbor, Hungarian matching) help rule out the worry that L_mot just learns any detection-to-detection mapping. Code is released, which makes the ablation reproducible.\n\nSoft spots, in proportion: the OSNet confound is the main one. For the single-view claim to hold as stated, the authors need to report results with and without those features, or retitle the claim. The L_mot degeneracy is acknowledged and partially addressed by the baselines, so I would call that a minor concern. Matching training frame intervals to the test FPS (Table A.1) is sensible but means the low-FPS numbers are a tuned-regime result, not a free lunch. Single-view tables have no variance bars. At 30 FPS, the method only matches ByteTrack, which is fine given the paper's focus.\n\nWho should read it: people working on tracking-by-detection, weakly supervised motion, and multi-view tracking. It deserves a serious referee, but the revision should either remove OSNet from the single-view pipeline or weaken the 'no identity' language accordingly.","headline":"Multi-view motion-supervision story is solid, but the 'no identity' claim is untested in single-view because OSNet re-ID features are baked into the input.","tokens_in":19601,"tokens_out":3989,"would_cite":true,"duration_ms":32194,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"People tracking can be trained from detection labels alone: the paper learns a 2D motion field by warping one detection heatmap onto the next and enforcing consistency, with no identity or motion annotations.","keywords":["multi-object tracking","detection-only supervision","motion estimation","heatmap warping","differentiable reconstruction","low frame rate tracking","multi-view tracking"],"falsifier":"Train the model with $L_{\\mathrm{mot}}$ but without $L_{\\mathrm{fb}}$ and $L_{\\mathrm{se}}$ on data with known ground-truth displacements; if the predicted offsets keep near-zero $L_{\\mathrm{mot}}$ while L1 error against true motion stays high, the core supervision alone does not recover motion.","tokens_in":18631,"feed_emoji":"🚶","tokens_out":5250,"duration_ms":45641,"temperature":0.7,"pith_summary":"This paper argues that motion estimation for people tracking does not need identity or motion annotations: the consistency between two detection heatmaps, linked by a predicted 2D displacement field, is itself enough supervision. The authors predict detection heatmaps at times $t$ and $t+1$ plus an offset map, warp the first heatmap by the offsets, and penalize disagreement with the second. This couples information across frames during training and is shown to produce motion offsets of near-fully-supervised quality, while improving single-view low-frame-rate MOTA by more than 10 points over ByteTrack and giving higher multi-view tracking numbers than the compared baselines on WILDTRACK. The method works with existing detectors and trackers, so the practical payoff is cheaper annotation for robust tracking in crowded scenes and low-frame-rate video.","feed_headline":"Detection labels alone train motion for people tracking","feed_subtitle":"Warping one detection heatmap onto the next supplies motion supervision, lifting low-FPS MOTA by 10 points.","key_machinery":"The load-bearing mechanism is the differentiable reconstruction from motion: $\\hat{x}^{t+1}_j = \\sum_{i \\in G} x^t_i \\, W\\big(d(j,\\, i + \\delta^{t,t+1}_i)\\big)$, with $W(l) = 1/(1 + e^{4\\lambda_r l - 10})$ a sigmoidal distance weight. It turns the offset map into a predicted future heatmap, so the detection loss backpropagates into the motion predictor; $\\lambda_r$ starts small for easy optimization and increases to sharpen the reconstruction. The forward/backward loss $L_{\\mathrm{fb}}$ (reversing frame order and requiring $\\delta^{t+1,t} \\approx -\\delta^{t,t+1}$) and the spatial-extent loss $L_{\\mathrm{se}}$ (encouraging offset smoothness within detection peaks) are what break the symmetry of $L_{\\mathrm{mot}}$'s multiple global minima.","core_discovery":"The central claim is that motion can be supervised through detection. For each pair of frames, the network outputs detection heatmaps $X^t$ and $X^{t+1}$ and a 2D offset map $\\Delta^{t,t+1}$; a differentiable reconstruction module uses $\\Delta$ to warp $X^t$ into a predicted $\\hat{X}^{t+1}$, and the loss $L_{\\mathrm{mot}} = \\|\\hat{X}^{t+1} - X^{t+1}_{\\mathrm{gt}}\\|^2$ pushes the offsets to explain the observed presence changes. Two regularizers, a forward/backward consistency $L_{\\mathrm{fb}}$ and a spatial-extent $L_{\\mathrm{se}}$, select the physically meaningful displacement among the many matchings that would zero $L_{\\mathrm{mot}}$. On WILDTRACK the resulting offsets reach L1 error 0.58 versus 0.55 for a fully supervised counterpart and 1.06 for RAFT optical flow; on MOT17 at 2 FPS the modified ByteTrack improves MOTA from 59.1 to 65.5. The authors take this as evidence that the motion signal can be extracted from detection-level annotation alone.","pith_inferences":["Because $L_{\\mathrm{mot}}$ alone is satisfied by any detection-to-detection matching, the physical-motion claim rests on $L_{\\mathrm{fb}}$ and $L_{\\mathrm{se}}$; on scenes with repeated or symmetric motion patterns the offsets could degrade into a valid matching that is not true displacement.","The same consistency-by-warping recipe should transfer to other dense prediction tasks where temporal pairs exist but labels are scarce, such as keypoint tracking, instance segmentation, or generic scene flow.","A testable extension is to train the motion predictor jointly with the detector end-to-end, instead of using a frozen detector, and measure whether the consistency signal also sharpens detection in crowded scenes."],"forward_implications":["Training a motion estimator no longer requires identity or motion labels; detection annotations, which are cheaper and more common, suffice.","Low-frame-rate tracking improves substantially: at 2 FPS on MOT17, MOTA rises from 59.1 to 65.5 over the ByteTrack baseline, and the gap widens as frame rate drops.","Learned offsets rival full supervision: L1 error 0.58 versus 0.55 on WILDTRACK, far ahead of RAFT optical flow at 1.06.","The motion field plugs into existing trackers, ByteTrack and muSSP, improving association without changing the detector or the tracking graph.","Multi-view tracking on WILDTRACK reaches MOTA 91.7, ahead of the compared methods, and the single-view approach transfers to MOT20 with similar low-FPS gains."],"supporting_citations":[{"why":"Supplies the ByteTrack baseline and association framework whose Kalman filter is replaced by the learned motion offsets.","marker":"[52]"},{"why":"Provides the MVFlow multi-view tracking baseline and the weakly supervised motion-prediction setting this work builds on.","marker":"[16]"},{"why":"Supplies the RAFT optical-flow baseline for motion-offset comparison.","marker":"[43]"},{"why":"Supplies the WILDTRACK multi-view dataset and its evaluation protocol.","marker":"[10]"},{"why":"Supplies the MOT17 single-view dataset and its train/val split used for evaluation.","marker":"[32]"},{"why":"Supplies the muSSP min-cost-flow tracker that consumes the predicted motion offsets in the multi-view pipeline.","marker":"[47]"}],"fun_headline_variants":["Motion from detections alone: no identity labels","Detections alone supervise motion: no identities needed","Learning motion from detection heatmaps only","No identity annotations: motion from detection pairs","Tracking motion learned from detections alone"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The consistency loss has many equally valid solutions, because any offset field that maps every detection at $t$ to some detection at $t+1$ gives zero loss, so the claim that the network learns true physical motion depends on the two regularizers and the training schedule selecting the right solution.","fun_headline_variants_meta":{"raw":{"variants":["Motion from detections alone: no identity labels","Detections alone supervise motion: no identities needed","Learning motion from detection heatmaps only","No identity annotations: motion from detection pairs","Tracking motion learned from detections alone"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000961,"raw_usage":{"total_tokens":4098,"prompt_tokens":953,"completion_tokens":3145,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":569,"completion_tokens_details":{"reasoning_tokens":3079}},"tokens_in":569,"tokens_out":3145,"duration_ms":18157,"temperature":1.0,"reasoning_tokens":3079,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:04:21.887696+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the model with $L_{\\mathrm{mot}}$ but without $L_{\\mathrm{fb}}$ and $L_{\\mathrm{se}}$ on data with known ground-truth displacements; if the predicted offsets keep near-zero $L_{\\mathrm{mot}}$ while L1 error against true motion stays high, the core supervision alone does not recover motion.","supporting_citations":[{"cited_title":"ByteTrack: Multi-Object Tracking by Associating Every Detection Box, 2022","cited_arxiv_id":null,"evidence_quote":"Supplies the ByteTrack baseline and association framework whose Kalman filter is replaced by the learned motion offsets."},{"cited_title":"Engilberge, W","cited_arxiv_id":null,"evidence_quote":"Provides the MVFlow multi-view tracking baseline and the weakly supervised motion-prediction setting this work builds on."},{"cited_title":"Teed and J","cited_arxiv_id":null,"evidence_quote":"Supplies the RAFT optical-flow baseline for motion-offset comparison."},{"cited_title":"Chavdarova, P","cited_arxiv_id":null,"evidence_quote":"Supplies the WILDTRACK multi-view dataset and its evaluation protocol."},{"cited_title":"Milan, L","cited_arxiv_id":null,"evidence_quote":"Supplies the MOT17 single-view dataset and its train/val split used for evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the muSSP min-cost-flow tracker that consumes the predicted motion offsets in the multi-view pipeline."}],"review_version":1}