{"id":"b80e97a9-2993-48a5-a107-fd47d12b9946","arxiv_id":"2504.12186","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"CoMotion tracks multiple people's 3D poses online from monocular video using recurrent pose updates from image features, reaching 71.4 MOTA on PoseTrack21 and running about 12x faster than a strong prior 3D pose tracker.","lead":"CoMotion is a system that detects and tracks the 3D poses of many people from a single video camera, updating each person's pose directly from new frames instead of matching separate detections. It reports state-of-the-art tracking accuracy on PoseTrack21 and runs about 12x faster than the strongest prior 3D pose tracker.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline tracking gains are computed under the PoseTrack21 evaluation code the authors themselves flag as buggy; the corrected protocol is only run for CoMotion and one baseline, leaving 'prior state of the art' unverified.","rationale":"The reader's CONDITIONAL verdict is reasonable. I considered the pseudo-label close-proximity filter (Sec. 4.1) as the weakest assumption, and the paper explicitly concedes in the conclusion that missing close-proximity annotations is 'arguably an area that would be most important to supervise well.' However, the paper partially mitigates this with BEDLAM and WHAC-A-MOLE synthetic ground truth that includes crowded, interacting people, and Table 5 shows that including PoseTrack/DanceTrack pseudolabels improves tracking. The reader's concern is legitimate but not the single most load-bearing issue because it is disclosed and partially covered by independent supervision. The more consequential issue is the evaluation protocol mismatch: the abstract's +14%/+12% claim is based on the official PoseTrack21 code that Appendix A.1 demonstrates is buggy, while the corrected code is applied to only CoMotion and 4DHumans. Since MOTA/IDF1 are sensitive to how ignore regions are counted, and since the authors themselves show that annotation completeness can swing results by over ten points (Table 2), the relative ranking against 'prior state of the art' is not yet established under a consistent, corrected protocol. This does not diminish the architectural contribution or the strong two-method comparison against 4DHumans, but it makes the headline SOTA claim conditional on rerunning all baselines under the fixed evaluation code. The released code makes that check straightforward, which is why I keep the verdict CONDITIONAL rather than moving to REJECT.","tokens_in":21074,"tokens_out":8413,"duration_ms":87388,"concrete_test":"Rerun Tracktor++, FairMOT, CorrTrack+ReID, TRMOT, and 4DHumans with released code and checkpoints under the fixed PoseTrack21 evaluation described in Appendix A.1 on the same validation split. Then recompute CoMotion's relative MOTA and IDF1 gain against the best baseline under this single protocol. If CoMotion retains at least a 10% relative advantage on both metrics, the headline claim is robust; if another method closes the gap or overtakes it, the claim should be re-scoped to a two-method comparison. This is feasible because the authors release code and weights, and all listed baselines have public implementations.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central quantitative claim, 'MOTA +14%, IDF1 +12% relative to prior SOTA', is taken from Table 1 (top), which uses the official PoseTrack21 evaluation code. Appendix A.1 shows this code mishandles ignore regions: it computes IOU between a predicted box and the ignore polygon, so when the ignore region is large, correct detections fall below the discard threshold and are counted as false positives. The authors fix this with a one-line change (fraction of box area inside the ignore region, threshold 0.3) and report Table 1 (bottom). But under the corrected protocol only CoMotion and 4DHumans are evaluated; the other baselines (Tracktor++ 59.5 MOTA, FairMOT 56.3, etc.) are not rerun because, as the paper states, numbers cannot be carried over from a different evaluation code. The headline improvement therefore compares CoMotion to prior published numbers under a protocol the authors reject, while the corrected comparison establishes superiority over only one baseline. MOTA and IDF1 are highly sensitive to false-positive accounting, and the authors' own Table 2 shows large metric swings from annotation completeness. If the ignore-region bug penalizes methods unequally, the claimed margin over 'prior state of the art' may not survive the corrected code. This is a benchmark-control gap, not an accusation: the authors were transparent, but the headline claim is not yet supported under a single consistent evaluation protocol.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"CoMotion proposes a multi-person 3D pose tracking method that replaces the standard detect-and-associate pipeline with a recurrent pose update module: existing tracks are maintained as SMPL parameterizations plus hidden states, and each new frame directly updates those tracks via cross-attention to image features, while a parallel detection head proposes new tracks. The authors train on a mixture of pseudo-labeled image and video datasets (InstaVariety, COCO, MPII, PoseTrack, DanceTrack) and synthetic data (BEDLAM, WHAC-A-MOLE) using a three-stage curriculum, and they evaluate on PoseTrack21 for tracking, COCO/PoseTrack for 2D pose, and 3DPW for 3D pose, alongside ablations on EgoHumans and controlled experiments. The headline claims are that CoMotion improves PoseTrack21 MOTA by 14% and IDF1 by 12% relative to prior state of the art, is an order of magnitude faster than 4DHumans, and tracks through occlusion and crowds robustly.","tokens_in":21396,"tokens_out":3737,"duration_ms":37249,"significance":"If the claims hold, this is a meaningful step forward for online multi-person 3D pose tracking: the system is simple, fast, and addresses a real gap, since most prior multi-person 3D tracking is offline or two-stage. The paper is unusually transparent: it reports and fixes a bug in the PoseTrack21 evaluation code, releases code and weights, and includes thorough ablations including a pseudolabel-data ablation and a hidden-state ablation. The external grounding on 3DPW and EgoHumans, and the fact that the central architecture is a learned update rather than a fitted association module, give the contributions credibility beyond a single benchmark.","major_comments":[{"comment":"The abstract and Section 5 claim that 'CoMotion improves MOTA by 14% and IDF1 by 12% relative to the prior state of the art' on PoseTrack21. This claim is taken from Table 1 (top), which uses the official PoseTrack21 evaluation code that the authors themselves demonstrate is buggy in Appendix A.1. Under the corrected evaluation protocol (Table 1, bottom), only CoMotion and 4DHumans are evaluated; the other prior state-of-the-art baselines (Tracktor++, FairMOT, CorrTrack+ReID, TRMOT) are not rerun. Because MOTA and IDF1 are highly sensitive to false-positive accounting, and because the ignore-region bug can penalize different detectors unequally, the claimed margin over 'prior state of the art' is not established under a single consistent evaluation protocol. This is a load-bearing issue for the central quantitative claim; the authors should either rerun the other baselines under the corrected evaluation code or restrict the claim to the comparison against 4DHumans.","section":"§5, Table 1, Appendix A.1"},{"comment":"The real-world video training signal for the pose update module is NLF pseudo-labels on PoseTrack and DanceTrack, and the authors state that predicted labels are unreliable for people in close proximity, so those samples are flagged and ignored during training. This directly removes from the training distribution the crowded, close-interaction cases that the paper's headline capability ('tracking through occlusion') is meant to address. The paper's own conclusion acknowledges this as an important limitation, but the empirical consequence is not quantified. To support the claim that the learned update tracks through occlusion, the authors should report how many training samples are discarded and evaluate on occlusion-heavy or close-proximity subsets of PoseTrack21 or EgoHumans, where ground truth through occlusion is available.","section":"§4.1, §4.2, Appendix A.4"}],"minor_comments":[{"comment":"The 'one-line change' to the evaluation code actually involves two modifications: replacing the IOU-over-ignore-region test with a fraction-of-box-overlap test and changing the discard threshold from 0.1 to 0.3. Please describe this clearly as two changes.","section":"§A.1"},{"comment":"The column layout for the CoMotionstrict and 4DHumans rows is difficult to read because of the merged metrics; please ensure all columns (HOTA, IDs, MOTA on PoseTrack18, MOTA, IDF1, IDP, IDR, FP, FN, FPS) are clearly aligned.","section":"Table 2"},{"comment":"The controlled experiments in Appendix A.2 use oracle initialization: tracks are matched to ground-truth annotations on the first frame and then unrolled without track management. This is fine for architecture ablations, but it should be stated more prominently in the main text so readers do not infer that the full tracking stack was used in those ablations.","section":"§A.2"},{"comment":"The runtime claim 'an order of magnitude faster than prior state of the art' is specifically demonstrated only against 4DHumans (12x); the comparison to PARE is 1.4x. Please specify which prior method is meant by 'state of the art' in the speed claim, or report runtimes for all baselines in Table 1.","section":"§5"},{"comment":"The paper does not report variance or multiple runs for the main benchmark numbers; given the sensitivity of MOTA to evaluation details, a sentence on the determinism of the evaluation and any seed sensitivity would be helpful.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The main risk to the paper is the inconsistency between the headline PoseTrack21 comparison (official buggy evaluator) and the corrected evaluator, which covers only CoMotion and 4DHumans. The authors are refreshingly transparent about this, and the corrected comparison does show superiority over the strongest 3D pose tracking baseline, but the margin over the broader prior state of the art is unverified. I would encourage the editor to require the authors to either produce a consistent corrected comparison or soften the headline claim. The pseudo-label filtering issue is a secondary concern that needs a quantitative sensitivity check rather than a rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"CoMotion is a genuine step forward for online multi-person 3D pose tracking. The core move—maintain SMPL tracks and update them directly from a new frame's features via cross-attention and a GRU, instead of associating per-frame detections—isn't brand new (TrackFormer and MeMOTR did tracking-by-attention for boxes), but this is the first time it's made to work for detailed 3D multi-person pose, and the payoff is real: better MOTA/IDF1 on PoseTrack21, roughly 12x faster than 4DHumans, and pose accuracy on 3DPW that beats the top-down baselines. The system is well engineered, the ablations actually test the design decisions (hidden state, GRU, pseudo-labeled video datasets, curriculum), and code and weights are released.\n\nCredit where it's due: the authors found a genuine bug in the PoseTrack21 ignore-region handling, fixed it, and showed it can swing a scene from roughly -1500 MOTA to normal. That's the kind of benchmarking work the community under-values, and they're transparent about it.\n\nThe main soft spot is the one the stress-test flags: the headline 14%/12% gains are computed with the official PoseTrack21 evaluation code, which the paper argues is buggy. Under the corrected code they rerun only CoMotion and 4DHumans, so the 'prior state of the art' baselines (Tracktor++, FairMOT, etc.) are never re-evaluated under a consistent protocol. This is a real benchmark-control gap. I don't think it sinks the paper: the corrected comparison shows CoMotion ahead of the strongest 3D-capable baseline by a wide margin (71.4 vs 56.7 MOTA), and since the buggy code penalized detections inside ignore regions, a high-recall detector like CoMotion likely suffered more false positives under the old code than the weaker baselines—so the fixed protocol probably helps CoMotion more, not less. But the authors should rerun the baselines under the corrected eval; the current paper leaves the exact size of the improvement imprecise.\n\nMinor concerns: no error bars or multiple seeds (normal for this literature, still worth noting), and the video training signal relies on NLF pseudo-labels with close-proximity samples discarded—which is precisely the hard case PoseTrack21 measures. The authors acknowledge this in Sec. 4.1 and the conclusion. It's not circular: 3DPW and EgoHumans provide external grounding, and the pose estimation results stand independently.\n\nVerdict: a desk reject would be wrong; this deserves serious peer review. The eval-protocol gap should be addressed in revision, but the central claim—a faster, accurate, online multi-person 3D tracker—holds up.","headline":"Solid systems paper with a genuinely new recurrent tracking-by-attention design for multi-person 3D pose; the headline SOTA margins rest on an eval bug fix not applied to all baselines, but the corrected comparison and external benchmarks still support the core claim.","tokens_in":21916,"tokens_out":3499,"would_cite":true,"duration_ms":33990,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CoMotion proposes that online multi-person 3D pose tracking should be a recurrent update of existing tracks from new image pixels, rather than detection followed by association, and reports gains in accuracy, stability, and speed.","keywords":["multi-person 3D pose estimation","online pose tracking","tracking-by-attention","recurrent pose update","SMPL","pseudo-labeling","PoseTrack21","occlusion"],"falsifier":"A video with ground-truth identities in which two similar-looking people fully occlude each other for several seconds and then separate; if CoMotion's identity label for either person switches at the reunion, the claim that updating tracks directly from pixels supports online tracking through occlusion is not supported.","tokens_in":20869,"feed_emoji":"🧍","tokens_out":9082,"duration_ms":79803,"temperature":0.7,"pith_summary":"The paper introduces CoMotion, a system that detects and tracks detailed 3D poses of multiple people from a single monocular camera, online. Its central proposal is to replace the standard detect-then-associate pipeline: instead of finding poses independently in each frame and linking them, the model keeps a set of tracked poses and updates all of them directly from the pixels of each new frame, so a person can be followed even while partly or fully occluded. The paper claims this yields both strong per-frame pose accuracy and much better tracking stability, reporting on PoseTrack21 a 14% improvement in MOTA and 12% in IDF1 over the prior state of the art, with an order-of-magnitude reduction in runtime. If true, this makes live multi-person 3D pose tracking practical in crowded real-world scenes.","feed_headline":"One pipeline updates 3D poses instead of matching detections","feed_subtitle":"CoMotion recurrently updates tracks from image pixels, improving PoseTrack21 MOTA 14% at an order-of-magnitude faster runtime.","key_machinery":"The machinery that carries the argument is a recurrent track state $X=(\\gamma,\\theta,\\beta,h)$, where $(\\gamma,\\theta,\\beta)$ are SMPL parameters — translation, joint angles, and body shape — and $h$ is a per-person hidden state. Each new frame is encoded once into image features; a single-shot detection head proposes candidate SMPL poses for new tracks, while the pose-update module turns each existing track into tokens, attends cross-wise to the image features, refines poses through transformer layers, and rolls the hidden state through a GRU. Track management is handled by a modified Object Keypoint Similarity score that compares projected 2D keypoints, deciding when to start, keep, or delete tracks. This design lets the update module use partial evidence, such as a pair of feet, to keep a track alive through occlusion.","core_discovery":"CoMotion's central claim is that online multi-person 3D pose tracking should be formulated as a recurrent pose-update problem rather than a detection-association problem. Given a stream of monocular RGB frames, a single image encoder produces features for the current frame; a detection module proposes candidate SMPL poses for new people, while a pose-update module takes the previous timestep's tracks — SMPL parameters, projected 2D keypoints, and a per-track hidden state — and updates them by cross-attending to the full set of image features. Because tracks are updated directly from pixels, the model can follow a person through occlusion using partial evidence such as a pair of feet, and it does not need to re-identify or link per-frame detections. Trained on a mixture of pseudo-labeled real images and videos and synthetic data with perfect ground truth, CoMotion reports simultaneous strong single-frame pose accuracy and large tracking gains on PoseTrack21, with MOTA up 14% and IDF1 up 12% over the prior state of the art, at roughly an order-of-magnitude lower runtime.","pith_inferences":["Editorial inference: If the reported speed and accuracy hold when input resolution and model size scale, real-time 3D pose tracking of crowds could move from offline post-processing to live streaming systems; the authors note scaling as future work.","Editorial inference: The paper's full-image pose results suggest the standard oracle-crop evaluation protocol may understate single-shot multi-person systems, and future benchmarks could reasonably report full-image as the primary pose-estimation setting.","Editorial inference: The same recurrent update mechanism could transfer to other articulated objects — animals, hands, or robots — by replacing the SMPL parameterization with the appropriate body model and re-training on pseudo-labeled or synthetic video.","Editorial inference: Because the paper discards exactly the close-proximity pseudo-labels that matter most for crowd tracking, an explicit test is whether a pseudo-labeler designed to disentangle overlapping bodies improves MOTA further without changing the architecture."],"forward_implications":["If the central claim is correct, online streaming systems can track multiple people's 3D poses without waiting for future frames, because the update module propagates and refines tracks using only the current and past frames.","On crowded benchmarks, identity preservation should improve substantially: the reported 14% MOTA and 12% IDF1 gains over prior work mean fewer missed tracks, false positives, and identity switches.","Runtime should drop by an order of magnitude relative to top-down tracking-by-detection systems, because the model processes the full image once instead of running a pose estimator on every person crop.","The system should tolerate approximate camera intrinsics across typical focal lengths, which matters for in-the-wild deployment where calibration is unknown."],"supporting_citations":[{"why":"Supplies the tracking-by-attention paradigm that CoMotion adopts: joint detection and track update via cross-attention between image features and query tokens.","marker":"(Meinhardt et al., 2022)"},{"why":"The strongest prior baseline for multi-person 3D pose tracking; CoMotion compares against it on PoseTrack18/21 and inherits the idea of training on a mixture of pseudo-labeled image datasets.","marker":"(Goel et al., 2023)"},{"why":"Defines SMPL, the body parameterization used for every pose, track state, and detection output.","marker":"(Loper et al., 2015)"},{"why":"Provides PoseTrack21, the benchmark used for tracking evaluation and one of the two real-world video datasets pseudo-labeled for training.","marker":"(Doering et al., 2022)"},{"why":"The single-person model used to pseudo-label InstaVariety, COCO, MPII, PoseTrack, and DanceTrack, supplying the 3D supervision that makes in-the-wild video training possible.","marker":"(Sárándi & Pons-Moll, 2024)"},{"why":"BEDLAM provides synthetic multi-person sequences with perfect 3D ground truth used in stages 1 and 2 of training.","marker":"(Black et al., 2023)"},{"why":"WHAC-A-MOLE provides synthetic dancing-couple clips with ground truth, used for video training of the pose-update module.","marker":"(Yin et al., 2024)"},{"why":"DanceTrack is the other pseudo-labeled real-world video dataset; its bounding-box tracks let the authors unroll and supervise pose updates on crowded motion.","marker":"(Sun et al., 2022)"},{"why":"PoseTrack18 is used for backward-compatible pose and tracking evaluation and to show the effect of incomplete annotations.","marker":"(Andriluka et al., 2018)"},{"why":"COCO supplies the Object Keypoint Similarity metric that CoMotion modifies, and is one of the pseudo-labeled image datasets used for detection pretraining.","marker":"(Lin et al., 2014)"}],"fun_headline_variants":[],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The system's crowd and occlusion tracking depends on pseudo-labels from a single-person model being accurate enough to teach the recurrent update module how to follow people through occlusion, even though the authors discard pseudo-labeled samples where people are close together.","fun_headline_variants_meta":{"error":"Client error '402 Payment Required' for url 'https://api.deepseek.com/chat/completions'\nFor more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/402"},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:35:58.925779+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A video with ground-truth identities in which two similar-looking people fully occlude each other for several seconds and then separate; if CoMotion's identity label for either person switches at the reunion, the claim that updating tracks directly from pixels supports online tracking through occlusion is not supported.","supporting_citations":[{"cited_title":"All estimates are made in the camera coordinate frame","cited_arxiv_id":null,"evidence_quote":"Defines SMPL, the body parameterization used for every pose, track state, and detection output."},{"cited_title":"PoseTrack only provides 2D keypoint annotations while DanceTrack only offers bounding boxes, so we pseudo-label the videos with NLF","cited_arxiv_id":null,"evidence_quote":"DanceTrack is the other pseudo-labeled real-world video dataset; its bounding-box tracks let the authors unroll and supervise pose updates on crowded motion."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"PoseTrack18 is used for backward-compatible pose and tracking evaluation and to show the effect of incomplete annotations."}],"review_version":1}