REVIEW 2 major objections 5 minor 28 references
CoMotion: Concurrent Multi-person 3D Motion
T0 review · 2 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read CoMotion proposes that online multi-person 3D pose tracking should be a recurrent update of existing tracks from new image pixels, rather than detection followed by association, and reports gains in accuracy, stability, and speed.
desk verdict Solid systems paper with a genuinely new recurrent tracking-by-attention design for multi-person 3D pose; the headline SOTA margins rest on an eval bug fix not applied to all baselines, but the corrected comparison and external benchmarks still support the core claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery that carries the argument is a recurrent track state $X=(\gamma,\theta,\beta,h)$, where $(\gamma,\theta,\beta)$ are SMPL parameters — translation, joint angles, and body shape — and $h$ is a per-person hidden state. Each new frame is encoded once into image features; a single-shot detection head proposes candidate SMPL poses for new tracks, while the pose-update module turns each existing track into tokens, attends cross-wise to the image features, refines poses through transformer layers, and rolls the hidden state through a GRU. Track management is handled by a modified Object Keypoint Similarity score that compares projected 2D keypoints, deciding when to start, keep, or delete tracks. This design lets the update module use partial evidence, such as a pair of feet, to keep a track alive through occlusion.
What would settle it
A video with ground-truth identities in which two similar-looking people fully occlude each other for several seconds and then separate; if CoMotion's identity label for either person switches at the reunion, the claim that updating tracks directly from pixels supports online tracking through occlusion is not supported.
Extended reading notes
Core claim
CoMotion's central claim is that online multi-person 3D pose tracking should be formulated as a recurrent pose-update problem rather than a detection-association problem. Given a stream of monocular RGB frames, a single image encoder produces features for the current frame; a detection module proposes candidate SMPL poses for new people, while a pose-update module takes the previous timestep's tracks — SMPL parameters, projected 2D keypoints, and a per-track hidden state — and updates them by cross-attending to the full set of image features. Because tracks are updated directly from pixels, the model can follow a person through occlusion using partial evidence such as a pair of feet, and it does not need to re-identify or link per-frame detections. Trained on a mixture of pseudo-labeled real images and videos and synthetic data with perfect ground truth, CoMotion reports simultaneous strong single-frame pose accuracy and large tracking gains on PoseTrack21, with MOTA up 14% and IDF1 up 12% over the prior state of the art, at roughly an order-of-magnitude lower runtime.
Load-bearing premise
The system's crowd and occlusion tracking depends on pseudo-labels from a single-person model being accurate enough to teach the recurrent update module how to follow people through occlusion, even though the authors discard pseudo-labeled samples where people are close together.
Editorial extensions
If this is right
- If the central claim is correct, online streaming systems can track multiple people's 3D poses without waiting for future frames, because the update module propagates and refines tracks using only the current and past frames.
- On crowded benchmarks, identity preservation should improve substantially: the reported 14% MOTA and 12% IDF1 gains over prior work mean fewer missed tracks, false positives, and identity switches.
- Runtime should drop by an order of magnitude relative to top-down tracking-by-detection systems, because the model processes the full image once instead of running a pose estimator on every person crop.
- The system should tolerate approximate camera intrinsics across typical focal lengths, which matters for in-the-wild deployment where calibration is unknown.
Reading between the lines
- Editorial inference: If the reported speed and accuracy hold when input resolution and model size scale, real-time 3D pose tracking of crowds could move from offline post-processing to live streaming systems; the authors note scaling as future work.
- Editorial inference: The paper's full-image pose results suggest the standard oracle-crop evaluation protocol may understate single-shot multi-person systems, and future benchmarks could reasonably report full-image as the primary pose-estimation setting.
- Editorial inference: The same recurrent update mechanism could transfer to other articulated objects — animals, hands, or robots — by replacing the SMPL parameterization with the appropriate body model and re-training on pseudo-labeled or synthetic video.
- Editorial inference: Because the paper discards exactly the close-proximity pseudo-labels that matter most for crowd tracking, an explicit test is whether a pseudo-labeler designed to disentangle overlapping bodies improves MOTA further without changing the architecture.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. CoMotion proposes a multi-person 3D pose tracking method that replaces the standard detect-and-associate pipeline with a recurrent pose update module: existing tracks are maintained as SMPL parameterizations plus hidden states, and each new frame directly updates those tracks via cross-attention to image features, while a parallel detection head proposes new tracks. The authors train on a mixture of pseudo-labeled image and video datasets (InstaVariety, COCO, MPII, PoseTrack, DanceTrack) and synthetic data (BEDLAM, WHAC-A-MOLE) using a three-stage curriculum, and they evaluate on PoseTrack21 for tracking, COCO/PoseTrack for 2D pose, and 3DPW for 3D pose, alongside ablations on EgoHumans and controlled experiments. The headline claims are that CoMotion improves PoseTrack21 MOTA by 14% and IDF1 by 12% relative to prior state of the art, is an order of magnitude faster than 4DHumans, and tracks through occlusion and crowds robustly.
Significance. If the claims hold, this is a meaningful step forward for online multi-person 3D pose tracking: the system is simple, fast, and addresses a real gap, since most prior multi-person 3D tracking is offline or two-stage. The paper is unusually transparent: it reports and fixes a bug in the PoseTrack21 evaluation code, releases code and weights, and includes thorough ablations including a pseudolabel-data ablation and a hidden-state ablation. The external grounding on 3DPW and EgoHumans, and the fact that the central architecture is a learned update rather than a fitted association module, give the contributions credibility beyond a single benchmark.
major comments (2)
- [§5, Table 1, Appendix A.1] The abstract and Section 5 claim that 'CoMotion improves MOTA by 14% and IDF1 by 12% relative to the prior state of the art' on PoseTrack21. This claim is taken from Table 1 (top), which uses the official PoseTrack21 evaluation code that the authors themselves demonstrate is buggy in Appendix A.1. Under the corrected evaluation protocol (Table 1, bottom), only CoMotion and 4DHumans are evaluated; the other prior state-of-the-art baselines (Tracktor++, FairMOT, CorrTrack+ReID, TRMOT) are not rerun. Because MOTA and IDF1 are highly sensitive to false-positive accounting, and because the ignore-region bug can penalize different detectors unequally, the claimed margin over 'prior state of the art' is not established under a single consistent evaluation protocol. This is a load-bearing issue for the central quantitative claim; the authors should either rerun the other baselines under the corrected evaluation code or restrict the claim to the comparison against 4DHumans.
- [§4.1, §4.2, Appendix A.4] The real-world video training signal for the pose update module is NLF pseudo-labels on PoseTrack and DanceTrack, and the authors state that predicted labels are unreliable for people in close proximity, so those samples are flagged and ignored during training. This directly removes from the training distribution the crowded, close-interaction cases that the paper's headline capability ('tracking through occlusion') is meant to address. The paper's own conclusion acknowledges this as an important limitation, but the empirical consequence is not quantified. To support the claim that the learned update tracks through occlusion, the authors should report how many training samples are discarded and evaluate on occlusion-heavy or close-proximity subsets of PoseTrack21 or EgoHumans, where ground truth through occlusion is available.
minor comments (5)
- [§A.1] The 'one-line change' to the evaluation code actually involves two modifications: replacing the IOU-over-ignore-region test with a fraction-of-box-overlap test and changing the discard threshold from 0.1 to 0.3. Please describe this clearly as two changes.
- [Table 2] The column layout for the CoMotionstrict and 4DHumans rows is difficult to read because of the merged metrics; please ensure all columns (HOTA, IDs, MOTA on PoseTrack18, MOTA, IDF1, IDP, IDR, FP, FN, FPS) are clearly aligned.
- [§A.2] The controlled experiments in Appendix A.2 use oracle initialization: tracks are matched to ground-truth annotations on the first frame and then unrolled without track management. This is fine for architecture ablations, but it should be stated more prominently in the main text so readers do not infer that the full tracking stack was used in those ablations.
- [§5] The runtime claim 'an order of magnitude faster than prior state of the art' is specifically demonstrated only against 4DHumans (12x); the comparison to PARE is 1.4x. Please specify which prior method is meant by 'state of the art' in the speed claim, or report runtimes for all baselines in Table 1.
- [Throughout] The paper does not report variance or multiple runs for the main benchmark numbers; given the sensitivity of MOTA to evaluation details, a sentence on the determinism of the evaluation and any seed sensitivity would be helpful.
Circularity Check
No circularity: CoMotion is an empirically trained system whose claims are benchmark results, not derivations from their own inputs.
full rationale
CoMotion does not present a derivation chain in the sense of the circularity audit; it is a supervised multi-person 3D pose tracking system trained on pseudo-labeled and synthetic data and evaluated on external benchmarks. None of the seven circularity patterns is exhibited with a quotable equation-level reduction. The tracking claim is not defined in terms of the training labels: MOTA and IDF1 are not loss terms, and the model never sees the PoseTrack21 evaluation metric during training. The closest concern is that PoseTrack pseudo-labels are used for training while PoseTrack21 is used for tracking evaluation; this is an in-domain supervision and benchmark-control issue, not a self-definitional or fitted-input-as-prediction circularity, and the paper also reports external grounding on 3DPW, EgoHumans, and controlled ablations. The paper's use of prior work (4D Humans, TrackFormer, NLF) is through standard dataset, architecture, and pseudo-label citations that are not load-bearing uniqueness claims, and no "prediction" is produced by construction from a fitted constant. The acknowledged bug in the PoseTrack21 evaluation code and the fact that the corrected protocol reruns only CoMotion and 4DHumans is a legitimate benchmark-comparison weakness, but it concerns whether the headline margin over prior state of the art is verified under one consistent protocol, not whether the derivation reduces to its inputs. Accordingly, no specific circular step can be quoted, and the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (5)
- z_default depth scaling =
not stated in paper
- Track management thresholds =
instantiation OKS 0.2; deletion 0.15; collapse 0.6 for 20 frames; EMA alpha 0.2
- Candidate poses per grid location =
4
- Track token count M and feature dim D =
M=24, D=512
- Modified OKS single kappa =
not stated
assumptions (5)
- domain assumption SMPL is an adequate body representation for all people in the scene
- domain assumption A fixed pinhole camera with provided intrinsics K is sufficient; camera motion and absolute scale are not modeled
- domain assumption NLF pseudo-labels on real images and videos are accurate enough for 3D pose supervision after filtering close-proximity samples
- ad hoc to paper Training on fixed short clips with a fixed number of tracks transfers to open-ended online tracking with dynamic tracks
- ad hoc to paper Hand-tuned track management heuristics preserve identity without a learned association module
Cite this review
Pith. "Pith review of CoMotion: Concurrent Multi-person 3D Motion." pith.science (2026). https://pith.science/paper/6E4GPVAJ
@misc{pith2026250412186,
author = {Pith},
title = {Pith review of: CoMotion: Concurrent Multi-person 3D Motion},
year = {2026},
howpublished = {\url{https://pith.science/paper/6E4GPVAJ}},
note = {Machine review of arXiv:2504.12186}
}
read the original abstract
We introduce an approach for detecting and tracking detailed 3D poses of multiple people from a single monocular camera stream. Our system maintains temporally coherent predictions in crowded scenes filled with difficult poses and occlusions. Our model performs both strong per-frame detection and a learned pose update to track people from frame to frame. Rather than match detections across time, poses are updated directly from a new input image, which enables online tracking through occlusion. We train on numerous image and video datasets leveraging pseudo-labeled annotations to produce a model that matches state-of-the-art systems in 3D pose estimation accuracy while being faster and more accurate in tracking multiple people through time. Code and weights are provided at https://github.com/apple/ml-comotion
Figures
Figures from the paper (10 more)
Reference graph
Works this paper leans on
-
[1]
nd a learned pose update to track people from frame to frame. Rather than match detections across time, poses are updated directly from a new input image, which enables online tracking through occlusion. We train on numerous image and video datasets leveraging pseudo- labeled annotations to produce a model that matches state-of-the-art systems in 3D pose ...
arXiv 2017
-
[3]
Note that no video information is used here
We follow the same setup as prior work, with the same bounding boxes, cropping, and resizing of each subject. Note that no video information is used here. As opposed to other methods, our model outputs multiple detected poses given an input image; we return the detection whose IOU is highest with the provided bounding box. On oracle-cropped inputs, we obs...
work page 2024
-
[4]
Performance is slightly worse without the GRU, and even worse without any hidden state at all
The first row represents our full approach, the second row drops the GRU but preserves a learned hidden state that is passed through time, and the third row drops the hidden state altogether. Performance is slightly worse without the GRU, and even worse without any hidden state at all. Without a hidden state, the only information passed across consecutive...
work page 2025
-
[7]
All estimates are made in the camera coordinate frame
and parameterize each person with SMPL (Loper et al., 2015), which consists of a translation termγ∈ R3, joint anglesθ∈ R72 (including a global orientation), and shape parametersβ∈ R10. All estimates are made in the camera coordinate frame. We do not explicitly model any changes to the camera pose or intrinsics (e.g., due to camera motion or zooming). Duri...
work page 2015
-
[8]
1 × 1 Conv ConvNeXt Layer Non-Max Suppression SMPL Predictions FlattenandConcat Confidence SMPL Candidates ... ... Image Encoder 1 × 1 Conv Detection Head ConvNeXt V2 Stage 4 𝑁 ×𝛾,𝜃,𝛽 (e.g., 𝑁=2 ) Stage 2 Stage 1 1 × 1 Conv 1 × 1 Conv 1 × 1 Conv 1 × 1 Conv Reshape Reshape Reshape Reshape Figure 9: Architecture of the image encoder and detection step. We v...
work page 2025
-
[9]
and DanceTrack (Sun et al., 2022). PoseTrack only provides 2D keypoint annotations while DanceTrack only offers bounding boxes, so we pseudo-label the videos with NLF. While this yields 3D annotations for challenging real-world video sequences that would be impossible to obtain otherwise, we note one drawback. The predicted labels are unreliable for peopl...
work page 2022
-
[10]
We sample sequences of length 96 from DanceTrack and length 32 from WHAC-A-MOLE
Stage 3 extends training of the pose update module to longer video sequences. We sample sequences of length 96 from DanceTrack and length 32 from WHAC-A-MOLE. Samples from such long sequences are particularly slow to train on so we include a mix of shorter 8-frame clips from PoseTrack and images from InstaVariety. To reduce GPU memory consumption, we enab...
work page 2025
-
[12]
but not on the more accurately and completely annotated PoseTrack21. Unfortunately, due to the drastically incomplete annotations in PoseTrack18 (which motivated the creation of PoseTrack21), we observe that tracking evaluation on PoseTrack18 can be misleading. Many sequences in PoseTrack18 only have annotations for a fraction of the people in the scene, ...
work page 2023
Show all 28 references
-
[16]
BEDLAM: A synthetic dataset of bodies exhibiting detailed lifelike animated motion
10 Published as a conference paper at ICLR 2025 Michael J Black, Priyanka Patel, Joachim Tesch, and Jinlong Yang. BEDLAM: A synthetic dataset of bodies exhibiting detailed lifelike animated motion. In CVPR,
2025
-
[17]
SSD: Single shot multibox detector
11 Published as a conference paper at ICLR 2025 Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. SSD: Single shot multibox detector. In ECCV,
2025
-
[18]
TransTrack: Multiple-object tracking with transformer
Peize Sun, Yi Jiang, Rufeng Zhang, Enze Xie, Jinkun Cao, Xinting Hu, Tao Kong, Zehuan Yuan, Changhu Wang, and Ping Luo. TransTrack: Multiple-object tracking with transformer. arXiv:2012.15460,
2012 arXiv
-
[19]
Simple baselines for human pose estimation and tracking
12 Published as a conference paper at ICLR 2025 Bin Xiao, Haiping Wu, and Yichen Wei. Simple baselines for human pose estimation and tracking. In ECCV,
2025
-
[20]
false positives
13 Published as a conference paper at ICLR 2025 A A P P E N D I X A . 1 T R A C K I N G E VA L U AT I O N D E TA I L S Figure 4: Incorrect handling of missing annotations in PoseTrack18. Due to incomplete annota- tions in PoseTrack18, predicted tracks may be incorrectly regard...
2025
-
[21]
ignore regions
PoseTrack21 addresses the incompleteness of PoseTrack18 annotations by providing ‘ignore’ regions to accompany the annotated tracks. For the frame on the left, the center image illustrates the annotation of the person in the center (shown in green) and a polygon defining the ‘...
2022
-
[23]
(c h2 w2) h w -> ... c (h h2) (w w2)
All measurements were made on the same hardware using the code provided by the authors of each method. We measure the time to run the complete tracking stack unrolled across all PoseTrack validation videos. We find that CoMotion significantly outperforms prior work. Specifical...
2000
-
[24]
As with most convolutional backbones, features are extracted in consecutive stages to progressively lower resolutions
implementation provided in the timm library (Wightman, 2019). As with most convolutional backbones, features are extracted in consecutive stages to progressively lower resolutions. There are four stages yielding features at 1/4th, 1/8th, 1/16th, and 1/32nd of the input image r...
2019
-
[25]
At each feature scale we apply 1x1 convolutions to decode candidate detections
to yield a low-resolution feature pyramid. At each feature scale we apply 1x1 convolutions to decode candidate detections. The model produces multiple candidate poses at each spatial location. The number of poses per location is a hyperparameter, and we found 4 to provide a go...
2025
-
[27]
Object Keypoint Similarity
The goal of the above steps is to have a flexible way to operate on an unordered discrete set of tracks and efficiently attend to dense per-pixel image information without explicitly defining what information might be relevant for which tracks. We use standard attention as the...
2023
-
[28]
We ran this model off-the-shelf on the full set of InstaVariety, COCO, and MPII images
which offered an opportunity to produce new pseudolabels. We ran this model off-the-shelf on the full set of InstaVariety, COCO, and MPII images. With a strong detector, the images are much more comprehensively annotated than before and the quality of poses improved. In partic...
2024
-
[2014]
and PoseTrack18 (Andriluka et al., 2018), and Mean Per-Joint Position Error (MPJPE) on 3DPW (von Marcard et al., 2018). PCK calculates the percentage of estimates whose distance to the ground truth falls under a given threshold, while MPJPE is the mean distance between 3D poin...
2018
-
[2015]
and new architecture. These approaches achieve excellent results on modern benchmarks but scale poorly on videos with many subjects, since they run a pose estimator independently on each cropped detection. In contrast to these tracking-by-detection methods, CoMotion estimates ...
2025
-
[2018]
More recently, PHALP (Rajasegaran et al.,
as well as 2D motion (Xiao et al., 2018). More recently, PHALP (Rajasegaran et al.,
2018
-
[2019]
projecting
59.5 69.3 76.4 63.5 CoMotion (ours) 67.6 77.9 83.4 73.0 4DHumans (Goel et al., 2023)† 56.7 70.9 87.1 59.7 CoMotion (ours)† 71.4 79.5 87.1 73.0 Also, we can fit much larger batches during video training since we do not have to calculate gradients for or update the weights of th...
2022
-
[2020]
detects new objects and updates existing tracks jointly by performing cross-attention between an image and a set of learned per-track and new-object query tokens. While many of these works condition the query tokens only on the previous few frames (Zeng et al., 2022), MeMOTR m...
2022
-
[2021]
Due to making independent predictions per frame, we observe that 4D Humans occasionally makes abrupt changes to the estimated pose (see green track on the right)
0.72 0.91 0.79 0.93 82.0 50.9 PyMAF-X (Zhang et al., 2023a)0.79 0.93 0.85 0.95 78.0 47.1 HMR 2.0a (Goel et al., 2023)0.79 0.95 0.86 0.97 70.0 44.5 HMR 2.0b (Goel et al., 2023)0.86 0.96 0.90 0.98 81.3 54.3 CoMotion 0.79 0.93 0.90 0.97 63.6 36.1 fullCoMotion 0.79 0.92 0.88 0.96 ...
2023
-
[2022]
We are unaware of any prior work in this area that trains on a comparable heterogenous mix of datasets
that would be exceedingly difficult to annotate otherwise. We are unaware of any prior work in this area that trains on a comparable heterogenous mix of datasets. In particular, there are no multi-person 3D pose methods that supervise temporally unrolled predictions on complex...
2018
-
[2023]
At each level of the pyramid, a 1x1 convolution decodes candidate detections
to the given image featuresF t, yielding a low-resolution feature pyramid. At each level of the pyramid, a 1x1 convolution decodes candidate detections. Instead of multiple anchor boxes as in a traditional bounding-box approach, our detector produces multiple candidate poses a...
2014
-
[2024]
Our approach also uses a GRU to run inference on video
uses a denoising diffusion model to decode subsequent frames. Our approach also uses a GRU to run inference on video. However, where these prior methods rely on an external tracking solution and can only perform inference on a single subject, our method does its own tracking t...
2023
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.