{"id":"80a26965-a96d-417a-9c65-8108e64f0188","arxiv_id":"2607.15868","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A distributed ego-exo HMD framework that fuses head/wrist tracking with an observer's camera views via ray canonicalization and learned gating reconstructs full-body motion, beating egocentric and exocentric baselines on Nymeria and EgoHumans.","lead":"EgoExoMoCap uses two (or more) people wearing smart glasses to capture each other's full-body motion: each wearer's head/wrist tracking is combined with camera views from the other person. If it works, lightweight in-the-wild motion capture for VR/AR and embodied AI could replace bulky suits and multi-camera studios.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Known-shape assumption inflates reported MPJPE; the 'two people wearing glasses' claim needs a shape-estimation step or mean-shape numbers.","rationale":"The reader's weakest assumption correctly identifies the known-shape requirement as structurally separate from the fusion claim. My stress-test agrees: the network predicts only rotations, and joint positions are obtained via SMPL forward kinematics with a shape that is not predicted. The paper's own Discussion admits this, but the abstract and Section 4.2 present absolute MPJPE results without that caveat. This is the most load-bearing concern because the central practical claim (lightweight, 'as simple as two people wearing glasses') depends on not needing per-subject calibration. The relative fusion advantage over baselines is likely unaffected, since all methods are evaluated with GT shape, so the verdict should remain CONDITIONAL rather than moving to REJECT. I also note a smaller internal inconsistency in Section 4.2: it says 'across all metrics ... outperforms' but immediately acknowledges RPM has lower jitter; this is a wording issue and not the primary concern. The concrete mean-shape rerun would settle whether the absolute numbers are representative of the advertised deployment scenario.","tokens_in":20634,"tokens_out":6706,"duration_ms":70084,"concrete_test":"Fix the trained EgoExoMoCap and all baselines; re-run the Nymeria 3-point and 1-point evaluations twice: once with GT subject-specific SMPL shape (current Table 1) and once with the SMPL mean shape for every subject. Report MPJPE, U-PE, and L-PE for both. If the mean-shape MPJPE increases by a few cm or more, the abstract's lightweight 'two people wearing glasses' framing and the absolute SOTA numbers require qualification.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The method's absolute error numbers are conditional on subject-specific SMPL shape, which the network never estimates. Section 3.1 states the model does not predict SMPL shape parameters; Section 4.4 acknowledges 'we mainly focus on motion reconstruction and assume subject shape is provided when calculating the joint positions'; and Section 4.1 states the evaluation uses 'SMPL identity ground-truth parameters' for all methods. Thus the reported MPJPE values (e.g., 5.72 cm three-point and 8.28 cm one-point on Nymeria) are achievable only when the true body shape is known at inference. The abstract's 'as simple as two people wearing glasses' framing is therefore not supported as stated: a shape-calibration step is required, and if the mean SMPL shape is substituted, the forward-kinematics-computed joint positions will change by an amount the paper does not report. Because absolute MPJPE is the headline quantitative evidence, this is load-bearing for the practical claim, even though the relative ego-exo fusion advantage may survive unchanged.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"EgoExoMoCap proposes a distributed HMD-based human motion capture system. A target subject ('wearer') wears smart glasses providing head (and optionally wrist) tracking via SLAM, while one or more nearby observers wearing the same glasses provide RGB views of the wearer. An ego-only EgoNet produces coarse SMPL poses, which are used to crop the observer images. ViTPose 2D keypoints in those crops are unprojected to 3D rays, scaled by observer–wearer distance, canonicalized into the wearer's head frame (Section 3.4), and gated by DINOv3-derived confidence scores (Section 3.5). Ego and gated exo tokens are fused through spatial and temporal transformers to regress SMPL joint rotations; root translation is recovered analytically from the HMD head position. Training is two-stage with L1 losses on root orientation, joint rotations, and joint positions. Experiments on Nymeria and cross-dataset evaluation on EgoHumans compare against AvatarPoser, EgoPoser, EgoAllo, RPM, PromptHMR, and a naive ego-exo fusion baseline, in both 1-point and 3-point tracking settings, with ablations. The paper reports consistently lower MPJPE for the proposed method and claims that the approach is 'as simple as two (or more) people, each wearing a pair of smart glasses.'","tokens_in":20864,"tokens_out":7141,"duration_ms":77607,"significance":"The central idea is timely and the system design is well motivated. The paper provides a useful demonstration that combining egocentric device tracking with intermittent exocentric HMD views improves full-body pose accuracy without multi-camera rigs or mocap suits. The ray-based canonicalization is a clean solution to the observer ego-motion problem, and the DINOv3-based gating is a sensible, well-ablated component. The cross-dataset evaluation on EgoHumans and the robustness-to-EgoNet perturbation test are also valuable. However, two issues condition the headline claims: the absolute MPJPE numbers are computed with ground-truth subject-specific SMPL shape, and the EgoHumans 3-point evaluation uses wrist signals synthesized from ground-truth body parameters. If these are addressed, the contribution would be solid; as written, the practical 'two people wearing glasses' claim is stronger than the evidence supports.","major_comments":[{"comment":"The model does not predict SMPL shape parameters (Section 3.1), and Section 4.1 states that evaluation uses 'SMPL identity ground-truth parameters.' Section 4.4 acknowledges: 'we assume subject shape is provided when calculating the joint positions.' Consequently, the reported MPJPE values (e.g., 5.72 cm three-point and 8.28 cm one-point on Nymeria) are achievable only when the true body shape is known at inference. This is a load-bearing caveat for the abstract's claim that the system is 'as simple as two people wearing glasses': a shape-calibration or shape-estimation step is missing from the stated pipeline. Please report numbers with the SMPL mean shape or with shape estimated from the available image/sensor signals, and re-scope the accessibility claim accordingly. The relative fusion advantage may survive, but the absolute errors and the practical deployment claim are currently ove","section":"§3.1, §4.1, §4.4, Tables 1–2"},{"comment":"For EgoHumans, the 3-point tracking results use 'synthesized 6DoF tracking signals from ground-truth body parameters' because the dataset does not provide wrist tracking. This bypasses the noise, dropout, and drift characteristics of real wrist trackers, and it can inflate the 3-point numbers and the apparent cross-dataset generality. Please evaluate on real or realistically perturbed wrist signals, or present EgoHumans results primarily in the 1-point setting. At minimum, add a noise/dropout analysis to quantify how sensitive the reported 3-point gains are to the synthetic-signal assumption.","section":"§4.1, Table 2"},{"comment":"The central claim that 'across all metrics and both tracking setups, our method outperforms the baselines' rests on a single random split with no error bars or significance testing. The margins over the best ego-exo baseline are relatively small (e.g., Nymeria 5.72 vs 6.47 cm three-point, 8.28 vs 9.03 cm one-point). Report mean and standard deviation over subject-disjoint splits or bootstraps, and assess whether the reported ordering is statistically reliable. The ablation table has the same issue; without variance, the component-level conclusions are difficult to evaluate quantitatively.","section":"§4.2, Tables 1–3"}],"minor_comments":[{"comment":"The sentence 'Across all metrics and both tracking setups, our method outperforms the baselines. The only exception is Jitter, where RPM achieves the best performance' is internally inconsistent: Tables 1 and 2 show RPM has lower Jitter in all configurations. Please qualify the 'all metrics' statement in the text.","section":"§4.2, Tables 1–2"},{"comment":"The numeric labels inside the skeleton diagrams and the color scale for the DINO scores are not explained. Please add a legend or caption describing what the numbers represent and what the color mapping means.","section":"Figure 6"},{"comment":"The relationship between the full-set 'EgoExoMoCap (Ours)' row and the subset rows marked with '*' (EgoExo-single-observer, EgoExo-triangulation, EgoExo-multi-observer) is unclear. Clarify whether the subset is the same across rows and whether the comparison is fair.","section":"Table 2"},{"comment":"The paper states that the baselines AvatarPoser and EgoPoser were 'improved' by predicting a sequence instead of just the last frame. This changes the original methods; please describe the modification in more detail and indicate whether the retrained baselines and data splits will be released to support reproducibility. The paper currently does not mention any code release.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The two main concerns — ground-truth shape at inference and synthetic wrist signals on EgoHumans — are load-bearing for the practical claims but fixable within the scope of a revision. The core ego-exo fusion idea appears supported by the ablations and cross-dataset trend, so I would not reject. I recommend requiring mean-shape or estimated-shape results and a realistic wrist-signal evaluation before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead this one if you work on HMD-based or ego-exo motion capture. The takeaway: a credible fusion system with a real limitation the authors mostly own up to. The headline numbers assume known body shape, so the \"as simple as two people wearing glasses\" pitch is a step ahead of what's actually evaluated.\n\nWhat's new: the wearer-conditioned ray canonicalization (Eqs. 1–4) is a clean way to lift a second person's 2D keypoints into the wearer's head frame, factoring out observer head motion and inter-person distance. The DINOv3-based per-keypoint gating is a neat idea, and the ablation shows it helps under occlusion. The full pipeline—spatial transformer for ego-exo fusion, temporal transformer for smoothness—beats ego-only baselines (AvatarPoser, EgoPoser, EgoAllo, RPM) and an exocentric PromptHMR baseline on Nymeria, and it transfers zero-shot to EgoHumans. The exocentric baseline is even given ground-truth root pose, so beating it says the fusion is doing real work.\n\nSoft spots, in rough order of severity. First, the known-shape assumption. Section 3.1 says the model doesn't predict shape; Section 4.1 says all methods are evaluated with ground-truth SMPL identity parameters; Section 4.4 admits \"we assume subject shape is provided.\" So the absolute MPJPE numbers—5.7 cm three-point on Nymeria—are conditional on knowing the person's body shape. The relative fusion gain likely survives, since baselines get the same shape, but the practical \"two people wearing glasses\" claim needs either a shape-estimation step or a mean-shape evaluation. The paper's own Discussion is honest about this, which counts in its favor. Second, no error bars or variance across splits, and no code release, so exact reproducibility can't be checked. Third, the EgoHumans three-point evaluation synthesizes wrist signals from ground-truth body parameters, which is a break from the real sensor setup. Fourth, the closest prior work—LAMP and EgoBody—is cited but not in the quantitative comparison. The strongest baseline is a custom PromptHMR+EgoPoser fusion, and the margin over it is modest (about 0.7–1.2 cm MPJPE). The gains are consistent across settings and the ablations are supportive, so the central claim holds, but \"state-of-the-art\" needs a qualifier given the missing comparisons.\n\nWho's it for: researchers building practical mocap from commodity HMDs, and anyone working on ego-exo fusion or embodied AI data collection. I'd cite it for the ray canonicalization and gating ideas. It deserves a serious referee. A revision should add error bars, release code, and address the shape assumption directly—either report mean-shape numbers or add a shape-estimation module. Send it out.","headline":"Credible ego-exo fusion system; the known-shape assumption means the 'two glasses' pitch is ahead of what's evaluated, but the core result holds up.","tokens_in":21437,"tokens_out":4578,"would_cite":true,"duration_ms":44687,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Distributed HMD fusion — each wearer's ego-tracking plus another wearer's camera view — reconstructs full-body motion in the wild more accurately than either signal alone.","keywords":["distributed motion capture","egocentric tracking","exocentric fusion","head-mounted devices","ray-based keypoint lifting","learned visibility gating","SMPL body model","in-the-wild mocap"],"falsifier":"Recompute the reported MPJPEs with the ground-truth subject shape replaced by the SMPL mean body (or by a shape estimated from a single image). If the fused model's error rises toward the ego-only baseline, the claimed fusion advantage depends on knowing body shape in advance rather than on the fusion itself. Separately, in a long sequence with the wearer fully outside the observer's field of view, the DINO gate should reduce exo weights to near zero and output should match the ego-only fallback; any large divergence would falsify the gating claim.","tokens_in":20486,"feed_emoji":"🕶️","tokens_out":8797,"duration_ms":80172,"temperature":0.7,"pith_summary":"EgoExoMoCap aims to establish that full-body motion capture in the wild can be done by two or more people wearing head-mounted smart glasses, with each person's inertial and image streams serving as both self-tracking and mutual observation. The paper claims that fusing the wearer's egocentric head (and optional wrist) tracking with exocentric images from another wearer's glasses beats egocentric-only, exocentric-only, and naive ego-exo fusion on two in-the-wild datasets, lowering mean per-joint position error from 6.47 cm to 5.72 cm in the 3-point Nymeria setup. The technical core is a ray-based geometric canonicalization of the observer's 2D keypoints into the wearer's head-local frame, plus a learned context-aware gate that suppresses exocentric signals exactly when they are unreliable — under occlusion, truncation, or out-of-view motion. A sympathetic reader would care because this points toward scalable, comfortable, infrastructure-free collection of real-world body motion for embodied AI, VR/AR, and human-interaction applications.","feed_headline":"Two smart-glasses wearers beat single-view motion capture","feed_subtitle":"Each wearer's ego-tracking fuses with the other's camera view to reconstruct occluded, out-of-view bodies in the wild.","key_machinery":"The load-bearing machinery has three parts. Ray-based pose canonicalization unprojects each 2D keypoint from the observer's image into a 3D ray, scales it by observer–wearer head distance, and rotates it into the wearer's head-local frame, making the exocentric signal invariant to observer head motion and wearer global position. Learned visibility gating maps a DINOv3 CLS token of the cropped region through a small MLP to per-joint confidence scores that scale each ray, suppressing unreliable detections under occlusion or out-of-view. Ego-exo token fusion passes an Ego Token (egocentric tracking plus coarse EgoNet pose) and an Exo Token (gated rays) through a spatial transformer biased towar","core_discovery":"The paper claims that a distributed setup — two or more people wearing head-mounted devices — can capture full-body motion in the wild more accurately than ego-only or exo-only methods. On Nymeria, the fused model reports 5.72 cm MPJPE versus 6.47 cm for the strongest ego-exo baseline (3-point), and 8.28 versus 9.03 cm (1-point); a similar gap holds on EgoHumans, the exception being jitter, where a rolling-prediction baseline is smoother. The mechanism: a coarse ego-only pose localizes the wearer in the observer's image; 2D keypoints become 3D rays, scaled by inter-person distance and rotated into the wearer's head frame; a DINOv3-context gate scales each ray by learned visibility; spatial a","pith_inferences":["Beyond the reported results, the shape assumption means a fully self-contained pipeline would need image- or sensor-based SMPL shape estimation; the reported MPJPEs are an upper bound until then.","Beyond the paper, the DINO-context gating is a general mechanism for per-signal reliability, testable in any multi-view fusion setup that currently trusts 2D detector confidence.","Beyond the paper, since global translation is recovered from HMD SLAM, absolute trajectory accuracy is bounded by localization drift; improving head tracking may matter more than further pose refinement.","Beyond the paper, the framework yields pairwise relative motion in a group of wearers, which could enable social-interaction capture without external cameras."],"forward_implications":["Full-body motion capture in the wild works with two people wearing smart glasses; no camera rigs, mocap suits, or fixed infrastructure.","Lower-body accuracy, the classic weakness of ego-only tracking, improves markedly: 10.05 cm lower-body MPJPE versus 13.96 cm for the best ego-only baseline (3-point, Nymeria).","Adding more observers improves accuracy with no architectural change: on EgoHumans, multi-observer fusion reaches 7.11 cm MPJPE versus 8.80 cm for the best single-observer variant.","The method tolerates missing wrist signals (1-point tracking) and degrades gracefully when the coarse egocentric prior is noisy: 10 cm of injected noise raises final MPJPE by only 0.54 cm.","Because wrist signals are optional, the same pipeline applies to glasses-only wearers and to VR users with hand controllers."],"fun_headline_variants":["Ego-exo fusion: two heads better than one for mocap","Smart glasses duo capture full-body motion in the wild","Pair of HMDs reconstruct occluded bodies better than single view","Distributed ego-exo mocap: teamwork makes accurate motion"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The reported absolute accuracies hold only if each subject's SMPL body shape is known in advance; the network predicts joint rotations, not shape, so with an unknown or wrong shape the joint-position errors would be larger than the numbers reported.","fun_headline_variants_meta":{"raw":{"variants":["Ego-exo fusion: two heads better than one for mocap","Smart glasses duo capture full-body motion in the wild","Pair of HMDs reconstruct occluded bodies better than single view","Distributed ego-exo mocap: teamwork makes accurate motion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000539,"raw_usage":{"total_tokens":2437,"prompt_tokens":776,"completion_tokens":1661,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":520,"completion_tokens_details":{"reasoning_tokens":1589}},"tokens_in":520,"tokens_out":1661,"duration_ms":10600,"temperature":1.0,"reasoning_tokens":1589,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T22:05:03.264262+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the reported MPJPEs with the ground-truth subject shape replaced by the SMPL mean body (or by a shape estimated from a single image). If the fused model's error rises toward the ego-only baseline, the claimed fusion advantage depends on knowing body shape in advance rather than on the fusion itself. Separately, in a long sequence with the wearer fully outside the observer's field of view, the DINO gate should reduce exo weights to near zero and output should match the ego-only fallback; any large divergence would falsify the gating claim.","supporting_citations":[],"review_version":1}