{"id":"9760f095-05e0-4438-8778-60fa70f4be93","arxiv_id":"2608.01600","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":12,"one_line_summary":"A humanoid robot learned eight construction motions from human video demonstrations via pose retargeting and reinforcement learning, with an average 82.45 mm joint tracking error.","lead":"This paper builds a two-stage system that turns video of construction workers into motions a humanoid robot can copy, and tests it on eight tasks such as carrying a pipe and pushing a wheelbarrow. It reports an average joint-position tracking error of about 82 mm, with a few physical-robot trials, and is an early feasibility check for human-shaped robots on construction sites.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's only quantitative execution metric (82.45 mm MPJPE) comes from sim-to-sim transfer (IsaacGym→MuJoCo), not from the physical robot; real-world 'reliably executed' is asserted without MPJPE, success rates, or trial counts.","rationale":"The reader's formal weakest_assumption—manual retargeting labels—is a genuine concern: the 48.46 mm retargeting MPJPE is measured against hand-generated targets that are never validated by inverse kinematics, physical feasibility, or task outcome. However, the most load-bearing problem for the abstract's central claim is that the only quantitative execution metric (82.45 mm MPJPE) is a simulation number. This issue is more direct because it does not rely on additional assumptions about label quality: the paper's own reporting shows the metric comes from Table 4 in the sim-to-sim section, not from the physical robot. The real-robot section is qualitative only, and the conclusion's 'reliable task success rate' is unsupported by any reported rate. This gap would remain even if every retargeting label were perfect, making it the point where the central claim is least secure. The reader's rationale does mention 'no quantitative sim-to-real results,' so we partially agree, but the reader did not designate this as the weakest assumption. A concrete check—recomputing real-robot MPJPE and success counts—would settle whether the abstract's pairing of 'reliably executed' with 82.45 mm MPJPE is justified. If the real measurements are comparable, the claim survives; if not, the paper must be restricted to simulation-only evidence or rejected until real-robot data are provided.","tokens_in":30857,"tokens_out":5081,"duration_ms":57794,"concrete_test":"Request the real-robot deployment logs and recompute per-action MPJPE for each of the eight actions using an independent ground truth (e.g., a second motion-capture system or a markerless 3D pose tracker on the deployment videos). Report per-action MPJPE, the number of trials, and the percentage of trials that completed without falling or losing balance. If the average real-robot MPJPE is not within, say, 2x of the 82.45 mm simulation value, or if any action cannot be assigned a quantitative real-robot error or success rate, the headline claim should be revised to simulation-only success.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim in the abstract pairs 'reliably executed eight construction-related actions' with 'average motion-tracking error of 82.45 mm MPJPE.' That number, however, is measured in simulation. Section 5.2 reports Table 4's MPJPE after describing the IsaacGym-to-MuJoCo sim-to-sim evaluation; the following paragraph on physical deployment says only that the G1 'successfully reproduced all eight actions' and that performance was 'consistent with sim-to-sim evaluation trends,' but provides no real-robot MPJPE, no per-action numbers, no trial counts, and no success criteria. Section 7 then asserts a 'reliable task success rate' without reporting any such rate. Thus the single quantitative piece of evidence offered for the headline claim does not concern physical execution at all. Even if the manual retargeting labels are correct and the simulation numbers are reproducible, the paper as written does not establish that a real robot achieved 82.45 mm MPJPE or that any construction-related task was 'reliably' completed. This is a direct evidence gap for the central claim, independent of the validity of the retargeting ground truth identified by the reader.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Vision-based Perception-and-Action (VPA), a two-module pipeline for teaching a Unitree G1 humanoid robot construction-related actions from RGB videos of human workers. Humanoid-PoseNet combines an off-the-shelf 3D pose estimator with a learned human-to-humanoid retargeting network (encoders/decoder with triplet, reconstruction, and latent-consistency losses). Humanoid-ActionNet is a teacher-student PPO architecture that tracks the retargeted pose trajectories under physics-aware rewards, with a student policy distilled for deployable proprioceptive observations. The authors report retargeting MPJPE of 48.46 mm, sim-to-sim (IsaacGym-to-MuJoCo) action-tracking MPJPE of 82.45 mm across eight actions, and qualitative real-robot deployment on a physical G1. The abstract and conclusion claim that the robot 'reliably executed eight construction-related actions' and achieved 'a reliable task success rate.'","tokens_in":31186,"tokens_out":3741,"duration_ms":43042,"significance":"If substantiated, the system would be a meaningful early step toward learning construction tasks from human demonstration on a full humanoid platform, connecting vision-based pose extraction, retargeting, and RL whole-body control. The paper has several strengths that deserve explicit credit: the architecture is clearly described; the data-collection protocol (five subjects, 30 actions, 450 trials) is documented in detail; the retargeting module is ablated; and a comparison against four whole-body control baselines (ExBody, ExBody+AMP, OmniH2O, ExBody2) is included. However, the headline quantitative claim is not supported as written: the only action-tracking number (82.45 mm MPJPE) comes from sim-to-sim transfer, not from the physical robot, and the retargeting ground truth is a manual process that is never independently validated. The paper's internal-consistency metrics are plausible as engineering-development evidence, but they do not yet establish reliable physical task execution.","major_comments":[{"comment":"The abstract states that 'the humanoid robot reliably executed eight construction-related actions, achieving an average motion-tracking error of 82.45 mm MPJPE,' but that number is measured in simulation. Section 5.2 reports Table 4 after describing the IsaacGym-to-MuJoCo sim-to-sim evaluation; the following paragraph on physical deployment says only that the G1 'successfully reproduced all eight tested actions' and that performance was 'consistent with the sim-to-sim evaluation trends,' with no real-robot MPJPE, per-action numbers, trial counts, or success criteria. Section 7 then asserts a 'reliable task success rate' without reporting any rate. The single quantitative piece of evidence for the central claim therefore does not concern physical execution at all. This is a direct evidence gap that must be fixed by either adding real-robot quantitative tracking results or substantially qu","section":"§5.2, Table 4, Abstract"},{"comment":"The retargeting ground truth is produced by a manual process: 'the authors manually extracted and aligned the T-poses of the human and humanoid, and adjusted the body scale.' The reported retargeting MPJPE (48.46 mm, Table 3) is computed against these manual labels, and the ActionNet MPJPE (82.45 mm) compares executed poses to reference trajectories generated by Humanoid-PoseNet, which was trained to reproduce the same manual labels. Thus the primary performance measures are internal-consistency scores. The paper never independently verifies the manual labels via inverse kinematics, physical feasibility checks, or task outcomes. If the labels are not mechanically correct or task-faithful, the entire reference-trajectory chain is invalid. Please provide an external validation of the retargeting ground truth (e.g., IK-based joint-angle feasibility, contact/balance checks, or human evaluati","section":"§4.2, Table 3, Eq. (5)"},{"comment":"The eight actions are selected after a sim-to-sim screening procedure, but the paper does not report how many actions were attempted, how many failed, or the selection criteria. Figure 14 shows representative failure cases in MuJoCo (loss of balance, unstable recovery, tracking breakdown), and Section 7 acknowledges 'non-negligible failure cases across the eight actions,' yet Table 4 reports only average per-joint MPJPE with no variance, no trial counts, and no success definition. The claim 'reliably executed' is not supported without per-action success rates and repeated-trial statistics. Please report the number of trials per action, standard deviations, and a defined success threshold (e.g., MPJPE below a threshold or posture-stability criterion), and discuss how the failure cases affect the reported averages.","section":"§5.2, Table 4, Fig. 14"}],"minor_comments":[{"comment":"Minor notation: 'Where' should be lowercase 'where'; also the set B is used for matched bones but the superscripts h and r on u are not defined until the following sentence. Consider defining the skeleton notation before the equation.","section":"§3.2, Eq. (1)"},{"comment":"The 'Regularization rewards' row refers to prior work (Cheng et al., 2024; He et al., 2024; Ji et al., 2025) without specifying the actual reward expressions. This makes the reward design not fully reproducible from the paper alone; please include the explicit formulas or a supplementary appendix.","section":"§3.3, Table 1"},{"comment":"The data-splitting description says '80% of the data for model development (training and evaluation), with performance evaluated via 5-fold cross-validation. The remaining 20% was used as a test set.' It would be clearer to state whether the 5-fold cross-validation is on the 80% split and what the test-set size is, especially since Table 3 reports test-set MPJPE.","section":"§4.2"},{"comment":"The comparison against baselines reports only point estimates of MPJPE with no variance or statistical significance. Adding standard deviations and, if feasible, paired tests across the same evaluation trials would strengthen the comparison.","section":"§5.2, Table 8"},{"comment":"The Conclusion says 'a reliable task success rate' but no rate is given anywhere. Please either report a measured success rate or remove the phrase.","section":"§7"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is attractive and the system description is unusually complete for a construction-robotics paper. The main risk is that the quantitative claims in the abstract and conclusion outrun the evidence: the only action-tracking number is sim-to-sim, and the retargeting ground truth is manual and unvalidated. Both issues are fixable in revision, but they are load-bearing for the paper's central message. I would also encourage the editor to ask for a data-availability statement including the manual retargeting protocol, since reproducibility hinges on the labels."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a coherent feasibility study that integrates human pose estimation, retargeting, and teacher-student RL for construction-like whole-body motions. The new content is the construction-specific integration and a new 30-action dataset with five subjects. That dataset and the careful documentation of the pipeline have value.\n\nThe paper does some things well. The ablation on the retargeting losses shows each contributes. The sim-to-sim transfer to MuJoCo is a sensible safety gate. Comparing against ExBody, ExBody2, and OmniH2O on the same motion benchmark is the right kind of evaluation, even if the baselines are not optimized for construction postures.\n\nThe soft spots are in the evidence for the central claim. The 82.45 mm MPJPE in the abstract is a sim-to-sim number from IsaacGym to MuJoCo. The real-robot section says the G1 reproduced all eight actions and that performance was consistent with sim-to-sim, but there are no real-robot MPJPE numbers, no trial counts, no success criteria, and only three photos. So the phrase 'reliably executed' goes beyond what is measured. Also, the retargeting ground truth is a manual retargeting process from the authors; the 48.46 mm retargeting MPJPE is with respect to those manual labels, so it is an internal consistency check rather than an absolute measure. The paper does acknowledge its own limitations—small subject pool, no tool interaction, sim-to-sim failures—which is good, but those limitations undercut the abstraction claim.\n\nThe citation pattern is fine; the related work is up to date and the baselines are legitimate. The math and pipeline are internally coherent, and I do not see a fatal flaw. What is missing is the real-robot measurement. A serious editor should send this to peer review because the dataset and integrated system warrant scrutiny, but the authors should be asked to either add real-robot quantitative results or soften the abstract claim to sim-to-sim demonstration plus preliminary real-world execution.","headline":"The integration is real and the dataset is useful, but the headline claim of reliable real-world execution leans on a sim-to-sim number.","tokens_in":31710,"tokens_out":2294,"would_cite":true,"duration_ms":24543,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-stage perception-and-action pipeline lets a humanoid robot learn construction actions from RGB worker demonstrations, reproducing eight actions with an average mean-per-joint position error of 82.45 mm.","keywords":["Humanoid robot","Construction robotics","Human motion estimation","Motion retargeting","Whole-body control","Reinforcement learning","Sim-to-real transfer","Worker demonstration"],"falsifier":"Take the manually retargeted ground-truth humanoid poses, run them through the physical robot's inverse kinematics and a balance/feasibility check, and count how many fall outside joint limits, violate foot-contact constraints, or would pitch the robot over. If a substantial share are infeasible, the 48.46 mm retargeting error and the 82.45 mm execution error are measured against targets the robot cannot physically follow, and the central claim fails.","tokens_in":30695,"feed_emoji":"🤖","tokens_out":9524,"duration_ms":95445,"temperature":0.7,"pith_summary":"Construction sites are built for human bodies, so a bipedal humanoid that could copy what workers do might take over heavy or dangerous tasks without needing redesigned workspaces. This paper argues that a two-stage pipeline can do that from ordinary RGB video: the first stage estimates a worker's 3D pose and retargets it into joint positions a humanoid can physically adopt; the second stage trains a reinforcement-learning controller to follow those retargeted motions while keeping balance and foot contact. The supporting experiment used demonstrations of 30 construction-related actions from five subjects and reports that eight selected actions were reproduced in simulation and on a physical humanoid robot, with an average whole-body tracking error of 82.45 mm MPJPE. If those numbers hold, the system is an early proof that construction skills can be transferred to humanoids from observation alone, without task-specific programming. The claim is about motion-tracking fidelity, not about whether tools or materials are successfully manipulated.","feed_headline":"Humanoid robot replays eight construction actions from video demos","feed_subtitle":"A two-stage pipeline turns worker videos into stable whole-body robot motion, no task-specific programming required.","key_machinery":"The load-bearing piece is the pair of learned modules. Humanoid-PoseNet is a human-to-humanoid retargeting network: two encoders, one for human poses and one for robot poses, map poses into a shared latent space, and a decoder produces humanoid pose targets. A triplet loss built on bone-direction angular error aligns similar human and robot poses, while reconstruction and latent-consistency losses keep outputs inside the robot's feasible configuration. Humanoid-ActionNet is a teacher-student reinforcement-learning controller: a teacher policy trained with PPO on privileged simulation observations and physics-aware rewards—motion tracking, balance, root velocity, body orientation, foot contac","core_discovery":"On the paper's own terms, the central discovery is that a humanoid robot can learn a useful set of construction motions from human demonstrations through a vision-based perception-and-action (VPA) system, and that the learned motions transfer from simulation to the physical robot. The system's Humanoid-PoseNet reconstructs 3D human keypoints from RGB video and maps them through a shared latent space into a 14-joint humanoid pose representation, reporting a human-to-humanoid retargeting error of 48.46 mm MPJPE on held-out data. Humanoid-ActionNet then trains a teacher policy with PPO and distills it into a student policy that executes the retargeted poses on a 23-DoF humanoid while preserving","pith_inferences":["Editorial inference: The reported 82.45 mm MPJPE measures how closely the robot's joints track reference poses, not whether a brick was placed or a pipe delivered; task success is a separate quantity the paper does not measure.","Editorial inference: The manual retargeting step that produced ground-truth labels is the pipeline's hidden dependency; unless those labels are checked against inverse kinematics or balance feasibility, both the retargeting error and the execution error are relative to labels that may themselves be imperfect.","Editorial inference: Because physical deployment tested only the actions that passed the cross-simulator filter, the system's real-world generality is conditioned on that filter; actions that fail in one simulator remain untested on hardware.","Editorial inference: The latent-space retargeting suggests a route to cross-robot transfer: the same human demonstration could be decoded into pose targets for different humanoids without re-collecting demonstrations, provided the target robot's kinematics are represented in the same way."],"forward_implications":["If the reported numbers are representative, construction-specific humanoid skills can be initialized from ordinary video of human workers, so no specialized teleoperation rig or task-specific controller is needed to capture the motions.","The retargeting module is not tied to one robot: the paper shows it also produces coherent poses for a different humanoid configuration, suggesting the same learned latent space can be reused for other humanoids with compatible kinematics.","Because the off-the-shelf 3D pose estimators compared in the paper give near-identical downstream retargeting quality, the system's front end can be swapped as pose-estimation models improve.","The sim-to-sim evaluation in a second physics engine acts as a safety gate: actions that do not survive transfer are not deployed on hardware, so the physical-robot results are the filtered subset that already passed that robustness check.","The whole pipeline is an early path toward autonomous humanoid collaborators on construction sites, where the near-term expectation is worker-like whole-body motion, not dexterous tool use."],"supporting_citations":[{"why":"Supplies the PoseNet architecture chosen as the 3D pose-estimation backbone of Humanoid-PoseNet after comparative evaluation.","marker":"Moon et al., 2019"},{"why":"Provides the public pretraining dataset that is fine-tuned with the self-collected RGB-3D samples.","marker":"Ionescu et al., 2014"},{"why":"Serves as the VideoPose3D comparative baseline in the pose-estimation and retargeting evaluation.","marker":"Pavllo et al., 2019"},{"why":"Defines the ExBody whole-body control baseline and informs the reward design and teacher-student structure.","marker":"Cheng et al., 2024"},{"why":"Provides the OmniH2O baseline and the teacher-student distillation approach for deployable policies.","marker":"He, Luo, He et al., 2024"},{"why":"Provides the ExBody2 baseline and additional evidence for the observation design used in Humanoid-ActionNet.","marker":"Ji et al., 2025"},{"why":"Supplies PPO, the actor-critic optimization algorithm used to train the teacher policy.","marker":"Schulman et al., 2017"},{"why":"Supplies DAgger, the dataset-aggregation procedure used to distill the teacher policy into the student policy.","marker":"Ross et al., 2011"},{"why":"Motivates the cross-simulator safety evaluation by documenting physics-engine mismatches.","marker":"Erez et al., 2015"}],"fun_headline_variants":["Video-trained humanoid executes 8 construction tasks","Humanoid robot learns 8 construction actions from video","Two-stage pipeline maps human video to humanoid actions","No task-specific code: humanoid learns 8 construction actions","From worker demo to robot motion: 8 construction actions"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The pipeline assumes the manually retargeted humanoid poses used as ground truth for both training and evaluation are correct, since the reported retargeting and execution errors are measured against those labels.","fun_headline_variants_meta":{"raw":{"variants":["Video-trained humanoid executes 8 construction tasks","Humanoid robot learns 8 construction actions from video","Two-stage pipeline maps human video to humanoid actions","No task-specific code: humanoid learns 8 construction actions","From worker demo to robot motion: 8 construction actions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001137,"raw_usage":{"total_tokens":4544,"prompt_tokens":713,"completion_tokens":3831,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":457,"completion_tokens_details":{"reasoning_tokens":3763}},"tokens_in":457,"tokens_out":3831,"duration_ms":33322,"temperature":1.0,"reasoning_tokens":3763,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T00:19:21.247152+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the manually retargeted ground-truth humanoid poses, run them through the physical robot's inverse kinematics and a balance/feasibility check, and count how many fall outside joint limits, violate foot-contact constraints, or would pitch the robot over. If a substantial share are infeasible, the 48.46 mm retargeting error and the 82.45 mm execution error are measured against targets the robot cannot physically follow, and the central claim fails.","supporting_citations":[],"review_version":1}