{"id":"7da1af63-deb9-4a9b-8464-c33ccc74835b","arxiv_id":"2611.17060","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"A large-scale dataset pairing metric 3D scene scans with egocentric human activity recordings, with a pipeline for registration and reuse in robot learning.","lead":"Ropedia Xperience 2 combines 1,589 metric 3D scans of real environments with 4,230 egocentric recordings of 74 everyday tasks, totaling 131 hours of activity. The dataset provides a shared spatial frame for human behavior and scene geometry, enabling robot learning applications without requiring robots during data collection.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"No significant objection identified: the central claims are supported under the paper's stated scope, with scale-level registration quality as the residual uncertainty and all relevant failure modes already acknowledged by the authors.","rationale":"The reader's weakest assumption (registration reliability) is the right place to look, and it is indeed the least-secure link in the reuse argument. But the paper does not depend on unqualified reliability: it explicitly labels motion estimates as estimated (b210), acknowledges the single-transform limitation (b110, b211, b373), the rigid-registration regime under 2 m (b371), and the downward-looking localization difficulty (b372), and it provides registration QC labels in Section 3.3 and Appendix B.3 so users can filter. The four reuse routes are presented as demonstrations on a small number of scenes, not as evidence that all 4,230 recordings pass registration, and the benchmarks are explicitly scoped to 20 scenes. No internally inconsistent or unsupported load-bearing step remains. The one check worth running is a scale-level PASS/WEAK/FAIL distribution; if that distribution were very poor, the scaling-law framing would weaken, but the current manuscript is not wrong as written. Therefore the reader's ACCEPT verdict stands unchanged.","tokens_in":25064,"tokens_out":9709,"duration_ms":68931,"concrete_test":"Run the Appendix B.3 registration QC over all released recordings, or over a stratified random sample of at least 100 recordings spanning the seven task actions and the below-2 m rigid-registration regime, and publish the PASS/WEAK/FAIL counts plus the per-stratum median and p90 held-out error. If the PASS fraction is materially below what the 4,230-recording inventory implies, the scaling-law framing should be revised to describe a filterable pipeline rather than a fully registered joint dataset.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest candidate concern is whether scan–HOMIE registration is reliable enough at the scale claimed: the reuse routes in Section 5.2 all presuppose the Section 3.3 transform, yet the paper reports detailed registration QC for one example recording (Figure 5: held-out error 0.7/1.8 cm) and four retargeting demonstrations in one scene. If most of the 4,230 recordings were unregistered or FAIL-labeled, the 'shared metric frame' novelty would be much weaker. However, this concern does not land as a decisive objection. The paper explicitly acknowledges the failure modes that would make registration unreliable: a single transform cannot correct local SLAM drift (b110, b211, b373), recordings with trajectory span below about 2 m are registered rigidly (b371), downward-looking frames localize less often (b372), and hand and body motion are estimated rather than motion-capture ground truth (b210). It also ships per-registration QC labels so downstream users can audit and filter. The central claim is a data-collection protocol and released resource, not a claim that every recording achieves a specified registration accuracy; the benchmarks are explicitly scoped to 20 evaluation scenes and the reuse routes are presented as demonstrations. Under the stated scope, the argument is internally consistent and the acknowledged limitations do not undermine acceptance.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"The paper introduces Ropedia Xperience 2, a dataset comprising 1,589 metric-scale 3D scans and 4,230 egocentric recordings (131 hours) of 74 tasks, along with a pipeline for registering recordings to scans. It proposes this as a gold standard for physical AI data collection, supporting benchmarks (depth/geometry, 3D detection, spatial reasoning) and four reuse routes (real-to-sim, human-robot retargeting, virtual teleoperation, long-horizon trajectory generation). The central claim is that joint capture of environments and behaviors, with spatial registration, provides a data foundation for scaling laws in physical AI.","tokens_in":25440,"tokens_out":289,"duration_ms":41329,"significance":"If the dataset and pipeline deliver on their promises, this is a significant contribution. It addresses a bottleneck in physical AI: the lack of paired, metric-scale environment geometry and natural human behavior demonstrations. The scale (1,589 scans, 131 hours of recordings) is substantial, and the four reuse routes offer concrete paths for generating robot training data without requiring robots on-site. The benchmarks provide standardized evaluations. The paper is transparent about limitations (estimated motion, registration drift, scene composition biases). The resource is released, which enables reproducibility and community use.","major_comments":[],"minor_comments":[],"recommendation":"accept"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"What you should know: this is a well-built, honestly-scoped dataset paper, and the one number I most want from it — the aggregate PASS/WEAK/FAIL distribution over the 4,230 recordings — is the one it doesn't report. The residual risk is real, but the core claim holds under the stated scope.\n\nWhat's genuinely new is the combination, at this scale, of metric LiDAR scans with egocentric task recordings registered into a shared frame. ScanNet++/DL3DV give you scenes; Ego4D/Nymeria give you egocentric activity; nobody ships both with a registration pipeline and per-recording QC labels. The collection protocol is simple — scan once, record N tasks with the head rig, no robots or mocap on site — and the four reuse routes are demonstrated with concrete numbers: median fingertip error 0.40–0.55 cm over 10.3 minutes of retargeted motion, and 30/30 held-out rollouts on an eight-subtask chained task.\n\nThe QC transparency is the best part. Exact grade counts (148 Perfect, 934 Usable, 507 Reuse of 1,589 retained scans), policy overrides disclosed (91 scans retained without explicit QC, 33 with conflicting grades resolved), and a limitations section that reads like people who actually ran the pipeline: estimated hand/body motion, model-generated captions, single-transform registration cannot fix local SLAM drift, sub-2m recordings registered rigidly, downward-looking frames localize poorly. They ship per-recording PASS/WEAK/FAIL labels, so downstream users can filter.\n\nSoft spots, in proportion. The largest is registration QC reporting. Figure 5 gives held-out error for one example recording, and the retargeting demos run in a single scene. The paper never states how many of the 4,230 recordings pass registration. If a large share are FAIL, the shared-metric-frame novelty shrinks to the pipeline plus a minority of recordings. That does not sink the paper — the claim is the protocol and the resource, and the labels let users audit — but it is the first question a referee will ask, and the answer is in data the authors already have.\n\nSecond, the reuse routes are demonstrations, not evaluations: one teleoperation interaction, one eight-subtask generation task. Fine as proofs of concept; the gold-standard and scaling-laws framing is aspirational, and the abstract's wording will raise expectations the paper does not meet by design.\n\nThird, the data is home-heavy (87.4%) and 31.9% of scans are Reuse grade, much worse in kitchens and bathrooms. Disclosed, but it caps the diversity claim for scaling-law study.\n\nMinor: the abstract mixes released scans (1,589) with inventory counts (4,230 recordings); Table 3 is precise, so it is a headline-phrasing issue.\n\nWho it is for: egocentric 3D, robot learning from human demonstrations, physical-AI data curation. It deserves a serious referee; in revision I would ask for the aggregate registration QC distribution and a registration-success breakdown by scene grade.\n\nMy recommendation: engage — accept with that request.","headline":"A solid, honestly-scoped dataset release pairing metric scans with egocentric recordings; the one missing number is the aggregate registration QC distribution, and that is what I would ask for in revision.","tokens_in":25996,"tokens_out":4845,"duration_ms":35602},"pith_extraction":{"msc":["68T45","68T40"],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"Ropedia Xperience 2 claims that physical AI data must pair 3D environments with human behaviors, and delivers 1,589 scans plus 4,230 egocentric task recordings fused by registration into robot-ready training data.","keywords":["physical AI","egocentric video dataset","3D scene reconstruction","scan-recording registration","human-robot retargeting","real-to-sim","long-horizon manipulation","spatial reasoning benchmark"],"falsifier":"Run a HOMIE recording in a room equipped with optical motion capture, register it to the scene's LiDAR scan, and compare the pipeline's SMPL-H body joints and camera trajectory against marker-based ground truth across recordings of increasing length (roughly 2, 10, and 30 minutes): if median joint error grows with recording duration, or exceeds the fingertip thresholds reported for the four demonstrations, then single-transform registration is the bottleneck for the retargeting and trajectory-reuse claims.","tokens_in":26586,"feed_emoji":"🤖","tokens_out":11754,"duration_ms":68377,"temperature":0.7,"pith_summary":"Ropedia Xperience 2 proposes that the data problem for physical AI is not human behavior alone or environments alone, but the two captured together in a shared metric frame. The paper introduces exactly such a resource: 1,589 released LiDAR scans of real homes, restaurants, stores, offices, and outdoor sites, paired with 4,230 egocentric recordings of 74 everyday manipulation tasks totalling 131 hours, collected by a single operator with no robots or motion-capture equipment on site. The load-bearing mechanism is a registration pipeline that places each recording's camera trajectory and estimated hand and body motion into the scanned scene's coordinate frame, so video, human motion, language annotations, and interactive 3D geometry (mesh, 3D Gaussian splatting, simulator scene) all describe one physical space. The paper then demonstrates four ways a capture is reused for robot learning — real-to-sim conversion, human-robot retargeting, virtual teleoperation, and automated long-horizon trajectory generation — and argues that this joint capture is a gold standard that lays the data foundation for scaling laws over environments and behaviors.","feed_headline":"1,589 scans fused with 131 hours of human tasks feed robot learning","feed_subtitle":"One capture yields simulator scenes, robot retargets, and long-horizon rollouts, with no robot on site.","key_machinery":"The carrying mechanism is the scan–HOMIE registration pipeline (Section 3.3), fed by the two-stage capture protocol. A scene is scanned once into a metric, gravity-aligned mesh and 3D Gaussian splatting model; each assigned task is then recorded with a head-mounted four-fisheye rig whose SLAM trajectory, MANO hand estimates (21 joints per hand), and SMPL-H body estimates (52 joints) are computed in the recording's own frame. Registration places the recording into the scene frame by SIFT matching, RANSAC, and PnP against scan keypoints lifted onto the mesh, followed by a robust Umeyama alignment of camera centres, and grades the result PASS, WEAK, or FAIL using held-out frames, IMU gravity consistency, and an ICP check of stereo depth against the mesh. This single shared coordinate frame is what lets the four reuse routes draw on the same capture: the scan becomes a physics-enabled simulator scene, registered human motion becomes robot joint trajectories, and the simulator scene becomes the stage for teleoperation and automated rollouts.","core_discovery":"The central claim is that a two-stage capture protocol — scanning a static scene once with a handheld LiDAR device, then recording its assigned tasks with a head-mounted egocentric camera rig — yields a capture from which both environments and behaviors can be scaled for robot learning without further on-site collection. The paper reports 1,589 released scans retained from 1,610 graded completed scans (98.7%), graded Perfect, Usable, or Reuse, and an inventory spanning 596 locations, 4,230 task recordings, 74 tasks, and 131 hours of video. On the reuse side it reports that four full recordings, 10.3 minutes of motion, retarget to a wheeled humanoid with median fingertip errors of 0.40–0.55 cm and every frame passing motion, balance, and collision checks; that a converted simulator scene supports live teleoperation of a kettle grasp, transfer, and placement; and that an eight-subtask long-horizon task succeeds from all 30 held-out start states with no task-rule violations, at a cost of 10–16 wall-clock minutes per trajectory. The same 20 scanned scenes function as reference geometry for three benchmarks — depth and geometry estimation across three input densities, 3D object detection, and spatial reasoning — where current models leave clear headroom, notably no method above 7 AP50 on tight 3D boxes. The paper's stated aim is to establish joint capture, spatial registration, and reuse as the gold standard for physical AI data collection and thereby the foundation for studying how capability scales with environments and behaviors.","pith_inferences":["If the registration premise holds, the dataset's effective volume for training grows combinatorially: one scan plus one recording expands into many simulated rollouts, so further scaling could become a compute question rather than a capture question.","The paper leaves per-segment drift correction to future work; the natural experiment is to register long recordings in overlapping windows and test whether retargeting success and usable trajectory length improve, which would isolate the single-transform limit as the binding constraint.","The scaling-law framing invites a test the paper does not run: with the capture protocol held fixed, add scenes and recordings incrementally and fit a curve of downstream policy success against environment and behavior counts — evidence of a power law would confirm the gold-standard framing.","The wide gaps on the spatial-reasoning and detection benchmarks suggest the metric scene geometry itself is under-used by current models; methods that consume the LiDAR reference directly rather than only posed images are a plausible next step that the dataset makes possible."],"forward_implications":["Robot data scales with operators instead of robots: one scene scan supports many task recordings, and each registered recording yields robot demonstrations without deploying any hardware on site.","Beyond the initial capture, additional robot behavior costs only compute: the paper reports 30/30 rollouts of an eight-subtask long-horizon task from 30 held-out start states, at 10–16 wall-clock minutes per trajectory on one simulation worker.","One collected scene serves two purposes at once — as reference geometry for benchmarking existing vision models and as a source of interactive environments and demonstrations for robot learning.","The three benchmarks quantify where current models stand: no single geometry model leads across all metrics, no detection method exceeds 7 AP50, and no spatial-reasoning model exceeds 34% on distance questions, so the released scans give the field a fixed reference against which future progress can be measured."],"supporting_citations":[{"why":"The predecessor egocentric dataset this work extends; supplies the behaviour-capture lineage that Xperience 2 augments with metric scans and registration.","marker":"[47]"},{"why":"Produces the 3D Gaussian splatting scene model used for photoreal rendering, registration views, and simulator appearance.","marker":"[20]"},{"why":"Defines the MANO hand and SMPL-H body models that represent the estimated human motion later retargeted to robots.","marker":"[46]"},{"why":"Estimates full-body motion from the SLAM head trajectory and the pipeline's hand estimates, including the unobserved legs.","marker":"[62]"},{"why":"Provides the SIFT keypoint matcher that initialises the recording-to-scan registration.","marker":"[31]"},{"why":"Supplies the RANSAC robust-matching scheme used to find consistent correspondences during registration.","marker":"[15]"},{"why":"Gives the robust least-squares alignment that fits the recording-to-scene transform from camera centres.","marker":"[54]"},{"why":"Provides the ICP geometry check used to grade registrations and flag mismatched spaces.","marker":"[4]"},{"why":"Supplies the scan camera model and reference poses used for registration inputs and benchmark ground truth.","marker":"[48]"},{"why":"The real-robot teleoperation dataset used as the reference cost and output baseline for the four reuse routes.","marker":"[21]"}],"fun_headline_variants":["One capture protocol yields 1,589 scenes and 131 hours of robot training data","Scans plus head-mounted task videos become simulator scenes and robot trajectories","Ropedia Xperience 2: one static scan and one video pass feed robot learning","From captured scenes to long-horizon rollouts without a robot on site"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The whole reuse economy rests on the assumption that registration and motion estimates are accurate enough for real robot learning; the paper itself states that a single transform cannot correct local SLAM drift (b110, b211, b373), that recordings with trajectories under about 2 metres are registered rigidly with unconstrained scale (b371), that downward-looking frames localize less reliably (b372), and that hand and body motion are estimated rather than motion-capture ground truth (b210) — if these estimates degrade on long or hard recordings, the retargeted trajectories and simulated rollouts inherit the error.","fun_headline_variants_meta":{"raw":{"variants":["One capture protocol yields 1,589 scenes and 131 hours of robot training data","Scans plus head-mounted task videos become simulator scenes and robot trajectories","Ropedia Xperience 2: one static scan and one video pass feed robot learning","From captured scenes to long-horizon rollouts without a robot on site"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.000372,"raw_usage":{"cost":0.0003718078,"is_byok":false,"cost_details":{"upstream_inference_cost":0.0003718078,"upstream_inference_prompt_cost":3.91118e-05,"upstream_inference_completions_cost":0.000332696},"total_tokens":3021,"prompt_tokens":1193,"completion_tokens":1828,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":954,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":1750}},"tokens_in":1193,"tokens_out":1828,"cost_source":"openrouter_usage","duration_ms":13033,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":1750,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-11-13T10:38:57.334364+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"Run a HOMIE recording in a room equipped with optical motion capture, register it to the scene's LiDAR scan, and compare the pipeline's SMPL-H body joints and camera trajectory against marker-based ground truth across recordings of increasing length (roughly 2, 10, and 30 minutes): if median joint error grows with recording duration, or exceeds the fingertip thresholds reported for the four demonstrations, then single-transform registration is the bottleneck for the retargeting and trajectory-reuse claims.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The predecessor egocentric dataset this work extends; supplies the behaviour-capture lineage that Xperience 2 augments with metric scans and registration."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Produces the 3D Gaussian splatting scene model used for photoreal rendering, registration views, and simulator appearance."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the MANO hand and SMPL-H body models that represent the estimated human motion later retargeted to robots."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Estimates full-body motion from the SLAM head trajectory and the pipeline's hand estimates, including the unobserved legs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the SIFT keypoint matcher that initialises the recording-to-scan registration."},{"cited_title":"Fischler and Robert C","cited_arxiv_id":null,"evidence_quote":"Supplies the RANSAC robust-matching scheme used to find consistent correspondences during registration."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the robust least-squares alignment that fits the recording-to-scene transform from camera centres."},{"cited_title":"Besl and Neil D","cited_arxiv_id":null,"evidence_quote":"Provides the ICP geometry check used to grade registrations and flag mismatched spaces."},{"cited_title":"Schönberger and Jan-Michael Frahm","cited_arxiv_id":null,"evidence_quote":"Supplies the scan camera model and reference poses used for registration inputs and benchmark ground truth."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The real-robot teleoperation dataset used as the reference cost and output baseline for the four reuse routes."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}