Pith.

Computer Vision and Pattern Recognition · arXiv:2611.17060 v1 · submitted 12 Nov 2026 · 69 references

Ropedia Xperience 2: A Gold Standard for Scaling Physical AI Environments and Behaviors

Hao Li, Fangzhou Hong, Zhaoxi Chen, Hui En Pang, Kairui Hu, Mingze Sun, Runmao Yao, Xinhao Ji

show 13 more authors

Xiyuan Zheng, Xuanli Chen, Yalun Dai, Yap Kim Hui, Dingwen Zhang, Yuanyuan Gao, Ziang Cao, Cheng Huang Yean, Jackson Peng, Rui Duan, Yaobo Liu, Zetong Wang, Ziwei Liu

The paper says

“Large foundation models such as GPT-6 Astra and Claude Fable 5.1 have recently been shown to control robots directly from visual observations. They nevertheless remain brittle on long-horizon, contact-rich tasks in real-world environments.”

more of the abstract

Further progress depends on two complementary resources: demonstrations of human behavior that supervise how tasks are performed, and interactive environments in which policies can be rolled out, evaluated, and improved in closed loop. Language models have advanced along well-characterized scaling laws; the corresponding question for physical AI is how capability scales with environments and behaviors, which first requires identifying which data to scale and how to produce and reuse such data continuously. To address this question, we introduce Ropedia Xperience 2, comprising 1,589 released metric-scale 3D scans and a collection of 4,230 egocentric recordings of 74 tasks (131 hours). We further show that each capture can be repurposed for robot learning: scenes are converted into physically interactive simulation environments, human motion is retargeted to whole-body robot trajectories, and additional robot behaviors are obtained through virtual teleoperation and automated long-horizon trajectory generation. We propose this joint capture of environments and behaviors as a gold standard for physical AI data collection, laying a data foundation for establishing scaling laws over environments and behaviors.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 13 Nov 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up. A solid, honestly-scoped dataset release pairing metric scans with egocentric recordings; the one missing number is the aggregate registration QC distribution, and that is what I would ask for in revision. This line is wrong

no objections raised

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 69 references say.

What would settle it

Run the Appendix B.3 registration QC over all released recordings, or over a stratified random sample of at least 100 recordings spanning the seven task actions and the below-2 m rigid-registration regime, and publish the PASS/WEAK/FAIL counts plus the per-stratum median and p90 held-out error. If the PASS fraction is materially below what the 4,230-recording inventory implies, the scaling-law framing should be revised to describe a filterable pipeline rather than a fully registered joint dataset.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 13 Nov 2026

  1. Reader holds up confidence high
  2. Skeptic partly agrees 1 min 9 s · 25,064 tokens read, 9,709 written
  3. Editor wrote the verdict above 36 s · 25,996 tokens read, 4,845 written
  4. Referee accept: no objections raised 41 s · 25,440 tokens read, 289 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

What you should know: this is a well-built, honestly-scoped dataset paper, and the one number I most want from it — the aggregate PASS/WEAK/FAIL distribution over the 4,230 recordings — is the one it doesn't report. The residual risk is real, but the core claim holds under the stated scope.

What's genuinely new is the combination, at this scale, of metric LiDAR scans with egocentric task recordings registered into a shared frame. ScanNet++/DL3DV give you scenes; Ego4D/Nymeria give you egocentric activity; nobody ships both with a registration pipeline and per-recording QC labels. The collection protocol is simple — scan once, record N tasks with the head rig, no robots or mocap on site — and the four reuse routes are demonstrated with concrete numbers: median fingertip error 0.40–0.55 cm over 10.3 minutes of retargeted motion, and 30/30 held-out rollouts on an eight-subtask chained task.

The QC transparency is the best part. Exact grade counts (148 Perfect, 934 Usable, 507 Reuse of 1,589 retained scans), policy overrides disclosed (91 scans retained without explicit QC, 33 with conflicting grades resolved), and a limitations section that reads like people who actually ran the pipeline: estimated hand/body motion, model-generated captions, single-transform registration cannot fix local SLAM drift, sub-2m recordings registered rigidly, downward-looking frames localize poorly. They ship per-recording PASS/WEAK/FAIL labels, so downstream users can filter.

Soft spots, in proportion. The largest is registration QC reporting. Figure 5 gives held-out error for one example recording, and the retargeting demos run in a single scene. The paper never states how many of the 4,230 recordings pass registration. If a large share are FAIL, the shared-metric-frame novelty shrinks to the pipeline plus a minority of recordings. That does not sink the paper — the claim is the protocol and the resource, and the labels let users audit — but it is the first question a referee will ask, and the answer is in data the authors already have.

Second, the reuse routes are demonstrations, not evaluations: one teleoperation interaction, one eight-subtask generation task. Fine as proofs of concept; the gold-standard and scaling-laws framing is aspirational, and the abstract's wording will raise expectations the paper does not meet by design.

Third, the data is home-heavy (87.4%) and 31.9% of scans are Reuse grade, much worse in kitchens and bathrooms. Disclosed, but it caps the diversity claim for scaling-law study.

Minor: the abstract mixes released scans (1,589) with inventory counts (4,230 recordings); Table 3 is precise, so it is a headline-phrasing issue.

Who it is for: egocentric 3D, robot learning from human demonstrations, physical-AI data curation. It deserves a serious referee; in revision I would ask for the aggregate registration QC distribution and a registration-success breakdown by scene grade.

My recommendation: engage — accept with that request.

The strongest objection

The strongest candidate concern is whether scan–HOMIE registration is reliable enough at the scale claimed: the reuse routes in Section 5.2 all presuppose the Section 3.3 transform, yet the paper reports detailed registration QC for one example recording (Figure 5: held-out error 0.7/1.8 cm) and four retargeting demonstrations in one scene. If most of the 4,230 recordings were unregistered or FAIL-labeled, the 'shared metric frame' novelty would be much weaker. However, this concern does not land as a decisive objection. The paper explicitly acknowledges the failure modes that would make registration unreliable: a single transform cannot correct local SLAM drift (b110, b211, b373), recordings with trajectory span below about 2 m are registered rigidly (b371), downward-looking frames localize less often (b372), and hand and body motion are estimated rather than motion-capture ground truth (b210). It also ships per-registration QC labels so downstream users can audit and filter. The central claim is a data-collection protocol and released resource, not a claim that every recording achieves a specified registration accuracy; the benchmarks are explicitly scoped to 20 evaluation scenes and the reuse routes are presented as demonstrations. Under the stated scope, the argument is internally consistent and the acknowledged limitations do not undermine acceptance.

The reader's weakest assumption (registration reliability) is the right place to look, and it is indeed the least-secure link in the reuse argument. But the paper does not depend on unqualified reliability: it explicitly labels motion estimates as estimated (b210), acknowledges the single-transform limitation (b110, b211, b373), the rigid-registration regime under 2 m (b371), and the downward-looking localization difficulty (b372), and it provides registration QC labels in Section 3.3 and Appendix B.3 so users can filter. The four reuse routes are presented as demonstrations on a small number of scenes, not as evidence that all 4,230 recordings pass registration, and the benchmarks are explicitly scoped to 20 scenes. No internally inconsistent or unsupported load-bearing step remains. The one check worth running is a scale-level PASS/WEAK/FAIL distribution; if that distribution were very poor, the scaling-law framing would weaken, but the current manuscript is not wrong as written. Therefore the reader's ACCEPT verdict stands unchanged.

The referee's report

If the dataset and pipeline deliver on their promises, this is a significant contribution. It addresses a bottleneck in physical AI: the lack of paired, metric-scale environment geometry and natural human behavior demonstrations. The scale (1,589 scans, 131 hours of recordings) is substantial, and the four reuse routes offer concrete paths for generating robot training data without requiring robots on-site. The benchmarks provide standardized evaluations. The paper is transparent about limitations (estimated motion, registration drift, scene composition biases). The resource is released, which enables reproducibility and community use.

Pith's reading of the paper (a machine's summary, not the authors')

Ropedia Xperience 2 proposes that the data problem for physical AI is not human behavior alone or environments alone, but the two captured together in a shared metric frame. The paper introduces exactly such a resource: 1,589 released LiDAR scans of real homes, restaurants, stores, offices, and outdoor sites, paired with 4,230 egocentric recordings of 74 everyday manipulation tasks totalling 131 hours, collected by a single operator with no robots or motion-capture equipment on site. The load-bearing mechanism is a registration pipeline that places each recording's camera trajectory and estimated hand and body motion into the scanned scene's coordinate frame, so video, human motion, language annotations, and interactive 3D geometry (mesh, 3D Gaussian splatting, simulator scene) all describe one physical space. The paper then demonstrates four ways a capture is reused for robot learning — real-to-sim conversion, human-robot retargeting, virtual teleoperation, and automated long-horizon trajectory generation — and argues that this joint capture is a gold standard that lays the data foundation for scaling laws over environments and behaviors.

What carries the argument. The carrying mechanism is the scan–HOMIE registration pipeline (Section 3.3), fed by the two-stage capture protocol. A scene is scanned once into a metric, gravity-aligned mesh and 3D Gaussian splatting model; each assigned task is then recorded with a head-mounted four-fisheye rig whose SLAM trajectory, MANO hand estimates (21 joints per hand), and SMPL-H body estimates (52 joints) are computed in the recording's own frame. Registration places the recording into the scene frame by SIFT matching, RANSAC, and PnP against scan keypoints lifted onto the mesh, followed by a robust Umeyama alignment of camera centres, and grades the result PASS, WEAK, or FAIL using held-out frames, IMU gravity consistency, and an ICP check of stereo depth against the mesh. This single shared coordinate frame is what lets the four reuse routes draw on the same capture: the scan becomes a physics-enabled simulator scene, registered human motion becomes robot joint trajectories, and the simulator scene becomes the stage for teleoperation and automated rollouts.

Core claim. The central claim is that a two-stage capture protocol — scanning a static scene once with a handheld LiDAR device, then recording its assigned tasks with a head-mounted egocentric camera rig — yields a capture from which both environments and behaviors can be scaled for robot learning without further on-site collection. The paper reports 1,589 released scans retained from 1,610 graded completed scans (98.7%), graded Perfect, Usable, or Reuse, and an inventory spanning 596 locations, 4,230 task recordings, 74 tasks, and 131 hours of video. On the reuse side it reports that four full recordings, 10.3 minutes of motion, retarget to a wheeled humanoid with median fingertip errors of 0.40–0.55 cm and every frame passing motion, balance, and collision checks; that a converted simulator scene supports live teleoperation of a kettle grasp, transfer, and placement; and that an eight-subtask long-horizon task succeeds from all 30 held-out start states with no task-rule violations, at a cost of 10–16 wall-clock minutes per trajectory. The same 20 scanned scenes function as reference geometry for three benchmarks — depth and geometry estimation across three input densities, 3D object detection, and spatial reasoning — where current models leave clear headroom, notably no method above 7 AP50 on tight 3D boxes. The paper's stated aim is to establish joint capture, spatial registration, and reuse as the gold standard for physical AI data collection and thereby the foundation for studying how capability scales with environments and behaviors.

The premise it rests on. The whole reuse economy rests on the assumption that registration and motion estimates are accurate enough for real robot learning; the paper itself states that a single transform cannot correct local SLAM drift (b110, b211, b373), that recordings with trajectories under about 2 metres are registered rigidly with unconstrained scale (b371), that downward-looking frames localize less reliably (b372), and that hand and body motion are estimated rather than motion-capture ground truth (b210) — if these estimates degrade on long or hard recordings, the retargeted trajectories and simulated rollouts inherit the error.

If the paper is right (Pith's inferences, not the authors')

Robot data scales with operators instead of robots: one scene scan supports many task recordings, and each registered recording yields robot demonstrations without deploying any hardware on site.

Beyond the initial capture, additional robot behavior costs only compute: the paper reports 30/30 rollouts of an eight-subtask long-horizon task from 30 held-out start states, at 10–16 wall-clock minutes per trajectory on one simulation worker.

One collected scene serves two purposes at once — as reference geometry for benchmarking existing vision models and as a source of interactive environments and demonstrations for robot learning.

The three benchmarks quantify where current models stand: no single geometry model leads across all metrics, no detection method exceeds 7 AP50, and no spatial-reasoning model exceeds 34% on distance questions, so the released scans give the field a fixed reference against which future progress can be measured.

If the registration premise holds, the dataset's effective volume for training grows combinatorially: one scan plus one recording expands into many simulated rollouts, so further scaling could become a compute question rather than a capture question.

The paper leaves per-segment drift correction to future work; the natural experiment is to register long recordings in overlapping windows and test whether retargeting success and usable trajectory length improve, which would isolate the single-transform limit as the binding constraint.

The scaling-law framing invites a test the paper does not run: with the capture protocol held fixed, add scenes and recordings incrementally and fit a curve of downstream policy success against environment and behavior counts — evidence of a power law would confirm the gold-standard framing.

The wide gaps on the spatial-reasoning and detection benchmarks suggest the metric scene geometry itself is under-used by current models; methods that consume the LiDAR reference directly rather than only posed images are a plausible next step that the dataset makes possible.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Large foundation models such as GPT-6 Astra and Claude Fable 5.1 have recently been shown to control robots directly from visual observations. They nevertheless remain brittle on long-horizon, contact-rich tasks in real-world environments. Further progress depends on two complementary resources: demonstrations of human behavior that supervise how tasks are performed, and interactive environments in which policies can be rolled out, evaluated, and improved in closed loop. Language models have advanced along well-characterized scaling laws; the corresponding question for physical AI is how capability scales with environments and behaviors, which first requires identifying which data to scale and how to produce and reuse such data continuously. To address this question, we introduce Ropedia Xperience 2, comprising 1,589 released metric-scale 3D scans and a collection of 4,230 egocentric recordings of 74 tasks (131 hours). We further show that each capture can be repurposed for robot learning: scenes are converted into physically interactive simulation environments, human motion is retargeted to whole-body robot trajectories, and additional robot behaviors are obtained through virtual teleoperation and automated long-horizon trajectory generation. We propose this joint capture of environments and behaviors as a gold standard for physical AI data collection, laying a data foundation for establishing scaling laws over environments and behaviors.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-14

The authors' main claim, as the classifier picked it out 1.00

“We propose this joint capture of environments and behaviors as a gold standard for physical AI data collection, laying a data foundation for establishing scaling laws over environments and behaviors.”

How the main claim is stated: asserted 0.97

What the abstract itself states, as probabilities: claims first 0.13 · states limitation 0.15 · human participants 0.71 · controlled comparison 0.03 · randomized assignment 0.01 · states quantitative result 0.96 · compares with prior results 0.17 · mentions code or data availability 0.85

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Large foundation models such as GPT-6 Astra and Claude Fable 5.1 have recently been shown to control robots directly from visual observations. background 1.00 0.05 0.02 asserted 1.00
They nevertheless remain brittle on long-horizon, contact-rich tasks in real-world environments. background 0.66 0.13 0.03 asserted 1.00
Further progress depends on two complementary resources: demonstrations of human behavior that supervise how tasks are performed, and interactive environments in which policies can be rolled out, evaluated, and improved in closed loop. background 0.52 0.07 0.08 asserted 0.94
Language models have advanced along well-characterized scaling laws; the corresponding question for physical AI is how capability scales with environments and behaviors, which first requires identifying which data to scale and how to produce and reuse such data continuously. gap 0.59 0.06 0.06 asserted 0.95
To address this question, we introduce Ropedia Xperience 2, comprising 1,589 released metric-scale 3D scans and a collection of 4,230 egocentric recordings of 74 tasks (131 hours). contribution 0.95 0.94 0.12 asserted 1.00
We further show that each capture can be repurposed for robot learning: scenes are converted into physically interactive simulation environments, human motion is retargeted to whole-body robot trajectories, and additional robot behaviors are obtained through virtual teleoperation and automated long-horizon trajectory generation. method 0.89 0.94 0.12 asserted 1.00
We propose this joint capture of environments and behaviors as a gold standard for physical AI data collection, laying a data foundation for establishing scaling laws over environments and behaviors. contribution 0.60 0.62 0.79 asserted 0.97

What kind of paper the classifier reports this is: dataset or benchmark 1.00 · evidence from observation or experiment 0.92

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-14: 1 request, 5,781 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (11)
Figure 2 of the paper
Figure 2. From composite-scene modalities to robot data. Left: scene assets from the 3D scan and HOMIE behavior streams; lines link each modality to the applications that use it. Right: the output, gains, and cost per collection of each application, with real-robot teleoperation as the reference; chips mark applications that build on Real-to-Sim scenes. Bars are qualitative five-step ratings, not measurements; the measured and demonstrated notes refer to results from Section 5.2, and the reference row quotes DROID [21]. recorder, so that one scan provides the spatial context for multiple…
Figure 3 of the paper
Figure 3. Overview of data collection. (a) Before capture, each scene (Level 1) is assigned a set of tasks (Level 2). (b) The scene is scanned once with the 3D scanner, with overlapping coverage, revisits, detail views, and high and low viewpoints. (c) Its tasks are then recorded individually with HOMIE; each recording begins before the task and ends after it, and the saved file is verified. One scene scan provides the environmental context for multiple task recordings. Rooms, tasks, viewpoints, and device views are illustrative. • Long-horizon trajectory generation rolls out robots on new…
Figure 4 of the paper
Figure 4. Tasks by scene and action in the collection inventory. (a, b) Number of tasks per scene type and per action; each thumbnail is a representative HOMIE frame of that category. (c) Number of uploaded recordings per action and scene type, with log-scaled color; counts do not require completed processing or registration. 3. Processing and Annotation For each scene, the 3D scan provides a static, metric 3D reconstruction (Section 3.1). The HOMIE processing pipeline can turn task recordings into synchronized video, a device trajectory, and hand, body, and activity annotations (Section…
Figure 5 of the paper
Figure 5. Egocentric recordings registered into 3D Gaussian scene scans. Top: synchronized scene, motion, video, and activity annotation views of one recording. Orange/blue denote the body’s left/right sides; pink/grey show the past 20 s/next 12 s of the camera path. Bottom: 45 s motion excerpts from three further scenes, with ground tracks coloured by time and poses at the start, middle, and end of each window (earlier poses in pale colours). 3.1. 3D Scan Processing The 3D scan exports comprise fisheye images with their camera poses and a sparse point model in COLMAP format [48], a…
Figure 6 of the paper
Figure 6. QC decisions and released-scene composition. Of 1,610 completed, graded scans, 1,589 are retained (98.7%) and 21 are Delete (1.3%). Within the retained set, Perfect, Usable, and Reuse account for 9.3%, 58.8%, and 31.9%, respectively, summing to 100%. QC-viewer examples and their recorded findings illustrate each grade. Released scans Not released Collection sites Category PERFECT USABLE REUSE DELETE SITES sqrt scale HOME Living room 19 220 100 0 Bedroom 60 434 143 8 Bathroom 3 39 58 9 Kitchen 1 13 62 3 Multi-room 6 133 98 0 WORK / RETAIL Office 0 2 12 0 Meeting room 0 5 4 0 Store…
Figure 7 of the paper
Figure 7. Released scene grades, Delete decisions, and collection sites. Left: counts in 12 categories; shading for Perfect, Usable, and Reuse encodes the share within their row sum, excluding Delete. Delete counts are shown separately. Right: the 596 sites in the broader collection inventory, on a square-root scale; 443 homes span all home categories. Per-category totals are in
Figure 8 of the paper
Figure 8. Real-to-sim conversion of a scanned scene. (a) The converted simulator scene, with zoom-ins A and B rendered from the same camera in our scene, with the 3D box of each object, and in the scan’s 3DGS. (b) Three further views rendered from the 3DGS, from a scene built by GPT-6 Astra [41] from images without 3DGS input, and from our scene, with the 3D boxes of four matched objects. 5.2.1. Real-to-Sim Neural scene representations such as NeRF [35] support novel-view rendering, while Habi￾tat 2.0 and BEHAVIOR-1K provide environments for physics-based robot interaction [51, 26]. RialTo…
Figure 9 of the paper
Figure 9. Human-robot retargeting in a 3D scan. Left: one frame of a HOMIE recording retargeted to a wheeled humanoid, with the robot head view, a scene overview, and a timeline of base driving and per-frame fingertip error (clipped at 2 cm). Right: three further recordings in the same scene, each showing the scene overview with the robot boxed, a third-person zoom-in, and the robot head view. Tags give the median fingertip error over each recording. The orange skeleton is the operator; the grey body is the robot. Head views are pitched 40◦ below the head direction. 5.2.2. Human-Robot…
Figure 10 of the paper
Figure 10. Virtual teleoperation in a converted scene. (a) The operator setup, with a head-mounted display and hand input devices; arrows indicate operator commands and robot-view feedback. (b) Three stages of a kettle-transfer interaction, each shown from an external camera (top) and the robot camera (bottom): grasping the handle, lifting and transferring the kettle, and placing it on the cooktop and releasing it. The operator photograph illustrates the setup; the paired simulation views show the interaction stages. collection of new interactive robot demonstrations. The scene can be…
Figure 11 of the paper
Figure 11. Long-horizon trajectory generation. Rollouts of one eight-subtask task in a converted home environment. Left: an example rollout (rendered with a chase camera), with a frame as each subtask completes, the time since the start, and the steps that subtask’s policy program logged; the last cell gives the results of the 30 held-out rollouts per subtask and for the full task. Right: the scene’s top-down plan with the example base path coloured by subtask, the places where its subtasks end, and the start poses of the 30 held-out rollouts. 5.2.4. Long-Horizon Trajectory Generation…

Showing 10 of the paper's 11 figures. Figures are the authors', reproduced from the paper.

References (69)
  1. [1] Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Daniel Ho, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Eric Jang, Rosario Jauregui Ruano, Kyle Jeffrey, ... on PithPith's reviewarXiv
  2. [2] Armen Avetisyan, Christopher Xie, Henry Howard-Jenkins, Tsun-Yi Yang, Samir Aroudj, Suvam Patra, Fuyang Zhang, Duncan Frost, Luke Holland, Campbell Orme, Jakob Engel, Edward Miller, Richard Newcombe, and Vasileios Balntas. 2024. SceneScript: Reconstructing scenes with an autor... on Pith
  3. [3] Shuai Bai, Keqin Chen, Xuejing Liu, and 1 others. 2025. Qwen2.5-VL technical report. arXiv preprint arXiv:2502.13923. 17 on PithPith's reviewarXiv
  4. [4] Paul J. Besl and Neil D. McKay. 1992. A method for registration of 3-D shapes.IEEE T ransactions on Pattern Analysis and Machine Intelligence, 14(2):239–256 on Pith
  5. [5] Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, and... on PithPith's reviewarXiv
  6. [6] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alexander Herzog, Jasm... on PithPith's reviewarXiv
  7. [7] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakr- ishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Tomas Jackson, Sally Jesmonth, Nikhil J Joshi, Ryan Julian, Dmitry Kalashniko... on PithPith's reviewarXiv
  8. [8] Tianxing Chen, Yue Chen, Zixuan Li, and 1 others. 2026. RoboDojo: A unified sim-and-real benchmark for comprehensive evaluation of generalist robot manipulation policies. arXiv preprint arXiv:2607.04434 on PithPith's reviewarXiv
  9. [9] Xingyu Chen, Yue Chen, Yuliang Xiu, Andreas Geiger, and Anpei Chen. 2025. TTT3R: 3D reconstruction as test-time training. arXiv preprint arXiv:2509.26645 on PithPith's reviewarXiv
  10. [10] Cheng Chi, Zhenjia Xu, Chuer Pan, Eric Cousineau, Benjamin Burchfiel, Siyuan Feng, Russ Tedrake, and Shuran Song. 2024. Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots. arXiv preprint arXiv:2402.10329 on PithPith's reviewarXiv
  11. [11] Gheorghe Comanici, Eric Bieber, Mike Schaekermann, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261 on PithPith's reviewarXiv
  12. [12] Kai Deng, Zexin Ti, Jiawei Xu, Jian Yang, and Jin Xie. 2025. VGGT-Long: Chunk it, loop it, align it—pushing VGGT’s limits on kilometer-scale long RGB sequences. arXiv preprint arXiv:2507.16443 on PithPith's reviewarXiv
  13. [13] Daniel DeTone, Tianwei Shen, Fan Zhang, Lingni Ma, Julian Straub, Richard Newcombe, and Jakob Engel. 2026. Boxer: Robust lifting of open-world 2D bounding boxes to 3D. arXiv preprint arXiv:2604.05212 on PithPith's reviewarXiv
  14. [14] Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duck- worth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussai... on PithPith's reviewarXiv
  15. [15] Martin A. Fischler and Robert C. Bolles. 1981. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM, 24(6):381–395 on Pith
  16. [16] Galbot Team, Xuchuan Chen, Xiaoqian Cheng, Yu Deng, Lihe Ding, Shaocong Dong, and 1 others. 2026. System- atically exploring the capabilities of GPT-6 Astra as embodied policies.arXiv preprint arXiv:2609.38537 on PithPith's reviewarXiv
  17. [17] Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, Miguel Martin, Tushar Nagarajan, Ilija Radosavovic, Santhosh Ku- mar Ramakrishnan, Fiona Ryan, Jayant Sharma, Michael Wray, Meng... on Pith
  18. [18] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models.arXiv preprint arXiv:2001.08361 on PithPith's reviewarXiv
  19. [19] Nikhil Keetha, Norman Müller, Johannes Schönberger, Lorenzo Porzi, Yuchen Zhang, Tobias Fischer, Arno Knapitsch, Duncan Zauss, Ethan Weber, Nelson Antunes, Jonathon Luiten, Manuel Lopez-Antequera, Samuel Rota Bulò, Christian Richardt, Deva Ramanan, Sebastian Scherer, and Peter... on PithPith's reviewarXiv
  20. [20] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 2023. 3D gaussian splatting for real-time radiance field rendering.ACM T ransactions on Graphics, 42(4) on Pith
show 49 more
  1. [21] Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, and 1 others. 2024. DROID: A large-scale in-the-wild robot manipulation dataset. InRobotics: Sc... on Pith
  2. [22] Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. 2024. O... on PithPith's reviewarXiv
  3. [23] Alexander Ku, Peter Anderson, Roma Patel, Eugene Ie, and Jason Baldridge. 2020. Room-Across-Room: Multilin- gual vision-and-language navigation with dense spatiotemporal grounding. arXiv preprint arXiv:2010.07954 on PithPith's reviewarXiv
  4. [24] Vincent Leroy, Yohann Cabon, and Jérôme Revaud. 2024. Grounding image matching in 3D with MASt3R. arXiv preprint arXiv:2406.09756 on PithPith's reviewarXiv
  5. [25] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2024. LLaVA-OneVision: Easy visual task transfer. arXiv preprint arXiv:2408.03326. 18 on PithPith's reviewarXiv
  6. [26] Chengshu Li, Ruohan Zhang, Josiah Wong, Cem Gokmen, Sanjana Srivastava, Roberto Martín-Martín, Chen Wang, Gabrael Levine, Wensi Ai, Benjamin Martinez, Hang Yin, Michael Lingelbach, Minjune Hwang, Ayano Hiranaka, Sujay Garlanka, Arman Aydin, Sharon Lee, Jiankai Sun, Mona Anvari... on PithPith's reviewarXiv
  7. [27] Hao Li, Xiyuan Zheng, Yuanyuan Gao, Yalun Dai, Xinhao Ji, Xuan-Li Chen, Jing Zhao, Fangzhou Hong, Zhaoxi Chen, and Ziwei Liu. 2026. Holi-World: Building holistic world intelligence from human-ego experience.arXiv preprint arXiv:2610.26549 on PithPith's reviewarXiv
  8. [28] Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng on Pith
  9. [29] Haotong Lin, Sili Chen, Junhao Liew, Donny Y. Chen, Zhenyu Li, Guang Shi, Jiashi Feng, and Bingyi Kang. 2025. Depth Anything 3: Recovering the visual space from any views. arXiv preprint arXiv:2511.10647 on PithPith's reviewarXiv
  10. [30] Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, Xuanmao Li, Xingpeng Sun, Rohan Ashok, Aniruddha Mukherjee, Hao Kang, Xiangrui Kong, Gang Hua, Tianyi Zhang, Bedrich Benes, and Aniket Bera. 2024. DL3DV-10K: A large-sca... on Pith
  11. [31] David G. Lowe. 2004. Distinctive image features from scale-invariant keypoints.International Journal of Computer Vision, 60(2):91–110 on Pith
  12. [32] Lingni Ma, Yuting Ye, Fangzhou Hong, Vladimir Guzov, Yifeng Jiang, Rowan Postyeni, Luis Pesqueira, Alexander Gamino, Vijay Baiyya, Hyo Jin Kim, Kevin Bailey, David Soriano Fosas, C. Karen Liu, Ziwei Liu, Jakob Engel, Renzo De Nardi, and Richard Newcombe. 2024. Nymeria: A massi... on PithPith's reviewarXiv
  13. [33] Ajay Mandlekar, Soroush Nasiriany, Bowen Wen, Iretiayo Akinola, Yashraj Narang, Linxi Fan, Yuke Zhu, and Dieter Fox. 2023. MimicGen: A data generation system for scalable robot learning using human demonstrations. arXiv preprint arXiv:2310.17596 on PithPith's reviewarXiv
  14. [34] Yongsen Mao, Junhao Zhong, Chuan Fang, Jia Zheng, Rui Tang, Hao Zhu, Ping Tan, and Zihan Zhou. 2025. SpatialLM: Training large language models for structured indoor modeling. arXiv preprint arXiv:2506.07491 on PitharXiv
  15. [35] Ben Mildenhall, Pratul P . Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. 2020. NeRF: Representing scenes as neural radiance fields for view synthesis. InEuropean Conference on Computer Vision on Pith
  16. [36] Riku Murai, Eric Dexheimer, and Andrew J. Davison. 2024. MASt3R-SLAM: Real-time dense SLAM with 3D reconstruction priors. arXiv preprint arXiv:2412.12392 on PithPith's reviewarXiv
  17. [37] Open X-Embodiment Collaboration. 2023. Open X-Embodiment: Robotic learning datasets and RT-X models. arXiv preprint arXiv:2310.08864 on PithPith's reviewarXiv
  18. [38] OpenAI. 2024. GPT-4o system card. arXiv preprint arXiv:2410.21276 on PithPith's reviewarXiv
  19. [39] OpenAI. 2025. GPT-5 system card. OpenAI on Pith
  20. [40] OpenAI. 2025. OpenAI o3 and o4-mini system card. OpenAI on Pith
  21. [41] OpenAI. 2026. GPT-6 Astra system card. OpenAI on Pith
  22. [42] Haosong Peng, Hao Li, Jiaqi Chen, Yuhao Pan, Runmao Yao, Yalun Dai, Fushuo Huo, Fangzhou Hong, Zhaoxi Chen, Haozhao Wang, Dingwen Zhang, Ziwei Liu, and Wenchao Xu. 2026. SpatialBench: Is your spatial foundation model an all-round player?arXiv preprint arXiv:2605.27367 on PithPith's reviewarXiv
  23. [43] Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y. Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin Le... on PithPith's reviewarXiv
  24. [44] Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister. 2024. LangSplat: 3D language gaussian splatting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition on Pith
  25. [45] Santhosh K. Ramakrishnan, Aaron Gokaslan, Erik Wijmans, and 1 others. 2021. Habitat-Matterport 3D Dataset (HM3D): 1000 large-scale 3D environments for embodied AI. arXiv preprint arXiv:2109.08238 on PithPith's reviewarXiv
  26. [46] Javier Romero, Dimitrios Tzionas, and Michael J. Black. 2017. Embodied hands: Modeling and capturing hands and bodies together.ACM T ransactions on Graphics, 36(6) on Pith
  27. [47] Ropedia. 2026. Xperience-10m: A large-scale egocentric multimodal dataset with structured 3d/4d annotations. Dataset on Pith
  28. [48] Johannes L. Schönberger and Jan-Michael Frahm. 2016. Structure-from-motion revisited. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition on Pith
  29. [49] SenseNova, Joint Research Team, Zhongang Cai, Ziang Cao, Tailai Chen, and 1 others. 2026. Has GPT-6-Astra achieved embodied intelligence?arXiv preprint arXiv:2610.22471 on PithPith's reviewarXiv
  30. [50] Satvik Sharma, Hengyuan Hu, Mihika Kamat, Dorsa Sadigh, and Jeannette Bohg. 2026. RoboARCade: Bench- marking agentic robot control. arXiv preprint arXiv:2611.08209 on PithPith's reviewarXiv
  31. [51] Andrew Szot, Alex Clegg, Eric Undersander, Erik Wijmans, Yili Zhao, John Turner, Noah Maestre, Mustafa Mukadam, Devendra Chaplot, Oleksandr Maksymets, Aaron Gokaslan, Vladimir Vondrus, Sameer Dharur, Franziska Meier, Wojciech Galuba, Angel Chang, Zsolt Kira, Vladlen Koltun, Ji... on PithPith's reviewarXiv
  32. [52] Marcel Torne, Anthony Simeonov, Zechu Li, April Chan, Tao Chen, Abhishek Gupta, and Pulkit Agrawal. 2024. Reconciling reality through simulation: A real-to-sim-to-real approach for robust manipulation. InRobotics: Science and Systems on Pith
  33. [53] Vadim Tschernezki, Ahmad Darkhalil, Zhifan Zhu, David Fouhey, Iro Laina, Diane Larlus, Dima Damen, and Andrea Vedaldi. 2023. EPIC Fields: Marrying 3D geometry and video understanding. InAdvances in Neural Information Processing Systems on Pith
  34. [54] Shinji Umeyama. 1991. Least-squares estimation of transformation parameters between two point patterns.IEEE T ransactions on Pattern Analysis and Machine Intelligence, 13(4):376–380 on Pith
  35. [55] Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. 2025. VGGT: Visual geometry grounded transformer. arXiv preprint arXiv:2503.11651 on PithPith's reviewarXiv
  36. [56] Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A. Efros, and Angjoo Kanazawa. 2025. Continuous 3D perception model with persistent state. arXiv preprint arXiv:2501.12387 on PithPith's reviewarXiv
  37. [57] Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. 2023. DUSt3R: Geometric 3D vision made easy. arXiv preprint arXiv:2312.14132 on PithPith's reviewarXiv
  38. [58] Yifan Wang, Jianjun Zhou, Haoyi Zhu, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Jiangmiao Pang, Chunhua Shen, and Tong He. 2025. 𝜋3: Permutation-equivariant visual geometry learning. arXiv preprint arXiv:2507.13347 on PithPith's reviewarXiv
  39. [59] Sihan Yang, Runsen Xu, Yiman Xie, Sizhe Yang, Mo Li, Jingli Lin, Chenming Zhu, Xiaochen Chen, Haodong Duan, Xiangyu Yue, Dahua Lin, Tai Wang, and Jiangmiao Pang. 2026. MMSI-Bench: A benchmark for multi-image spatial intelligence. InInternational Conference on Learning Representations on Pith
  40. [60] Yung-Hsu Yang, Luigi Piccinelli, Samuel Rota Bulò, Sunghwan Hong, Denis Rozumny, Johannes Schönberger, Zuria Bauer, Hermann Blum, Peter Kontschieder, and Marc Pollefeys. 2026. Map-Det3D: Metric feed-forward 3D reconstruction prior for multi-view 3D object detection from stream... on PithPith's reviewarXiv
  41. [61] Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai. 2023. ScanNet++: A high-fidelity dataset of 3D indoor scenes. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 12–22 on Pith
  42. [62] Brent Yi, Vickie Ye, Maya Zheng, Yunqi Li, Lea Müller, Georgios Pavlakos, Yi Ma, Jitendra Malik, and Angjoo Kanazawa. 2025. Estimating body and hand motion in an ego-sensed world. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition on Pith
  43. [63] Wenbo Zhang, Kaixuan Wang, Yutao Ouyang, Xiaoyu Huang, Liyang Li, Kailun Su, Weiyang Jin, Wenhao Chai, Haotian Liang, Zhiyang Dou, Yue Chen, and Tianxing Chen. 2026. An unexpected robot policy: Early evaluations of GPT-6 Astra on RoboDojo and beyond.arXiv preprint arXiv:2609.24170 on PithPith's reviewarXiv
  44. [64] Chenming Zhu, Tai Wang, Wenwei Zhang, Jiangmiao Pang, and Xihui Liu. 2024. LLaVA-3D: A simple yet effective pathway to empowering LMMs with 3D-awareness. arXiv preprint arXiv:2409.18125 on PithPith's reviewarXiv
  45. [65] Jinguo Zhu, Weiyun Wang, Zhe Chen, and 1 others. 2025. InternVL3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479 on PithPith's reviewarXiv
  46. [66] Dong Zhuo, Wenzhao Zheng, Jiahe Guo, Yuqi Wu, Jie Zhou, and Jiwen Lu. 2025. Streaming 4D visual geometry transformer. arXiv preprint arXiv:2507.11539 on PithPith's reviewarXiv
  47. [67] Xueyan Zou, Yuchen Song, Ri-Zhao Qiu, Xuanbin Peng, Jianglong Ye, Sifei Liu, and Xiaolong Wang. 2025. M3: 3D-spatial multimodal memory. InInternational Conference on Learning Representations. Appendix Contents A Contributors 21 B Processing and Annotation Details 21 B.1 Synchr... on Pith
  48. [2022] Ego4D: Around the world in 3,000 hours of egocentric video. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18995–19012 on Pith
  49. [2023] Code as policies: Language model programs for embodied control. InIEEE International Conference on Robotics and Automation on Pith

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment