Pith. sign in

REVIEW 5 cited by

Real2Code: Reconstruct Articulated Objects via Code Generation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.08474 v2 pith:TIBC4LK5 submitted 2024-06-12 cs.CV cs.AIcs.LG

Zhao Mandi , Yijia Weng , Dominik Bauer , Shuran Song This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords objectsarticulatedmodelreal2codeapproachcodepartsfirst
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We present Real2Code, a novel approach to reconstructing articulated objects via code generation. Given visual observations of an object, we first reconstruct its part geometry using an image segmentation model and a shape completion model. We then represent the object parts with oriented bounding boxes, which are input to a fine-tuned large language model (LLM) to predict joint articulation as code. By leveraging pre-trained vision and language models, our approach scales elegantly with the number of articulated parts, and generalizes from synthetic training data to real world objects in unstructured environments. Experimental results demonstrate that Real2Code significantly outperforms previous state-of-the-art in reconstruction accuracy, and is the first approach to extrapolate beyond objects' structural complexity in the training set, and reconstructs objects with up to 10 articulated parts. When incorporated with a stereo reconstruction model, Real2Code also generalizes to real world objects from a handful of multi-view RGB images, without the need for depth or camera information.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Fail2Progress: Learning from Real-World Robot Failures with Stein Variational Inference

    cs.RO 2025-09 conditional novelty 7.0 of 10

    Fail2Progress generates failure-targeted simulation data via Stein variational inference and fine-tunes skill effect models, improving long-horizon manipulation success rates and generalizing to unseen object counts a...

  2. SplArt: Articulation Estimation and Part-Level Reconstruction with 3D Gaussian Splatting

    cs.GR 2025-06 conditional novelty 6.0 of 10

    SplArt estimates revolute or prismatic joint parameters and part-level 3D Gaussian geometry from two sets of posed RGB images using self-supervised multi-stage optimization.

  3. A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards

    cs.RO 2025-02 conditional novelty 6.0 of 10

    IKER uses VLM-generated keypoint rewards to train manipulation policies in simulation that transfer to a real robot, enabling multi-step tasks and replanning.

  4. Articulate That Object Part (ATOP): 3D Part Articulation via Text and Motion Personalization

    cs.CV 2025-02 conditional novelty 6.0 of 10

    ATOP personalizes a pre-trained multi-view diffusion model with a few reference videos to generate part motion from text and masks, then lifts that motion to a 3D articulation axis via score distillation.

  5. SPLATART: Articulated Gaussian Splatting with Estimated Object Structure

    cs.RO 2025-06 conditional novelty 5.0 of 10

    SPLATART builds Gaussian splat renderers that can redraw articulated objects in new configurations, using sparse part segmentations and estimating joint structure for deep kinematic chains.

Pith tools