REVIEW 4 cited by
Differentiable Robot Rendering
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Vision foundation models trained on massive amounts of visual data have shown unprecedented reasoning and planning skills in open-world settings. A key challenge in applying them to robotic tasks is the modality gap between visual data and action data. We introduce differentiable robot rendering, a method allowing the visual appearance of a robot body to be directly differentiable with respect to its control parameters. Our model integrates a kinematics-aware deformable model and Gaussians Splatting and is compatible with any robot form factors and degrees of freedom. We demonstrate its capability and usage in applications including reconstruction of robot poses from images and controlling robots through vision language models. Quantitative and qualitative results show that our differentiable rendering model provides effective gradients for robotic control directly from pixels, setting the foundation for the future applications of vision foundation models in robotics.
Forward citations
Cited by 4 Pith papers
-
RoboLight: A Dataset with Linearly Composable Illumination for Robotic Manipulation
A dataset that records identical robot manipulation tasks under 14 controlled lighting conditions and uses HDR linearity to synthesize 196,000 additional lighting-varied episodes.
-
ScrewSplat: An End-to-End Method for Articulated Object Recognition
A method that recovers the 3D shape and the rotation or sliding axis of each movable part of an object from RGB video alone, by jointly optimizing randomly initialized screw axes with Gaussian Splatting.
-
ArtGS:3D Gaussian Splatting for Interactive Visual-Physical Modeling and Manipulation of Articulated Objects
ArtGS combines multi-view 3D reconstruction, language-model joint initialization, and closed-loop optimization to improve articulated object manipulation.
-
Morpheus: A Neural-driven Animatronic Face with Hybrid Actuation and Diverse Emotion Control
A hybrid-actuated robot face, controlled by a learned inverse model, that turns emotional speech into distinct facial expressions such as happy, angry, disgust, and fear.
Discussion (0). Sign in to comment.