REVIEW 10 cited by
DeformGS: Scene Flow in Highly Deformable Scenes for Deformable Object Manipulation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
DeformGS: Scene Flow in Highly Deformable Scenes for Deformable Object Manipulation
read the original abstract
Teaching robots to fold, drape, or reposition deformable objects such as cloth will unlock a variety of automation applications. While remarkable progress has been made for rigid object manipulation, manipulating deformable objects poses unique challenges, including frequent occlusions, infinite-dimensional state spaces and complex dynamics. Just as object pose estimation and tracking have aided robots for rigid manipulation, dense 3D tracking (scene flow) of highly deformable objects will enable new applications in robotics while aiding existing approaches, such as imitation learning or creating digital twins with real2sim transfer. We propose DeformGS, an approach to recover scene flow in highly deformable scenes, using simultaneous video captures of a dynamic scene from multiple cameras. DeformGS builds on recent advances in Gaussian splatting, a method that learns the properties of a large number of Gaussians for state-of-the-art and fast novel-view synthesis. DeformGS learns a deformation function to project a set of Gaussians with canonical properties into world space. The deformation function uses a neural-voxel encoding and a multilayer perceptron (MLP) to infer Gaussian position, rotation, and a shadow scalar. We enforce physics-inspired regularization terms based on conservation of momentum and isometry, which leads to trajectories with smaller trajectory errors. We also leverage existing foundation models SAM and XMEM to produce noisy masks, and learn a per-Gaussian mask for better physics-inspired regularization. DeformGS achieves high-quality 3D tracking on highly deformable scenes with shadows and occlusions. In experiments, DeformGS improves 3D tracking by an average of 55.8% compared to the state-of-the-art. With sufficient texture, DeformGS achieves a median tracking error of 3.3 mm on a cloth of 1.5 x 1.5 m in area. Website: https://deformgs.github.io
Forward citations
Cited by 10 Pith papers
-
ASTRA: Asynchronous Spatio-Temporal Reconstruction via Trajectory Alignment
ASTRA jointly estimates camera time offsets and dynamic Gaussian geometry by aligning projected 3D motion with observed 2D trajectory tracks, improving robustness to large asynchrony.
-
Deform360: A Massive Multi-view Visuotactile Dataset for Deformable World Models
Deform360 supplies 215+ hours of synchronized multi-view video and tactile data plus markerless 3D tracks, revealing that 3D particle models win in low data while 2D video models generalize better at scale.
-
DLGStream: Dynamic Language-embedded Guassian Splatting for Open-vocabulary Enabled Free-viewpoint Video Streaming
DLGStream introduces dual-opacity dynamic language Gaussians and an interpolation deformation field to enable language-embedded FVV streaming at 43 KB per frame with improved open-vocabulary segmentation and reconstru...
-
Realizing Immersive Volumetric Video: A Multimodal Framework for 6-DoF VR Engagement
The paper presents a multimodal framework, dataset, and reconstruction pipeline to create immersive volumetric videos supporting large 6-DoF audiovisual interaction from real multi-view captures.
-
GEAR: GEometry-motion Alternating Refinement for Articulated Object Modeling with Gaussian Splatting
GEAR is an EM-style alternating optimization framework that jointly models geometry and motion in Gaussian Splatting to improve reconstruction of complex articulated objects.
-
PD-4DGS:Progressive Decomposition of 4D Gaussian Splatting for Bandwidth-Adaptive Dynamic Scene Streaming
PD-4DGS decomposes 4DGS into static scaffold, global deformation, and local refinement layers using hierarchical decomposition and custom losses, achieving over 60% bitstream reduction and reducing first-frame latency...
-
WARPED: Wrist-Aligned Rendering for Robot Policy Learning from Egocentric Human Demonstrations
WARPED synthesizes realistic wrist-view observations from monocular egocentric human videos via foundation models, hand-object tracking, retargeting, and Gaussian Splatting to train visuomotor policies that match tele...
-
LIVE-GS: LLM Powers Interactive VR Experience with Physics-Aware Gaussian Splatting
LIVE-GS uses an LLM to predict physical parameters from static Gaussian assets in 10 seconds for physics-aware VR interactions, validated by interviews, baseline comparisons, and user studies.
-
From Concept to Capability: Evaluating 3D Gaussian Splatting for Synthetic Scene Editing in Autonomous Driving
A framework is introduced to systematically assess the reconstruction fidelity of 3D Gaussian Splatting for vehicles and pedestrians in autonomous driving scenes from novel lateral and longitudinal viewpoints.
-
Advances in 4D Representation: Geometry, Motion, and Interaction
A representation-centric survey of 4D generation and reconstruction, organized by geometry, motion, and interaction, with qualitative trade-off comparisons across seven representation families.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.