REVIEW 3 major objections 7 minor 37 references
PIG: Physically-based Multi-Material Interaction with 3D Gaussians
T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read PIG couples object-level segmentation with MLS-MPM simulation so a static 3D Gaussian scene can be re-simulated as interacting soft, rigid, and sand materials with high rendering fidelity.
desk verdict Good segmentation pipeline and a practical artifact fix, but the multi-material claim is not demonstrated: all simulated objects use the same Young's modulus and Poisson's ratio. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism has two halves. First, inverse projection: a surface depth map is rendered from the Gaussians using a transmittance threshold, DEVA provides multi-view masks with consistent object IDs, and each Gaussian is assigned the ID of the pixel whose ray is within a relative depth threshold; the most frequent ID across views becomes the object label. Second, adaptive eigen-clamping: the deformation gradient's covariance update is factored into rotation and scaling via eigenvalue decomposition, the scale components are clamped between tau_min and tau_max, and the small residual corrections are blended back with constraint scales lambda_R and lambda_S to preserve geometric fidelity.
What would settle it
Reconstruct a scene that includes a transparent object such as a glass cup, run the inverse projection, and check whether the glass Gaussians share one consistent ID. If their IDs scatter across the background and occluded objects, the interaction simulation will start from the wrong primitives, breaking the multi-material interaction claim.
Extended reading notes
Core claim
PIG is presented as the first pipeline to combine 3D object-level segmentation with MLS-MPM simulation directly on 3D Gaussian primitives. Segmented objects receive independent physical parameters, enabling interactions such as a rigid apple striking a soft sheep or sand pouring around a plush toy, while the rendering side clamps the eigenvalues of the Gaussian covariance to keep deformed primitives from becoming needles or points. The authors claim this produces state-of-the-art performance in 3D object-level segmentation and artifact removal, and that it is the only method among recent competitors to simultaneously support diverse physical properties, object-level segmentation, and artifact suppression.
Load-bearing premise
The segmentation presumes every object has an unambiguous visible surface depth, so transparent and semi-transparent objects cannot be assigned correct IDs.
Editorial extensions
If this is right
- A multi-view capture of a static tabletop can be turned into a video where objects are knocked over, squashed, or scattered as distinct materials with different stiffness and response.
- Because segmentation is a fast projection step rather than per-Gaussian feature distillation, the pipeline requires no extra training for a new scene beyond the standard 3DGS reconstruction.
- Adaptive eigen-clamping keeps large-deformation interactions visually coherent, removing the needle-like and point-like artifacts that appear when covariance is driven directly by deformation gradients.
- The same object-level IDs can be reused for scene editing tasks beyond simulation, such as select-and-remove or select-and-retexture on the reconstructed Gaussians.
- The pipeline offers a concrete three-way combination—diverse materials, object-level segmentation, and artifact removal—that defines a baseline for future physically realistic scene generation.
Reading between the lines
- The depth-based ID assignment could serve as a cheap proxy for tracking Gaussians across a dynamic sequence, since DEVA mask IDs stay consistent across views and frames.
- The fixed global clamp thresholds and constraint scales could be made per-object or tied to material stiffness, which might extend the method to extreme deformations such as tearing or fracturing.
- The transparent-object failure is a natural next boundary: a transmittance-aware or multi-view-depth segmentation could extend the pipeline to glass and liquids, which are currently out of scope.
- The paper's comparative setup implies a benchmark where methods are scored on all three axes simultaneously (material diversity, object-level selection, artifact-free rendering), which future work could adopt directly.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes PIG, a pipeline that segments objects in a 3D Gaussian Splatting scene by inverse-projecting DEVA 2D masks through a rendered surface depth map, then simulates the segmented objects with MLS-MPM and applies an 'adaptive eigen-clamping' correction to the Gaussian rotations and scales to suppress rendering artifacts. The paper reports quantitative segmentation results (mIoU/mBIoU) on the LERF-Mask and 3D-OVS datasets, qualitative comparisons against PhysGaussian, PhysDreamer, and Feature Splatting for multi-material interaction, and a qualitative ablation of the artifact-removal step.
Significance. If the central claims are properly supported, the pipeline would be a practically useful integration of object-level segmentation with particle-based simulation inside 3DGS, with potential value for scene editing and digital-twin applications. The segmentation component has quantitative evidence on two benchmarks, and the idea of clamping eigenvalue components of the covariance to avoid needle-like artifacts is plausible and addresses a known failure mode of deformation-gradient-driven covariance updates. However, the current manuscript does not demonstrate the claimed multi-material physical-property assignment, and the artifact-removal and visual-quality claims rest on qualitative figures. The significance is therefore conditional: the pipeline is promising, but the central simulation and rendering claims need stronger evidence.
major comments (3)
- [Section 4.2.2 and Section 3.1.2] The central claim of multi-material interaction is not supported by the experimental setup. Section 4.2.2 states 'we set the Young's modulus to 10^7, the Poisson's ratio to 0.2' for all subsequent experiments, and the only per-object variation listed is initial velocity. Section 3.1.2 presents a generic elastic energy density for MLS-MPM and does not introduce a plasticity model for sand (e.g., Drucker-Prager) or a rigid-body constraint for objects labeled rigid in Figures 1 and 5. Consequently, the scenes described as 'soft', 'rigid', and 'sand' are simulated as a single elastic material with different initial conditions. This directly contradicts the abstract's claim that the method 'assign[s] unique physical properties to correspondingly segmented objects' and the assertion in Table 1 that the method supports 'Diverse Physical Properties'. The paper must either specify distinct constitutive models and per-object parameters or substantially revise the multi-material claims.
- [Section 4.3 and Eq. (6)] The claim that adaptive eigen-clamping 'eliminates artifacts and achieve[s] geometric fidelity and visual consistency' is not quantitatively supported. The ablation in Figure 6 is purely qualitative, and the comparison in Figure 5 is also qualitative; no rendering metrics such as PSNR, SSIM, or LPIPS are reported for the ablation or for the comparison against baselines. The constraint scales lambda_R = 1.2 and lambda_S = 0.8 are introduced in Eq. (6) as fine-tuned parameters, but no sensitivity analysis or cross-validation is provided, and the clamping thresholds tau_min and tau_max are not specified in the main text. Without quantitative evidence, the claimed superiority in 'artifact removal' and 'visual quality' (abstract and contribution list) remains unverified. Please add quantitative rendering metrics and a sensitivity analysis for lambda_R, lambda_S, tau_min, and tau_max.
- [Section 3.2.2 and Limitations] The inverse-projection segmentation assumes that a single sharp depth value can be found for every pixel, as the transmittance threshold tau_T identifies the first surface along the ray. The Limitations paragraph concedes that transparent and semi-transparent objects produce unreliable depth because their accurate depth is difficult to determine. Since this depth-based mapping is the foundation of the entire pipeline, the current claims should be explicitly restricted to opaque objects, or the evaluation should include transparent/semi-transparent objects to characterize the failure mode. As written, the paper overstates the scope of the segmentation method relative to its own acknowledged limitation.
minor comments (7)
- [Eq. (1)] In the definition of T_i, the product is over j from 1 to i-1, but the displayed factor uses alpha_i; it should be alpha_j.
- [Tables 2 and 3] The abbreviation mBIoU is not defined in the text; please define it (presumably mean boundary IoU) at first use.
- [Table 3] The scene name 'figuriens' appears to be a typo for 'figurines'.
- [Section 4.2.4 and Figure 5] The method is called 'Feature Splatting' in the baseline list but 'Featuresplatting' in the caption and comparison text; please use a consistent name.
- [Introduction and Reference [33]] The introduction attributes 'Junhao Cai et al.' to reference [33], but the reference list gives 'Licheng Zhong, Hong-Xing Yu, Jiajun Wu, and Yunzhu Li'; please correct the in-text citation.
- [Section 3.3.4] The thresholds tau_min and tau_max are introduced but their values are never given in the main text; please state them or include the supplementary material with concrete values.
- [Availability] No code or supplementary document is provided with the arXiv version, which makes the hyperparameter optimization and the eigen-decomposition details mentioned in the text unverifiable; please make these available.
Circularity Check
No significant circularity: the pipeline is an integration of independent external methods, and no prediction reduces to its inputs by construction.
full rationale
The claimed derivation chain is assembled from independently published components: 3D Gaussian Splatting (Kerbl et al.), DEVA video segmentation (Cheng et al.), and MLS-MPM (Hu et al.). The segmentation step maps DEVA 2D masks into 3D by inverse projection using the rendered surface depth map (Eqs. 4-5); the resulting 3D ID assignment is a lifting of an external 2D prior, and the evaluation compares re-rendered masks against manually annotated ground truth, not against the input masks. The simulation step uses the standard MLS-MPM update equations (Eqs. 2-3) with deformation gradients; no material parameter is fitted to a target outcome and then reported as a prediction. The adaptive eigen-clamping in Eq. 6 introduces hyperparameters lambda_R and lambda_S that are tuned on the same scenes used for the qualitative ablation, which is a mild evaluation caveat, but it is not a prediction forced by construction and does not make the central claim self-referential. The paper contains no self-citations by the authors and imports no 'uniqueness theorem' from prior work. The acknowledged limitation about transparent objects (Limitations section) affects robustness and scope, not circularity. The observation that all experimental objects share the same Young's modulus and Poisson's ratio (Sec. 4.2.2) undermines the strength of the multi-material claim, but this is an internal-support gap between claims and experiments, not a circular derivation.
Assumptions & free parameters
free parameters (7)
- tau_T =
0.5
- tau_d =
0.03
- lambda_R =
1.2
- lambda_S =
0.8
- Young's modulus =
1e7
- Poisson's ratio =
0.2
- Initial velocities of objects =
Miku (2,0,0), India Cat (1,-1,0), others (0,0,0)
assumptions (5)
- domain assumption Reconstructed 3D Gaussians faithfully represent scene geometry and appearance for depth and color rendering.
- domain assumption DEVA provides consistent object IDs across all multi-view images.
- domain assumption A transmittance threshold marks the first surface along each ray, so each Gaussian has a unique depth comparison.
- domain assumption Segmented objects behave as continuum materials with uniform Young's modulus and Poisson's ratio under MLS-MPM.
- ad hoc to paper Clamping Gaussian scale eigenvalues and blending corrected rotations and scales with the originals preserves visual fidelity.
Cite this review
Pith. "Pith review of PIG: Physically-based Multi-Material Interaction with 3D Gaussians." pith.science (2026). https://pith.science/paper/ORXNEK7S
@misc{pith2026250607657,
author = {Pith},
title = {Pith review of: PIG: Physically-based Multi-Material Interaction with 3D Gaussians},
year = {2026},
howpublished = {\url{https://pith.science/paper/ORXNEK7S}},
note = {Machine review of arXiv:2506.07657}
}
read the original abstract
3D Gaussian Splatting has achieved remarkable success in reconstructing both static and dynamic 3D scenes. However, in a scene represented by 3D Gaussian primitives, interactions between objects suffer from inaccurate 3D segmentation, imprecise deformation among different materials, and severe rendering artifacts. To address these challenges, we introduce PIG: Physically-Based Multi-Material Interaction with 3D Gaussians, a novel approach that combines 3D object segmentation with the simulation of interacting objects in high precision. Firstly, our method facilitates fast and accurate mapping from 2D pixels to 3D Gaussians, enabling precise 3D object-level segmentation. Secondly, we assign unique physical properties to correspondingly segmented objects within the scene for multi-material coupled interactions. Finally, we have successfully embedded constraint scales into deformation gradients, specifically clamping the scaling and rotation properties of the Gaussian primitives to eliminate artifacts and achieve geometric fidelity and visual consistency. Experimental results demonstrate that our method not only outperforms the state-of-the-art (SOTA) in terms of visual quality, but also opens up new directions and pipelines for the field of physically realistic scene generation.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Jiazhong Cen, Jiemin Fang, Chen Yang, Lingxi Xie, Xiaopeng Zhang, Wei Shen, and Qi Tian. 2023. Segment Any 3D Gaussians.arXiv preprint arXiv:2312.00860 (2023)
arXiv 2023
-
[2]
Ho Kei Cheng, Seoung Wug Oh, Brian Price, Alexander Schwing, and Joon-Young Lee. 2023. Tracking Anything with Decoupled Video Segmentation. InICCV
work page 2023
-
[3]
Gilles Daviet and Florence Bertails-Descoubes. 2016. A semi-implicit material point method for the continuum simulation of granular materials.ACM Transac- tions on Graphics (TOG)35, 4 (2016), 1–13
work page 2016
-
[4]
Yu Fang, Yuanming Hu, Shi-Min Hu, and Chenfanfu Jiang. 2018. A temporally adaptive material point method with regional time stepping. InComputer graphics forum, Vol. 37. Wiley Online Library, 195–204
work page 2018
-
[5]
Jun Guo, Xiaojian Ma, Yue Fan, Huaping Liu, and Qing Li. 2024. Semantic Gaussians: Open-Vocabulary Scene Understanding with 3D Gaussian Splatting. arXiv:2403.15624 [cs.CV]
arXiv 2024
-
[6]
Yuanming Hu, Yu Fang, Ziheng Ge, Ziyin Qu, Yixin Zhu, Andre Pradhana, and Chenfanfu Jiang. 2018. A moving least squares material point method with displacement discontinuity and two-way rigid body coupling.ACM Transactions on Graphics (TOG)37, 4 (2018), 1–14
2018
-
[7]
Chenfanfu Jiang, Theodore Gast, and Joseph Teran. 2017. Anisotropic elastoplas- ticity for cloth, knit and hair frictional contact.ACM Transactions on Graphics (TOG)36, 4 (2017), 1–14
2017
-
[8]
Chenfanfu Jiang, Craig Schroeder, Andrew Selle, Joseph Teran, and Alexey Stom- akhin. 2015. The affine particle-in-cell method.ACM Transactions on Graphics (TOG)34, 4 (2015), 1–10
2015
Show all 37 references
-
[9]
Chenfanfu Jiang, Craig Schroeder, Joseph Teran, Alexey Stomakhin, and Andrew Selle. 2016. The material point method for simulating continuum materials. In ACM SIGGRAPH 2016 Courses(Anaheim, California)(SIGGRAPH ’16). Association for Computing Machinery, New York, NY, USA, Arti...
2016
-
[10]
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis
-
[11]
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al
-
[12]
Agelos Kratimenos, Jiahui Lei, and Kostas Daniilidis. 2024. Dynmf: Neural motion factorization for real-time dynamic view synthesis with 3d gaussian splatting. In European Conference on Computer Vision. Springer, 252–269
2024
-
[13]
InProceedings of the IEEE/CVF International Conference on Computer Vision
Segment anything. InProceedings of the IEEE/CVF International Conference on Computer Vision. 4015–4026
-
[14]
Kunhao Liu, Fangneng Zhan, Jiahui Zhang, Muyu Xu, Yingchen Yu, Abdulmo- taleb El Saddik, Christian Theobalt, Eric Xing, and Shijian Lu. 2023. Weakly Supervised 3D Open-vocabulary Segmentation.arXiv preprint arXiv:2305.14093 (2023)
2023 arXiv
-
[15]
Yiqing Liang, Numair Khan, Zhengqin Li, Thu Nguyen-Phuoc, Douglas Lanman, James Tompkin, and Lei Xiao. 2023. Gaufre: Gaussian deformation fields for real-time dynamic novel view synthesis.arXiv preprint arXiv:2312.11458(2023)
2023 arXiv
-
[16]
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. 2021. Nerf: Representing scenes as neural radiance fields for view synthesis.Commun. ACM65, 1 (2021), 99–106
2021
-
[17]
Tiantian Liu, Adam W Bargteil, James F O’Brien, and Ladislav Kavan. 2013. Fast simulation of mass-spring systems.ACM Transactions on Graphics (TOG)32, 6 (2013), 1–7
2013
-
[18]
Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister
-
[19]
Yicong Peng, Yichao Yan, Shengqi Liu, Yuhao Cheng, Shanyan Guan, Bowen Pan, Guangtao Zhai, and Xiaokang Yang. 2022. Cagenerf: Cage-based neural radiance field for generalized 3d deformation and animation.Advances in Neural Information Processing Systems35 (2022), 31402–31415
2022
-
[20]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. InInternational conference on machine learnin...
2021
-
[21]
Daniel Ram, Theodore Gast, Chenfanfu Jiang, Craig Schroeder, Alexey Stom- akhin, Joseph Teran, and Pirouz Kavehpour. 2015. A material point method for viscoelastic fluids, foams and sponges. InProceedings of the 14th ACM SIG- GRAPH/Eurographics Symposium on Computer Animation. 157–163
2015
-
[22]
Ri-Zhao Qiu, Ge Yang, Weijia Zeng, and Xiaolong Wang. 2024. Feature Splatting: Language-Driven Physics-Based Scene Synthesis and Editing.arXiv preprint arXiv:2404.01223(2024)
2024 arXiv
-
[23]
Alexey Stomakhin, Craig Schroeder, Chenfanfu Jiang, Lawrence Chai, Joseph Teran, and Andrew Selle. 2014. Augmented MPM for phase-change and varied materials.ACM Transactions on Graphics (TOG)33, 4 (2014), 1–11
2014
-
[24]
Deborah Sulsky, Shi-Jian Zhou, and Howard L Schreyer. 1995. Application of a particle-in-cell method to solid mechanics.Computer physics communications87, 1-2 (1995), 236–252
1995
-
[25]
Alexey Stomakhin, Craig Schroeder, Lawrence Chai, Joseph Teran, and Andrew Selle. 2013. A material point method for snow simulation.ACM Transactions on Graphics (TOG)32, 4 (2013), 1–10
2013
-
[26]
Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 2024. 4d gaussian splatting for real-time dynamic scene rendering. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 20310–20320
2024
-
[27]
Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, and Chen- fanfu Jiang. 2024. Physgaussian: Physics-integrated 3d gaussians for generative dynamics. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4389–4398
2024
-
[28]
Andre Pradhana Tampubolon, Theodore Gast, Gergely Klár, Chuyuan Fu, Joseph Teran, Chenfanfu Jiang, and Ken Museth. 2017. Multi-species simulation of porous sand and water mixtures.ACM Transactions on Graphics (TOG)36, 4 (2017), 1–11
2017
-
[29]
Mingqiao Ye, Martin Danelljan, Fisher Yu, and Lei Ke. 2024. Gaussian Grouping: Segment and Edit Anything in 3D Scenes. InECCV
2024
-
[30]
Yu-Jie Yuan, Yang-Tian Sun, Yu-Kun Lai, Yuewen Ma, Rongfei Jia, and Lin Gao
-
[31]
Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. 2024. Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 20331–20341
2024
-
[32]
Tianyuan Zhang, Hong-Xing Yu, Rundi Wu, Brandon Y Feng, Changxi Zheng, Noah Snavely, Jiajun Wu, and William T Freeman. 2024. Physdreamer: Physics- based interaction with 3d objects via video generation. InEuropean Conference on Computer Vision. Springer, 388–406
2024
-
[33]
Licheng Zhong, Hong-Xing Yu, Jiajun Wu, and Yunzhu Li. 2025. Reconstruction and simulation of elastic objects with spring-mass 3d gaussians. InEuropean Conference on Computer Vision. Springer, 407–423
2025
-
[35]
Yonghao Yue, Breannan Smith, Christopher Batty, Changxi Zheng, and Eitan Grinspun. 2015. Continuum foam: A material point method for shear-dependent flows.ACM Transactions on Graphics (TOG)34, 5 (2015), 1–20
2015
-
[2022]
InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Nerf-editing: geometry editing of neural radiance fields. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 18353–18364
-
[2023]
Graph.42, 4 (2023), 139–1
3D Gaussian Splatting for Real-Time Radiance Field Rendering.ACM Trans. Graph.42, 4 (2023), 139–1
2023
-
[2024]
InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Langsplat: 3d language gaussian splatting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 20051–20060
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.