Pith. sign in

REVIEW 4 major objections 7 minor 42 references

SiPhy claims that a single RGB image plus its estimated depth can predict an object's mass, density, and Young's modulus as accurately as—or better than—multi-view 3D reconstruction systems.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 05:00 UTC pith:2M22XYAS

load-bearing objection A real advance in single-image physical reasoning, but the mass numbers rest on an unvalidated depth-scale assumption and an under-specified HAT module; major revision needed before the claims hold. the 4 major comments →

arxiv 2607.22355 v1 pith:2M22XYAS submitted 2026-07-24 cs.CV cs.AI

SiPhy: Single-Image Physical Property Reasoning

Hoang Le , Joonwoo Kwon , Elkhan Ismayilzada , Yufei Zhang , Zijun Cui This is my paper
classification cs.CV cs.AI
keywords physical property reasoning from imagesmass estimationmaterial segmentationvision-language modelCLIPpseudo-voxel samplingsingle-image depthYoung's modulus prediction
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

SiPhy is a framework for inferring physical properties—mass, density, stiffness, and material—from one photograph. The paper argues that the advantages of multi-view reconstruction (structured geometry, consistent material reasoning, physics-aware aggregation) can be approximated from a single view by building a coarse pseudo-voxel grid whose spacing adapts to estimated depth, and by grounding visual patches to material candidates proposed by a vision-language model. On object benchmarks, it reports state-of-the-art single-image mass prediction, surpassing multi-view reconstruction baselines (up to 93% improvement in MnRE vs. PUGS), and lower errors in density and Young's modulus. A sympathetic reading: the paper aims to show that expensive NeRF/Gaussian pipelines are not necessary for physical property estimation, opening the door to single-view annotation at scale.

Core claim

The central claim is that a depth-adaptive 2D sampling strategy—spacing s = d*sqrt(fx*fy)/z—can emulate voxelization from a single RGB-D view, and that CLIP features aligned with VLM-proposed material candidates yield physically consistent material probabilities. Aggregating these probabilities over pseudo-voxels delivers object-level mass, while a part-based contrastive alignment enforces within-part material coherence and a heaviness-aware thickness refinement corrects volume errors on dense objects. The paper reports state-of-the-art single-image results on ABO-500, MVImgNet-100, and PhysXNet-100, with real hand-object interaction validation, suggesting the framework can serve as a data a

What carries the argument

The load-bearing object is the pseudo-voxel grid: the paper replaces explicit multi-view reconstruction with an adaptive 2D sampling grid whose pixel spacing is s = d*sqrt(fx*fy)/z, where d is the virtual cube side length, fx/fy are camera intrinsics, and z is the estimated object depth. This turns a single view into a coarse set of 3D-aware voxel centers. Material probabilities per voxel come from a fine-tuned VLM proposing K candidate materials, CLIP visual-text similarity, a part-based contrastive aggregator (self-attention within SAM-derived masks plus supervised contrastive loss), and a heaviness-aware thickness refinement module that biases thickness toward plausible ranges based on he

Load-bearing premise

The single strongest assumption is that the depth value z in Eq. (1) is true metric distance aligned with the camera intrinsics; if it is off by a constant factor, every predicted volume and mass is systematically wrong by that factor squared.

What would settle it

Hold the RGB image fixed and multiply the depth map by a constant factor (e.g., 2x); feed the modified depth through SiPhy. If predicted mass changes by roughly 4x instead of staying the same, the metric-depth assumption is violated and the central volume/mass results are scale-ambiguous.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Single-view physical property prediction becomes practical, removing the need for multi-view capture and heavy optimization for mass, density, and stiffness estimation.
  • Hand-object interaction datasets lacking physical labels can be annotated automatically, as demonstrated on HO3D and ARCTIC.
  • Material priors from SiPhy improve downstream image-to-audio generation, correcting appearance-driven misjudgments such as a metal locker being classified as wood.
  • The depth-adaptive spacing formula offers a cheap voxelization prior that could be reused in other vision-language-physics pipelines.
  • Embodied AI agents could estimate object mass and stiffness from a single camera frame before grasping or manipulation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the depth scaling assumption holds, the paper implies that most of the 'information' in multi-view physical reasoning is reconstructable from a single image, suggesting the heavy computational cost of NeRF/Gaussian baselines may be unnecessary for this task.
  • A natural testable extension is coupling SiPhy with a metric depth sensor (e.g., stereo or RGB-D) instead of a monocular estimator; the paper's +/-10% sensitivity analysis does not cover scale-ambiguous depth, which is the actual practical risk.
  • The heaviness-aware thickness refinement is a discrete binary prior; extending it to continuous thickness regression or compositional thickness estimation could generalize to objects with non-uniform density.
  • The framework could be repurposed as an automatic annotation engine to scale up physical property datasets, which would in turn enable training of even stronger single-image estimators.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes SiPhy, a single-image pipeline for estimating object-level physical properties (mass) and pixel-level maps (density, Young's modulus). The method samples pseudo-voxel points using a depth-adaptive spacing formula, extracts CLIP features, aligns them with VLM-proposed material candidates via a part-based contrastive module, and refines thickness with a heaviness-aware module. Evaluations on ABO-500, MVImgNet-100, PhysXNet-100, and real hand-object datasets claim state-of-the-art or competitive performance against multi-view NeRF/3DGS baselines.

Significance. If the central claim holds—that a single RGB image plus monocular depth can match or beat multi-view NeRF/3DGS methods on mass, density, and stiffness—this would be a practically significant result for embodied AI, simulation, and annotation. The paper contributes a creative combination of monocular depth, CLIP, and VLM material reasoning, and it includes downstream applications. However, the validation has critical gaps: the depth scale assumption is unverified, the key HAT module is under-specified and partially circular, and a supporting proof is not provided. These issues prevent acceptance of the claimed SOTA result as stated.

major comments (4)
  1. [Sec. 3.2, Eq. (1)] The adaptive spacing formula assumes z is metric depth consistent with fx and fy. The paper cites Depth Anything but gives no scale-calibration or per-sequence alignment. Because the number of sampled voxels over a fixed projected area scales as (1/s)^2, and the per-voxel volume uses a fixed d and a thickness that is not rescaled with z, predicted mass scales approximately as z^2. Monocular depth scale errors of 1.5–3x would change mass by ~2–9x. Table 5 perturbs depth/intrinsics by only ±10%, which does not cover global scale ambiguity. Please report either per-dataset metric-depth validation, a global-depth-scale sweep, or use of a calibrated depth source. Without this, the mass results cannot be attributed to the proposed method.
  2. [Sec. 3.4, Table 4] The heaviness-aware thickness (HAT) module is described only qualitatively ('biasing the prediction toward thickness ranges that are physically plausible'), with no equation, threshold, or adjustment rule. The classification into heavy/light uses SiPhy's own initial mass predictions, and the final mass is then adjusted through thickness—an apparent circularity. Table 4 shows HAT improves heavy-object ADE (18.86 to 15.59) but degrades light-object ADE (2.07 to 3.57) and APE (0.57 to 1.29), yet no error bars or multiple seeds are given. Specify the exact algorithm, report seeded variance, and test sensitivity to the threshold (e.g., cross-validated) to rule out overfitting the test split.
  3. [Sec. 3.4] The paper states 'We found that the material prediction has weak correlation to mass (proof in Supplementary)' but provides no proof or supporting analysis in the main text, and the supplement is not part of the submitted PDF. Since this claim motivates HAT, it is load-bearing. Please include the supporting evidence (e.g., correlation coefficient, scatter plot) in the main paper or a clearly accessible appendix.
  4. [Tables 2–3 vs. Abstract/Introduction] The claim that SiPhy 'achieves state-of-the-art performance in mass prediction, material segmentation, density estimation, and Young's modulus' is contradicted by the reported numbers: on ABO-500, GaussianProperty achieves mIoU 0.29 vs. SiPhy 0.25, and on PhysXNet-100, PUGS achieves density MAE 1297 vs. SiPhy 1315. The paper should qualify the claim (e.g., 'state-of-the-art among single-view methods' or 'competitive with multi-view baselines on specific properties') and explicitly acknowledge cases where multi-view methods are superior.
minor comments (7)
  1. [Table 5] The sensitivity analysis only covers ±10% perturbations; this is insufficient to address global depth-scale ambiguity. Please state this limitation or extend the analysis.
  2. [Tables 1–4, 6–7] No error bars, standard deviations, or statistical significance tests are reported. Given the small test splits (100 objects), this is necessary to assess whether observed differences are meaningful.
  3. [Abstract] Typo: 'an VLM' should be 'a VLM'. Also, capitalization of 'SiPhy' is inconsistent in some paragraphs.
  4. [Sec. 3.2] Clarify when Depth Anything is used versus other depth sources. The paper says 'In scenarios characterized by significant background clutter,' but it does not state which benchmarks use which setting.
  5. [Table 4] The phrase 'Improvements are marked in purple' is unusual and unhelpful in a black-and-white printout. Use arrows or bold formatting instead.
  6. [Fig. 6 caption] The caption refers to 'Objects 5 and 6' but the figure panel does not show numbered objects. Please add panel labels or revise the caption.
  7. [Sec. 4.1] The text says 'We report results on both the official test split and the full dataset for all models,' but only test-split results are shown. Either provide the full-dataset table or remove the sentence.

Circularity Check

1 steps flagged

HAT uses SiPhy's own mass prediction as the heavy/light gate, so the reported mass improvement is self-conditioned rather than independent.

specific steps
  1. fitted input called prediction [Sec. 3.4, 'Heaviness-aware thickness'; Table 4]
    "We found that the material prediction has weak correlation to mass (proof in Supplementary)... Therefore, we propose a thickness estimation method that are more mass-aware, using GPT4 to refine the mass based on heaviness of the object. We first use SiPhy’s initial predictions to classify the object as heavy or light. This classification is then used to adjust thickness prediction, biasing the prediction toward thickness ranges that are physically plausible."

    The heavy/light gate is computed from SiPhy's own initial mass prediction, and the gate then changes thickness, which changes final mass. Thus the final mass prediction is a function of the initial mass prediction: the model conditions its output on its own output. The Table 4 'improvement' for heavy objects (ADE 18.86 to 15.59, MnRE 0.57 to 0.65) is therefore a self-recalibration, not independent evidence that thickness/volume errors were corrected from material or geometric cues. Light-object ADE degrades from 2.07 to 3.57, the signature of a mass-derived class prior rather than a principled thickness model.

full rationale

The main derivation chain before HAT is mostly self-contained: Eq. (1) is a standard metric-depth-to-pixel-spacing conversion; material likelihoods come from CLIP and a fine-tuned VLM; Eq. (6) is an expectation over material attributes. I found no load-bearing self-citations or imported uniqueness theorems; comparisons are against external baselines. The unvalidated scale of the monocular depth prior is a real correctness/robustness risk, not a circularity, and the ±10% sensitivity analysis does not address common metric-scale ambiguity. However, HAT injects the target quantity into its own refinement: initial mass determines heavy/light, which adjusts thickness, which changes mass, and the resulting heavy/light mass improvements are reported as prediction gains. This makes the central single-image mass claim partially circular. It is not a full 8-10 because the base mass estimate still contains independent visual/material information and HAT is a bounded calibration rather than an identity, so score 6.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 2 invented entities

The central contribution is an architecture whose output scale is set by unverified depth scale d/z, whose material attributes come from GPT-4, and whose mass result is corrected by a heavy/light prior computed from the model's own prediction. The paper introduces no independently checkable entity or constant; every quantitative output depends on at least one unposted or unverified parameter.

free parameters (6)
  • Virtual cube side length d = not reported
    Predefined input to Eq. (1) that sets the pseudo-voxel scale; total mass scales with volume, so a wrong d shifts all mass predictions, but no value or selection rule is given.
  • Candidate material count K = 5
    Set to K=5; ablation on K in {3,5,10} is deferred to Supplementary, so the reported results depend on this specific choice.
  • Contrastive temperature tau = 0.1
    Temperature in the supervised contrastive loss (Eq. 3); affects learned alignment strength and is fixed without sensitivity analysis in the main text.
  • Alignment weight lambda = not reported
    Balances L_CL and L_align in Eq. (5); value is not reported, but the final material likelihood matrix depends on it.
  • Heavy/light threshold and thickness adjustment ranges (HAT) = not reported
    Sec. 3.4: classification threshold and 'physically plausible' thickness ranges are not specified; this is a post-hoc correction fitted to improve heavy-object mass errors.
  • GPT-4 material attribute table (density, Young's modulus, thickness) = not reported
    Material property values V_j used in Eq. (6) are generated by GPT-4; they are central to every property estimate and are not independently benchmarked or released.
axioms (6)
  • standard math Pinhole projection spacing s = d*sqrt(fx*fy)/z
    Eq. (1) is standard perspective projection, but only if z is true metric depth and fx, fy are calibrated intrinsics.
  • domain assumption Monocular depth estimate z is metric and aligned with camera intrinsics
    Sec. 3.2 uses 'metric depth estimates'; Depth Anything v2 is cited, but no scale calibration is described, and the sensitivity table only covers ±10% perturbations.
  • domain assumption CLIP patch similarity and VLM material proposals capture physical material identity
    Sec. 3.3 treats CLIP embeddings and VLM text candidates as proxies for material; no physical measurement anchors this correspondence.
  • domain assumption SAM part masks correspond to physically homogeneous material regions
    The part-based contrastive loss (Eq. 3) enforces intra-part consistency; if a SAM part spans multiple materials, the alignment is actively wrong.
  • domain assumption GPT-4-generated material attributes are accurate
    Density, Young's modulus, and thickness values are generated by GPT-4 and used as V_j in Eq. (6), with no external verification.
  • domain assumption Material likelihood expectation gives physical property
    Eq. (6) computes the property as an expectation over material likelihoods; the paper itself states that material prediction has weak correlation with mass, weakening this axiom for the central quantity.
invented entities (2)
  • Pseudo-voxel / virtual unit cube no independent evidence
    purpose: Approximates 3D volumetric integration from a single-view 2D grid; mass is the sum of density times pseudo-voxel volume over all sampled points.
    A computational abstraction with no independent handle; its volume depends on the unverified scale of d and z, so it cannot be validated outside the model.
  • Heaviness prior (heavy/light classification) no independent evidence
    purpose: Adjusts thickness for dense objects to reduce mass error in the HAT module.
    A discrete label derived from the model's own initial mass prediction; no external measure of heaviness, and no equation or threshold is given.

pith-pipeline@v1.3.0-alltime-deepseek · 12556 in / 12963 out tokens · 128896 ms · 2026-08-01T05:00:02.570165+00:00 · methodology

0 comments
read the original abstract

Inferring physical properties such as mass, stiffness, and elasticity from a single image is essential for simulation and embodied AI, yet most existing approaches rely on multi-view reconstruction or physics-based supervision. We introduce SiPhy, a unified framework for single-image physical property reasoning that aligns 3D-aware visual cues, depth with language-based material knowledge. From one RGB image, SiPhy samples pseudo-voxel points, extracts CLIP features, and grounds them to material candidates proposed by a VLM. A part-based contrastive aggregator enforces region consistency, while a heaviness-aware refinement improves thickness and volume estimation for dense objects. Across ABO-500, MVImgNet-100, and PhysXNet-100, SiPhy achieves state-of-the-art single-image performance, surpassing multi-view reconstruction methods by improving mass MnRE by up to 93% (vs. PUGS), reducing density MAE by 35.5% (vs. NeRF2Physics), and lowering Young's modulus error by 23.5%. We further validate SiPhy on real hand-object interaction datasets, demonstrating its potential as a data annotation engine for physical understanding from single-view imagery.

Figures

Figures reproduced from arXiv: 2607.22355 by Elkhan Ismayilzada, Hoang Le, Joonwoo Kwon, Yufei Zhang, Zijun Cui.

Figure 1
Figure 1. Figure 1: Overall Architecture of the SiPhy. (1) 3D-aware 2D sampling: spa￾tially distributed 2D points are sampled from the object surface to approximate coarse geometry from a single view. (2) Material probability estimation: CLIP visual features, guided by SAM masks, are aligned with material candidates proposed by a finetuned VLM to produce material probability maps. (3) Physical property pre￾diction: localized … view at source ↗
Figure 2
Figure 2. Figure 2: SiPhy VLM (Sec. 3.1). For each SAM-derived object part, the model receives three visual inputs (part mask, cropped region, full object image) alongside a text prompt and a GPT-4-generated part description. Vicuna-7B-v1.5 jointly processes all inputs and predicts the material category (e.g., “Metal”). 3.2 3D-Aware Visual Sampling Pixel-level properties can be directly inferred from local visual cues and are… view at source ↗
Figure 3
Figure 3. Figure 3: 3D-aware pseudo-voxelization. (a) Depth-adaptive sampling voxels (col￾ored rectangulars). (b) Zoomed comparison: fixed uniform spacing (top) causes voxel overlap and redundancy; our adaptive spacing (bottom) produces non-overlapping cov￾erage that better approximates a surface voxel grid. 3.3 Material Probability Estimation To infer material categories and their associated physical attributes from local ap… view at source ↗
Figure 4
Figure 4. Figure 4: Heaviness-aware thickness refinement (HAT). [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative results on ABO500. Material segmentation (left) and [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative results on HO3D, ARCTIC, ABO500, and MVImgNET. [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Edge-case analysis. From left to right: the first 2 objects are transparent, the next 3 (pillows, same image) are highly textured, and the last is reflective. Predictions are bordered with a green box if correct and a red box if incorrect. SiPhy handles transparent and textured objects well; reflective surfaces remain a failure mode for both methods [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Material-grounded audio generation. Mel-spectrograms for three objects under ChatGPT-only, Vanilla (VLM caption → MakeAnAudio), and Vanilla + SiPhy. SiPhy’s material prior corrects appearance-driven errors (e.g., metal locker misidenti￾fied as wood), producing spectrograms that better match the reference audio. 5 Conclusion We presented SiPhy, a single-image framework that unifies geometric cues, se￾mantic… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references · 1 canonical work pages

  1. [1]

    Alayrac, J.B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., Ring, R., Rutherford, E., Cabi, S., Han, T., Gong, Z., Samangooei, S., Monteiro, M., Menick, J., Borgeaud, S., Brock, A., Nematzadeh, A., Sharifzadeh, S., Binkowski, M., Barreira, R., Vinyals, O., Zisser- man, A., Simonyan, K.: Flamingo:...

    Pith/arXiv arXiv 2022

  2. [2]

    Bakhtin, A., van der Maaten, L., Johnson, J., Gustafson, L., Girshick, R.: Phyre: A new benchmark for physical reasoning (2019),https://arxiv.org/abs/1908. 05656

    2019

  3. [3]

    In: Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR)

    Bell, S., Upchurch, P., Snavely, N., Bala, K.: Material recognition in the wild with the materials in context database. In: Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR). p. – (2015), https://openaccess.thecvf.com/content_cvpr_2015/papers/Bell_Material_ Recognition_in_2015_CVPR_paper.pdf, materials in Context Datab...

    2015

  4. [4]

    Bordes, F., Garrido, Q., Kao, J.T., Williams, A., Rabbat, M., Dupoux, E.: Intphys 2: Benchmarking intuitive physics understanding in complex synthetic environ- ments (2025),https://arxiv.org/abs/2506.09849

    Pith/arXiv arXiv 2025

  5. [5]

    arXiv preprint arXiv:2507.12465 (2025)

    Cao, Z., Chen, Z., Pan, L., Liu, Z.: Physx-3d: Physical-grounded 3d asset genera- tion. arXiv preprint arXiv:2507.12465 (2025)

    arXiv 2025

  6. [6]

    Chaudhari, A., Guntuboina, C., Huang, H., Farimani, A.B.: Alloybert: Alloy prop- erty prediction with large language models (2024),https://arxiv.org/abs/2403. 19783

    2024

  7. [7]

    Chen, Z., Yi, K., Li, Y., Ding, M., Torralba, A., Tenenbaum, J.B., Gan, C.: Com- phy: Compositional physical reasoning of objects and events from videos (2022), https://arxiv.org/abs/2205.01089

    Pith/arXiv arXiv 2022

  8. [8]

    Cherian, A., Corcodel, R., Jain, S., Romeres, D.: Llmphy: Complex physical rea- soning using large language models and world models (2024),https://arxiv.org/ abs/2411.08027

    Pith/arXiv arXiv 2024

  9. [9]

    In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Cherti, M., Beaumont, R., Wightman, R., Wortsman, M., Ilharco, G., Gordon, C., Schuhmann, C., Schmidt, L., Jitsev, J.: Reproducible scaling laws for contrastive language-image learning. In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). p. 2818–2829. IEEE (Jun 2023).https://doi. org/10.1109/cvpr52729.2023.00276,http://dx.doi.o...

    arXiv 2023

  10. [10]

    Chiang, W.L., Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y., Gonzalez, J.E., Stoica, I., Xing, E.P.: Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality (March 2023),https://lmsys.org/ blog/2023-03-30-vicuna/

    2023

  11. [11]

    CoRRabs/2110.06199(2021), https://arxiv.org/abs/2110.06199

    Collins, J., Goel, S., Luthra, A., Xu, L., Deng, K., Zhang, X., Vicente, T.F.Y., Arora, H., Dideriksen, T., Guillaumin, M., Malik, J.: ABO: dataset and bench- marks for real-world 3d object understanding. CoRRabs/2110.06199(2021), https://arxiv.org/abs/2110.06199

    Pith/arXiv arXiv 2021

  12. [12]

    arXiv preprint arXiv:2510.22975 (2025)

    Dagli, R., Xiang, D., Modi, V., Loop, C., Tsang, C.F., Chen, A.H., Hu, A., State, G., Levin, D.I., Shugrina, M.: Vomp: Predicting volumetric mechanical property fields. arXiv preprint arXiv:2510.22975 (2025)

    arXiv 2025

  13. [13]

    Dai, W., Li, J., Li, D., Tiong, A.M.H., Zhao, J., Wang, W., Li, B., Fung, P., Hoi, S.: Instructblip: Towards general-purpose vision-language models with instruction tuning (2023) SiPhy: Single-Image Physical Property Reasoning 17

    2023

  14. [14]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Dou, Y., Oh, W., Luo, Y., Loquercio, A., Owens, A.: Hearing hands: Generating sounds from physical interactions in 3d scenes. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 1795–1804 (2025)

    2025

  15. [15]

    In: Pro- ceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2023)

    Fan, Z., Taheri, O., Tzionas, D., Kocabas, M., Kaufmann, M., Black, M.J., Hilliges, O.: ARCTIC: A dataset for dexterous bimanual hand-object manipulation. In: Pro- ceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2023)

    2023

  16. [16]

    In: CVPR (2020)

    Hampali, S., Rad, M., Oberweger, M., Lepetit, V.: Honnotate: A method for 3d annotation of hand and object poses. In: CVPR (2020)

    2020

  17. [17]

    In: Proceedings of the 33rd annual conference of the cognitive science society

    Hamrick, J., Battaglia, P., Tenenbaum, J.B.: Internal physics models guide prob- abilistic judgments about object dynamics. In: Proceedings of the 33rd annual conference of the cognitive science society. vol. 2. Cognitive Science Society (2011)

    2011

  18. [18]

    In: 2025 International Conference on 3D Vision (3DV)

    Hsu, H.Y., Lin, C.H., Zhai, A.J., Xia, H., Wang, S.: Autovfx: Physically realistic video editing from natural language instructions. In: 2025 International Conference on 3D Vision (3DV). pp. 769–780. IEEE (2025)

    2025

  19. [19]

    arXiv preprint arXiv:2301.12661 (2023)

    Huang, R., Huang, J., Yang, D., Ren, Y., Liu, L., Li, M., Ye, Z., Liu, J., Yin, X., Zhao,Z.:Make-an-audio:Text-to-audiogenerationwithprompt-enhanceddiffusion models. arXiv preprint arXiv:2301.12661 (2023)

    Pith/arXiv arXiv 2023

  20. [20]

    org/10.5281/zenodo.5143773, if you use this software, please cite it as below

    Ilharco, G., Wortsman, M., Wightman, R., Gordon, C., Carlini, N., Taori, R., Dave, A., Shankar, V., Namkoong, H., Miller, J., Hajishirzi, H., Farhadi, A., Schmidt, L.: Openclip (Jul 2021).https://doi.org/10.5281/zenodo.5143773,https://doi. org/10.5281/zenodo.5143773, if you use this software, please cite it as below

    doi:10.5281/zenodo.5143773 2021

  21. [21]

    Jacobs, R., Polak, M.P., Schultz, L.E., Mahdavi, H., Honavar, V., Morgan, D.: Regression with large language models for materials and molecular property pre- diction (2024),https://arxiv.org/abs/2409.06080

    Pith/arXiv arXiv 2024

  22. [22]

    arXiv preprint arXiv:2503.17973 (2025)

    Jiang, H., Hsu, H.Y., Zhang, K., Yu, H.N., Wang, S., Li, Y.: Phystwin: Physics- informed reconstruction and simulation of deformable objects from videos. arXiv preprint arXiv:2503.17973 (2025)

    Pith/arXiv arXiv 2025

  23. [23]

    Kerbl, B., Kopanas, G., Leimkühler, T., Drettakis, G.: 3d gaussian splatting for real-time radiance field rendering (2023),https://arxiv.org/abs/2308.04079

    Pith/arXiv arXiv 2023

  24. [24]

    org/abs/2004.11362

    Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., Krishnan, D.: Supervised contrastive learning (2021),https://arxiv. org/abs/2004.11362

    Pith/arXiv arXiv 2021

  25. [25]

    Li, J., Li, D., Savarese, S., Hoi, S.: Blip-2: Bootstrapping language-image pre- training with frozen image encoders and large language models (2023),https: //arxiv.org/abs/2301.12597

    Pith/arXiv arXiv 2023

  26. [26]

    Li, S., Wu, K., Zhang, C., Zhu, Y.: I-phyre: Interactive physical reasoning (2024), https://arxiv.org/abs/2312.03009

    Pith/arXiv arXiv 2024

  27. [27]

    see doi: https://doi.org/10.1039/d4dd00199k

    Li, Y., Gupta, V., Kilic, M.N.T., Choudhary, K., Wines, D., keng Liao, W., Choud- hary,A.,Agrawal,A.:Hybrid-llm-gnn:integratinglargelanguagemodelsandgraph neural networks for enhanced materials property prediction††electronic supple- mentary information (esi) available. see doi: https://doi.org/10.1039/d4dd00199k. Digital Discovery4(2), 376–383 (2024).htt...

    work page doi:10.1039/d4dd00199k 2024

  28. [28]

    Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning (2023),https:// arxiv.org/abs/2304.08485

    Pith/arXiv arXiv 2023

  29. [29]

    In: ECCV (2020) 18 H

    Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: Nerf: Representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) 18 H. Le et al

    2020

  30. [30]

    Rubungo,A.N.,Arnold,C.,Rand,B.P.,Dieng,A.B.:Llm-prop:Predictingphysical and electronic properties of crystalline solids from their text descriptions (2023), https://arxiv.org/abs/2310.14029

    Pith/arXiv arXiv 2023

  31. [31]

    International Journal of Computer Vision103(3), 348–371 (July 2013).https://doi.org/10.1007/s11263-013-0609-0

    Sharan, L., Liu, C., Rosenholtz, R., Adelson, E.H.: Recognizing materials using perceptually inspired features. International Journal of Computer Vision103(3), 348–371 (July 2013).https://doi.org/10.1007/s11263-013-0609-0

    doi:10.1007/s11263-013-0609-0 2013

  32. [32]

    Shuai, Y., Yu, R., Chen, Y., Jiang, Z., Song, X., Wang, N., Zheng, J., Ma, J., Yang, M., Wang, Z., Ding, W., Zhao, H.: Pugs: Zero-shot physical understanding with gaussian splatting (2025),https://arxiv.org/abs/2502.12231

    Pith/arXiv arXiv 2025

  33. [33]

    In: Levine, S., Vanhoucke, V., Goldberg, K

    Standley, T., Sener, O., Chen, D., Savarese, S.: image2mass: Estimating the mass of an object from its image. In: Levine, S., Vanhoucke, V., Goldberg, K. (eds.) Proceedings of the 1st Annual Conference on Robot Learning. Proceedings of Machine Learning Research, vol. 78, pp. 324–333. PMLR (13–15 Nov 2017), https://proceedings.mlr.press/v78/standley17a.html

    2017

  34. [34]

    In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R

    Wu, J., Lu, E., Kohli, P., Freeman, B., Tenenbaum, J.: Learning to see physics via visual de-animation. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 30. Curran Associates, Inc. (2017),https : / / proceedings . neurips . cc / paper _ files / p...

    2017

  35. [35]

    In: Cortes, C., Lawrence, N., Lee, D., Sugiyama, M., Garnett, R

    Wu, J., Yildirim, I., Lim, J.J., Freeman, B., Tenenbaum, J.: Galileo: Perceiv- ing physical object properties by integrating a physics engine with deep learn- ing. In: Cortes, C., Lawrence, N., Lee, D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 28. Curran Associates, Inc. (2015),https://proceedings.neurips.cc...

    2015

  36. [36]

    arXiv preprint arXiv:2412.11258 (2024)

    Xu, X., Ge, W., Qiu, D., Chen, Z., Yan, D., Liu, Z., Zhao, H., Zhao, H., Zhang, S., Liang, J., Chen, Y.C.: Gaussianproperty: Integrating physical properties to 3d gaussians with lmms. arXiv preprint arXiv:2412.11258 (2024)

    Pith/arXiv arXiv 2024

  37. [37]

    Xu, Z., Wu, J., Zeng, A., Tenenbaum, J.B., Song, S.: Densephysnet: Learning dense physical object representations via multi-step dynamic interactions (2019), https://arxiv.org/abs/1906.03853

    Pith/arXiv arXiv 2019

  38. [38]

    arXiv:2406.09414 (2024)

    Yang, L., Kang, B., Huang, Z., Zhao, Z., Xu, X., Feng, J., Zhao, H.: Depth anything v2. arXiv:2406.09414 (2024)

    Pith/arXiv arXiv 2024

  39. [39]

    arXiv preprint arXiv:2405.02794 (2024)

    Yu, S., Lin, K., Xiao, A., Duan, J., Soh, H.: Octopi: Object property reasoning with large tactile-language models. arXiv preprint arXiv:2405.02794 (2024)

    Pith/arXiv arXiv 2024

  40. [40]

    In: Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Yu, X., Xu, M., Zhang, Y., Liu, H., Ye, C., Wu, Y., Yan, Z., Zhu, C., Xiong, Z., Liang, T., et al.: Mvimgnet: A large-scale dataset of multi-view images. In: Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 9150–9161 (2023)

    2023

  41. [41]

    In: CVPR (2024)

    Zhai, A.J., Shen, Y., Chen, E.Y., Wang, G.X., Wang, X., Wang, S., Guan, K., Wang, S.: Physical property understanding from language-embedded feature fields. In: CVPR (2024)

    2024

  42. [42]

    Zheng, Z., Yan, X., Chen, Z., Wang, J., Lim, Q.Z.E., Tenenbaum, J.B., Gan, C.: Contphy: Continuum physical concept learning and reasoning from videos (2024), https://arxiv.org/abs/2402.06119

    Pith/arXiv arXiv 2024