REVIEW 2 major objections 3 minor 28 references
Future rendering quality does not track future surface accuracy in dynamic-scene reconstruction; a new benchmark measures the gap.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 07:19 UTC pith:JWIJZ4QO
load-bearing objection FutureSurf is a genuinely useful evaluation contribution with a clean protocol, but the abstract overstates two results and the decoupling claim needs cross-method evidence. the 2 major comments →
Future Rendering neq Future Surface: A Benchmark and Dataset for Dynamic Surface Reconstruction Beyond the Observed Window
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central discovery is a measured decoupling: for the tested dynamic-surface backbones, future rendering quality and future-surface accuracy do not move together. On six animated asset scenes and a suite of eight controlled motions with exact future ground truth, per-frame future Chamfer distance is 2.0–6.6× the observed-window error, and per-frame |ρ(LPIPS,CD)| averages 0.13, with a linear fit explaining under 6% of variance. The future error is structured, concentrating where the surface moves, and it persists after removing a global Sim(3) gauge, so it is genuine non-rigid shape error. The paper frames this as evidence that the field's standard evaluation protocol measures the wrong qua
What carries the argument
The benchmark's load-bearing instrument is the future/observed gap: the ratio of per-frame bidirectional Chamfer distance between extracted and ground-truth meshes on the held-out future window to the same quantity on the observed window. It converts 'how good is the future surface' into a normalized diagnostic that isolates extrapolation failure from interpolation quality, and it is paired with three falsification controls (a surface-invariant twist, a rigid-rotation gauge control, and a frozen-future stop control) designed to expose a broken metric or alignment. A second instrument, the ground-truth-side recoverability oracle, fits simple and learned per-vertex temporal rules to observed g
Load-bearing premise
The paper frames the measured future-surface gap and rendering–surface decoupling as a property of dynamic-scene reconstruction in general, but both tested backbones share the same time-conditioned deformation-MLP temporal model; if a temporally distinct representation extrapolates accurately, the gap would be an artifact of that architecture family rather than a field-wide finding.
What would settle it
Train a dynamic reconstruction method with a temporally distinct representation (e.g., an explicit 4D grid or physics-based simulation) on the released FutureSurf splits; if its future Chamfer gap drops to near unity (future/observed ≈ 1) on the recoverable controlled motions and its per-frame |ρ(LPIPS,CD)| rises substantially, the paper's claim of a general field-wide decoupling would be refuted.
If this is right
- The standard practice of reporting PSNR/SSIM/LPIPS on future frames does not certify future-surface quality; a separate per-frame mesh metric is needed for deployment claims.
- Any method evaluated only inside the observed window may overstate its usefulness for future-time tasks such as AR overlays, robot interaction, and anticipatory planning.
- The gap persists across two deformation-MLP backbones and six varied scenes, suggesting the limitation is not scene-specific but tied to the temporal model family.
- The error's concentration where the surface moves implies that future-surface failure is predictable in location, and that motion-aware diagnostics (per-vertex maps) should accompany scores.
- The recoverability oracle shows that some futures are known in principle from observed motion; the gap on those is a representation or extrapolation issue, not an information-theoretic one.
Where Pith is reading between the lines
- A natural next test is to train a temporally distinct representation—for example, an explicit 4D grid, a physics-based simulator, or a neural-SDF flow—on the released splits; if its future/observed gap drops to near unity on the recoverable controlled motions, the paper's family-level conclusion would be refined into an architecture-specific one.
- The decoupling result implies that prior dynamic-scene forecasting papers that report only rendering metrics may be unknowingly releasing methods whose future geometry is poor; re-evaluating them with the released mesh protocol would be a high-value, low-cost extension.
- The benchmark's synthetic-only design could be extended toward real captures by using a fitted proxy for future ground truth (e.g., a high-fidelity offline reconstruction) or depth sensors, though the paper's point that exact future GT requires analytic motion suggests a hybrid evaluation may be needed.
- If the gap generalizes to other architecture families, it would motivate treating future-surface accuracy as a first-class benchmark axis alongside rendering, and could drive new training objectives that include temporal-extrapolation regularization or drift penalties on static futures.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. FutureSurf proposes a standardized benchmark and dataset for evaluating dynamic surface reconstruction at held-out future times. It contributes eight analytically defined controlled motions (five surface-changing, three falsification controls) with exact per-frame ground-truth meshes, a 75/25 train/future split, a method-agnostic per-frame Chamfer protocol, a ground-truth-side recoverability oracle, benchmark card, and Croissant metadata. Using DG-Mesh and Deformable-3DGS as backbones, the paper reports a 2.0–6.6× future/observed Chamfer gap on asset scenes and 2.7–4.1× on the controlled motions, with falsification controls behaving as designed; a per-motion oracle recovers four of five constructed futures using simple matched rules; and a within-method analysis finds weak rank correlation between future rendering quality (LPIPS) and future-surface CD (mean |ρ|=0.13), which the paper interprets as a rendering–surface decoupling. The authors release splits, scoring code, and metadata. The paper is explicitly framed as a diagnostic benchmark rather than a new reconstruction method.
Significance. If taken as scoped, this is a valuable contribution: it makes an unmeasured quantity—future-surface mesh accuracy—measurable, and provides exact ground truth, falsification controls, and reproducible scripts. The recoverability oracle is honestly scoped as an optimistic, ground-truth-side reference, and the paper carefully separates removable gauge error from non-rigid error. The main limitations are the breadth of tested temporal models (both backbones share a deformation-MLP temporal family) and the within-method nature of the decoupling evidence; these limit the generality of the headline 'Future Rendering ≠ Future Surface' but do not undermine the benchmark itself. The release of code, splits, and a CPU-run oracle is a concrete strength for reproducibility.
major comments (2)
- [Abstract and §1 (Contribution 2) vs §6] The abstract and Contribution 2 state that 'four of five recoverable from observed motion by a fixed rule.' This is contradicted by §6: Table 6 shows the best rule varies per motion (harmonic K=2 for wave/compound, cubic for stretch, quadratic for accel, velocity for bulge) and the text explicitly says 'No single per-vertex extrapolation rule covers the suite.' The oracle establishes recoverability only under per-motion matched rule families, not a fixed rule. This phrasing is load-bearing because it is used to argue that the 2.7–4.1× gap reflects a backbone limitation rather than intrinsic unknowability. Please rephrase to 'per-motion matched rules' or 'simple rules' and align the abstract/contributions with §6.
- [§7, Table 4] The decoupling conclusion is based on per-frame Spearman correlations computed within each DG-Mesh scene over future frames (mean |ρ(LPIPS,CD)| = 0.13). This supports only the within-method statement that, for DG-Mesh, rendering quality does not track surface error over the prediction horizon. It does not support the broader claim that 'the NVS metrics the field reports do not track future geometry,' which is a cross-method claim. No cross-method ranking analysis is provided; the second backbone is not included in the decoupling analysis, and §1 itself states the decoupling is reported on DG-Mesh. Please either add a cross-method analysis (e.g., rank-correlating method-level future PSNR/LPIPS with future CD across methods/scenes) or restrict the claim to the tested DG-Mesh backbone.
minor comments (3)
- [§3, §5] Spacing typos throughout: e.g., 'a monocular orbit camera,200frames, the first75%' (§3) and 'gap:2.7–4.1×by mean' (§5). These are likely LaTeX artifacts but should be cleaned before publication.
- [Table 4] The column header 'Non-rigid' should explicitly say it is the Sim(3)-gauge-removed future gap; otherwise a reader may confuse it with a non-rigid motion class. The caption explains it, but the header itself is ambiguous.
- [§6, Table 6] The oracle CD is bounding-box-normalized linear Chamfer, which the text says is on a different scale from Table 5's absolute mesh CD. It would help to state in the table caption that the ratios vs. the freeze reference are the primary comparison, not the absolute CD values.
Circularity Check
No significant circularity: the paper's central claims are empirical benchmark measurements, not derivations from fitted inputs or self-citations.
full rationale
FutureSurf is an evaluation contribution. Its headline results—future CD gaps of 2.0–6.6× and the weak per-frame rendering–surface rank correlation (|ρ(LPIPS,CD)|=0.13)—are direct measurements on trained backbones scored against exact analytic ground-truth meshes with released CPU scoring code. No parameter is fitted to a target quantity and then renamed as a prediction; the recoverability oracle is explicitly a ground-truth-side diagnostic ('This is deliberately an oracle evaluation... not a baseline method'), and its rule families are matched to the analytic construction, so it does not enter the gap or decoupling measurements. The paper contains no load-bearing self-citations; all cited prior work is external. The acknowledged limitation that both backbones share a deformation-MLP temporal model, and the concern that the decoupling is reported only within one backbone over time, are threats to external validity, not circularity. The falsification controls are sanity checks of the protocol rather than derived predictions. The derivation chain is therefore self-contained and the empirical claims stand on their own measurements.
Axiom & Free-Parameter Ledger
free parameters (3)
- Controlled motion parameters =
e.g., wave amplitude 0.22, bulge amplitude 0.34, bulge width 0.28
- Split ratio (75/25) =
75% observed / 25% future
- Oracle rule families per motion =
wave/compound harmonic K=2; stretch cubic; accel quadratic; bulge ≈ freeze
axioms (4)
- domain assumption The analytic motion maps F_m(x,t) define exact per-frame ground-truth surfaces.
- standard math Bidirectional Chamfer on vertex sets in a shared canonical frame is a valid measure of surface accuracy for the primary score.
- domain assumption The tested backbones (DG-Mesh, Deformable-3DGS) are representative of the class of dynamic reconstruction methods.
- domain assumption Training on observed frames only is enforced; backbones do not observe future frames during training.
read the original abstract
Dynamic-scene reconstruction is almost always evaluated inside the observed time window, yet deployment settings such as AR overlays, robot interaction, and anticipatory planning need the future surface: the geometry at times beyond those captured. No standard benchmark measures this. We introduce FutureSurf, a controlled diagnostic benchmark and dataset for future-time surface reconstruction that trades scene diversity for exact future ground truth and falsification controls. A method trains on the observed first 75% of a sequence; we score its extracted per-frame surface on the held-out future by Chamfer distance, reporting absolute future CD as the primary score and the future/observed gap as a diagnostic. The dataset contains eight analytically defined controlled motions, including three falsification controls, with exact per-frame ground-truth meshes. We also provide a ground-truth-side recoverability oracle. The release includes split files, scoring code, a benchmark card, and Croissant metadata. On the controlled motions, the DG-Mesh backbone leaves a 2.7-4.1$\times$ gap even for futures predictable in principle (four of five recoverable from observed motion by a fixed rule), while the falsification controls behave as designed (the surface-invariant motion shows no gap). Beyond the contributed dataset, the gap persists across six animated DG-Mesh asset scenes and a second backbone, Deformable-3DGS (2.0-6.6$\times$; both share a deformation-MLP temporal model). The benchmark also shows that future rendering quality and future-surface accuracy are statistically decoupled, so the novel-view-synthesis metrics the field reports do not track future geometry. The future error is structured, concentrating where the surface moves. The dataset, evaluation toolkit, and scoring code are available on Hugging Face and GitHub (https://github.com/Ricky-S/futuresurf).
Figures
Reference graph
Works this paper leans on
-
[1]
Croissant: A metadata format for ML- ready datasets
Mubashara Akhtar, Omar Benjelloun, Costanza Conforti, Luca Foschini, Pieter Gijsbers, Joan Giner-Miguelez, Su- jata Goswami, Nitisha Jain, Michalis Karamousadakis, Satyapriya Krishna, Michael Kuchnik, Sylvain Lesage, Quentin Lhoest, Pierre Marcenac, Manil Maskey, Peter Mattson, Luis Oala, Hamidah Oderinwale, Pierre Ruyssen, Tim Santos, Rajat Shinde, Elena...
2024
-
[2]
4D Gaussian splatting as a learned dynamical system.arXiv preprint arXiv:2512.19648, 2025
Arnold Caleb Asiimwe and Carl V ondrick. 4D Gaussian splatting as a learned dynamical system.arXiv preprint arXiv:2512.19648, 2025. 3
arXiv 2025
-
[3]
GIC: Gaussian-informed continuum for physical property identi- fication and simulation
Junhao Cai, Yuji Yang, Weihao Yuan, Yisheng He, Zilong Dong, Liefeng Bo, Hui Cheng, and Qifeng Chen. GIC: Gaussian-informed continuum for physical property identi- fication and simulation. InAdvances in Neural Information Processing Systems (NeurIPS), 2024. arXiv:2406.14927. 3
Pith/arXiv arXiv 2024
-
[4]
Weiwei Cai, Weicai Ye, Peng Ye, Tong He, and Tao Chen. DynaSurfGS: Dynamic surface reconstruction with planar- based Gaussian splatting.arXiv preprint arXiv:2408.13972,
-
[5]
Monocular dynamic view synthesis: A reality check
Hang Gao, Ruilong Li, Shubham Tulsiani, Bryan Russell, and Angjoo Kanazawa. Monocular dynamic view synthesis: A reality check. InNeurIPS, 2022. 4
2022
-
[6]
Fluid- Nexus: 3D fluid reconstruction and prediction from a single video
Yue Gao, Hong-Xing Yu, Bo Zhu, and Jiajun Wu. Fluid- Nexus: 3D fluid reconstruction and prediction from a single video. InCVPR, 2025. arXiv:2503.04720. 3
Pith/arXiv arXiv 2025
-
[7]
Datasheets for datasets.Communications of the ACM, 2021
Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jen- nifer Wortman Vaughan, Hanna Wallach, Hal Daum´e III, and Kate Crawford. Datasheets for datasets.Communications of the ACM, 2021. 3
2021
-
[8]
Klaus Greff, Francois Belletti, Lucas Beyer, Carl Do- ersch, Yilun Du, Daniel Duckworth, David J. Fleet, Dan Gnanapragasam, Florian Golemo, Charles Herrmann, Thomas Kipf, Abhijit Kundu, Dmitry Lagun, Issam Laradji, Hsueh-Ti Derek Liu, Henning Meyer, Yishu Miao, Derek Nowrouzezahrai, Cengiz Oztireli, Etienne Pot, Noha Rad- wan, Daniel Rebain, Sara Sabour, ...
2022
-
[9]
2D Gaussian splatting for geometrically ac- curate radiance fields
Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2D Gaussian splatting for geometrically ac- curate radiance fields. InSIGGRAPH 2024 Conference Pa- pers, 2024. 3
2024
-
[10]
3D Gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics,
Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3D Gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics,
-
[11]
Mijeong Kim, Jungtaek Kim, and Bohyung Han. GP- 4DGS: Probabilistic 4D Gaussian splatting from monocular video via variational Gaussian processes. InCVPR, 2026. arXiv:2604.02915. 3
Pith/arXiv arXiv 2026
-
[12]
Minje Kim, Younghyun Noh, Jaesoon Kim, and Tae-Kyun Kim. DySurface: Consistent 4D surface reconstruction via bridging explicit Gaussians and implicit functions.arXiv preprint arXiv:2605.10360, 2026. 3
Pith/arXiv arXiv 2026
-
[13]
Space-time forecasting of dynamic scenes with motion-aware Gaussian grouping
Junmyeong Lee, Hoseung Choi, and Minsu Cho. Space-time forecasting of dynamic scenes with motion-aware Gaussian grouping. InCVPR, 2026. arXiv:2602.21668. 3
Pith/arXiv arXiv 2026
-
[14]
Dynamic Gaus- sians mesh: Consistent mesh reconstruction from dynamic scenes
Isabella Liu, Hao Su, and Xiaolong Wang. Dynamic Gaus- sians mesh: Consistent mesh reconstruction from dynamic scenes. InInternational Conference on Learning Represen- tations (ICLR), 2025. arXiv:2404.12379. 3, 4
Pith/arXiv arXiv 2025
-
[15]
Model cards for model reporting
Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, In- ioluwa Deborah Raji, and Timnit Gebru. Model cards for model reporting. InProceedings of the Conference on Fair- ness, Accountability, and Transparency (FAT* ’19), 2019. 3
2019
-
[16]
Barron, Sofien Bouaziz, Dan B
Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T. Barron, Sofien Bouaziz, Dan B. Goldman, Ricardo Martin- Brualla, and Steven M. Seitz. HyperNeRF: A higher- dimensional representation for topologically varying neural radiance fields.ACM Transactions on Graphics, 2021. Proc. SIGGRAPH Asia 2021. 3
2021
-
[17]
D-NeRF: Neural radiance fields for dynamic scenes
Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-NeRF: Neural radiance fields for dynamic scenes. InCVPR, 2021. 3
2021
-
[18]
Boyuan Wang, Xiaofeng Wang, Yongkang Li, Zheng Zhu, Yifan Chang, Angen Ye, Guosheng Zhao, Chaojun Ni, Guan Huang, Yijie Ren, Yueqi Duan, and Xingang Wang. Recon- Phys: Reconstruct appearance and physical attributes from single video.arXiv preprint arXiv:2604.07882, 2026. 3
Pith/arXiv arXiv 2026
-
[19]
ODE-GS: Latent ODEs for dynamic scene extrapolation with 3D Gaussian splat- ting
Daniel Wang, Patrick Rim, Tian Tian, Dong Lao, Alex Wong, and Ganesh Sundaramoorthi. ODE-GS: Latent ODEs for dynamic scene extrapolation with 3D Gaussian splat- ting. InInternational Conference on Learning Representa- tions (ICLR), 2026. arXiv:2506.05480. 3
Pith/arXiv arXiv 2026
-
[20]
Shuo Wang, Binbin Huang, Ruoyu Wang, and Shenghua Gao. Space-time 2D Gaussian splatting for accurate sur- face reconstruction under complex dynamic scenes.arXiv preprint arXiv:2409.18852, 2024. 3
Pith/arXiv arXiv 2024
-
[21]
DUSt3R: Geometric 3D vision made easy
Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and J ´erˆome Revaud. DUSt3R: Geometric 3D vision made easy. InCVPR, 2024. arXiv:2312.14132. 3
Pith/arXiv arXiv 2024
-
[22]
4D Gaussian splatting for real-time dynamic scene render- ing
Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4D Gaussian splatting for real-time dynamic scene render- ing. InCVPR, 2024. arXiv:2310.08528. 3
Pith/arXiv arXiv 2024
-
[23]
Renjie Wu, Hongdong Li, Jose M. Alvarez, and Miaomiao Liu. 4DSurf: High-fidelity dynamic scene surface recon- struction. InCVPR, 2026. arXiv:2603.28064. 3
arXiv 2026
-
[24]
PhysGaussian: Physics- integrated 3D Gaussians for generative dynamics
Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, and Chenfanfu Jiang. PhysGaussian: Physics- integrated 3D Gaussians for generative dynamics. InCVPR,
-
[25]
Deformable 3D Gaussians for 9 high-fidelity monocular dynamic scene reconstruction
Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3D Gaussians for 9 high-fidelity monocular dynamic scene reconstruction. In CVPR, 2024. arXiv:2309.13101. 3
Pith/arXiv arXiv 2024
-
[26]
Gaus- sianPrediction: Dynamic 3D Gaussian prediction for motion extrapolation and free view synthesis
Boming Zhao, Yuan Li, Ziyu Sun, Lin Zeng, Yujun Shen, Rui Ma, Yinda Zhang, Hujun Bao, and Zhaopeng Cui. Gaus- sianPrediction: Dynamic 3D Gaussian prediction for motion extrapolation and free view synthesis. InSIGGRAPH 2024 Conference Papers, 2024. 3
2024
-
[27]
GauSTAR: Gaussian surface tracking and reconstruction
Chengwei Zheng, Lixin Xue, Juan Zarate, and Jie Song. GauSTAR: Gaussian surface tracking and reconstruction. In CVPR, 2025. arXiv:2501.10283. 3
Pith/arXiv arXiv 2025
-
[28]
Reconstruction and simulation of elastic objects with spring- mass 3D Gaussians
Licheng Zhong, Hong-Xing Yu, Jiajun Wu, and Yunzhu Li. Reconstruction and simulation of elastic objects with spring- mass 3D Gaussians. InEuropean Conference on Computer Vision (ECCV), 2024. arXiv:2403.09434. 3 10
Pith/arXiv arXiv 2024
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.