Pith. sign in

REVIEW 3 major objections 5 minor 35 references

SGC scores 3D geometric consistency of generated videos by measuring how much local camera poses from static background patches disagree.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 22:13 UTC pith:5VY3AGNO

load-bearing objection Solid new diagnostic for background geometric rigidity in generative video; estimator domain-shift is the real soft spot, not a fatal one. the 3 major comments →

arxiv 2603.19048 v2 pith:5VY3AGNO submitted 2026-03-19 cs.CV

Measuring 3D Spatial Geometric Consistency in Dynamic Video Generation

Weijia Dou , Wenzhao Zheng , Weiliang Chen , Yu Zheng , Jie Zhou , Jiwen Lu This is my paper
classification cs.CV
keywords video generationgeometric consistency3D spatial consistencycamera pose estimationevaluation metricstatic background isolationdepth clustering
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Generative video models often look sharp frame by frame yet break the 3D geometry of the scene: static buildings warp, distant mountains squeeze, objects flicker or stick to moving things. Fidelity scores such as FVD barely notice these failures, while many consistency scores punish valid foreground motion. The paper introduces SGC, a metric that first masks out moving objects, splits the remaining static background into depth-coherent patches, recovers a local camera pose for each patch, and scores how much those poses diverge from one another and from a global trajectory. On real driving, robot, and web videos the score stays low; on many current generators it rises sharply, exposing geometric collapse that other metrics miss. The claim is that this divergence is a reliable, motion-robust measure of whether a video could have been filmed in a single coherent 3D world.

Core claim

In a geometrically consistent video every static background point must obey one shared camera transformation. Measuring the variance among local camera poses estimated from distinct static sub-regions, together with their agreement to a global trajectory and cross-frame depth alignment, yields a scalar SGC score that quantifies 3D spatial geometric inconsistency and flags failures overlooked by FVD, VBench background-consistency, and 2-D feature-matching metrics.

What carries the argument

SGC: after motion segmentation isolates the static background, depth clustering partitions it into sub-regions; PnP recovers a local relative pose for each; inter-region and global pose variances plus depth-consistency error are PCA-weighted into a single score.

Load-bearing premise

The off-the-shelf depth, tracking, pose, and motion-segmentation models must be accurate enough on both real and generated video that residual pose divergence really reflects geometric failure rather than estimator error.

What would settle it

A generator that produces videos scoring low on SGC yet still shows clear static-background warping, perspective collapse, or object impermanence under human inspection would falsify the claim that SGC captures the relevant geometric failures.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces SGC, a diagnostic metric for 3D spatial geometric consistency in dynamically generated videos. It isolates the static background (SegAnyMo), partitions it by 1-D depth k-means, estimates per-subregion relative camera poses via PnP on tracked points and predicted depths, and aggregates local/global pose-variance terms plus cross-frame depth error into a PCA-weighted scalar. On a 1,296-video mix of GenWorld generative clips and real anchors (nuScenes, RT-1, OpenVid), SGC ranks models differently from FVD, VBench-BC, and MEt3R, with real videos lowest and models such as Latte/Seine highest. Controlled geometric corruptions on nuScenes produce monotonic SGC increases; ablations of MOS and clustering behave as expected.

Significance. Geometric instability in high-fidelity video generators is a genuine evaluation gap: fidelity metrics ignore multi-plane rigidity, while many consistency metrics either break under foreground motion or conflate valid dynamics with failure. A physically motivated, motion-disentangled metric that reports local pose divergence is therefore useful to the community. Strengths include a clear pipeline (Fig. 2, Algs. 1–2), public code, controlled synthetic-perturbation tests (Fig. 5) that causally link geometric degradation to score, motion-stratified analyses (Fig. 6), and component-level construct checks (Fig. 7). If the attribution of residual variance to scene geometry (rather than estimator domain shift) can be secured, SGC would be a practical complementary diagnostic for T2V/I2V/V2V research.

major comments (3)
  1. [§3.1–3.2, §4.1–4.3, Table 1, Fig. 5] Central attribution claim (Abstract; §4.2 Table 1; Fig. 4) rests on residual local-pose / depth variance being dominated by scene geometry, not by domain failure of the fixed foundation stack (VGGT, Video Depth Anything, DELTA, SegAnyMo; §3.1–3.2, Unified Protocol §4.1). Controlled corruptions (§4.3, Fig. 5) only inject geometry errors into real videos; they do not test the reverse—whether the same estimators systematically inflate variance on clean generative content that is merely OOD in texture, lighting, or mild non-rigidity. Without estimator-confidence diagnostics, cross-estimator agreement, or a held-out set of generative clips judged geometrically coherent by humans, the ranking that “identifies critical failures missed by FVD/VBench/MEt3R” remains partly confounded. This is load-bearing for the paper’s diagnostic claim and should be addressed experimentally or with strong, quant
  2. [§4.1, Appendix A.2, Table 1] Appendix A.2 and §4.1 state that PCA weights are learned once on the full 1,196-video calibration set that is later used for model ranking (Table 1). This couples the scalar definition to the evaluation distribution and introduces mild circularity: the same data both define the metric and produce the leaderboard. The appendix also calls the result “parameter-free,” which is inconsistent with data-dependent weights (and with free choices of k-means granularity and log/Z/min-max transforms). Report weights from a held-out split, or fix weights a priori and show rank stability under re-estimation; drop or qualify the “parameter-free” language.
  3. [Table 1, Fig. 6, §4.2] Table 1 reports point estimates only for 10 generative models and 3 real anchors, with no error bars, bootstrap intervals, or significance tests on pairwise rank differences. Given video-level variance visible in the motion-stratified violins (Fig. 6) and the modest size of some model subsets, claims that Cosmos is “most competitive” or that Latte/Seine exhibit “pronounced difficulties” need uncertainty quantification. Add per-model standard errors or non-parametric tests so that the ranking can be interpreted as stable rather than sample noise.
minor comments (5)
  1. [Title, Abstract] The manuscript title and abstract use “Dynamic Video Generation” / “dynamically generated videos”; the body also uses “Dynamic Generated Videos.” Standardize terminology.
  2. [Fig. 2] Fig. 2 pipeline diagram is hard to parse in the provided text rendering (garbled labels). Ensure high-resolution vector graphics and legible component names in the camera-ready version.
  3. [§3.3, §4.4] k for depth k-means and the precise definition of ECD (Estimated Camera Dynamics) are only sketched; state defaults and any sensitivity to k in the main text or a short appendix table.
  4. [§2] Related-work coverage of concurrent geometric / physical video metrics (e.g., VideoPhy is cited briefly) could note more explicitly how SGC differs from multi-view consistency scores beyond MEt3R.
  5. [§3] Occasional notation drift: M_static vs Mstatic, P_local vs P^local; unify subscript/superscript style.

Circularity Check

1 steps flagged

Mild data-dependent circularity only in PCA aggregation weights fitted on the same 1196-video set later ranked; core pose-divergence principle and controlled validations remain independent.

specific steps
  1. fitted input called prediction [Sec. 4.1 Unified Evaluation Protocol; Appendix A.2 Final PCA Component Weights]
    "to aggregate SGC components, PCA weights are learned exactly once on the full dataset. ... these components are weighted objectively using Principal Component Analysis (PCA) on the normalized score matrix from our 1196-video calibration dataset. The weights are derived from the normalized loadings of the first principal component (PC1)."

    The five component metrics (local/global translational and rotational variances plus depth error) are first computed on the 1196-video set; PCA loadings of PC1 on that same matrix then become the fixed weights for the final SGC scalar used to rank those identical videos. The composite score is therefore partially tuned to the principal mode of variance already present in the evaluation data rather than an a-priori fixed combination, introducing mild statistical dependence between the fitted aggregator and the reported rankings.

full rationale

The paper's central derivation is not circular. SGC is defined from a physical premise (static background points must share one camera transform) via explicit pipeline steps: MOS isolation of M_static, depth-based sub-region partitioning, local PnP poses, and variance/depth-error components (Secs. 3.1–3.4, Fig. 2, Algs. 1–2). These components are computed from off-the-shelf estimators without fitting to the target ranking. Controlled synthetic perturbations on real nuScenes videos (Sec. 4.3, Fig. 5) produce monotonic SGC increases independent of any generative-model fit, and ablations (Tab. 2) isolate MOS/clustering effects. The sole mild circularity is that final scalar aggregation uses PCA loadings learned once on the full 1196-video calibration/evaluation set (Sec. 4.1 Unified Protocol; Appendix A.2), so the composite weights are data-dependent on the same videos later ranked. This is a fitted-input issue for the scalar only; individual components, real-vs-gen separation, and perturbation sensitivity do not reduce to that fit. No self-definitional equations, no load-bearing self-citation uniqueness theorems, no ansatz smuggling, and no renaming of a known result as a first-principles derivation. Score 3 reflects one non-central fitted aggregation step while the diagnostic claim retains independent content.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 1 invented entities

The metric rests on standard multi-view geometry plus four off-the-shelf estimators treated as oracles, a depth-clustering heuristic, and PCA weights fit to the evaluation corpus. No new physical entities are postulated; free parameters are the clustering and aggregation choices.

free parameters (3)
  • PCA component weights (w_k) = trans_glob 0.2459, trans_loc 0.2403, depth 0.2307, rot_glob 0.1665, rot_loc 0.1167
    Derived once from PC1 loadings on the full 1 196-video calibration set and then frozen for all reported scores (Appendix A.2).
  • k-means cluster count / depth partitioning granularity
    Number of depth-coherent sub-regions used for local PnP; chosen by GPU k-means on 1-D depth values inside M_static (Sec. 4.1).
  • log + Z-score + min-max normalization constants
    Dataset-level statistics used to standardize raw variance components before PCA weighting (Appendix A.1).
axioms (4)
  • domain assumption All static-background 3D points must be consistent with a single shared camera transformation T_cam.
    Stated as the physical premise of SGC (Sec. 1 and Sec. 3 opening).
  • domain assumption Off-the-shelf estimators (VGGT, Video Depth Anything, DELTA, SegAnyMo) supply sufficiently accurate depth, pose, tracks and motion masks on both real and generative video.
    Unified Evaluation Protocol (Sec. 4.1) freezes these models as the sole sources of geometric input.
  • ad hoc to paper 1-D depth k-means yields spatially coherent sub-regions adequate for stable PnP.
    Chosen after ablation against grid and depth+spatial variants (Table 2, Sec. 4.5).
  • standard math Perspective-n-Point with RANSAC recovers a reliable local relative pose from tracked 2-D points and back-projected 3-D points.
    Standard multi-view geometry invoked in Sec. 3.3–3.4.
invented entities (1)
  • SGC score (composite of local/global pose variances + depth consistency error) no independent evidence
    purpose: Scalar diagnostic of 3D spatial geometric consistency for dynamic video.
    Defined by the aggregation pipeline in Sec. 3.4 and Appendix A; no independent physical existence outside the metric.

pith-pipeline@v1.1.0-grok45 · 18238 in / 2934 out tokens · 23266 ms · 2026-07-13T22:13:01.754551+00:00 · methodology

0 comments
read the original abstract

Recent generative models can produce high-fidelity videos, yet they often exhibit 3D spatial geometric inconsistencies. Existing evaluation methods fail to accurately characterize these inconsistencies: fidelity-centric metrics like FVD are insensitive to geometric distortions, while consistency-focused benchmarks often penalize valid foreground dynamics. To address this gap, we introduce SGC, a metric for evaluating 3D \textbf{S}patial \textbf{G}eometric \textbf{C}onsistency in dynamically generated videos. We quantify geometric consistency by measuring the divergence among multiple camera poses estimated from distinct local regions. Our approach first separates static from dynamic regions, then partitions the static background into spatially coherent sub-regions. We predict depth for each pixel, estimate a local camera pose for each subregion, and compute the divergence among these poses to quantify geometric consistency. Experiments on real and generative videos demonstrate that SGC robustly quantifies geometric inconsistencies, effectively identifying critical failures missed by existing metrics.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 16 linked inside Pith

  1. [1]

    Hotshot.https://hotshot.co/(2023) 8, 9, 4

    2023

  2. [2]

    Zeroscope-v2-xl.https://zeroscope.replicate.dev/(2024) 8, 9, 4

    2024

  3. [3]

    arXiv preprint arXiv:2501.03575 (2025) 1, 3, 8, 9, 4

    Agarwal, N., Ali, A., Bala, M., Balaji, Y., Barker, E., Cai, T., Chattopadhyay, P., Chen, Y., Cui, Y., Ding, Y., et al.: Cosmos world foundation model platform for physical ai. arXiv preprint arXiv:2501.03575 (2025) 1, 3, 8, 9, 4

    Pith/arXiv arXiv 2025

  4. [4]

    arXiv preprint arXiv:2505.00209 (2025) 2, 3, 8, 9

    Allen, K., Doersch, C., Zhou, G., Suhail, M., Driess, D., Rocco, I., Rubanova, Y., Kipf, T., Sajjadi, M.S., Murphy, K., et al.: Direct motion models for assessing generated videos. arXiv preprint arXiv:2505.00209 (2025) 2, 3, 8, 9

    Pith/arXiv arXiv 2025

  5. [5]

    In: CVPR (2024) 3, 8, 9

    Asim, M., Wewer, C., Wimmer, T., Schiele, B., Lenssen, J.E.: Met3r: Measuring multi-view consistency in generated images. In: CVPR (2024) 3, 8, 9

    2024

  6. [6]

    arXiv preprint arXiv:2406.03520 (2024) 3

    Bansal, H., Lin, Z., Xie, T., Zong, Z., Yarom, M., Bitton, Y., Jiang, C., Sun, Y., Chang, K.W., Grover, A.: Videophy: Evaluating physical commonsense for video generation. arXiv preprint arXiv:2406.03520 (2024) 3

    Pith/arXiv arXiv 2024

  7. [7]

    In: SIGGRAPH Asia 2024 Conference Papers

    Bar-Tal, O., Chefer, H., Tov, O., Herrmann, C., Paiss, R., Zada, S., Ephrat, A., Hur, J., Liu, G., Raj, A., et al.: Lumiere: A space-time diffusion model for video generation. In: SIGGRAPH Asia 2024 Conference Papers. pp. 1–11 (2024) 1, 3

    2024

  8. [8]

    arXiv preprint arXiv:2311.15127 (2023) 3

    Blattmann, A., Dockhorn, T., Kulal, S., Mendelevitch, D., Kilian, M., Lorenz, D., Levi, Y., English, Z., Voleti, V., Letts, A., et al.: Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127 (2023) 3

    Pith/arXiv arXiv 2023

  9. [9]

    In: CVPR

    Blattmann, A., Rombach, R., Ling, H., Dockhorn, T., Kim, S.W., Fidler, S., Kreis, K.: Align your latents: High-resolution video synthesis with latent diffusion models. In: CVPR. pp. 22563–22575 (2023) 3

    2023

  10. [10]

    arXiv preprint arXiv:2212.06817 (2022) 8, 9, 3, 4

    Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Dabis, J., Finn, C., Gopalakr- ishnan, K., Hausman, K., Herzog, A., Hsu, J., et al.: Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817 (2022) 8, 9, 3, 4

    Pith/arXiv arXiv 2022

  11. [11]

    In: CVPR

    Burgert, R., Xu, Y., Xian, W., Pilarski, O., Clausen, P., He, M., Ma, L., Deng, Y., Li, L., Mousavi, M., et al.: Go-with-the-flow: Motion-controllable video diffusion models using real-time warped noise. In: CVPR. pp. 13–23 (2025) 3

    2025

  12. [12]

    In: CVPR

    Caesar, H., Bankiti, V., Lang, A.H., Vora, S., Liong, V.E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., Beijbom, O.: nuscenes: A multimodal dataset for autonomous driving. In: CVPR. pp. 11621–11631 (2020) 8, 9, 3, 4

    2020

  13. [13]

    In: CVPR

    Chen, H., Zhang, Y., Cun, X., Xia, M., Wang, X., Weng, C., Shan, Y.: Videocrafter2: Overcoming data limitations for high-quality video diffusion models. In: CVPR. pp. 7310–7320 (2024) 3, 8, 9, 4

    2024

  14. [14]

    arXiv:2501.12375 (2025) 5, 8, 9

    Chen, S., Guo, H., Zhu, S., Zhang, F., Huang, Z., Feng, J., Kang, B.: Video depth anything: Consistent depth estimation for super-long videos. arXiv:2501.12375 (2025) 5, 8, 9

    Pith/arXiv arXiv 2025

  15. [15]

    Chen, W., Zheng, W., Zheng, Y., Chen, L., Zhou, J., Lu, J., Duan, Y.: Gen- world: Towards detecting ai-generated real-world simulation videos (2025),https: //arxiv.org/abs/2506.109758

    Pith/arXiv arXiv 2025

  16. [16]

    In: ICLR (2023) 3, 8, 9, 4

    Chen, X., Wang, Y., Zhang, L., Zhuang, S., Ma, X., Yu, J., Wang, Y., Lin, D., Qiao, Y., Liu, Z.: Seine: Short-to-long video diffusion model for generative transition and prediction. In: ICLR (2023) 3, 8, 9, 4

    2023

  17. [17]

    In: CVPR

    Ge, S., Mahapatra, A., Parmar, G., Zhu, J.Y., Huang, J.B.: On the content bias in fréchet video distance. In: CVPR. pp. 7277–7288 (2024) 3

    2024

  18. [18]

    In: ICCV

    Ge, S., Nah, S., Liu, G., Poon, T., Tao, A., Catanzaro, B., Jacobs, D., Huang, J.B., Liu, M.Y., Balaji, Y.: Preserve your own correlation: A noise prior for video diffusion models. In: ICCV. pp. 22930–22941 (2023) 3 SGC 17

    2023

  19. [19]

    arXiv preprint arXiv:2209.14792 (2022) 3

    Singer, U., Polyak, A., Hayes, T., Yin, X., An, J., Zhang, S., Hu, Q., Yang, H., Ashual, O., Gafni, O., et al.: Make-a-video: Text-to-video generation without text- video data. arXiv preprint arXiv:2209.14792 (2022) 3

    Pith/arXiv arXiv 2022

  20. [20]

    In: CVPR

    Skorokhodov, I., Menapace, W., Siarohin, A., Tulyakov, S.: Hierarchical patch diffusion models for high-resolution video generation. In: CVPR. pp. 7569–7579 (2024) 3

    2024

  21. [21]

    arXiv preprint arXiv:1812.01717 (2018) 1, 2, 3, 8, 9, 10

    Unterthiner, T., Van Steenkiste, S., Kurach, K., Marinier, R., Michalski, M., Gelly, S.: Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717 (2018) 1, 2, 3, 8, 9, 10

    Pith/arXiv arXiv 2018

  22. [22]

    arXiv preprint arXiv:2210.02399 (2022) 3

    Villegas, R., Babaeizadeh, M., Kindermans, P.J., Moraldo, H., Zhang, H., Saffar, M.T., Castro, S., Kunze, J., Erhan, D.: Phenaki: Variable length video generation from open domain textual description. arXiv preprint arXiv:2210.02399 (2022) 3

    Pith/arXiv arXiv 2022

  23. [23]

    In: CVPR

    Wang, H., Ma, C.Y., Liu, Y.C., Hou, J., Xu, T., Wang, J., Juefei-Xu, F., Luo, Y., Zhang, P., Hou, T., et al.: Lingen: Towards high-resolution minute-length text-to- video generation with linear computational complexity. In: CVPR. pp. 2578–2588 (2025) 3

    2025

  24. [24]

    In: CVPR (2025) 5, 8, 9

    Wang, J., Chen, M., Karaev, N., Vedaldi, A., Rupprecht, C., Novotny, D.: Vggt: Visual geometry grounded transformer. In: CVPR (2025) 5, 8, 9

    2025

  25. [25]

    arXiv preprint arXiv:2308.06571 (2023) 3, 8, 9, 4

    Wang, J., Yuan, H., Chen, D., Zhang, Y., Wang, X., Zhang, S.: Modelscope text- to-video technical report. arXiv preprint arXiv:2308.06571 (2023) 3, 8, 9, 4

    Pith/arXiv arXiv 2023

  26. [26]

    IJCV133(5), 3059–3078 (2025) 3, 8, 9, 4

    Wang, Y., Chen, X., Ma, X., Zhou, S., Huang, Z., Wang, Y., Yang, C., He, Y., Yu, J., Yang, P., et al.: Lavie: High-quality video generation with cascaded latent diffusion models. IJCV133(5), 3059–3078 (2025) 3, 8, 9, 4

    2025

  27. [27]

    arXiv preprint arXiv:2210.04628 (2022) 2

    Watson, D., Chan, W., Martin-Brualla, R., Ho, J., Tagliasacchi, A., Norouzi, M.: Novel view synthesis with diffusion models. arXiv preprint arXiv:2210.04628 (2022) 2

    Pith/arXiv arXiv 2022

  28. [28]

    In: ECCV

    Xing,J.,Xia,M.,Zhang,Y.,Chen,H.,Yu,W.,Liu,H.,Liu,G.,Wang,X.,Shan,Y., Wong, T.T.: Dynamicrafter: Animating open-domain images with video diffusion priors. In: ECCV. pp. 399–417. Springer (2024) 3

    2024

  29. [29]

    In: ICCV

    Yu, J.J., Forghani, F., Derpanis, K.G., Brubaker, M.A.: Long-term photometric consistent novel view synthesis with diffusion models. In: ICCV. pp. 7094–7104 (2023) 2, 3

    2023

  30. [30]

    Yuan, X., Baek, J., Xu, K., Tov, O., Fei, H.: Inflation with diffusion: Efficient temporal adaptation for text-to-video super-resolution. pp. 489–496 (2024) 3

    2024

  31. [31]

    In: CVPR

    Zeng, Y., Wei, G., Zheng, J., Zou, J., Wei, Y., Zhang, Y., Li, H.: Make pixels dance: High-dynamic video generation. In: CVPR. pp. 8850–8860 (2024) 3

    2024

  32. [32]

    arXiv preprint arXiv:2401.01827 (2024) 3

    Zhang, D.J., Li, D., Le, H., Shou, M.Z., Xiong, C., Sahoo, D.: Moonshot: To- wards controllable video generation and editing with multimodal conditions. arXiv preprint arXiv:2401.01827 (2024) 3

    Pith/arXiv arXiv 2024

  33. [33]

    arXiv preprint arXiv:2311.04145 (2023) 3

    Zhang, S., Wang, J., Zhang, Y., Zhao, K., Yuan, H., Qin, Z., Wang, X., Zhao, D., Zhou, J.: I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models. arXiv preprint arXiv:2311.04145 (2023) 3

    Pith/arXiv arXiv 2023

  34. [34]

    arXiv preprint arXiv:2305.13077 (2023) 3

    Zhang, Y., Wei, Y., Jiang, D., Zhang, X., Zuo, W., Tian, Q.: Con- trolvideo: Training-free controllable text-to-video generation. arXiv preprint arXiv:2305.13077 (2023) 3

    Pith/arXiv arXiv 2023

  35. [35]

    Zheng, Z., Peng, X., Yang, T., Shen, C., Li, S., Liu, H., Zhou, Y., Li, T., You, Y.: Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:2412.20404 (2024) 3, 8, 9, 4 SGC 1 Appendix A More Methodology and Algorithmic Details A.1 Method Overview The complete SGC evaluation pipeline is detailed in Algorithms 1 and 2. Ini- tially...

    Pith/arXiv arXiv 2024