The reviewed record of science sign in
Pith

arxiv: 2607.05243 · v1 · pith:32JYSGA4 · submitted 2026-07-06 · cs.CV

GUSH3R: Everyone Everywhere All at Once as Gaussians

Keito Abe , Kaede Shiohara , Takashi Otonari , Toshihiko Yamasaki This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-07-07 21:54 UTCglm-5.2pith:32JYSGA4record.jsonopen to challenge →

classification cs.CV
keywords dynamichuman-scenemonoculareveryoneeverywherefeed-forwardgaussiansgeometry
0
0 comments X

The pith

One forward pass rebuilds people and rooms as 3D Gaussians

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

GUSH3R is a feed-forward framework that takes a monocular video and, in a single forward pass per frame, reconstructs both moving humans and the static scene as a unified set of 3D Gaussian Splatting primitives suitable for photorealistic novel view synthesis. The system builds on a frozen human-scene foundation model called Human3R, which provides camera poses, scene point clouds, human detections, and SMPL-X body mesh parameters. Two new decoder modules sit on top of this foundation: a Scene Gaussian Decoder that uses DPT-style dense prediction to convert point clouds and image features into per-pixel scene Gaussians, and a Human Gaussian Decoder that uses a cross-attention transformer to transfer appearance onto SMPL-X mesh vertices, which are then placed in 3D via linear blend skinning. The two Gaussian sets are merged in a shared metric space and rendered with standard Gaussian splatting. The central claim is that this architecture achieves competitive novel view synthesis quality compared to both optimization-based methods and decomposition-based feed-forward baselines, while being roughly ten times faster than the latter and avoiding the composition artifacts that arise when humans and scenes are reconstructed separately and then stitched together.

Core claim

The paper demonstrates that a frozen geometric foundation model producing point clouds and parametric body meshes can be lifted into a unified, renderable 3D Gaussian representation by attaching two lightweight decoder branches, one for scene appearance and one for human appearance, without any per-scene optimization. The Human Gaussian Transformer is the key mechanism for human rendering: it uses cross-attention to transfer image appearance features onto canonical SMPL-X vertices, with memory tokens that persist per tracked person to maintain appearance consistency across frames and through occlusion. For scenes, the DPT decoder fuses foundation-model image tokens with CNN features to预测 per

What carries the argument

Human3R foundation model (frozen), Scene Gaussian Decoder with DPT and voxelization, Human Gaussian Decoder with cross-attention Human Gaussian Transformer (HGT), memory tokens for per-person appearance persistence, linear blend skinning (LBS) for placing canonical-space human Gaussians into posed space

If this is right

  • Feed-forward photorealistic 4D reconstruction from monocular video becomes practical at ~1.7 FPS, opening paths toward near-real-time AR/VR applications that currently require minutes to hours of per-scene optimization.
  • The architecture of freezing a geometric foundation model and training only lightweight Gaussian decoder branches suggests a general recipe: any feed-forward geometry predictor could be upgraded to photorealistic rendering by adding analogous decoder modules.
  • The appearance memory mechanism for maintaining person-specific appearance through occlusion could be extended to other non-rigid dynamic objects beyond humans, such as animals or articulated tools, if analogous parametric models exist.
  • The voxelization scheme for bounded memory growth in streaming reconstruction could inform design choices for long-duration or continuous-capture scenarios where unbounded Gaussian accumulation is a bottleneck.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the frozen foundation model were replaced or jointly fine-tuned with the Gaussian decoders, systematic biases in pose estimation or scene geometry could potentially be corrected downstream, possibly narrowing the quality gap with optimization-based methods that currently achieve higher PSNR and SSIM.
  • The batch-streaming trade-off noted by the authors suggests that a hybrid architecture processing short temporal windows in batch mode while maintaining streaming state could improve static-scene consistency without sacrificing the dynamic-human handling that streaming provides.
  • The cross-attention transfer from image tokens to body-part tokens could in principle be inverted or regularized to produce semantic part segmentations as a byproduct, since the transformer must implicitly localize body parts to transfer correct appearance.

Load-bearing premise

The entire system depends on the frozen Human3R model producing accurate camera poses, scene point maps, human detections, and SMPL-X body parameters. Because the foundation model is not jointly fine-tuned with the Gaussian decoders, any systematic errors in its predictions propagate directly to the final reconstruction and cannot be corrected downstream.

What would settle it

If the SMPL-X body pose estimated by Human3R is substantially wrong for a given person, the human Gaussians anchored to those mesh vertices are placed in incorrect 3D locations, producing visible misalignment between the rendered human and the scene that no amount of appearance decoding can fix.

Figures

Figures reproduced from arXiv: 2607.05243 by Kaede Shiohara, Keito Abe, Takashi Otonari, Toshihiko Yamasaki.

Figure 1
Figure 1. Figure 1: GUSH3R (Gaussian-Unified Scene Human 3D Reconstruction) takes a monocular video as input and produces dynamic human-scene representations using 3D Gaussians. Abstract Reconstructing dynamic human-scene environments from monocular videos is a challenging problem that requires jointly modeling scene geometry, camera motion, and non-rigid human dynamics while enabling photorealistic rendering. Recent feed-for… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the proposed framework. GUSH3R reconstructs a dynamic human-scene representation from a monocular video using two newly introduced branches: the Scene Gaussian Decoder and the Human Gaussian Decoder. Each frame is processed by the foundation model Human3R [10] to extract human token H′ t and image token I ′ t along with scene point clouds Xt and human mesh vertices Vt. The Scene Gaussian Decode… view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative comparison on single-human scene reconstruction against the baseline using NeuMan [24]. The baseline refers to the decomposition-based baseline; a combination of AnySplat [23], LHM [47], and Human3R [10]. Although our method works in a streaming setting using only past frames, it achieves comparable reconstruction quality while providing faster inference. Method NeuMan [24] (4-view) NeuMan [24]… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison on multi-human scene reconstruction against the baseline us￾ing BEDLAM [5]. The baseline approach refers to the decomposition-based baseline; a combination of AnySplat [23], LHM [47], and Human3R [10]. Method Human-Scene Scene Human FPS PSNR↑ SSIM↑ LPIPS↓ PSNR↑ SSIM↑ LPIPS↓ PSNR↑ SSIM↑ LPIPS↓ ↑ AnySplat [23] 15.9 0.43 0.42 16.2 0.50 0.37 14.3 0.87 0.14 (6.77) AnySplat [23]+LHM [47]+H… view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative 4D human-scene reconstruction results. GUSH3R produces coherent dynamic human-scene reconstructions across diverse scenarios with different numbers of people, body poses, and camera viewpoints from a monocular video. Setting / Variant 4-view 8-view 16-view PSNR↑ SSIM↑ LPIPS↓ PSNR↑ SSIM↑ LPIPS↓ PSNR↑ SSIM↑ LPIPS↓ Scene Full model 19.7 0.60 0.26 17.8 0.49 0.37 17.4 0.47 0.39 w/o depth loss 19.3 0… view at source ↗
read the original abstract

Reconstructing dynamic human-scene environments from monocular videos is a challenging problem that requires jointly modeling scene geometry, camera motion, and non-rigid human dynamics while enabling photorealistic rendering. Recent feed-forward methods can efficiently predict geometry, but they are often limited to non-photorealistic representations such as point clouds and meshes, or they fail to handle non-rigid objects, particularly dynamic humans. To fill this gap, we present GUSH3R (Gaussian-Unified Scene Human 3D Reconstruction), a feed-forward framework for online dynamic human-scene reconstruction. From a monocular human-scene video, our method reconstructs dynamic humans (everyone) and static scenes (everywhere) in a single forward pass (all at once) as 3D Gaussian Splatting (3DGS) primitives (as gaussians), which are geometrically consistent and capable of novel view synthesis. Experiments on monocular human-scene datasets demonstrate that our approach achieves competitive novel view synthesis quality while significantly improving inference efficiency compared to optimization-based methods.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 6 minor

Summary. This paper presents GUSH3R, a feed-forward framework for reconstructing dynamic human-scene environments from monocular video as a unified 3D Gaussian Splatting (3DGS) representation. The method builds upon the frozen Human3R foundation model, which provides scene point maps, camera poses, and SMPL-X meshes. The authors introduce two new components trained on top of these frozen features: a Scene Gaussian Decoder that predicts per-pixel Gaussians from DPT and CNN features, and a Human Gaussian Transformer (HGT) that predicts per-vertex Gaussians using cross-attention with memory tokens for temporal consistency. Experiments on NeuMan, EMDB, and BEDLAM compare GUSH3R against optimization-based methods (HSR) and author-constructed decomposition baselines (AnySplat+LHM+Human3R), demonstrating improved efficiency (1.70 FPS vs. 0.16 FPS) and competitive novel view synthesis quality.

Significance. The paper addresses a genuine gap in the literature: no existing feed-forward method simultaneously handles dynamic humans and static scenes while producing photorealistic, renderable 3DGS representations from monocular video. The decomposition into scene and human decoders leveraging frozen foundation model priors is a reasonable architectural choice. The inclusion of appearance memory tokens for temporal human consistency is a notable design contribution. However, the significance of the 'competitive quality' claim is tempered by the evaluation gaps detailed below, particularly the reliance on author-constructed baselines and the low absolute PSNR values on human regions.

major comments (3)
  1. §4.2, Table 2: The primary decomposition-based baselines (AnySplat+LHM+Human3R, AnySplat+LHM+GT) are author-constructed pipelines, not established methods. The oracle variant using ground-truth SMPL-X (AnySplat+LHM+GT) achieves only 14.6–15.9 PSNR on full scenes, which is suspiciously low for an oracle and suggests the composition procedure itself is suboptimal. This raises the concern that the baseline's poor performance reflects implementation choices in the composition step rather than a fundamental limitation of the decomposition approach, thereby inflating GUSH3R's relative advantage. The authors should either justify the composition procedure more rigorously or temper the claim of superiority over decomposition-based approaches.
  2. §4.2, Table 2 and §4.3, Table 3: The paper claims 'photorealistic rendering' (Abstract, §1), yet human-only PSNR is 11–13 dB (Table 4 ablation: 11.6–13.0 dB; Table 3: 13.5 dB). These values are far below what 'photorealistic' typically implies in the novel view synthesis literature. The paper should either substantiate the 'photorealistic' claim with stronger evidence (e.g., higher-fidelity comparisons, user studies) or qualify the language to 'photorealistic-capable representation' or similar, making clear that current quantitative quality is limited.
  3. Appendix B.2, Table 6: On background-only evaluation, GUSH3R significantly underperforms existing feed-forward methods: 19.7 vs. 24.0 PSNR for AnySplat on NeuMan 4-view. The paper attributes this to a 'batch-streaming trade-off' but does not test this hypothesis. A straightforward falsifiable test would be to run the Scene Gaussian Decoder in batch mode (processing all frames simultaneously) to see if the gap closes. If it does not, the quality limitation is architectural rather than a streaming trade-off, and the current explanation is insufficient.
minor comments (6)
  1. Table 1: The 'Photo-reality' column labels existing methods (VGGT, CUT3R, Human3R) as '✗' while labeling GUSH3R as '✓'. Given the low human PSNR values (11–13 dB) noted above, the '✓' for GUSH3R may overstate the current rendering quality. Consider clarifying that this denotes 'renderable representation' rather than achieved photorealistic quality.
  2. §3.5, Eq. (9): The scale regularization threshold τ is listed as a free parameter in the axiom ledger but its value is not specified in the main text or Appendix A.1. Please report the value used.
  3. §3.3: The voxel size used for the voxelization scheme (Eq. 5) is mentioned as 'a fixed voxel size in real-world scale' but the actual value is not provided. Please specify this hyperparameter.
  4. Table 2: AnySplat's FPS is reported as '(6.77)' in parentheses, while other methods' FPS are without parentheses. The convention is unclear—please clarify whether this denotes a different measurement condition.
  5. §3.4: The identity association across frames using 'matching based on SMPL-X parameters' is mentioned but the specific matching metric or threshold is not described. A brief clarification would aid reproducibility.
  6. Figure 2: The notation switches between F_t (image tokens before decoder) and F'_t (image tokens after decoder) in the text, but Figure 2 uses I'_t for the image tokens input to the decoders. Please harmonize the notation between text and figure.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive review. The referee correctly identifies that GUSH3R addresses a genuine gap—no prior feed-forward method simultaneously handles dynamic humans and static scenes while producing renderable 3DGS representations from monocular video. We address each major comment below. In brief: (1) we will add further justification for the composition procedure and temper the superiority claim; (2) we will qualify the 'photorealistic' language throughout the manuscript; and (3) we will conduct the batch-mode experiment the referee proposes and report the result. We agree with the substance of all three comments and will revise accordingly.

read point-by-point responses

  1. Referee: §4.2, Table 2: The primary decomposition-based baselines are author-constructed pipelines, not established methods. The oracle variant using ground-truth SMPL-X achieves only 14.6–15.9 PSNR, which is suspiciously low for an oracle and suggests the composition procedure itself is suboptimal, inflating GUSH3R's relative advantage.

    Authors: The referee raises a valid concern. We acknowledge that the decomposition-based baselines are author-constructed and that the oracle variant's PSNR is lower than one might expect given ground-truth SMPL-X. We offer two points of clarification, and will also revise the manuscript. First, the oracle variant uses ground-truth SMPL-X parameters and human masks, but the scene reconstruction (AnySplat) and human reconstruction (LHM) components still operate independently—each is a feed-forward model with its own error profile. The composition step aligns these independently reconstructed elements in a common coordinate frame via Umeyama alignment, but misalignment between separately reconstructed humans and scenes introduces artifacts that no amount of oracle-quality pose information can fully eliminate. This is precisely the limitation of the decomposition approach that motivates our unified framework. Second, we agree that the low oracle PSNR warrants more discussion. In the revision, we will add a more detailed justification of the composition procedure (including the alignment method and its failure modes) in Appendix A.2, and we will temper the claim of superiority over decomposition-based approaches to make clear that the comparison reflects the difficulty of post-hoc composition specifically, not a fundamental limitation of decomposition as a paradigm. We will also note explicitly in Table 2 that these baselines are author-constructed. revision: partial

  2. Referee: §4.2, Table 2 and §4.3, Table 3: The paper claims 'photorealistic rendering' (Abstract, §1), yet human-only PSNR is 11–13 dB, far below what 'photorealistic' typically implies in the NVS literature.

    Authors: The referee is correct. Human-region PSNR values of 11–13 dB are below what the term 'photorealistic' conventionally implies in the novel view synthesis literature, where photorealistic results typically achieve 25+ dB PSNR. We used 'photorealistic' to distinguish our 3DGS-based renderable representation from non-renderable outputs (point clouds, meshes) of prior feed-forward methods, but the referee is right that the term overstates the current quantitative quality. In the revision, we will qualify the language throughout the manuscript. Specifically, we will replace 'photorealistic rendering' with more precise phrasing such as 'a renderable representation capable of novel view synthesis' in the Abstract and Introduction, and we will add an explicit note in the experimental section that current quantitative quality—particularly for human regions—remains limited and that the term refers to the representation format (3DGS with view-dependent appearance) rather than implying state-of-the-art rendering fidelity. We will retain 'photorealistic' only where it refers to the representation type (i.e., 3DGS as a photorealistic-capable representation) with appropriate qualification. revision: yes

  3. Referee: Appendix B.2, Table 6: On background-only evaluation, GUSH3R significantly underperforms existing feed-forward methods (19.7 vs. 24.0 PSNR for AnySplat on NeuMan 4-view). The paper attributes this to a 'batch-streaming trade-off' but does not test this hypothesis. A straightforward falsifiable test would be to run the Scene Gaussian Decoder in batch mode.

    Authors: This is a fair and actionable suggestion. We agree that the batch-streaming trade-off explanation should be empirically tested rather than merely asserted. We will conduct the experiment the referee proposes: running the Scene Gaussian Decoder in batch mode (processing all input frames simultaneously rather than frame-by-frame) and reporting background-only PSNR/SSIM/LPIPS on NeuMan. If the gap with AnySplat closes, this supports the streaming trade-off explanation. If it does not, we will acknowledge that the quality limitation is at least partly architectural—our Scene Gaussian Decoder is trained on top of frozen Human3R features and may not fully exploit multi-view consistency even in batch mode, unlike AnySplat which is designed specifically for batch multi-view scene reconstruction. We will report the result honestly regardless of outcome and revise the explanation in Appendix B.2 accordingly. We note that even in the batch experiment, our method retains the advantage of jointly modeling dynamic humans, which AnySplat does not support. revision: yes

Circularity Check

0 steps flagged

No significant circularity: the derivation chain is self-contained, with only a minor evaluation-protocol concern.

full rationale

The paper builds on Human3R [10], an external model with no author overlap. The Gaussian decoders (Scene and Human) are trained on external datasets (BEDLAM, DL3DV, Motion-X++) with standard losses (MSE, LPIPS, depth, silhouette BCE). No prediction reduces to a fitted input by construction. The Gaussian centers are initialized from Human3R point maps and SMPL-X vertices, but the remaining Gaussian parameters (opacity, rotation, scale, color) are predicted by trained MLPs from image tokens — these are not definitional re-derivations of the inputs. The one mild concern is that the decomposition baselines (AnySplat+LHM+Human3R) use Human3R-estimated SMPL-X parameters and masks, meaning the baseline and the proposed method share a common upstream dependency. However, this is an evaluation-protocol issue (baseline construction), not a circularity in the paper's own derivation chain. The paper's central claim — feed-forward photorealistic human-scene reconstruction as 3DGS — is supported by independently trained decoders evaluated against external benchmarks (NeuMan, EMDB, BEDLAM). No self-citation chain is load-bearing for the central result.

Axiom & Free-Parameter Ledger

10 free parameters · 5 axioms · 3 invented entities

The free parameters are mostly loss weights with stated values, plus two unstated hyperparameters (τ and voxel size). The axioms are reasonable domain assumptions, with the Human3R dependency being the most load-bearing. The invented entities are architectural components that are ablated, though the memory token ablation shows limited impact.

free parameters (10)
  • λ_mse (scene) = 1.0
    Loss weight for scene MSE loss, set by the authors.
  • λ_lpips (scene) = 0.2
    Loss weight for scene LPIPS loss.
  • λ_reg (scene) = 0.05
    Loss weight for scene Gaussian scale regularization.
  • λ_dep (scene) = 0.1
    Loss weight for scene depth supervision loss.
  • λ_mse (human) = 1.0
    Loss weight for human MSE loss.
  • λ_part (human) = 0.5
    Loss weight for partial LPIPS loss.
  • λ_reg (human) = 100.0
    Loss weight for human Gaussian scale regularization; notably 2000x larger than scene λ_reg.
  • λ_sil (human) = 1.0
    Loss weight for silhouette BCE loss.
  • τ (scale ratio threshold)
    Threshold for Gaussian scale regularization (Eq. 9); value not stated in the paper.
  • voxel size
    Fixed real-world scale voxel size for scene Gaussian aggregation (Sec. 3.3); value not stated.
axioms (5)
  • domain assumption Human3R [10] provides sufficiently accurate camera poses, scene point maps, human detections, and SMPL-X parameters for downstream Gaussian prediction.
    The entire pipeline depends on frozen Human3R outputs (Sec. 3.2). Acknowledged as a limitation in Appendix C.
  • domain assumption SMPL-X mesh vertices serve as valid geometric anchors for human Gaussians.
    The Human Gaussian Decoder attaches Gaussians to SMPL-X vertices (Sec. 3.4). If vertex positions are inaccurate, Gaussian placement is wrong.
  • standard math Linear blend skinning (LBS) correctly transforms canonical-space Gaussians to posed space.
    Standard SMPL/SMPL-X skinning assumption (Sec. 3.4). Well-established in the literature.
  • domain assumption 3D Gaussian Splatting differentiable rendering is suitable for feed-forward parameter prediction (not just per-scene optimization).
    Follows AnySplat [23] precedent; the paper predicts Gaussian parameters via MLPs rather than optimizing them (Sec. 3.3-3.4).
  • domain assumption Identity association across frames using SMPL-X parameter matching is reliable enough for memory token reuse.
    Sec. 3.4 states identities are associated using SMPL-X parameters. Acknowledged as challenging under occlusion in Appendix C.
invented entities (3)
  • Scene Gaussian Decoder independent evidence
    purpose: Predicts per-pixel 3DGS parameters from DPT-decoded image tokens and CNN features, using Human3R point clouds as geometric anchors.
    Ablated in Table 4 (w/o depth loss, w/o DL3DV); produces measurable changes in PSNR/SSIM/LPIPS.
  • Human Gaussian Transformer (HGT) independent evidence
    purpose: Cross-attention transformer that transfers image appearance features to canonical body space using human, vertex, and memory tokens as queries.
    Ablated in Table 4 (w/o cross-attention shows largest degradation: ~1.4 dB PSNR drop).
  • Appearance memory tokens independent evidence
    purpose: Per-person persistent tokens storing accumulated appearance features across frames for temporal consistency.
    Ablated in Table 4 (w/o memory tokens); shows minimal impact on PSNR/SSIM, raising questions about effectiveness.

pith-pipeline@v1.1.0-glm · 18796 in / 3623 out tokens · 714682 ms · 2026-07-07T21:54:53.557647+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

75 extracted references · 75 canonical work pages · 4 internal anchors

  1. [1]

    Agarwal, N

    S. Agarwal, N. Snavely, I. Simon, S. M. Seitz, and R. Szeliski. Building rome in a day. InICCV, 2009

    work page 2009

  2. [2]

    M. L. Antequera, P. Gargallo, M. Hofinger, S. R. Bulo, Y . Kuang, and P. Kontschieder. Mapillary planet-scale depth dataset. InECCV, 2020

    work page 2020

  3. [3]

    Bansal, M

    A. Bansal, M. V o, Y . Sheikh, D. Ramanan, and S. Narasimhan. 4d visualization of dynamic events from unconstrained multi-view videos. InCVPR, 2020

    work page 2020

  4. [4]

    Baradel, M

    F. Baradel, M. Armando, S. Galaaoui, R. Brégier, P. Weinzaepfel, G. Rogez, and T. Lucas. Multi-hmr: Multi-person whole-body human mesh recovery in a single shot. InECCV, 2024

    work page 2024

  5. [5]

    M. J. Black, P. Patel, J. Tesch, and J. Yang. BEDLAM: A synthetic dataset of bodies exhibiting detailed lifelike animated motion. InCVPR, 2023

    work page 2023

  6. [6]

    Virtual KITTI 2

    Y . Cabon, N. Murray, and M. Humenberger. Virtual kitti 2.arXiv preprint arXiv:2001.10773, 2020

    work page internal anchor Pith review Pith/arXiv arXiv 2001

  7. [7]

    Cao and J

    A. Cao and J. Johnson. Hexplane: A fast representation for dynamic scenes. InCVPR, 2023

    work page 2023

  8. [8]

    X. Chen, Y . Chen, Y . Xiu, A. Geiger, and A. Chen. Ttt3r: 3d reconstruction as test-time training. InICLR, 2026

    work page 2026

  9. [9]

    X. Chen, Z. Xiong, Y . Chen, G. Li, N. Wang, H. Luo, L. Chen, H. Sun, B. WANG, G. Chen, et al. Dggt: Feedforward 4d reconstruction of dynamic driving scenes using unposed images. arXiv preprint arXiv:2512.03004, 2025

    work page arXiv 2025

  10. [10]

    Y . Chen, X. Chen, Y . Xue, A. Chen, Y . Xiu, and P.-M. Gerard. Human3r: Everyone everywhere all at once. InICLR, 2026

    work page 2026

  11. [11]

    Y . Chen, Q. Wu, M. Harandi, and J. Cai. How far can we compress instant-ngp-based nerf? In CVPR, 2024

    work page 2024

  12. [12]

    A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nießner. Scannet: Richly- annotated 3d reconstructions of indoor scenes. InCVPR, 2017

    work page 2017

  13. [13]

    Deitke, D

    M. Deitke, D. Schwenk, J. Salvador, L. Weihs, O. Michel, E. VanderBilt, L. Schmidt, K. Ehsani, A. Kembhavi, and A. Farhadi. Objaverse: A universe of annotated 3d objects. InCVPR, 2023

    work page 2023

  14. [14]

    Dosovitskiy, L

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. InICLR, 2021

    work page 2021

  15. [15]

    B. P. Duisterhof, L. Zust, P. Weinzaepfel, V . Leroy, Y . Cabon, and J. Revaud. Mast3r-sfm: a fully-integrated solution for unconstrained structure-from-motion. In3DV, 2025

    work page 2025

  16. [16]

    Frahm, P

    J.-M. Frahm, P. Fite-Georgel, D. Gallup, T. Johnson, R. Raguram, C. Wu, Y .-H. Jen, E. Dunn, B. Clipp, S. Lazebnik, and M. Pollefeys. Building rome on a cloudless day. InECCV, 2010

    work page 2010

  17. [17]

    Fridovich-Keil, G

    S. Fridovich-Keil, G. Meanti, F. R. Warburg, B. Recht, and A. Kanazawa. K-planes: Explicit radiance fields in space, time, and appearance. InCVPR, 2023

    work page 2023

  18. [18]

    Galliani, K

    S. Galliani, K. Lasinger, and K. Schindler. Massively parallel multiview stereopsis by surface normal diffusion. InICCV, 2015. 10

    work page 2015

  19. [19]

    Greff, F

    K. Greff, F. Belletti, L. Beyer, C. Doersch, Y . Du, D. Duckworth, D. J. Fleet, D. Gnanapragasam, F. Golemo, C. Herrmann, T. Kipf, A. Kundu, D. Lagun, I. Laradji, H.-T. (Derek)Liu, H. Meyer, Y . Miao, D. Nowrouzezahrai, C. Oztireli, E. Pot, N. Radwan, D. Rebain, S. Sabour, M. S. M. Sajjadi, M. Sela, V . Sitzmann, A. Stone, D. Sun, S. V ora, Z. Wang, T. Wu...

    work page 2022

  20. [20]

    C. Guo, T. Jiang, X. Chen, J. Song, and O. Hilliges. Vid2avatar: 3d avatar reconstruction from videos in the wild via self-supervised scene decomposition. InCVPR, 2023

    work page 2023

  21. [21]

    Huang, K

    P.-H. Huang, K. Matzen, J. Kopf, N. Ahuja, and J.-B. Huang. Deepmvs: Learning multi-view stereopsis. InCVPR, 2018

    work page 2018

  22. [22]

    Hyung, S

    J. Hyung, S. Hong, S. Hwang, J. Lee, J. Choo, and J.-H. Kim. Effective rank analysis and regularization for enhanced 3d gaussian splatting. InNeurIPS, 2024

    work page 2024

  23. [23]

    Jiang, Y

    L. Jiang, Y . Mao, L. Xu, T. Lu, K. Ren, Y . Jin, X. Xu, M. Yu, J. Pang, F. Zhao, et al. Anysplat: Feed-forward 3d gaussian splatting from unconstrained views.TOG, 44(6):1–16, 2025

    work page 2025

  24. [24]

    Jiang, K

    W. Jiang, K. M. Yi, G. Samei, O. Tuzel, and A. Ranjan. Neuman: Neural human radiance field from a single video. InECCV, 2022

    work page 2022

  25. [25]

    Kaufmann, J

    M. Kaufmann, J. Song, C. Guo, K. Shen, T. Jiang, C. Tang, J. J. Zárate, and O. Hilliges. EMDB: The Electromagnetic Database of Global 3D Human Pose and Shape in the Wild. InICCV, 2023

    work page 2023

  26. [26]

    Kerbl, G

    B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis. 3d gaussian splatting for real-time radiance field rendering.TOG, 42(4):1–14, 2023

    work page 2023

  27. [27]

    Leroy, Y

    V . Leroy, Y . Cabon, and J. Revaud. Grounding image matching in 3d with mast3r. InECCV, 2024

    work page 2024

  28. [28]

    S. Li, H. Li, Y . Wang, Y . Liao, and L. Yu. Steernerf: Accelerating nerf rendering via smooth viewpoint trajectory. InCVPR, 2023

    work page 2023

  29. [29]

    T. Li, M. Slavcheva, M. Zollhoefer, S. Green, C. Lassner, C. Kim, T. Schmidt, S. Lovegrove, M. Goesele, R. Newcombe, and Z. Lv. Neural 3d video synthesis from multi-view video. In CVPR, 2022

    work page 2022

  30. [30]

    Li and N

    Z. Li and N. Snavely. Megadepth: Learning single-view depth prediction from internet photos. InCVPR, 2018

    work page 2018

  31. [31]

    Z. Li, R. Tucker, F. Cole, Q. Wang, L. Jin, V . Ye, A. Kanazawa, A. Holynski, and N. Snavely. Megasam: Accurate, fast and robust structure and motion from casual dynamic videos. In CVPR, 2025

    work page 2025

  32. [32]

    Liang, J

    H. Liang, J. Ren, A. Mirzaei, A. Torralba, Z. Liu, I. Gilitschenski, S. Fidler, C. Oztireli, H. Ling, Z. Gojcic, and J. Huang. Feed-forward bullet-time reconstruction of dynamic scenes from monocular videos. InNeurIPS, 2025

    work page 2025

  33. [33]

    H. Lin, S. Chen, J. H. Liew, D. Y . Chen, Z. Li, Y . Zhao, S. Peng, H. Guo, X. Zhou, G. Shi, J. Feng, and B. Kang. Depth anything 3: Recovering the visual space from any views. InICLR, 2026

    work page 2026

  34. [34]

    L. Ling, Y . Sheng, Z. Tu, W. Zhao, C. Xin, K. Wan, L. Yu, Q. Guo, Z. Yu, Y . Lu, X. Li, X. Sun, R. Ashok, A. Mukherjee, H. Kang, X. Kong, G. Hua, T. Zhang, B. Benes, and A. Bera. Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. InCVPR, 2024

    work page 2024

  35. [35]

    S. Liu, Y . Gao, T. Zhang, R. Pautrat, J. L. Schönberger, V . Larsson, and M. Pollefeys. Robust incremental structure-from-motion with hybrid features. InECCV, 2024

    work page 2024

  36. [36]

    X. Liu, C. R. Qi, and L. J. Guibas. Flownet3d: Learning scene flow in 3d point clouds. In CVPR, 2019

    work page 2019

  37. [37]

    Z. Liu, J. Lin, W. Wu, and B. Zhou. Joint optimization for 4d human-scene reconstruction in the wild. InICLR, 2026. 11

    work page 2026

  38. [38]

    Loper, N

    M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black. SMPL: A skinned multi-person linear model.TOG, 34(6):1–16, 2015

    work page 2015

  39. [39]

    Loshchilov and F

    I. Loshchilov and F. Hutter. Decoupled weight decay regularization. InICLR, 2019

    work page 2019

  40. [40]

    Luiten, G

    J. Luiten, G. Kopanas, B. Leibe, and D. Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. In3DV, 2024

    work page 2024

  41. [41]

    Mildenhall, P

    B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. InECCV, 2020

    work page 2020

  42. [42]

    Müller, H

    L. Müller, H. Choi, A. Zhang, B. Yi, J. Malik, and A. Kanazawa. Reconstructing people, places, and cameras. InCVPR, 2025

    work page 2025

  43. [43]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V . V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. HAZIZA, F. Massa, A. El-Nouby, M. Assran, N. Ballas, W. Galuba, R. Howes, P.-Y . Huang, S.-W. Li, I. Misra, M. Rabbat, V . Sharma, G. Synnaeve, H. Xu, H. Jegou, J. Mairal, P. Labatut, A. Joulin, and P. Bojanowski. DINOv2: Learning robust visual features without s...

    work page 2024

  44. [44]

    X. Pan, N. Charron, Y . Yang, S. Peters, T. Whelan, C. Kong, O. Parkhi, R. Newcombe, and C. Y . Ren. Aria digital twin: A new benchmark dataset for egocentric 3d machine perception. In ICCV, 2023

    work page 2023

  45. [45]

    Pavlakos, V

    G. Pavlakos, V . Choutas, N. Ghorbani, T. Bolkart, A. A. A. Osman, D. Tzionas, and M. J. Black. Expressive body capture: 3D hands, face, and body from a single image. InCVPR, 2019

    work page 2019

  46. [46]

    Pumarola, E

    A. Pumarola, E. Corona, G. Pons-Moll, and F. Moreno-Noguer. D-nerf: Neural radiance fields for dynamic scenes. InCVPR, 2021

    work page 2021

  47. [47]

    L. Qiu, X. Gu, P. Li, Q. Zuo, W. Shen, J. Zhang, K. Qiu, W. Yuan, G. Chen, Z. Dong, and L. Bo. Lhm: Large animatable human reconstruction model for single image to 3d in seconds. In ICCV, 2025

    work page 2025

  48. [48]

    Ranftl, A

    R. Ranftl, A. Bochkovskiy, and V . Koltun. Vision transformers for dense prediction. InICCV, 2021

    work page 2021

  49. [49]

    Reizenstein, R

    J. Reizenstein, R. Shapovalov, P. Henzler, L. Sbordone, P. Labatut, and D. Novotny. Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. In ICCV, 2021

    work page 2021

  50. [50]

    Roberts, J

    M. Roberts, J. Ramapuram, A. Ranjan, A. Kumar, M. A. Bautista, N. Paczan, R. Webb, and J. M. Susskind. Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding. InICCV, 2021

    work page 2021

  51. [51]

    Rojas, M

    S. Rojas, M. Armando, B. Ghanem, P. Weinzaepfel, V . Leroy, and G. Rogez. Hamst3r: Human- aware multi-view stereo 3d reconstruction. InICCV, 2025

    work page 2025

  52. [52]

    J. L. Schonberger and J.-M. Frahm. Structure-from-motion revisited. InCVPR, 2016

    work page 2016

  53. [53]

    J. L. Schönberger, E. Zheng, J.-M. Frahm, and M. Pollefeys. Pixelwise view selection for unstructured multi-view stereo. InECCV, 2016

    work page 2016

  54. [54]

    The Replica Dataset: A Digital Replica of Indoor Spaces

    J. Straub, T. Whelan, L. Ma, Y . Chen, E. Wijmans, S. Green, J. J. Engel, R. Mur-Artal, C. Ren, S. Verma, A. Clarkson, M. Yan, B. Budge, Y . Yan, X. Pan, J. Yon, Y . Zou, K. Leon, N. Carter, J. Briales, T. Gillingham, E. Mueggler, L. Pesqueira, M. Savva, D. Batra, H. M. Strasdat, R. D. Nardi, M. Goesele, S. Lovegrove, and R. Newcombe. The replica dataset:...

    work page internal anchor Pith review Pith/arXiv arXiv 1906

  55. [55]

    A. Szot, A. Clegg, E. Undersander, E. Wijmans, Y . Zhao, J. Turner, N. Maestre, M. Mukadam, D. Chaplot, O. Maksymets, A. Gokaslan, V . V ondrus, S. Dharur, F. Meier, W. Galuba, A. Chang, Z. Kira, V . Koltun, J. Malik, M. Savva, and D. Batra. Habitat 2.0: Training home assistants to rearrange their habitat. InNeurIPS, 2021. 12

    work page 2021

  56. [56]

    S. Umeyama. Least-squares estimation of transformation parameters between two point patterns. TPAMI, 13(4):376–380, 1991

    work page 1991

  57. [57]

    von Marcard, R

    T. von Marcard, R. Henschel, M. Black, B. Rosenhahn, and G. Pons-Moll. Recovering accurate 3d human pose in the wild using imus and a moving camera. InECCV, 2018

    work page 2018

  58. [58]

    J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny. Vggt: Visual geometry grounded transformer. InCVPR, 2025

    work page 2025

  59. [59]

    Q. Wang, V . Ye, H. Gao, W. Zeng, J. Austin, Z. Li, and A. Kanazawa. Shape of motion: 4d reconstruction from a single video. InICCV, 2025

    work page 2025

  60. [60]

    Q. Wang, Y . Zhang, A. Holynski, A. A. Efros, and A. Kanazawa. Continuous 3d perception model with persistent state. InCVPR, 2025

    work page 2025

  61. [61]

    S. Wang, V . Leroy, Y . Cabon, B. Chidlovskii, and J. Revaud. Dust3r: Geometric 3d vision made easy. InCVPR, 2024

    work page 2024

  62. [62]

    Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity.TIP, 13(4):600–612, 2004

    work page 2004

  63. [63]

    G. Wu, T. Yi, J. Fang, L. Xie, X. Zhang, W. Wei, W. Liu, Q. Tian, and X. Wang. 4d gaussian splatting for real-time dynamic scene rendering. InCVPR, 2024

    work page 2024

  64. [64]

    H. Xia, Y . Fu, S. Liu, and X. Wang. Rgbd objects in the wild: Scaling real-world 3d object learning from rgb-d videos.arXiv preprint arXiv:2401.12592, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  65. [65]

    L. Xue, C. Guo, C. Zheng, F. Wang, T. Jiang, H.-I. Ho, M. Kaufmann, J. Song, and O. Hilliges. Hsr: holistic 3d human-scene reconstruction from monocular videos. InECCV, 2024

    work page 2024

  66. [66]

    Y . Yao, Z. Luo, S. Li, J. Zhang, Y . Ren, L. Zhou, T. Fang, and L. Quan. Blendedmvs: A large-scale dataset for generalized multi-view stereo networks. InCVPR, 2020

    work page 2020

  67. [67]

    B. Ye, B. Chen, H. Xu, D. Barath, and M. Pollefeys. Yonosplat: You only need one model for feedforward 3d gaussian splatting. InICLR, 2026

    work page 2026

  68. [68]

    H. Zhai, X. Zhang, B. Zhao, H. Li, Y . He, Z. Cui, H. Bao, and G. Zhang. Splatloc: 3d gaussian splatting-based visual localization for augmented reality.TVCG, 31(5):3591–3601, 2025

    work page 2025

  69. [69]

    Zhang, C

    J. Zhang, C. Herrmann, J. Hur, V . Jampani, T. Darrell, F. Cole, D. Sun, and M.-H. Yang. Monst3r: A simple approach for estimating geometry in the presence of motion. InICLR, 2025

    work page 2025

  70. [70]

    Zhang, P

    R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features as a perceptual metric. InCVPR, 2018

    work page 2018

  71. [71]

    Motion-X++: A Large-Scale Multimodal 3D Whole-body Human Motion Dataset

    Y . Zhang, J. Lin, A. Zeng, G. Wu, S. Lu, Y . Fu, Y . Cai, R. Zhang, H. Wang, and L. Zhang. Motion-x++: A large-scale multimodal 3d whole-body human motion dataset.arXiv preprint arXiv:2501.05098, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  72. [72]

    Zhang, M

    Z. Zhang, M. Kaufmann, L. Xue, J. Song, and M. R. Oswald. Odhsr: Online dense 3d reconstruction of humans and scenes from monocular videos. InCVPR, 2025

    work page 2025

  73. [73]

    Zheng, A

    Y . Zheng, A. W. Harley, B. Shen, G. Wetzstein, and L. J. Guibas. Pointodyssey: A large-scale synthetic dataset for long-term point tracking. InICCV, 2023

    work page 2023

  74. [74]

    H. Zhou, J. Shao, L. Xu, D. Bai, W. Qiu, B. Liu, Y . Wang, A. Geiger, and Y . Liao. Hugs: Holistic urban 3d scene understanding via gaussian splatting. InCVPR, 2024

    work page 2024

  75. [75]

    D. Zhuo, W. Zheng, J. Guo, Y . Wu, J. Zhou, and J. Lu. Streaming 4d visual geometry transformer. InICLR, 2026. 13 A Implementation Details A.1 Training details We freeze the pretrained Human3R [10] backbone and train only the Gaussian prediction modules. For the Scene Gaussian Decoder, we train the DPT-style feature decoder, CNN feature decoder and Gaussi...

    work page 2026