{"id":"1d64df5f-2c3f-4910-9759-658d7127a5c0","arxiv_id":"2607.05243","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":10,"one_line_summary":"A feed-forward framework predicts unified 3D Gaussian representations of dynamic humans and static scenes from monocular video in a single forward pass.","lead":"GUSH3R reconstructs both moving people and static backgrounds from a single video in one pass, outputting 3D Gaussian primitives that can be rendered from new viewpoints. It could enable fast, photorealistic 3D video capture for AR, VR, and digital content creation without per-scene optimization.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"The 'competitive quality' claim rests on comparisons against author-constructed decomposition baselines; human-only PSNR (11–13 dB) and background-only PSNR (19.7 vs AnySplat's 24.0) suggest the claim is not yet well-substantiated across evaluation settings.","rationale":"The reader correctly identified the CONDITIONAL verdict and many of the same evaluation gaps (low human PSNR, background underperformance, author-constructed baselines). However, the reader's weakest_assumption focused on Human3R error propagation, which is acknowledged by the authors and is inherent to any approach building on a frozen foundation model. The more load-bearing concern is whether the 'competitive quality' claim is substantiated by the evaluation design itself: the baselines GUSH3R outperforms are author-constructed and potentially suboptimal (the GT oracle only reaching ~15 PSNR is a red flag), while GUSH3R underperforms existing methods on both background-only and human-only metrics. That said, the paper does make a legitimate contribution as a first attempt at this problem setting, the efficiency advantage is clear (1.70 vs 0.16 FPS), and the architecture is reasonable. The CONDITIONAL verdict is appropriate — the contribution is real but the 'competitive quality' claim needs stronger evidence. The concrete test (batch-mode scene decoder) would directly test the paper's own explanation for the background quality gap and clarify whether the limitation is fundamental or a streaming trade-off. I recommend UNCHANGED because the reader's verdict already captures the right level of caution, even though the specific weakest_assumption could be sharpened.","tokens_in":15820,"tokens_out":2878,"duration_ms":397647,"concrete_test":"Run the Scene Gaussian Decoder in non-streaming (batch) mode on the NeuMan and EMDB benchmarks — processing all input frames jointly rather than frame-by-frame — and recompute the background-only PSNR/SSIM/LPIPS in Table 6. If the gap to AnySplat (currently 19.7 vs 24.0 PSNR on NeuMan 4-view) closes substantially (>3 dB improvement), the streaming architecture is confirmed as the cause and the 'competitive' claim holds for the streaming setting. If the gap persists, the quality limitation is in the decoder design itself, and the 'competitive' claim weakens.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of 'competitive novel view synthesis quality' is load-bearing on the evaluation being fair and the baselines being meaningful. Three issues weaken this: (1) The primary baselines (AnySplat+LHM+Human3R, AnySplat+LHM+GT) are author-constructed decomposition pipelines, not established methods. The oracle variant with GT SMPL-X still only achieves 14.6–15.9 PSNR on full scenes (Table 2), which is suspiciously low for an oracle — suggesting the composition procedure itself is suboptimal rather than GUSH3R being strong. (2) Human-only PSNR is 11–13 dB (Tables 3, 4), which is far below what 'photorealistic' implies, yet the paper claims photorealistic rendering. (3) On background-only evaluation (Table 6), GUSH3R significantly underperforms existing feed-forward methods: 19.7 vs 24.0 PSNR for AnySplat on NeuMan 4-view. The paper attributes this to a 'batch-streaming trade-off,' but does not test this hypothesis. If the streaming architecture is the cause, running the scene decoder in batch mode should close the gap; if it doesn't, the quality limitation is architectural rather than a streaming trade-off. The reader's weakest_assumption (Human3R error propagation) is a valid concern but is acknowledged by the authors and is inherent to the approach; the more pressing issue is whether the 'competitive' claim is substantiated by the evidence presented.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"This paper presents GUSH3R, a feed-forward framework for reconstructing dynamic human-scene environments from monocular video as a unified 3D Gaussian Splatting (3DGS) representation. The method builds upon the frozen Human3R foundation model, which provides scene point maps, camera poses, and SMPL-X meshes. The authors introduce two new components trained on top of these frozen features: a Scene Gaussian Decoder that predicts per-pixel Gaussians from DPT and CNN features, and a Human Gaussian Transformer (HGT) that predicts per-vertex Gaussians using cross-attention with memory tokens for temporal consistency. Experiments on NeuMan, EMDB, and BEDLAM compare GUSH3R against optimization-based methods (HSR) and author-constructed decomposition baselines (AnySplat+LHM+Human3R), demonstrating improved efficiency (1.70 FPS vs. 0.16 FPS) and competitive novel view synthesis quality.","tokens_in":16070,"tokens_out":1367,"duration_ms":339998,"significance":"The paper addresses a genuine gap in the literature: no existing feed-forward method simultaneously handles dynamic humans and static scenes while producing photorealistic, renderable 3DGS representations from monocular video. The decomposition into scene and human decoders leveraging frozen foundation model priors is a reasonable architectural choice. The inclusion of appearance memory tokens for temporal human consistency is a notable design contribution. However, the significance of the 'competitive quality' claim is tempered by the evaluation gaps detailed below, particularly the reliance on author-constructed baselines and the low absolute PSNR values on human regions.","major_comments":[{"comment":"§4.2, Table 2: The primary decomposition-based baselines (AnySplat+LHM+Human3R, AnySplat+LHM+GT) are author-constructed pipelines, not established methods. The oracle variant using ground-truth SMPL-X (AnySplat+LHM+GT) achieves only 14.6–15.9 PSNR on full scenes, which is suspiciously low for an oracle and suggests the composition procedure itself is suboptimal. This raises the concern that the baseline's poor performance reflects implementation choices in the composition step rather than a fundamental limitation of the decomposition approach, thereby inflating GUSH3R's relative advantage. The authors should either justify the composition procedure more rigorously or temper the claim of superiority over decomposition-based approaches.","section":null},{"comment":"§4.2, Table 2 and §4.3, Table 3: The paper claims 'photorealistic rendering' (Abstract, §1), yet human-only PSNR is 11–13 dB (Table 4 ablation: 11.6–13.0 dB; Table 3: 13.5 dB). These values are far below what 'photorealistic' typically implies in the novel view synthesis literature. The paper should either substantiate the 'photorealistic' claim with stronger evidence (e.g., higher-fidelity comparisons, user studies) or qualify the language to 'photorealistic-capable representation' or similar, making clear that current quantitative quality is limited.","section":null},{"comment":"Appendix B.2, Table 6: On background-only evaluation, GUSH3R significantly underperforms existing feed-forward methods: 19.7 vs. 24.0 PSNR for AnySplat on NeuMan 4-view. The paper attributes this to a 'batch-streaming trade-off' but does not test this hypothesis. A straightforward falsifiable test would be to run the Scene Gaussian Decoder in batch mode (processing all frames simultaneously) to see if the gap closes. If it does not, the quality limitation is architectural rather than a streaming trade-off, and the current explanation is insufficient.","section":null}],"minor_comments":[{"comment":"Table 1: The 'Photo-reality' column labels existing methods (VGGT, CUT3R, Human3R) as '✗' while labeling GUSH3R as '✓'. Given the low human PSNR values (11–13 dB) noted above, the '✓' for GUSH3R may overstate the current rendering quality. Consider clarifying that this denotes 'renderable representation' rather than achieved photorealistic quality.","section":null},{"comment":"§3.5, Eq. (9): The scale regularization threshold τ is listed as a free parameter in the axiom ledger but its value is not specified in the main text or Appendix A.1. Please report the value used.","section":null},{"comment":"§3.3: The voxel size used for the voxelization scheme (Eq. 5) is mentioned as 'a fixed voxel size in real-world scale' but the actual value is not provided. Please specify this hyperparameter.","section":null},{"comment":"Table 2: AnySplat's FPS is reported as '(6.77)' in parentheses, while other methods' FPS are without parentheses. The convention is unclear—please clarify whether this denotes a different measurement condition.","section":null},{"comment":"§3.4: The identity association across frames using 'matching based on SMPL-X parameters' is mentioned but the specific matching metric or threshold is not described. A brief clarification would aid reproducibility.","section":null},{"comment":"Figure 2: The notation switches between F_t (image tokens before decoder) and F'_t (image tokens after decoder) in the text, but Figure 2 uses I'_t for the image tokens input to the decoders. Please harmonize the notation between text and figure.","section":null}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about the 'competitive quality' claim being load-bearing on fair baselines is well-founded. The author-constructed decomposition baselines are the weakest point: the oracle variant's poor performance (14.6–15.9 PSNR with GT SMPL-X) is difficult to interpret without knowing whether the composition procedure is optimized. The background-only comparison (Table 6) against established feed-forward methods like AnySplat is actually more informative and shows GUSH3R is not yet competitive on scene quality alone. The 'photorealistic' language in the abstract is not well-supported by 11–13 dB human PSNR. These are fixable issues (tempering claims, adding the batch-mode ablation, justifying the composition baseline) but they are load-bearing for the paper's central claims."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful and constructive review. The referee correctly identifies that GUSH3R addresses a genuine gap—no prior feed-forward method simultaneously handles dynamic humans and static scenes while producing renderable 3DGS representations from monocular video. We address each major comment below. In brief: (1) we will add further justification for the composition procedure and temper the superiority claim; (2) we will qualify the 'photorealistic' language throughout the manuscript; and (3) we will conduct the batch-mode experiment the referee proposes and report the result. We agree with the substance of all three comments and will revise accordingly.","responses":[{"response":"The referee raises a valid concern. We acknowledge that the decomposition-based baselines are author-constructed and that the oracle variant's PSNR is lower than one might expect given ground-truth SMPL-X. We offer two points of clarification, and will also revise the manuscript. First, the oracle variant uses ground-truth SMPL-X parameters and human masks, but the scene reconstruction (AnySplat) and human reconstruction (LHM) components still operate independently—each is a feed-forward model with its own error profile. The composition step aligns these independently reconstructed elements in a common coordinate frame via Umeyama alignment, but misalignment between separately reconstructed humans and scenes introduces artifacts that no amount of oracle-quality pose information can fully eliminate. This is precisely the limitation of the decomposition approach that motivates our unified framework. Second, we agree that the low oracle PSNR warrants more discussion. In the revision, we will add a more detailed justification of the composition procedure (including the alignment method and its failure modes) in Appendix A.2, and we will temper the claim of superiority over decomposition-based approaches to make clear that the comparison reflects the difficulty of post-hoc composition specifically, not a fundamental limitation of decomposition as a paradigm. We will also note explicitly in Table 2 that these baselines are author-constructed.","revision_made":"partial","referee_comment":"§4.2, Table 2: The primary decomposition-based baselines are author-constructed pipelines, not established methods. The oracle variant using ground-truth SMPL-X achieves only 14.6–15.9 PSNR, which is suspiciously low for an oracle and suggests the composition procedure itself is suboptimal, inflating GUSH3R's relative advantage."},{"response":"The referee is correct. Human-region PSNR values of 11–13 dB are below what the term 'photorealistic' conventionally implies in the novel view synthesis literature, where photorealistic results typically achieve 25+ dB PSNR. We used 'photorealistic' to distinguish our 3DGS-based renderable representation from non-renderable outputs (point clouds, meshes) of prior feed-forward methods, but the referee is right that the term overstates the current quantitative quality. In the revision, we will qualify the language throughout the manuscript. Specifically, we will replace 'photorealistic rendering' with more precise phrasing such as 'a renderable representation capable of novel view synthesis' in the Abstract and Introduction, and we will add an explicit note in the experimental section that current quantitative quality—particularly for human regions—remains limited and that the term refers to the representation format (3DGS with view-dependent appearance) rather than implying state-of-the-art rendering fidelity. We will retain 'photorealistic' only where it refers to the representation type (i.e., 3DGS as a photorealistic-capable representation) with appropriate qualification.","revision_made":"yes","referee_comment":"§4.2, Table 2 and §4.3, Table 3: The paper claims 'photorealistic rendering' (Abstract, §1), yet human-only PSNR is 11–13 dB, far below what 'photorealistic' typically implies in the NVS literature."},{"response":"This is a fair and actionable suggestion. We agree that the batch-streaming trade-off explanation should be empirically tested rather than merely asserted. We will conduct the experiment the referee proposes: running the Scene Gaussian Decoder in batch mode (processing all input frames simultaneously rather than frame-by-frame) and reporting background-only PSNR/SSIM/LPIPS on NeuMan. If the gap with AnySplat closes, this supports the streaming trade-off explanation. If it does not, we will acknowledge that the quality limitation is at least partly architectural—our Scene Gaussian Decoder is trained on top of frozen Human3R features and may not fully exploit multi-view consistency even in batch mode, unlike AnySplat which is designed specifically for batch multi-view scene reconstruction. We will report the result honestly regardless of outcome and revise the explanation in Appendix B.2 accordingly. We note that even in the batch experiment, our method retains the advantage of jointly modeling dynamic humans, which AnySplat does not support.","revision_made":"yes","referee_comment":"Appendix B.2, Table 6: On background-only evaluation, GUSH3R significantly underperforms existing feed-forward methods (19.7 vs. 24.0 PSNR for AnySplat on NeuMan 4-view). The paper attributes this to a 'batch-streaming trade-off' but does not test this hypothesis. A straightforward falsifiable test would be to run the Scene Gaussian Decoder in batch mode."}],"tokens_in":15582,"tokens_out":1151,"duration_ms":248107,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"GUSH3R tackles a real gap: no prior feed-forward method handles both dynamic humans and photorealistic 3DGS rendering from monocular video simultaneously. The architecture is a reasonable combination of Human3R priors, a DPT-based scene decoder, and a cross-attention human decoder with SMPL-X vertex anchoring and appearance memory tokens. The problem setting is genuinely new, and the streaming design with separate human/scene Gaussian branches is a clean architectural choice. The ablations are informative — cross-attention removal causes the largest human quality drop, and depth supervision helps scene geometry. The depth evaluation (Table 5) showing improvement over both Human3R and DepthAnything3 is a nice independent check on geometric accuracy. Credit is due for shipping a working end-to-end pipeline on a problem nobody else addresses in this formulation. The soft spots are real but proportional. The stress-test concern about baselines lands: the decomposition baselines (AnySplat+LHM+Human3R, AnySplat+LHM+GT) are author-constructed, and the oracle variant with GT SMPL-X still only hits 14.6–15.9 PSNR on full scenes, which suggests the composition procedure itself is suboptimal rather than GUSH3R being strong. Human-only PSNR of 11–13 dB is low and undercuts the 'photorealistic' framing — the paper should temper that claim. Background-only evaluation (Table 6) shows GUSH3R at 19.7 vs AnySplat's 24.0 PSNR on NeuMan 4-view, and the 'batch-streaming trade-off' explanation is plausible but untested; running the scene decoder in batch mode would confirm or refute it. The dependence on frozen Human3R is acknowledged in Appendix C but not quantified — a sensitivity analysis to upstream pose errors would strengthen the paper. The missing hyperparameters (τ, voxel size) are a minor reproducibility gap. These are fixable issues that don't undermine the core contribution. The paper is for researchers in feed-forward 3D reconstruction and human-scene modeling who need a starting point for dynamic photorealistic reconstruction. It deserves a serious referee who can push the authors to strengthen the evaluation — better baselines, tempered claims, and the batch-mode ablation.","headline":"First feed-forward method to jointly reconstruct dynamic humans and static scenes as 3DGS from monocular video; quality claims need stronger baselines.","tokens_in":16899,"tokens_out":550,"would_cite":true,"duration_ms":277388,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"One forward pass rebuilds people and rooms as 3D Gaussians","keywords":[],"falsifier":"If the SMPL-X body pose estimated by Human3R is substantially wrong for a given person, the human Gaussians anchored to those mesh vertices are placed in incorrect 3D locations, producing visible misalignment between the rendered human and the scene that no amount of appearance decoding can fix.","tokens_in":16007,"feed_emoji":"🎥","tokens_out":976,"duration_ms":140434,"temperature":0.7,"pith_summary":"GUSH3R is a feed-forward framework that takes a monocular video and, in a single forward pass per frame, reconstructs both moving humans and the static scene as a unified set of 3D Gaussian Splatting primitives suitable for photorealistic novel view synthesis. The system builds on a frozen human-scene foundation model called Human3R, which provides camera poses, scene point clouds, human detections, and SMPL-X body mesh parameters. Two new decoder modules sit on top of this foundation: a Scene Gaussian Decoder that uses DPT-style dense prediction to convert point clouds and image features into per-pixel scene Gaussians, and a Human Gaussian Decoder that uses a cross-attention transformer to transfer appearance onto SMPL-X mesh vertices, which are then placed in 3D via linear blend skinning. The two Gaussian sets are merged in a shared metric space and rendered with standard Gaussian splatting. The central claim is that this architecture achieves competitive novel view synthesis quality compared to both optimization-based methods and decomposition-based feed-forward baselines, while being roughly ten times faster than the latter and avoiding the composition artifacts that arise when humans and scenes are reconstructed separately and then stitched together.","feed_headline":"One forward pass rebuilds people and rooms as 3D Gaussians","feed_subtitle":"A frozen geometry model plus two lightweight decoders turns monocular video into photorealistic 4D scenes at 10x the speed of decomposition.","key_machinery":"Human3R foundation model (frozen), Scene Gaussian Decoder with DPT and voxelization, Human Gaussian Decoder with cross-attention Human Gaussian Transformer (HGT), memory tokens for per-person appearance persistence, linear blend skinning (LBS) for placing canonical-space human Gaussians into posed space","core_discovery":"The paper demonstrates that a frozen geometric foundation model producing point clouds and parametric body meshes can be lifted into a unified, renderable 3D Gaussian representation by attaching two lightweight decoder branches, one for scene appearance and one for human appearance, without any per-scene optimization. The Human Gaussian Transformer is the key mechanism for human rendering: it uses cross-attention to transfer image appearance features onto canonical SMPL-X vertices, with memory tokens that persist per tracked person to maintain appearance consistency across frames and through occlusion. For scenes, the DPT decoder fuses foundation-model image tokens with CNN features to预测 per","pith_inferences":["If the frozen foundation model were replaced or jointly fine-tuned with the Gaussian decoders, systematic biases in pose estimation or scene geometry could potentially be corrected downstream, possibly narrowing the quality gap with optimization-based methods that currently achieve higher PSNR and SSIM.","The batch-streaming trade-off noted by the authors suggests that a hybrid architecture processing short temporal windows in batch mode while maintaining streaming state could improve static-scene consistency without sacrificing the dynamic-human handling that streaming provides.","The cross-attention transfer from image tokens to body-part tokens could in principle be inverted or regularized to produce semantic part segmentations as a byproduct, since the transformer must implicitly localize body parts to transfer correct appearance."],"forward_implications":["Feed-forward photorealistic 4D reconstruction from monocular video becomes practical at ~1.7 FPS, opening paths toward near-real-time AR/VR applications that currently require minutes to hours of per-scene optimization.","The architecture of freezing a geometric foundation model and training only lightweight Gaussian decoder branches suggests a general recipe: any feed-forward geometry predictor could be upgraded to photorealistic rendering by adding analogous decoder modules.","The appearance memory mechanism for maintaining person-specific appearance through occlusion could be extended to other non-rigid dynamic objects beyond humans, such as animals or articulated tools, if analogous parametric models exist.","The voxelization scheme for bounded memory growth in streaming reconstruction could inform design choices for long-duration or continuous-capture scenarios where unbounded Gaussian accumulation is a bottleneck."],"fun_headline_variants":["Frozen geometry model lifted to renderable 3D Gaussians via two decoders","Cross-attention maps persistent appearance onto SMPL-X for human rendering","DPT decoder fuses foundation tokens with CNN features for scene Gaussians","Memory tokens sustain per-person appearance across frames and occlusion","Feed-forward 3DGS from monocular video skips per-scene optimization"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The entire system depends on the frozen Human3R model producing accurate camera poses, scene point maps, human detections, and SMPL-X body parameters. Because the foundation model is not jointly fine-tuned with the Gaussian decoders, any systematic errors in its predictions propagate directly to the final reconstruction and cannot be corrected downstream.","fun_headline_variants_meta":{"raw":{"variants":["Frozen geometry model lifted to renderable 3D Gaussians via two decoders","Cross-attention maps persistent appearance onto SMPL-X for human rendering","DPT decoder fuses foundation tokens with CNN features for scene Gaussians","Memory tokens sustain per-person appearance across frames and occlusion","Feed-forward 3DGS from monocular video skips per-scene optimization"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":632,"prompt_tokens":538,"completion_tokens":94,"prompt_tokens_details":null},"tokens_in":538,"tokens_out":94,"duration_ms":11738,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-07T21:54:53.557647+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If the SMPL-X body pose estimated by Human3R is substantially wrong for a given person, the human Gaussians anchored to those mesh vertices are placed in incorrect 3D locations, producing visible misalignment between the rendered human and the scene that no amount of appearance decoding can fix.","supporting_citations":[],"review_version":1}