REVIEW 3 major objections 5 minor
InfiniSplat claims that single-image 3D view synthesis improves when Gaussians are anchored to depth-derived surface patches rather than fixed pixel centers, with a shared implicit decoder predicting their attributes.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
2026-08-04 07:18 UTC pith:K2HAVUAX
load-bearing objection A solid empirical step for single-image feed-forward 3DGS, with the geometric mechanism less proven than the headlines suggest. the 3 major comments →
InfiniSplat: Implicit Gaussian Decoding for Large-Baseline Monocular View Synthesis
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
InfiniSplat claims that pixel-aligned Gaussian generation — primitives tied to fixed image-grid positions — is the key bottleneck in single-image view synthesis. It proposes a surface-aligned representation: geometry-guided sampling places Gaussian supports by area-weighting local back-projected depth triangles, and a shared implicit decoder predicts bounded attribute updates at those supports. Decoupling placement and prediction from the pixel grid lets Gaussians assemble into coherent surfaces, reducing cracks, holes, and drift under large viewpoint changes, in one feed-forward pass. The paper supports this with cross-dataset metrics, ablations, and normal-map visualizations.
What carries the argument
The mechanism is the pairing of geometry-guided support sampling with query-conditioned implicit decoding. Sampling back-projects a monocular depth map, discards triangles spanning depth discontinuities, and allocates support points proportional to 3D triangle area, so Gaussians concentrate on slanted or extended surfaces. The decoder is a shared function of support coordinates: it bilinearly samples semantic and texture features, fuses them with a gate, and predicts bounded residual updates to position, scale, rotation, color, and opacity on a base scaffold. Each stage requires the other: irregular supports demand a query-based decoder, and the decoder lacks surface awareness without the ge
Load-bearing premise
The paper's central premise is that the frozen monocular depth model's predicted depth is accurate enough that geometry-guided sampling places supports on true scene surfaces; the paper's own limitation section acknowledges that on reflective, transparent, thin, and textureless regions this scaffold fails, so the surface-aligned advantage degrades there.
What would settle it
Use a benchmark with ground-truth depth and compare the full model against the same model whose supports are sampled uniformly at random at equal density, keeping the decoder identical: if the geometry-guided supports do not beat random supports on large-baseline metrics, surface alignment is not the driver of the reported gains. A complementary check: corrupt the depth input (inverted or heavily blurred) at inference and measure whether large-baseline coherence degrades smoothly; the paper's claim predicts graceful decline on ordinary scenes but severe failure where depth is wrong.
If this is right
- If the surface-aligned representation holds, single-image 3DGS can generate renderable scenes that remain stable at substantially larger camera baselines than nearby-view splatting.
- Because the decoder is query-conditioned over arbitrary supports, the same trained framework can accept different support densities and arrangements, including sparse depth prompts, without retraining the core decoder.
- Area-weighted sampling naturally allocates more Gaussians to slanted or geometrically demanding surface patches, showing that adaptive resource allocation is compatible with feed-forward prediction.
- Cross-dataset zero-shot generalization from synthetic indoor scenes to real-world captures becomes achievable in a single forward pass, supporting real-time interactive browsing and AR/VR viewing.
- LiDAR-conditioned variants show that more reliable geometric scaffolding further improves the representation, indicating the framework's gains are tied to the geometric scaffold rather than a specific depth source.
Where Pith is reading between the lines
- A natural extension is uncertainty-aware support sampling: instead of treating monocular depth as a single scaffold, sample supports from a depth distribution, which could mitigate the claimed failure on reflective and textureless regions.
- The separation of 'where to place primitives' from 'what attributes to give them' may generalize beyond 3DGS to other primitive-based scene representations such as point clouds or surfels, and could be combined with diffusion-based priors as a post-refinement step.
- Because the decoder is a shared function of coordinates, the framework could be trained on varied camera intrinsics and resolutions by re-scaling query coordinates, potentially improving resolution robustness — a hypothesis the paper does not test.
- If the surface-alignment claim is correct, the benefit should be measurable not only in image metrics but in geometric metrics such as normal consistency or chamfer distance on the predicted Gaussian centers; the paper's normal-map visualizations support this, but a quantitative geometry metric would make it testable.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces InfiniSplat, a feed-forward single-image 3D Gaussian Splatting method. Instead of predicting Gaussians at fixed pixel-grid locations, InfiniSplat first samples 2D supports using depth-induced local surface areas (geometry-guided sampling), then predicts Gaussian attributes at these supports with a query-conditioned implicit decoder. The authors claim that this moves single-image 3DGS from pixel-aligned splat expansion to a surface-aligned representation, reducing cracks and improving large-baseline novel-view synthesis. They train on Hypersim and evaluate zero-shot on ETH3D, ScanNet++, Tanks-and-Temples, and DL3DV, reporting state-of-the-art PSNR/SSIM/LPIPS over feed-forward RGB-only baselines and a LiDAR-conditioned variant over ADGaussian. Ablations support the contribution of learned updates, dual-branch features, regularization, geometry-guided sampling, and implicit decoding, and a support-budget table shows diminishing returns beyond 1.5M supports.
Significance. If the empirical results hold, the paper makes a meaningful advance: shifting Gaussian placement away from the regular image lattice is a natural direction, and the cross-dataset zero-shot gains over SHARP and Flash3D are substantial (averaging +1.92 PSNR over SHARP, with larger gains on DL3DV). The paper has several strengths: the comparison controls the depth backend by including a Flash3D-DepthPro variant; the LiDAR-conditioned variant isolates whether gains depend on the depth source; the component ablations in Table 2 are systematic; Table 3 studies support budget; and Table 4 characterizes robustness to prompt-depth noise. The limitations section is candid about failures on reflective, transparent, thin, and textureless regions. The central mechanism, however, is not yet cleanly isolated from a simpler confound: continuous off-grid support placement alone may explain part or most of the benefit attributed to depth-guided surface alignment. The large-baseline claim also currently rests on a dataset-averaged table rather than per-baseline evidence. These issues are addressable and do not invalidate the empirical contribution, but they need to be resolved before the paper's sign
major comments (3)
- [Section 4.1 / Table 1] The paper's headline claim is robustness under large-baseline viewpoint changes, and Section 4.1 states that the 512 pairs per dataset are constructed by targeting 128 pairs in each of four baseline ranges [0,0.5), [0.5,1), [1,2), and [2,∞) meters. Yet Table 1 reports only dataset-level averages. Averages over these bins can be dominated by easy nearby views or by the distribution of actual baselines within each bin. To support the 'large-baseline' claim, please report per-baseline-bin PSNR/SSIM/LPIPS for at least InfiniSplat-RGB and SHARP (and ideally all baselines). If the advantage concentrates in the [0,0.5) bin, the surface-alignment story would be weakened; if it persists or grows in the [1,2) and [2,∞) bins, it would directly support the abstract.
- [Section 3.2 / Table 2 / Table 3] The 'w/o Geometry-guided Sampling' ablation in Table 2 replaces geometry-guided sampled supports with pixel-aligned supports while keeping the implicit decoder. This varies two factors at once: (i) support coordinates lose continuous, off-grid placement, and (ii) the depth-area allocation is removed. The observed drop (from 22.240 to 21.576 PSNR on ScanNet++) may therefore be caused entirely by the loss of coordinate flexibility rather than by geometric guidance. This is load-bearing for the central mechanistic claim that depth-induced area-weighted sampling creates surface-aligned Gaussians. The confound is compounded by Table 3: increasing the support budget from 1.0M to 1.5M yields +1.06 PSNR on ScanNet++, a larger gain than removing geometry-guided sampling. The authors should add a control with a depth-agnostic continuous sampler at the same support budget (e.g., stratified jittered
- [Section 3.2 / Section 5] The geometry-guided sampling pipeline relies on the quality of a frozen monocular depth scaffold, and Section 5 acknowledges that the scaffold fails on reflective, transparent, thin, and textureless regions. This is a structural limitation, not a flaw in the experiments, and the authors are transparent about it. However, the paper does not quantify how often such failures occur in the evaluation datasets or how much the reported averages are affected. Since the central claim is that surface alignment improves large-baseline rendering, a short analysis of failure-mode frequency (e.g., counting cases where the source depth is clearly wrong on planar regions, or reporting a subset of pairs with strong depth discontinuities) would help readers calibrate the strength of the claim. I view this as a needed clarification rather than a reason to reject.
minor comments (5)
- [Section 4.1] The evaluation protocol says pairs are selected with 'a fixed random seed' and 'a scene-level sampling cap', but the seed value and cap are not given. Please state these for reproducibility.
- [Section 4.2 / Table 1] The LPIPS improvement on Tanks-and-Temples is only 0.004 relative to SHARP (0.310 vs 0.306), whereas PSNR/SSIM gains are clearer. The text acknowledges this, but a brief explanation of why (e.g., scene scale, texture distribution) would be useful.
- [Table 3] The support-budget table reports inference and rendering times, but the paper does not state the hardware and software environment (GPU, PyTorch version, rasterizer). Please add this information so the runtimes are interpretable.
- [Section 3.3] In the equation for the fused descriptor h_i, the gating notation is clear, but the projection P('f_cnn_i') is not defined. Please state the projection layer (e.g., 1x1 conv or linear) and its output dimension.
- [Figures 3 and 4] The qualitative figures are compelling, but the color/normal color maps are not always accompanied by a legend or a description of the normalization. Adding a short caption note would make the normal visualizations easier to interpret.
Circularity Check
No significant circularity: the model is learned from rendered supervision on Hypersim and evaluated on held-out datasets, so the central claim is empirical rather than definitional.
full rationale
InfiniSplat's central claim is that geometry-guided support sampling plus query-conditioned implicit decoding produces surface-aligned Gaussians that improve large-baseline novel-view synthesis. This is not circular: Gaussian parameters are predicted by a trainable MLP, supervised by rendering losses on Hypersim target views, and evaluated zero-shot on ETH3D, ScanNet++, Tanks-and-Temples, and DL3DV. The geometry scaffold in Eq. (3), X(p) = Π^{-1}(p, D(p); K̂), is obtained from a frozen pretrained depth model (DepthPro for RGB, InfiniDepth-Metric for LiDAR), not from the evaluation target, and the decoder predicts bounded residual updates rather than reproducing a fitted quantity. The ablation 'w/o Geometry-guided Sampling' may conflate depth-guided placement with continuous off-grid coordinates, and the support-budget ablation shows coverage effects, but these are attribution or experimental-design concerns, not instances where a prediction reduces to its input by construction. The only notable self-citations (InfiniDepth for the implicit-decoder inspiration and the LiDAR variant's depth backbone) are not load-bearing for the RGB results, and no uniqueness theorem or prior ansatz is imported to force the method's choices. Section 5's acknowledged limitations about depth-prior failures weaken robustness but do not make the derivation tautological.
Axiom & Free-Parameter Ledger
free parameters (6)
- λ_xy, λ_z =
not reported
- λ_scale, λ_smooth =
0.1, 0.1
- ℓ_min, ℓ_max =
[-8, -3]
- γ =
10
- support budget =
1.5M
- training schedule =
100K steps, batch 8, LR 5e-5
axioms (4)
- domain assumption Predicted depth D and intrinsics K̂ from the frozen model are accurate enough to define surface structure
- domain assumption Local triangle areas from neighboring pixels approximate the local 3D surface area for sampling density
- domain assumption Training on Hypersim and supervising target-view rendering transfers to the evaluation datasets
- domain assumption DINOv3 and CNN features provide sufficient conditioning for Gaussian attributes
Cite this review
Pith. "Pith review of InfiniSplat: Implicit Gaussian Decoding for Large-Baseline Monocular View Synthesis." pith.science (2026). https://pith.science/paper/K2HAVUAX
@misc{pith2026260802437,
author = {Pith},
title = {Pith review of: InfiniSplat: Implicit Gaussian Decoding for Large-Baseline Monocular View Synthesis},
year = {2026},
howpublished = {\url{https://pith.science/paper/K2HAVUAX}},
note = {Machine review of arXiv:2608.02437}
}
read the original abstract
Single-image feed-forward 3D Gaussian Splatting (3DGS) aims to directly generate a renderable 3D scene representation from one input image, avoiding the cost of multi-view capture and per-scene optimization. However, existing methods are often constrained by a pixel-aligned representation, where Gaussians are predicted from fixed image-grid locations. Such pixel-aligned primitives can produce promising nearby-view renderings, but they remain weakly coupled to underlying scene surfaces and struggle to preserve coherent structures under large viewpoint shifts. We present InfiniSplat, a feed-forward single-image 3DGS framework that moves from a pixel-aligned representation toward a surface-aligned representation. InfiniSplat constructs this representation by first using geometry-guided sampling to place 2D supports according to depth-induced local surface structure, and then applying a query-conditioned implicit decoder to predict Gaussian attributes from the image features queried at these supports. By grounding support locations in geometry while decoupling Gaussian prediction from fixed pixel centers, InfiniSplat produces Gaussian layouts that better follow scene surfaces and reduce scattered primitives caused by grid discretization. Across multiple cross-dataset NVS evaluations, InfiniSplat achieves state-of-the-art performance compared with single-image feed-forward baselines, and demonstrates zero-shot generalization from Hypersim indoor synthetic training to complex open-world scenes. Project page: https://zju3dv.github.io/InfiniSplat.
Figures
This paper was first reviewed by deepseek-v4-flash on August 4, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.