REVIEW 4 major objections 5 minor 1 cited by
FreeStreamGS, an online feed-forward 3D Gaussian splatting system, reconstructs scenes from unposed streaming images in one pass per frame, matching the rendering quality of offline methods.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-02 12:27 UTC pith:BQ2L25QL
load-bearing objection A solid incremental step in online feed-forward 3DGS, but the 'unposed streaming' claim quietly assumes fixed intrinsics and skips the closest baselines. the 4 major comments →
OF³GS: On-the-Fly Feed-Forward 3D Gaussian Splatting from Unposed Images
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that the two failure modes of causal 3DGS—intrinsic drift and rigid unprojection—are separable and fixable. DIR-Head predicts a normalized focal length from the first frame's features, then scales by image width, so all frames share one intrinsic matrix; this removes cumulative scale drift. DPR-Offsets predicts a per-pixel 3D residual added to the unprojected Gaussian center, so primitives can detach from the viewing ray and snap to the accumulated geometry. Under a training scheme that supervises novel views more heavily and a recursive voxel-based fusion that merges new Gaussians into a cached global state, the paper reports PSNR/SSIM/LPIPS that beat the online optimiz
What carries the argument
Decoupled Intrinsic Recovery Head (DIR-Head): a lightweight MLP that reads only the first frame's feature and outputs a normalized focal length, scaled by image width to form the full intrinsic matrix; it fixes the projection scale for the whole stream. Dynamic Point Refinement Offsets (DPR-Offsets): a per-pixel 3D residual map predicted by the Gaussian decoder, added to the depth-unprojected center of each Gaussian to compensate for coupled pose–depth drift. Online Recursive Gaussian Fusion: a voxel cache with confidence-weighted running averages that merges new primitives into the global representation without revisiting past frames.
Load-bearing premise
The stream is captured with fixed camera intrinsics that can be read from the first frame, and the frozen pretrained teacher provides reliable camera and depth priors for whatever footage appears.
What would settle it
Feed the system a video shot with a zoom lens whose focal length changes and check whether novel views drift in scale or develop ghosting; the fixed-intrinsic first-frame anchor cannot represent zoom.
If this is right
- One forward pass per frame can replace iterative per-frame optimization for online NVS, roughly halving latency while improving rendering quality.
- Dense input streams (tens of frames) become tractable because recursive fusion keeps memory constant, whereas global-context offline methods run out of memory.
- First-frame anchoring lets the system process raw handheld video from a fixed-lens camera without calibration.
- Novel-view-weighted supervision can be adopted by other feed-forward 3DGS systems to avoid input-view overfitting.
Where Pith is reading between the lines
- The same decoupling idea—anchoring static camera intrinsics to the first frame—could transfer to other streaming geometry tasks like online SLAM or point-cloud fusion, wherever recursive bias accumulates.
- DPR-Offsets illustrates a general pattern: relaxing hard geometric constraints with learned residuals is a cheap way to absorb estimator drift; a similar mechanism might improve online mesh or neural radiance field reconstruction.
- A stress test worth running is footage with a zoom lens or a camera swap mid-stream; the fixed-intrinsic assumption would break and likely cause scale drift, suggesting that re-anchoring logic would be needed for such inputs.
- Because training relies on a frozen pretrained geometry teacher, the system inherits that teacher's blind spots (e.g., transparent surfaces or unusual scenes), so the practical ceiling is partly the teacher's quality.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FreeStreamGS, a feed-forward framework for online 3D Gaussian Splatting (3DGS) reconstruction from unposed streaming image sequences. The method processes images causally, maintaining a full-history cached feature state. To stabilize geometry, it introduces a Decoupled Intrinsic Recovery Head (DIR-Head) that anchors the focal length to the first frame, and Dynamic Point Refinement Offsets (DPR-Offsets) that relax the rigid per-pixel unprojection by adding a predicted 3D residual. The framework trains with teacher-forced intrinsic distillation from a frozen VGGT model, scale-shift invariant depth supervision from Depth Anything V3, and a novel-view-weighted photometric loss. Experiments on DL3DV-140, RE10K, and NYUv2 report competitive novel-view synthesis quality versus offline feed-forward methods and versus the online optimization-based baseline OnTheFly-NVS, with a per-frame latency of 250 ms.
Significance. If the claims are substantiated, this is a useful step toward low-latency online 3DGS, addressing a realistic streaming scenario. The method is practical and achieves reasonable rendering quality at 5 view inputs, with a clear architectural rationale. However, the evaluation omits the most relevant online feed-forward baselines because their code is unavailable, and the reported quality degrades substantially with more input views. The fixed-intrinsic assumption is not acknowledged as a limitation despite being load-bearing for the 'unposed streaming' claim. These issues materially affect the strength of the central claims and require revision before the paper can be accepted.
major comments (4)
- [Sec. 5.1, footnote 4 and Table 1] The paper's abstract claims that FreeStreamGS 'outperforms online baselines,' but the only online baseline is OnTheFly-NVS, which is optimization-based. The closest category—online feed-forward methods (StreamGS, SaLon3R, StreamSplat)—is excluded because code is unavailable. Without a comparison to at least one such method, the central claim is not established. Please either report published numbers from these methods on the same benchmarks, reproduce them if possible, or restrict the claim to 'outperforms the online optimization-based baseline OnTheFly-NVS' and explicitly discuss the missing comparisons as a limitation.
- [Table 1 and Sec. 5.2] Performance degrades sharply as the number of input views increases: DL3DV PSNR drops from 21.884 (5 views) to 19.952 (10) to 17.190 (64), and RE10K drops from 25.797 to 24.536 to 21.562. At 64 views on DL3DV, FreeStreamGS is worse than the optimization-based OnTheFly-NVS (17.190 vs. 18.478). This contradicts the claim in Supplement E that the method 'scales to longer input sequences' and produces 'stable, high-fidelity renderings' at dense inputs. An online method should benefit from more observations; the observed degradation raises concerns about the recursive fusion mechanism's ability to integrate long histories. Please explain this behavior and provide evidence for long-sequence operation, or temper the claims accordingly.
- [Sec. 4.1, Eqs. (3)-(4), and Limitations] The method assumes fixed camera intrinsics: the focal length is predicted from the first frame feature F0 and never updated. The paper mentions this in Sec. 4.1 ('intrinsics are physically static properties of the lens') but does not list it in the Limitations section, and the title/abstract claim 'unposed streaming inputs' without qualification. Any zoom or crop after frame 0 would change the true focal length, and DPR-Offsets, being per-pixel 3D residuals, cannot repair a global focal-length-dependent scale error. Either demonstrate robustness to varying focal lengths (e.g., synthetic or real zoom sequences) or explicitly state the fixed-intrinsic scope as a limitation of the method.
- [Sec. 4.1 and Table 4] The DIR-Head is trained by distillation from a frozen VGGT teacher, and the student's focal length is not quantitatively evaluated against the teacher or ground truth. Since ablation shows DIR-Head is the single most important component, the reader cannot judge whether the student is accurate enough for real deployment. Please report intrinsic prediction error (e.g., relative focal-length error) and, if possible, compare with the teacher's predictions on held-out data.
minor comments (5)
- [Title/Abstract] The full text is titled 'FreeStreamGS' but the review package lists the title as 'OF^3GS: On-the-Fly Feed-Forward 3D Gaussian Splatting from Unposed Images.' Please ensure the final manuscript has a consistent title across all versions.
- [Sec. 4.2] The phrase 'DPR-Offsetshandlethefine-grainedspatialcorrections' has a missing space; minor typographical errors throughout the text should be corrected.
- [Figure 3] The diagram is dense and some labels (e.g., 'Pretrained Model') are ambiguous. Clarifying which components are frozen and which are trainable would improve reproducibility.
- [Sec. 5.2, Table 1] No error bars or repeated-run statistics are provided. Given the small differences in some comparisons (e.g., RE10K 5-view PSNR 25.797 vs. WorldMirror 25.419), reporting standard deviations would help assess significance.
- [Supplement B, Eq. (B.1)] The least-squares alignment is described only in the supplement; the main text says 'detailed in Suppl.' but a one-sentence summary of the alignment would make the main paper more self-contained.
Circularity Check
No significant circularity; the derivation is self-contained and evaluated against external benchmarks.
full rationale
FreeStreamGS does not define any predicted quantity in terms of itself. The intrinsic focal length is regressed from the first-frame feature F0 and trained by L2 distillation against the frozen VGGT teacher's focal estimate (Eqs. 3-5); this is amortized imitation of an external pretrained model, not a self-referential fit. The DPR-Offsets are learned residuals supervised by photometric, geometric, and novel-view losses (Eqs. 6-11), and the fused Gaussian representation is evaluated on independent DL3DV, RE10K, and NYUv2 benchmarks against external baselines. The frozen StreamVGGT/VGGT components are disclosed and constitute independent pretrained support rather than self-citation; no authors overlap with those cited models. The fixed-intrinsics assumption is a scope restriction that limits generality, but it is not a circular step because the method never claims to estimate time-varying intrinsics in the tested regime. No equation or fitted parameter is renamed as a prediction, and no uniqueness claim is imported from the authors' own prior work. Therefore the central claim, while bounded by teacher accuracy and the fixed-intrinsics assumption, is not circular.
Axiom & Free-Parameter Ledger
free parameters (2)
- Focal length from DIR-Head (normalized ratio scaled by W) =
learned (distilled from VGGT)
- Loss weights λdepth=1.0, λint=10.0, λnovel=1.5, λ1=λ2=0.05 =
1.0 / 10.0 / 1.5 / 0.05
axioms (5)
- domain assumption Fixed intrinsics over the sequence: square pixels, centered principal point, constant focal length inferred from first frame.
- domain assumption Frozen StreamVGGT provides reliable causal features and relative camera extrinsics.
- domain assumption VGGT teacher focal length is a stable and accurate reference for distillation.
- domain assumption Depth Anything V3 pseudo-depth provides useful geometric signal after scale-shift alignment.
- domain assumption The scene is static; dynamic objects are not explicitly modeled.
Cite this review
Pith. "Pith review of OF$^3$GS: On-the-Fly Feed-Forward 3D Gaussian Splatting from Unposed Images." pith.science (2026). https://pith.science/paper/BQ2L25QL
@misc{pith2026260603254,
author = {Pith},
title = {Pith review of: OF$^3$GS: On-the-Fly Feed-Forward 3D Gaussian Splatting from Unposed Images},
year = {2026},
howpublished = {\url{https://pith.science/paper/BQ2L25QL}},
note = {Machine review of arXiv:2606.03254}
}
read the original abstract
Feed-forward 3D Gaussian Splatting (3DGS) enables efficient and high-fidelity novel view synthesis (NVS) from offline image sequences. However, achieving on-the-fly NVS from unposed images remains challenging: the system must reconstruct renderable 3D Gaussians as images arrive, without access to future observations. Although online feed-forward geometry methods have been developed for causal depth and point-cloud recovery, directly adapting them to NVS often leads to severe rendering artifacts because Gaussian-based rendering demands stricter multi-view consistency in primitive scale and pose-geometry alignment. Even minor deviations can accumulate under causal inference and visibly degrade rendering quality. To this end, we propose OF$^3$GS, a feed-forward framework for efficient and high-quality on-the-fly NVS from sparse-view unposed images under causal constraints. We introduce two mechanisms for causal geometric stability: a Decoupled Intrinsic Recovery Head that mitigates cumulative camera-intrinsic bias and scene-scale jitter, and Dynamic Point Refinement Offsets that relax rigid unprojection to compensate for coupled pose-depth drift. Extensive experiments show that OF$^3$GS outperforms online baselines and approaches offline feed-forward 3DGS methods under comparable sparse-input settings. It also remains memory-feasible with denser inputs. Homepage: https://richardchen225.github.io/of3gs/
Figures
Forward citations
Cited by 1 Pith paper
-
StreamSplat: Streaming Feed-Forward 3D Gaussian Splatting
StreamSplat incrementally builds a 3D Gaussian scene from a causal view stream using a memory-bounded voxel cache, and scales to hundreds of views where fixed-context baselines run out of memory.
Reference graph
Works this paper leans on
-
[1]
Charatan, D., Li, S.L., Tagliasacchi, A., Sitzmann, V.: Pixelsplat: 3d gaussian splatsfromimagepairsforscalablegeneralizable3dreconstruction.In:Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 19457–19467 (2024)
2024
-
[2]
In: European Conference on Computer Vision
Chen, Y., Xu, H., Zheng, C., Zhuang, B., Pollefeys, M., Geiger, A., Cham, T.J., Cai, J.: Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images. In: European Conference on Computer Vision. pp. 370–386. Springer (2024)
2024
-
[3]
arXiv preprint arXiv:2510.15072 (2025)
Guo, J., Guan, T., Dong, W., Zheng, W., Wang, W., Wang, Y., Yam, Y., Liu, Y.H.: Salon3r: Structure-aware long-term generalizable 3d reconstruction from unposed images. arXiv preprint arXiv:2510.15072 (2025)
arXiv 2025
-
[4]
arXiv preprint arXiv:2410.22128 (2024)
Hong, S., Jung, J., Shin, H., Han, J., Yang, J., Luo, C., Kim, S.: Pf3plat: Pose-free feed-forward 3d gaussian splatting. arXiv preprint arXiv:2410.22128 (2024)
Pith/arXiv arXiv 2024
-
[5]
arXiv preprint arXiv:2507.16144 (2025) 22 R
Huang, G., Wang, R., Gao, X., Sun, C., Wu, Y., Gao, S., Jia, Y.: Longsplat: On- line generalizable 3d gaussian splatting from long sequence images. arXiv preprint arXiv:2507.16144 (2025) 22 R. Chen et al
Pith/arXiv arXiv 2025
-
[6]
ACM Transactions on Graphics (TOG)44(6), 1–16 (2025)
Jiang, L., Mao, Y., Xu, L., Lu, T., Ren, K., Jin, Y., Xu, X., Yu, M., Pang, J., Zhao, F., et al.: Anysplat: Feed-forward 3d gaussian splatting from unconstrained views. ACM Transactions on Graphics (TOG)44(6), 1–16 (2025)
2025
-
[7]
In: ACM SIGGRAPH 2024 conference papers
Jiang, Y., Yu, C., Xie, T., Li, X., Feng, Y., Wang, H., Li, M., Lau, H., Gao, F., Yang, Y., et al.: Vr-gs: A physical dynamics-aware interactive gaussian splatting system in virtual reality. In: ACM SIGGRAPH 2024 conference papers. pp. 1–1 (2024)
2024
-
[8]
In: Proceedings of the Computer Vision and Pattern Recognition Conference
Keetha, N., Karhade, J., Jatavallabhula, K.M., Yang, G., Scherer, S., Ramanan, D., Luiten, J.: Splatam: Splat track & map 3d gaussians for dense rgb-d slam. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 21357–21366 (2024)
2024
-
[9]
In: International Conference on Learning Representations (2026)
Lan,Y.,Luo,Y.,Hong,F.,Zhou,S.,Chen,H.,Lyu,Z.,Yang,S.,Dai,B.,Loy,C.C., Pan, X.: STream3R: Scalable sequential 3D reconstruction with causal transformer. In: International Conference on Learning Representations (2026)
2026
-
[10]
arXiv preprint arXiv:2510.08551 (2025)
Li, G., Ren, K., Xu, L., Zheng, Z., Jiang, C., Gao, X., Dai, B., Pu, J., Yu, M., Pang, J.: Artdeco: Towards efficient and high-fidelity on-the-fly 3d reconstruction with structured scene representation. arXiv preprint arXiv:2510.08551 (2025)
arXiv 2025
-
[11]
arXiv preprint arXiv:2503.06235 (2025)
Li, Y., Wang, J., Chu, L., Li, X., Kao, S.h., Chen, Y.C., Lu, Y.: Streamgs: Online generalizable gaussian splatting reconstruction for unposed image streams. arXiv preprint arXiv:2503.06235 (2025)
Pith/arXiv arXiv 2025
-
[12]
arXiv preprint arXiv:2503.10286 (2025)
Li, Z., Dong, C., Chen, Y., Huang, Z., Liu, P.: Vicasplat: A single run is all you need for 3d gaussian splatting and camera estimation from unposed video frames. arXiv preprint arXiv:2503.10286 (2025)
Pith/arXiv arXiv 2025
-
[13]
In: Proceedings of the Computer Vision and Pattern Recognition Conference
Lin, C.Y., Sun, C., Yang, F.E., Chen, M.H., Lin, Y.Y., Liu, Y.L.: Longsplat: Ro- bust unposed 3d gaussian splatting for casual long videos. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 27412–27422 (2025)
2025
-
[14]
arXiv preprint arXiv:2511.10647 (2025)
Lin, H., Chen, S., Liew, J.H., Chen, D.Y., Li, Z., Shi, G., Feng, J., Kang, B.: Depth anything 3: Recovering the visual space from any views. arXiv preprint arXiv:2511.10647 (2025)
Pith/arXiv arXiv 2025
-
[15]
In: Proceedings of the Computer Vision and Pattern Recognition Conference
Ling, L., Sheng, Y., Tu, Z., Zhao, W., Xin, C., Wan, K., Yu, L., Guo, Q., Yu, Z., Lu, Y., et al.: Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 22160–22169 (2024)
2024
-
[16]
Liu, Y., Min, Z., Wang, Z., Wu, J., Wang, T., Yuan, Y., Luo, Y., Guo, C.: World- mirror:Universal3dworldreconstructionwithany-priorprompting.arXivpreprint arXiv:2510.10726 (2025)
arXiv 2025
-
[17]
arXiv preprint arXiv:1711.05101 (2017)
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)
Pith/arXiv arXiv 2017
-
[18]
ACM Transactions on Graphics (TOG)44(4) (2025)
Meuleman, A., Shah, I., Lanvin, A., Kerbl, B., Drettakis, G.: On-the-fly reconstruc- tion for large-scale novel view synthesis from unposed images. ACM Transactions on Graphics (TOG)44(4) (2025)
2025
-
[19]
In: European Conference on Computer Vision (2012)
Nathan Silberman, Derek Hoiem, P.K., Fergus, R.: Indoor segmentation and sup- port inference from rgbd images. In: European Conference on Computer Vision (2012)
2012
-
[20]
In: Proceedings of the Computer Vision and Pattern Recognition Conference
Ranftl, R., Bochkovskiy, A., Koltun, V.: Vision transformers for dense prediction. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 12179–12188 (2021)
2021
-
[21]
Analytical chemistry36(8), 1627–1639 (1964) FreeStreamGS 23
Savitzky, A., Golay, M.J.: Smoothing and differentiation of data by simplified least squares procedures. Analytical chemistry36(8), 1627–1639 (1964) FreeStreamGS 23
1964
-
[22]
In: Proceedings of the Computer Vision and Pattern Recognition Conference
Wang, J., Chen, M., Karaev, N., Vedaldi, A., Rupprecht, C., Novotny, D.: Vggt: Visual geometry grounded transformer. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 5294–5306 (2025)
2025
-
[23]
Advances in Neural Infor- mation Processing Systems37, 107326–107349 (2024)
Wang, Y., Huang, T., Chen, H., Lee, G.H.: Freesplat: Generalizable 3d gaussian splatting towards free view synthesis of indoor scenes. Advances in Neural Infor- mation Processing Systems37, 107326–107349 (2024)
2024
-
[24]
arXiv preprint arXiv:2506.08862 (2025)
Wu, Z., Yan, Q., Yi, X., Wang, L., Liao, R.: Streamsplat: Towards online dynamic 3d reconstruction from uncalibrated video streams. arXiv preprint arXiv:2506.08862 (2025)
arXiv 2025
-
[25]
In: Proceedings of the Computer Vision and Pattern Recognition Conference
Xie, T., Zong, Z., Qiu, Y., Li, X., Feng, Y., Yang, Y., Jiang, C.: Physgaussian: Physics-integrated 3d gaussians for generative dynamics. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 4389–4398 (2024)
2024
-
[26]
In: Proceedings of the Computer Vision and Pattern Recognition Conference
Xu, H., Peng, S., Wang, F., Blum, H., Barath, D., Geiger, A., Pollefeys, M.: Depth- splat: Connecting gaussian splatting and depth. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 16453–16463 (2025)
2025
-
[27]
IEEE Robotics and Automation Letters11(1), 426–433 (2025)
Xu, Y., Yu, Y., Gan, W., Wang, T., Zhan, Z., Cheng, H., Wang, X.: Gaussian on-the-fly splatting: A progressive framework for robust near real-time 3dgs opti- mization. IEEE Robotics and Automation Letters11(1), 426–433 (2025)
2025
-
[28]
In: Proceedings of the Computer Vision and Pattern Recognition Conference
Yan, C., Qu, D., Xu, D., Zhao, B., Wang, Z., Wang, D., Li, X.: Gs-slam: Dense visual slam with 3d gaussian splatting. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 19595–19604 (2024)
2024
-
[29]
arXiv preprint arXiv:2410.24207 (2024)
Ye, B., Liu, S., Xu, H., Li, X., Pollefeys, M., Yang, M.H., Peng, S.: No pose, no problem: Surprisingly simple 3d gaussian splats from sparse unposed images. arXiv preprint arXiv:2410.24207 (2024)
Pith/arXiv arXiv 2024
-
[30]
In: Pro- ceedings of the Special Interest Group on Computer Graphics and Interactive Tech- niques Conference Conference Papers
You, Z., Georgoulis, S., Chen, A., Tang, S., Dai, D.: Gavs: 3d-grounded video stabilization via temporally-consistent local reconstruction and rendering. In: Pro- ceedings of the Special Interest Group on Computer Graphics and Interactive Tech- niques Conference Conference Papers. pp. 1–12 (2025)
2025
-
[31]
In: Proceedings of the Computer Vision and Pattern Recognition Conference
Zhang, S., Wang, J., Xu, Y., Xue, N., Rupprecht, C., Zhou, X., Shen, Y., Wet- zstein, G.: Flare: Feed-forward geometry, appearance and camera estimation from uncalibrated sparse views. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 21936–21947 (2025)
2025
-
[32]
In: Proceedings of the Computer Vision and Pattern Recognition Confer- ence
Zheng, S., Zhou, B., Shao, R., Liu, B., Zhang, S., Nie, L., Liu, Y.: Gps-gaussian: Generalizable pixel-wise 3d gaussian splatting for real-time human novel view syn- thesis. In: Proceedings of the Computer Vision and Pattern Recognition Confer- ence. pp. 19680–19690 (2024)
2024
-
[33]
Zhou, T., Tucker, R., Flynn, J., Fyffe, G., Snavely, N.: Stereo magnification: Learn- ingviewsynthesisusingmultiplaneimages.arXivpreprintarXiv:1805.09817(2018)
Pith/arXiv arXiv 2018
-
[34]
arXiv preprint arXiv:2507.11539 (2025) 24 R
Zhuo, D., Zheng, W., Guo, J., Wu, Y., Zhou, J., Lu, J.: Streaming 4d visual geometry transformer. arXiv preprint arXiv:2507.11539 (2025) 24 R. Chen et al. GT Ours OnTheFly- NVS [18] World Mirror [16] AnySplat [6] FLARE [31] Fig. E.5:Qualitative comparison on DL3DV-140 [15] datasets under 10 input views. FreeStreamGS 25 GT Ours OnTheFly- NVS [18] World Mir...
Pith/arXiv arXiv 2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.