{"id":"40b4ac41-aff7-4886-b31d-d4da921f2dfe","arxiv_id":"2501.09302","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A comparative study of three 3D Gaussian Splatting pipelines for virtual environments finds that novel view synthesis gives the highest user presence and maintains real-time frame rates.","lead":"This paper compares three ways to build virtual reality environments from 3D Gaussian Splatting: recreating a scene from photos, turning it into a textured mesh, and generating a 360-degree panorama. A small user test with eight people found the photo-based novel view synthesis approach felt most immersive and ran at about 70 frames per second on a high-end GPU.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SUS-based superiority of novel view synthesis over mesh reconstruction is not statistically established: differences are small (d≈0.35) and no significance test is reported.","rationale":"The central claim has two components: (1) 3DGS can render VEs in real time on XR hardware, and (2) among the three pipelines, novel view synthesis provides the best user presence. Component (1) is supported by a rough FPS observation, and while measurement details are thin, it is not the main point of the paper. Component (2) is the differentiator and the basis for the title's 'comparative study.' The only quantitative evidence for it is Table 1, which is underpowered and unreported in terms of statistical testing. The reader's weakest assumption focuses on implementation effort; that is a plausible confound, but it is not the strongest attack because the paper's own Discussion already concedes the other two approaches have intrinsic limitations (fixed viewpoint for panoramic, reconstruction incompleteness for mesh). The stronger issue is that even if all confounds were removed, the observed difference between the top two approaches is small relative to the variance. A reanalysis with proper tests could easily flip the ranking or render it inconclusive. Thus the condition on the paper should be that the authors provide participant-level data and demonstrate that the headline contrast is statistically robust. This does not change the reader's CONDITIONAL verdict; it sharpens the condition.","tokens_in":3559,"tokens_out":7156,"duration_ms":69067,"concrete_test":"Obtain the per-participant SUS scores underlying Table 1 and run a paired permutation test (or Wilcoxon signed-rank test) comparing Novel View Synthesis and Mesh Reconstruction on SUS Mean, with a two-sided α=0.05 and a 95% bootstrap confidence interval for the mean difference. If the CI includes zero or p>0.05, the claimed superiority of novel view synthesis over mesh reconstruction is not statistically supported and the conclusion should be softened to feasibility only.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central comparative claim — that the novel view synthesis approach achieves the highest SUS Count and Mean and thus provides the best user presence — is supported only by Table 1, which reports means and standard deviations for n=8 with no inferential statistics. The decisive comparison is Novel View Synthesis vs. Mesh Reconstruction: SUS Mean 4.9±1.2 vs 4.5±1.1 (Cohen's d ≈ 0.35), Q2 4.3±1.3 vs 4.0±1.4, Q6 4.0±2.0 vs 3.8±1.8. With eight participants, a paired t-test on such small gaps would almost certainly not reach conventional significance; the paper provides no p-values, confidence intervals, effect sizes, or correction for multiple comparisons across the six SUS questions and two aggregate metrics. Consequently, the observed ordering could be sampling noise. This is more fundamental than the reader-identified implementation-effort confound: even with perfectly equal tuning, the reported evidence cannot distinguish the two leading approaches. The 70 FPS feasibility claim is less vulnerable, but the 'best presence' conclusion is not established by the data as presented.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents a comparative study of three 3D Gaussian Splatting (3DGS) pipelines for creating virtual environments for extended reality: a novel-view-synthesis approach, a mesh reconstruction approach (SuGaR with Poisson surface reconstruction), and a panoramic 3D scene approach. The authors render the resulting environments in Unity, display them on a Meta Quest 3 HMD, and report qualitative impressions, rendering speed (approximately 70 FPS for all approaches), and a user study with 8 participants using the Slater-Usoh-Steed (SUS) questionnaire. Based on Table 1, they claim that the novel view synthesis approach achieves the highest presence scores and conclude that 3DGS is feasible for real-time XR virtual environments, while also discussing limitations and future directions.","tokens_in":3757,"tokens_out":3592,"duration_ms":38758,"significance":"If the comparative claims were statistically and experimentally well supported, this would be a useful contribution to the XR and 3DGS communities, since practical evaluations of 3DGS in head-mounted displays are scarce. The paper has concrete strengths: it considers three distinct reconstruction/rendering pipelines, uses an actual HMD rather than only a desktop display, reports a standard presence questionnaire, and includes qualitative examples of artifacts. However, the evidence is currently preliminary. The headline claim that novel view synthesis provides the best user presence rests on a small single-scene user study with no inferential statistics, and the feasibility claim rests on a one-line FPS report without measurement details. The manuscript is therefore more convincing as an experience report than as a rigorous comparative evaluation.","major_comments":[{"comment":"The central claim that \"the novel view synthesis approach achieves the highest SUS Count and Mean metrics\" is not statistically supported by the reported data. Table 1 gives means and standard deviations for n=8, but no p-values, confidence intervals, effect sizes, or inferential tests are provided. The decisive comparison is Novel View Synthesis versus Mesh Reconstruction: SUS Mean 4.9±1.2 versus 4.5±1.1 (Cohen's d ≈ 0.35), Q2 4.3±1.3 versus 4.0±1.4, and Q6 4.0±2.0 versus 3.8±1.8. With eight participants, differences of this magnitude are likely to be within sampling error, and no correction is applied for the multiple SUS questions and aggregate metrics. The paper should either report appropriate paired significance tests (e.g., Wilcoxon signed-rank test with a stated correction), present confidence intervals, or explicitly soften the conclusion to a preliminary observation. As written, the observed ordering between the two leading approaches could be noise.","section":"Section 3, Table 1"},{"comment":"The feasibility claim \"all approaches achieve approximately 70 FPS\" is reported without the information needed to assess it. The authors do not state how FPS was measured (e.g., in-HMD measurement, screen capture, or Unity Profiler), how many runs were averaged, whether the 70 FPS figure is a mean, median, or peak value, or whether it varied across the approaches and during user movement. Since real-time XR feasibility is one of the two main claims, this needs a measurement protocol and per-approach results with variance. Additionally, the evaluation uses only a single scene, so the generality of both the FPS and the user-study findings is unclear.","section":"Section 3"},{"comment":"The comparison suffers from a potential implementation-effort confound. Each pipeline is described in only a few sentences with no hyperparameters, training times, numbers of Gaussians, mesh resolutions, or failure cases. The user study is therefore comparing not just the three conceptual approaches but also the authors' proficiency with each pipeline. For example, if the panoramic pipeline was under-tuned relative to the novel-view pipeline, then the large SUS differences (SUS Mean 1.9±0.9 versus 4.9±1.2) would be an artifact of implementation quality rather than an intrinsic property of the approach. The authors should provide more implementation details or acknowledge that the comparison is demonstration-oriented rather than a controlled benchmark.","section":"Section 2"},{"comment":"The statement that \"3DGS-based mesh reconstruction could hardly compete with traditional TSDF methods\" is made without presenting the results or protocol of the \"additional tests\" that support it. This is a comparative claim and should either be backed by data (even a small quantitative comparison) or be moved to the limitations/future-work paragraph without an unverifiable comparison.","section":"Section 4"}],"minor_comments":[{"comment":"The phrase \"conducting a comparable study\" should be \"conducting a comparative study\", and \"system evaluations\" in Section 5 should be clarified because the only quantitative system evaluation reported is the FPS statement.","section":"Introduction"},{"comment":"The text has typographical spacing errors: \"theSlater\" and \"Slater , Usoh\" should be \"the Slater\" and \"Slater, Usoh\". Also, the SUS acronym is used for the Slater-Usoh-Steed questionnaire rather than the more common System Usability Scale; the authors should explicitly define the scoring of \"SUS Count\" and \"SUS Mean\" in the text, since these are not standard from the acronym alone.","section":"Section 3"},{"comment":"Table 1 would benefit from a clearer layout: the column header \"Virtual environments created using the approaches of\" is awkward, and the table should state which questions Q1 through Q6 correspond to, since the SUS questionnaire items are not included in the paper.","section":"Table 1"},{"comment":"Reference [7] is a self-citation to the authors' position paper; this is not a problem for circularity, but the phrase \"straightforward scene manipulation [7]\" cites a position paper instead of a technical paper demonstrating manipulability, which would be a stronger support.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is close to a workshop or short-paper feasibility report. The main obstacle at a journal level is the gap between the headline comparative conclusion and the statistical evidence in Table 1. The single scene, the absent FPS measurement details, and the sparse implementation descriptions reinforce the need for either a substantially expanded evaluation or a reframed, more modest claim. I would be willing to look at a revision that addresses these points."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the bottom line: this is a legitimate small empirical comparison of three 3DGS pipelines for XR virtual environments, and the qualitative material is worth a look for people working on VR scene creation. But the central quantitative claim—that novel view synthesis beats mesh reconstruction on presence—is not backed by the reported numbers.\n\nWhat's actually new: the authors integrate three known methods (vanilla 3DGS, SuGaR-based mesh extraction, DreamScene360-style panoramas) into Unity for HMD use and run an eight-person SUS study. That integration is real work, and the qualitative observations about motion blur, incomplete reconstructions, and panoramic viewpoint restrictions are plausible and useful. The discussion section is honest: they note mesh reconstruction can't compete with TSDF on distant objects and that the panorama approach fails on free movement. That counts for something.\n\nThe soft spots are mostly statistical. With n=8, the SUS differences between novel view (4.9±1.2) and mesh (4.5±1.1) are small; the stress-test estimate of d≈0.35 is right, and without a paired test or effect sizes the ordering is within noise. The paper also gives no objective rendering quality metrics, tests only one scene, and describes each pipeline in a few sentences, so the implementation-effort confound the reader flagged is real. If the mesh pipeline was tuned less carefully, the gap could be an artifact. Even under equal tuning, the evidence can't distinguish the two top methods. The 70 FPS claim is less fragile, but even there no variance is reported.\n\nCitation pattern is fine; the one self-citation is a position paper and isn't used to construct the results. The SUS findings are external ratings, so no circularity problem.\n\nWho's this for? XR developers who want a quick sense of what 3DGS-based VE creation can do today, and researchers doing follow-up comparisons. It deserves a serious referee because the comparison question is worth answering, but it needs a revision with more participants, inferential statistics, and ideally objective quality metrics and more scenes. As it stands, treat the headline claim as preliminary.","headline":"A legitimate but statistically thin comparison of three 3DGS-based VE creation methods; the claim that novel view synthesis gives the best presence outruns the data.","tokens_in":4277,"tokens_out":1819,"would_cite":false,"duration_ms":16728,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that among three 3D Gaussian Splatting pipelines for building virtual environments, direct novel view synthesis gives the strongest user presence while all three sustain roughly 70 frames per second on a headset.","keywords":["3D Gaussian Splatting","virtual environments","extended reality","novel view synthesis","mesh reconstruction","panoramic scene","user presence","SUS questionnaire"],"falsifier":"Re-run the three pipelines on a fixed set of scenes with matched training budgets and hyperparameter tuning, collect SUS ratings from a larger blinded participant pool, and measure per-frame view-consistency error along a scripted head path; if novel view synthesis no longer scores highest or no longer has the lowest consistency error, the paper's central ordering fails.","tokens_in":3369,"feed_emoji":"🥽","tokens_out":6390,"duration_ms":56096,"temperature":0.7,"pith_summary":"This paper asks whether 3D Gaussian Splatting (3DGS) can serve as a practical basis for building virtual environments for extended reality. It builds a virtual environment by three routes: direct novel-view synthesis from multiview photographs, surface-aligned Gaussian meshing followed by Poisson reconstruction and texturing, and a 360-degree panoramic Gaussian scene generated from a fixed viewpoint. In a user study with eight participants, the direct novel-view approach scored highest on the Slater-Usoh-Steed presence questionnaire, and all three approaches ran at roughly 70 frames per second on a desktop GPU driving a Meta Quest 3. The paper concludes that 3DGS is feasible for real-time XR virtual environments, while flagging the need for better object-level interaction and the fixed-viewpoint weakness of the panoramic route.","feed_headline":"Direct 3D Gaussian view beats meshes and panoramas for VR presence","feed_subtitle":"User study: direct Gaussian view synthesis gives the strongest presence among three 3DGS pipelines.","key_machinery":"The object that carries the argument is the 3D Gaussian Splatting representation: a scene stored as an explicit cloud of 3D Gaussian points that can be rendered in real time. The paper compares three pipelines built on that representation: direct view synthesis (COLMAP poses plus standard 3DGS training, rendered in Unity with an open-source shader), mesh reconstruction (SuGaR's surface-aligned Gaussian regularization, Poisson surface reconstruction, and texture cues from trained Gaussians), and panoramic scene synthesis (multiview images fused into a 360-degree Gaussian panorama using monocular depth estimation and an optimizable geometric field). The user-presence measurement is the SUS questionnaire, which turns the comparison into numbers the authors use to rank the approaches.","core_discovery":"The central discovery is that the way 3DGS is used to create a virtual environment changes how present users feel in it. The novel-view-synthesis approach, which keeps the scene as a trained cloud of 3D Gaussians and renders them directly, produced the highest SUS Count (2.8 ± 1.8) and SUS Mean (4.9 ± 1.2) and won five of the six individual presence questions; participants said visual quality stayed consistent as they moved. The mesh-reconstruction route scored lower (SUS Mean 4.5 ± 1.1) and showed reconstruction incompleteness and difficulty with distant objects, while the panoramic route scored far lower (SUS Mean 1.9 ± 0.9) and produced artifacts when users shifted their standing position. The paper treats the first route as the most suitable immediate choice for immersive free-movement XR, and the other two as receding or constrained options.","pith_inferences":["Editorial inference: with eight participants and no statistical significance test, the reported SUS ordering should be read as a pilot result; a larger pre-registered study would be needed to confirm that the advantage is intrinsic to the approach.","Editorial inference: a natural testable extension is to measure view-consistency error along a scripted head path for the three pipelines; the paper's explanation predicts that direct view synthesis will also win on objective consistency metrics.","Editorial inference: the fixed-viewpoint failure of the panoramic route suggests that stitching several multiview panoramas into one 3DGS scene could extend it to larger spaces, a direction the paper flags as future work.","Editorial inference: for XR developers, the result argues for investing in Gaussian-level selection and interaction primitives instead of treating mesh conversion as the default integration path."],"forward_implications":["If the ordering holds, teams building XR virtual environments should favor direct Gaussian view synthesis over converting the scene to a textured mesh or a fixed-viewpoint panorama when users need to move freely.","3DGS-based mesh reconstruction still has to catch up with traditional TSDF methods, especially for distant objects, before it can serve as a drop-in mesh source for XR.","Panoramic Gaussian scenes remain useful for stationary virtual tours but not for room-scale or larger XR experiences, since even slight changes in standing position degrade the display.","Real-time performance around 70 FPS is reachable with 3DGS on current consumer GPU and head-mounted-display hardware, which removes a major practical obstacle to XR use.","Object-level interaction in direct Gaussian scenes requires a mechanism to identify and prioritize Gaussians belonging to specific objects, rather than treating all Gaussians uniformly."],"supporting_citations":[{"why":"Supplies the original 3D Gaussian Splatting representation and real-time rendering method that all three pipelines build on.","marker":"[5]"},{"why":"Supplies SuGaR's surface-aligned regularization used to train evenly distributed Gaussians for the mesh reconstruction route.","marker":"[1]"},{"why":"Supplies the panoramic generation idea (monocular depth plus optimizable geometric field and pseudo-views) for the panoramic scene route.","marker":"[10]"},{"why":"Supplies camera pose estimation and sparse point clouds from multiview images, the input to the novel-view-synthesis pipeline.","marker":"[8]"},{"why":"Supplies the Slater-Usoh-Steed questionnaire that measures presence in the user study.","marker":"[9]"},{"why":"Supplies the Poisson surface reconstruction algorithm that converts sampled Gaussians into a triangular mesh.","marker":"[4]"},{"why":"Supplies the texture extraction cues used to texture the reconstructed mesh from the trained Gaussians.","marker":"[2]"}],"fun_headline_variants":["Direct 3DGS view wins presence over meshes and panoramas","For VR, direct 3D Gaussian rendering beats mesh and panorama paths","Study: direct 3DGS pipeline yields strongest sense of presence in VR","3DGS direct rendering tops mesh and panorama for VR immersion","Direct 3DGS beats mesh and panorama for strongest VR presence"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison assumes the three pipelines were implemented and tuned to a comparable level, so the large difference in presence scores reflects the approaches themselves rather than the amount of engineering effort each one received in this study.","fun_headline_variants_meta":{"raw":{"variants":["Direct 3DGS view wins presence over meshes and panoramas","For VR, direct 3D Gaussian rendering beats mesh and panorama paths","Study: direct 3DGS pipeline yields strongest sense of presence in VR","3DGS direct rendering tops mesh and panorama for VR immersion","Direct 3DGS beats mesh and panorama for strongest VR presence"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000901,"raw_usage":{"total_tokens":3821,"prompt_tokens":828,"completion_tokens":2993,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":444,"completion_tokens_details":{"reasoning_tokens":2899}},"tokens_in":444,"tokens_out":2993,"duration_ms":22089,"temperature":1.0,"reasoning_tokens":2899,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:05:53.902107+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the three pipelines on a fixed set of scenes with matched training budgets and hyperparameter tuning, collect SUS ratings from a larger blinded participant pool, and measure per-frame view-consistency error along a scripted head path; if novel view synthesis no longer scores highest or no longer has the lowest consistency error, the paper's central ordering fails.","supporting_citations":[{"cited_title":"Gu \\'e don and V","cited_arxiv_id":null,"evidence_quote":"Supplies the original 3D Gaussian Splatting representation and real-time rendering method that all three pipelines build on."},{"cited_title":"Mildenhall, P","cited_arxiv_id":null,"evidence_quote":"Supplies the panoramic generation idea (monocular depth plus optimizable geometric field and pseudo-views) for the panoramic scene route."},{"cited_title":"Kazhdan, M","cited_arxiv_id":null,"evidence_quote":"Supplies camera pose estimation and sparse point clouds from multiview images, the input to the novel-view-synthesis pipeline."},{"cited_title":"Kerbl, G","cited_arxiv_id":null,"evidence_quote":"Supplies the Slater-Usoh-Steed questionnaire that measures presence in the user study."}],"review_version":1}