{"id":"cf479352-0826-4209-bf86-fe7896b0eb9e","arxiv_id":"2412.10078","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Toy-GS partitions large free camera trajectories into local Gaussian models and fuses them to improve rendering quality while reducing GPU memory.","lead":"This paper describes a 3D rendering method that splits large scenes into smaller pieces, trains a 3D Gaussian scene model for each piece, and blends the pieces to render new camera views along long, irregular paths. It reports better image quality and lower GPU memory use than standard 3D Gaussian Splatting on several large outdoor and indoor datasets, including a new dataset the authors collected.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation 2's 1/N camera-selection threshold can exclude the sparse-background cameras that local models most need, so the claimed free-trajectory gains may not generalize beyond the tested scenes.","rationale":"The central claim is that Toy-GS provides state-of-the-art rendering quality and memory savings on large-scale free camera trajectories. Three conditions have to hold: (i) the adaptive division assigns the right cameras to each local model; (ii) PatchMatch and PPAC improve texture and background rendering; (iii) local-global rendering beats global-only rendering on free trajectories. The paper provides internal evidence for (ii) and (iii): Table 4 shows each component adds PSNR, and the reported numbers are consistent with the abstract (1.19 dB on SCUTic, 0.4 dB on Tanks and Temples, 1.35 dB on Free). Condition (i) is the weakest because the camera-selection threshold in Eq. 2 is introduced without motivation or ablation, and it interacts directly with the paper's own description of free-trajectory view imbalance. A camera with a dense foreground and a sliver of background in view has a large Mc and small MAi for the background region, so it can be discarded from that region despite being one of the few views of it. This is not a disagreement with consensus; it is an internal tension between the stated scene characteristics and the selection rule. The proposed test (relaxing the mask to any visibility) would settle whether the threshold is necessary or harmful. I therefore recommend the verdict remain CONDITIONAL: the paper is promising but needs either code/artifacts or an ablation of the selection threshold to support the generality of the headline gains. I agree with the reader's weakest_assumption identification; no separate concern is more load-bearing.","tokens_in":11593,"tokens_out":13956,"duration_ms":141168,"concrete_test":"Re-run the full pipeline on the Free and SCUTic datasets with N=3, replacing the Eq. 2 mask condition MAi/Mc > 1/N with mask = 1 whenever MAi > 0 (i.e., keep every camera that sees at least one point in the region), holding all other settings identical. Measure PSNR, SSIM, and LPIPS on the same test split, plus the fraction of test images containing any uncovered (zero-opacity) pixels. If the relaxed selection matches or improves quality on the sparse-background scenes, the 1/N threshold is too aggressive and the paper should justify it or show it is beneficial; if quality drops or holes appear, the threshold is not the weak point.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The most load-bearing assumption is the camera-selection rule of §3.1 (Eq. 2): a camera is used to train region Ai only if MAi/Mc > 1/N, where MAi is the number of visible COLMAP points in Ai and Mc is the camera's total visible point count. The paper's own Section 1 states that free trajectories give dense, close views of foreground objects and sparse, distant views of background areas. Under that inhomogeneity, a camera that sees a dense foreground (large Mc) plus a small piece of a background region (small MAi) will fail the threshold and be excluded from exactly the background region it could help constrain. Because background coverage is already sparse in these datasets, dropping such cameras can leave local models with holes or weak geometry. The paper reports no ablation of this threshold: Table 4 varies the number of areas, PatchMatch/PPAC, and global vs. local-global rendering, but never the selection criterion. The central claim that Toy-GS achieves state-of-the-art results on large-scale free camera trajectories therefore rests on an unvalidated choice of 1/N; on scenes where background regions are visible only to foreground-pointing cameras, the method could lose the reported 0.4–1.35 dB gains.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Toy-GS, a 3D Gaussian Splatting system for large-scale free camera trajectories. It partitions cameras and the point cloud with k-means on camera poses, expands or prunes cameras by a visibility ratio (Eq. 2), trains per-region 3DGS models, and enhances them with a PatchMatch-based multi-view constraint and position-aware point adaptive control (PPAC). A local-global rendering rule selects local Gaussians for new viewpoints that pass distance and visibility tests and falls back to a global Gaussian otherwise. The authors report state-of-the-art PSNR, SSIM, and LPIPS on the Free dataset, the Tanks and Temples advanced set, and a new SCUTic dataset, with GPU memory savings of several GB relative to 3DGS.","tokens_in":11910,"tokens_out":9293,"duration_ms":93395,"significance":"If the results hold, Toy-GS is a practical step toward applying 3DGS to trajectories that are neither object-centric nor aerial: it reduces memory through parallel region training and shows quality gains from region-local optimization plus a fusion strategy. The empirical evidence is broad, covering three datasets, and Figure 7 supports the quality-versus-region-count trend. The paper's value is as a system paper, not as a theoretical contribution, and it would be publishable once the central mechanisms are specified and the statistical uncertainty is addressed. The claimed gains are not yet fully supported because the PatchMatch-to-Gaussian update is unspecified, the camera-selection threshold is unablated, and the memory accounting is ambiguous.","major_comments":[{"comment":"The 1/N camera-selection threshold is load-bearing but unvalidated. The paper's introduction states that free trajectories have dense foreground views and sparse background views. Under Eq. (2), a camera that sees many foreground points (large Mc) and only a few points in a background region Ai (small MAi) is discarded from exactly the region whose training data are already sparse. No ablation varies this threshold; Table 4 varies the number of areas, PatchMatch/PPAC, and global versus local-global rendering, but never the selection rule. Since the headline gains (e.g., 1.19 dB on SCUTic) depend on the quality of each local model, the authors should report sensitivity to the threshold or replace the rule with one that cannot reject informative sparse-background cameras.","section":"Section 3.1, Eq. (2)"},{"comment":"The paper claims PatchMatch moves the Gaussian ellipsoids to their correct positions, but it does not specify the matching cost, the number or choice of source views, or how the optimized depth maps are converted into Gaussian mean updates. The only implementation details given are an iteration interval and a patch size in the Appendix. Because the components labeled 2P are ablated jointly in Table 4, the individual contribution of the PatchMatch constraint cannot be isolated, and the method cannot be reproduced or independently verified as written.","section":"Section 3.2"},{"comment":"The GPU-memory accounting is unclear. In Tables 1 and 2 the 'Ours (3 Areas)' memory values equal the VastGaussian values to the megabyte, and in Table 4 the RG and RLG rows have identical memory for the same number of areas. Section 3.3 states that viewpoints failing the local criteria are rendered by the global Gaussian used with VastGaussian, so the authors need to state whether that global model is trained and retained, and if so why its memory is not reflected. A per-component memory breakdown is needed to support the memory-saving claim in the abstract and Figure 7.","section":"Tables 1, 2, and 4; Section 3.3"},{"comment":"All tables report a single run with no error bars, seeds, or statistical tests, and the train/test split is not defined for the SCUTic and Tanks and Temples evaluations. With headline differences as small as 0.4 dB (Table 2) and ablation differences of 0.02 in SSIM (Table 4), readers cannot assess whether the differences are within run-to-run variation. Please provide multi-seed results or confidence intervals for the main comparisons.","section":"Experimental protocol"}],"minor_comments":[{"comment":"The symbol gamma is used both for the PPAC scale factor and for the local-rendering distance threshold; please rename one to avoid confusion.","section":"Section 3.2, Eqs. (7)-(8)"},{"comment":"Equation (1) writes zp = KTP without defining the homogeneous coordinate convention; please state explicitly that z is the depth scale and that P and p are homogeneous vectors.","section":"Section 3.1, Eq. (1)"},{"comment":"'VastGaussain' is a typo for VastGaussian.","section":"Section 3.3"},{"comment":"The text 'improves by 1.8 dB, 0.42 dB, and 0.032 dB in PSNR, SSIM, and LPIPS' should read 0.042 for SSIM, and the values are not all in dB.","section":"Ablation Studies, Table 4 discussion"},{"comment":"The checkmarks for the RG and RLG columns are not visually distinguishable in the rendered text; please use explicit column entries so the comparisons described in the prose can be verified.","section":"Table 4"},{"comment":"The '1.19 dB' gain should be identified as the SCUTic overall result rather than appearing to be an average over all benchmarks.","section":"Abstract"},{"comment":"Please provide a public link, release plan, or access mechanism for the SCUTic dataset, since the paper bills the dataset as a contribution.","section":"SCUTic dataset"}],"recommendation":"major_revision","confidential_remarks":"This is an empirical system paper with no citation-pattern or novelty-disclosure concerns that I can identify. The central risk is reproducibility: the unspecified PatchMatch-to-Gaussian update and the identical memory numbers for Ours and VastGaussian need to be resolved, and I would ask the authors to provide code or raw logs for the main tables. The paper fits the scope of a computer vision/graphics venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Toy-GS is a believable incremental step for 3DGS on free trajectories, but the unvalidated 1/N camera-selection threshold and the missing code/error bars keep me from fully trusting the numbers.\n\nRead this as a system paper: it takes known pieces (k-means pose partitioning from F2-NeRF, regional training from VastGaussian, PatchMatch and PPAC from prior work) and assembles them into a pipeline that beats 3DGS and VastGaussian on the Free dataset by about 1.35 dB PSNR and on their new SCUTic dataset while using less memory. The local-global rendering rule (Eq. 8–9) is genuinely new, and the SCUTic dataset itself is a useful resource with over 600 images per category.\n\nCredit where it is due: the tables are internally consistent across three datasets (PSNR, SSIM, LPIPS all improve), the ablation trend makes sense, and Figure 7 backs the memory/quality tradeoff. For a system paper, that is a solid level of evidence.\n\nNow the soft spots, roughly in order of seriousness.\n\nFirst, the 1/N threshold in Eq. 2 is never ablated. This is the rule that decides whether a camera gets to train a region, and the paper's own intro says free trajectories have dense foreground and sparse background. A camera pointed at a dense foreground object will often see only a small piece of a background region, fail the MAi/Mc > 1/N test, and be excluded from the region that needs more views. The authors claim the ratio is stable across densities, but they never vary the threshold or compare against a different selection rule. The reported gains might hold, but this is a genuine unknown.\n\nSecond, no error bars or seeds. A 0.4 dB gain on Tanks and Temples is small enough that a couple of unlucky runs could change the conclusion.\n\nThird, the PatchMatch integration is underspecified. We are told it optimizes depth maps and then moves Gaussians to fix incorrect depths, but the exact update is not given. This is fixable in a page.\n\nFourth, the ablation prose has small mismatches with the table (e.g., 0.42 vs 0.042 for SSIM). Copyediting, but sloppy.\n\nFifth, no code or data link, and CityGaussian—mentioned in Related Work—is not compared.\n\nOverall, the central idea is sound and the contribution is usable. I would send it to a serious referee now, but I would expect the review to insist on artifacts and an ablation of the selection threshold. If you are in large-scale 3DGS, read it and cite it.","headline":"Toy-GS is a believable incremental step for 3DGS on free trajectories, but the unvalidated 1/N camera-selection threshold and the missing code/error bars keep me from fully trusting the numbers.","tokens_in":12425,"tokens_out":4522,"would_cite":true,"duration_ms":43135,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Toy-GS claims that splitting large free-trajectory scenes into pose-aligned regions and training a local 3D Gaussian Splatting model per region—then fusing local and global models at render time—raises rendering quality while cutting GPU…","keywords":["3D Gaussian Splatting","novel view synthesis","large-scale scenes","free camera trajectories","adaptive spatial partitioning","local-global rendering","multi-view constraints","GPU memory reduction"],"falsifier":"On a free-trajectory scene, re-run training with the selection threshold in Eq. (2) varied (e.g., $1/(2N)$ and $2/N$) while keeping everything else fixed; if a non-$1/N$ threshold raises PSNR or removes holes in regions with sparse background coverage, the ratio test's critical assumption fails.","tokens_in":11420,"feed_emoji":"🎥","tokens_out":6834,"duration_ms":61867,"temperature":0.7,"pith_summary":"Large-scale free camera trajectories—long, irregular paths with dense foreground and sparse background views—break whole-scene 3D Gaussian Splatting, both in quality and GPU memory. Toy-GS argues the cure is to divide the cameras and point cloud into regions by camera pose, train a separate Gaussian model per region, and then fuse local models at render time instead of merging them into one global model. On top of this, PatchMatch-style multi-view constraints reposition Gaussians to correct depth and geometry, and position-aware point adaptive control (PPAC) assigns larger Gaussians to distant areas. The paper reports that this combination raises PSNR by 1.35 dB on the Free dataset, 0.4 dB on Tanks and Temples advanced, and 1.19 dB on the new SCUTic dataset versus vanilla 3DGS, while cutting GPU memory by about 7 G. If correct, this makes high-quality rendering of long, messy camera paths feasible on a single GPU.","feed_headline":"Local Gaussians add 1.19 dB PSNR and save 7 G of GPU memory","feed_subtitle":"Toy-GS splits a scene by camera pose, trains one Gaussian per region, and fuses them for free-path rendering.","key_machinery":"The load-bearing structure is the adaptive spatial partitioning plus local-global rendering. K-means on camera poses yields regions $\\{A_n\\}_{n=1}^N$; a camera is kept for region $A_i$ iff the fraction of points it sees that lie in $A_i$ exceeds $1/N$ (Eq. 2), which is meant to be robust to varying point-cloud density because both numerator and denominator move together. Within each region, a separate 3D Gaussian Splatting model is trained with two additions: PatchMatch homography warping (Eqs. 5–6) between reference and source views to refine depth and normals and relocate Gaussians onto correct geometry, and PPAC, which scales Gaussian size by $\\gamma(\\mu)=\\|\\mu\\|_2/r-1$ for $\\|\\mu\\|_2 \\geq 2r$ so distant regions are covered by larger splats. At render time, a new viewpoint first locates its region by k-means, then passes the distance test $\\|C_{\\text{new}}-O\\|<\\gamma$ and the visibility test $M'_{A_i}/M'_{c_{\\text{new}}}>1/N$; if both hold, the local Gaussian renders it, otherwise the global Gaussian does. This fusion is what prevents the holes that arise when each region's Gaussian is simply merged and pruned as in VastGaussian.","core_discovery":"On free camera trajectories, the spatial coverage of views is strongly inhomogeneous: foreground objects are captured densely and up close, while backgrounds are seen sparsely from far away, and parts of the scene may be invisible from every camera. Toy-GS's central claim is that this regime requires splitting the scene, not to save memory alone, but to let each local Gaussian model concentrate on the texture detail it can actually see, and that the split must follow the camera poses rather than axis-aligned boxes. It therefore clusters cameras with k-means on their poses, assigns each point in the sparse SfM cloud to a region, prunes and expands cameras per region using a visibility ratio test, trains a 3DGS per region in parallel, and then decides per new viewpoint, by distance and visibility constraints, whether to render with that region's local Gaussian or with a global Gaussian. The multi-view constraint (PatchMatch on rendered depth and normal maps) and PPAC (scale factor $\\gamma(\\mu)=\\|\\mu\\|_2/r-1$ for points beyond radius $r$) are added to improve geometry and distant rendering. The reported result is consistent gains in PSNR/SSIM/LPIPS over 3DGS and over VastGaussian on two public datasets and the authors' SCUTic dataset, with memory usage that falls as region count rises.","pith_inferences":["Because the camera-selection threshold $1/N$ is fixed by region count, scenes with extreme foreground/background imbalance may silently exclude cameras whose small in-region share is still geometrically essential; an adaptive threshold or a view-coverage prior is a natural next step, though untested in the paper.","The local-global fusion rule is only demonstrated for up to seven regions; at finer granularity the distance test $\\|C_{\\text{new}}-O\\|<\\gamma$ likely needs re-tuning, since the max-distance threshold grows with region spread and may admit the wrong local model.","Since the base is 3DGS, the same divide-train-fuse recipe could be dropped into other point-based or splatting renderers, and the SCUTic dataset (2–3 times larger than Free) could serve as a stress test for trajectory-level generalization.","The reported memory figures are peak GPU usage on one RTX 3090; the claimed 7 G saving is an average over benchmarks, so actual savings vary by scene and region count."],"forward_implications":["On the Free dataset, Toy-GS with three regions improves PSNR by 1.35 dB over 3DGS and by 1.37 dB over VastGaussian, with better SSIM and LPIPS.","On the Tanks and Temples advanced split, the method gains 0.4 dB over 3DGS (21.27 vs 20.87) and 1.16 dB over VastGaussian, while using about 1 G less memory than 3DGS.","On the SCUTic dataset, Toy-GS with three areas beats 3DGS by 1.19 dB overall (23.76 vs 22.57), beats VastGaussian by 1.79 dB, and cuts total GPU memory from about 16.4 G to 9.6 G.","Ablations show that PatchMatch+PPAC alone add 0.6 dB, the local-global renderer adds about 1.8 dB over global-only rendering, and increasing region count from 1 to 7 raises PSNR by 1.14 dB while reducing memory by about 6 G.","Rendering quality improves monotonically as the number of regions grows from 1 to 7, without the degradation seen in VastGaussian's global merging."],"supporting_citations":[{"why":"Supplies the base 3D Gaussian Splatting renderer that Toy-GS divides, augments, and compares against.","marker":"(Kerbl et al. 2023)"},{"why":"Defines the free-trajectory problem, introduces adaptive spatial division for such trajectories, and contributes the Free dataset used for evaluation.","marker":"(Wang et al. 2023)"},{"why":"VastGaussian is the divide-and-conquer large-scene baseline whose global-only rendering Toy-GS replaces with local-global fusion.","marker":"(Lin et al. 2024)"},{"why":"PatchMatch is the multi-view correspondence algorithm used to refine rendered depth and normals and reposition Gaussians for better geometry.","marker":"(Barnes et al. 2009)"},{"why":"Supplies the position-aware point adaptive control (PPAC) that scales Gaussian size for distant scenery.","marker":"(Chen et al. 2023)"},{"why":"Provides the Tanks and Temples advanced dataset used to benchmark rendering on free trajectories.","marker":"(Knapitsch et al. 2017)"}],"fun_headline_variants":["Toy-GS: pose-split Gaussians render free camera paths","Split by camera pose, train local Gaussians, save 7 G","Toy-GS: 1.19 dB PSNR up, 7 G memory down on free paths","Pose-aware splitting boosts 3D Gaussian rendering on free trajectories"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The camera-selection test assumes that the share of points a camera sees inside a region reliably tells whether that camera should train that region; on free trajectories where foreground is dense and background sparse, this threshold can drop informative cameras and leave holes.","fun_headline_variants_meta":{"raw":{"variants":["Toy-GS: pose-split Gaussians render free camera paths","Split by camera pose, train local Gaussians, save 7 G","Toy-GS: 1.19 dB PSNR up, 7 G memory down on free paths","Pose-aware splitting boosts 3D Gaussian rendering on free trajectories"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001688,"raw_usage":{"total_tokens":6756,"prompt_tokens":1077,"completion_tokens":5679,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":693,"completion_tokens_details":{"reasoning_tokens":5594}},"tokens_in":693,"tokens_out":5679,"duration_ms":44129,"temperature":1.0,"reasoning_tokens":5594,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:22:35.390131+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a free-trajectory scene, re-run training with the selection threshold in Eq. (2) varied (e.g., $1/(2N)$ and $2/N$) while keeping everything else fixed; if a non-$1/N$ threshold raises PSNR or removes holes in regions with sparse background coverage, the ratio test's critical assumption fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the free-trajectory problem, introduces adaptive spatial division for such trajectories, and contributes the Free dataset used for evaluation."}],"review_version":1}