{"id":"2fbe33c2-3ec8-413c-ab94-13a7c33f6cfc","arxiv_id":"2505.15294","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"R3GS integrates appearance-conditioned hash features, a fine-tuned human-detector visibility map, and a fixed sky sphere into 3D Gaussian Splatting to improve novel view synthesis and relocalization on Phototourism scenes.","lead":"This paper presents R3GS, a 3D Gaussian Splatting system for reconstructing outdoor scenes from unconstrained photo collections, and for relocalizing cameras in those scenes. It combines per-image appearance features, a transient object removal mask, and a fixed sky sphere to improve rendering, plus a place-recognition-based pose initialization for robust relocalization.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table I contradicts the SOTA claim: GS-W beats R3GS on Brandenburg Gate in PSNR and SSIM and on Trevi Fountain in LPIPS, while the text says R3GS outperforms all concurrent works.","rationale":"The reader's weakest assumption was the generalization of the fine-tuned human detector to arbitrary transient objects. That concern is valid, but it concerns a component whose effect on test-set metrics is acknowledged to be limited because the test set contains no transient objects. The Table I inconsistency is more immediate: it is a verifiable mismatch between the paper's headline claim and its own quantitative table. Both issues are addressable with additional experiments and rewording, so a conditional verdict remains appropriate. The disagreement with the reader is about which concern is most load-bearing, not about the overall verdict.","tokens_in":10130,"tokens_out":8479,"duration_ms":77184,"concrete_test":"Re-run the comparison at a matched input resolution: either run GS-W at full resolution (if feasible) or run R3GS at the same 2x-downsampled resolution used for GS-W, then recompute per-scene PSNR, SSIM, and LPIPS on the identical test split. If GS-W still exceeds R3GS on any scene or metric, the SOTA claim must be qualified or removed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that R3GS achieves state-of-the-art novel view synthesis on unconstrained collections. Table I does not support that claim as stated. GS-W [22] reports 28.39 PSNR / 0.9306 SSIM / 0.0833 LPIPS on Brandenburg Gate versus R3GS's 27.02 / 0.9240 / 0.1317, and on Trevi Fountain GS-W reports better LPIPS (0.1640 vs. 0.1950). The paper's own text in Section IV-B first says 'Our method outperforms all concurrent works in PSNR, SSIM, and LPIPS' and then says 'GS-W shows the best performance, but its results are based on downsampled images.' The dagger note that GS-W needs 2x downsampling may justify excluding it from the headline comparison, but the current wording is internally inconsistent. If GS-W is excluded on resolution grounds, the claim should be explicitly restricted to full-resolution methods; if it is included, the claim is false. Since the abstract and introduction rest on this SOTA claim, this inconsistency is a direct, load-bearing weakness in the paper's central assertion.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes R3GS, a 3D Gaussian Splatting framework for unconstrained photo collections. It combines per-image CNN appearance features with per-anchor multiresolution hash-grid features to predict Gaussian attributes, fine-tunes a lightweight human detector to produce visibility maps for masking transient objects, handles sky regions by constraining sky Gaussians to a large-radius sphere with a depth prior, and introduces a relocalization procedure that initializes poses via CosPlace retrieval and refines them with photometric loss. Experiments on three Phototourism scenes report novel-view-synthesis quality (Table I), relocalization accuracy (Table II), ablations (Table III), and training/rendering speed.","tokens_in":10383,"tokens_out":5320,"duration_ms":46541,"significance":"If the claims were fully supported, the hybrid anchor representation and the relocalization-by-appearance-tuning pipeline would be useful contributions to in-the-wild 3DGS, and the paper demonstrates real-time rendering with competitive NVS numbers. The paper also states that code will be open-sourced, which would aid reproducibility. However, the current quantitative evidence does not support the unconditional state-of-the-art claim: Table I shows GS-W outperforming R3GS on Brandenburg Gate and on Trevi Fountain LPIPS, the sky-handling ablation has an unexplained SSIM regression, and the visibility-map generalization claim lacks mask-level evaluation. The approach is promising, but the claims need to be aligned with the evidence before the paper is publishable.","major_comments":[{"comment":"The text states 'Our method outperforms all concurrent works in PSNR, SSIM, and LPIPS' and the caption of Table I says 'We significantly outperform all baseline methods', but Table I contradicts this. On Brandenburg Gate, GS-W scores 28.39 PSNR / 0.9306 SSIM / 0.0833 LPIPS versus R3GS's 27.02 / 0.9240 / 0.1317, and on Trevi Fountain GS-W has a better LPIPS (0.1640 versus 0.1950). The footnote that GS-W downsamples input by 2x and would overflow memory at full resolution may justify excluding it from a full-resolution comparison, but then the SOTA claim must be explicitly restricted to full-resolution methods. As written, the abstract, introduction, and Section IV-B make an unconditional claim that the table does not support; please rephrase the claim or add a full-resolution comparison with GS-W if feasible.","section":"Section IV-B, Table I"},{"comment":"Table III shows the full model's SSIM on Brandenburg Gate (0.9240) is lower than the no-sky variant (0.9421), while the text says removing the sky module causes the visibility map to treat the sky as transient and disrupts sky learning. The paper does not explain this metric regression. Since sky handling is a stated contribution, either provide an explanation (for example, if the sky mask incorrectly masks foreground pixels or the depth prior biases SSIM) or temper the claim that sky handling improves rendering fidelity; without this, the benefit of the sky module is not established.","section":"Section IV-C, Table III"},{"comment":"The transient-object removal module is a core contribution but is underspecified. The paper states that a pre-trained LRASPP-MobileNet human detector is fine-tuned for 20,000-25,000 iterations in Section III-C, yet Section IV-A says fine-tuning converges after 5,000 iterations, and the supervision signal for this fine-tuning is never stated. The claim that the visibility map 'efficiently generalizes to other transient objects (such as posters, banners, and cars)' is supported only by final rendering metrics; no mask accuracy, intersection-over-union, or qualitative mask statistics are reported. Because Section IV-C notes that the test set contains no transient objects, the NVS metrics cannot demonstrate that transient handling improves test-time rendering. Please specify the fine-tuning protocol, report mask-level metrics if possible, and discuss the generalization evidence.","section":"Section III-C, Section IV-A"},{"comment":"The relocalization experiment compares only with a 3DGS baseline, which is insufficient to support the claim of 'robust relocalization' or 'superior relocalization accuracy'. Established pose-estimation and relocalization baselines should be included, or the claim should be limited to 'improves over direct 3DGS pose optimization'. In addition, on Brandenburg Gate R3GS's rotation distance (0.582) is slightly worse than 3DGS's (0.576), and the paper does not discuss this. The ATE improvements are large, but the single-baseline comparison and the unaddressed rotation regression weaken the conclusion.","section":"Section IV-B, Table II"}],"minor_comments":[{"comment":"There are numerous typos, including 'ffne-tune' in the abstract, 'golobal' in Section III-B, and 'adress' in the introduction; the paper needs a careful proofread.","section":"Abstract, Section III"},{"comment":"The statement that the highest and lowest ATE values are excluded in each group is not accompanied by the number of test images or the variance of the remaining values, making it hard to judge the stability of the reported improvements.","section":"Section IV-B, Relocalization"},{"comment":"Equation (4) defines alpha using sigma and delta, but these quantities are not defined in the preliminaries; in standard 3DGS the opacity alpha is a learnable attribute, so the equation should be aligned with the notation used in the rest of the paper.","section":"Equation (4)"},{"comment":"Equation (10) for ATE is malformed: the square root appears to apply to an underspecified sum, and no averaging or mean operation is shown; please rewrite the ATE definition clearly.","section":"Equation (10)"},{"comment":"Reference [38] is cited for COLMAP poses, but the cited title is 'Colmap: A memory-efficient occupancy grid mapping framework', which is a different work; the correct COLMAP reference should be used.","section":"References, [38]"},{"comment":"In Table II the first row reads '0.4010.576' without a clear separator between the ATE and Rot Dist values; please format the numbers consistently.","section":"Table II"}],"recommendation":"major_revision","confidential_remarks":"The core approach is interesting and the paper can likely be revised to a publishable state, but the state-of-the-art claim, the sky-handling ablation, and the visibility-map evidence need to be addressed before acceptance. I would also ask the authors to verify the reference list, as [38] appears to cite the wrong COLMAP paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper combines several known ingredients into a 3DGS pipeline for unconstrained photo collections: appearance-conditioned hash anchors (Scaffold-GS + per-image CNN feature), a fine-tuned human detector for visibility maps, a fixed sky sphere with a depth prior, and CosPlace-initialized photometric relocalization. That exact combination is new, and the system does what it promises at 239 FPS, which is the practical selling point.\n\nWhat it does well: the visibility-map fine-tuning is clever and cheap, and the ablation shows PSNR drops from 27.02 to 25.96 when removed. The sky sphere with fixed anchor positions is a sensible fix for floaters. The relocalization, while only compared against a vanilla 3DGS baseline, at least demonstrates the concept works.\n\nThe soft spots are real. The SOTA claim is contradicted by its own Table I: GS-W beats R3GS on Brandenburg Gate on PSNR, SSIM, and LPIPS, and on Trevi Fountain on LPIPS. The text first says 'outperforms all concurrent works' and then concedes GS-W's numbers come from downsampled images. If GS-W is excluded on resolution grounds, the claim must be explicitly restricted; as written, it is false. That is load-bearing because the abstract and introduction rest on it.\n\nSecond, the ablation in Table III shows the full model's SSIM (0.9240) is lower than w/o sky (0.9421) on Brandenburg. The paper doesn't explain this. Removing the sky module shouldn't improve SSIM by 0.018. Either the sky sphere introduces structural artifacts or there's a bug; either way it needs addressing.\n\nThird, the depth prior loss is mentioned but never specified. Section III-D ends with 'The following losses were applied' and no equations appear. That's a missing core detail.\n\nFourth, the visibility map's generalization to vehicles and posters is asserted, not measured. No mask IoU, no accuracy numbers, and the test set has no transient objects. The fine-tuning supervision is also unspecified. The indirect evidence from rendering metrics is suggestive but not conclusive.\n\nThe relocalization benchmark is thin—one baseline, three scenes, trimmed outliers—but acceptable for a preliminary result.\n\nOverall: a plausible engineering contribution with a few genuinely useful ideas, but the central claim needs rewriting and the experimental gaps need filling. It deserves a serious referee, not a desk reject. I'd send it back for major revision: qualify the SOTA claim, explain the SSIM ablation, write out the depth loss, and report mask accuracy. For your reading group, worth a look for the visibility-map trick, but I'd wait for the revised version before citing it.","headline":"Useful engineering combination, but the SOTA claim doesn't survive contact with Table I.","tokens_in":10928,"tokens_out":3928,"would_cite":false,"duration_ms":31699,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"R3GS is a 3D Gaussian Splatting framework that jointly handles appearance variation, transient objects, and sky in unconstrained photo collections, improving novel-view synthesis and relocalization while keeping real-time rendering.","keywords":["3D Gaussian Splatting","novel view synthesis","relocalization","unconstrained image collections","transient object removal","sky handling","appearance modeling","photometric pose optimization"],"falsifier":"On a Phototourism-style scene, take held-out test images that do contain transient objects with manually labeled masks, and compare rendering and relocalization with and without R3GS's visibility map; if novel-view PSNR/LPIPS on the transient-containing frames does not improve over the no-mask baseline, or if the fine-tuned detector's IoU on non-human transient classes falls well below its human-class IoU, then the transient-generalization claim is contradicted. A cheaper check is to report the visibility map's precision and recall on held-out transients, which the paper currently omits.","tokens_in":9924,"feed_emoji":"🏛️","tokens_out":9790,"duration_ms":71962,"temperature":0.7,"pith_summary":"This paper argues that 3D Gaussian Splatting can be made to work on unconstrained photo collections—tourist snapshots with changing daylight, passing pedestrians, cars, posters, and bright skies—without giving up real-time rendering. R3GS, the proposed framework, attaches to each Gaussian anchor a global per-image appearance feature from a CNN plus a local feature from a multiresolution hash grid, then predicts color, opacity, and covariance with shallow MLPs. A fine-tuned lightweight human detector supplies a visibility map that masks transient objects during optimization, while a large-radius sky sphere with a depth prior keeps the sky from collapsing into the foreground. The paper reports state-of-the-art novel-view-synthesis numbers on three Phototourism landmarks and a large drop in relocalization error versus plain 3DGS, at training times and frame rates close to the original method. If correct, R3GS makes high-quality reconstruction and pose estimation practical for large, messy real-world image collections.","feed_headline":"R3GS makes 3D Gaussian scenes robust to crowds, light, and sky","feed_subtitle":"Hybrid encoding plus transient masking and sky depth prior keeps real-time rendering while improving pose accuracy.","key_machinery":"The load-bearing object is the hybrid anchor: per-anchor, a 32-dimensional local feature is queried from a multiresolution hash grid and concatenated with a 64-dimensional global appearance feature extracted from the reference image by a CNN encoder; three shallow MLPs map the concatenation to the Gaussian attributes (opacity, color, covariance). Around this core sit two supporting mechanisms. The first is a visibility map: a pretrained LRASPP-MobileNet human detector, fine-tuned for 20,000–25,000 iterations on the scene, is used to mask transient foreground objects while leaving the sky untouched. The second is a sky sphere: 20,000 Gaussians sampled by Fibonacci spherical mapping on a sphere of radius ten times the scene radius, with fixed positions, a OneFormer sky mask, and a depth prior that keeps sky geometry at infinity. Relocalization rests on CosPlace: the query's place-recognition descriptor is compared with a bank of training descriptors, the three nearest neighbors' poses are combined with distance-inverse weights to initialize the camera, and photometric loss refines the pose with Gaussian splatting gradients.","core_discovery":"The central claim is that the three long-standing failure modes of 3DGS on in-the-wild data—appearance variation, transient objects, and sky—can be handled by one hybrid architecture without sacrificing the method's speed or memory footprint. The scene is represented by anchors, each carrying a local hash-grid code and a per-image CNN appearance code, feeding three shallow MLPs that emit Gaussian color, opacity, and covariance. Transient content is suppressed by a visibility map produced by a fine-tuned LRASPP-MobileNet detector, and the sky is rendered by Gaussians pinned to a sphere of radius ten times the scene radius, with a depth prior preventing foreground-sky depth collisions. For relocalization, an appearance-conditioned standard 3DGS model is rendered from a query image, initialized with a CosPlace-weighted pose from the three nearest training views, and refined by photometric loss. The result, the authors claim, is state-of-the-art PSNR, SSIM, and LPIPS on Brandenburg Gate, Sacre Coeur, and Trevi Fountain, and ATE of 0.101, 0.021, and 0.367 respectively against a 3DGS baseline, while rendering at 239 FPS and training in roughly 0.31 hours per 10,000 iterations.","pith_inferences":["A natural extension the paper does not test is whether the fine-tuned detector transfers to a different landmark scene; the paper reports no mask-accuracy numbers, so held-out transient categories could behave worse than the training-time masks.","The CosPlace-based pose initialization only needs a feature bank of training poses, so the same relocalization idea could scale to very large collections where full bundle adjustment is impractical, assuming the three-nearest-neighbor retrieval is geographically reliable.","Because each image supplies its own appearance code, the exported 3DGS model could in principle be re-rendered with a different appearance code to simulate time-of-day or weather changes, a capability the paper notes but does not demonstrate.","The visibility map is applied only to the foreground so clouds are not masked; a direct test of the sky-handling claim would be to compare floaters and depth consistency on scenes with partial cloud cover against a no-sky-prior baseline."],"forward_implications":["On the three Phototourism scenes, R3GS exceeds NeRF-W, Ha-NeRF, CR-NeRF, WildGaussians, and the downsampled GS-W on PSNR, SSIM, and LPIPS, while rendering at 239 FPS without caching.","The same pipeline estimates camera poses with ATE of 0.101 (Brandenburg Gate), 0.021 (Sacre Coeur), and 0.367 (Trevi Fountain), a large improvement over the 3DGS baseline (0.401, 0.381, 1.377).","Because the appearance code is per-image and the exported model is a standard 3DGS point cloud, the scene can be rendered with a chosen appearance at a given pose without storing per-Gaussian high-dimensional features.","Masking transient objects improves visual quality in training views, and applying the mask only to the foreground prevents clouds from being misclassified as transient.","Fixed sky-Gaussian positions plus a depth prior eliminate the depth overlap between sky and foreground that causes floaters."],"supporting_citations":[{"why":"Supplies the explicit Gaussian representation and differentiable rasterizer on which R3GS builds and whose real-time speed it preserves.","marker":"[1]"},{"why":"Provides the anchor-based neural-Gaussian structure that R3GS adapts for per-anchor features and MLP attribute prediction.","marker":"[11]"},{"why":"Supplies multiresolution hash-grid encoding for the position-aware local feature of each anchor.","marker":"[2]"},{"why":"Defines the unconstrained photo-collection task, the per-image appearance latent idea, and the train/test split used for evaluation.","marker":"[4]"},{"why":"Introduces the CNN appearance-feature extraction that R3GS adopts and is a primary NeRF-based baseline to beat.","marker":"[5]"},{"why":"Supplies the sky-sphere initialization strategy and is a direct 3DGS-based baseline in the comparison.","marker":"[23]"},{"why":"The strongest competing 3DGS method for unconstrained collections; its downsampling requirement is the comparison point for R3GS's memory advantage.","marker":"[22]"},{"why":"Provides the CosPlace retrieval descriptor used to initialize the relocalization pose from the three nearest training views.","marker":"[31]"},{"why":"Generates the sky masks that separate foreground and sky during R3GS optimization.","marker":"[10]"},{"why":"Provides the lightweight LRASPP-MobileNet human detector that is fine-tuned into the visibility map for transient-object masking.","marker":"[28]"}],"fun_headline_variants":["R3GS handles crowds, light, and sky in 3D Gaussian scenes","Hybrid anchors make Gaussian splatting robust in the wild","One-step fix for transient objects and sky in 3DGS","R3GS: robust relocalization for unconstrained 3D Gaussian scenes","Gaussian splatting that ignores pedestrians and lighting changes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that fine-tuning a human detector on the training scene yields a visibility map that correctly masks every kind of transient object—people, cars, posters, banners—and that the map is accurate enough to exclude those pixels during optimization without erasing useful scene content.","fun_headline_variants_meta":{"raw":{"variants":["R3GS handles crowds, light, and sky in 3D Gaussian scenes","Hybrid anchors make Gaussian splatting robust in the wild","One-step fix for transient objects and sky in 3DGS","R3GS: robust relocalization for unconstrained 3D Gaussian scenes","Gaussian splatting that ignores pedestrians and lighting changes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000371,"raw_usage":{"total_tokens":2068,"prompt_tokens":1107,"completion_tokens":961,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":723,"completion_tokens_details":{"reasoning_tokens":867}},"tokens_in":723,"tokens_out":961,"duration_ms":6667,"temperature":1.0,"reasoning_tokens":867,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:20:16.868586+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a Phototourism-style scene, take held-out test images that do contain transient objects with manually labeled masks, and compare rendering and relocalization with and without R3GS's visibility map; if novel-view PSNR/LPIPS on the transient-containing frames does not improve over the no-mask baseline, or if the fine-tuned detector's IoU on non-human transient classes falls well below its human-class IoU, then the transient-generalization claim is contradicted. A cheaper check is to report the visibility map's precision and recall on held-out transients, which the paper currently omits.","supporting_citations":[{"cited_title":"3D Gaussian Splatting for Real-Time Radiance Field Rendering,","cited_arxiv_id":null,"evidence_quote":"Supplies the explicit Gaussian representation and differentiable rasterizer on which R3GS builds and whose real-time speed it preserves."},{"cited_title":"Scaffold-gs: Structured 3d gaussians for view-adaptive rendering,","cited_arxiv_id":null,"evidence_quote":"Provides the anchor-based neural-Gaussian structure that R3GS adapts for per-anchor features and MLP attribute prediction."},{"cited_title":"Instant Neural Graphics Primitives with a Multiresolution Hash Encoding,","cited_arxiv_id":null,"evidence_quote":"Supplies multiresolution hash-grid encoding for the position-aware local feature of each anchor."},{"cited_title":"Nerf in the wild: Neural radiance fields for unconstrained photo collections,","cited_arxiv_id":null,"evidence_quote":"Defines the unconstrained photo-collection task, the per-image appearance latent idea, and the train/test split used for evaluation."},{"cited_title":"Hallucinated neural radiance fields in the wild,","cited_arxiv_id":null,"evidence_quote":"Introduces the CNN appearance-feature extraction that R3GS adopts and is a primary NeRF-based baseline to beat."},{"cited_title":"Rethinking visual geo- localization for large-scale applications,","cited_arxiv_id":null,"evidence_quote":"Provides the CosPlace retrieval descriptor used to initialize the relocalization pose from the three nearest training views."},{"cited_title":"Oneformer: One transformer to rule universal image segmentation,","cited_arxiv_id":null,"evidence_quote":"Generates the sky masks that separate foreground and sky during R3GS optimization."},{"cited_title":"Searching for mobilenetv3,","cited_arxiv_id":null,"evidence_quote":"Provides the lightweight LRASPP-MobileNet human detector that is fine-tuned into the visibility map for transient-object masking."}],"review_version":1}