{"id":"b9c7d9d6-2f5c-473a-b389-cf8de9786456","arxiv_id":"2508.18971","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A privacy attack exposes fine details in NeRF geometry, and a segmentation-supervised neural field (ppNeSF) avoids this while achieving competitive visual localization.","lead":"The paper shows that NeRF scene models trained on RGB images keep fine private details in their geometry, even when the color output is removed. It then introduces ppNeSF, a NeRF variant trained only on self-supervised segmentation maps, which hides those details while still supporting accurate camera pose estimation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Privacy claim rests only on a single cross-scene grayscale inversion attack; a same-scene white-box or color attack is untested, so 'strong privacy' is not established.","rationale":"The method's localization contribution is benchmarked on public datasets and shows consistent gains over privacy-preserving baselines, so that part of the central claim is credible. The privacy contribution, however, is the more novel and load-bearing part, and it depends entirely on the inversion attack being a faithful upper bound on adversarial reconstruction. The reader's conditional verdict captures this: the manuscript never tests a stronger adversary, never probes Φ, and even acknowledges in Sec. 3 that cross-scene transfer of the attack is not guaranteed. Because the paper's own limitation statement admits the weak point, the concern is not manufactured. A same-scene white-box inversion test with RGB output directly targets the gap: if it still cannot recover held-out view detail, the current evidence is much stronger; if it can, the headline privacy claim would need to be softened. The localization accuracy, ablations, and cost analysis do not require revision, so the verdict should remain conditional rather than being upgraded or downgraded.","tokens_in":25157,"tokens_out":10062,"duration_ms":100570,"concrete_test":"Train a same-scene white-box inversion attack on a target ppNeSF model: render internal geometry features, segmentation maps, and depth on held-out views; train a U-Net with RGB output and LPIPS+L1 loss on training views only; evaluate on held-out views. Independently, train a second inversion network on Φ's multi-scale feature maps instead of rendered features. If either attack yields LPIPS < 0.4, FID below the random-input control, or LLaVa captions identifying scene objects/text, the privacy claim is falsified; if held-out reconstruction stays at the level of the current cross-scene attack, the bounded-adversary concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that ppNeSF is privacy-preserving is supported only by the failure of one U-Net inversion model trained to reconstruct grayscale images from rendered internal features. The paper itself states in Sec. 3 that nothing constrains the internal representations of independently trained NIFs to lie in a common embedding space, so cross-dataset transfer failure is unsurprising and provides no upper bound on what a scene-specific or white-box adversary could extract. The attack also discards color by design, so color/texture leakage is not measured. In addition, the RGB-trained image encoder Φ is part of the proposed architecture and is never probed for invertibility; if Φ is retained in the deployed server-side model, it is an untested leak channel. Thus the privacy guarantee is conditional on an unspecified weak-adversary assumption rather than demonstrated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses privacy in NeRF-based visual localization. It first proposes a privacy attack and evaluation protocol for neural implicit fields (NIFs), arguing that NeRFs trained with photometric/RGB losses store fine-grained scene details in their geometry branch, so removing the color head is insufficient. It then introduces ppNeSF, a neural segmentation field trained with self-supervised segmentation labels derived via an optimal-transport labeling scheme, and shows that the resulting segmentation space supports accurate pose refinement. Localization experiments on 7Scenes, Cambridge Landmarks, and Indoor6 report competitive or state-of-the-art results among privacy-preserving baselines, with ablations and training details in the appendix.","tokens_in":25411,"tokens_out":5672,"duration_ms":55978,"significance":"If the privacy claim were fully established, the paper would be a valuable first step toward privacy-preserving neural implicit localization, replacing RGB supervision with learned segmentation targets. The localization contribution is technically solid: it uses standard benchmarks, compares against relevant privacy-preserving baselines, includes ablations, and reports consistent gains in most scenes. The proposed privacy protocol is also creative, using VLM-based semantic similarity in addition to perceptual metrics. However, the privacy evidence is currently based on a single bounded-adversary attack, with no error bars, no independent corroboration, and with at least one acknowledged limitation that undermines the strength of the conclusion. The central privacy claim therefore needs substantial additional support before the paper can be accepted as it stands.","major_comments":[{"comment":"The privacy claim rests entirely on a single cross-scene inversion attack. The paper itself states: 'nothing constrains the internal representations of NIFs trained on different scenes ... to lie in a common embedding space. As such, there are no guarantees that the inversion model can generalize to NIFs trained on scenes unseen during training.' This is precisely the scenario used to conclude that ppNeSF is privacy-preserving. A cross-scene attack can fail because of representation shift even when a scene-specific adversary could extract fine details. To support the central claim, the authors should report same-scene/white-box attacks, attacks trained per scene, or an explicit argument why the cross-scene failure is indicative of the information content rather than the attack's inability to transfer. The current evidence does not establish 'strong privacy.'","section":"Section 3, Table 1"},{"comment":"The inversion model reconstructs grayscale images only. The paper defines privacy as the inability to retrieve 'textures, text, or fine details'; color is often essential to such content (e.g., documents, signs, clothing). The stated rationale for grayscale reconstruction is to improve cross-dataset generalization, but this choice systematically blinds the attack to color-dependent leakage. An evaluation of color reconstruction, or at least a discussion of why grayscale is a sufficient proxy for the privacy definition, is needed before the privacy-preservation claim can be accepted.","section":"Section 3, Appendix C (attack design)"},{"comment":"The deployment description is ambiguous about which components are exposed to the server. The architecture includes an image encoder Phi that is trained on RGB images (though without photometric loss) and is never probed for invertibility. In 'a cloud-based localization setting, the server stores ... the ppNeSF model'; if Phi is part of that stored model, it is an untested leak channel. If Phi is only client-side, the paper should state this explicitly and justify why the client-side processing is outside the privacy threat model. The privacy evaluation should cover every component that an adversary can access.","section":"Section 4.1 and Section 5.1 (deployment)"},{"comment":"The quantitative gap between the supposedly vulnerable baseline (ZipNeRF-wo-RGB) and ppNeSF is modest: average LPIPS increases from 0.55 to 0.59 and FID from 250 to 322 on mip360, with similar differences on other datasets. Both reconstructions are of rather low quality, suggesting that the attack may be weak even on a model known to contain texture information. There is no positive control showing that the same attack can produce high-fidelity images from a full RGB-supervised NeRF with the color head retained. Without such a control, the observed failure on ppNeSF may reflect attack capacity rather than true privacy. The paper should include a positive-control experiment or otherwise calibrate the attack's sensitivity.","section":"Section 3, Table 1"}],"minor_comments":[{"comment":"References [11] and [12] appear to be the same paper (Chelani et al., 'How Privacy-Preserving Are Line Clouds?') and should be merged or disambiguated.","section":"References"},{"comment":"In the Indoor6 block, the RGB-ppNeSF row for scene6 reads '0.56/0.53/303'; the extra value likely indicates a formatting error in the reported LPIPS/FID pair.","section":"Table 8"},{"comment":"Minor typos and wording issues: 'discriminativeness' in the abstract should likely be 'discriminative' or 'discriminativeness' in a grammatical context; 'Contrarily to' in Section 1 should be 'In contrast to'.","section":"Abstract / Section 1"},{"comment":"The paper does not specify whether code or model weights will be released. Given the privacy claims, releasing the attack implementation and the training pipeline would materially help reviewers and the community reproduce the results.","section":"Section 5.1"}],"recommendation":"major_revision","confidential_remarks":"The localization results are likely sufficient for a publication if the privacy claim is reframed as resistance to a specific bounded adversary. However, the current title and abstract claim 'strong privacy' and 'privacy-preserving' without acknowledging that the attack is a single self-defined, grayscale, cross-scene probe. I would encourage the editor to request a revision that either substantially strengthens the privacy evaluation or softens the privacy claims to match the evidence. The paper's heavy reliance on the authors' own prior work is not problematic per se, but the novelty of the privacy protocol should be positioned more carefully against the acknowledged limitations of the carried-out attack."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: read this for the localization. ppNeSF trains a ZipNeRF backbone with self-supervised segmentation labels instead of RGB, and on 7Scenes, Cambridge, and Indoor6 it consistently beats the prior privacy-preserving methods (SegLoc, GSFF) and stays competitive with non-privacy methods like STDLoc. That is a genuine result, backed by solid ablations and standard benchmarks. The innovation is the OT-based hierarchical labeling scheme that trains a joint 2D/3D embedding from scratch, and the way segmentation uncertainty is used to down-weight ambiguous pixels during pose refinement.\n\nThe attack section is also worth attention. The finding that photometrically trained NeRFs retain texture in the geometry branch even after the color head is removed is plausible, and the extra baseline with a photometric loss added to ppNeSF confirms the mechanism. If you want to make a NeRF-based localizer privacy-preserving, this paper shows you can't just chop off the RGB head.\n\nThe soft spot is the privacy claim itself. The evaluation uses a single inversion model: a U-Net trained on rendered internal features from one set of scenes, tested on other scenes, reconstructing grayscale only. The authors themselves note that nothing constrains internal NIF representations across scenes to share an embedding space, so a cross-dataset failure is unsurprising and doesn't bound what a same-scene or white-box adversary could do. The RGB-trained image encoder Phi is never probed. If it lives only on the client, fine; if the server stores it, that's an untested leak channel. So the phrase 'high level of privacy' is stronger than the evidence. What's demonstrated is robust resistance to a simple transfer attack, not privacy against a serious adversary. Also, no code, no error bars on the privacy metrics; they are consistent across scenes, but still.\n\nWho is this for? The visual localization community, specifically people building privacy-preserving representations. It deserves a serious referee: the localization side is solid, the privacy question is important, and the paper is honest about most of its design choices. I'd send it out, with the expectation that the privacy claims get softened or the attack gets made substantially stronger (same-scene, color-aware, probe Phi). If the authors reframe the claim as 'robust against a simple cross-dataset inversion attack,' the paper would be in much better shape.","headline":"Localization results are the real contribution; the privacy claim rests on a single cross-dataset grayscale attack and needs a stronger adversary before it can be taken literally.","tokens_in":25849,"tokens_out":3441,"would_cite":true,"duration_ms":31684,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RGB-trained NeRFs leak fine scene details even without the color head, and a segmentation-only neural field is proposed to fix it.","keywords":["neural radiance fields","visual localization","privacy-preserving representation","segmentation supervision","self-supervised learning","optimal transport","pose refinement","inversion attack"],"falsifier":"Train an inversion network not on rendered internal features from other datasets but on the image encoder's outputs, or on RGB images of the same scenes, and try to reconstruct text, faces, or readable scene texture from ppNeSF's deployed components. Recovering a recognizable image would refute the privacy claim; likewise, demonstrating that ppNeSF's internal features can be separated into a recoverable appearance channel would refute it.","tokens_in":1671,"feed_emoji":"🔒","tokens_out":1536,"duration_ms":77698,"temperature":0.7,"pith_summary":"Visual localization in the cloud typically needs a server-side scene model, and if the model is a neural radiance field, the server holds more than geometry. The paper shows that NeRFs trained with ordinary RGB reconstruction errors store recognizable texture and fine detail inside the geometry branch itself, so simply deleting the color head after training leaves a privacy leak. The proposed fix, ppNeSF, never trains on images: it learns coarse and fine segmentation labels self-supervised, by aligning 2D image features and 3D rendered features in a shared embedding and assigning labels with optimal transport. The resulting field supports accurate pose refinement by aligning those segmentations, and it beats previous privacy-preserving localization methods while staying close to non-private ones on three benchmarks. If right, this gives a practical way to keep scene-level detail useful for localization without storing what the scene actually looks like.","feed_headline":"RGB-trained NeRFs leak details even without color head","feed_subtitle":"A segmentation-only neural field blocks image recovery while matching non-private localization accuracy on three benchmarks.","key_machinery":"The load-bearing object is the rendered internal feature of the geometry branch—the MLP output shared by density and downstream heads. Because volume rendering alpha-composites those features, they act as a view-dependent image of the scene; the inversion attack reads exactly this and shows that photometric training writes texture there. ppNeSF's counter-machinery is the joint 2D/3D embedding plus optimal-transport labeling: a contrastive loss aligns image-encoder features with features rendered from a temporary feature field, prototypes define a learned class structure, and the Sinkhorn algorithm turns feature-prototype similarities into segmentation targets. The geometry field and segmenta","core_discovery":"Neural fields trained with photometric losses embed image appearance into the feature output of the geometry MLP, the same internal representation shared by NeRF-style models. An inversion network that renders those features and maps them to grayscale images can recover scene structure, shading, and sometimes readable texture even when the color head is removed—so the standard 'train with RGB, then strip the RGB head' answer does not make a NeRF private. ppNeSF instead supervises the field only with segmentation label distributions, obtained by optimal transport between pixels and EMA-updated prototypes in a joint 2D/3D embedding space, with coarse (20) and fine (100) classes and predicted u","pith_inferences":["The deployment still sends client-computed segmentations and a global descriptor derived from raw RGB images; if the image encoder itself were ever exposed or invertible, privacy would depend on that encoder, which the paper's attack does not test.","The privacy claim covers the deployed field, not the training process: training ppNeSF locally requires posed RGB images, and a malicious cloud participant could potentially exploit intermediate feature-field weights or gradients during that phase.","The class count is effectively a privacy knob: more classes improve localization but likely increase recoverable information, and the paper measures the trade-off only for localization accuracy, not privacy.","The same segmentation-supervision recipe could transfer to explicit dense representations such as Gaussian splatting, letting other non-photometric localization pipelines inherit a similar privacy property."],"forward_implications":["Removing the RGB head after training does not protect a NeRF; any NeRF-based localization system trained with a photometric loss inherits the exposure.","A neural field can be trained for localization without ever being shown an image: segmentation labels from optimal transport over a joint 2D/3D embedding supply enough supervision for geometry.","Segmentation-based pose refinement is accurate enough to outperform prior privacy-preserving methods and stay competitive with non-private feature-rendering baselines on 7Scenes, Cambridge Landmarks, and Indoor6.","The new privacy evaluation protocol—inversion attack plus perceptual metrics and vision-language caption similarity—offers a reusable way to compare how much scene detail any implicit field stores.","Cloud localization with ppNeSF can be served by sending only coarse-to-fine segmentation maps and a global descriptor, rather than features that carry appearance."],"supporting_citations":[{"why":"Supplies the ZipNeRF geometry field architecture and anti-aliased volume rendering that ppNeSF builds on.","marker":"[4]"},{"why":"Defines the NeRF/volume-rendering framework whose internal geometry representation is analysed by the privacy attack.","marker":"[50]"},{"why":"Prior segmentation-based privacy-preserving localization; provides the non-injective segmentation argument and a pose-refinement baseline.","marker":"[58]"},{"why":"Main privacy-preserving baseline the paper compares against and outperforms on multiple datasets.","marker":"[60]"},{"why":"Establishes the inversion-attack methodology for reconstructing images from scene representations, which the paper adapts to implicit fields.","marker":"[61]"},{"why":"Supplies the vision-language model used to measure semantic similarity between original and reconstructed images in the privacy protocol.","marker":"[47]"},{"why":"Provides the Sinkhorn optimal-transport algorithm used to derive segmentation label targets during self-supervised training.","marker":"[19]"},{"why":"Prior self-supervised neural implicit feature field for pose refinement, serving as both related work and a non-private baseline.","marker":"[59]"},{"why":"Supplies the contrastive objective used to align 2D and 3D features in the shared embedding space.","marker":"[77]"}],"fun_headline_variants":["NeRF geometry retains recoverable scene details","No color head? NeRF still leaks private details","Segmentation-only NeRF matches accuracy, blocks leaks","Stripping color head fails to make NeRF private"],"cache_read_input_tokens":27648,"weakest_assumption_plain":"The privacy guarantee is only as strong as the bounded attacker the paper builds: if a stronger inversion model—trained on the same scenes or on RGB data from the same pipeline—can decode recognizable details from ppNeSF's weights, features, or client-side encoder, the central claim fails.","fun_headline_variants_meta":{"raw":{"variants":["NeRF geometry retains recoverable scene details","No color head? NeRF still leaks private details","Segmentation-only NeRF matches accuracy, blocks leaks","Stripping color head fails to make NeRF private"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000173,"raw_usage":{"total_tokens":1119,"prompt_tokens":748,"completion_tokens":371,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":492,"completion_tokens_details":{"reasoning_tokens":309}},"tokens_in":492,"tokens_out":371,"duration_ms":4800,"temperature":1.0,"reasoning_tokens":309,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T16:02:43.056568+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train an inversion network not on rendered internal features from other datasets but on the image encoder's outputs, or on RGB images of the same scenes, and try to reconstruct text, faces, or readable scene texture from ppNeSF's deployed components. Recovering a recognizable image would refute the privacy claim; likewise, demonstrating that ppNeSF's internal features can be separated into a recoverable appearance channel would refute it.","supporting_citations":[{"cited_title":"Srinivasan, Matthew Tancik, Jonathan T","cited_arxiv_id":null,"evidence_quote":"Defines the NeRF/volume-rendering framework whose internal geometry representation is analysed by the privacy attack."},{"cited_title":"SegLoc: Learn- ing Segmentation-Based Representations for Privacy- Preserving Visual Localization","cited_arxiv_id":null,"evidence_quote":"Prior segmentation-based privacy-preserving localization; provides the non-injective segmentation argument and a pose-refinement baseline."},{"cited_title":"Gaussian Splatting Feature Fields for (Privacy- Preserving) Visual Localization","cited_arxiv_id":null,"evidence_quote":"Main privacy-preserving baseline the paper compares against and outperforms on multiple datasets."},{"cited_title":"Koppal, Sing Bing Kang, and Sudipta N","cited_arxiv_id":null,"evidence_quote":"Establishes the inversion-attack methodology for reconstructing images from scene representations, which the paper adapts to implicit fields."},{"cited_title":"Visual Instruction Tuning","cited_arxiv_id":null,"evidence_quote":"Supplies the vision-language model used to measure semantic similarity between original and reconstructed images in the privacy protocol."},{"cited_title":"Self-Supervised Learning of Neural Implicit Feature Fields for Camera Pose Re- finement","cited_arxiv_id":null,"evidence_quote":"Prior self-supervised neural implicit feature field for pose refinement, serving as both related work and a non-private baseline."}],"review_version":1}