Pith. sign in

REVIEW 4 major objections 4 minor 100 references

Can we make NeRF-based visual localization privacy-preserving?

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RGB-trained NeRFs leak fine scene details even without the color head, and a segmentation-only neural field is proposed to fix it.

desk verdict Localization results are the real contribution; the privacy claim rests on a single cross-dataset grayscale attack and needs a stronger adversary before it can be taken literally. read the letter →

arxiv 2508.18971 v1 pith:KE6S7DZT submitted 2025-08-26 cs.CV

Maxime Pietrantoni , Martin Humenberger , Torsten Sattler , Gabriela Csurka This is my paper · ORCID
classification cs.CV
keywords neuralradiancefieldsvisuallocalizationprivacy-preservingrepresentationsegmentationsupervisionself-supervisedlearningoptimaltransportposerefinementinversionattack
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Visual localization in the cloud typically needs a server-side scene model, and if the model is a neural radiance field, the server holds more than geometry. The paper shows that NeRFs trained with ordinary RGB reconstruction errors store recognizable texture and fine detail inside the geometry branch itself, so simply deleting the color head after training leaves a privacy leak. The proposed fix, ppNeSF, never trains on images: it learns coarse and fine segmentation labels self-supervised, by aligning 2D image features and 3D rendered features in a shared embedding and assigning labels with optimal transport. The resulting field supports accurate pose refinement by aligning those segmentations, and it beats previous privacy-preserving localization methods while staying close to non-private ones on three benchmarks. If right, this gives a practical way to keep scene-level detail useful for localization without storing what the scene actually looks like.

What carries the argument

The load-bearing object is the rendered internal feature of the geometry branch—the MLP output shared by density and downstream heads. Because volume rendering alpha-composites those features, they act as a view-dependent image of the scene; the inversion attack reads exactly this and shows that photometric training writes texture there. ppNeSF's counter-machinery is the joint 2D/3D embedding plus optimal-transport labeling: a contrastive loss aligns image-encoder features with features rendered from a temporary feature field, prototypes define a learned class structure, and the Sinkhorn algorithm turns feature-prototype similarities into segmentation targets. The geometry field and segmenta

What would settle it

Train an inversion network not on rendered internal features from other datasets but on the image encoder's outputs, or on RGB images of the same scenes, and try to reconstruct text, faces, or readable scene texture from ppNeSF's deployed components. Recovering a recognizable image would refute the privacy claim; likewise, demonstrating that ppNeSF's internal features can be separated into a recoverable appearance channel would refute it.

Watch

Extended reading notes

Core claim

Neural fields trained with photometric losses embed image appearance into the feature output of the geometry MLP, the same internal representation shared by NeRF-style models. An inversion network that renders those features and maps them to grayscale images can recover scene structure, shading, and sometimes readable texture even when the color head is removed—so the standard 'train with RGB, then strip the RGB head' answer does not make a NeRF private. ppNeSF instead supervises the field only with segmentation label distributions, obtained by optimal transport between pixels and EMA-updated prototypes in a joint 2D/3D embedding space, with coarse (20) and fine (100) classes and predicted u

Load-bearing premise

The privacy guarantee is only as strong as the bounded attacker the paper builds: if a stronger inversion model—trained on the same scenes or on RGB data from the same pipeline—can decode recognizable details from ppNeSF's weights, features, or client-side encoder, the central claim fails.

Editorial extensions

If this is right

  • Removing the RGB head after training does not protect a NeRF; any NeRF-based localization system trained with a photometric loss inherits the exposure.
  • A neural field can be trained for localization without ever being shown an image: segmentation labels from optimal transport over a joint 2D/3D embedding supply enough supervision for geometry.
  • Segmentation-based pose refinement is accurate enough to outperform prior privacy-preserving methods and stay competitive with non-private feature-rendering baselines on 7Scenes, Cambridge Landmarks, and Indoor6.
  • The new privacy evaluation protocol—inversion attack plus perceptual metrics and vision-language caption similarity—offers a reusable way to compare how much scene detail any implicit field stores.
  • Cloud localization with ppNeSF can be served by sending only coarse-to-fine segmentation maps and a global descriptor, rather than features that carry appearance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The deployment still sends client-computed segmentations and a global descriptor derived from raw RGB images; if the image encoder itself were ever exposed or invertible, privacy would depend on that encoder, which the paper's attack does not test.
  • The privacy claim covers the deployed field, not the training process: training ppNeSF locally requires posed RGB images, and a malicious cloud participant could potentially exploit intermediate feature-field weights or gradients during that phase.
  • The class count is effectively a privacy knob: more classes improve localization but likely increase recoverable information, and the paper measures the trade-off only for localization accuracy, not privacy.
  • The same segmentation-supervision recipe could transfer to explicit dense representations such as Gaussian splatting, letting other non-photometric localization pipelines inherit a similar privacy property.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper addresses privacy in NeRF-based visual localization. It first proposes a privacy attack and evaluation protocol for neural implicit fields (NIFs), arguing that NeRFs trained with photometric/RGB losses store fine-grained scene details in their geometry branch, so removing the color head is insufficient. It then introduces ppNeSF, a neural segmentation field trained with self-supervised segmentation labels derived via an optimal-transport labeling scheme, and shows that the resulting segmentation space supports accurate pose refinement. Localization experiments on 7Scenes, Cambridge Landmarks, and Indoor6 report competitive or state-of-the-art results among privacy-preserving baselines, with ablations and training details in the appendix.

Significance. If the privacy claim were fully established, the paper would be a valuable first step toward privacy-preserving neural implicit localization, replacing RGB supervision with learned segmentation targets. The localization contribution is technically solid: it uses standard benchmarks, compares against relevant privacy-preserving baselines, includes ablations, and reports consistent gains in most scenes. The proposed privacy protocol is also creative, using VLM-based semantic similarity in addition to perceptual metrics. However, the privacy evidence is currently based on a single bounded-adversary attack, with no error bars, no independent corroboration, and with at least one acknowledged limitation that undermines the strength of the conclusion. The central privacy claim therefore needs substantial additional support before the paper can be accepted as it stands.

major comments (4)
  1. [Section 3, Table 1] The privacy claim rests entirely on a single cross-scene inversion attack. The paper itself states: 'nothing constrains the internal representations of NIFs trained on different scenes ... to lie in a common embedding space. As such, there are no guarantees that the inversion model can generalize to NIFs trained on scenes unseen during training.' This is precisely the scenario used to conclude that ppNeSF is privacy-preserving. A cross-scene attack can fail because of representation shift even when a scene-specific adversary could extract fine details. To support the central claim, the authors should report same-scene/white-box attacks, attacks trained per scene, or an explicit argument why the cross-scene failure is indicative of the information content rather than the attack's inability to transfer. The current evidence does not establish 'strong privacy.'
  2. [Section 3, Appendix C (attack design)] The inversion model reconstructs grayscale images only. The paper defines privacy as the inability to retrieve 'textures, text, or fine details'; color is often essential to such content (e.g., documents, signs, clothing). The stated rationale for grayscale reconstruction is to improve cross-dataset generalization, but this choice systematically blinds the attack to color-dependent leakage. An evaluation of color reconstruction, or at least a discussion of why grayscale is a sufficient proxy for the privacy definition, is needed before the privacy-preservation claim can be accepted.
  3. [Section 4.1 and Section 5.1 (deployment)] The deployment description is ambiguous about which components are exposed to the server. The architecture includes an image encoder Phi that is trained on RGB images (though without photometric loss) and is never probed for invertibility. In 'a cloud-based localization setting, the server stores ... the ppNeSF model'; if Phi is part of that stored model, it is an untested leak channel. If Phi is only client-side, the paper should state this explicitly and justify why the client-side processing is outside the privacy threat model. The privacy evaluation should cover every component that an adversary can access.
  4. [Section 3, Table 1] The quantitative gap between the supposedly vulnerable baseline (ZipNeRF-wo-RGB) and ppNeSF is modest: average LPIPS increases from 0.55 to 0.59 and FID from 250 to 322 on mip360, with similar differences on other datasets. Both reconstructions are of rather low quality, suggesting that the attack may be weak even on a model known to contain texture information. There is no positive control showing that the same attack can produce high-fidelity images from a full RGB-supervised NeRF with the color head retained. Without such a control, the observed failure on ppNeSF may reflect attack capacity rather than true privacy. The paper should include a positive-control experiment or otherwise calibrate the attack's sensitivity.
minor comments (4)
  1. [References] References [11] and [12] appear to be the same paper (Chelani et al., 'How Privacy-Preserving Are Line Clouds?') and should be merged or disambiguated.
  2. [Table 8] In the Indoor6 block, the RGB-ppNeSF row for scene6 reads '0.56/0.53/303'; the extra value likely indicates a formatting error in the reported LPIPS/FID pair.
  3. [Abstract / Section 1] Minor typos and wording issues: 'discriminativeness' in the abstract should likely be 'discriminative' or 'discriminativeness' in a grammatical context; 'Contrarily to' in Section 1 should be 'In contrast to'.
  4. [Section 5.1] The paper does not specify whether code or model weights will be released. Given the privacy claims, releasing the attack implementation and the training pipeline would materially help reviewers and the community reproduce the results.

Circularity Check

0 steps flagged · score 2.0 of 10

No construction-level circularity: localization claims are externally benchmarked; privacy claim uses a self-designed attack and a supportive same-group citation, but no result is forced by definition.

full rationale

The central localization claim of ppNeSF is validated against public benchmarks (7Scenes, Cambridge Landmarks, Indoor6) with external baselines, so the pose accuracy results are independent of the training objective's inputs. The privacy claim, however, rests on the authors' own inversion attack and protocol (Sec. 3 and App. C), which is a bounded-adversary evaluation rather than a circular reduction: the same protocol yields higher-quality reconstructions from ZipNeRF-wo-RGB than from ppNeSF, giving the comparison empirical content. The paper itself concedes that per-scene NIFs have no common embedding space, which weakens the transferability of the attack but does not make the privacy conclusion an identity with the inputs. The only notable same-group dependency is App. C.2's reliance on [58] for the privacy of segmentation maps, but that is a separate published result and is supplemented by original experiments (Table 9). No equation equates the reported predictions to fitted parameters, and no result is imported as a 'uniqueness theorem' from the authors' prior work. Overall, the derivation chain is not circular; the moderate concern is limited generality of the privacy protocol, not construction-level circularity.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central claims rest on a modest set of conceptual assumptions about what constitutes privacy and about the strength of the inversion attack, plus a large number of empirically tuned hyperparameters. No formal guarantees are provided.

free parameters (4)
  • Number of segmentation classes (K, Kf) = K=20 coarse, Kf=100 fine
    Chosen empirically as a trade-off between localization accuracy and training cost (Sec. 5, Fig. 6). Ablation shows accuracy is sensitive below ~50-100 classes.
  • EMA momentum (mu) and beta schedule = not reported numerically; beta scheduled 0 to 0.5
    Prototype update hyperparameters (Sec. 4.2.1); values are not reported, affecting segmentation target stability.
  • Loss weights (depth, distortion, interlevel, NCE, CE coarse, CE fine, hierarchical) = 2, 0.5, 0.1, 0.2, 0.2, 0.2, 0.05
    Algorithm 1; hand-tuned, no sensitivity analysis except the No L_hierar ablation.
  • Training iterations and ray samples = 50k iterations, 4096 rays (7Scenes/Indoor6) or 8192 (Cambridge)
    Training budget and sampling density are tuned for performance; no ablation on these values.
assumptions (6)
  • domain assumption Privacy is defined as the inability to retrieve personally identifiable information (text, fine details); broad semantics/geometry are not a privacy breach.
    Sec. 1; this framing is inherited from prior work [11,58,84,95].
  • domain assumption Segmentation labels are non-injective mappings from RGB pixels to classes and therefore hinder inversion.
    Sec. 4, citing [58]; empirical support in Table 9 but assumed as a design principle.
  • domain assumption The geometry feature vector (pre-color MLP) is a shared component across NeRF architectures whose inversion reveals stored details.
    Sec. 3; demonstrated for ZipNeRF but generalized to all NeRF-based localization methods [18,53,59,96].
  • ad hoc to paper The inversion model trained on rendered features from one set of scenes generalizes to unseen scenes/datasets.
    Appendix C; the attack's strength is bounded by this generalization assumption, and no cross-architecture test is done.
  • domain assumption The image encoder, trained on RGB images, does not itself leak privacy when deployed client-side.
    Section 4.1 and privacy evaluation only attack the 3D field internals; the 2D encoder is not probed.
  • domain assumption Depth supervision from external monocular estimators provides geometry priors without leaking texture.
    Appendix B; depth maps are used as supervision, and it is assumed they do not carry texture/private info.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Can we make NeRF-based visual localization privacy-preserving?." pith.science (2026). https://pith.science/paper/KE6S7DZT

@misc{pith2026250818971,
  author       = {Pith},
  title        = {Pith review of: Can we make NeRF-based visual localization privacy-preserving?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KE6S7DZT}},
  note         = {Machine review of arXiv:2508.18971}
}
read the original abstract

Visual localization (VL) is the task of estimating the camera pose in a known scene. VL methods, a.o., can be distinguished based on how they represent the scene, e.g., explicitly through a (sparse) point cloud or a collection of images or implicitly through the weights of a neural network. Recently, NeRF-based methods have become popular for VL. While NeRFs offer high-quality novel view synthesis, they inadvertently encode fine scene details, raising privacy concerns when deployed in cloud-based localization services as sensitive information could be recovered. In this paper, we tackle this challenge on two ends. We first propose a new protocol to assess privacy-preservation of NeRF-based representations. We show that NeRFs trained with photometric losses store fine-grained details in their geometry representations, making them vulnerable to privacy attacks, even if the head that predicts colors is removed. Second, we propose ppNeSF (Privacy-Preserving Neural Segmentation Field), a NeRF variant trained with segmentation supervision instead of RGB images. These segmentation labels are learned in a self-supervised manner, ensuring they are coarse enough to obscure identifiable scene details while remaining discriminativeness in 3D. The segmentation space of ppNeSF can be used for accurate visual localization, yielding state-of-the-art results.

Figures

Figures reproduced from arXiv: 2508.18971 by the authors.

Figure 1
Figure 1. NeRFs optimized with photometric losses store fine-grained scene details, making them susceptible [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Left to right: ground truth image and images reconstructed from ZipNeRF-wo-RGB and ppNeSF via our [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The ppNeSF architecture. Given an input image and its associated input pose (𝑅, 𝑡), we extract image-based features and segmentations from the Image Encoder Φ, render volumetric features from the feature field Γ and segmentations from Ω. The 2D and 3D features are aligned through hierarchical contrastive losses (top). Segmentation targets Q are derived through optimal transport based on feature/prototype similaritie… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: From left to right: original image, rendered depth, coarse image-based segmentation [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: From left to right: original image, rendered depth, coarse image-based segmentation [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Refinement with different number 𝐾𝑓 = 𝑛 ∗ 𝐾 of fine classes (varying number 𝐾 of coarse classes, 𝑛 = 5 fine classes per coarse class). The green curve corresponds to the translation and the blue curve to the rotation error. [38], the initial learning rate set to 1e-3 a…
Figure 7
Figure 7. Figure 7: From left to right: Ground truth image, image reconstructed from ZipNeRF-wo-RGB, image reconstructed [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Left to right: GT image, images reconstructed from ZipNeRF-wo-RGB/ ppNeSF/ Q-Segs / ppNeSF+R￾Segs. Segmentations provide little additional information. LPIPS(↑)/ FID (↑) / Captions similarity (↓) Model ZipNeRF-wo-RGB PPNeSF Q-Segs PPNeSF+R-Segs Metrics 0.52 / 298 / 0.5…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

100 extracted references · 75 canonical work pages

  1. [1]

    Pho- tometric Bundle Adjustment for Vision-Based SLAM

    Hatem Alismail, Brett Browning, and Simon Lucey. Pho- tometric Bundle Adjustment for Vision-Based SLAM. In ACCV, 2017. 3

    2017

  2. [2]

    Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P

    Jonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P. Srinivasan. Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields. InICCV, 2021. 13

    2021

  3. [3]

    Mip-nerf 360: Un- bounded anti-aliased neural radiance fields

    JonathanTBarron,BenMildenhall,DorVerbin,PratulP Srinivasan, and Peter Hedman. Mip-nerf 360: Un- bounded anti-aliased neural radiance fields. InCVPR,

  4. [4]

    Barron, Ben Mildenhall, Dor Verbin, Pratul P

    Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields. In ICCV, 2023. 4, 5, 13, 14, 15

    2023

  5. [5]

    Zoedepth: Zero-shot transfer by combining relative and metric depth.arXiv preprint arXiv:2302.12288, 2023

    Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias Müller. Zoedepth: Zero-shot transfer by combining relative and metric depth.arXiv preprint arXiv:2302.12288, 2023. 15

    arXiv 2023

  6. [6]

    Cali- brated and Partially Calibrated Semi-Generalized Ho- mographies

    Snehal Bhayani, Torsten Sattler, Daniel Barath, Patrik Beliansky, Janne Heikkilä, and Zuzana Kukelova. Cali- brated and Partially Calibrated Semi-Generalized Ho- mographies. In ICCV, 2021. 2

    2021

  7. [7]

    Visual Camera Re- Localization from RGB and RGB-D Images Using DSAC

    Eric Brachmann and Carsten Rother. Visual Camera Re- Localization from RGB and RGB-D Images Using DSAC. TPAMI, 44(9):5847–5865, 2021. 8, 9

    2021

  8. [8]

    DSAC - Differentiable RANSAC for Cam- era Localization

    Eric Brachmann, Alexander Krull, Sebastian Nowozin, Jamie Shotton, Frank Michel, Stefan Gumhold, and Carsten Rother. DSAC - Differentiable RANSAC for Cam- era Localization. InCVPR, 2017. 2

    2017

Show all 100 references
  1. [9]

    On the Limits of Pseudo Ground Truth in Visual Camera Re-Localisation

    Eric Brachmann, Martin Humenberger, Carsten Rother, and Torsten Sattler. On the Limits of Pseudo Ground Truth in Visual Camera Re-Localisation. InICCV, 2021. 8

    2021

  2. [10]

    Prentice Hall, 1963

    Robert Goodell Brown.Smoothing Forecasting and Pre- diction of Discrete Time Series. Prentice Hall, 1963. 7

    1963

  3. [12]

    How Privacy-Preserving Are Line Clouds? Recovering Scene Details From 3D Lines

    Kunal Chelani, Fredrik Kahl, and Torsten Sattler. How Privacy-Preserving Are Line Clouds? Recovering Scene Details From 3D Lines. InCVPR, 2021. 2

    2021

  4. [13]

    Obfuscation Based Pri- vacy Preserving Representations are Recoverable Using Neighborhood Information

    Kunal Chelani, Assia Benbihi, Fredrik Kahl, Torsten Sattler, and Zuzana Kukelova. Obfuscation Based Pri- vacy Preserving Representations are Recoverable Using Neighborhood Information. In3DV, 2025. 2

    2025

  5. [14]

    TensoRF: Tensorial Radiance Fields

    Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. TensoRF: Tensorial Radiance Fields. InECCV,

  6. [15]

    FactorFields: AUnifiedFramework for Neural Fields and Beyond

    AnpeiChen,ZexiangXu,XinyueWei,SiyuTang,HaoSu, andAndreasGeiger. FactorFields: AUnifiedFramework for Neural Fields and Beyond. arXiv:2302.01226, 2023. 13

    2023 arXiv

  7. [16]

    Direct- PoseNet: Absolute Pose Regression with Photometric Consistency

    ShuaiChen,ZiruiWang,andVictorA.Prisacariu. Direct- PoseNet: Absolute Pose Regression with Photometric Consistency. In 3DV, 2021. 2

    2021

  8. [17]

    Prisacariu

    Shuai Chen, Xinghui Li, Zirui Wang, and Victor A. Prisacariu. DFNet: Enhance Absolute Pose Regression with Direct Feature Matching. InECCV, 2022. 2, 8

    2022

  9. [18]

    Prisacariu

    Shuai Chen, Yash Bhalgat, Xinghui Li, Jiawang Bian, Kejie Li, Zirui Wang, and Victor A. Prisacariu. Neural Refinement for Absolute Pose Regression with Feature Synthesis. In CVPR, 2024. 2, 3, 5, 8, 9

    2024

  10. [19]

    Sinkhorn Distances: Lightspeed Com- putation of Optimal Transport

    Marco Cuturi. Sinkhorn Distances: Lightspeed Com- putation of Optimal Transport. InNeurIPS, 2013. 2, 7

    2013

  11. [20]

    Learning To Detect Scene Land- marks for Camera Localization

    Tien Do, Ondrej Miksik, Joseph DeGol, Hyun Soo Park, and Sudipta N Sinha. Learning To Detect Scene Land- marks for Camera Localization. InCVPR, 2022. 4, 8

    2022

  12. [21]

    Schönberger, Sudipta N

    Mihai Dusmanu, Johannes L. Schönberger, Sudipta N. Sinha, and Marc Pollefeys. Privacy-Preserving Image Features via Adversarial Affine Subspace Embeddings. In CVPR, 2021. 2

    2021

  13. [22]

    LSD- SLAM: Large-scale Direct Monocular SLAM

    Jakob Engel, Thomas Schöps, and Daniel Cremers. LSD- SLAM: Large-scale Direct Monocular SLAM. InECCV,

  14. [23]

    Direct Sparse Odometry

    JakobEngel,VladlenKoltun,andDanielCremers. Direct Sparse Odometry. TPAMI, 40(3):611–625, 2017. 3

    2017

  15. [24]

    Plenox- els: Radiance Fields without Neural Networks

    Sara Fridovich-Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenox- els: Radiance Fields without Neural Networks. InCVPR,

  16. [25]

    Schönberger, and Marc Pollefeys

    Marcel Geppert, Viktor Larsson, Pablo Speciale, Jo- hannes L. Schönberger, and Marc Pollefeys. Privacy Preserving Structure-from-Motion. InECCV, 2020. 2

    2020

  17. [26]

    Schönberger, and Marc Pollefeys

    Marcel Geppert, Viktor Larsson, Pablo Speciale, Jo- hannes L. Schönberger, and Marc Pollefeys. Privacy Pre- serving Localization and Mapping From Uncalibrated Cameras. In CVPR, 2021. 2

    2021

  18. [27]

    Schön- berger, and Marc Pollefeys

    Marcel Geppert, Viktor Larsson, Johannes L. Schön- berger, and Marc Pollefeys. Privacy Preserving Partial Localization. In CVPR, 2022. 2

    2022

  19. [28]

    Fea- ture Query Networks: Neural Surface Description for Camera Pose Refinement

    Hugo Germain, Daniel DeTone, Geoffrey Pascoe, Tan- ner Schmidt, David Novotny, Richard Newcombe, Chris Sweeney, Richard Szeliski, and Vasileios Balntas. Fea- ture Query Networks: Neural Surface Description for Camera Pose Refinement. InCVPR Workshops , 2022. 3, 8

    2022

  20. [29]

    KeyBERT: Minimal Keyword Extraction with BERT

    Maarten Grootendorst. KeyBERT: Minimal Keyword Extraction with BERT. zenodo.4461265, 2020. 4

    2020

  21. [30]

    Project AutoVi- sion: Localization and 3D Scene Perception for an Au- tonomous Vehicle with a Multi-Camera System

    Lionel Heng, Benjamin Choi, Zhaopeng Cui, Marcel Geppert, Sixing Hu, Benson Kuan, Peidong Liu, Rang Nguyen, Ye Chuan Yeo, Andreas Geiger, Gim Hee Lee, Marc Pollefeys, and Torsten Sattler. Project AutoVi- sion: Localization and 3D Scene Perception for an Au- tonomous Vehicle wi...

  22. [31]

    GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. InNeurIPS, 2017. 4

    2017

  23. [32]

    From Sparse to Dense: Camera Re- localization with Scene-Specific Detector from Feature Gaussian Splatting

    Zhiwei Huang, Hailin Yu, Yichun Shentu, Jin Yuan, and Guofeng Zhang. From Sparse to Dense: Camera Re- localization with Scene-Specific Detector from Feature Gaussian Splatting. InCVPR, 2025. 3, 8, 9

    2025

  24. [33]

    Investigating the Role of Image Retrieval for Visual Localization.IJCV, 130(7):1811–1836, 2022

    Martin Humenberger, Yohann Cabon, Noé Pion, Philippe Weinzaepfel, Donghwan Lee, Nicolas Guérin, Torsten Sattler, and Gabriela Csurka. Investigating the Role of Image Retrieval for Visual Localization.IJCV, 130(7):1811–1836, 2022. 2

    2022

  25. [34]

    Kajiya and Brian P

    James T. Kajiya and Brian P. Von Herzen. Ray Tracing Volume Densities. ACM SIGGRAPH Computer Graphics, 18(3):0097–8930, 1984. 13

    1984

  26. [35]

    Re- purposingDiffusion-BasedImageGeneratorsforMonoc- ular Depth Estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, RodrigoCayeDaudt, andKonradSchindler. Re- purposingDiffusion-BasedImageGeneratorsforMonoc- ular Depth Estimation. InCVPR, 2024. 15

    2024

  27. [36]

    What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? In NeurIPS, 2017

    Alex Kendall and Yarin Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? In NeurIPS, 2017. 7

    2017

  28. [37]

    PoseNet: a Convolutional Network for Real-Time 6-DOF Camera Relocalization

    Alex Kendall, Matthew Grimes, and Roberto Cipolla. PoseNet: a Convolutional Network for Real-Time 6-DOF Camera Relocalization. InICCV, 2015. 2, 8

    2015

  29. [38]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. InICLR, 2015. 17

    2015

  30. [39]

    Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980,

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980,

    arXiv

  31. [40]

    Identity-Obscured Neural Radiance Fields: Privacy-Preserving 3D Facial Reconstruction

    Jiayi Kong, Baixin Xu, Xurui Song, Chen Qian, Jun Luo, and Ying He. Identity-Obscured Neural Radiance Fields: Privacy-Preserving 3D Facial Reconstruction. arXiv preprint arXiv:2312.04106, 2023. 2

    2023 arXiv

  32. [41]

    Paired-Point Lifting for Enhanced Privacy-Preserving Visual Localization

    Chunghwan Lee, Jaihoon Kim, Chanhyuk Yun, and Je Hyeong Hong. Paired-Point Lifting for Enhanced Privacy-Preserving Visual Localization. InCVPR, 2023. 2

    2023

  33. [42]

    Sinha, Michael F

    Hyon Lim, Sudipta N. Sinha, Michael F. Cohen, Matt Uyttendaele, and H. Jin Kim. Real-time Monocular Image-based 6-DoF Localization.International Journal of Robotics Research, 34(4–5):476–492, 2015. 1

    2015

  34. [43]

    Feature pyra- mid networks for object detection

    Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyra- mid networks for object detection. InCVPR, 2017. 13

    2017

  35. [44]

    Vela, and Stan Birchfield

    Yunzhi Lin, Thomas Müller, Jonathan Tremblay, Bowen Wen, Stephen Tyree, Alex Evans, Patricio A. Vela, and Stan Birchfield. Parallel Inversion of Neural Radiance Fields for Robust Pose Estimation. InICRA, 2023. 3

    2023

  36. [45]

    Pixel-Perfect Structure-from- Motion with Featuremetric Refinement

    Philipp Lindenberger, Paul-Edouard Sarlin, Viktor Lars- son, and Marc Pollefeys. Pixel-Perfect Structure-from- Motion with Featuremetric Refinement. InICCV, 2021. 2, 3

    2021

  37. [46]

    GS-CPR: Efficient Camera Pose Refine- ment via 3D Gaussian Splatting

    Changkun Liu, Shuai Chen, Yash Bhalgat, Siyan Hu, Zirui Wang, Ming Cheng, Victor Adrian Prisacariu, and Tristan Braud. GS-CPR: Efficient Camera Pose Refine- ment via 3D Gaussian Splatting. InICLR, 2025. 3, 8, 9

    2025

  38. [47]

    Visual Instruction Tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual Instruction Tuning. InNeurIPS, 2024. 2, 3, 4

    2024

  39. [48]

    Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. InICCV, 2021. 13

    2021

  40. [49]

    Distance-Based Image Classifica- tion: Generalizing to New Classes at near Zero Cost

    Thomas Mensink, Jakob Verbeek, Florent Perronnin, and Gabriela Csurka. Distance-Based Image Classifica- tion: Generalizing to New Classes at near Zero Cost. TPAMI, 35(11):2624 –2637, 2013. 7

    2013

  41. [50]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. InECCV, 2020. 2, 4, 13

    2020

  42. [51]

    Efficient Privacy-Preserving Visual Localization Using 3D Ray Clouds

    Heejoon Moon, Chunghwan Lee, and Je Hyeong Hong. Efficient Privacy-Preserving Visual Localization Using 3D Ray Clouds. InCVPR, 2024. 2

    2024

  43. [52]

    LENS: Localization Enhanced by NeRF Synthesis

    Arthur Moreau, Nathan Piasco, Dzmitry Tsishkou, Bog- dan Stanciulescu, and Arnaud e La Fortelle. LENS: Localization Enhanced by NeRF Synthesis. In CoRL,

  44. [53]

    CROSSFIRE: Camera Relocalization on Self- Supervised Features from an Implicit Representation

    Arthur Moreau, Nathan Piasco, Moussab Bennehar, Dzmitry Tsishkou, Bogdan Stanciulescu, and Arnaud de La Fortelle. CROSSFIRE: Camera Relocalization on Self- Supervised Features from an Implicit Representation. In ICCV, 2023. 2, 3, 5, 8, 9

    2023

  45. [54]

    Instant Neural Graphics Primitives with a Multiresolution Hash Encoding.ACM Transac- tions on Graphics, 41(4):1–15, 2022

    Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant Neural Graphics Primitives with a Multiresolution Hash Encoding.ACM Transac- tions on Graphics, 41(4):1–15, 2022. 13

    2022

  46. [55]

    Mip-Grid: Anti-aliased Grid Representa- tions for Neural Radiance Fields

    Seungtae Nam, Daniel Rho, Jong Hwan Ko, and Eun- byung Park. Mip-Grid: Anti-aliased Grid Representa- tions for Neural Radiance Fields. In NeurIPS, 2023. 13

    2023

  47. [56]

    RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs

    Michael Niemeyer, Jonathan T Barron, Ben Mildenhall, Mehdi SM Sajjadi, Andreas Geiger, and Noha Radwan. RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs. InCVPR, 2022. 13

    2022

  48. [57]

    Privacy Preserving Localization via Coordinate Permutations

    Linfei Pan, Johannes L Schönberger, Viktor Larsson, and Marc Pollefeys. Privacy Preserving Localization via Coordinate Permutations. InICCV, 2023. 2

    2023

  49. [58]

    SegLoc: Learn- ing Segmentation-Based Representations for Privacy- Preserving Visual Localization

    Maxime Pietrantoni, Martin Humenberger, Torsten Sattler, and Gabriela Csurka. SegLoc: Learn- ing Segmentation-Based Representations for Privacy- Preserving Visual Localization. InCVPR, 2023. 2, 3, 8, 9, 19

    2023

  50. [59]

    Self-Supervised Learning of Neural Implicit Feature Fields for Camera Pose Re- finement

    Maxime Pietrantoni, Gabriela Csurka, Martin Humen- berger, and Torsten Sattler. Self-Supervised Learning of Neural Implicit Feature Fields for Camera Pose Re- finement. In 3DV, 2024. 2, 3, 5, 8, 9

    2024

  51. [60]

    Gaussian Splatting Feature Fields for (Privacy- Preserving) Visual Localization

    Maxime Pietrantoni, Gabriela Csurka, and Torsten Sat- tler. Gaussian Splatting Feature Fields for (Privacy- Preserving) Visual Localization. InCVPR, 2025. 2, 3, 8, 9 11 Can we make NeRF-based visual localization privacy-preserving?

    2025

  52. [61]

    Koppal, Sing Bing Kang, and Sudipta N

    Francesco Pittaluga, Sanjeev J. Koppal, Sing Bing Kang, and Sudipta N. Sinha. Revealing Scenes by Inverting Structure from Motion Reconstructions. InCVPR, 2019. 2

    2019

  53. [62]

    U-Net: Convolutional Networks for Biomedical Image Segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional Networks for Biomedical Image Segmentation. In MICCAI, 2015. 16

    2015

  54. [63]

    From Coarse to Fine: Robust Hierarchical Localization at Large Scale

    Paul-Edouard Sarlin, Cesar Cadena, Roland Siegwart, and Marcin Dymczyk. From Coarse to Fine: Robust Hierarchical Localization at Large Scale. InCVPR, 2019. 2, 8, 9

    2019

  55. [64]

    Back to the Feature: Learning Robust Camera Localization From Pixels To Pose

    Paul-Edouard Sarlin, Ajaykumar Unagar, Mans Larsson, HugoGermain,CarlToft,ViktorLarsson,MarcPollefeys, Vincent Lepetit, Lars Hammarstrand, Fredrik Kahl, and Torsten Sattler. Back to the Feature: Learning Robust Camera Localization From Pixels To Pose. In CVPR,

  56. [65]

    Hyperpoints and Fine Vocabularies for Large-Scale Location Recognition

    Torsten Sattler, Michal Havlena, Filip Radenović, Kon- rad Schindler, and Marc Pollefeys. Hyperpoints and Fine Vocabularies for Large-Scale Location Recognition. In ICCV, 2015. 2

    2015

  57. [66]

    Understanding the Limitations of CNN- based Absolute Camera Pose Regression

    Torsten Sattler, Qunjie Zhou, Marc Pollefeys, and Laura Leal-Taixé. Understanding the Limitations of CNN- based Absolute Camera Pose Regression. InCVPR, 2019. 2

    2019

  58. [67]

    Schönberger, Silvano Gal- liani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and Andreas Geiger

    Thomas Schöps, Johannes L. Schönberger, Silvano Gal- liani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and Andreas Geiger. A Multi-view Stereo Benchmark with High-Resolution Images and Multi-camera Videos. In CVPR, 2017. 3

    2017

  59. [68]

    BAD SLAM: Bundle Adjusted Direct RGB-D SLAM

    Thomas Schöps, Torsten Sattler, and Marc Pollefeys. BAD SLAM: Bundle Adjusted Direct RGB-D SLAM. In CVPR, 2019. 3

    2019

  60. [69]

    Privacy Preserving Visual SLAM

    Mikiya Shibuya, Shinya Sumikura, and Ken Sakurada. Privacy Preserving Visual SLAM. InECCV, 2020. 2

    2020

  61. [70]

    Scene Coordinate Regression Forests for Camera Relocaliza- tion in RGB-D Images

    JamieShotton,BenGlocker,ChristopherZach,Shahram Izadi, Antonio Criminisi, and Andrew Fitzgibbon. Scene Coordinate Regression Forests for Camera Relocaliza- tion in RGB-D Images. InCVPR, 2013. 4, 8

    2013

  62. [71]

    GSplatLoc: Grounding Keypoint Descriptors into 3D Gaussian Splat- ting for Improved Visual Localization.arXiv preprint arXiv:2409.16502, 2024

    Gennady Sidorov, Malik Mohrat, Ksenia Lebedeva, Ruslan Rakhimov, and Sergey Kolyubin. GSplatLoc: Grounding Keypoint Descriptors into 3D Gaussian Splat- ting for Improved Visual Localization.arXiv preprint arXiv:2409.16502, 2024. 3, 8, 9

    2024 arXiv

  63. [72]

    Schönberger, Sudipta N

    Pablo Speciale, Johannes L. Schönberger, Sudipta N. Sinha, and Marc Pollefeys. Privacy Preserving Image Queries for Camera Localization. InICCV, 2019. 2

    2019

  64. [73]

    Schönberger, Sing Bing Kang, Sudipta N

    Pablo Speciale, Johannes L. Schönberger, Sing Bing Kang, Sudipta N. Sinha, and Marc Pollefeys. Privacy Preserving Image-Based Localization. InCVPR, 2019. 2

    2019

  65. [74]

    24/7 Place Recognition by View Synthesis

    Akihiko Torii, Relja Arandjelović, Josef Sivic, Masatoshi Okutomi, and Tomáš Pajdla. 24/7 Place Recognition by View Synthesis. TPAMI, 40(2):257–271, 2018. 8

    2018

  66. [75]

    The Unreasonable Effectiveness of Pre- Trained Features for Camera Pose Refinement

    Gabriele Trivigno, Carlo Masone, Barbara Caputo, and Torsten Sattler. The Unreasonable Effectiveness of Pre- Trained Features for Camera Pose Refinement. InCVPR,

  67. [76]

    SPARF: Neural Radiance Fields from Sparse and Noisy Poses

    Prune Truong, Marie-Julie Rakotosaona, Fabian Man- hardt, and Federico Tombari. SPARF: Neural Radiance Fields from Sparse and Noisy Poses. InCVPR, 2023. 13

    2023

  68. [77]

    Repre- sentation Learning with Contrastive Predictive Coding

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Repre- sentation Learning with Contrastive Predictive Coding. arXiv preprint arXiv:1807.03748, 2018. 6

    2018 arXiv

  69. [78]

    GN-Net: The Gauss-Newton Loss for Multi-Weather Relocalization

    Lukas Von Stumberg, Patrick Wenzel, Qadeer Khan, and Daniel Cremers. GN-Net: The Gauss-Newton Loss for Multi-Weather Relocalization. IEEE Robotics and Automation Letters, 5(2):890–897, 2020. 2, 3

    2020

  70. [79]

    LM-Reloc: Levenberg-MarquardtBased Direct Visual Relocalization

    Lukas von Stumberg, Patrick Wenzel, Nan Yang, and DanielCremers. LM-Reloc: Levenberg-MarquardtBased Direct Visual Relocalization. In3DV, 2020. 2, 3

    2020

  71. [80]

    Image-based Localization Using LSTMs for Struc- tured Feature Correlation

    Florian Walch, Caner Hazirbas, Laura Leal-Taixé, Torsten Sattler, Sebastian Hilsenbeck, and Daniel Cre- mers. Image-based Localization Using LSTMs for Struc- tured Feature Correlation. InICCV, 2017. 2

    2017

  72. [81]

    SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis

    Guangcong Wang, Zhaoxi Chen, Chen Change Loy, and Ziwei Liu. SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis. InICCV, 2023. 13

    2023

  73. [82]

    Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time

    Liao Wang, Jiakai Zhang, Xinhang Liu, Fuqiang Zhao, Yanshun Zhang, Yingliang Zhang, Minye Wu, Jingyi Yu, and Lan Xu. Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time. InCVPR, 2022. 13

    2022

  74. [83]

    NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi- view Reconstruction

    Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi- view Reconstruction. InNeurIPS, 2021. 4

    2021

  75. [84]

    DGC- GNN: Descriptor-free Geometric-Color Graph Neural Network for 2D-3D Matching

    Shuzhe Wang, Juho Kannala, and Daniel Barath. DGC- GNN: Descriptor-free Geometric-Color Graph Neural Network for 2D-3D Matching. InCVPR, 2024. 2, 3, 8, 9

    2024

  76. [85]

    Planet-photo Geolocation with Convolutional Neural Networks

    Tobias Weyand, Ilya Kostrikov, and James Philbin. Planet-photo Geolocation with Convolutional Neural Networks. In ECCV, 2016. 2

    2016

  77. [86]

    Deep Probabilistic Feature-metric Tracking.IEEE Robotics and Automation Letters , 6(1):223 – 230, 2021

    Binbin Xu, Andrew Davison, and Stefan Leuteneg- ger. Deep Probabilistic Feature-metric Tracking.IEEE Robotics and Automation Letters , 6(1):223 – 230, 2021. 2, 3

    2021

  78. [87]

    Barron, Al- berto Rodriguez, Phillip Isola, and Tsung-Yi Lin

    Lin Yen-Chen, Pete Florence, Jonathan T. Barron, Al- berto Rodriguez, Phillip Isola, and Tsung-Yi Lin. INeRF: Inverting Neural Radiance Fields for Pose Estimation. In IROS, 2021. 3

    2021

  79. [88]

    PlenOctrees for Real-time Ren- dering of Neural Radiance Fields

    Alex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng, and Angjoo Kanazawa. PlenOctrees for Real-time Ren- dering of Neural Radiance Fields. InICCV, 2021. 13

    2021

  80. [89]

    MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction

    Zehao Yu, Songyou Peng, Michael Niemeyer, Torsten Sattler, and Andreas Geiger. MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction. In NeurIPS, 2022. 5, 13, 15

    2022

  81. [90]

    Efros, Eli Shecht- man, and Oliver Wang

    Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shecht- man, and Oliver Wang. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. InCVPR, 2018. 4

    2018

  82. [91]

    Use all the labels: A hierarchical multi-label contrastive learning framework

    Shu Zhang, Ran Xu, Caiming Xiong, and Chetan Ra- maiah. Use all the labels: A hierarchical multi-label contrastive learning framework. InCVPR, 2022. 7

    2022

  83. [92]

    Zhang and J

    W. Zhang and J. Kosecka. Image based Localization in Urban Environments. In3DPVT, 2006. 2 12 Can we make NeRF-based visual localization privacy-preserving?

    2006

  84. [93]

    Structure From Motion Using Structure-Less Resection

    Enliang Zheng and Changchang Wu. Structure From Motion Using Structure-Less Resection. InICCV, 2015. 2

    2015

  85. [94]

    ToLearnornottoLearn: VisualLocalization from Essential Matrices

    Qunjie Zhou, Torsten Sattler, Marc Pollefeys, and Laura Leal-Taixé. ToLearnornottoLearn: VisualLocalization from Essential Matrices. InICRA, 2020. 2

    2020

  86. [95]

    Is Geometry Enough for Matching in Visual Localization? In ECCV, 2022

    Qunjie Zhou, Sergio Agostinho, Aljosa Osep, and Laura Leal-Taixe. Is Geometry Enough for Matching in Visual Localization? In ECCV, 2022. 2, 3, 8, 9

    2022

  87. [96]

    The NeRFect Match: Exploring NeRF Fea- tures for Visual Localization

    Qunjie Zhou, Maxim Maximov, Or Litany, and Laura Leal-Taixé. The NeRFect Match: Exploring NeRF Fea- tures for Visual Localization. InECCV, 2024. 2, 3, 5, 8, 9 APPENDIX This section is structured as follows. In Appendix A we first provide comprehensive details regarding the ppN...

    2024

  88. [98]

    to output two final feature maps of stride 4 and 2. They are processed through segmentation heads (three conv2D layers with ReLU activation and internal dimen- sion of 128, one upsampling layer of scale 4 or 2) which yield the pixel aligned coarse and fine segmentation maps wi...

  89. [100]

    marigold-lcm-v1-0

    The geometric MLP contains two linear layers and ReLU activations (internal dimension 64), scale featur- ization is used. Coarse and fine semantic heads consist of a three layer MLP with ReLU activations (internal dimension 128). The number of samples per proposal network is s...

  90. [101]

    liuhaotian/llava-v1.5-7b

    Skip connections are used for the decoder blocks. The inversion model is trained with an Adam optimizer 16 Can we make NeRF-based visual localization privacy-preserving? Figure 6: Refinement with different number𝐾𝑓 = 𝑛∗𝐾 of fine classes (varying number𝐾 of coarse classes,𝑛 = 5...

  91. [128]

    The output feature has a dimension of 96 to match with the encoder-based feature dimension

    with ReLU activations. The output feature has a dimension of 96 to match with the encoder-based feature dimension. Feature rendering uses the opacity weight from the main neural implicit field. Gradients are detached from these weights. NIF (neural implicit field) is a represe...

  92. [2019]

    1 10 Can we make NeRF-based visual localization privacy-preserving?

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.