{"id":"ba57d85d-fcde-46bf-a726-ee91d4f75fc8","arxiv_id":"2507.18881","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Self-supervised 3D geometric priors, trained with multi-view and view-scene contrastive constraints, transferred into an F3Loc-style observation model improve visual floorplan localization accuracy on Structured3D and Gibson benchmarks.","lead":"A visual localization method for 2D floorplans was upgraded by first teaching its image encoder 3D geometry through contrastive learning on indoor scans, then fine-tuning it for floorplan matching. On two benchmark suites, this pre-training substantially lifted localization success rates without adding inference cost.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 3D-prior attribution is confounded: Table 5 varies pretraining data and loss simultaneously, so gains may come from indoor-domain self-supervised pretraining rather than the hard geometric constraints.","rationale":"The reader's weakest assumption is that ScanNet RGB-D geometric priors transfer to monocular RGB inference on Structured3D and Gibson. That is a real risk, but the paper's own experiments already attempt the transfer, and the point estimates show improvements. The more immediate threat to the central claim is internal to the comparison: the causal role of the geometric priors is not isolated because the pretraining data and the loss vary together across all compared initializations. If a ScanNet-trained non-geometric contrastive baseline produces similar downstream gains, the methods would still be useful but the paper's specific 3D-prior story would be unsupported. This concern is compatible with the reader's CONDITIONAL verdict: the work is promising and the main tables are internally consistent, but the attribution needs a controlled same-data ablation, plus code and error bars to rule out seed noise. I therefore keep the verdict unchanged rather than escalating to reject or unverified.","tokens_in":17528,"tokens_out":11663,"duration_ms":124619,"concrete_test":"Run a controlled pretraining experiment: take the exact ScanNet RGB-D frame pairs (approximately 840k pairs) used for GCVI and train the same UNet with a non-geometric contrastive loss, e.g., SimCLR-style augmentations or random-pair InfoNCE with no depth, surface chunks, or 3D correspondence, for the same 60k iterations, batch size, and learning-rate schedule. Then fine-tune on Gibson(g) with the same F3Loc pipeline and compare SR@0.1m, SR@0.5m, SR@1m, and SR@1m30 against Table 2 and Table 5. If the non-geometric ScanNet pretraining matches or exceeds the GCVI/VSAG gain, the attribution to 3D geometric priors fails; if the geometric variant is clearly better, the concern is settled.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is causal: the hard 3D geometric pretraining (GCVI and VSAG) is what improves visual FLoc. The strongest evidence for this is the comparison in Table 5, where our pretrained initialization is contrasted with Scratch, ImageNet, SimCLR, CRL, Ego2-MAP, ECL, and SPA. But these baselines differ simultaneously in pretraining data, objective, and compute: SimCLR uses ImageNet, CRL/Ego2-MAP/ECL use embodied scenes, and SPA uses neural rendering on other inputs. None holds fixed the exact ScanNet RGB-D frame pairs and 840k pair budget while replacing the geometric correspondence loss with a non-geometric contrastive loss. Thus the observed gain over F3Loc could be driven by domain-matched self-supervised pretraining on indoor RGB, not by the 2 cm geometric correspondences or view-scene alignment. Table 4 shows that adding GCVI and VSAG helps relative to using only one of them, but it contains no 'neither prior' row, so it cannot separate the geometric prior from the indoor pretraining data. Because F3Loc is initialized from these weights and fine-tuned end-to-end, any better indoor-aware initialization could explain part or all of the gains. This does not make the empirical result false, but it means the paper's headline statement that 3D geometric priors help is not yet demonstrated with the reported experiments alone.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes to improve 2D visual floorplan localization (FLoc) by pre-training the visual encoder with two self-supervised contrastive objectives on ScanNet RGB-D data: geometry-constrained view invariance (GCVI), which uses 2 cm 3D-point correspondences between frames as positive pairs, and view-scene aligned geometry (VSAG), which aligns RGB frames with surface point-cloud chunks. The pre-trained encoder is then fine-tuned inside the F3Loc observation model for single-frame, multi-frame, and adaptive inference. Experiments on Structured3D and Gibson report consistent improvements in success rate at several accuracy thresholds over F3Loc and other pretraining baselines, plus ablations and qualitative trajectory comparisons.","tokens_in":17792,"tokens_out":7337,"duration_ms":69174,"significance":"If the improvements are truly caused by the hard geometric constraints, the contribution is practically valuable: it is a simple, label-free encoder pre-training recipe that improves FLoc success and accuracy at no additional inference cost. The paper's main strengths are the clear formulation of the two priors, the use of public ScanNet data, and the breadth of the evaluation (two datasets, three inference modes, four metrics). The weakness is that the experimental design does not yet isolate the geometric prior from correlated factors such as indoor-domain pretraining data, the contrastive objective itself, and the fine-tuning protocol. The absence of variance estimates further limits the strength of the claims. With additional controlled experiments and uncertainty reporting, the paper could fully support its central claim.","major_comments":[{"comment":"The central claim that the hard 3D geometric constraints (GCVI, VSAG) drive the gains is confounded. Table 5 varies pretraining data, objective, and input modality simultaneously across baselines (SimCLR on ImageNet, CRL/Ego2-MAP/ECL on embodied RGB or semantic maps, SPA on neural rendering), while Table 4 only removes one of the two proposed losses and leaves the other in place; there is no control that keeps the same ScanNet RGB-D frame pairs and 840k pair budget while replacing the geometric correspondence loss with a non-geometric contrastive loss. Consequently, the improvement over F3Loc could be explained by domain-matched indoor self-supervised pretraining rather than by the 2 cm pixel/point constraints. Please add a control (e.g., the same ScanNet data and pair budget with a SimCLR-style or random-pair contrastive loss) and report it in Table 4 or Table 5.","section":"§4.3 (Tables 4 and 5)"},{"comment":"All success rates are single numbers from one run; no error bars, multiple seeds, or significance tests are reported. The smallest claimed advantage over the strongest baseline is +1.1 percentage points at SR@0.1m in Table 5 (Ours vs. SPA), which is within the range of typical run-to-run variance for this kind of fine-tuning. Please report mean and standard deviation over at least three seeds (or a paired significance test) for the main comparisons, and in particular for Table 5, so the reader can assess whether the remaining gains over SPA are reliable.","section":"All tables, especially Table 5"},{"comment":"The paper does not state whether the F3Loc baseline numbers are re-implemented under the same fine-tuning protocol (optimizer, epochs, loss, ray counts) or taken from the original F3Loc paper. Since the paper's headline gains are relative to F3Loc, this matters: if the baselines use a different training budget, part of the improvement may be due to fine-tuning configuration rather than to the pretrained geometric priors. Please clarify the source of each baseline number and, if available, provide a same-protocol re-implementation of F3Loc to make the comparison controlled.","section":"§4.1–4.2 (Tables 1–3)"}],"minor_comments":[{"comment":"The sentence 'Ours_f further benefits from the performance gains of Ours_f and Ours_m' should read 'Ours_s and Ours_m'.","section":"§4.2"},{"comment":"The denominator as printed is missing the summation symbol; the formula is ambiguous. Please use a clear summation notation, e.g., sum over all negative keys, to match the PointInfoNCE definition.","section":"Eq. (3)"},{"comment":"The method requires RGB-D depth maps and camera poses (from SLAM) to define the geometric correspondences during pre-training; the statement 'no additional geometric annotations are required' should be qualified to clarify that metric depth and poses are used as supervision signals, though they are not manually annotated.","section":"§3.2 and Abstract"},{"comment":"The text says the contributions of GCVI and VSAG are 'almost equal'; Table 4 shows similar SR@1m values (38.3 vs. 37.5) but the difference at SR@0.1m is small (6.2 vs. 6.4). This summary is acceptable, but given the absence of variance estimates, the 'almost equal' claim should be stated more cautiously.","section":"§4.3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to be the ACM MM 2025 camera-ready version (see the copyright line and DOI on page 1). If this is a journal submission, please confirm the journal's policy on prior conference publication and ask for a clear statement of the extended contribution. Also, the authors state that 'All data and code will be released after the anonymous review'; for a reproducibility-focused journal, consider whether code should be made available to reviewers during the revision process."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: the paper delivers real, consistent gains on visual floorplan localization by feeding a pretrained visual backbone into F3Loc, and it's the first in the cited literature to bring 3D geometric contrastive priors to this task. But the headline attribution - that the hard 2 cm geometric constraints in GCVI and VSAG are what cause the gains - is not proven by the experiments as run.\n\nWhat's new and good: the combination is sensible, and the gains are not marginal. Single-frame SR@1m on Structured3D jumps from 22.4 to 55.5; Gibson(g) shows similar improvements. The gains hold across two datasets, three inference settings (single, multi, adaptive), and against strong baselines including SPA and ECL. The ablations in Table 4 show each loss contributes, and the paper is honest about limitations (no semantics, lack of indoor datasets). The pretraining uses ScanNet, separate from the evaluation datasets, so there's no circularity.\n\nThe soft spots are mostly about attribution. Table 5 compares against Scratch, ImageNet, SimCLR, CRL, Ego2-MAP, ECL, SPA, but these differ in pretraining data, loss, and compute simultaneously. You can't tell whether the improvement comes from the 2 cm pixel/point constraints or simply from indoor-domain self-supervised pretraining on RGB-D scans. Table 4 adds the two losses individually and together, but has no 'neither prior' row; the closest proxies are the Scratch and ImageNet rows in Table 5, which again change the data. So the paper shows the full recipe works, not that the geometric hard constraints are the active ingredient. To their credit, Ours does beat SPA and ECL, which are also indoor/embodied, so there's some evidence the geometric priors add value, but it's not clean isolation.\n\nThe other issues are more minor: no error bars or multi-seed statistics - every number is a single run; and the code/data are promised only 'after the anonymous review' in a non-anonymous arXiv paper, which is a red flag for verification. These are fixable in revision.\n\nWho this is for: people working on floorplan localization, visual localization with lightweight maps, or self-supervised geometric representation learning. It's a useful data point even if the causal story needs tightening. I'd send it to peer review; the right referee will ask for an ablation that holds pretraining data fixed (same ScanNet pairs, a non-geometric contrastive loss) before accepting the 3D-prior claim. With that, it would be close to publishable as is.","headline":"Solid gains in floorplan localization, but the causal claim that 3D geometric priors drive them is underdetermined by the experiments as run.","tokens_in":18344,"tokens_out":3407,"would_cite":true,"duration_ms":34330,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Injecting 3D geometric priors learned from RGB-D scans into a floorplan localizer's observation model lifts single-frame success at 0.1 m from 1.5% to 5.6% on Structured3D and from 8.3% to 9.4% on Gibson(g), with no added inference cost.","keywords":["Floorplan Localization","Geometry-Constrained View Invariance","View-Scene Aligned Geometric Prior","Contrastive Learning","3D Geometric Priors","Self-Supervised Pretraining","Visual Localization","RGB-D Scene Understanding"],"falsifier":"Reproduce the same fine-tuning pipeline on Gibson(g) but initialize the observation encoder with a contrastive pretraining that uses the same ScanNet frames with the correspondence threshold relaxed from 2 cm to 20 cm, or with the VSAG alignment removed. If success at 0.1 m stays near the reported 9.4% rather than falling toward the 4.3% scratch baseline, the hard geometric constraints are not what drives the gain.","tokens_in":17305,"feed_emoji":"📍","tokens_out":13027,"duration_ms":108509,"temperature":0.7,"pith_summary":"Visual floorplan localization asks a camera to find its pose on a 2D floorplan from RGB images alone, despite the gap between sparse geometric maps and appearance-rich photos. This paper argues that the missing link is 3D geometry: if the visual encoder is first taught, through self-supervised contrastive learning on ScanNet RGB-D scans, what stays invariant across views of the same 3D points and how views align with 3D surface geometry, the resulting representation yields better localization once fine-tuned in the F3Loc framework. On Structured3D, single-frame success at 0.1 m rises from 1.5% (F3Loc) to 5.6%, and success at 1 m from 22.4% to 55.5%; on the harder Gibson(g) set the method reaches 9.4% success at 0.1 m, above the 8.3% of the best prior representation-learning baseline. The gains come without any test-time depth, extra semantic labels, or added inference cost, because the priors are folded into the same observation encoder used by the existing pipeline.","feed_headline":"3D priors lift floorplan localization success from 1.5% to 5.6%","feed_subtitle":"Self-supervised 3D priors transfer from RGB-D scans to monocular RGB localization at no extra inference cost.","key_machinery":"The load-bearing object is the visual encoder $F_\\theta$, a UNet-style ResNet50, pretrained with two contrastive objectives. GCVI builds positive pairs from frame pairs whose pixels correspond to the same 3D points, established by projecting depth with the pinhole model and requiring the world coordinates to agree within 2 cm for at least 30% of pixels. VSAG builds positive pairs from an RGB frame and the cropped point-cloud chunk of the scene surface reconstruction falling in that frame's frustum, at 2 cm resolution, aligning color with geometry. Both use the PointInfoNCE loss, and the second objective also trains a PointNet++ branch $F_\\phi$; after pretraining, $F_\\theta$ is dropped into the F3Loc observation model and fine-tuned end-to-end with the histogram filter, so the geometric knowledge enters only through the encoder weights and adds no runtime cost.","core_discovery":"The paper's central claim is that 3D geometric priors can be distilled into a 2D visual encoder, and that this distillation is what lifts floorplan localization accuracy. Two priors are defined. Geometry-Constrained View Invariance (GCVI) treats two frames as a positive pair when, via known poses and depth, at least 30% of their pixels project to 3D points within 2 cm of each other. View-Scene Aligned Geometry (VSAG) aligns each RGB frame with the point-cloud chunk of the scanned surface that lies in its view frustum, again under a 2 cm pixel-to-point correspondence. Both are trained with a PointInfoNCE contrastive loss on ScanNet, using BundleFusion poses and reconstructions, with no manual annotations. The pretrained encoder is then transferred into the observation model of F3Loc, a front-end ray and depth predictor plus back-end histogram filter, and fine-tuned on the target floorplan datasets. The measured consequence is that localization success at tight tolerances rises well above the F3Loc baseline and above prior 2D and 3D contrastive pretraining methods, without changing what is computed at inference time.","pith_inferences":["If the transfer story holds, the same self-supervised pretraining recipe could be dropped into other localization front-ends, such as particle-filter observation models, retrieval-based pose estimators, or map-matching networks; the paper demonstrates only the F3Loc case.","The 2 cm threshold and 30% overlap rule are design choices, not theorems. A natural stress test is to coarsen the threshold to, say, 20 cm and see whether Gibson(g) success at 0.1 m degrades; if it does not, the hard geometric alignment is not the active ingredient.","A practical corollary the paper does not test directly: since depth is used only in pretraining, the same encoder could be deployed on an ordinary phone camera with no depth sensor, as long as the scene floorplan and rough gravity direction are available."],"forward_implications":["On Structured3D full, single-frame success at 0.1 m improves from 1.5% (F3Loc_s) to 5.6%, and success at 1 m from 22.4% to 55.5%.","On Gibson(g), single-frame success at 0.1 m reaches 9.4% versus 8.3% for the strongest prior representation-learning baseline, and long-sequence success at 0.2 m reaches 70.3% for the Gibson(g)-trained single-frame model versus 35.1% for F3Loc_s.","Ablations show GCVI and VSAG contribute roughly equally, and either prior alone leaves success at 0.1 m near 6% on Gibson(g), well below the combined 9.4%.","Because the pretrained encoder replaces only the initialization of the same observation model, the improvements add no extra inference computation.","The pretraining needs no semantic labels: positive and negative pairs are defined purely by scan geometry, so the recipe extends to any RGB-D scan collection with poses."],"supporting_citations":[{"why":"Supplies the F3Loc front-end observation model and histogram filter that the method enhances, along with the baseline numbers in Tables 1-3.","marker":"[9]"},{"why":"Provides the ScanNet RGB-D sequences, camera poses, and 3D surface reconstructions used to define the 2 cm pixel/point correspondences for pretraining.","marker":"[13]"},{"why":"BundleFusion generates the globally consistent poses and surface reconstructions that make the 2 cm hard correspondences possible.","marker":"[14]"},{"why":"PointInfoNCE is the contrastive loss used for both GCVI and VSAG pretraining objectives.","marker":"[55]"},{"why":"Structured3D is one of the two evaluation datasets; Table 1 reports the single-frame gains over F3Loc_s there.","marker":"[59]"},{"why":"Gibson is the second evaluation dataset; Tables 2, 3, and 5 report single-frame, multi-frame, and long-sequence results there.","marker":"[54]"},{"why":"SimCLR serves as a self-supervised pretraining baseline in Table 5, showing where the method stands relative to generic contrastive learning.","marker":"[11]"},{"why":"SPA is the strongest representation-learning baseline in Table 5, with the 8.3% SR@0.1m that the method surpasses on Gibson(g).","marker":"[61]"},{"why":"LASER is a 2D-feature FLoc baseline whose lower numbers anchor the claim that 3D priors beat 2D geometric cues.","marker":"[37]"}],"fun_headline_variants":["3D geometric priors boost floorplan localization 3.7x","Self-supervised 3D priors sharpen floorplan localization at no extra cost","Zero-annotation 3D priors improve floorplan localization","Floorplan localization gets 3D priors at no inference cost","3D priors from scans, no labels, sharpen floorplan localization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that geometric invariance and view-to-scene alignment learned on ScanNet RGB-D scans with BundleFusion poses and 2 cm pixel/point constraints transfer to monocular RGB inference on Structured3D and Gibson, where no depth is available at test time.","fun_headline_variants_meta":{"raw":{"variants":["3D geometric priors boost floorplan localization 3.7x","Self-supervised 3D priors sharpen floorplan localization at no extra cost","Zero-annotation 3D priors improve floorplan localization","Floorplan localization gets 3D priors at no inference cost","3D priors from scans, no labels, sharpen floorplan localization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000866,"raw_usage":{"total_tokens":3832,"prompt_tokens":1099,"completion_tokens":2733,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":715,"completion_tokens_details":{"reasoning_tokens":2638}},"tokens_in":715,"tokens_out":2733,"duration_ms":21670,"temperature":1.0,"reasoning_tokens":2638,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:06:34.980916+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Reproduce the same fine-tuning pipeline on Gibson(g) but initialize the observation encoder with a contrastive pretraining that uses the same ScanNet frames with the correspondence threshold relaxed from 2 cm to 20 cm, or with the VSAG alignment removed. If success at 0.1 m stays near the reported 9.4% rather than falling toward the 4.3% scratch baseline, the hard geometric constraints are not what drives the gain.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the F3Loc front-end observation model and histogram filter that the method enhances, along with the baseline numbers in Tables 1-3."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Structured3D is one of the two evaluation datasets; Table 1 reports the single-frame gains over F3Loc_s there."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"LASER is a 2D-feature FLoc baseline whose lower numbers anchor the claim that 3D priors beat 2D geometric cues."}],"review_version":2}