{"id":"ea7e1754-457e-44be-b322-add7b1b2b1f1","arxiv_id":"2506.13073","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"SuperPlace demonstrates that improved classical aggregators (G2M and NVL-FT2) match or beat 2024-era VPR methods at much lower feature dimensions.","lead":"SuperPlace shows that lightly modified classical image-encoding methods (GeM and NetVLAD) beat more complex recent approaches for visual place recognition, while using one-tenth the descriptor size. Its two new building blocks, a compact GeM variant and a staged fine-tuning strategy, plus a way to combine four training datasets, make robot and augmented-reality localization faster and more memory-efficient.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"G2M's controlled accuracy gain is selected on the same Pitts-30k-test split that is later reported as the final benchmark, and the chosen configuration is not consistently better on Tokyo-24/7 or MSLS-val, so the compact-aggregation claim needs a held-out validation check.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: G2M hyperparameters were selected on Pitts-30k-test, which is then used as a final benchmark. This concern is supported by explicit text in Section 4.4, and it matters because the controlled G2M gain over GeM is small and not consistent across the three test sets reported in Table 7. Under GPMS training in Table 10, G2M is even slightly worse than GeM on Pitts, which further weakens the case that the GCA module is a general improvement rather than a configuration fitted to one benchmark. This is not an accusation of misconduct; it is a benchmark-protocol issue that can be resolved with a held-out selection split. The paper does contain useful controlled experiments, especially the NVL-FT2 comparisons in Tables 9 and 11, and the SLA ablations in Table 5 provide genuine evidence for the multi-dataset training contribution. However, because the abstract's headline claim about G2M depends on the small, possibly fitted margin, a conditional acceptance that requires an independent held-out hyperparameter evaluation is the appropriate outcome. Secondary concerns, such as training-set differences in the Table 3 SOTA comparison and the unverified MSLS leaderboard claim, are also real but do not change the verdict beyond conditional.","tokens_in":16326,"tokens_out":6963,"duration_ms":79689,"concrete_test":"Re-run the Table 8 hyperparameter sweep using a held-out validation split that is not Pitts-30k-test (for example, a disjoint district subset of the Pittsburgh data, or another GSV-based set such as Tokyo-24/7), select the GCA rank and activation there, then evaluate the frozen configuration on Pitts-30k-test, Tokyo-24/7, and MSLS-val. Include rank values 3, 32, 64, and 128 with both GELU and ReLU, and repeat each run with at least two seeds to estimate variance. If the selected configuration is not rank=64/GELU, or if the Pitts-30k-test R@1 margin over plain GeM drops below the Table 7 spread, the G2M-specific claim should be reworded; if rank=64/GELU is selected on the held-out split and the margin holds across seeds, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's G2M accuracy claim rests on a closed selection loop. Section 4.1 fixes the GCA configuration to rank 64 and GELU, and Section 4.4 states: \"Since the distributions of GSV-Cities and Pitts-30k were closely related, we mainly selected parameters based on the results of Pitts-30k\" (Table 8). That same Pitts-30k-test split is then reported as a final benchmark in Tables 3, 7, 10, and 11. In Table 8, rank=64/GELU beats plain GeM on Pitts-30k-test R@1 by +0.7 (92.6 vs 91.9), but on Tokyo-24/7 it gives 94.0, below rank=3 (95.2) and below ReLU (94.9); on MSLS-val it gives 90.4, below rank=3 (90.9) and rank=32 (90.5). Thus the only benchmark on which the chosen hyperparameter is clearly best is the benchmark on which it was selected. This is load-bearing because the controlled contribution of GCA is small and inconsistent: under the same GSV-Cities training, G2M's margins over GeM are +0.7, -0.3, and +0.1 R@1 on Pitts, Tokyo, and MSLS-val respectively (Table 7). Under GPMS training (Table 10), G2M is actually 0.2 points below GeM on Pitts-30k-test (92.2 vs 92.4) and below SALAD on both Pitts and MSLS-val. If the +0.7 Pitts margin is a selection artifact, the evidence that GCA improves over plain GeM reduces to noise-level, inconsistent changes, and the \"one-tenth dimensions\" framing loses its controlled support. The 768-dimensional efficiency advantage is real, but the claimed accuracy equivalence across domains is not independently established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SuperPlace, a VPR framework built on DINOv2, with two classical aggregators: G2M (GeM plus a channel-attention branch) and NVL-FT2 (NetVLAD with a linear compression layer trained with a second fine-tuning stage), together with a supervised label alignment (SLA) method that unifies training labels across GSV-Cities, Pitts-250k, MSLS, and SF-XL. Experiments compare SuperPlace against recent VPR methods on 12 test sets, report ablations of each component, and claim that G2M reaches competitive accuracy with 768-d descriptors while NVL-FT2 ranks first on the MSLS leaderboard.","tokens_in":16726,"tokens_out":6162,"duration_ms":57185,"significance":"The paper's strength is its systematic ablation methodology: within each table, methods are trained on the same set, isolating the effect of the aggregator (Tables 7, 10, 11) and of the training-set composition (Table 5). If the claims held, the paper would show that simple classical aggregators remain competitive with more complex recent designs at a fraction of the descriptor dimensionality, and that multi-dataset training via label alignment is a valuable recipe. Efficiency figures (parameter counts, inference times) are also reported. However, the headline SOTA comparison is confounded by training-set differences, and the GCA hyperparameters are selected on a test set that is later reported as a benchmark, which undermines the controlled evidence for G2M's advantage.","major_comments":[{"comment":"The paper explicitly states that GCA rank and activation were selected 'mainly ... based on the results of Pitts-30k' (Section 4.4), and the same Pitts-30k-test split is used as a final benchmark in Tables 3, 7, 10, and 11. In Table 8, rank=64 with GELU improves over GeM on Pitts-30k-test (+0.7 R@1) but is worse on Tokyo-24/7 (-0.3) and only marginally better on MSLS-val (+0.1); on Tokyo, rank=3 and ReLU are both better than the chosen configuration. In Table 10 (GPMS training), G2M is actually 0.2 R@1 below GeM on Pitts-30k-test. This constitutes selection on the test set, and the small, inconsistent gains mean the 'one-tenth dimensions' claim is not independently established. Please validate the GCA configuration on a held-out validation set (e.g., a separated split or a different benchmark), report multiple seeds or confidence intervals, and avoid using the benchmark set for model selection.","section":"Section 4.4, Table 8"},{"comment":"The abstract's statement that 'G2M achieves promising results with only one-tenth of the feature dimensions compared to recent methods' is based primarily on Table 3, where SP-G2M and SP-NVL-FT2 are trained on GPMS while SALAD, BoQ, and other baselines use GSV-Cities (this is acknowledged in Section 4.1). The different training sets make the comparison non-matching; the performance gap conflates the aggregation design with the SLA contribution. Please provide a matched comparison where baselines are also trained on GPMS (or SuperPlace on GSV-Cities only), or explicitly separate the contribution of SLA from the aggregation gains in the claims.","section":"Table 3 and Abstract"},{"comment":"Under GPMS training, the GCA module actually hurts performance on Pitts-30k-test (92.2 vs 92.4 for GeM) and leaves G2M below SALAD (92.8). This result is inconsistent with the claim that G2M is a generally superior low-dimensional aggregation. Please temper the claim or provide an analysis of why the benefit disappears with the multi-dataset training recipe.","section":"Table 10"}],"minor_comments":[{"comment":"The sentence 'The use of VFMs in VPR has been limited compared to other vision tasks, compared to other vision tasks' repeats 'compared to other vision tasks'.","section":"Section 2"},{"comment":"The phrase 'between the patches' should likely be 'on the patches' or 'across the patches'.","section":"Figure 3 caption"},{"comment":"The training set for SALAD-SLA is not specified; please state it to make the comparison with SALAD-CM controlled.","section":"Table 6"},{"comment":"The number of images after label alignment is said to be 'recorded in Tab. 2', but Table 2 lists test sets; please add the aligned training-set statistics.","section":"Section 3.3"},{"comment":"The row 'NVL-MLP✓8192' is ambiguous about whether the CLS token is concatenated; please define the checkmark columns clearly.","section":"Table 9"},{"comment":"The claim that NVL-FT2 'ranks first on the MSLS leaderboard' is not verifiable from the manuscript; please provide a citation, link, or snapshot with a date.","section":"Abstract and Section 1"}],"recommendation":"major_revision","confidential_remarks":"The reader's concern about selection on Pitts-30k is well-founded and is explicitly acknowledged in Section 4.4. Beyond the requested revisions, the authors should be asked to either provide a genuine validation split or to reframe the claims so that they do not rest on a benchmark used for model selection. The MSLS leaderboard claim should also be backed by evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper before committing time to it. First, the core claim—that lightly modified classical aggregators like GeM and NetVLAD remain competitive with fancier recent modules—is actually backed by a lot of careful ablations. Second, the specific accuracy advantage claimed for G2M is shakier than the paper's framing suggests, because the authors selected GCA hyperparameters (rank 64, GELU) on the Pitts-30k test split and then report that same split as a final benchmark. The stress-test note nails this: in Table 8, rank 64/GELU is best on Pitts-30k by +0.7 R@1 over plain GeM, but on Tokyo-24/7 it is worse than rank 3 and worse than ReLU, and on MSLS-val it is worse than rank 3 and rank 32. Under GPMS training (Table 10), G2M actually trails plain GeM on Pitts-30k (92.2 vs 92.4). So the controlled evidence that the GCA module helps is essentially noise-level and inconsistent. That said, the paper does several things well. The SLA label-alignment method is a legitimate attempt to train on multiple datasets with heterogeneous supervision, and the comparison against CliqueMining in Table 6 is a fair, controlled test. The FT2 strategy for NetVLAD-Linear is a clean idea and the ablations in Table 9 support it: NVL-FT2 matches NV-PCA at a fraction of the parameters and beats one-shot NVL. The 768-dim G2M descriptor is a genuine efficiency contribution, even if its accuracy parity with larger descriptors is not independently established on held-out benchmarks. The main soft spots are the selection loop on Pitts-30k, the confounded SOTA table (GPMS vs GSV-Cities), and the exclusion of CricaVPR based on an unverified leakage claim. These are addressable, but they need to be fixed before the headline claims are taken at face value. Who is this for? VPR researchers who care about descriptor dimensionality and multi-dataset training recipes. It deserves a serious referee—the engineering is solid and the questions are important—but the referee should push for a held-out validation of G2M's hyperparameters, for example by reporting results on a split not used for selection, or by showing consistent gains across multiple unseen sets. I'd send it to review, but my own verdict would be conditional pending those changes.","headline":"A solid empirical paper with a real efficiency contribution, but the G2M accuracy gain is partly selected on a test set that is then reported as final, and the headline SOTA comparison is confounded by training-set differences.","tokens_in":17348,"tokens_out":1105,"would_cite":false,"duration_ms":13151,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Lightly upgraded GeM and NetVLAD aggregators, trained on multiple datasets with aligned labels, match or beat recent VPR methods at one-tenth the descriptor size, and one variant ranks first on MSLS.","keywords":["visual place recognition","feature aggregation","GeM pooling","NetVLAD","DINOv2","multi-dataset training","label alignment","compact descriptors"],"falsifier":"Retrain G2M with the GCA rank and activation selected on a validation split that is disjoint from every reported test set, then evaluate on Pitts-30k-test; if the Recall@1 advantage over plain GeM shrinks to well below the reported +0.7, the compact-aggregation claim is partly an artifact of test-set tuning.","tokens_in":16064,"feed_emoji":"📍","tokens_out":13007,"duration_ms":132053,"temperature":0.7,"pith_summary":"The paper's thesis is that the recently invented feature-aggregation modules in visual place recognition are not necessary: the classical GeM and NetVLAD pooling modules, upgraded in small ways and trained on multiple datasets at once, match or exceed them with far smaller descriptors. The upgrades are a second GeM that learns the principal components of a feature map across channels and uses them to calibrate the first GeM's output, and a two-stage fine-tuning schedule in which NetVLAD first learns a high-dimensional representation and then compresses it through a single linear layer. A third ingredient, supervised label alignment, turns GPS-distance labels into class labels using UTM grid cells and local feature matching, so four large datasets can be pooled into one training framework. If the claims hold, a 768-dimensional descriptor is enough for competitive place recognition, and the remaining large gains come from training on more data rather than from inventing more complex aggregators.","feed_headline":"Two classical aggregators top modern VPR at 1/10 the descriptor size","feed_subtitle":"Light upgrades plus multi-dataset training rank SuperPlace first on the MSLS leaderboard at a fraction of the descriptor size.","key_machinery":"Three mechanisms carry the argument. Generalized Channel Attention (GCA) is a second GeM pooling branch that computes a channel-wise summary, passes it through a low-rank MLP, GELU, and sigmoid, and re-weights the main GeM's descriptor; it is the component that lets the compact model stay at 768 dimensions. Secondary fine-tuning (FT2) applies to NetVLAD-Linear: first the backbone and NetVLAD are fine-tuned in a high-dimensional space, then the single linear compression layer is fine-tuned while the rest is frozen; this two-stage schedule is what makes the linear layer competitive with PCA-based compression. Supervised label alignment (SLA) converts the distance-based supervision of four datasets into class labels by partitioning UTM coordinates into grid cells, adding heading classes where available, and refining classes with local feature matching, so one model can train on all datasets simultaneously.","core_discovery":"The central discovery is that heavily engineered aggregation layers are not the source of recent progress in VPR. Starting from DINOv2 as a pre-trained backbone, the paper shows that a plain GeM with the last four layers fine-tuned already achieves strong results, and that adding a Generalized Channel Attention (GCA) branch — a second GeM pooling that summarizes each channel, passes it through a low-rank MLP with GELU and sigmoid, and re-weights the output of the first GeM — improves accuracy while keeping the descriptor at 768 dimensions. For high-dimensional retrieval, the paper shows that a linear projection after NetVLAD underperforms until it is trained in a second stage after the high-dimensional NetVLAD representation is already learned; this secondary fine-tuning updates only 0.11% of the model's parameters and closes most of the gap with full NetVLAD. The supervised label alignment then combines GSV-Cities, Pittsburgh-250k, MSLS, and SF-XL into one training mixture, and this multi-dataset training is what pushes the NetVLAD variant to first place on the MSLS challenge leaderboard and gives the 768-dimensional model its strongest results.","pith_inferences":["An extension the paper does not run: attach the same GCA calibration branch to other aggregators such as SALAD or BoQ and see whether the 768-dimension advantage persists; if it does, channel calibration is a general cheap upgrade rather than a GeM-specific fix.","The SLA recipe generalizes beyond VPR: any geo-tagged image task with inconsistent supervision could reuse the UTM-grid-plus-local-matching trick to merge datasets, turning multi-source metric learning into a classification problem.","If the 768-dimension result survives a strict tuning-free evaluation, the practical bottleneck for large-scale place recognition shifts from descriptor expressiveness to database search cost, and re-ranking becomes optional rather than necessary for top performance."],"forward_implications":["SP-G2M's 768-dimensional descriptors imply that large VPR databases can be searched with roughly one-tenth the memory and computation of recent methods that use 4,096 to 12,288 dimensions.","NVL-FT2 reaches first place on the MSLS challenge leaderboard with a single-stage pipeline, so two-stage re-ranking is not required for top-level accuracy.","Training on the aligned mixture helps on every test set that matches one of the constituent datasets, indicating that the four datasets carry complementary information rather than being redundant.","The FT2 second stage is cheap because it updates only 0.11% of the model's parameters, making the accuracy gain over one-shot NetVLAD-Linear available at low extra training cost.","With DINOv2-L and the aligned training mixture, SP-NVL-FT2 reaches 84.8% Recall@1 on MSLS-challenge, which the paper reports as 11.3 points above a strong re-ranking baseline."],"supporting_citations":[{"why":"defines NetVLAD and contributes the Pittsburgh-250k dataset and benchmark that the paper builds on and compares against.","marker":"[6]"},{"why":"introduces GeM pooling and the fine-tuning recipe for image retrieval that G2M upgrades.","marker":"[38]"},{"why":"describes SALAD, the main recent aggregation baseline, and supplies the FT4 fine-tuning used in training.","marker":"[23]"},{"why":"describes BoQ, the recent query-based aggregator whose 12,288-dimensional descriptor is the main high-dimension comparison.","marker":"[3]"},{"why":"introduces CosPlace and the UTM grid-cell class partition with N×L grouping that SLA adapts for multiple datasets.","marker":"[9]"},{"why":"provides GSV-Cities, the base training set whose original place-ID labels anchor the aligned label framework.","marker":"[1]"},{"why":"is the concurrent multi-dataset training work (CliqueMining) that SLA is directly compared against in Table 6.","marker":"[24]"},{"why":"provides the multi-similarity loss used to train all SuperPlace models.","marker":"[49]"}],"fun_headline_variants":["Classic GeM and NetVLAD beat foundation-model VPR tricks","SuperPlace revives classical aggregators for top VPR","Two GeMs cut descriptor size tenfold, top VPR benchmarks","Secondary fine-tuning pushes NetVLAD to #1 on MSLS"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the gain of the new two-GeM module over a plain GeM was not inflated by tuning its rank and activation on the same Pittsburgh test set that is later reported as the final result; the paper notes in Section 4.4 that those settings were selected using Pitts-30k-test outcomes.","fun_headline_variants_meta":{"raw":{"variants":["Classic GeM and NetVLAD beat foundation-model VPR tricks","SuperPlace revives classical aggregators for top VPR","Two GeMs cut descriptor size tenfold, top VPR benchmarks","Secondary fine-tuning pushes NetVLAD to #1 on MSLS"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000252,"raw_usage":{"total_tokens":1607,"prompt_tokens":1037,"completion_tokens":570,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":653,"completion_tokens_details":{"reasoning_tokens":496}},"tokens_in":653,"tokens_out":570,"duration_ms":6975,"temperature":1.0,"reasoning_tokens":496,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:36:52.903968+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain G2M with the GCA rank and activation selected on a validation split that is disjoint from every reported test set, then evaluate on Pitts-30k-test; if the Recall@1 advantage over plain GeM shrinks to well below the reported +0.7, the compact-aggregation claim is partly an artifact of test-set tuning.","supporting_citations":[{"cited_title":"Netvlad: Cnn architecture for weakly supervised place recognition","cited_arxiv_id":null,"evidence_quote":"defines NetVLAD and contributes the Pittsburgh-250k dataset and benchmark that the paper builds on and compares against."},{"cited_title":"Fine- tuning cnn image retrieval with no human annotation.IEEE transactions on pattern analysis and machine intelligence, 41(7):1655–1668, 2018","cited_arxiv_id":null,"evidence_quote":"introduces GeM pooling and the fine-tuning recipe for image retrieval that G2M upgrades."},{"cited_title":"Optimal transport ag- gregation for visual place recognition","cited_arxiv_id":null,"evidence_quote":"describes SALAD, the main recent aggregation baseline, and supplies the FT4 fine-tuning used in training."},{"cited_title":"Boq: A place is worth a bag of learnable queries","cited_arxiv_id":null,"evidence_quote":"describes BoQ, the recent query-based aggregator whose 12,288-dimensional descriptor is the main high-dimension comparison."},{"cited_title":"Re- thinking visual geo-localization for large-scale applications","cited_arxiv_id":null,"evidence_quote":"introduces CosPlace and the UTM grid-cell class partition with N×L grouping that SLA adapts for multiple datasets."},{"cited_title":"Gsv-cities: Toward appropriate supervised visual place recognition.Neurocomputing, 513:194–203, 2022","cited_arxiv_id":null,"evidence_quote":"provides GSV-Cities, the base training set whose original place-ID labels anchor the aligned label framework."},{"cited_title":"Close, But Not There: Boosting Geographic Distance Sensitivity in Visual Place Recognition","cited_arxiv_id":"2407.02422","evidence_quote":"is the concurrent multi-dataset training work (CliqueMining) that SLA is directly compared against in Table 6."},{"cited_title":"Multi-similarity loss with general pair weighting for deep metric learning","cited_arxiv_id":null,"evidence_quote":"provides the multi-similarity loss used to train all SuperPlace models."}],"review_version":1}