{"id":"0e12d408-8403-4c79-8e7c-0ac2bb9fb53c","arxiv_id":"2505.07198","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"KDF reduces catastrophic forgetting in LiDAR place recognition by distilling soft ranking information from the old model and concatenating old and new model features at test time.","lead":"This paper introduces KDF, a continual learning framework for LiDAR place recognition that combines ranking-aware knowledge distillation with feature fusion. It reports better mean Recall@1 and lower forgetting scores than prior continual LPR methods across four network backbones.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Knowledge fusion confounds the forgetting metric; a freeze-and-concatenate baseline may reproduce the gains, so the distillation's contribution is unverified.","rationale":"The reader's weakest assumption correctly flags that knowledge fusion retains the old model at test time, making the reported forgetting score partly a byproduct of the evaluation design. I agree that this is load-bearing, but I would sharpen it: the issue is not only whether the assumption is 'allowed' but that the comparison to single-model baselines is unfair and the source of the improvement is unidentified. The paper's ablation (Table V) shows that adding fusion improves mR@1 by roughly 2 points and F by about 2.3 points over the best no-fusion variant, but there is no control that adds fusion to a fine-tuning-only model. Without such a control, the central claim that ranking-aware distillation enables continual LPR is not established; the fusion module may be doing most of the work. The multi-step ambiguity compounds this: Eq. 11 only defines fusion for one old and one new model, and the caption of Fig. 2 says the fused knowledge is 'in preparation for future steps' without specifying how a concatenated feature is used to train the next model. The concrete test I propose would settle whether the distillation losses contribute beyond the fusion mechanism. A CONDITIONAL verdict is appropriate: the method may be sound, but the paper must supply this control and clarify the multi-step fusion protocol before the central claim can be accepted.","tokens_in":12533,"tokens_out":7463,"duration_ms":74164,"concrete_test":"Run a fusion-only baseline under the 4-step protocol: at each new task, fine-tune the current model from the previous model using only the triplet/contrastive loss (no LRKD, no LDKD), but at evaluation concatenate the previous model's feature with the current model's feature as in Eq. 11. Compare mean Recall@1 and forgetting score to KDF. If the fusion-only baseline matches KDF's forgetting score within a small margin, then the ranking-aware distillation is not the source of the forgetting improvement. Also report the new model's alone (no fusion) forgetting score after step 4; if it is close to Fine-Tuning's, the 'overcoming forgetting' claim applies to the system, not the learned encoder.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that KDF 'overcomes catastrophic forgetting' is confounded by the knowledge fusion module. At inference, Eq. 11 concatenates the frozen previous model's descriptor with the new model's descriptor. The forgetting score F (Eq. 12) is computed on this fused system, so the previous model's own retained knowledge on old tasks directly inflates R_{T,t}. The paper provides no control experiment isolating the fusion benefit from the distillation benefit; a trivial baseline that fine-tunes the new model without any KD loss and concatenates the old and new features at test time might achieve similarly low forgetting scores. If so, the reported gains over InCloud/CCL/MICL are due to extra inference-time capacity (two encoders) rather than to ranking-aware distillation. Additionally, the paper does not specify how the fused representation is maintained across more than two environments: if at step 3 the 'old' model is the step-2 model, the original step-1 model is discarded, and the chain's ability to retain early-task knowledge is unverified; if instead a fused model is used for subsequent distillation, the training procedure is not described.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes KDF, a continual learning framework for LiDAR place recognition. It introduces a ranking-aware knowledge distillation loss that combines a ranking-based distillation term with a symmetric KL-divergence distribution-based term, and a knowledge fusion module that concatenates the old and new models' descriptors at test time. Experiments are conducted with a 4-step protocol on Oxford, DCC, Riverside, and In-house using four backbones (PointNetVLAD, LoGG3D-Net, MinkLoc3D, TransLoc3D), reporting mean Recall@1 and forgetting scores, plus KITTI generalization results. The authors claim that KDF can be applied to different networks and surpasses InCloud, CCL, and MICL in mean Recall@1 and forgetting score.","tokens_in":12742,"tokens_out":5256,"duration_ms":52307,"significance":"Ranking-aware distillation is a sensible and potentially useful idea: adapting Smooth-AP-style ranking relaxation to continual LPR is novel, and the loss is cleanly defined and architecture-agnostic. The experimental breadth is a strength: four backbones, two training losses, comparisons with three recent continual LPR methods, and a generalization protocol on KITTI. The ablation study is well structured and confirms that each proposed loss term contributes positively. However, the central forgetting claim is weakened by the evaluation design: the forgetting score is computed on a fused system that includes the frozen old model, so the reported low forgetting is partly an artifact of retaining the old model rather than evidence that the new model retains old-domain knowledge. The paper is potentially publishable after the evaluation is deconfounded and the multi-step fusion protocol is specified.","major_comments":[{"comment":"The forgetting score F is computed on the fused descriptor f_fuse = {f^t ⊕ f^{t+1}}, which includes the frozen old model's descriptor. Since the old model has already seen the old domains, the fused system's Recall@1 on old tasks is inflated by the old model's retained knowledge; the new model's own forgetting is not measured. The paper's own ablation (Table V) shows that adding Knowledge Fusion to the full distillation configuration reduces F from 5.17 to 1.20, and fusion alone reduces F from 7.99 to 4.64, so the fusion module, rather than the ranking-aware distillation, drives most of the reported forgetting improvement. Please report Recall@1 and F for the new model alone (without fusion), and add a control baseline that fine-tunes without any KD loss but concatenates the frozen-old and new descriptors at test time; without this control, the contribution of ranking-aware distillation to the forgetting reduction is unverified.","section":"Section IV-C, Eq. (11), Section V-C, Eq. (12)"},{"comment":"The paper does not specify how the fused representation is maintained across the four sequential environments. At step 3, if the \"old model\" is the model after step 2, the step-1 model is discarded; if instead the fused descriptor from step 2 is used as the old knowledge, the training procedure for that fused feature extractor is not described. Consequently, the reported 4-step forgetting scores are not reproducible, and the claim that early-task knowledge is retained beyond two steps is unverified. Please define the exact fusion protocol for steps t=2,3,4, including which model is frozen and what is concatenated at test time for each step.","section":"Section IV-C and Section V-A"}],"minor_comments":[{"comment":"The denominator N_b^3 in Eq. (5) appears inconsistent with the summation over N_b^2 pairs (q,i); as written, the ranking distillation loss magnitude scales inversely with batch size, which is particularly relevant because the paper uses batch expansion. If the normalization is intentional, please clarify the scaling rationale.","section":"Section IV-A, Eq. (5)"},{"comment":"The notation switches between E^t, E^{t+1}, f^t, and f^{t+1} without clarifying the relationship; please define the embedding notation once and use it consistently throughout.","section":"Section IV, Eqs. (5)-(11)"},{"comment":"The ablation table uses checkmarks without explicit row configuration labels; as presented, the reader cannot unambiguously map each row to the exact set of enabled components. Please enumerate the configurations.","section":"Table V"},{"comment":"The claim that \"KDF achieves the best performance on the 02, 07, and 08 sequences\" is not accurate for the triplet variant on sequence 08 (Ours: 26.10 vs MICL-T: 29.33 and MICL-C: 32.71); it is Ours-C that is best on 08. Please report per-variant results precisely and avoid conflating the two variants.","section":"Section V-E, last paragraph"},{"comment":"Several comparisons in Table II are within a fraction of a percentage point (e.g., Ours-C vs CCL in mean Recall@1), and no variance or multiple-seed results are reported. Please report standard deviations or at least clarify how many runs the reported numbers are based on.","section":"Section V-D, Table II"}],"recommendation":"major_revision","confidential_remarks":"The knowledge fusion module is effectively a test-time ensemble with the frozen old model, and the ablation shows that it dominates the forgetting improvement. The editor may wish to require the unfused new-model results and the freeze-and-concatenate control as a condition of acceptance. If the authors cannot provide those results, the paper should be re-evaluated for rejection. The paper also does not release code, which combined with the unspecified multi-step fusion protocol further limits reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the bottom line: the paper's main claim is undercut by an unfair evaluation setup. The knowledge fusion module concatenates the frozen old model's descriptor with the new model's at test time (Eq. 11). Since the old model never changes, it retains its old-domain knowledge perfectly, so the forgetting score (Eq. 12) on the fused system is largely predetermined. A trivial baseline that fine-tunes without any distillation and concatenates old and new features would likely produce low forgetting too. The paper never reports such a control, so you cannot attribute the improvements to the ranking-aware distillation.\n\nThat said, the ranking-aware distillation idea is genuinely new and plausible. Adapting Smooth AP to preserve retrieval ranking across domains is a reasonable way to transfer place-recognition knowledge. The paper also does a broad empirical job: four backbones, four datasets, several baselines including recent MICL, and a KITTI generalization test. The ablation shows the distillation losses help even without fusion: forgetting drops from 7.99 to 5.17 on MinkLoc3D. So there is a real effect, just not the one advertised.\n\nThe soft spots are concentrated in the evaluation. Besides the fusion confound, the multi-step protocol is underspecified. After training on a third environment, do you keep both prior descriptors? If you keep only the immediately previous model, early-task knowledge is lost; if you keep all, the dimension grows and the training procedure is not described. The distribution distillation loss (Eq. 7) applies KL divergence to embeddings without explaining how vectors become distributions. Also, no code, no error bars, and key hyperparameters (tau, m) are missing.\n\nThe citation pattern is fine; it engages with the relevant prior work.\n\nOverall, this paper is for a niche audience of continual-LPR researchers. The ranking-aware loss is worth exploring, but the current manuscript needs a major revision. The authors should compare against a like-for-like baseline that either uses only the new model at test time or allows all methods the same old-model concatenation. They also need to spell out the protocol for more than two environments.\n\nI'd still send it to peer review, because there is a real idea here and the experiments are broad enough to deserve referee time. But I'd expect the reviewers to demand the control experiment and a rewritten evaluation section.\n\nReading group: maybe, as a case study in evaluation pitfalls.","headline":"Useful distillation idea for continual LiDAR place recognition, but the fusion module makes the main forgetting comparison unfair; needs a baseline that isolates distillation from old-model concatenation.","tokens_in":13251,"tokens_out":5060,"would_cite":false,"duration_ms":47389,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"KDF claims LiDAR place-recognition models can learn new environments without forgetting old ones if knowledge distillation preserves retrieval ranking and the old model's features are fused with the new model's at test time.","keywords":["continual learning","LiDAR place recognition","knowledge distillation","catastrophic forgetting","knowledge fusion","ranking-aware loss","point cloud retrieval","SLAM"],"falsifier":"Run the four-step protocol but evaluate using only the new model's descriptor at test time, without concatenating the old model's features; if mean Recall@1 and forgetting score then fall to the no-fusion ablation level or below InCloud and CCL, the claim that the learned model itself resists forgetting is unsupported.","tokens_in":12347,"feed_emoji":"🗺️","tokens_out":6871,"duration_ms":61006,"temperature":0.7,"pith_summary":"The paper is trying to establish that a retrieval model can learn new places without forgetting old ones if, during training, it is forced to preserve not just feature similarity but the order in which old and new models rank candidate places, and if, at test time, the old model's knowledge is explicitly kept alongside the new model's. The authors build a continual-learning wrapper, KDF, around any LiDAR place-recognition encoder. They show it reduces catastrophic forgetting on a four-dataset sequential benchmark and improves generalization to an unseen city, across four different encoder backbones. If true, place-recognition systems in SLAM and autonomous driving could be updated incrementally as vehicles enter new environments, instead of being retrained from scratch.","feed_headline":"Ranking-aware distillation cuts LiDAR place-recognition forgetting","feed_subtitle":"Keeping the old model's features alongside the new model's beats state-of-the-art continual place recognition on four backbones.","key_machinery":"The central machinery is the soft ranking function $R(q,i)=1+\\sum_{j\\neq i} G((S(q,j)-S(q,i))/\\tau)$, a Smooth-AP-style differentiable proxy for the rank of candidate $i$ against query $q$. KDF computes this ranking for embeddings from the old and new encoders and minimizes their absolute difference, then adds a symmetric KL divergence between the two embedding distributions; place-recognition metric learning with a triplet margin loss runs on top. A separate knowledge fusion module then concatenates old and new descriptors into a fused descriptor used at retrieval time.","core_discovery":"On the paper's own terms, the discovery is that catastrophic forgetting in LiDAR place recognition is not a fixed cost of sequential training: it can be cut sharply by aligning the old and new models on the quantity the retrieval task actually cares about, the ranking of candidate matches, and by refusing to discard the old model. The paper reports KDF improving mean Recall@1 over InCloud by 5.56 points and lowering the forgetting score by 9.93 points when averaged across four encoder backbones under the triplet-loss protocol, with the best single backbone (MinkLoc3D) reaching a forgetting score of 1.20. On an unseen KITTI evaluation, the paper reports KDF achieving the best overall Recall@1 on sequences 02, 07, and 08 among the compared methods.","pith_inferences":["Beyond the paper, a deployment-time descriptor that concatenates every old model grows linearly with the number of environments; a practical system would need a pruning or re-distillation policy that KDF does not specify.","Beyond the paper, the ranking-distillation idea should transfer to visual and multimodal place recognition, since the retrieval objective is identical; the paper lists visual place recognition as future work.","Beyond the paper, the current experiments fuse only the two most recent models, so how KDF behaves on a third sequential environment is untested; training on a fifth domain and checking early-domain Recall@1 would settle it."],"forward_implications":["A robot or vehicle entering a new city can update its place-recognition model on new LiDAR data while keeping high Recall@1 on previously visited cities, without storing the full old training set.","The ranking-aware distillation loss can be dropped into existing descriptors such as PointNetVLAD, LoGG3D-Net, MinkLoc3D, and TransLoc3D, so continual place recognition does not require a new architecture.","Because retrieval is the actual downstream task, preserving the ranking of candidate matches transfers the knowledge that matters for place recognition rather than only matching intermediate feature statistics.","The fused old-plus-new descriptor improves generalization: after sequential training, the model scores higher on an unseen dataset than several baselines, suggesting the old model contributes transferable knowledge.","The framework's components are separable: ablations show ranking distillation, symmetric-KL distribution distillation, and fusion each add to mean Recall@1 and reduce forgetting on MinkLoc3D."],"supporting_citations":[{"why":"Supplies the PointNetVLAD encoder and the In-house dataset used in the sequential and generalization protocols.","marker":"[4]"},{"why":"Supplies the ranking-based loss formulation that the ranking-aware distillation loss adapts.","marker":"[7]"},{"why":"Defines the continual-LPR benchmark, the 4-step protocol, and the angle-based distillation baseline plus distillation relaxation.","marker":"[13]"},{"why":"Provides the contrastive continual-LPR baseline and loss variant used for the Ours-C comparison.","marker":"[14]"},{"why":"Supplies the Smooth-AP ranking approximation that makes the ranking-aware distillation loss differentiable.","marker":"[21]"},{"why":"Provides the mutual-information continual-learning baseline KDF is compared against.","marker":"[38]"},{"why":"Inspires the dual-model knowledge fusion mechanism.","marker":"[40]"},{"why":"Supplies the Oxford RobotCar dataset used as the first sequential environment.","marker":"[41]"},{"why":"Supplies the DCC and Riverside MulRan environments used in the 4-step protocol.","marker":"[42]"},{"why":"Supplies the KITTI dataset used for the unseen-domain generalization evaluation.","marker":"[43]"}],"fun_headline_variants":["Ranking-aware distillation and fusion cut LiDAR place-recognition forgetting","Fusing old and new models staves off LiDAR place-recognition forgetting","Teach the ranking, keep the old model: less forgetting in LiDAR place recognition","New loss and fusion beat catastrophic forgetting in LiDAR place recognition","Ranking-aware knowledge distillation reduces LiDAR place-recognition forgetting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's forgetting gains assume that the previous model stays frozen and available at test time so its features can be concatenated with the new model's; if a system cannot store that old model, most of the reported improvement disappears, and the paper does not spell out how the fused representation is maintained after a third environment is added.","fun_headline_variants_meta":{"raw":{"variants":["Ranking-aware distillation and fusion cut LiDAR place-recognition forgetting","Fusing old and new models staves off LiDAR place-recognition forgetting","Teach the ranking, keep the old model: less forgetting in LiDAR place recognition","New loss and fusion beat catastrophic forgetting in LiDAR place recognition","Ranking-aware knowledge distillation reduces LiDAR place-recognition forgetting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000432,"raw_usage":{"total_tokens":2163,"prompt_tokens":864,"completion_tokens":1299,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":480,"completion_tokens_details":{"reasoning_tokens":1205}},"tokens_in":480,"tokens_out":1299,"duration_ms":10788,"temperature":1.0,"reasoning_tokens":1205,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:21:47.899948+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the four-step protocol but evaluate using only the new model's descriptor at test time, without concatenating the old model's features; if mean Recall@1 and forgetting score then fall to the no-fusion ablation level or below InCloud and CCL, the claim that the learned model itself resists forgetting is unsupported.","supporting_citations":[{"cited_title":"Pointnetvlad: Deep point cloud based retrieval for large-scale place recognition,","cited_arxiv_id":null,"evidence_quote":"Supplies the PointNetVLAD encoder and the In-house dataset used in the sequential and generalization protocols."},{"cited_title":"Improving point cloud based place recognition with ranking-based loss and large batch training,","cited_arxiv_id":null,"evidence_quote":"Supplies the ranking-based loss formulation that the ranking-aware distillation loss adapts."},{"cited_title":"Incloud: Incremental learning for point cloud place recognition,","cited_arxiv_id":null,"evidence_quote":"Defines the continual-LPR benchmark, the 4-step protocol, and the angle-based distillation baseline plus distillation relaxation."},{"cited_title":"Ccl: Continual contrastive learning for lidar place recognition,","cited_arxiv_id":null,"evidence_quote":"Provides the contrastive continual-LPR baseline and loss variant used for the Ours-C comparison."},{"cited_title":"Smooth-ap: Smoothing the path towards large-scale image retrieval,","cited_arxiv_id":null,"evidence_quote":"Supplies the Smooth-AP ranking approximation that makes the ranking-aware distillation loss differentiable."},{"cited_title":"Micl: Mutual information guided continual learning for lidar place recognition,","cited_arxiv_id":null,"evidence_quote":"Provides the mutual-information continual-learning baseline KDF is compared against."},{"cited_title":"Lifelong person re- identification via knowledge refreshing and consolidation,","cited_arxiv_id":null,"evidence_quote":"Inspires the dual-model knowledge fusion mechanism."},{"cited_title":"Mulran: Multimodal range dataset for urban place recognition,","cited_arxiv_id":null,"evidence_quote":"Supplies the DCC and Riverside MulRan environments used in the 4-step protocol."}],"review_version":1}