{"id":"0b9bf952-5667-4255-afc5-92fceeaf91a7","arxiv_id":"2608.05356","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":10,"one_line_summary":"An LoD-aware object-level pipeline plus the new LoDA benchmark achieves 95.0% ACC / 83.0% mIoU on Subiaco LiDAR maps and 96.81% mAcc / 89.52% mIoUch on Urb3DCD-V2, beating reported baselines.","lead":"This paper introduces LoDA, a new benchmark for object-level 3D change detection in multi-temporal urban LiDAR maps, and a pipeline that labels objects as Added, Removed, Increased, Decreased, or Unchanged using detection-limit-aware gating. The method reports state-of-the-art numbers on LoDA and on the public Urb3DCD-V2 dataset, which matters because keeping HD maps current means knowing exactly which objects changed and whether the change is observable.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Urb3DCD-V2 transfer result rests on an unspecified projection from five object-level labels to seven point-level classes; without that mapping, the 3.18 mIoUch gain is not reproducible.","rationale":"The reader's weakest assumption focuses on possible leakage from the LoDA annotation protocol: the LoD95 observability aid and exclusion of QA-ambiguous cases could encode the same suppression the method performs. That is a real concern, but it remains speculative because the text explicitly says the aid 'does not directly determine the released label' and that ambiguous cases were excluded rather than assigned. The label-mapping omission on Urb3DCD-V2 is concrete and directly visible in the manuscript: Section 5.4 says only 'projecting each instance-level decision to point labels,' yet Table 3 Panel B reports per-class IoUs for classes that do not align one-to-one with the method's output. Since Urb3DCD-V2 is the independent, publicly available validation, an unspecified projection undermines the strongest evidence for generalization. The LoDA benchmark issue and the Urb3DCD-V2 projection issue are both evaluation-fairness problems, hence 'partial' agreement, but the projection omission is more easily settled and more load-bearing for the external claim. The verdict remains CONDITIONAL because the issue is addressable by releasing the mapping and re-running the projection; no rejection is warranted without evidence of intentional bias.","tokens_in":17868,"tokens_out":9376,"duration_ms":86635,"concrete_test":"Publish the exact label projection and the code that implements it, for example a mapping from (semantic class, five-class change label) to Urb3DCD-V2 point classes. Then re-run Table 3 with two alternative plausible mappings: (a) Increased building maps to New building and Decreased vegetation maps to Missing vegetation; (b) both Increased and Decreased map to Unchanged for buildings and vegetation. If mIoUch changes by more than about 1 point, or if the published mapping cannot be applied without further ad hoc rules, the external-transfer claim is not robust and Table 3 should be re-reported with the mapping fixed in advance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest independent evidence for the central claim is the Urb3DCD-V2 transfer test (Sec. 5.4), but the paper never specifies how the method's five object-level labels (Added, Removed, Increased, Decreased, Unchanged) are projected onto Urb3DCD-V2's seven point-wise classes (Unchanged, New building, Demolition, New vegetation, Vegetation growth, Missing vegetation, Mobile). The mapping is not one-to-one: for example, a building labeled Increased has no exact target class, and decreased vegetation could plausibly map to Missing vegetation or Unchanged. Despite this, Table 3 Panel B reports per-class IoUs for all seven classes, which implies a concrete projection that is never stated. Because the paper emphasizes frozen thresholds and 'no per-dataset retuning,' the only benchmark-specific adaptation on Urb3DCD-V2 is this projection; if the projection was chosen on the Urb3DCD-V2 validation split, or if it is arbitrary, the 3.18 point mIoUch advantage over supervised baselines is not a controlled comparison. This is a verifiable omission in the manuscript text, not a claim about author intent.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LoDA, a new object-level change detection benchmark for multitemporal vehicle LiDAR, and a three-stage pipeline that couples detection-limit-aware registration, geometric object proxies with rule-based semantic/instance segmentation, and LoD-gated height/volume/normal-displacement cues to assign five change labels (Added, Removed, Increased, Decreased, Unchanged). The method is evaluated on LoDA and, with frozen thresholds, on Urb3DCD-V2 under the official point-wise protocol, reporting strong improvements over published baselines (95.0% ACC, 90.8% mF1, 83.0% mIoU on LoDA; 96.81% mAcc, 89.52% mIoUch on Urb3DCD-V2).","tokens_in":18140,"tokens_out":5946,"duration_ms":47705,"significance":"If the results are reproducible, the paper makes a valuable contribution: it formalizes a level-of-detection notion for object-level change detection, releases a benchmark with explicit LoD information, and demonstrates cross-dataset transfer without retuning. The manuscript is notably detailed in reporting thresholds, ablations, and runtime, and the external Urb3DCD-V2 evaluation is a strong design choice. However, the unspecified baseline-to-object mapping on LoDA and the unspecified label projection on Urb3DCD-V2, together with the potential benchmark circularity from LoD-assisted annotation, leave the central quantitative claims insufficiently supported.","major_comments":[{"comment":"Section 5.4 states that instance-level decisions are 'projected to point labels' on Urb3DCD-V2, but the projection from the five object-level labels (Added, Removed, Increased, Decreased, Unchanged) to the seven point-wise classes (Unchanged, New building, Demolition, New vegetation, Vegetation growth, Missing vegetation, Mobile) is never specified. Table 3 Panel B reports per-class IoUs for all seven classes, which implicitly requires a concrete mapping (e.g., how a building labeled 'Increased' is assigned, or how vegetation 'Decreased' maps to 'Missing vegetation' versus 'Unchanged'). Without this mapping, the reported 96.81% mAcc and 89.52% mIoUch are not reproducible, and the comparison against supervised point-wise baselines is not controlled. Please provide the exact label projection rule, specify whether it was fixed before seeing Urb3DCD-V2 labels, and state how unmatched object predictions are converted to point labels.","section":"Section 5.4 and Table 3, Panel B"},{"comment":"The annotation protocol in Section 3 states that 'The local LoD95 map is available during annotation only as an observability aid' and that 'Cases that remain ambiguous after QA are excluded from the benchmark statistics rather than forced to follow the output of the proposed method.' This creates a potential circularity: if annotators relied on the LoD95 map when deciding that a low-observability change is indeterminate, and if those ambiguous cases are then excluded from evaluation, the benchmark's ground truth may encode the same suppression of low-observability changes that the method's LoD gating performs. The paper should report how many candidate instances were excluded at each stage, the reasons for exclusion, and ideally include a secondary evaluation that assigns a default label (e.g., Unchanged) to all excluded cases, so readers can assess whether the LoDA results are inflated by the exclusion. This is load-bearing because the LoDA benchmark is the primary vehicle for the central quantitative claim.","section":"Section 3 (Data Collection)"},{"comment":"The LoDA evaluation compares object-based outputs against point-wise and voxel-wise baselines, but the protocol for converting baseline per-point predictions into the object-level five-class labels used in Tables 1 and 2 is not described. The paper does not state whether baseline predictions are aggregated by the same instance segmentation, by connected components, by voting, or by ground-truth instances, nor how a baseline that emits no object prediction is handled. Without this protocol, the reported 8.7 mIoU improvement over EFS KPConv is not a controlled comparison. Please specify the exact mapping from baseline point scores/classes to object predictions (including thresholds and grouping), or report baselines under their native point-wise protocol with an additional object-level aggregation step that is described in full.","section":"Section 5.1 and Tables 1/2"}],"minor_comments":[{"comment":"Equation (2) is presented as the definition of LoD95, but it is the M3C2-EP error propagation formula from [54]; please cite [54] at the equation and clarify which terms are adopted unchanged and which are estimated in this work.","section":"Section 4.1, Eq. (2)"},{"comment":"The sensitivity sweep for α_LoD over {1.0, 1.1, 1.2} reports that performance was stable, but no validation numbers are given; please report the validation mIoU for each α value.","section":"Section 4.3"},{"comment":"The statement that 'All rule thresholds ... were selected once on the LoDA validation split' should be reconciled with the many precise thresholds (e.g., 0.60 m, 0.12, 0.35 m, 0.18) that also appear in Section 4.3; please clarify whether any thresholds were taken from prior work or set by physical reasoning rather than the validation sweep.","section":"Section 5.1"},{"comment":"The paper says 'final labels are independently verified using synchronized annotation views and QA, reducing protocol bias,' but then states that the LoD95 map is used as an observability aid; please clarify the exact role of LoD95 in the verification step and how the annotators were instructed to use it.","section":"Section 3"},{"comment":"The reference numbering for EFS KPConv and DC3DCD is ambiguous because [20] is cited for both; please disambiguate the entries.","section":"Table 1"},{"comment":"The sentence 'Because compatible five-label object-level baselines are unavailable' should be supported by a short description of the adaptation protocol used for existing point-wise baselines, otherwise it is unclear whether the comparison is apples-to-apples.","section":"Section 5.2"}],"recommendation":"major_revision","confidential_remarks":"The paper's benchmark-release claims should be verified: no URL or download link is provided. Additionally, the author list includes affiliations with a commercial entity; the acknowledgments disclose funding, which is fine. The referee should ask the editor to ensure that the data release statement is complete. The citation pattern seems reasonable. The main risk is the two unspecified evaluation protocols; both are fixable within the scope of a revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a serious, unusually detailed engineering paper with a new benchmark (LoDA) and a sensible LoD-gating idea. The central claim is plausible and the ablations support it, but the evaluation has two reproducibility gaps—the LoDA benchmark's ambiguous-case exclusion and the unspecified five-to-seven-class projection on Urb3DCD-V2—that need to be fixed before the numbers, especially the 8.7-point mIoU margin, can be taken at face value.\n\nWhat is actually new: the LoDA benchmark itself (bi-temporal vehicle LiDAR of Subiaco with semantic, instance, and five-class object-level change labels) is a concrete contribution for the urban mapping community. The method integrates pose covariance into a spatially varying detection limit (M3C2-EP style), uses geometry-only proxies before semantic/instance formation, and gates height/volume/normal cues by the local LoD. That's a sensible, honest way to suppress false positives from misalignment and density variation. The paper is transparent about thresholds, with ablations on LoDA and a frozen-threshold transfer to Urb3DCD-V2. The ablation showing a big drop without LoD gating is the strongest evidence for the core idea.\n\nWhere it gets soft. First, the Urb3DCD-V2 transfer is the paper's best independent evidence, but the projection from the method's five object-level labels to Urb3DCD-V2's seven point-wise classes is never stated. Table 3 lists per-class IoUs for all seven classes, so a concrete mapping exists; without it, the 3.18-point mIoUch gain is not reproducible, and if the mapping was tuned on the validation split, the comparison to supervised baselines is less clean. This is a straightforward omission to fix.\n\nSecond, the LoDA benchmark construction has a mild circularity risk. Annotators had the LoD95 map as an 'observability aid' and QA-ambiguous cases were excluded from statistics. If that exclusion correlates with low-observability changes, the benchmark is skewed toward exactly what the method's LoD gating suppresses. The paper discloses this, which is good, but the risk deserves a sensitivity analysis—e.g., re-annotating a sample of excluded cases or reporting how many were excluded.\n\nThird, the baseline comparison on LoDA is not fully specified: point-based baselines output point-wise scores, and the protocol to aggregate them to object-level five-class labels is absent. Without it, the 8.7-point margin over the best baseline isn't a clean measurement.\n\nThe citation pattern is fine; M3C2-EP and cut-pursuit are properly credited. Who this is for: researchers working on urban LiDAR map updating and change detection, especially people who care about explicit observability constraints. The benchmark, once available, is a useful evaluation resource.\n\nRecommendation: send it to peer review. The engineering is real, the paper is honest, and the flaws are addressable in revision. A serious referee should ask for the projection mapping, the baseline aggregation protocol, and a sensitivity analysis around the benchmark's ambiguous-case exclusion.","headline":"A detailed, honest object-level change-detection paper with a new benchmark and a sensible LoD-gating idea; the evaluation needs two reproducibility fixes before the headline margins are trustworthy.","tokens_in":18695,"tokens_out":3948,"would_cite":true,"duration_ms":31867,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By gating change decisions on a spatially varying level of detection, an object-level LiDAR change-detection pipeline beats the strongest baselines by 8.7 mIoU points on its own benchmark and transfers with frozen thresholds to Urb3DCD-V2.","keywords":["LiDAR change detection","object-level change detection","level of detection","multi-temporal urban mapping","benchmark dataset","point cloud registration","instance segmentation","map updating"],"falsifier":"Re-annotate a random sample of the LoDA cases that were excluded as ambiguous after QA, together with a stratified sample of grid cells with high LoD95, without letting annotators see any LoD map; then run the full pipeline and the strongest baseline on that sample. If the pipeline's mIoU margin over the baseline drops below the ablation's LoD-gating effect of 7.8 points, the benchmark construction rather than the method explains the result.","tokens_in":17658,"feed_emoji":"🗺️","tokens_out":9868,"duration_ms":79904,"temperature":0.7,"pith_summary":"The paper argues that object-level change detection in multi-temporal urban LiDAR should be driven by an explicit, spatially varying detection limit rather than fixed thresholds or implicit learned priors. To that end, it builds a pipeline that fine-registers two vehicle-survey point clouds, computes a per-location Level of Detection (LoD95) from local roughness, sampling density, and pose uncertainty, and labels geometry-based, class-consistent instance pairs as Added, Removed, Increased, Decreased, or Unchanged using height, volume, and normal-displacement cues gated by LoD95. The paper also releases LoDA, an object-level benchmark built from 2023 and 2025 multimodal surveys of Subiaco, with fused maps, semantic and instance annotations, and five-class change labels. On LoDA the pipeline reports 95.0% accuracy, 90.8% macro F1, and 83.0% macro IoU, exceeding the strongest baseline by 8.7 IoU and 4.4 F1 points; with frozen thresholds it transfers to Urb3DCD-V2 at 96.81% mean accuracy and 89.52% mean change IoU. A reader should care because the output is a structured, audit-friendly map-update record, and because the interpretable knob—declare a change only if geometry moved more than the local sensor can detect—is where the paper locates the improvement.","feed_headline":"Local detectability gates lift LiDAR change detection by 8.7 points","feed_subtitle":"Added, removed, grown, and shrunk objects become structured update records rather than point scores.","key_machinery":"The central object is the spatially varying Level of Detection, $\\mathrm{LoD}_{95}$, from Equation (2): a per-cell, normal-direction minimum detectable displacement that fuses local roughness (MAD-to-standard-deviation plane residuals), sampling density, and the translational block of the ICP pose covariance. It carries the argument because it converts a vague notion of observability into a quantitative gate: the change cues $\\Delta h$, $\\Delta V$, and $D_\\perp$ are only trusted when they exceed $1.10\\,\\mathrm{LoD}_{95}$, and the pairwise detection limit is the median $\\mathrm{LoD}_{95}$ over the overlap region. Around this gate the pipeline is organized as decoupled stages—Tukey-weighted registration, geometry-only proxy correspondence solved by Hungarian assignment, cut-pursuit superpoints with rule-based semantic and instance segmentation, and deterministic five-class decision rules—so that each stage's uncertainty is propagated forward instead of being absorbed by a learned black box.","core_discovery":"At the paper's center is the claim that a change label is only warranted when the observed displacement exceeds what the sensing system could plausibly produce through noise, misalignment, or density variation. The mechanism is the LoD95 field of Equation (2), a per-cell normal-direction detection limit computed as $\\mathrm{LoD}_{95}(g)=1.96\\sqrt{\\sigma^2_{23}(g)/N_{23}(g)+\\sigma^2_{25}(g)/N_{25}(g)+n_g^{\\top}\\Sigma_t n_g}$, where the two variance terms come from robust plane-fit residuals in each epoch, the $N$'s are local sample counts, and $\\Sigma_t$ is the translational block of the ICP pose covariance. Height differences $\\Delta h$ and normal-direction displacements $D_\\perp$ are only accepted when their magnitude exceeds $1.10\\,\\mathrm{LoD}_{95}$, and the volume and overlap statistics back the decision. The paper supports this claim with a controlled experiment: all thresholds are fixed once on the LoDA validation split, frozen for the test split and Urb3DCD-V2, and the ablation shows that removing the LoD gate costs 7.8 mIoU points, more than removing any single geometric cue. Stated at its strongest, the paper's discovery is that explicit observability modelling, not a larger network, is what separates reliable object-level change detection from per-point tile-based prediction.","pith_inferences":["A direct test of the paper's main hypothesis would be to take a strong learning baseline and apply the same LoD gate to its point-wise predictions; if the margin on LoDA shrinks toward zero, the gate rather than the geometry pipeline is doing the work.","The LoD95 formula is generic enough to be dropped into other multi-temporal sensing settings, but the pose-covariance term would need to be replaced by the uncertainty of whatever trajectory solution is used; a useful extension is per-sensor calibration of the fixed sensor term.","The benchmark's annotation protocol had access to the LoD95 map as an observability aid, so a user should treat the reported margin as conditional on labels that may already be biased toward detectable changes; re-annotating the excluded ambiguous cases without the aid would settle this.","The per-block runtime scaling (47.6 to 75.8 seconds as points grow from 1.12 to 2.18 million) suggests the rule-based pipeline is practical at city-block scale, and the deterministic nature of all decisions makes it auditable for mapping products."],"forward_implications":["A map-update service can consume the output directly as structured records: each instance is assigned exactly one of Added, Removed, Increased, Decreased, or Unchanged, with a deterministic confidence score for ranking.","The method transfers across cities without per-dataset retuning: frozen thresholds yield 96.81% mAcc and 89.52% mIoUch on Urb3DCD-V2 under the official point-wise protocol, beating the strongest published baselines by 3.18 mIoUch points.","On LoDA, the largest gains are in the classes deep baselines handle worst: Increased and Decreased improve by 23.1 and 15.3 IoU points over the strongest baseline.","The ablation attributes the gains to the LoD gate: removing it cuts mIoU from 83.0 to 75.2, a larger drop than removing any single geometric cue such as height, volume, or normal displacement.","Because all thresholds are frozen, the benchmark enables controlled object-level evaluation without hidden per-dataset tuning; candidate instance pairs are generated automatically and final labels are QA-verified."],"supporting_citations":[{"why":"supplies the normal-direction uncertainty model based on roughness and sampling density that LoD95 adapts into Equation (2).","marker":"[28]"},{"why":"extends the M3C2 comparison with error propagation, supporting the pose-covariance term in LoD95.","marker":"[54]"},{"why":"provides the closed-form ICP covariance estimate used for the translational pose uncertainty in LoD95.","marker":"[11]"},{"why":"defines the strongest point-wise change-detection baselines and feature-stabilization ablations that the pipeline must beat on Urb3DCD-V2.","marker":"[18]"},{"why":"supplies the EFSKPConv and DC3DCD baselines and object-proxy change cues closest to the proposed object-level setting.","marker":"[20]"},{"why":"introduces the Urb3DCD change-detection dataset and Siamese KPConv training protocol used for the public transfer evaluation.","marker":"[19]"},{"why":"provides the KPConv backbone used by the SiamKPConv baseline family on both benchmarks.","marker":"[50]"},{"why":"provides the cut-pursuit algorithm used to form superpoints for semantic and instance segmentation.","marker":"[29]"},{"why":"supplies the Hungarian assignment used for the gated bipartite matching of geometry-only object proxies.","marker":"[37]"}],"fun_headline_variants":["LoDA: detectability gates lift change detection by 8.7 IoU","LoD gating lifts change detection by 8.7 IoU points","LoDA: 8.7-point IoU gain via level-of-detection gates","Explicit detectability limits lift LiDAR change detection","LoDA: new benchmark and 8.7-point IoU lift"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"LoDA's ground-truth labels are assumed to be independent of the method's observability reasoning, but annotators used the LoD95 map as an observability aid and excluded ambiguous cases from benchmark statistics; if that aid and exclusion already suppress the same low-observability changes the pipeline is designed to ignore, the reported 8.7-point margin is partly a consequence of the benchmark construction.","fun_headline_variants_meta":{"raw":{"variants":["LoDA: detectability gates lift change detection by 8.7 IoU","LoD gating lifts change detection by 8.7 IoU points","LoDA: 8.7-point IoU gain via level-of-detection gates","Explicit detectability limits lift LiDAR change detection","LoDA: new benchmark and 8.7-point IoU lift"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001378,"raw_usage":{"total_tokens":5691,"prompt_tokens":1164,"completion_tokens":4527,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":780,"completion_tokens_details":{"reasoning_tokens":4429}},"tokens_in":780,"tokens_out":4527,"duration_ms":31406,"temperature":1.0,"reasoning_tokens":4429,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T14:43:28.429292+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-annotate a random sample of the LoDA cases that were excluded as ambiguous after QA, together with a stratified sample of grid cells with high LoD95, without letting annotators see any LoD map; then run the full pipeline and the strongest baseline on that sample. If the pipeline's mIoU margin over the baseline drops below the ablation's LoD-gating effect of 7.8 points, the benchmark construction rather than the method explains the result.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the KPConv backbone used by the SiamKPConv baseline family on both benchmarks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the normal-direction uncertainty model based on roughness and sampling density that LoD95 adapts into Equation (2)."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"extends the M3C2 comparison with error propagation, supporting the pose-covariance term in LoD95."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the closed-form ICP covariance estimate used for the translational pose uncertainty in LoD95."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"defines the strongest point-wise change-detection baselines and feature-stabilization ablations that the pipeline must beat on Urb3DCD-V2."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the EFSKPConv and DC3DCD baselines and object-proxy change cues closest to the proposed object-level setting."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"introduces the Urb3DCD change-detection dataset and Siamese KPConv training protocol used for the public transfer evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the cut-pursuit algorithm used to form superpoints for semantic and instance segmentation."}],"review_version":1}