{"id":"0453920e-d9e9-4549-ad62-c674c471d651","arxiv_id":"2411.13620","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A pose-confidence method using a detached view-direction-free color network, Monte Carlo re-localization, and dynamic scene graph updates improves neural surface reconstruction under outlier camera poses.","lead":"Robust SG-NeRF improves 3D surface reconstruction when some camera poses are badly wrong, using a second color network that ignores viewing direction to tell good poses from bad ones, then re-localizing the bad ones. It reports large gains over SG-NeRF on both reconstruction quality and pose accuracy on the SG-NeRF and DTU datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The view-direction-free PSNR-gap outlier detector is never validated with known labels or a threshold sweep; with τ1=9 fixed, a misfiring confidence signal would misdirect both ray sampling and Monte Carlo re-localization.","rationale":"The reader's weakest-assumption analysis is correct: the confidence indicator is the linchpin of the method, and it is not quantitatively validated. My independent reading confirms that Sec. 3.1 introduces the |PSNR_o − PSNR_n| > τ1 rule without any justification beyond a qualitative figure, and no ablation removes or varies the confidence scheme. The DTU setup is an ideal testbed because outlier poses are synthetically injected with known labels, yet the paper does not use it to measure detection accuracy. A threshold sweep and a label-based ROC analysis would settle whether the mechanism actually works. I also note that the headline claim of 'consistently outperforming' is slightly overstated: on the Bell scene in Table 1, the proposed method has higher Chamfer distance and lower F-score than SG-NeRF (1.22 vs 0.98 and 0.65 vs 0.71). This is a secondary issue, but it supports keeping the verdict conditional rather than elevating it. The evaluation protocol for pose accuracy is also partially self-referential because the same indicator is used to filter outliers before alignment, which could bias the reported two-orders-of-magnitude improvements; however, the threshold-validation concern is more fundamental and is the one I would prioritize. Overall, the paper is a reasonable empirical contribution, but the core detector needs a direct, quantitative test before the robustness claims can be accepted unconditionally.","tokens_in":15003,"tokens_out":4914,"duration_ms":54745,"concrete_test":"Use the DTU injection protocol to obtain ground-truth inlier/outlier labels. During training on one or more DTU scenes, record PSNR_o and PSNR_n per image and compute the ROC curve and AUC for the discriminator |PSNR_o − PSNR_n|. Then re-run the full pipeline with τ1 ∈ {3, 6, 9, 12, 15} and report Chamfer distance, APE, and RPE per scene. If the AUC is not close to 1, or if the optimal τ1 varies substantially across scenes, the fixed-threshold confidence claim is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The method's central mechanism is the claim that |PSNR_o − PSNR_n| reliably separates inlier from outlier poses, with a fixed threshold τ1 = 9 (Sec. 3.1). This is load-bearing because the confidence update in Eq. 2, the outlier/inlier split that gates the re-projection and IoU losses, and the Monte Carlo re-localization in Sec. 3.3 all depend on this flag. The paper supports the indicator only with qualitative examples (Fig. 2) and favorable end results; it provides no ROC analysis, no per-scene classification accuracy, and no sensitivity study for τ1. This gap is especially conspicuous because the DTU experiments inject known outlier poses into 1/7 to 1/4 of images (Sec. 4.1), which would allow a direct label-based evaluation of the detector. The paper instead reports only that Monte Carlo re-localization was never triggered on DTU, so the DTU results do not validate the outlier-handling machinery at all. If the indicator misfires in either direction, the pipeline either starves inlier poses of sampling or wastes re-localization on healthy poses, and the quantitative gains over SG-NeRF could be driven by other components or by favorable dataset-specific behavior rather than by the proposed confidence scheme.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes RSG-NeRF, a neural surface reconstruction method built on NeuS that aims to handle both small pose noise and large outlier poses. The method uses a scene graph to obtain initial per-image confidence scores, adds a detached color network that omits viewing direction, and classifies poses as inliers or outliers based on the PSNR gap between the standard color network and the view-direction-free network with a fixed threshold tau1 = 9. It then samples rays preferentially from inlier poses, applies re-projection and IoU losses to inlier-inlier pairs, employs Monte Carlo re-localization for outlier poses, and periodically updates the scene graph by filtering keypoint matches with a re-projection threshold. Experiments on the SG-NeRF and DTU datasets report improved Chamfer distance, F-score, APE, and RPE compared with SG-NeRF and other baselines.","tokens_in":15266,"tokens_out":4754,"duration_ms":45501,"significance":"If the reported gains hold, the paper makes a meaningful contribution to robust neural surface reconstruction under severely noisy camera poses, particularly for mirror-symmetry ambiguity. The paper's strengths include a clear problem formulation, a novel detached color network for confidence estimation, a reasonably detailed ablation study, and a public project page with visual results. However, the central claim of superior pose accuracy currently rests on an evaluation that filters poses using the method's own classifier, and the outlier detector itself is not validated against known labels or threshold sensitivity. These issues must be resolved before the significance claim becomes fully credible.","major_comments":[{"comment":"The APE/RPE results in Table 1 are computed after filtering out outliers using the proposed indicator (|PSNR_o − PSNR_n| > τ1), as stated in Sec. 4.3: 'we utilize our indicator to filter out outliers.' Because the same indicator determines which poses are optimized as inliers and which are re-localized, the reported pose error measures the method's self-consistency rather than an independent accuracy. This is especially problematic when the same filter is applied to SG-NeRF poses, since SG-NeRF does not use this confidence model. Please recompute APE/RPE on all poses without filtering, or use ground-truth inlier/outlier labels (available for the DTU noise-injection protocol) to select poses for the SIM(3) alignment; otherwise the claim of improvement 'by more than two orders of magnitude' is not supported.","section":"§4.3, pose accuracy evaluation"},{"comment":"The inlier/outlier classification is load-bearing: it gates confidence-based ray sampling, the re-projection and IoU losses, and Monte Carlo re-localization. Yet the classifier is justified only by a qualitative example in Fig. 2 and by favorable end results; no ROC analysis, confusion matrix, or threshold sweep is provided. The DTU experiments inject known outlier poses (Sec. 4.1) and therefore permit a direct label-based evaluation of the detector, but this is not reported. Please add a sensitivity analysis for τ1 (and ideally for the loss weights α, β, λ) and report per-scene classification accuracy or an ROC curve on the labeled DTU outliers; this is necessary to establish that the confidence signal, rather than dataset-specific behavior, drives the improvements.","section":"§3.1, Eq. (2) and threshold τ1"},{"comment":"The paper states that Monte Carlo re-localization 'has not been triggered' on the DTU dataset, so the DTU experiments do not validate the outlier-handling mechanism that is a central contribution. The DTU results therefore only demonstrate an improved SG-NeRF with additional geometric losses. Since the authors claim a robust solution for outlier poses, the DTU evaluation should either inject sufficiently large noise to trigger re-localization, or the paper should explicitly state that the re-localization component remains unvalidated on DTU. Please also clarify in the contributions which components are supported by which experiments.","section":"§4.1 and §4.3, DTU experiments"},{"comment":"The abstract and Sec. 4.3 state that the method 'consistently' outperforms SG-NeRF, but Tables 1 and 2 show two counterexamples: the Bell scene in Table 1 (Chamfer distance 1.22 vs. 0.98 for SG-NeRF) and Scan 55 in Table 2 (0.44 vs. 0.38 for SG-NeRF). Please qualify the claim to reflect scene-dependent performance or provide an explanation for these regressions.","section":"Abstract and Sec. 4.3, Tables 1–2"}],"minor_comments":[{"comment":"The phrase 'donated as P SN Ro' should be 'denoted as PSNR_o'.","section":"§3.1"},{"comment":"The caption contains a typo: 'The fist row' should be 'The first row'.","section":"Fig. 6 caption"},{"comment":"The reference list entries for [13] and [36] end with stray page/citation markers ', 1' that appear to be formatting artifacts.","section":"References [13] and [36]"},{"comment":"The term 'Intersection-of-Union' is more commonly rendered as 'Intersection-over-Union'; consider standardizing the terminology.","section":"§3.2, Eqs. (6)–(8)"},{"comment":"The sentence 'In DTU [6] dataset' should reference the DTU dataset [25], not [6].","section":"§4.3, Table 2 discussion"},{"comment":"The Huber loss Lδ is not fully specified; please state the Huber parameter in the implementation details.","section":"§3.2, Eq. (9)"}],"recommendation":"major_revision","confidential_remarks":"The pose-evaluation circularity is the main obstacle to acceptance; the authors should be asked for an independent evaluation protocol, ideally using the known outlier labels from the DTU noise-injection setup to validate the confidence indicator. The 'consistently' claim in the abstract should also be softened in light of the Bell and Scan 55 results. This is a promising method that could become acceptable after these revisions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid extension of SG-NeRF, not a breakthrough. The one idea worth remembering is the detached view-direction-free color network as a pose-confidence indicator. It is simple, cheap, and directly targets the shape-radiance ambiguity that makes PSNR-based confidence updates misbehave. On the SG-NeRF benchmark the reconstruction numbers do improve (mean Chamfer 0.32 vs 0.46), and the ablations show Monte Carlo re-localization and scene graph updating each pull their weight. The writing is clear and the method is reproducible in principle.\n\nThe soft spots are real, and the stress-test note is on target. The outlier detector is load-bearing: a single threshold τ1=9 gates sampling, the re-projection/IoU losses, and the re-localization. It is validated only by the qualitative figure and by end-to-end results. There is no sensitivity analysis for τ1, no ROC curve, and no classification accuracy against known labels — even though the DTU experiments inject known outliers, which would make that evaluation trivial. If the indicator misfires, both the ray sampling and the re-localization pipeline are misdirected.\n\nThe pose-accuracy evaluation is the weakest part. The paper uses its own confidence indicator to filter outliers before computing the SIM(3) alignment, then reports APE/RPE on the remaining poses. That is circular: you are measuring the indicator, not the pose optimization. The 'two orders of magnitude' claim should be treated as an upper bound until a neutral protocol is used — e.g., freeze the indicator, use ground-truth outlier labels, or report pose errors before filtering.\n\nAlso note that DTU never triggers Monte Carlo re-localization, so the outlier-handling machinery is only demonstrated on the 8 SG-NeRF scenes, without error bars. No code is released, which makes threshold sensitivity harder to check.\n\nWho is this for? People working on joint pose-NeRF optimization, especially robustness to large pose noise. It is a within-subfield improvement, not a paradigm shift. It deserves a serious referee: the core idea is plausible, the reconstruction evidence is non-circular, and the limitations are acknowledged in the text. I would send it to review with a request for code, a sensitivity study on τ1, and a pose evaluation that does not rely on the method's own outlier filter.","headline":"A credible incremental advance over SG-NeRF with a nice confidence-signal idea; the pose-accuracy results are inflated by a self-referential evaluation, but the reconstruction gains look real.","tokens_in":15791,"tokens_out":2523,"would_cite":true,"duration_ms":25554,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A second, blind color network can flag outlier camera poses, and using that signal to resample rays and re-localize the outliers markedly improves neural surface reconstruction.","keywords":["neural surface reconstruction","camera pose outliers","scene graph","shape-radiance ambiguity","view-direction-free color network","Monte Carlo re-localization","signed distance function","pose-NeRF joint optimization"],"falsifier":"Take a held-out set of poses with known errors and plot $|PSNR_o - PSNR_n|$ against the true pose error after training. If any pose with a large true error stays below the $\\tau_1 = 9$ line, or any accurate pose crosses it, the indicator misfires. A concrete failure mode would be a textureless or repetitive scene where both color networks render confidently from wrong poses, collapsing the gap.","tokens_in":14802,"feed_emoji":"🧊","tokens_out":7351,"duration_ms":65770,"temperature":0.7,"pith_summary":"Neural surface reconstruction typically assumes camera poses are correct, but poses from COLMAP or ARKit can be badly wrong, and joint pose-scene optimization handles small errors yet fails on large ones such as mirrored views. This paper argues that a rendering-quality gap that standard PSNR misses can be exposed by training a second color network that ignores viewing direction: when the two color networks disagree strongly, the pose is an outlier. The method uses that signal to sample more rays from inlier poses, applies re-projection and intersection-over-union losses to tighten inlier geometry, and runs Monte Carlo re-localization on outliers, with the scene graph updated as training proceeds. On the SG-NeRF benchmark the approach lowers mean Chamfer distance from 0.46 to 0.32 and improves pose errors by over two orders of magnitude, while also improving DTU results.","feed_headline":"A second, blind color network flags bad camera poses in 3D scans","feed_subtitle":"Re-localizing the flagged outliers cuts mean surface error from 0.46 to 0.32 Chamfer distance on the SG-NeRF benchmark.","key_machinery":"The load-bearing object is the detached view-direction-free color network $C_n$, a shallow color MLP that shares sampling points and geometry features with the standard color network $C_o$ but never receives the viewing direction and is cut off from the main optimization graph. Its rendered PSNR, $PSNR_n$, replaces the usual $PSNR_o$ in the confidence update, and the rule $|PSNR_o - PSNR_n| > \\tau_1 = 9$ labels a pose as outlier. Confidence scores from the scene graph are updated as $CS(v_i) = CS(v_i) + \\lambda_c PSNR_n(v_i)$, so rays are drawn preferentially from inlier poses. Around this indicator the method builds two correction paths: inlier pairs get re-projection and intersection-over-union losses, while outlier poses are re-initialized by particle poses $R_{axis}(i \\cdot 2\\pi/N_p) \\cdot R_o$ about the coarse main axis and optimized with $PSNR_n$ as the selection score.","core_discovery":"On the paper's own terms, the central discovery is that shape-radiance ambiguity breaks PSNR as a pose-confidence signal: a NeRF with a badly wrong pose can still render images with high PSNR, so inliers and outliers are confused. The authors show that a color network which omits viewing direction, detached from the main pose-NeRF optimization graph, renders the outlier images much worse, and the absolute gap $|PSNR_o - PSNR_n|$ between the standard and detached networks separates the two classes. With this indicator, the framework identifies inliers and outliers, concentrates ray sampling on inlier poses, strengthens inlier matching with re-projection and IoU losses, and re-localizes outlier poses through Monte Carlo sampling around the scene's main axis. The result is consistently better surface geometry and pose accuracy than the scene-graph baseline it builds on, for example mean Chamfer distance 0.32 versus 0.46 on SG-NeRF and absolute pose error 0.053 versus 1.44.","pith_inferences":["The fixed threshold $\\tau_1 = 9$ invites a calibration study: plotting the two-PSNR gap against known pose error per scene would reveal whether one threshold survives across datasets or should be tuned per scene.","Because the indicator only needs two color heads, it could be grafted onto other NeRF-based surface backbones, not just NeuS, as a plug-in outlier detector.","The method's acknowledged dependence on a substantial number of inlier poses suggests it would degrade in sparse-view settings, so prior-based pose priors would be a direct extension.","The PSNR gap may also serve as an unsupervised signal for detecting mirror-symmetry ambiguity in structure-from-motion itself."],"forward_implications":["Up to roughly a third of input poses can be badly wrong and the reconstruction still recovers high-fidelity geometry in one training stage.","Mirrored or otherwise severely wrong poses are corrected rather than discarded, so all captured images contribute to the model.","The same confidence signal drives both ray sampling and outlier re-localization, making the two mechanisms a single feedback loop.","Pose accuracy improves by more than two orders of magnitude over the scene-graph baseline on the SG-NeRF benchmark, which would make downstream AR and robotics tasks far more reliable.","Dynamic scene graph updating prunes wrong keypoint matches during training, strengthening constraints as geometry improves."],"supporting_citations":[{"why":"Provides the scene graph confidence formulation, the baseline results, and the benchmark that the method extends and improves.","marker":"[6]"},{"why":"Supplies the NeuS SDF-based surface reconstruction backbone that the two color networks plug into.","marker":"[59]"},{"why":"Supplies the initial camera poses and scene graph from structure-from-motion that all methods start from.","marker":"[50]"},{"why":"Documents the shape-radiance ambiguity that makes standard PSNR an unreliable pose-confidence signal.","marker":"[17]"},{"why":"Supports the shallow-MLP color network design and view-dependence analysis behind the detached network.","marker":"[73]"},{"why":"Provides the Monte Carlo particle localization mechanism adapted to re-localize outlier poses.","marker":"[13]"},{"why":"Shows Monte Carlo localization inside a NeRF, the direct template for the re-localization stage.","marker":"[36]"},{"why":"Provides a cross-view correspondence baseline whose epipolar loss motivates the chosen re-projection loss.","marker":"[2]"},{"why":"Gives the multi-view geometry behind the re-projection loss used on inlier image pairs.","marker":"[23]"},{"why":"Supplies the DTU benchmark images used for the second set of reconstruction experiments.","marker":"[25]"}],"fun_headline_variants":["Detached color network exposes bad poses in neural surfaces","View-direction-free network separates inlier from outlier poses","Flag mirrored poses via a second, blind color network","Re-localize bad camera poses with Monte Carlo sampling"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole pipeline rests on the assumption that a fixed gap of 9 between the two PSNRs cleanly separates good poses from bad ones in every scene; if that gap mislabels poses, ray sampling and re-localization are pointed at the wrong images.","fun_headline_variants_meta":{"raw":{"variants":["Detached color network exposes bad poses in neural surfaces","View-direction-free network separates inlier from outlier poses","Flag mirrored poses via a second, blind color network","Re-localize bad camera poses with Monte Carlo sampling"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000459,"raw_usage":{"total_tokens":2336,"prompt_tokens":1014,"completion_tokens":1322,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":630,"completion_tokens_details":{"reasoning_tokens":1259}},"tokens_in":630,"tokens_out":1322,"duration_ms":9555,"temperature":1.0,"reasoning_tokens":1259,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:44:48.325164+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of poses with known errors and plot $|PSNR_o - PSNR_n|$ against the true pose error after training. If any pose with a large true error stays below the $\\tau_1 = 9$ line, or any accurate pose crosses it, the indicator misfires. A concrete failure mode would be a textureless or repetitive scene where both color networks render confidently from wrong poses, collapsing the gap.","supporting_citations":[{"cited_title":"Sg-nerf: Neural surface recon- struction with scene graph optimization","cited_arxiv_id":null,"evidence_quote":"Provides the scene graph confidence formulation, the baseline results, and the benchmark that the method extends and improves."},{"cited_title":"Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction","cited_arxiv_id":null,"evidence_quote":"Supplies the NeuS SDF-based surface reconstruction backbone that the two color networks plug into."},{"cited_title":"Structure-from-motion revisited","cited_arxiv_id":null,"evidence_quote":"Supplies the initial camera poses and scene graph from structure-from-motion that all methods start from."},{"cited_title":"Re- ducing shape-radiance ambiguity in radiance fields with a closed-form color estimation method","cited_arxiv_id":null,"evidence_quote":"Documents the shape-radiance ambiguity that makes standard PSNR an unreliable pose-confidence signal."},{"cited_title":"Monte carlo localization for mobile robots","cited_arxiv_id":null,"evidence_quote":"Provides the Monte Carlo particle localization mechanism adapted to re-localize outlier poses."},{"cited_title":"Loc-nerf: Monte carlo local- ization using neural radiance fields","cited_arxiv_id":null,"evidence_quote":"Shows Monte Carlo localization inside a NeRF, the direct template for the re-localization stage."},{"cited_title":"Porf: Pose residual field for accurate neural sur- face reconstruction","cited_arxiv_id":null,"evidence_quote":"Provides a cross-view correspondence baseline whose epipolar loss motivates the chosen re-projection loss."},{"cited_title":"Large scale multi-view stereopsis eval- uation","cited_arxiv_id":null,"evidence_quote":"Supplies the DTU benchmark images used for the second set of reconstruction experiments."}],"review_version":1}