{"id":"4a85a5e0-ce85-4ae3-a817-a0a83fed4001","arxiv_id":"2501.09947","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A neural surface representation with foreground and background modules performs self-supervised object segmentation from multi-view images, producing finer masks than NeRF-based counterparts.","lead":"Surface-SOS is a self-supervised method that separates foreground objects from backgrounds in multi-view images by reconstructing 3D surfaces with signed distance functions. It reports finer segmentation masks than prior NeRF-based approaches and can refine masks produced by single-view models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The core 'self-supervised / always finer' claims are not supported: main tables use Mask R-CNN mask initialization, the mask-free ablation degrades sharply, and BlendedMVS mIoU favors Semantic-NeRF.","rationale":"This pass reads the paper as a scene-specific multi-view refinement and segmentation method built on hash-encoded SDFs. The central claims are (i) a first self-supervised surface-based SOS method, (ii) no dependence on large annotated data or strong constraints, and (iii) always finer masks than NeRF-based baselines and large gains over supervised single-view methods. The method has independent value: it combines two SDF modules, sparsity and Eikonal regularization, and releases code, and the qualitative examples show plausible multi-view consistency. The most load-bearing weakness is evidentiary. The implementation always applies Mask R-CNN to obtain coarse masks, and the main tables are produced with this initialization. The paper describes the mask as optional and as merely expediting convergence, but the ablation in Table II shows it is far more than that: without mask initialization, Teddy SAD goes from 8.685 to 14.188 and Cat mIoU from 0.946 to 0.726. The mask-free variant is not evaluated on any of the four standard benchmarks, so the 'break the dependence on annotated data' claim is not demonstrated on the domains where the paper claims superiority. Additionally, Table I contradicts the literal 'always finer' claim: on BlendedMVS, Semantic-NeRF has mIoU 0.935 vs Surface-SOS 0.931; on LLFF, Acc. ties at 0.918. These results come from only 2-3 scenes per dataset, with author-created labels and no error bars. In light of this, the verdict should remain conditional: the method is plausible and probably contributes a useful refinement mechanism, but the headline claims need to be narrowed and the mask-free configuration needs quantitative evaluation on the standard benchmarks.","tokens_in":19359,"tokens_out":8908,"duration_ms":92149,"concrete_test":"Run the released code on all three LLFF scenes and both BlendedMVS scenes in the mask-free configuration (no Mask R-CNN / SAM initialization) and report SAD, MSE, mIoU, and Acc. with error bars over at least 3 runs; also rerun the same scenes with mask initialization. If mask-free performance on any benchmark falls materially below Semantic-NeRF or NeRF-SOS, the self-supervised claim and the 'always finer' claim should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption behind the 'self-supervised / no large annotated data' claim is that the coarse mask is a minor optional accelerator. The experiments contradict this. Section IV-A states that Mask R-CNN is applied to segment the foreground in each view, and the main quantitative tables (Table I and Table III) are produced with such mask initialization. The only mask-free ablation, Table II, shows large degradation: Teddy SAD rises from 8.685 to 14.188 and mIoU drops from 0.950 to 0.857; Cat mIoU drops from 0.946 to 0.726. So the method's headline performance depends on a COCO-supervised segmentation network, not on pure multi-view surface cues. Furthermore, Table I itself does not support the 'always finer' claim: on BlendedMVS, Semantic-NeRF achieves mIoU 0.935 vs Surface-SOS 0.931, and the LLFF Acc. values tie at 0.918. With only 2-3 scenes per benchmark, no error bars, and author-created labels, the alleged universal superiority is an overstatement.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Surface-SOS, a framework for segmenting a foreground object from multi-view images by decomposing the scene into two complementary neural representations: a Foreground Consistent Representation (FoCoR) based on a signed distance function (SDF) and a Background Completion (BaCo) module. The two modules are trained end-to-end with photometric, Eikonal, sparsity, and optional mask losses, with multi-resolution hash encoding for efficiency. The authors claim that this is the first self-supervised approach using neural surface representation to break the dependence on large annotated data and strong constraints, that it always yields finer object masks than NeRF-based counterparts, and that it remarkably surpasses supervised single-view baselines. Experiments are reported on LLFF, CO3D, BlendedMVS, TUM, and custom video sequences, with qualitative and quantitative comparisons against SAM, DINO-CoSeg, Semantic-NeRF, NeRF-SOS, RFP, RVM, and LNA.","tokens_in":19600,"tokens_out":5216,"duration_ms":46813,"significance":"If the central claims were fully supported, the paper would be a meaningful step toward annotation-free object segmentation by exploiting multi-view geometric consistency. The technical core has merit: the two-module SDF-based decomposition is a sensible design, the use of multi-resolution hash encoding makes training practical, the sparsity loss is a reasonable regularization, and the mask-refinement use case in Table III is a clearly demonstrated contribution. Code is released, which is a strength. However, the evidence as presented does not support the headline claims of pure self-supervision and universal superiority over NeRF-based and supervised baselines; the main results rely on Mask R-CNN initialization, the mask-free ablation degrades sharply, and several quantitative comparisons in Table I go against the stated claim.","major_comments":[{"comment":"The 'self-supervised' claim is not supported because the main quantitative results are obtained with Mask R-CNN coarse masks as co-inputs and an explicit mask loss (Eq. 12). Mask R-CNN is trained on COCO with mask annotations, so it is a supervised prior. The mask-free ablation in Table II shows a large degradation: for Teddy, SAD rises from 8.685 to 14.188 and mIoU drops from 0.950 to 0.857, and for Cat, mIoU drops from 0.946 to 0.726. This contradicts the statement in Fig. 2 and Section III-D that the coarse mask is optional and merely expedites convergence.","section":"IV-A, Table I, Table II, Eq. (12)"},{"comment":"The abstract's claim that Surface-SOS 'always yields finer object masks than its NeRF-based counterparts' is contradicted by Table I: on BlendedMVS, Semantic-NeRF achieves mIoU 0.935 and Acc. 0.955, while Surface-SOS achieves mIoU 0.931 and Acc. 0.941; on LLFF, the Acc. values tie at 0.918. With only 2-3 scenes per benchmark and no error bars or significance tests, the claimed universal superiority is not established.","section":"Table I (BlendedMVS and LLFF rows)"},{"comment":"The evaluation protocol is a concern because the authors manually labeled all views for LLFF, BlendedMVS, and TUM, and these same labels are used to train the supervised Semantic-NeRF baseline. CO3D labels are described as incomplete. With no inter-annotator agreement, no per-scene breakdown, and no error bars, the quantitative evidence does not support the strong comparative claims made in Section IV-C.","section":"IV-A and IV-C"}],"minor_comments":[{"comment":"The word 'Biclcle' in the caption should be 'Bicycle'.","section":"Fig. 5 caption"},{"comment":"The phrase 'coarse mutilated mask' appears to be a typo; it should likely read 'coarse initial mask' or 'coarse mask'.","section":"Section IV-D"},{"comment":"Equation (4) writes 'BB_geo' for the background geometry feature, which appears to be a typo for 'FB_geo'; Eq. (11) should clarify the exponent, e.g., using exp(-tau * |sigma_F|)^2 with unambiguous parentheses.","section":"Eq. (4) and Eq. (11)"},{"comment":"The sentence 'our method outperforms ... the supervised NeRF-based segmentation method (i.e., Semantic-NeRF [46])' cites the wrong reference: Semantic-NeRF is [21], not [46], and the claim is not true on BlendedMVS in Table I.","section":"Section IV-C"},{"comment":"Table III reports improvements on only three scenes, so the conclusion's assertion of 'significantly improving the supervised 2D single-view object segmentation results' would benefit from a per-scene table and some measure of variability or significance.","section":"Table III and Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid technical core, but the framing substantially overstates the self-supervised nature. The main quantitative results depend on Mask R-CNN initialization, and the mask-free ablation shows a large degradation. I recommend requiring a major revision that either repositions the paper as a multi-view mask-refinement method with a supervised initializer, or reports the mask-free variant as the main result and tempers the claims accordingly. The lack of error bars, the small number of scenes, and the reliance on author-created labels are additional concerns for a journal submission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on Surface-SOS. The paper's real contribution is straightforward: it takes the NeuS/Instant-NGP SDF machinery, adds a foreground/background decomposition (FoCoR/BaCo) plus a sparsity loss, and shows that this can segment objects from multi-view images without per-scene mask supervision. That's a legitimate extension, and the qualitative results on fine structures (bicycle spokes, shoelaces) are genuinely nicer than the NeRF-density baselines. The mask-refinement application (Table III) is also practically useful, and training time of 25 minutes per scene is reasonable.\n\nThe soft spots are exactly where the stress-test lands. First, the 'self-supervised' label is doing heavy lifting. The default pipeline applies Mask R-CNN to get coarse masks in every view, and those masks are COCO-supervised. The authors call them optional and say they only speed up convergence, but Table II shows that removing them hurts a lot: mIoU on Teddy drops from 0.950 to 0.857, and on Cat from 0.946 to 0.726. The mask-free version is clearly the less impressive method. That doesn't kill the paper, but the abstract and intro should not imply the headline numbers are annotation-free.\n\nSecond, the 'always finer' claim is not supported by Table I. On BlendedMVS, Semantic-NeRF gets mIoU 0.935 and Acc. 0.955 while Surface-SOS gets 0.931 and 0.941. The LLFF Acc. ties at 0.918. With only 2-3 scenes per dataset, no error bars, and author-made labels, the claim of universal superiority over supervised baselines is an overstatement. The method looks better on most scenes, but not 'always' and not 'remarkably' everywhere.\n\nWhat's solid: the method is not circular. The masks used for evaluation are held-out, the ablations are honest enough to show the mask-free degradation, and the code is public. The limitations section is also straightforward about unbounded scenes and extreme motion.\n\nVerdict: this deserves peer review. The core idea is timely and the experimental setup is fixable. A referee should ask for toned-down claims, the mask-free numbers in the main table, and either more scenes or error bars. If the authors deliver that, this would be a useful contribution.","headline":"A useful SDF-based extension for multi-view segmentation, but the 'self-supervised' and 'always finer' claims need serious toning down.","tokens_in":20127,"tokens_out":2958,"would_cite":true,"duration_ms":26407,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Self-supervised object segmentation can be driven by two signed-distance neural surfaces, one for the foreground object and one for a completed background, producing finer masks than NeRF-based methods and refining any rough single-view…","keywords":["self-supervised object segmentation","neural surface representation","signed distance function","multi-view object segmentation","scene decomposition","volume rendering","multi-resolution hash encoding","alpha matte refinement"],"falsifier":"Run the released code on a manually labeled unbounded 360-degree scene, such as the Tank and Temples Truck sequence: the paper itself predicts blurry masks and missed patches there. The decisive test is a scene with a single coherent foreground, moderate motion, and good poses where the object has strong color boundaries but no SDF-representable surface, like a translucent umbrella or a fur coat; if the output alpha matte stays sharp, the SDF assumption is not the limiting factor, and if it blurs or holes out, the central surface-consistency premise is the bottleneck.","tokens_in":19174,"feed_emoji":"🎭","tokens_out":6258,"duration_ms":62106,"temperature":0.7,"pith_summary":"The paper tries to establish that object segmentation can be learned from multi-view images with no manual annotations at all, using 3D surface geometry as the driving signal. It proposes Surface-SOS, which splits a scene into a foreground surface and a completed background, each represented by its own signed distance function, and renders both through volume rendering. The method claims this yields finer, more view-consistent masks than NeRF-based self-supervised or supervised alternatives, and that feeding it a rough mask from a single-view segmenter improves that segmenter's output. If true, it would mean segmentation of objects like performers in stage shows could be done from casually captured multi-camera video without collecting labeled data.","feed_headline":"Neural surface fields segment objects without any labels","feed_subtitle":"Two signed-distance fields — foreground and background — turn multi-view geometry into refined masks.","key_machinery":"The load-bearing object is a pair of SDF-based neural scene fields joined by alpha blending: a Foreground Consistent Representation (FoCoR) module whose zero-level set is the object surface, and a Background Completion (BaCo) module that inpaints the background from other views after the foreground volume is cropped out. Multi-resolution hash encoding is used to query features fast, SDF-based volume rendering converts signed distance into per-ray opacity, and an Eikonal term plus a sparsity term keep the surface compact and hole-free. The optional coarse mask only seeds the alpha channel; the geometry loss does the real work.","core_discovery":"Surface-SOS's central claim is that the right inductive bias for self-supervised object segmentation is 3D surface consistency, not pixel statistics or motion. Concretely, it models the scene as two complementary neural signed distance fields: FoCoR represents the foreground object's distance and radiance, BaCo represents the static background with the foreground volume cropped out, and $\\alpha$-blended composition $\\mathbf{c} = \\alpha \\mathbf{c}_F + (1-\\alpha) \\mathbf{c}_B$ renders the input views. Training with photometric, Eikonal, and sparsity losses—plus an optional coarse mask for initialization—makes the zero-level set of the foreground SDF lock onto the object, so the $\\alpha$ matte comes from geometry rather than appearance. The paper reports that this always produces finer object masks than NeRF-based counterparts on LLFF, CO3D, BlendedMVS, TUM, and real video scenes, and that refining masks from Mask R-CNN, SAM, or RVM with this representation substantially improves those single-view baselines.","pith_inferences":["The same two-SDF decomposition could be applied per object for scenes with multiple foreground instances, giving each object its own FoCoR-style surface and composing the alpha maps, an extension the paper does not explore.","If surface geometry is what drives mask quality, then scenes with weak texture or translucent objects will remain hard even with perfect poses, because the SDF cannot form a clean zero-level set; this is an implicit limit beyond the paper's stated unbounded-scene failure.","The method's success at refining masks suggests a bootstrapping protocol: take any single-image segmenter, generate coarse masks on a multi-view capture, run Surface-SOS, and use the refined masks as pseudo-labels for training the single-view network, a loop the paper does not test."],"forward_implications":["Multi-view unlabeled images become sufficient supervision for object segmentation, removing the need for annotated segmentation datasets in scenes captured by several cameras or a moving phone.","Rough single-view masks from existing segmenters are refined into sharper, more view-consistent mattes when used as initialization, so the method acts as a geometry-driven post-processor.","Because the foreground mask is extracted from a signed-distance surface rather than a density field, the resulting masks avoid the interior holes and near-camera floaters typical of NeRF-based segmentation.","The framework extends to modestly dynamic videos, such as casual handheld captures, but not to unbounded 360-degree scenes or extreme object motion, per the paper's own failure cases.","Self-supervised performance matches or beats a supervised NeRF semantic segmenter on the reported benchmarks, so annotation cost can be shifted from labels to camera poses."],"supporting_citations":[{"why":"Supplies the SDF volume-rendering formulation that turns signed distance into opacity for surface extraction.","marker":"[19]"},{"why":"Supplies the multi-resolution hash grid features that make per-scene training fast enough for practical use.","marker":"[17]"},{"why":"Defines the base radiance-field framework that NeRF-based segmenters extend and that the paper compares against.","marker":"[15]"},{"why":"Provides the main self-supervised NeRF-based segmentation baseline whose density-field approach Surface-SOS claims to outperform.","marker":"[52]"},{"why":"Provides an unsupervised radiance-field segmentation baseline used as a comparison on the LLFF dataset.","marker":"[53]"},{"why":"Provides the supervised NeRF semantic-segmentation baseline that Surface-SOS claims to match or exceed without labels.","marker":"[21]"},{"why":"Provides a strong supervised single-view segmenter whose masks are used both as a baseline and as initialization to refine.","marker":"[32]"},{"why":"Supplies the coarse object masks used for initialization and as a single-view segmentation baseline.","marker":"[7]"},{"why":"Supplies the forward-facing multi-view benchmark used for evaluation on real scenes.","marker":"[25]"}],"fun_headline_variants":["Two SDFs segment objects with zero annotations and pure geometry","Foreground and background SDFs segment objects label-free","Two neural distance fields replace all object segmentation labels","Dual SDFs refine masks beyond supervised baselines without any labels","Surface-SOS uses complementary SDFs to yield self-supervised object splits"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the foreground object is one coherent signed-distance surface whose geometry lines up across views, and that the background is static or moves gently enough that other views can complete it, which the paper itself notes fails for unbounded scenes and extreme motion.","fun_headline_variants_meta":{"raw":{"variants":["Two SDFs segment objects with zero annotations and pure geometry","Foreground and background SDFs segment objects label-free","Two neural distance fields replace all object segmentation labels","Dual SDFs refine masks beyond supervised baselines without any labels","Surface-SOS uses complementary SDFs to yield self-supervised object splits"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001642,"raw_usage":{"total_tokens":6557,"prompt_tokens":1006,"completion_tokens":5551,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":622,"completion_tokens_details":{"reasoning_tokens":5464}},"tokens_in":622,"tokens_out":5551,"duration_ms":39167,"temperature":1.0,"reasoning_tokens":5464,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T19:29:50.770132+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the released code on a manually labeled unbounded 360-degree scene, such as the Tank and Temples Truck sequence: the paper itself predicts blurry masks and missed patches there. The decisive test is a scene with a single coherent foreground, moderate motion, and good poses where the object has strong color boundaries but no SDF-representable surface, like a translucent umbrella or a fur coat; if the output alpha matte stays sharp, the SDF assumption is not the limiting factor, and if it blurs or holes out, the central surface-consistency premise is the bottleneck.","supporting_citations":[{"cited_title":"Unsupervised multi- view object segmentation using radiance field propagation,","cited_arxiv_id":null,"evidence_quote":"Provides an unsupervised radiance-field segmentation baseline used as a comparison on the LLFF dataset."}],"review_version":1}