{"id":"8be72fca-b418-4ad6-a66d-5393271e9c9c","arxiv_id":"2602.11804","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"Adding monocular depth to EfficientViT-SAM improves point-prompted segmentation at 3 and 5 clicks after fine-tuning on 11.2k images, but universal gains and data-efficiency are not established.","lead":"A team proposes a lightweight version of the Segment Anything Model that also uses a depth map from a pretrained estimator, and says it beats a larger variant when trained on only 11,200 images. The headline result is only partly supported: it wins at 3 and 5 clicks but loses at 1 click, and the comparison baseline was not retrained on the same small dataset.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The data-efficiency claim is confounded because the RGB backbone and SAM head are inherited from full SA-1B pretraining, and no RGB-only limited-data baseline is provided to isolate depth's contribution.","rationale":"The reader's weakest assumption pinpoints exactly the load-bearing issue: the comparison between the proposed model and EfficientViT-SAM is not a controlled limited-data experiment because the RGB/SAM components likely carry full SA-1B pretraining. My stress-test confirms this concern. The paper's own text supplies evidence for the confound: Section 4.1 says 'we build on EfficientViT-SAM-L2' and describes end-to-end training without specifying weight initialization; the runtime and parameter counts match a full dual-encoder network; and the ablation is only a vague 'about 1%' statement without a table. This makes the central claim 'Trained on only 11.2k images ... surpasses EfficientViT-SAM' misleading. The 1-click regression in Table 2 is a secondary but concrete inconsistency with the unqualified 'higher accuracy' language. Since the reader already recommends REJECT and my analysis supports that, no verdict change is needed. A fair RGB-only baseline and transparent initialization would be required before the claim could be accepted conditionally.","tokens_in":5876,"tokens_out":3589,"duration_ms":32642,"concrete_test":"Fine-tune the unmodified EfficientViT-SAM-L2 on the exact same 11.2k SA-1B subset for 4 epochs with the same optimizer, batch size, and loss (including the auxiliary terms) but no depth branch. Evaluate this RGB-only baseline on the same COCO/LVIS point-prompt and box-prompt protocols, and report 1/3/5-click mIoU. If this baseline matches or exceeds the depth-model numbers in Table 2 and Table 3, the depth-prior story collapses. Additionally, re-derive the abstract claim to specify click counts and to state explicitly whether the RGB tower is initialized from full-data pretrained weights.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that training on only 11.2k images (\"<0.1% of SA-1B\") yields a model that surpasses EfficientViT-SAM, implying that depth priors enable data-efficient training. But the paper never discloses whether the RGB encoder and SAM head are initialized from the released EfficientViT-SAM-L2 weights (which were trained on the full SA-1B dataset) or from scratch. Section 4.1 says only 'we build on EfficientViT-SAM-L2' and then 'the entire model is trained end-to-end.' If, as is standard and as the reported performance strongly suggests, the RGB tower starts from full-data pretrained weights, then the 11.2k images are a fine-tuning budget, not a training-from-scratch budget. The comparison to EfficientViT-SAM-L2 therefore conflates two variables: (1) additional fine-tuning on 11.2k images and (2) the added depth branch. Without an RGB-only baseline fine-tuned on the same 11.2k images with the same loss and optimizer, the observed gains cannot be attributed to depth cues. The paper's own ablation is only described as 'removing depth fusion reduces accuracy by about 1% on small objects' (Section 4.4) with no full table, suggesting the depth contribution may be minor relative to the fine-tuning effect. Furthermore, Table 2 shows the method is worse than EfficientViT-SAM-L2 at 1 click (COCO: 52.6 vs 56.1; LVIS: 56.3 vs 57.8), directly contradicting the unqualified abstract claim of 'higher accuracy than EfficientViT-SAM.' Thus the central data-efficiency and depth-prior claims are not established as stated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Depth-Aware EfficientViT-SAM, a variant of EfficientViT-SAM-L2 that fuses monocular depth maps (from a pretrained DepthAnything estimator) with RGB features through a separate depth encoder, using simple additive fusion with a learned scaling coefficient. The framework is trained end-to-end for four epochs on an 11.2k-image subset of SA-1B and evaluated zero-shot on COCO and LVIS in both box-prompted and point-prompted settings. The central claim is that this limited-data training, aided by depth priors, yields a model that surpasses EfficientViT-SAM and even larger SAM variants in accuracy while remaining lightweight. The paper also reports runtime, parameter counts, and qualitative examples. The main evidence consists of comparisons in Tables 1–3, but the most striking point-prompted gains appear only at 3 and 5 clicks, and the comparison protocol does not isolate the effect of depth from the effect of additional fine-tuning.","tokens_in":6317,"tokens_out":3216,"duration_ms":29516,"significance":"If substantiated, the claimed result would be practically interesting: it would show that monocular depth priors can substantially reduce the data required to adapt an efficient SAM to interactive segmentation, and the proposed architecture is simple and easy to build on. The paper also reports a useful runtime/parameter analysis and targets a real deployment concern. However, the current version does not provide enough evidence to support the central claim. The key missing piece is an RGB-only baseline trained on the same 11.2k images with the same loss and protocol; without it, the observed improvements at 3/5 clicks cannot be attributed to depth. In addition, the unqualified abstract statement of superiority is contradicted by the 1-click results in Table 2. These are correctness-level issues, not presentation issues. The paper is a reasonable empirical exploration, but its claims outrun its experiments.","major_comments":[{"comment":"The abstract states that the method 'achieves higher accuracy than EfficientViT-SAM' without qualification. Table 2 shows the opposite at 1 click: COCO 52.6 vs. 56.1 and LVIS 56.3 vs. 57.8 for our method versus EfficientViT-SAM-L2. The claim must be restricted to the 3/5-click settings, and the 1-click behavior needs discussion. As written, the central claim is internally contradicted by the paper's own main point-prompted table.","section":"Abstract / §4.5, Table 2"},{"comment":"The manuscript never discloses whether the RGB encoder and SAM head are initialized from the released EfficientViT-SAM-L2 weights (pre-trained on the full SA-1B) or trained from scratch. Section 4.1 only says 'we build on EfficientViT-SAM-L2' and then 'the entire model is trained end-to-end.' If the RGB tower retains full-dataset pretrained weights, the 11.2k images are a fine-tuning budget, not a training-from-scratch budget. The comparison to EfficientViT-SAM-L2 then conflates two variables: additional fine-tuning on 11.2k images and the added depth branch. The paper needs an RGB-only baseline fine-tuned on the same 11.2k images with the same loss, optimizer, and click protocol, in order to isolate the contribution of depth. Without this baseline, the 'depth cues enable data-efficient training' conclusion is unsupported.","section":"§4.1 / §4.4, Tables 2 and 3"},{"comment":"The claimed gains in box-prompted settings are very small and are reported without error bars or multiple seeds. For example, on COCO-VitDet the gain over EfficientViT-SAM-L2 is 0.3 mAP (44.3 vs. 44.0), while on COCO-YOLOv8 our method is slightly worse (41.8 vs. 41.9). In the ground-truth box setting (Table 3), the small-object gain is 2.5 points on COCO but the large-object gain is negative. With no statistical testing or variance estimates, 'consistent improvements' cannot be distinguished from noise. The ablation mentioned in §4.4 ('removing depth fusion reduces accuracy by about 1% on small objects') is also reported only as a single prose sentence with no table, no setting, and no numbers. A full ablation table separating RGB-only, depth-fusion, and each auxiliary loss term is required.","section":"§4.4, Table 1 / Table 3"},{"comment":"The 'efficient' and 'lightweight' framing should be calibrated. The proposed model doubles parameters (61.3M→118.7M), doubles MACs (69G→137G), and halves throughput (62.8→31.9 images/s) relative to the EfficientViT-SAM-L2 baseline. It is lightweight only in comparison to SAM-ViT-H (>600M, ~3000G). Since the paper's stated goal is to improve EfficientViT-SAM, the efficiency claim should be quantified against that baseline more carefully, and the computational cost of the monocular depth estimator should be reported. As written, the efficiency contribution is overstated.","section":"§4.2 / Abstract"}],"minor_comments":[{"comment":"The qualitative examples are not quantified and no failure cases are shown. Since the paper claims sharper boundaries and better small-object handling, a few quantitative boundary/F-score examples or failure-mode discussion would strengthen the presentation.","section":"§4.3 / Fig. 2"},{"comment":"The text says 'depth maps are generated by DepthAnything' and 'depth images are replicated across three channels,' but the depth pre-processing (normalization, resizing, whether the same image transform is applied) is not specified. This detail is needed for reproducibility.","section":"§3.1 / Eq. (1)"},{"comment":"Training details are missing some standard items: learning rate schedule, warm-up, weight decay, and whether the RGB encoder is frozen in stage 1. The stage-1 stage-2 description is also ambiguous about which components are trained in each stage.","section":"§4.1"},{"comment":"References [8]–[11] are about 3D reconstruction and SLAM and are only loosely related to the segmentation argument. The authors may wish to replace them with more directly relevant RGB-D segmentation or efficient SAM works, or justify the connection more explicitly.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper reports a potentially useful empirical direction, but the current version does not support its headline claims. The most important fix is an RGB-only limited-data baseline and a clear statement of initialization. The abstract's unqualified 'higher accuracy than EfficientViT-SAM' is contradicted by Table 2 and must be corrected. If the authors can provide the missing baseline, error bars, and a full ablation, the paper could become publishable; without those, the depth contribution remains confounded by fine-tuning effects."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: there's a real effect here somewhere, but the paper doesn't isolate it. The most interesting numbers are in Table 2: at 3 and 5 clicks, their Depth-Aware EfficientViT-SAM beats EfficientViT-SAM-L2 by 4-8 mIoU on COCO and LVIS, and even edges out SAM-ViT-H. That's worth checking. The box-prompted results are more modest, with consistent small-object gains (2-3 mIoU) and slight large-object losses. The fusion itself is simple — additive with a learnable scale — and the loss is a standard bag of BCE/Dice/IoU/aux terms. No code, no seeds, no error bars.\n\nThe soft spot is the central claim. The paper says 'trained on only 11.2k images' and 'surpasses EfficientViT-SAM,' but the comparison confounds depth with fine-tuning and initialization. Section 4.1 never says whether the EfficientViT-SAM-L2 backbone and SAM head are initialized from the released full SA-1B weights. Given the performance, they almost certainly are. If so, the 11.2k images are a fine-tuning budget, not a training budget, and the 'data-efficient' framing collapses. To attribute any gain to depth, they need an RGB-only baseline fine-tuned on the same 11.2k images with the same loss and schedule. Their ablation is a single sentence — 'removing depth fusion reduces accuracy by about 1% on small objects' — with no table. That doesn't carry the weight.\n\nThere's also the 1-click contradiction: Table 2 shows they're 3-4 mIoU worse than EfficientViT-SAM-L2 at 1 click on both benchmarks, which directly contradicts the unqualified abstract claim. The multi-click gains are real in the numbers, but without error bars or a fair baseline I wouldn't trust the magnitude. Doubling parameters and halving throughput (62.8 to 31.9 im/s) also makes the 'efficient' part a stretch relative to their own baseline.\n\nOverall: this is a plausible empirical direction — monocular depth as a low-cost prior for fine-tuning an efficient SAM — but the paper as written doesn't establish it. The missing baseline and initialization disclosure are fixable, and the 3/5-click result is intriguing enough to want to see them fixed. I wouldn't desk reject it; I'd send it to a referee with instructions to ask for the RGB-only same-data baseline, error bars, and an explicit statement about pretrained weights. If those come back supporting the numbers, it could be a useful workshop or short-conference paper.","headline":"Intriguing multi-click gains, but the headline 'data-efficient' claim is unsupported because the RGB tower almost certainly inherits full SA-1B pretraining and no same-data RGB-only baseline is run.","tokens_in":6837,"tokens_out":3334,"would_cite":false,"duration_ms":29137,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding monocular depth cues lets a lightweight Segment Anything Model surpass a strong RGB-only baseline while training on less than 0.1% of the original dataset.","keywords":["Segment Anything Model","depth-aware fusion","monocular depth priors","data-efficient segmentation","zero-shot segmentation","interactive segmentation","EfficientViT-SAM","RGB-D fusion"],"falsifier":"Train an RGB-only EfficientViT-SAM on the same 11.2k images with the same schedule and loss; if it matches the depth-aware model's scores, the depth branch is not carrying the improvement. Likewise, re-running with the backbone randomly initialized would show whether the system is truly learning from 11.2k images or merely relying on pretrained weights.","tokens_in":5741,"feed_emoji":"🖼️","tokens_out":4528,"duration_ms":40718,"temperature":0.7,"pith_summary":"Depth-Aware EfficientViT-SAM is a lightweight Segment Anything variant that adds monocular depth as a second input stream, fused with RGB features by a simple learnable sum. The paper's central claim is that this depth prior, combined with a training recipe on just 11.2k images, lets the model surpass the RGB-only EfficientViT-SAM-L2 on zero-shot box-prompted and point-prompted segmentation, and even beat the original SAM and a larger efficient variant when users provide several clicks. If true, this would mean geometric depth cues can substitute for millions of extra training images in adapting a universal segmenter to practical, resource-limited settings. The work matters because it points to a cheap way to regain the accuracy of heavy SAM models without their data and compute appetite.","feed_headline":"Depth-aware SAM beats EfficientViT-SAM on 0.1% of its data","feed_subtitle":"Monocular depth priors let a lightweight model fine-tuned on 11.2k images top click-driven segmentation benchmarks.","key_machinery":"The mechanism is a second encoder with the same architecture as the RGB encoder, taking monocular depth maps (replicated to three channels) and outputting embeddings that are added to RGB embeddings with a learnable scaling factor alpha. A five-term loss (mask, dice, IoU regression, direct intermediate supervision, and a boundary-focused auxiliary term) trains first the depth encoder alone, then the whole model end-to-end. The depth signal supplies geometric structure that RGB textures lack, and this is what carries the boundary and small-object improvements.","core_discovery":"The authors claim that augmenting EfficientViT-SAM with a parallel depth encoder—fed by monocular depth maps and fused as F_fuse = F_rgb + alpha*F_dep—produces higher zero-shot interactive segmentation accuracy than the RGB-only model while training on only 11.2k images for four epochs. In point-prompted evaluation with three or five clicks, the approach outperforms EfficientViT-SAM-XL1 and SAM-ViT-H on COCO and LVIS; box-prompted results consistently improve on small objects. The depth branch roughly doubles parameters and computation but remains far lighter than SAM-ViT-H.","pith_inferences":["The paper's data-efficiency claim is sensitive to initialization: the setup appears to start from a pretrained EfficientViT-SAM-L2 and fine-tune with the depth branch, making the 11.2k-image budget a fine-tuning cost rather than a from-scratch training cost; a true from-scratch comparison would clarify this.","A natural test is to train an RGB-only EfficientViT-SAM-L2 on the same 11.2k images with the same schedule and loss; if the RGB-only model matches or exceeds these scores, depth fusion is not the active ingredient.","Extending beyond the paper, the same additive-fusion-with-alpha recipe could apply to other tasks where monocular depth is a cheap prior, such as video segmentation or object detection, provided the depth estimator generalizes to the target domain."],"forward_implications":["If depth priors supply the geometric structure that RGB fine-tuning lacks, SAM variants can be adapted to new domains with orders of magnitude fewer annotated images.","The consistent gains on small and medium objects imply depth fusion is most valuable where boundaries are ambiguous, so boundary-sensitive downstream tasks could use the same recipe.","Point-prompted gains that grow with click count suggest depth-aware fusion is especially effective in interactive segmentation workflows.","Because training completes in a few hours on commodity GPUs, the approach lowers the compute barrier for customizing universal segmentation models."],"fun_headline_variants":["Depth-aware SAM tops RGB-only on 0.1% of data","Monocular depth upgrades EfficientViT-SAM with 11k images","Depth priors let SAM beat EfficientViT with 0.1% training","Efficient SAM with depth fusion needs only 11.2k samples","RGB-D fusion makes efficient SAM more accurate than RGB SAM"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The comparison assumes the EfficientViT-SAM backbone is already pretrained on the full 11M-image dataset, so the 11.2k images are a fine-tuning budget; if the backbone were trained from scratch on those 11.2k images, the reported gains would likely not appear.","fun_headline_variants_meta":{"raw":{"variants":["Depth-aware SAM tops RGB-only on 0.1% of data","Monocular depth upgrades EfficientViT-SAM with 11k images","Depth priors let SAM beat EfficientViT with 0.1% training","Efficient SAM with depth fusion needs only 11.2k samples","RGB-D fusion makes efficient SAM more accurate than RGB SAM"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000654,"raw_usage":{"total_tokens":2781,"prompt_tokens":637,"completion_tokens":2144,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":381,"completion_tokens_details":{"reasoning_tokens":2059}},"tokens_in":381,"tokens_out":2144,"duration_ms":123658,"temperature":1.0,"reasoning_tokens":2059,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T23:57:24.295914+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train an RGB-only EfficientViT-SAM on the same 11.2k images with the same schedule and loss; if it matches the depth-aware model's scores, the depth branch is not carrying the improvement. Likewise, re-running with the backbone randomly initialized would show whether the system is truly learning from 11.2k images or merely relying on pretrained weights.","supporting_citations":[],"review_version":1}