{"id":"e45d301d-ee38-4f5d-8063-61fc9372e128","arxiv_id":"2502.01666","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"Swapping CLIP for SeeCoder in a Stable Diffusion depth model slightly improves KITTI metrics but the model fails on Waymo and the evaluation is unfair.","lead":"This paper replaces the text-based CLIP encoder in a Stable Diffusion depth model with an image-based encoder called SeeCoder, then tests it on driving datasets. It reports a small gain on KITTI, but the model's poor results on Waymo and a missing key comparison undermine the robustness claim.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The KITTI comparison rests on a non-standard test-time protocol (left/right split plus flip TTA and fusion) that likely inflates the proposed model's numbers relative to baselines; without a whole-image evaluation the central claim is unverifiable.","rationale":"The reader identified the test-time protocol as the weakest assumption, and my reading agrees. The paper's only strong quantitative evidence for the central claim is the KITTI table, and that table is not like-for-like: the proposed model is evaluated with a bespoke split-and-fusion protocol while baselines use standard whole-image evaluation. This is not a matter of consensus but of internal comparison validity; if the protocol inflates performance, the claimed advantage of SeeCoder over CLIP is unsupported. The Waymo results independently show that the model does not generalize outdoors, making the KITTI protocol issue even more decisive. I do not see a different, more load-bearing concern: the ablation is weak but secondary, and the missing VPD baseline is important but would only matter if the KITTI comparison were valid. Therefore the reader's REJECT verdict is appropriate, and no change is needed.","tokens_in":10136,"tokens_out":3862,"duration_ms":36117,"concrete_test":"Re-evaluate the trained model on the KITTI Eigen validation split using the standard whole-image protocol: no left/right split, no horizontal-flip TTA, no weighted fusion; apply the same Eigen crop and depth cap used for the AdaBins/DPT/ZoeDepth numbers. If delta1 falls below 0.970 or RMSE rises above 2.3, the Table III comparison is invalidated and the central claim is unsupported. Ideally also run the same whole-image protocol for a VPD baseline with CLIP embeddings to isolate the SeeCoder contribution.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that SeeCoder image embeddings make Stable Diffusion-based depth estimation comparable to state-of-the-art on outdoor data; the only quantitative support is Table III on KITTI. These numbers are produced by the test-time protocol in Section IV-C: each 1242x375 image is split into left and right segments, each segment is predicted with horizontal-flip augmentation, and the segments are weighted-fused. The cited baselines (Eigen, MonoDepth2, AdaBins, DPT, ZoeDepth) are scored on whole images using the standard Eigen evaluation. This mismatch is load-bearing for three reasons. First, splitting changes the receptive field and aspect ratio, so the model sees a different distribution than the baselines. Second, the flip-and-fusion TTA is a form of ensembling that averages errors; baselines are not granted an analogous procedure. Third, no whole-image result for the proposed model is reported, so the reader cannot tell how much of the 0.974 delta1 / 2.179 RMSE advantage is due to the method versus the protocol. The Waymo results (Table II) compound this: with delta1 at 0.044 or lower, the model fails to transfer outdoors, so the only positive evidence is the protocol-dependent KITTI number. If the protocol is not neutral, the central comparative claim collapses.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes replacing CLIP text embeddings with SeeCoder image-based semantic embeddings in the VPD Stable Diffusion framework for monocular depth estimation, adding a spatial enhancement module composed of dilated convolutions and spatial attention. The method is evaluated on the KITTI Eigen split and on three Waymo Open Dataset scenarios (normal, rainy, nighttime). The authors report KITTI validation metrics of δ1=0.974, RMSE=2.179, AbsRel=0.052, claim performance comparable to state-of-the-art models, and interpret the Waymo results as evidence of robustness across weather and lighting conditions.","tokens_in":10437,"tokens_out":4199,"duration_ms":40009,"significance":"The underlying idea—using image-derived semantic embeddings instead of text prompts in a diffusion-based depth estimator—is motivated by a real limitation of CLIP text conditioning in complex outdoor scenes. If the reported gains were reliable, the paper would provide a simple and potentially transferable alternative for dense prediction tasks. However, the empirical support is severely undercut by the non-standard KITTI evaluation protocol, the absence of the direct VPD/CLIP baseline, and Waymo numbers that contradict the robustness claim. The contribution is incremental: SeeCoder and the VPD framework are imported from prior work, and the only novel component is the spatial enhancement module, whose ablation shows individual modules hurt performance and whose combined gain is marginal. Consequently, the paper's significance as a standalone contribution is limited.","major_comments":[{"comment":"The KITTI comparison is compromised by a non-standard test-time protocol. The paper states that during evaluation the 1242×375 images are split into left and right segments, each segment is predicted with horizontal-flip augmentation, and the two segments are \"weighted and fused.\" All compared baselines in Table III (Eigen, MonoDepth2, AdaBins, DPT, ZoeDepth) are scored on whole images using the standard Eigen evaluation. The split changes the input aspect ratio and receptive field, and the flip-and-fusion procedure is a form of test-time ensembling that is not granted to the baselines. No whole-image result for the proposed model is reported, and the fusion weights are not specified. As the only quantitative support for the central claim of state-of-the-art-comparable performance is this protocol-dependent KITTI result, the comparative claim is not verifiable as presented.","section":"Section IV-C and Table III"},{"comment":"The paper's central motivation is that SeeCoder image embeddings outperform CLIP text embeddings in outdoor scenes, but Table III does not include VPD with CLIP on KITTI. Since the proposed method is exactly the VPD framework with the semantic encoder swapped and a spatial module added, the missing VPD/CLIP baseline is load-bearing: without it, the reader cannot attribute any observed gain to the image-based semantic encoding rather than to other differences in training or evaluation. The paper should report the VPD model (or an equivalent CLIP-conditioned Stable Diffusion depth model) under the same protocol and training setup.","section":"Section IV-E and Table III"},{"comment":"The Waymo results contradict the robustness claim. Table II reports δ1 values of 0.044 (normal), 0.012 (rainy), and 0.017 (nighttime); in daytime conditions similar to KITTI, 95.6% of pixels are outside the 1.25× accuracy threshold. The text interprets the relatively small variation in RMSE and AbsRel across scenes as evidence of \"generalization and robustness,\" but the absolute error levels are so high that the model is effectively failing on this dataset. At minimum, the paper should acknowledge that the model does not transfer to Waymo and should not claim robustness on the basis of these numbers.","section":"Section IV-D and Table II"},{"comment":"The ablation study does not support the claim that the spatial enhancement module improves performance. Adding dilated convolution (DC) alone drops δ1 from 0.973 to 0.876 and increases Sq Rel from 0.164 to 1.155; adding spatial attention (SA) alone drops δ1 to 0.883. The combined DC+SA yields δ1=0.974 versus 0.973 and RMSE=2.179 versus 2.216, a marginal difference with no significance testing. The proposed explanation that \"their combination compensates for each other's shortcomings\" is speculative. Given that each individual module is clearly harmful, the paper should provide a more thorough analysis and preferably report standard deviations or multiple seeds.","section":"Section III-B and Table I"}],"minor_comments":[{"comment":"The paper attributes adaptive binning to Tu et al. [32], but the cited reference is about binary neural networks (AdaBin), not the AdaBins depth estimation method. Please cite the original AdaBins paper (Bhat et al., CVPR 2021).","section":"References [32]"},{"comment":"The abstract claims performance \"comparable to state-of-the-art models,\" while the conclusion states the method \"does not yet surpass current state-of-the-art models\" and cites ECoDepth. Please reconcile these statements and specify which state-of-the-art models are being compared.","section":"Abstract and Conclusion"},{"comment":"The weighted fusion of the left and right segments is described only as \"weighted and fused.\" Please specify the weights or the rule used to compute them, as this is necessary for reproducibility.","section":"Section IV-C"},{"comment":"The statement that the denoising UNet is \"trained from scratch without relying on pre-trained weights\" is ambiguous because the VAE and SeeCoder backbones are pre-trained. Clarify whether only the UNet parameters are randomly initialized.","section":"Section III-C"},{"comment":"The notation \\bar{α}_t is used without a definition of the noise schedule; please define α_t and \\bar{α}_t.","section":"Equation (2)"},{"comment":"There is a typo: \"Semantic encoders play a important role\" should read \"an important role.\"","section":"Section II-C"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to be an early draft or workshop-style submission. The core issues are experimental: the KITTI evaluation protocol is not comparable to published baselines, the direct CLIP-baseline is missing, and the Waymo results are presented as evidence of robustness despite being far below any useful accuracy level. These are not merely presentation problems; they affect the validity of the central claim. Even with a revised protocol and added baselines, the paper would need substantial new experiments and a more careful framing of the Waymo results. Given the current state, I recommend rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The idea here is reasonable: swap CLIP text embeddings for a visual semantic encoder (SeeCoder) in a diffusion-based depth estimator, and add a spatial enhancement module. The combination appears to be new, and the writing is clear. I also give the authors credit for reporting the ablation honestly—adding dilated convolution or spatial attention alone hurts, and only the combination helps—and for admitting in the conclusion that the method does not surpass the latest SOTA. That is not how every paper behaves.\n\nThe problem is the KITTI evaluation. Section IV-C describes splitting each 1242×375 image into left and right segments, predicting each with horizontal-flip augmentation, and weighted-fusing the results. Every baseline in Table III is scored on whole images using the standard Eigen protocol. That is not a minor detail; it changes the receptive field and gives the model a form of ensembling that the baselines do not get. No whole-image numbers are reported, so we cannot tell how much of the apparent advantage comes from the protocol rather than the method. This is load-bearing, because the only quantitative support for the central claim is that table.\n\nThe missing direct baseline makes it worse. The paper is essentially VPD with SeeCoder instead of CLIP, but VPD itself is not in the comparison. Without that, we cannot isolate what SeeCoder contributes.\n\nThe Waymo results are also presented misleadingly. δ1 values of 0.044, 0.012, and 0.017 are catastrophic, and the paper says this \"demonstrates generalization and robustness.\" It demonstrates the opposite. The authors do note the decline, but the framing is still overreaching.\n\nThere are smaller issues: the ablation gains are tiny (δ1 0.973 → 0.974, RMSE 2.216 → 2.179), likely within noise; and there is a citation error—AdaBins is misattributed to a binary-network paper by Tu et al. The abstract also overclaims \"comparable to state-of-the-art\" while the conclusion admits it does not surpass current SOTA.\n\nWho is this for? Someone working on diffusion-based depth might find the SeeCoder application interesting, but the evidence does not support the claims. The paper needs a whole-image KITTI evaluation, a VPD baseline, and honest framing of the Waymo results before it is publishable. I would not send this to peer review in its current form; the central comparison is invalid, and the contribution is incremental. If the authors fix the protocol and re-run, a workshop-level paper might emerge, but not this one.","headline":"The core idea is plausible but the main KITTI comparison is invalidated by a non-standard split-image test protocol, so the paper's central claim is currently unverifiable.","tokens_in":743,"tokens_out":1713,"would_cite":false,"duration_ms":34899,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Replacing CLIP text with SeeCoder image embeddings lifts KITTI monocular depth accuracy to δ1=0.974 and generalizes to rainy and nighttime driving scenes.","keywords":["monocular depth estimation","Stable Diffusion","image semantic encoding","SeeCoder","CLIP embeddings","dilated convolution","spatial attention","KITTI and Waymo"],"falsifier":"Evaluate the trained model on the KITTI Eigen validation set using whole 1242×375 images, without the left/right split or horizontal-flip fusion, but with the same Garg crop and valid-depth mask, and compare δ1, RMSE, and AbsRel with the numbers in Table III. If the metrics fall to the SeeCoder-only row (δ1≈0.973, RMSE≈2.216) or below the listed AdaBins and ZoeDepth numbers, then the split-fusion protocol rather than the image-semantic encoder is carrying the reported improvement.","tokens_in":9933,"feed_emoji":"🚗","tokens_out":7991,"duration_ms":73437,"temperature":0.7,"pith_summary":"The paper tries to show that text-based semantic guidance is the wrong interface for conditioning a Stable Diffusion depth model on outdoor scenes, and that image-derived semantics work better. It swaps CLIP text embeddings for SeeCoder's visual embeddings, adds a paired dilated-convolution and spatial-attention module, and trains the denoising UNet from scratch on KITTI. On the KITTI Eigen validation split the resulting model reports δ1=0.974, RMSE=2.179, and AbsRel=0.052, ahead of the transformer baselines listed in its comparison table, and it keeps global error metrics within roughly ten percent across normal, rainy, and nighttime Waymo scenes. The authors also report that the two spatial modules only help when used together; each alone degrades accuracy. The significance would be that generative depth models can be conditioned directly on visual semantics, removing the need for hand-written prompts in complex environments.","feed_headline":"Image semantics beat text prompts for monocular depth estimation","feed_subtitle":"Swapping CLIP text for SeeCoder visual embeddings lifts KITTI δ1 to 0.974 and holds up in rain and night.","key_machinery":"The load-bearing component is the spatially enhanced SeeCoder. Its frozen SWIN-L backbone and decoder produce multi-scale feature maps; a Query Transformer turns these into 148 semantic vectors of dimension 768, of which 144 are local queries performing cross-attention with the feature maps and 4 are global queries that capture whole-scene context. On top of each SeeCoder transformer layer the paper adds dilated convolutions and a spatial-attention module (channel-pooled features concatenated and convolved into an attention map), and only these added modules are updated during training. The 148 vectors condition a from-scratch denoising UNet through cross-attention at every resolution, while a frozen VAE encoder supplies the latent representation; the UNet is trained to predict the added noise in the latent diffusion objective.","core_discovery":"The paper's central claim is that text-free semantic conditioning fixes a real failure mode of diffusion-based depth estimation in complex outdoor scenes. Its Table III reports δ1=0.974, RMSE=2.179, and AbsRel=0.052 on the KITTI Eigen validation split, ahead of the listed AdaBins, DPT, and ZoeDepth numbers. Table I shows that this comes from combining SeeCoder with both spatial modules, since SeeCoder alone reaches δ1=0.973 and either module alone collapses δ1 to 0.876–0.883. On Waymo, the model keeps global RMSE and AbsRel within roughly 10% across normal, rainy, and nighttime scenes, though δ1 falls sharply in rain and night. The paper acknowledges in its conclusion that it does not yet surpass the current state-of-the-art diffusion depth model and that textureless regions remain failure cases.","pith_inferences":["The split-and-fuse evaluation protocol is the most direct alternative explanation for the KITTI margin; comparing whole-image scores would separate the encoder's contribution from the test-time augmentation's contribution.","Because SeeCoder emits 144 local queries, the model is effectively learning a per-image set of visual prompts; varying that number would reveal how much semantic capacity depth estimation actually needs.","The same image-semantic conditioning should transfer to other dense prediction tasks currently tied to CLIP text prompts, such as referring segmentation or open-vocabulary detection, where scene captions are equally hard to generate.","The paired-module ablation suggests a caution for future work: adding a single spatial enhancement to a frozen semantic encoder can be destabilizing, so module interactions should be tested jointly rather than independently."],"forward_implications":["Hand-written text prompts are no longer a prerequisite for semantic conditioning in Stable-Diffusion-based depth estimation.","Spatial enhancement modules for SeeCoder should be used as a pair: using either dilated convolution or spatial attention alone degrades accuracy in the paper's ablation.","A model trained only on KITTI can be applied directly to Waymo normal, rainy, and nighttime scenes with global error metrics varying by only about 10%.","Textureless regions such as dark night areas and rain droplets on the lens remain systematic failure cases that future work must address.","The approach is positioned as a drop-in semantic condition, so other perception heads built on the same UNet-decoder design could adopt image-based semantics."],"supporting_citations":[{"why":"Supplies the image-based semantic encoder (SeeCoder) whose 148 visual queries replace CLIP text embeddings as the conditioning signal.","marker":"[14]"},{"why":"Provides the Stable-Diffusion-based perception framework (VPD) into which the semantic encoder is integrated and against which the approach is positioned.","marker":"[12]"},{"why":"Establishes latent diffusion and the frozen VAE encoder and denoising UNet machinery that the method builds on.","marker":"[11]"},{"why":"Defines the text-embedding conditioning that the paper argues is inadequate for complex outdoor scenes and that SeeCoder replaces.","marker":"[13]"},{"why":"Defines the Eigen split and the pioneering CNN baseline used for KITTI training, validation, and the comparison table.","marker":"[4]"},{"why":"Supplies the KITTI dataset and Eigen split protocol used for training and evaluation.","marker":"[15]"},{"why":"Supplies the Waymo scenes (normal, rainy, nighttime) used to test generalization.","marker":"[16]"},{"why":"Provides a recent transformer-based depth baseline (ZoeDepth) that the reported KITTI numbers are compared against.","marker":"[34]"},{"why":"Names the current diffusion-based depth state of the art that the paper acknowledges it does not yet surpass.","marker":"[9]"}],"fun_headline_variants":["SeeCoder visual embeddings outdo CLIP for monocular depth","Visual semantics improve depth estimation in rain and night","Text-free semantic encoding rivals SOTA depth on KITTI","Beyond CLIP: visual embeddings for robust depth estimation","Diffusion depth gains from visual semantics over text prompts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported KITTI numbers rely on a test-time protocol that splits each 1242×375 image into left and right halves, predicts each half with horizontal flip, and fuses the depth maps; the paper's comparison assumes this protocol does not inflate its metrics relative to baselines scored on whole images.","fun_headline_variants_meta":{"raw":{"variants":["SeeCoder visual embeddings outdo CLIP for monocular depth","Visual semantics improve depth estimation in rain and night","Text-free semantic encoding rivals SOTA depth on KITTI","Beyond CLIP: visual embeddings for robust depth estimation","Diffusion depth gains from visual semantics over text prompts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000501,"raw_usage":{"total_tokens":2433,"prompt_tokens":909,"completion_tokens":1524,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":1444}},"tokens_in":525,"tokens_out":1524,"duration_ms":11766,"temperature":1.0,"reasoning_tokens":1444,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T18:55:35.703196+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate the trained model on the KITTI Eigen validation set using whole 1242×375 images, without the left/right split or horizontal-flip fusion, but with the same Garg crop and valid-depth mask, and compare δ1, RMSE, and AbsRel with the numbers in Table III. If the metrics fall to the SeeCoder-only row (δ1≈0.973, RMSE≈2.216) or below the listed AdaBins and ZoeDepth numbers, then the split-fusion protocol rather than the image-semantic encoder is carrying the reported improvement.","supporting_citations":[{"cited_title":"Prompt- free diffusion: Taking","cited_arxiv_id":null,"evidence_quote":"Supplies the image-based semantic encoder (SeeCoder) whose 148 visual queries replace CLIP text embeddings as the conditioning signal."},{"cited_title":"Unleashing text-to-image diffusion models for visual perception","cited_arxiv_id":null,"evidence_quote":"Provides the Stable-Diffusion-based perception framework (VPD) into which the semantic encoder is integrated and against which the approach is positioned."},{"cited_title":"High-Resolution Image Synthesis with Latent Diffusion Models,","cited_arxiv_id":null,"evidence_quote":"Establishes latent diffusion and the frozen VAE encoder and denoising UNet machinery that the method builds on."},{"cited_title":"Learning transferable visual models from natural language supervision","cited_arxiv_id":null,"evidence_quote":"Defines the text-embedding conditioning that the paper argues is inadequate for complex outdoor scenes and that SeeCoder replaces."},{"cited_title":"Depth Map Prediction from a Single Image Using a Multi-Scale Deep Network,","cited_arxiv_id":null,"evidence_quote":"Defines the Eigen split and the pioneering CNN baseline used for KITTI training, validation, and the comparison table."},{"cited_title":"Vision Meets Robotics: The KITTI Dataset,","cited_arxiv_id":null,"evidence_quote":"Supplies the KITTI dataset and Eigen split protocol used for training and evaluation."},{"cited_title":"Scalability in perception for autonomous driving: Waymo open dataset","cited_arxiv_id":null,"evidence_quote":"Supplies the Waymo scenes (normal, rainy, nighttime) used to test generalization."},{"cited_title":"ECoDepth: Effective Condi- tioning of Diffusion Models for Monocular Depth Estimation,","cited_arxiv_id":null,"evidence_quote":"Names the current diffusion-based depth state of the art that the paper acknowledges it does not yet surpass."}],"review_version":1}