{"id":"15c6656f-51d5-4464-aa74-e51f4b64ee19","arxiv_id":"2505.02704","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"VGLD fuses CLIP image and text embeddings to predict the global scale and shift that convert relative monocular depth into metric depth, outperforming the text-only RSA baseline.","lead":"This paper adds visual cues from CLIP to language-based monocular depth scale recovery, predicting a global scale and shift that turn relative depth maps into metric depth. The authors report improved accuracy and stability over the text-only RSA method on NYUv2 and KITTI, with zero-shot transfer to other datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central disambiguation claim is unsupported because image-only VGLD matches or beats text+image on several zero-shot sets, and the caption-variance evidence covers only three images per benchmark.","rationale":"The reader's weakest assumption precisely identifies the load-bearing risk: the claimed disambiguation mechanism requires the text branch to carry scale-relevant, caption-dependent information that the visual branch can resolve. The paper's ablations support the alternative explanation that the visual branch predicts scale from image appearance while the text branch contributes little. My reading of Tables 5-13 confirms this pattern: TCI is often only marginally better than I, and on DIML the image-only variant is sometimes better. The sensitivity study in Tables 7-10 is the only direct test of stability across captions, and it uses just three images per benchmark, so it cannot establish the generality of the disambiguation claim. The Limitations section's own admission of caption sensitivity further weakens the strong abstract claim. However, this is not a fatal internal inconsistency: the method may still be a useful scale-recovery module, and the in-domain results are consistently positive. The appropriate disposition remains CONDITIONAL, exactly as the reader concluded, with the condition being stronger evidence that the text modality contributes beyond the image branch. I therefore recommend no change to the reader's verdict. Credit is due for the LM-based pseudo-label supervision, the domain routing idea, and the breadth of backbones tested, but none of these directly tests the linguistic-disambiguation mechanism.","tokens_in":23163,"tokens_out":6038,"duration_ms":67335,"concrete_test":"Run the caption-sensitivity protocol on the full NYUv2 and KITTI test splits using all 12 LLaVA-generated captions per image from the Table 14 prompts. For each image and each method (RSA, VGLD-T, VGLD-I, VGLD-TCI), compute per-caption AbsRel and predicted scale/shift, then average within each image and compute the across-caption standard deviation of AbsRel and of predicted scale. Perform a paired bootstrap or Wilcoxon test comparing VGLD-TCI versus VGLD-I on both mean AbsRel and across-caption variance. If VGLD-TCI is not significantly better than VGLD-I on both criteria, the visual-disambiguation mechanism is unsupported: the image branch alone already supplies the scale cue, and the text branch is redundant.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline mechanism is 'visually-guided linguistic disambiguation': visual semantics are claimed to resolve inconsistencies in text so that scale prediction is stable and accurate. The architecture concatenates frozen CLIP image and text embeddings into GlobalNet, so nothing forces the text branch to contribute; the load-bearing assumption is that fusing text with image outperforms image alone specifically under caption variation. The ablations do not establish this. On in-domain benchmarks, TCI improves over I by only 1-3% AbsRel, and on zero-shot DIML (Table 12) VGLD-NK-I beats VGLD-NK-TCI for MiDas-1 (0.188 vs 0.212) and DA V1 (0.193 vs 0.196). Across SUN-RGBD and DDAD the TCI margins are small. More importantly, the only direct evidence of disambiguation is a sensitivity study using three images per dataset and three captions each (Tables 7-10), with no full-test-set caption-variance statistics and no paired significance tests. The Limitations section concedes that VGLD 'may output inconsistent results' under different descriptions. Thus the current experiments cannot distinguish VGLD from a visual-only scale predictor; the text branch may be nearly redundant, and 'disambiguation' may be an interpretation imposed on what is effectively image-based scale regression. This is a correctness-risk issue for the central claim, not merely a disagreement with current consensus.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces VGLD, a lightweight post-hoc module that recovers metric depth scale and shift from a relative depth map by jointly encoding a CLIP text embedding of a caption and a CLIP image embedding, then predicting a global affine transformation. The method adds a domain router (indoor/outdoor) and a Levenberg-Marquardt pseudo-label loss to stabilize training. Experiments compare VGLD against RSA and other baselines on NYUv2, KITTI, and zero-shot SUN-RGBD, DIML, and DDAD, with ablations of the text-only, image-only, and text-plus-image variants. The stated central claim is that visual semantics disambiguate inconsistent textual descriptions, improving both accuracy and stability of scale recovery.","tokens_in":23423,"tokens_out":5808,"duration_ms":64435,"significance":"If the disambiguation claim were fully supported, VGLD would be a practical, low-cost contribution to monocular metric depth recovery, since it makes relative depth models metric without retraining them. The paper has clear strengths: it evaluates four relative-depth backbones, includes a domain-router ablation and an LM-loss ablation, reports computational cost, and is written with enough detail to be reproduced. However, the load-bearing mechanistic claim is currently under-supported: the image-only variant is competitive with or better than the text-plus-image variant in several zero-shot settings, the caption-robustness evidence is limited to three images per dataset, and the main tables lack error bars. The paper therefore establishes VGLD as a reasonably strong scale-recovery method, but it does not yet establish visually-guided linguistic disambiguation as an operating mechanism.","major_comments":[{"comment":"The central claim that adding the text branch to the image branch (TCI) improves over image-only (I) is not consistently supported. On DIML Indoor (Table 12), VGLD-NK-I outperforms VGLD-NK-TCI for MiDas-1 (AbsRel 0.188 vs 0.212) and for DA V1-vits (0.193 vs 0.196). On SUN-RGBD (Table 11) the MiDas-1 I variant also beats TCI (0.259 vs 0.262), while on DDAD (Table 13) TCI is better for all backbones. The margins are small and no paired significance tests or confidence intervals are reported, so the ranking of I vs TCI is not statistically meaningful. The authors should provide per-dataset paired tests across the full test sets, or explicitly temper the claim that visual grounding of text yields consistent gains.","section":"Supplementary Material, Tables 11-13 (zero-shot results)"},{"comment":"The only direct evidence for caption robustness is based on three images per dataset with three captions each, reported without summary statistics over a broader sample. The Limitations section concedes that VGLD 'may output inconsistent results' under different descriptions and deteriorates under incorrect descriptions. This is insufficient to support the abstract's claim that VGLD 'significantly mitigates scale estimation bias caused by inconsistent or ambiguous language.' The authors should run a caption-variance study on a random subset (or the full set) of test images, reporting the distribution of predicted (k,b) and of depth errors across captions, and test whether the text-plus-image model reduces variance relative to the image-only model.","section":"Sensitivity to Variations in Linguistic Descriptions (Figures 3-4; Supplementary Tables 7-10)"},{"comment":"The architecture concatenates CLIP image and text embeddings and feeds them to an MLP, with no gating, cross-attention, or other mechanism that forces the text branch to be modulated by visual content. Consequently, the network can in principle learn an image-only scale predictor while ignoring the text branch. The ablations show that the image-only variant is close to or better than the fused variant in several settings, so the 'disambiguation' claim is currently an interpretation rather than an observed property. A concrete test would be to measure, for a fixed image, the variance of predicted (k,b) across captions for the I and TCI variants, and to verify that TCI reduces this variance while also being at least as accurate; alternatively, the architecture should be revised so that visual features explicitly modulate or gate the text representation.","section":"Method (Figure 2; Eq. 5)"},{"comment":"The main tables report point estimates without error bars or confidence intervals, even though the differences between I and TCI variants are often only a few thousandths of AbsRel (e.g., Table 1, MiDas-1 NYUv2: 0.121 vs 0.119; MiDas-1 KITTI: 0.120 vs 0.120). Supplementary Figure 10 reports seed variance for a subset of models, but this is not carried into the principal comparison tables. Without uncertainty quantification, the claimed advantage of TCI over I cannot be evaluated, and the DIML results where I beats TCI cannot be dismissed as noise. The authors should add confidence intervals (or multiple-seed mean/std) to all main and zero-shot tables, and perform paired significance tests for the I-vs-TCI and VGLD-vs-RSA comparisons.","section":"Experimental Settings and Tables 1, 5-6, 11-13"}],"minor_comments":[{"comment":"The D1 value 0.942 reported for VGLD-N/K-I on KITTI is inconsistent with the more detailed Table 6, where the corresponding values are 0.733 (VGLD-K-I) and 0.740 (VGLD-NK-I); please correct the main table.","section":"Table 1, MiDas-2 KITTI row"},{"comment":"The text states a '32.1% improvement in Abs Rel on NYUv2' for the DA V1-vits backbone, but recomputing from Table 1 (RSA-N/K 0.147 vs VGLD-NK-TCI 0.112) gives approximately 23.8%, and comparing against RSA-NK (0.148) gives 24.3%; please verify the arithmetic and the reference.","section":"Table 4 and surrounding text"},{"comment":"The VGLD-NK-TCI rows for DA V2-vits and DA V1-vits are identical across all metrics (0.241, 0.545, 0.433, 0.189, 0.067, 0.779, 0.967, 0.993); this is likely a copy-paste error and should be corrected.","section":"Table 11"},{"comment":"The scalar columns (pred shift, LM shift, pred scale, LM scale) are visually garbled, with values from different rows running together; the tables need reformatting so each row has one aligned value per column.","section":"Supplementary Tables 7 and 9"},{"comment":"The sentence 'In our experiments, we set α and β to 0.1, as is customary' is vague; since β is ablated in Table 3, the choice for α should be justified or referenced, and 'customary' should be removed unless a citation is given.","section":"Eq. (5) and hyperparameters"},{"comment":"The manuscript contains several presentation issues: the typo 'Tabel 1' in Table 4, inconsistent spelling of 'MiDaS' vs 'MiDas', and the table footnote 'robust depth estimation‡' which ends mid-phrase; please proofread the final version.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The empirical core of the paper (VGLD outperforms RSA, DRM helps, LM loss helps) is credible, but the central 'visually-guided linguistic disambiguation' claim is not yet demonstrated, and some of the reported numbers (D1 typo in Table 1, identical TCI rows in Table 11, the 32.1% improvement figure) raise data-integrity concerns that should be resolved before the paper is accepted. I recommend major revision with an emphasis on adding full-set caption-variance experiments, paired significance tests, and confidence intervals, and on moderating the abstract claims to match the evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick read on arXiv:2505.02704. The paper takes RSA's language-guided scale recovery and adds a frozen CLIP image encoder, a domain router borrowed from ZoeDepth, and LM pseudo-label supervision. That works: on NYUv2 and KITTI, across four relative-depth backbones, the fused model beats RSA by a wide margin and lands close to what Levenberg-Marquardt fitting to ground truth achieves. Zero-shot results on SUN-RGBD, DIML, DDAD also show consistent gains over RSA. That is a real, if incremental, empirical contribution, and the ablations on the DRM and the LM loss are sensible.\n\nThe soft spot is the central mechanism claim. The paper frames VGLD as 'visually-guided linguistic disambiguation' — visual semantics resolve inconsistencies in text. But the ablations do not show that. Image-only VGLD is nearly as good as text-plus-image on in-domain benchmarks, and on several zero-shot sets it is better (e.g., DIML with MiDas-1: 0.188 vs 0.212). The only direct evidence of disambiguation is a sensitivity study that uses three images per dataset and three captions each, with no significance tests. Meanwhile the Limitations section concedes VGLD 'may output inconsistent results' under different descriptions. So the current evidence is fully consistent with the simpler story: the text branch is almost redundant and the model is doing visual scale regression. The authors should either provide a proper caption-variance study on the full test set or soften the disambiguation claim.\n\nOther issues: main tables lack error bars (the seed experiment in the supplement shows minimal variance, but that is not the same as run-to-run or test-set variance), and code is not available. A few supplementary tables also have formatting misalignments that make exact comparisons harder than they should be.\n\nI would not call this circular — the LM targets are a supervised training signal, not a self-referential definition of the output.\n\nBottom line: this is a useful, honest engineering paper that deserves a serious referee, but the mechanism claim needs evidence or rewriting. If I were handling it, I would send to review with a request for code, error bars, and a full-test-set caption-variance experiment before acceptance.\n\nMy take: read it if you work on language-conditioned depth; cite as related work if you need the RSA comparison. Not a paper that changes how I think about scale recovery.","headline":"A plausible incremental extension of RSA that delivers better benchmark numbers, but the headline disambiguation story is not supported by the paper's own ablations.","tokens_in":23961,"tokens_out":2133,"would_cite":false,"duration_ms":22689,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"VGLD adds visual semantics to caption-based scale recovery and claims this grounding stabilizes and improves metric depth estimates from relative depth maps.","keywords":["monocular depth estimation","metric scale recovery","visually-guided disambiguation","CLIP embeddings","domain routing","relative depth","zero-shot generalization"],"falsifier":"Run VGLD on the same image paired with several captions, including a semantically wrong one such as describing an indoor scene as a street; if the predicted scale and shift remain essentially unchanged, the text branch is inert and the disambiguation mechanism is not what produces the reported accuracy.","tokens_in":22915,"feed_emoji":"📏","tokens_out":7863,"duration_ms":78055,"temperature":0.7,"pith_summary":"VGLD claims that the unreliable scale information in natural-language captions can be stabilized by adding the image itself: a CLIP text encoder and a CLIP image encoder are fused and fed to a small network that predicts the global scale and shift needed to turn a relative depth map into metric depth. The paper shows that this fused approach outperforms the text-only language-guided baseline RSA on NYUv2 and KITTI across four relative depth backbones, and that the gain is largest when captions vary for the same image. It also introduces a domain router that sends the fused features to separate indoor and outdoor prediction heads, allowing one module trained on both datasets to work zero-shot on other indoor and outdoor benchmarks. The result matters because it suggests monocular depth scale can be recovered from language plus visual semantics without LiDAR, IMUs, or camera intrinsics.","feed_headline":"Visual grounding steadies language-based depth scale recovery","feed_subtitle":"Adding the image to caption-guided scale prediction cuts error and steadies results across different wordings.","key_machinery":"The load-bearing machinery is the reciprocal-linear scale transformation $\\hat{d}_{\\mathrm{pred}} = 1/(\\hat{k}\\,x + \\hat{b})$, in which the scale-less relative depth map $x$ is converted to metric depth by two global scalars. The scalars come from a fused representation formed by concatenating CLIP image and text embeddings, passed through GlobalNet, a three-layer MLP that produces a 256-dimensional latent vector. A Domain Router Mechanism, an MLP classifier inspired by ZoeDepth, predicts indoor versus outdoor from that latent vector and routes it to the corresponding domain-specific prediction head. Training is stabilized by pseudo-labels $(\\hat{k}_{\\mathrm{lm}}, \\hat{b}_{\\mathrm{lm}})$ obtained from Levenberg-Marquardt fitting of ground-truth depth, added to the metric L1 loss and the domain cross-entropy loss.","core_discovery":"The paper's central discovery is that jointly encoding the image with the caption removes most of the sensitivity that pure language-guided scale recovery has to wording. VGLD takes the relative depth map from a frozen backbone, extracts CLIP embeddings of both the caption and the image, concatenates them, and regresses two scalars $\\hat{k}$ and $\\hat{b}$ so that $\\hat{d}_{\\mathrm{pred}} = 1/(\\hat{k}\\,x + \\hat{b})$ matches metric depth. Supervised by a pixel-wise L1 loss, a domain classification loss, and regression targets obtained by Levenberg-Marquardt fitting of ground-truth depth, the model learns to output the same scale and shift for semantically equivalent paraphrases of a scene. Across MiDaS and Depth Anything backbones on NYUv2 and KITTI, VGLD's fused text-plus-image variant reports lower Abs Rel and RMSE than the RSA baseline, and zero-shot results on SUNRGBD, DIML, and DDAD come close to LM fitting on ground truth. The paper interprets this as evidence that visual semantics disambiguate the text rather than replace it.","pith_inferences":["The paper's zero-shot tables show image-only variants often matching or beating the text-plus-image variants, which suggests the practical driver of accuracy may be visual appearance itself rather than visual disambiguation of the caption; a stronger test would be to ablate the text branch entirely.","Because the predicted parameters are global scalars per image, VGLD could be attached to newer relative depth backbones, including diffusion-based zero-shot depth models, without retraining the backbone.","The domain routing idea could be extended beyond indoor and outdoor to finer scene categories, which might further reduce scale ambiguity in mixed-domain deployment.","The paper's stated limitation that wrong captions still shift predictions suggests a natural next step: use visual-textual similarity to reject or down-weight captions that do not match the image."],"forward_implications":["VGLD converts scale-less relative depth maps into metric depth using only a caption plus the image, without ground-truth depth at inference time.","Because it learns a single set of global scalars and a domain router, the same lightweight module transfers across MiDaS and Depth Anything backbones and across indoor and outdoor scenes.","Training on both NYUv2 and KITTI costs little accuracy per domain compared with single-dataset training, so the module can act as a universal alignment layer.","Zero-shot results on SUNRGBD, DIML, and DDAD indicate the alignment generalizes to datasets not seen in training, approaching the quality of nonlinear least-squares fitting on ground-truth depth.","Replacing text-only input with fused image-plus-text input reduces variance across different captions of the same image, particularly for diverse indoor captions."],"supporting_citations":[{"why":"Defines the language-guided scale recovery baseline RSA that VGLD extends and compares against, sharing its linear-transformation formulation and L1 loss.","marker":"Zeng et al. 2024b"},{"why":"Supplies the CLIP image and text encoders that produce the fused visual-linguistic embedding used for scale prediction.","marker":"Radford et al. 2021"},{"why":"Inspires the Domain Router Mechanism, which routes the fused embedding to indoor or outdoor scale-prediction heads.","marker":"Bhat et al. 2023"},{"why":"Provides the MiDaS 3.1 relative depth backbone whose scale-less predictions are aligned to metric depth by VGLD's scalars.","marker":"Reiner et al. 2023"},{"why":"Supplies the MiDaS 3.0 relative depth backbone used in evaluations and the least-squares and LM fitting baselines.","marker":"Ranftl et al. 2020"},{"why":"Provides the Depth Anything V1 relative depth backbone used in the paper's experiments.","marker":"Yang et al. 2024a"},{"why":"Supplies the NYUv2 indoor dataset used for training and evaluation.","marker":"Silberman et al. 2012"},{"why":"Supplies the KITTI outdoor dataset used for training and evaluation.","marker":"Geiger, Lenz, and Urtasun 2012"}],"fun_headline_variants":["Seeing the scene resolves caption ambiguity for depth scale","Image-text fusion boosts monocular depth scale accuracy","VGLD: Vision disambiguates language for metric depth","Don't trust the caption alone: add the image for depth scale","Ground depth scale in the image, not just the words"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole mechanism rests on the premise that captions carry scale-relevant information and that the image can pick the right interpretation of that information, rather than simply supplying the scale by itself.","fun_headline_variants_meta":{"raw":{"variants":["Seeing the scene resolves caption ambiguity for depth scale","Image-text fusion boosts monocular depth scale accuracy","VGLD: Vision disambiguates language for metric depth","Don't trust the caption alone: add the image for depth scale","Ground depth scale in the image, not just the words"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000608,"raw_usage":{"total_tokens":2869,"prompt_tokens":1019,"completion_tokens":1850,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":635,"completion_tokens_details":{"reasoning_tokens":1770}},"tokens_in":635,"tokens_out":1850,"duration_ms":14662,"temperature":1.0,"reasoning_tokens":1770,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:43:38.596965+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run VGLD on the same image paired with several captions, including a semantically wrong one such as describing an indoor scene as a street; if the predicted scale and shift remain essentially unchanged, the text branch is inert and the disambiguation mechanism is not what produces the reported accuracy.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the MiDaS 3.0 relative depth backbone used in evaluations and the least-squares and LM fitting baselines."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the KITTI outdoor dataset used for training and evaluation."}],"review_version":1}