{"id":"ff2ee530-eb65-44c6-b14c-04b8f4b1cd14","arxiv_id":"2411.17385","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A six-task benchmark shows newer large vision models encode human-like monocular depth cues, and cue understanding strongly correlates with their depth estimation performance.","lead":"DepthCues measures whether large vision models understand six monocular depth cues humans use, from shadow association to texture gradients. Testing 20 pre-trained models, the authors find the cues emerge in newer models and that cue understanding tracks depth estimation performance.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The DepthCues–depth correlation is computed with the same frozen-feature probing protocol on both sides; a shared 'probeability' confound may inflate R²=0.83/0.80, so the benchmark's validation claim should be re-tested using external depth metrics.","rationale":"The paper builds a genuinely useful benchmark: six tasks are derived from established depth cues, with human validation (95%±1.48%), end-to-end and coordinate baselines, and careful layer selection. The central claim has two legs: (i) depth cues emerge in recent models, and (ii) DepthCues is validated as a proxy for depth perception by its strong correlation with depth estimation. Leg (ii) is the load-bearing one: if the correlation is real, then 'emergence' has external grounding; if it is an artifact of the shared probing protocol, the benchmark's interpretation collapses to 'these features can be probed for several geometry-related tasks,' which is a much weaker statement. The authors' own appendix (A.1) concedes that their probing numbers differ from prior work, reinforcing that the correlation is computed within a specific, possibly idiosyncratic protocol. A concrete, feasible check is to correlate DepthCues with externally published depth metrics, or with depth scores obtained by a fixed decoder independent of the probing setup. I do not think this warrants rejection: the dataset, baselines, and fine-tuning experiments are valuable, and the concern is addressable. It does, however, justify keeping the conditional verdict. I partially agree with the reader's weakest assumption: the probe-to-understanding gap is real, but the more specific and testable vulnerability is the shared-protocol confound on the validation correlation.","tokens_in":28608,"tokens_out":7979,"duration_ms":81267,"concrete_test":"Recompute the Spearman correlation between DepthCues average accuracy and depth-estimation quality using externally reported metrics for the same backbones (e.g., δ1 or AbsRel on NYUv2 from the original papers, or a single fixed DPT decoder fine-tuned on each frozen backbone), instead of the paper's own linear probes. If the rank correlation drops below ~0.6 or loses significance, the validation in Figs. A1–A2 is an artifact of the shared probing protocol; if it remains high, the benchmark validation stands.","verdict_should_be":"UNCHANGED","load_bearing_attack":"DepthCues is validated primarily by the claim that model performance on it correlates strongly with depth-estimation performance (Sec. 5.1(v); Fig. 4; Figs. A1–A2, R²=0.83 for NYUv2 and 0.80 for DIW). Critically, both sides of this correlation are obtained by the same protocol: lightweight probes on the same frozen features, with shared layer selection and probe hyperparameters (Sec. 4). Any model-level factor that makes features more linearly or nonlinearly probeable—better conditioning, higher spatial resolution, larger feature dimensionality, or architecture choices favorable to a two-layer MLP—will tend to raise both DepthCues scores and linear-probe depth scores, producing a correlation that does not specifically certify depth-cue understanding. The low ImageNet correlation does not eliminate this confound: the authors note their ImageNet probing numbers differ from prior work (Appendix A.1), and ImageNet linear probing is sensitive to class-token usage and object-centric features, making it a poor control for 'general probeability.' The end-to-end and coordinate baselines establish task solvability floors but do not remove between-model variation in probeability. Without an external anchor for depth ability, the R² values may overstate how well DepthCues tracks true monocular depth perception.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces DepthCues, a benchmark suite of six tasks designed to probe whether large pre-trained vision models encode the monocular depth cues used by humans: elevation, light-shadow, occlusion, perspective, size, and texture gradient. The authors probe 20 frozen vision models with lightweight MLP or attentive probes, report accuracies per cue, and compare models by average performance. Their main findings are that recent self-supervised and geometry-estimation models score higher, that DepthCues scores correlate strongly with linear-probe depth estimation on NYUv2 and DIW (R^2 = 0.83 and 0.80), and that LoRA fine-tuning on DepthCues can improve downstream depth estimation when the new features are concatenated with the original features. The benchmark, baselines, human accuracy estimate, and code release are described in detail.","tokens_in":28890,"tokens_out":5433,"duration_ms":54552,"significance":"If the validation claim holds, DepthCues would be a useful and inexpensive proxy for probing monocular depth understanding in frozen vision models, complementary to dense depth evaluation. The benchmark is grounded in vision-science cues, includes six carefully constructed tasks with existing datasets plus a synthetic texture-gradient task, and ships with important controls: coordinate-map and end-to-end baselines, human accuracy, threshold validation, and layer selection. The breadth of 20 models spanning SSL, classification, VLM, generative, segmentation, and multi-view objectives is a strength. However, the headline claims—that depth cues emerge in more recent larger models and that DepthCues is validated by its correlation with depth estimation—rest on correlational and shared-protocol evidence that needs additional support before the benchmark can be adopted as a proxy for depth perception.","major_comments":[{"comment":"The central validation claim that DepthCues performance is highly correlated with depth estimation is based on two quantities obtained from the same frozen features: DepthCues uses layer-searched MLP/attentive probes (Sec. 4.2) while NYUv2 and DIW use a linear probe on patch tokens (Sec. 5.1). Any model-level property that makes features generally more probeable—feature conditioning, spatial resolution, feature dimensionality, or architecture—will tend to raise both sides of the correlation, so the reported R^2=0.83 and R^2=0.80 can overstate how specifically DepthCues tracks depth ability. The low ImageNet correlation does not remove this confound, because ImageNet linear probing is itself sensitive to class-token usage and object-centric features, as the authors note in Appendix A.1. Please re-validate DepthCues against depth metrics obtained from an independent protocol (e.g., published full-model depth accuracies or depth heads fine-tuned on depth data), or at minimum report partial correlations controlling for a measure of general probeability.","section":"5.1(v), Figs. A1-A2, Fig. 4"},{"comment":"The claim that human-like depth cues 'emerge in more recent larger models' is inferred from a scatter plot against release date in which models also differ in architecture, pre-training objective, and dataset size (Table A6). This is a confounded trend, not a controlled emergence result; for example, DepthAnythingv2 is initialized from DINOv2 and trained with dense depth supervision, so its top score cannot be attributed to recency. The appendix acknowledges these confounds (Appendix A.1, Fig. A3), but the abstract and Sec. 5.1 state the conclusion without qualification. Please either restrict the claim to a descriptive statement about the evaluated checkpoints or provide matched comparisons that vary scale/recency while holding architecture and objective fixed.","section":"5.1(i), Fig. 1, Abstract"},{"comment":"The benchmark's interpretational claim rests on the assumption that probe accuracy on frozen features measures the model's understanding of a depth cue. The coordinate-map and end-to-end baselines establish task-solvability floors, but they do not identify which feature dimensions the probe uses; a probe can solve a task from low-level artifacts (e.g., shadow boundaries, mask statistics, or texture statistics) without engaging any geometric representation. Without a control that removes or perturbs the cue-specific information, or an analysis of the probe's decision features, the statement that 'the better the model understands the task-specific cue' is stronger than what the protocol establishes. Please soften the wording or add such a control.","section":"4.2, Eqs. (1)-(2)"},{"comment":"The statement that fine-tuning on DepthCues 'resulted in improved depth perception' is not supported by the +DC rows alone: DINOv2+DC drops NYUv2 accuracy from 87.78 to 87.06, and CLIP+DC drops from 43.78 to 43.59; improvements appear only when the fine-tuned features are concatenated with the original features. The abstract and Sec. 1 claim that merely fine-tuning on DepthCues improves depth estimation. Please report the concatenation-dependent result as the actual finding, or provide a fine-tuning variant that improves the standalone features, and state the result at the level of precision the data support.","section":"5.2, Table 2"}],"minor_comments":[{"comment":"Because the size cue in DepthCues uses SUNRGBD images that overlap NYUv2, the NYUv2 numbers in Table 2 may be partially contaminated; the authors note this and point to DIW as fairer, but the main-text sentence should explicitly say that the NYUv2 improvements are subject to this caveat.","section":"5.2"},{"comment":"The threshold validation for the perspective and elevation accuracies is reported as a correlation with the raw error; please also report the absolute success rates at the chosen thresholds so readers can calibrate how strict the 0.2 and 0.1 thresholds are.","section":"C.2, Figs. A15-A16"},{"comment":"Minor wording and naming inconsistencies: 'DepthAnyv2' vs. 'DepthAnythingv2', 'light-shadow' vs. 'light and shadow', and the informal phrase 'we evaluate our own performance' in Sec. 5.1. These do not affect the results but should be cleaned up.","section":"Throughout"},{"comment":"The reported Pearson r=0.69 for pre-training data size vs. DepthCues is computed after excluding the three language-supervised models; the text should state how the exclusion criterion was chosen and note that the remaining set still mixes objectives and architectures.","section":"A.1, Fig. A3"}],"recommendation":"major_revision","confidential_remarks":"The benchmark itself is a solid contribution and the authors have included useful controls. My main concern is that the paper's two load-bearing claims—that DepthCues is validated by correlation with depth estimation and that depth cues emerge in recent models—are both supported by correlational or shared-protocol evidence. I would not reject the manuscript; the fixes are within scope: obtain or cite external depth metrics, add matched model comparisons or soften the emergence claim, and qualify the fine-tuning result as concatenation-dependent. The benchmark and code release make this a likely valuable resource for the community once these points are addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: DepthCues is a genuinely useful benchmark, and the paper is better than the average benchmark paper. The six cue tasks are carefully built, the synthetic texture-gradient task is clean, and the probing protocol includes the right controls: coordinate-map and end-to-end baselines, human accuracy, layer search, and multiple seeds. The evaluation of 20 models is systematic, and the appendix makes a decent effort to disentangle architecture and pre-training objective where the model zoo permits. Credit where it is due.\n\nThe soft spots are real, though. The central validation claim — that DepthCues performance correlates strongly with depth estimation (R²=0.83 on NYUv2, 0.80 on DIW) — rests on shakier footing than the raw numbers suggest. Both the DepthCues scores and the depth-estimation scores come from probing the same frozen features. The probes differ (MLP/attentive vs. linear), but a model whose features are more generally probeable for geometric tasks will tend to do better on both. The low ImageNet correlation is a partial control, not a strong one; the authors themselves note their ImageNet probing numbers differ from prior work, and ImageNet probeability is not the same as geometric probeability. An external anchor — actual zero-shot depth performance, or evaluation on a depth benchmark that does not use their probing pipeline — would make the correlation convincing. As it stands, the R² values likely overstate how specifically DepthCues tracks depth perception.\n\nThe second headline claim, that depth cues emerge in more recent larger models, is confounded. Release date, data scale, objective, and architecture all vary across the 20 models. The appendix tries to control for some of these, but the abstract and Fig. 1 present a causal-sounding trend the data cannot support. That needs to be softened.\n\nThe fine-tuning experiment is the weakest section. The gains are small — DINOv2 NYUv2 accuracy goes from 87.78 to 88.46 only when concatenating fine-tuned and original features — and the concatenation requirement suggests the fine-tuned features alone lose generalization. The concat(noise) control rules out parameter-count effects, but it does not control for fine-tuning on any sparse task. A non-depth fine-tuning control (random labels, or a different sparse task) is needed before claiming depth cue priors cause the improvement.\n\nMinor: code and data are promised but not yet public, which will slow adoption. The size task's SUN-RGBD/NYUv2 overlap is acknowledged and handled via DIW, which is fine.\n\nBottom line: a solid benchmark paper that deserves serious peer review. The benchmark itself will be useful regardless of the correlation findings, and the analysis is careful in most places. The validation and emergence claims should be re-framed or re-tested, but these are addressable in revision. Send it out.","headline":"A genuinely useful benchmark with a real validation confound: the correlation with depth is computed on the same frozen features on both sides, so the headline R² numbers are weaker than they look.","tokens_in":29419,"tokens_out":3791,"would_cite":true,"duration_ms":36933,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Benchmark probing shows human-like depth cues emerge in newer, larger vision models, including ones trained only on images with no depth supervision.","keywords":["monocular depth cues","probing","benchmark","vision foundation models","self-supervised learning","depth estimation","emergent properties","transfer learning"],"falsifier":"Run the exact DepthCues probing protocol on two controls: features from a randomly initialized network, and features whose spatial layout is destroyed by global average pooling or patch shuffling. If either control solves elevation or perspective well above the coordinate-map trivial baseline, those tasks are being solved from image statistics rather than learned geometric understanding, and the emergence claim would not survive.","tokens_in":28436,"feed_emoji":"👁","tokens_out":8413,"duration_ms":73673,"temperature":0.7,"pith_summary":"The paper introduces DepthCues, a benchmark of six tasks designed to test whether large pre-trained vision models understand the same monocular depth cues that human vision uses: elevation, light and shadow, occlusion, perspective, size, and texture gradient. The authors probe 20 pre-trained models by training lightweight classifiers on frozen features and find that recent, larger models, including DINOv2 and Stable Diffusion, neither of which was trained with depth supervision, score highly. They further show that models' DepthCues scores correlate strongly with their monocular depth estimation accuracy, which is why the benchmark is a credible proxy for depth perception. Finally, they show that fine-tuning a model on the sparse cue-level tasks of DepthCues improves its downstream depth estimation, suggesting that geometric priors can be injected without dense depth labels.","feed_headline":"Vision models learn human depth cues without depth training","feed_subtitle":"Six cue tasks rank 20 models, and scores predict depth accuracy, explaining why image-only pretraining transfers to depth.","key_machinery":"The central object is the DepthCues benchmark itself: six tasks, each operationalising one classic monocular depth cue from the human vision literature — horizon-line regression for elevation, shadow-object association for light-shadow, occlusion detection, vanishing-point estimation for perspective, 3D size comparison, and depth ordering on textured planes for texture-grad. The protocol that carries the argument is feature probing: for each frozen model, task-specific features are extracted by masked average pooling of object regions or by taking the full feature map, and a lightweight probe (an MLP for binary tasks, an attentive probe for the two regression tasks) is trained on them, with the best model layer chosen by validation performance. The benchmark's validity rests on its controls and correlations: human accuracy of about 95% shows the tasks are well posed, a coordinate-map baseline sets the floor, and DepthCues scores across the 20 models trace the same ranking as depth estimation accuracy while staying nearly uncorrelated with ImageNet classification, so the benchmark measures something geometric rather than generic visual quality.","core_discovery":"The paper's central claim is that the information humans use for monocular depth — horizon position, shadow-object relations, occlusion, vanishing points, familiar object sizes, and texture compression — is present in the frozen features of recent large vision models even when those models were never trained on depth. It asserts that this emergence is a measurable phenomenon: probes placed on the features of DINOv2, Stable Diffusion, and other newer models solve the six DepthCues tasks well above trivial baselines, and the ranking of models on DepthCues tracks their ranking on actual depth estimation tasks, with $R^2 = 0.83$ on NYUv2 and $R^2 = 0.80$ on DIW. The paper also argues the causal direction runs both ways: injecting these cues through fine-tuning on DepthCues, with far sparser supervision than dense depth maps, still improves downstream depth estimation. Together these findings position DepthCues as a diagnostic for asking how and where depth perception develops in vision models.","pith_inferences":["Because all six tasks share one probing regime, DepthCues could be re-run as a controlled experiment over pre-training objectives — freezing architecture and data scale while varying only the objective — to isolate which recipes produce geometric features; the authors note they cannot control these variables with public checkpoints.","The strong correlation between DepthCues and depth estimation could be partly driven by shared low-level image statistics rather than shared geometric understanding; an adversarial test would perturb images (for example, removing texture or contrast) and check whether the two scores degrade together.","The benchmark covers static-image cues only, so extending it to motion parallax and ego-motion cues would test whether the emergence generalises to temporal geometry, which the paper explicitly leaves out.","The fine-tuning results hint at a curriculum for depth: cue-level objectives might serve as a warm start that makes dense depth fine-tuning faster or more data-efficient, an extension the paper does not test."],"forward_implications":["DepthCues can serve as a cheap proxy for depth estimation quality: probing a model on six cue tasks reveals its geometric understanding without requiring dense depth ground truth.","The emergence result explains why self-supervised backbones such as DINOv2 transfer so well to downstream dense depth prediction: the geometric cues are already encoded in their features.","Sparse cue-level supervision is a viable route to improving depth perception, since fine-tuning on DepthCues improved NYUv2 and DIW depth results for both DINOv2 and CLIP without dense depth labels.","Multi-view pre-training leaves a specific signature: models trained across viewpoints, such as CroCo, LRM, and DUSt3R, dominate the local texture-gradient task, showing what each pre-training recipe contributes geometrically.","No model masters all six cues, so the benchmark isolates which cue each pre-training objective fails to capture, giving a map for targeted improvement of geometric understanding."],"supporting_citations":[{"why":"DINOv2 is the self-supervised model whose high cue scores drive the emergence claim.","marker":"[57]"},{"why":"DepthAnythingv2 is the top-scoring model and anchors the correlation between DepthCues and depth estimation.","marker":"[80]"},{"why":"Established the probing protocol for 3D awareness that DepthCues adapts, and supplies the NYUv2 probing comparison.","marker":"[18]"},{"why":"Provides the concurrent protocol for probing 3D physical understanding whose evaluation setup DepthCues follows.","marker":"[84]"},{"why":"The HLW dataset supplies the horizon-line annotations used for the elevation task.","marker":"[77]"},{"why":"The SOBA dataset supplies the object-shadow pairs used for the light-shadow task.","marker":"[73]"},{"why":"The COCOA dataset supplies the occlusion annotations used for the occlusion task.","marker":"[91]"},{"why":"The NaturalScene dataset supplies the vanishing-point annotations used for the perspective task.","marker":"[89]"},{"why":"The DTD dataset supplies the textures used to build the synthetic texture-gradient task.","marker":"[13]"},{"why":"KITTI, together with SUN-RGBD, supplies the 3D bounding boxes that define ground truth for the size task.","marker":"[24]"}],"fun_headline_variants":["DepthCues benchmark shows depth cues emerge in large vision models","Vision models pick up human depth cues without ever being taught","DepthCues: probing how large vision models see depth","No depth labels needed: large vision models still learn depth cues","Depth cues in vision models predict depth accuracy without depth training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a probe trained on a model's frozen features solving a task means the model understands that depth cue, when probe success only demonstrates that some learnable signal, potentially a low-level statistical shortcut, exists in the features.","fun_headline_variants_meta":{"raw":{"variants":["DepthCues benchmark shows depth cues emerge in large vision models","Vision models pick up human depth cues without ever being taught","DepthCues: probing how large vision models see depth","No depth labels needed: large vision models still learn depth cues","Depth cues in vision models predict depth accuracy without depth training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000661,"raw_usage":{"total_tokens":3006,"prompt_tokens":917,"completion_tokens":2089,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":533,"completion_tokens_details":{"reasoning_tokens":2006}},"tokens_in":533,"tokens_out":2089,"duration_ms":15139,"temperature":1.0,"reasoning_tokens":2006,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:09:40.032298+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the exact DepthCues probing protocol on two controls: features from a randomly initialized network, and features whose spatial layout is destroyed by global average pooling or patch shuffling. If either control solves elevation or perspective well above the coordinate-map trivial baseline, those tasks are being solved from image statistics rather than learned geometric understanding, and the emergence claim would not survive.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DINOv2 is the self-supervised model whose high cue scores drive the emergence claim."},{"cited_title":"Depth any- thing v2","cited_arxiv_id":null,"evidence_quote":"DepthAnythingv2 is the top-scoring model and anchors the correlation between DepthCues and depth estimation."},{"cited_title":"A general protocol to probe large vision models for 3d physical understanding","cited_arxiv_id":null,"evidence_quote":"Provides the concurrent protocol for probing 3D physical understanding whose evaluation setup DepthCues follows."},{"cited_title":"Horizon lines in the wild","cited_arxiv_id":null,"evidence_quote":"The HLW dataset supplies the horizon-line annotations used for the elevation task."},{"cited_title":"Instance shadow detection","cited_arxiv_id":null,"evidence_quote":"The SOBA dataset supplies the object-shadow pairs used for the light-shadow task."},{"cited_title":"Semantic amodal segmentation","cited_arxiv_id":null,"evidence_quote":"The COCOA dataset supplies the occlusion annotations used for the occlusion task."},{"cited_title":"Detect- ing dominant vanishing points in natural scenes with appli- cation to composition-sensitive image retrieval.Transactions on Multimedia, 2017","cited_arxiv_id":null,"evidence_quote":"The NaturalScene dataset supplies the vanishing-point annotations used for the perspective task."},{"cited_title":"Are we ready for autonomous driving? the kitti vision benchmark suite","cited_arxiv_id":null,"evidence_quote":"KITTI, together with SUN-RGBD, supplies the 3D bounding boxes that define ground truth for the size task."}],"review_version":1}