{"id":"c2244638-c1e8-4c63-a4cd-e2d2dcf2f447","arxiv_id":"2601.22054","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A sparse-prompt pretraining framework on 20M heterogeneous image-depth pairs yields a scaling trend and state-of-the-art metric depth across many downstream tasks.","lead":"The paper trains a depth-estimation model on about 20 million noisy image-depth pairs from many cameras and sensors, using random sparse depth points as prompts. It reports a clear scaling trend for metric depth and state-of-the-art results across depth completion, monocular depth, 3D reconstruction, robot planning, and spatial question answering.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Metric scaling claim depends on unverified per-source metric scale; robust MAE and prompt alignment cannot correct systematic calibration errors","rationale":"The reader's weakest assumption is the same one I identify: every source must be reliably convertible to per-pixel metric depth, with only pixel-level noise. This is load-bearing because the central claim is explicitly about metric depth scaling — if the labels carry source-level scale errors, the 'metric' part of the claim is not established. My analysis sharpens the reader's concern by noting that the robust MAE (Eq. 8) and prompt alignment cannot correct such systematic errors, and that the 'reconstructed 3D data' category in Sec 3.1 is especially vulnerable because SfM/SLAM/MVS outputs are algorithm-inferred, not sensor-measured. The paper's strong downstream results are not sufficient to dismiss this: zero-shot benchmarks (e.g., NuScenes, KITTI) use LiDAR-like sensors that may overlap with the dominant training sensors, and the student is distilled from the teacher, so both inherit the same potential bias. Other issues raised by the reader (DIODE hyperparameter tuning leak, no error bars, no code/data release) are real but secondary: they affect reproducibility and some SOTA claims, but the scaling claim itself would still be testable if the data were reliable. I agree with the CONDITIONAL verdict: the central idea is coherent and plausible, but the metric-scale premise must be audited before the 'first scaling trend' claim can be accepted. The proposed concrete test — restricting the scaling experiment to verified-metric sources and checking per-source scale errors — would directly resolve whether the concern lands. If the trend persists on verified-metric data, the concern is mitigated; if it does not, the scaling claim is an artifact of unverified sources. Therefore no change to the reader's verdict is needed.","tokens_in":34928,"tokens_out":9198,"duration_ms":99141,"concrete_test":"Run the scaling experiment of Sec 5.1 / Fig 2a using only sources with independently verified metric calibration (e.g., captured LiDAR/RGB-D with known intrinsics and sensor-to-camera pose, excluding SfM/MVS/SLAM outputs and other algorithm-generated depth). Also produce a per-source manifest that reports, for each source, the median scale error against an independent metric reference (e.g., overlapping LiDAR frames). If, after excluding sources with median scale error >5%, the monotonic scaling trend is substantially weakened, the central claim depends on unverified metric labels.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim — a clear scaling trend in metric depth from ~20M heterogeneous pairs — assumes the aggregated depth labels are metric (up to tolerable pixel-level noise). Section 3.1 aggregates 'reconstructed 3D data' from SfM, SLAM, MVS, and stereo algorithms without any per-source scale verification. Monocular SfM is scale-ambiguous; SLAM can drift; even stereo requires known baselines. The paper provides no dataset manifest, no per-source calibration audit, and no independent check that each source's depth is in meters along the camera z-axis. The robust MAE loss (Eq. 8) discards only the top-20% largest-error pixels per image; this mitigates sparse outliers but cannot correct a coherent scale/offset error affecting all pixels of a source. The prompt preparation (PDSA/GMDR) aligns the prompt to the sampled depth values, but the pretraining target remains the raw depth map G, so it does not fix systematic source bias. If any major source has a global scale error, the model learns a mixture of scales, and the reported scaling trend could reflect fitting source biases rather than acquiring true metric depth. The strong zero-shot results on KITTI-like or LiDAR-like benchmarks do not disprove this, because those benchmarks may share sensor characteristics with the biased sources. Without an audit, the 'metric' premise of the entire scaling claim is unsecured.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Metric Anything, a pretraining framework for metric depth estimation that aggregates roughly 20M image–depth pairs from reconstructed, captured, and rendered 3D data. The method uses randomly masked depth maps as \"Sparse Metric Prompts\" to train a depth-conditioned teacher, which is then distilled into a prompt-free student with a distance-balanced inverse-depth loss. The authors claim, for the first time, a clear scaling trend in metric depth pretraining, and report state-of-the-art zero-shot results across depth completion/super-resolution, radar–camera fusion, monocular depth estimation, camera intrinsics recovery, multi-view metric 3D reconstruction, VLA planning, and MLLM spatial reasoning. The paper emphasizes a minimalist, data-driven paradigm with no task-specific prompt engineering.","tokens_in":35358,"tokens_out":2373,"duration_ms":28646,"significance":"If the central claim holds, the paper would establish a valuable scaling route for metric depth estimation, analogous to progress in relative depth. The work is substantial in empirical scope: it aggregates a very large heterogeneous dataset, demonstrates a monotonic trend with data size, and evaluates across many downstream tasks. The paper also ships an open-source release, which is a concrete strength. The main scientific value lies in the scaling claim and in the demonstration that a simple random-mask prompt can unify heterogeneous sources. However, the current evidence for the scaling claim rests on a single benchmark, and at least one hyperparameter appears to have been tuned on an evaluation dataset, which tempers the strength of the reported SOTA results.","major_comments":[{"comment":"The student-loss hyperparameter C is selected on DIODE (Tab. 15, right), and DIODE also appears as an evaluation benchmark in the monocular geometry estimation tables (Tabs. 5 and 6). This contradicts the strict zero-shot protocol claimed in Sec. 4.2.1, where the six datasets are stated to be unused for hyper-parameter tuning. The DIODE rows in Tabs. 5–6 are therefore not trustworthy as zero-shot results, and the average ranks that include them are inflated. Please retune C on a held-out set disjoint from all evaluation benchmarks, or remove DIODE from the evaluation tables.","section":"Sec. 5.7 / Tab. 15 (right) and Tabs. 5–6"},{"comment":"The central scaling claim is supported only by δ1 accuracy on Middlebury (Fig. 2a), with no error bars, no multiple training runs, and no description of whether the data fractions are nested or independently sampled. A single benchmark without variance does not establish a 'clear scaling trend' robustly, especially given the noise inherent in heterogeneous data. Please report scaling curves on at least two or three benchmarks, include multiple seeds or confidence intervals, and clarify the subset construction.","section":"Sec. 5.1 / Fig. 2a"},{"comment":"The metric premise of the entire method is asserted but not audited. Section 3.1 includes 'reconstructed 3D data' from SfM, SLAM, MVS, and stereo, whose outputs are not automatically metric; monocular SfM is scale-ambiguous, SLAM can drift, and stereo requires known baselines. The robust MAE loss in Eq. (8) drops only the top-20% per-image largest errors, which cannot correct a coherent per-source scale or offset error. The prompt preparation step (Sec. 3.2) aligns prompts to sampled depth values but does not correct systematic source-level bias in the target G. Please provide a dataset manifest with per-source scale verification, or independent checks (e.g., comparison against LiDAR or known camera trajectories), to substantiate the claim that all 20M pairs are metric along the camera z-axis.","section":"Sec. 3.1 and Eq. (8)"},{"comment":"The prompt-free student is trained entirely on teacher-generated pseudo-labels. This is circular in the sense that any systematic metric error in the teacher (e.g., from the unverified sources above) is inherited by the student. The student results therefore cannot be used as evidence that the aggregated data are metric. This is not a flaw per se, but it means the student's SOTA numbers do not independently validate the metric premise; they only show consistency with the teacher.","section":"Sec. 3.3"}],"minor_comments":[{"comment":"Typo: 'MoGo-2' should be 'MoGe-2' (the method referenced in [110]).","section":"Sec. 7.3"},{"comment":"The text refers to 'Eq.14' when describing the balance weight C; the correct reference is Eq. (5).","section":"Sec. 5.7"},{"comment":"Typo: 'metric depth trick' should be 'metric depth track.'","section":"Sec. 9"},{"comment":"The term 'boundaris' should be 'boundaries' in the section heading and text.","section":"Sec. 4.2.3"},{"comment":"The benchmark is called VSI-Bench in the text and table, but 'VIS Benchmark' appears in Fig. 12; please make the naming consistent.","section":"Sec. 4.2.6 and Fig. 12"},{"comment":"Reference [36] and [35] appear to be the same paper; please deduplicate.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The central scaling claim is plausible and the empirical sweep is impressive, but the DIODE hyperparameter tuning issue and the unverified metric scale of reconstructed sources are load-bearing for the headline claims. These are fixable with a held-out tuning set, a per-source audit, and additional scaling evidence. I would not reject, but the current version overstates the strength of the zero-shot and scaling results."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. This is a serious scaling attempt for metric depth: 20M image-depth pairs from heterogeneous sources, a random-mask \"Sparse Metric Prompt\" as a universal interface, and a distilled prompt-free student. The zero-shot scaling curve on Middlebury is monotonic, and the downstream coverage is wide. The catch is that the central claim—a clear scaling trend in metric depth—rests on an unverified premise: that all 20M depth maps are actually metric. There is no dataset manifest, no per-source scale audit, and the robust MAE loss (drop top 20% errors per image) cannot correct a coherent scale or offset bias in a source.\n\nWhat is genuinely new: assembling a corpus of this size and showing that a simple random-mask prompt, with a DepthPro-style teacher and a distilled student, gives monotonic zero-shot improvements. The student's distance-balanced inverse-depth loss and the inverted skip connections are reasonable and are ablated. The evaluation breadth—depth completion, super-resolution, radar-camera fusion, monocular depth, camera intrinsics, multi-view reconstruction, VLA planning, MLLM spatial reasoning—is impressive, and the limitations section is honest (central-pinhole only, no model-scaling study).\n\nSoft spots, in order. (1) Metricity of the data. Reconstructed data from SfM/SLAM/MVS is scale-ambiguous; the paper asserts known calibration without showing it. If any large source has a global scale error, the model learns a mixture of scales and the scaling trend could reflect fitting those biases rather than acquiring genuine metric depth. The prompt-alignment step (PDSA/GMDR) only fixes the prompt, not the target G. This is the load-bearing weakness. (2) Test-set tuning: the student loss constant C is chosen on DIODE (Sec. 5.7/Tab. 15) and DIODE appears as an evaluation benchmark in Tabs. 5 and 6. That contaminates part of the SOTA claims. (3) The scaling evidence is thin: one benchmark for the student, one for the pretrained model, no error bars. (4) No code, data, or checkpoints are available yet, despite an open-source promise, so the results are not independently reproducible.\n\nWho should read it: anyone working on metric depth pretraining, depth completion, or camera calibration. It deserves a serious referee—the core idea is good and the flaws are fixable—provided the authors release the dataset manifest and per-source quality analysis, move the DIODE tuning off the test set, and add uncertainty to the scaling curves. If those checks hold, this could genuinely change how noisy cross-sensor data is used for metric depth. As it stands, treat it as a strong position paper with an unverified headline.","headline":"Promising scaling idea for metric depth, but the metricity of the 20M labels is unverified and one hyperparameter is tuned on an evaluation set; deserves a real referee but needs major fixes.","tokens_in":35778,"tokens_out":5332,"would_cite":false,"duration_ms":55849,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Metric Anything claims that metric depth estimation finally scales: roughly 20M heterogeneous image-depth pairs produce steady accuracy gains and transfer across depth completion, monocular depth, camera calibration, 3D reconstruction, and","keywords":["metric depth estimation","scaling laws","sparse prompts","heterogeneous 3D data","pretraining","distillation","zero-shot generalization","spatial reasoning"],"falsifier":"Take one large reconstructed source, multiply all its depth values by a fixed 15% scale, retrain at full data scale, and measure zero-shot relative error on several held-out datasets. If the model's error is sharply worse, the robust loss is absorbing source-level bias rather than pixel noise, and the scaling claim would not hold for arbitrary heterogeneous sources. A lighter version: report per-source residual histograms against a high-precision reference and look for a depth-ratio peak tied to a single source.","tokens_in":34876,"feed_emoji":"📏","tokens_out":6468,"duration_ms":69084,"temperature":0.7,"pith_summary":"This paper sets out to prove that metric depth estimation—predicting real physical distances from images—can be scaled like other vision foundation tasks, even when the training data is noisy and heterogeneous. The authors assemble about 20 million image-depth pairs from reconstructed, captured, and rendered 3D sources spanning over 10,000 camera models, and pretrain a single model with randomly masked depth maps as sparse prompts. Their central claim is that this is the first clear scaling trend in the metric depth track: more data consistently improves zero-shot depth accuracy. The same pretrained model, after distillation into a prompt-free student, reports strong results on monocular depth, camera-intrinsics recovery, multi-view 3D reconstruction, and vision-language-action planning, and its encoder improves spatial reasoning in vision-language models. If true, the practical implication is that large-scale 'data soup' pretraining, rather than small hand-curated datasets, is a viable route to general metric perception.","feed_headline":"20M noisy depth pairs make metric depth pretraining scale","feed_subtitle":"Reconstructed, captured, and rendered depth data become one foundation for depth and 3D perception.","key_machinery":"Sparse Metric Prompt—a depth map randomly subsampled to 2,000-40,000 valid pixels per image (about 1% of pixels), then regularized through pixel-wise depth scale alignment and global metric depth recovery against a prior depth prediction so that all data sources share one input format. The prompt is injected through a lightweight conditioned dense-prediction head (about 5% extra parameters), leaving the vision-transformer backbone untouched. Pretraining uses a robust MAE loss that discards the top 20% largest-error pixels per image to tolerate sensor and reconstruction noise. The teacher is distilled into a prompt-free student with a distance-balanced inverse-depth loss, which keeps near-fie","core_discovery":"The discovery, on the paper's own terms, is that the heterogeneity and noise that kept metric depth data from scaling can be absorbed by a single universal interface: the Sparse Metric Prompt, a randomly masked depth map. With roughly 20M such image-prompt pairs, the pretrained model exhibits a monotonic improvement in zero-shot accuracy as data grows, and the trend carries through distillation to a prompt-free student. The authors take this as evidence that metric depth estimation obeys the same scaling behavior as relative depth and other vision foundation tasks, and that a single pretrained metric representation can serve depth completion, monocular depth, camera calibration, 3D reconstru","pith_inferences":["The paper does not audit per-source quality; a direct consequence of its argument is that source diversity, not just raw count, drives the scaling curve, and a per-source diagnostic would reveal whether any single reconstruction corpus dominates the gain.","Because the student is trained on teacher pseudo-labels from a unified model, the framework implies a self-improving loop—using student outputs as new pseudo-labels and retraining—which the authors do not test but is a natural next step.","The success with radar, an unseen and extremely sparse sensor, suggests the same interface may extend to other sparse modalities such as event cameras or time-of-flight arrays, a testable prediction.","The vision-language results imply the encoder stores recoverable metric scale; an untested but plausible extension is using it for egocentric distance estimation in embodied navigation, where fine-grained metric errors may matter more than in the benchmarks shown."],"forward_implications":["Data quality trade-offs shift: a large, noisy, diverse collection can outperform a small clean one for metric depth, so future work can mine existing 3D datasets instead of curating small ones.","The pretrained prompt-conditional model is usable out of the box for depth completion and super-resolution across different prompt densities and unseen sensors, including radar.","The distilled student inherits metric scale without prompts, enabling monocular metric depth, focal-length recovery, and monocular 3D point-map prediction in camera-unknown settings.","The same visual encoder can be reused as a frozen backbone for vision-language models and robot policies, implying metric 3D pretraining is a general spatial representation rather than a task-specific module.","Zero-shot behavior on unseen camera configurations, night and rainy conditions, and panoramic/fisheye imagery follows directly from learning heterogeneous sources instead of modeling any one sensor."],"fun_headline_variants":["Masked depth prompts turn 20M noisy pairs into scalable metric depth","A masked-depth prompt unifies noisy 3D data for metric depth scaling","Metric depth scaling arrives via random depth masks on 20M pairs","20M noisy 3D pairs: one masked prompt to rule metric depth pretraining","Sparse Metric Prompt: the key to scaling metric depth with noisy data"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The claim assumes every aggregated source can be converted to per-pixel metric depth from known calibration and that residual errors are pixel-level noise, because the training loss actively discards the largest-error pixels; if a major source carries a systematic scale or calibration bias, the learned metric scale is corrupted instead of cleaned.","fun_headline_variants_meta":{"raw":{"variants":["Masked depth prompts turn 20M noisy pairs into scalable metric depth","A masked-depth prompt unifies noisy 3D data for metric depth scaling","Metric depth scaling arrives via random depth masks on 20M pairs","20M noisy 3D pairs: one masked prompt to rule metric depth pretraining","Sparse Metric Prompt: the key to scaling metric depth with noisy data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000514,"raw_usage":{"total_tokens":2357,"prompt_tokens":795,"completion_tokens":1562,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":539,"completion_tokens_details":{"reasoning_tokens":1463}},"tokens_in":539,"tokens_out":1562,"duration_ms":10920,"temperature":1.0,"reasoning_tokens":1463,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T06:43:59.952216+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take one large reconstructed source, multiply all its depth values by a fixed 15% scale, retrain at full data scale, and measure zero-shot relative error on several held-out datasets. If the model's error is sharply worse, the robust loss is absorbing source-level bias rather than pixel noise, and the scaling claim would not hold for arbitrary heterogeneous sources. A lighter version: report per-source residual histograms against a high-precision reference and look for a depth-ratio peak tied to a single source.","supporting_citations":[],"review_version":1}