{"id":"9a7cc0e6-5075-49bb-bee1-a350050b1b47","arxiv_id":"2411.10886","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"MetricGold fine-tunes Stable Diffusion's latent space on log-scaled synthetic depth to predict metric depth from single images, claiming efficient zero-shot generalization without reported benchmarks.","lead":"MetricGold adapts Stable Diffusion, a text-to-image model, to predict metric depth from a single photo by fine-tuning only the denoising network on synthetic indoor and outdoor images. The paper claims sharper zero-shot depth estimates than prior diffusion-based depth models, but reports no quantitative experiments to back that claim.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (3) log-depth normalization lacks any stated inverse or source for dmin/dmax, so the metric-scale recovery that defines MetricGold is undefined and the zero-shot metric depth claim is unsupported.","rationale":"I read the paper as a methods-only manuscript whose central empirical claim is asserted but not demonstrated. The strongest technical flaw is the unspecified log-depth normalization and its inverse, which directly undermines the metric-depth claim. This aligns with the reader's weakest_assumption. The absence of experimental evaluation is a separate confirmatory reason for rejection. A concrete test on the released code and a metric-depth dataset would settle whether the method works; until then the central claim is unsupported.","tokens_in":103,"tokens_out":2657,"duration_ms":39077,"concrete_test":"Inspect the released repository (github.com/AnshShah3009/MetricGold) and checkpoint to find how dmin and dmax are computed at inference and whether the inverse of Eq. (3) is applied to the decoded depth. Then run the provided inference script on the KITTI or NYUv2 test set with known metric ground truth and compute absolute relative error and mean absolute error. If the predicted depth is not in metric units, or if per-image ground-truth bounds are required at inference, the central claim is falsified. If global constants are used, reproduce the inverse transform analytically and confirm it exactly recovers dr from dlog for representative values.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract, §1, §5) is that MetricGold produces metric depth estimates that generalize zero-shot to unseen real datasets. The only description of metric scaling is Eq. (3), dlog = normalize(log(dr/dmin)/log(dmax/dmin)). The paper never states whether dmin/dmax are per-image ground-truth bounds, dataset-global constants, or learned values, and it never specifies the inverse transform applied to the decoder output to recover dr. If per-image bounds are used, the network output cannot be rescaled to metric units at inference without knowing the answer. If global constants are used, they must be reported and the inverse must be exact. This omission is load-bearing: relative depth can be produced without metric scale, but the entire contribution relative to Marigold is metric scale recovery. In addition, the abstract claims experiments 'demonstrate robust generalization' yet the paper contains no experimental section, no tables, and no quantitative error metrics, so the claim is also empirically unverified. The technical underspecification alone makes the method irreproducible and untestable as written.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MetricGold, a fine-tuning protocol that repurposes Stable Diffusion v2's latent diffusion model for monocular metric depth estimation. The method replaces the Stable Diffusion VAE with a depth VAE trained to reconstruct log-normalized metric depth from synthetic RGB-D data (Hypersim, Virtual KITTI, and TartanAir are named), concatenates image and depth latents as conditioning for the denoising U-Net, and fine-tunes the U-Net with a standard diffusion objective. The authors claim that this protocol trains efficiently on a single RTX 3090 in about two days and yields zero-shot generalization to unseen real datasets with sharper and more accurate metric depth estimates than existing approaches. The manuscript contains no quantitative evaluation, no benchmark tables, and no explicit inverse for the log-depth normalization.","tokens_in":5895,"tokens_out":3268,"duration_ms":35810,"significance":"If the central claims were substantiated, MetricGold would be a useful demonstration that a consumer-grade fine-tuning recipe can convert a text-to-image diffusion model into a zero-shot metric depth estimator using only synthetic data. The paper has some genuine strengths: it releases code, reports a concrete training budget, builds on external baselines rather than circular self-citation, and identifies a plausible design space (latent-space diffusion, log-scaled depth, synthetic-only training). However, the absence of any experimental validation makes the significance currently unverified; the claimed advantages over Marigold, DMD, and Depth Anything V2 are not established, and the metric-scale recovery mechanism is not fully specified.","major_comments":[{"comment":"The paper's central claim of \"robust generalization\" and \"sharper and accurate metric depth estimates compared to existing approaches\" is not supported by any quantitative evaluation. There is no experiments section, no benchmark tables, no error metrics (such as AbsRel, RMSE, or delta thresholds), no comparisons to Marigold, DMD, Depth Anything V2, or any other baseline, and no qualitative result figures beyond the teaser. The conclusion in Section 5 repeats the empirical claim without presenting evidence. This is load-bearing because the abstract explicitly promises experimental demonstration, and it makes the paper's central contribution untestable as written.","section":"Abstract; Section 5"},{"comment":"The log-depth normalization is underspecified in a way that affects the core claim of metric depth recovery. Eq. (3) defines dlog = normalize(log(dr/dmin)/log(dmax/dmin)), but the paper does not state whether dmin and dmax are per-image ground-truth bounds, dataset-global constants, or learned values, and it does not give the inverse transform used to recover dr from the network output at inference. If per-image bounds are used, the output cannot be rescaled to metric units at inference without knowing the answer in advance; if global constants are used, they must be reported and the inverse must be exact. Since metric scale recovery is the distinguishing contribution relative to relative-depth models such as Marigold, this omission makes the method irreproducible and the zero-shot metric-depth claim unsupported.","section":"Section 3.2, Eq. (3)"},{"comment":"There is an unresolved mismatch between the diffusion formulation and the reported training objective. Section 3.1 presents a standard noise-prediction objective, while Section 4 states that the model is trained with v-parameterization, which uses a different target. The paper does not reconcile these two descriptions or state the actual training loss used. This is a secondary but still load-bearing reproducibility issue because the reader cannot determine what objective was optimized or which inference procedure corresponds to the reported claims.","section":"Section 3.1; Section 4"}],"minor_comments":[{"comment":"There are typographical errors: \"doamins\" should be \"domains\" and \"WIth minimal changes\" should be \"With minimal changes.\"","section":"Section 3.2"},{"comment":"The name \"MariGold\" in the abstract and Section 1 should be \"Marigold\" to match the cited work [6].","section":"Abstract; Section 1"},{"comment":"The training data are described inconsistently: the abstract lists HyperSIM, VirtualKitti, and TartanAir, while Section 2.2 names only Hypersim and Virtual KITTI 2, and Section 4 gives no dataset composition. Please clarify which datasets were actually used and in what proportions.","section":"Abstract; Section 2.2; Section 4"},{"comment":"The VAE loss in Eq. (4) is written as L_VAE = E_{q(d|x)}[log p(log dlog | d_latent)] - D_KL(q(d_latent|x) || p(d_latent)), but the notation is confusing: dlog is described as a deterministic input to the encoder, not as a random variable conditioned on x. Please clarify the generative model and the exact reconstruction loss used for the depth VAE.","section":"Section 3.2, Eq. (4)"},{"comment":"Reference [4] has a malformed author list (\"Rare s, Ambrus, , and Adrien Gaidon\"), and the paper uses \"DMD\" without spelling out the method name or giving a dedicated reference; [15] appears to be the DMD paper but should be cited explicitly at first use.","section":"References"}],"recommendation":"reject","confidential_remarks":"The manuscript reads more like a technical report than a complete research paper: it proposes a sensible fine-tuning recipe but omits the experimental validation that would be required to support its central claims. The two main issues, missing evaluation and the underspecified metric-scale inverse, are not local presentation problems; they require substantial additional work (full benchmarks, ablations, and a precise specification of the normalization/inverse). The code release and training budget are useful, and a resubmission with proper evaluation would be a different, potentially publishable paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: MetricGold is a plausible fine-tuning recipe — adapt Stable Diffusion's latent space to output log-scaled metric depth, train on synthetic RGB-D only — but the preprint currently contains no experimental evidence. The central claim of zero-shot metric generalization is asserted, not shown. The stress-test concern is right: Eq. (3) defines dlog without ever saying where dmin/dmax come from or how the decoder output is inverted back to metric units. Without that, the method isn't reproducible, and the \"metric\" part of the contribution is literally undefined.\n\nWhat is actually new: the combination of latent-space diffusion (following Marigold) with DMD's metric-depth objective and Depth Anything V2's synthetic-only training is a reasonable next step, and the authors correctly identify DMD as the closest prior work. The training details — concatenating image and depth latents, doubling the U-Net input weights and halving them, fine-tuning a Depth VAE on log-normalized depth — are described concretely enough that someone could try to reproduce the pipeline. Training time on one RTX 3090 is a nice practical claim, and two days is plausible.\n\nThe soft spots are not minor. There is no experiments section: no tables, no error bars, no comparisons. The abstract and conclusion say the method produces \"sharper and accurate metric depth estimates compared to existing approaches\" but no measured support appears anywhere in the manuscript. This is the load-bearing claim, and it is unverified. Second, the normalization/inverse gap is technical and load-bearing: if dmin/dmax are per-image ground-truth bounds, inference requires knowing the answer; if they are dataset constants, they need to be stated, and the inverse transform must be exact. The paper reports neither. The linked code might resolve some of this, but I did not audit it, and the manuscript alone does not stand.\n\nI'd rather not cite this in its current form, and I would not send it to peer review as-is. It could become a short technical report with real numbers and a clear definition of the inverse transform. As it stands, a serious referee would bounce it for lack of evidence, and I would too. If you are working on diffusion-based depth, the recipe is worth a skim, but do not treat the claims as established.","headline":"A plausible latent-diffusion fine-tuning recipe for metric depth with synthetic-only data, but the preprint contains no experiments and the metric-scaling inverse is undefined.","tokens_in":6395,"tokens_out":2692,"would_cite":false,"duration_ms":25612,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MetricGold claims that a pretrained text-to-image latent diffusion model can be fine-tuned on photorealistic synthetic RGB-D data—in about two days on a single consumer GPU—to produce metric depth maps that zero-shot transfer to real…","keywords":["metric depth estimation","monocular depth estimation","latent diffusion models","Stable Diffusion fine-tuning","zero-shot generalization","synthetic RGB-D data","log-scaled depth representation"],"falsifier":"Take a real image with known depth, such as a LiDAR-registered KITTI frame or an indoor RGB-D scan, run the published MetricGold checkpoint, and compare its output in metres with the ground truth using absolute relative error. If the model's predictions are only correct up to an unknown per-image scaling that must be computed from the ground truth min/max depth, or if cropping the image changes the predicted scale, the metric-recovery claim fails. This experiment also discloses whether the inverse of Eq. (3) exists as a deployable transform.","tokens_in":5473,"feed_emoji":"📏","tokens_out":7979,"duration_ms":75429,"temperature":0.7,"pith_summary":"The paper sets out to show that a text-to-image generative model can be converted into a monocular metric depth estimator, a system that predicts a distance in metres for every pixel of a single photograph. MetricGold starts from Stable Diffusion, fine-tunes its variational autoencoder on log-normalized metric depth, and fine-tunes its denoising U-Net to translate an RGB image into a depth latent, using only photorealistic synthetic RGB-D data. The authors report that this protocol trains in about two days on a single consumer GPU and, without ever seeing real depth maps, zero-shot generalizes to real datasets, producing sharper and accurate metric depth estimates than existing approaches. The payoff, if correct, is that the visual knowledge inside a pretrained image generator can substitute for massive collections of real depth measurements.","feed_headline":"Text-to-image diffusion model becomes a metric depth estimator","feed_subtitle":"MetricGold fine-tunes Stable Diffusion on synthetic depth and generalizes to real RGB images unseen during training.","key_machinery":"The load-bearing mechanism is a fine-tuned latent diffusion pipeline. Stable Diffusion's VAE is re-trained as a Depth VAE to reconstruct log-normalized metric depth maps, while the denoising U-Net is fine-tuned with the RGB latent concatenated to the noisy depth latent, the first-layer weights duplicated and halved so the pretrained structure is preserved. The log-scaled depth representation, $d_\\text{log} = \\operatorname{normalize}\\bigl(\\log(d_r/d_{\\min}) / \\log(d_{\\max}/d_{\\min})\\bigr)$, spreads the depth distribution so that indoor scenes around a few metres and outdoor scenes up to tens of metres share one model; the paper identifies latent depth reconstruction as the main performance bottleneck.","core_discovery":"On its own terms, the paper's central claim is that latent diffusion priors from a text-to-image model are sufficient for monocular metric depth estimation. Depth prediction is reframed as conditional denoising in a compact latent space: an RGB image and a log-normalized metric depth map are each encoded with a variational autoencoder, the two latents are concatenated along the channel dimension, and a fine-tuned U-Net learns to remove noise from corrupted depth latents conditioned on the image. The only metric-depth supervision comes from photorealistic synthetic datasets, Hypersim for indoor scenes and Virtual KITTI 2 for outdoor driving. MetricGold then transfers to real datasets without retraining and outputs depth in metric units, rather than relative or affine-invariant depth, which is the qualitative step beyond prior diffusion-based depth estimators.","pith_inferences":["The paper leaves the inverse of Eq. (3) unspecified; a concrete next step is to report the exact $d_{\\min}$, $d_{\\max}$, and un-normalization used at inference, because without that the metric claim is not independently checkable.","If the recipe generalizes, it suggests a broader pattern: any strong latent diffusion model can be repurposed for dense visual prediction by fine-tuning only the output VAE and the U-Net's input layer, which could apply to surface normals, segmentation, or optical flow.","Because the metric scale is learned entirely from synthetic renderers, its accuracy on camera systems with unusual intrinsics or extreme depth ranges, such as fisheye or long-range LiDAR-only setups, is an untested boundary that would show where the diffusion prior stops carrying the scale."],"forward_implications":["A consumer-grade fine-tuning recipe—one RTX 3090 and roughly two days—can turn a pretrained text-to-image diffusion model into a zero-shot metric depth estimator, making such models practical for academic labs.","Training exclusively on photorealistic synthetic data avoids sensor-specific noise and bias that real RGB-D datasets carry, which the paper argues yields sharper depth boundaries.","Since MetricGold operates in latent space with v-parameterization and 50-step DDIM sampling, the same model is a candidate for consistency distillation, bringing inference to a handful of steps."],"supporting_citations":[{"why":"Supplies the pretrained Stable Diffusion VAE and U-Net that MetricGold fine-tunes into a depth estimator.","marker":"[13]"},{"why":"Establishes the prior result that diffusion image generators can be repurposed for monocular depth; MetricGold extends it from relative to metric depth.","marker":"[6]"},{"why":"Closest predecessor, diffusion-based metric depth estimation, which MetricGold makes more compute-efficient by denoising in latent space.","marker":"[15]"},{"why":"Provides the synthetic-only training argument and the photorealistic data strategy MetricGold adopts.","marker":"[17]"},{"why":"Contributes the indoor synthetic RGB-D training data, Hypersim.","marker":"[12]"},{"why":"Contributes the outdoor synthetic driving data, Virtual KITTI 2, used for training.","marker":"[2]"}],"fun_headline_variants":["Diffusion priors give metric depth from a single image","Synthetic data enables metric depth via diffusion","Fine-tuned diffusion model predicts metric depth","Latent diffusion turns RGB into metric depth"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that Eq. (3)'s log-normalized depth can be inverted to recover true metres on arbitrary real images, but the paper does not state whether the smallest and largest depth values, $d_{\\min}$ and $d_{\\max}$, are per-image ground-truth bounds, dataset-global constants, or learned values, nor what inverse transform is applied at test time.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion priors give metric depth from a single image","Synthetic data enables metric depth via diffusion","Fine-tuned diffusion model predicts metric depth","Latent diffusion turns RGB into metric depth"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000355,"raw_usage":{"total_tokens":1884,"prompt_tokens":859,"completion_tokens":1025,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":475,"completion_tokens_details":{"reasoning_tokens":967}},"tokens_in":475,"tokens_out":1025,"duration_ms":8229,"temperature":1.0,"reasoning_tokens":967,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T19:11:17.487551+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a real image with known depth, such as a LiDAR-registered KITTI frame or an indoor RGB-D scan, run the published MetricGold checkpoint, and compare its output in metres with the ground truth using absolute relative error. If the model's predictions are only correct up to an unknown per-image scaling that must be computed from the ground truth min/max depth, or if cropping the image changes the predicted scale, the metric-recovery claim fails. This experiment also discloses whether the inverse of Eq. (3) exists as a deployable transform.","supporting_citations":[{"cited_title":"Repurposing diffusion-based image generators for monocular depth estimation, 2024","cited_arxiv_id":null,"evidence_quote":"Establishes the prior result that diffusion image generators can be repurposed for monocular depth; MetricGold extends it from relative to metric depth."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Closest predecessor, diffusion-based metric depth estimation, which MetricGold makes more compute-efficient by denoising in latent space."}],"review_version":1}