{"id":"72796dc2-48e0-49dc-b4c1-0b4cf9835b93","arxiv_id":"2505.17358","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Two differently blurred images plus a pretrained diffusion depth prior are optimized together at inference time to recover metric depth without retraining.","lead":"The paper makes a pretrained relative-depth model, Marigold, output metric depth at inference time by using two photos with different apertures and a defocus-blur physics model. A smart generalist might read it because it shows a way to add physical scale information to learned depth estimators without retraining, using only camera metadata and two aligned images.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported advantage over baselines may reflect test-set-tuned apertures and dataset-informed scene bounds rather than the method's intrinsic accuracy; a held-out evaluation is needed.","rationale":"The reader's verdict is CONDITIONAL, and my analysis supports that rather than moving to accept or reject. The reader's stated weakest assumption is the physical forward model (PSF mismatch, occlusion, AIF assumptions). That is a legitimate concern, and the paper's own Fig. 2 and Limitations section acknowledge parts of it. However, the paper provides a meaningful ablation (Disc versus Gaussian PSF) and single-image ablation, which partially mitigates the forward-model risk. The more load-bearing weakness for the paper's strongest claim is the evaluation protocol: the aperture N=8 was selected after measuring performance on the same seven test scenes (Table 12), and the scene bounds used to parameterize alpha and beta were set from the real dataset. These choices can inflate the reported average advantage over baselines, and the per-scene tables show baselines winning on several scenes. A leave-one-scene-out evaluation with hyperparameters chosen only on training folds would provide a direct test of whether the method's advantage is genuine or an artifact of test-set tuning. This concern is addressable and does not invalidate the method's interesting premise, so the verdict remains CONDITIONAL.","tokens_in":18148,"tokens_out":14959,"duration_ms":125191,"concrete_test":"Run a leave-one-scene-out evaluation on the 7 collected scenes: for each held-out scene, choose the aperture N and scene bounds (smin, smax) using only the other 6 scenes (or a small separate validation capture), then evaluate the held-out scene with those choices. Report the mean and standard deviation of RMSE across folds against the same baselines. If the average gap over baselines narrows to within fold-to-fold variance, or if the method no longer wins on a majority of held-out scenes, the Table 1 claim of consistent superiority is not robust.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim is that the method outperforms MLPro, UniDepth, and Metric3D on average (Table 1). That claim rests on an evaluation protocol in which the most important hyperparameters were chosen using the same seven test scenes. Appendix G (Table 12) reports performance for N=4, 8, 11, 13, and 16 on the real dataset, and the method is then presented with N=8 because it is the best on those very scenes. This is test-set selection: the reported RMSE of 0.273 is the best of several aperture choices, not a fixed protocol. Similarly, the scene bounds smin=1.49 and smax=3.5 are described as a 'conservative upper bound on the potential maximum scale and offset in the real dataset,' meaning they were set from the evaluation dataset itself. Because alpha and beta are parameterized as bounded sigmoid functions of these values, the metric output is constrained by dataset statistics rather than by a priori scene knowledge. The per-scene tables also show that baselines win on several scenes (e.g., Metric3D on BOOKS and SHOERACK, UniDepth and Metric3D on THORDOG), so the average is driven by a small number of favorable scenes. With only 7 scenes and no significance testing, the claim of consistent superiority is not yet established. The forward-model mismatch is a real concern, but the paper includes a Gaussian-versus-Disc PSF ablation and a limitation discussion; the evaluation-protocol issue is the more load-bearing weakness for the headline comparison.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a training-free method that turns Marigold, a pretrained relative monocular depth diffusion model, into a metric depth estimator at inference time. Two raw images are captured at the same viewpoint: an all-in-focus image at a small aperture (F-stop 22) and a blurred image at a larger aperture (F-stop 8). The method optimizes the initial latent noise of Marigold-LCM together with affine scale and offset parameters (alpha and beta) by minimizing an L2 loss between the captured blurred image and a blurred image synthesized from the predicted metric depth through a disc-PSF defocus forward model. The authors collect a seven-scene dataset using a rigidly coupled DSLR and Intel RealSense camera, compare against MLPro, UniDepth, and Metric3D, and report lower average RMSE, REL, and log10 errors as well as higher delta accuracy metrics (Table 1). They also report ablations on the PSF model, aperture choice, alpha/beta initialization, noise-latent initialization, and the number of sampling steps.","tokens_in":18491,"tokens_out":4502,"duration_ms":33035,"significance":"The central idea, using physical defocus blur as an inference-time cue to convert a relative depth prior into metric depth, is original and potentially useful. The paper includes a physically grounded forward model, a clear ablation showing the disc PSF outperforms a Gaussian PSF, robustness results over ten noise-latent initializations with standard deviations (Table 10), and an analysis of aperture sensitivity. If the quantitative claim survives a properly protocoled evaluation, this would be a meaningful contribution to zero-shot metric depth estimation. The main weakness is that the headline numerical claim currently rests on an evaluation protocol in which key hyperparameters were selected on the same seven test scenes used for the final comparison.","major_comments":[{"comment":"The main result in Table 1 is obtained with the blurred image captured at F-stop N=8, but Appendix G, Table 12 reports performance for N=4, 8, 11, 13, and 16 on the same real test scenes, and the main text chooses N=8 because it is the best on those scenes. This is test-set selection: the reported RMSE of 0.273 is the best of several aperture choices rather than the result of a fixed protocol. To support the headline comparison, the aperture should be selected on held-out scenes or by a pre-registered criterion that does not use the evaluation scenes.","section":"Appendix G / Table 12"},{"comment":"The scene depth bounds smin=1.49 and smax=3.5 are described as 'a conservative upper bound on the potential maximum scale and offset in the real dataset,' meaning they were set using the evaluation dataset itself. Since alpha and beta are parameterized as sigmoid functions of smax and smin in Eq. (9), these dataset-derived bounds directly constrain the metric output. The paper should justify how these bounds are known a priori without access to the test scenes, and ideally include a sensitivity analysis showing that the results are stable over a range of smin and smax values.","section":"Appendix A / Eq. (9)"},{"comment":"The per-scene tables show that baselines outperform the proposed method on several scenes: for example, Metric3D has lower RMSE than the proposed method on BOOKS (0.263 vs. 0.312) and SHOERACK (0.209 vs. 0.251), and both UniDepth and Metric3D beat the proposed method on THORDOG (0.424 and 0.422 vs. 0.505). The average improvements in Table 1 are therefore driven by a small number of favorable scenes, such as PLANE and STAIRS. With only seven scenes, no error bars on Table 1, and no significance tests, the claim of consistent superiority is not established. The authors should report per-scene confidence intervals and a paired significance test, or substantially temper the claim.","section":"Appendix D / Tables 3-9"},{"comment":"The optimization signal for alpha, beta, and the latent depends on the accuracy of the defocus forward model, which assumes a depth-dependent shift-invariant disc PSF, neglects occlusion, and relies on the focus distance F being read manually from the lens's analog scale. The paper acknowledges occlusion-boundary errors in the Limitations section, but it does not quantify the sensitivity of the metric result to errors in F or to PSF falloff parameters. A sensitivity analysis for F and for the PSF shape would help establish that the recovered metric scale is not biased by forward-model mismatch.","section":"Eq. (6)-(7) / Limitations"}],"minor_comments":[{"comment":"The constraint ||z_T||_2 = sqrt(M) appears in Eq. (11), and Section 3 mentions rescaling to this norm, but the optimization procedure in Section 4 does not state how often or in what form this projection is applied during the 200 iterations; this should be specified for reproducibility.","section":"Eq. (11) / Section 3"},{"comment":"The text says 'we use the pyrealsense2library' without a space, and it would be clearer to say 'the pyrealsense2 library.'","section":"Appendix C"},{"comment":"The caption 'Optimizing with the noise latent with defocus cues (ours) improves upon optimizing only the affine parameters' should be rephrased for clarity, for example 'Optimizing the noise latent jointly with the affine parameters improves upon optimizing only the affine parameters.'","section":"Table 2 caption"},{"comment":"The caption contains a sentence fragment beginning 'recovers sharp details but fails at metric scale and relative depth accuracy for many of the scenes'; the subject of this sentence is missing and should be supplied.","section":"Fig. 4 caption"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is worth your time: take Marigold's relative depth latent, couple it to a thin-lens defocus forward model, and optimize scale, offset, and noise latent against the blur in a second aperture-diverse image. That is a genuinely new combination, and the paper shows it can fix texture-depth coupling and recover metric scale on toy and real scenes. The forward model is cleanly derived, the disc PSF with linear falloff is sensible, and they use raw images plus exposure compensation, which is the right way to handle the radiometric side. The ablations are honest: Gaussian vs disc PSF, noise-latent robustness, initialization, sampling steps, and a clear acknowledgment of occlusion-boundary and slow-inference limitations.\n\nThe main quantitative claim, however, rests on a protocol that doesn't hold up. The aperture N=8 was chosen after running the method at N=4,8,11,13,16 on the same seven real scenes (Appendix G, Table 12) and picking the best. That's test-set selection. The reported RMSE of 0.273 is the best of several runs, not a fixed protocol. The scene bounds smin=1.49 and smax=3.5 are also set from the real dataset, and since alpha and beta are bounded by those values, the metric output is constrained by dataset statistics rather than a priori knowledge. Additionally, baselines win on several individual scenes (Metric3D on BOOKS and SHOERACK, UniDepth and Metric3D on THORDOG), so the average is driven by a small number of favorable scenes. There are no error bars or significance tests on the headline table, even though the paper reports noise-latent robustness for two scenes. The forward-model mismatch is real but is handled reasonably; the evaluation protocol is the more load-bearing weakness.\n\nThis is not a fatal flaw. The idea is strong, the derivation is careful, and the dataset is provided (though no method code is released). A revision with a held-out dataset or fixed hyperparameters chosen without test-set peeking, per-scene confidence intervals, and code would make the central claim credible. I'd send it to peer review with that request. It's a good reading-group case study in how promising methods can overstate gains when the test set leaks into hyperparameter choice.","headline":"Clever training-free idea that deserves a serious referee, but the headline comparison is built on hyperparameters tuned on the same seven test scenes, so the stated edge over baselines isn't established yet.","tokens_in":19003,"tokens_out":3207,"would_cite":true,"duration_ms":34374,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Defocus blur cues, injected at inference time, turn a pre-trained relative-depth diffusion model into a metric-depth estimator without retraining.","keywords":["monocular metric depth estimation","defocus blur","depth from defocus","diffusion-based depth prior","inference-time optimization","circle of confusion","zero-shot generalization","latent consistency model"],"falsifier":"Capture a new indoor scene containing a strong occlusion boundary and a textured foreground, run the two-aperture optimization, and compare against the RealSense ground truth at the boundary; if the predicted depth is systematically wrong exactly where the depth changes abruptly while the blur-rendering loss stays small, the shift-invariant no-occlusion forward model is fitting a wrong explanation. A second check is to use a lens with strong spherical aberration or a non-circular aperture: the paper's own Fig. 2 predicts model mismatch, so a larger-than-expected error drop versus the Gaussian-PSF baseline would confirm the disc-PSF assumption is load-bearing.","tokens_in":17988,"feed_emoji":"📷","tokens_out":7859,"duration_ms":60935,"temperature":0.7,"pith_summary":"The paper tries to establish that a pre-trained relative monocular depth diffusion model, Marigold, can be made to output metric depth at inference time, with no retraining, by feeding it two images of the same scene: one all-in-focus small-aperture image and one blurred large-aperture image. The metric scale and offset, together with Marigold's initial noise latent, are optimized so that the predicted depth, when run through a defocus-blur image formation model, reproduces the observed blurred image. If correct, this would give zero-shot metric depth estimation a physics-based calibration signal that is independent of training-data scale and camera intrinsics, and would attack two known failure modes of learned depth models: wrong absolute scale and texture-depth coupling. On a self-collected seven-scene DSLR and RealSense dataset, the method reports lower average error than MLPro, UniDepth, and Metric3D.","feed_headline":"Two aperture shots give AI depth maps real-world scale","feed_subtitle":"A physics-based blur loss fixes scale at test time and beats three zero-shot baselines on seven scenes.","key_machinery":"The load-bearing object is the defocus-blur forward model, $\\hat{\\mathbf{x}}_b = g(\\mathbf{x}, d_m, f, F, N)$, which forms a blurred image as a spatially varying convolution between the all-in-focus image and a depth-dependent disc PSF with a smooth linear edge; the PSF radius is the circle of confusion $c(d) = \\frac{f^2}{N}\\frac{|d-F|}{d(F-f)s}$, with focal length $f$, focus distance $F$, F-stop $N$, and pixel size $s$ taken from camera metadata. The second ingredient is Marigold-LCM, a single-step latent consistency version of Marigold (a Stable Diffusion-based model trained for monocular relative depth), which acts as a differentiable map from a Gaussian noise latent $z_T^{(d)}$ and the AIF image to a depth map. The paper backpropagates the blur-rendering loss through that map into $z_T^{(d)}$ and through the affine parameters $\\alpha = s_{\\max}\\sigma(a)$, $\\beta = s_{\\min}\\sigma(b)$, rescaling $z_T^{(d)}$ to the Gaussian annulus norm so it stays on the prior manifold. The disc PSF with linear falloff, rather than a Gaussian approximation, is what keeps the gradient signal physically consistent with the real lens.","core_discovery":"At the center of the claim is an affine lift from relative to metric depth, $d_m = \\alpha \\cdot d + \\beta$, whose two parameters are learned per scene at inference time. The paper's discovery is that defocus blur is a sufficient physical signal to determine those parameters and to refine the relative depth map itself, because the blurred image can be synthesized, via a disc point-spread function whose radius is the circle of confusion, from the all-in-focus image and any candidate metric depth. Minimizing the $\\ell^2$ distance between the synthesized and the actually captured blurred image, with respect to $\\alpha$, $\\beta$, and Marigold's initial noise latent $z_T^{(d)}$, yields metric depth that is correctly scaled and less texture-coupled. On the authors' seven-scene real dataset, this yields an average RMSE of 0.273 m and a $\\delta_1$ accuracy of 0.879, compared with RMSE 0.459-0.644 and $\\delta_1$ 0.259-0.650 for the learned zero-shot baselines.","pith_inferences":["Going beyond the paper, the same inference-time scheme could be adapted to dual-pixel sensors, which encode a similar depth-dependent blur without needing two physical apertures, potentially bringing the physics-based scale correction to ordinary phone cameras.","An untested but natural extension is to jointly estimate the all-in-focus image and the depth map from a single blurred capture, removing the small-aperture requirement and making the method usable in light-limited settings.","The physics-plus-prior recipe is not limited to visible-light photography: any depth-dependent blur model, such as those in endoscopy, microscopy, or hyperspectral imaging, could be plugged into the same latent-noise optimization, and the authors point to these applications as future directions.","A testable prediction of the paper is that on scenes with severe occlusions or reflective and transparent surfaces, the optimization will fit the blur model by distorting depth in the wrong places; measuring per-pixel error against ground truth in those regions would quantify the cost of the no-occlusion assumption."],"forward_implications":["A relative-only depth foundation model can be converted into a metric estimator at test time with only two aperture settings and known camera parameters, requiring no training data and no camera calibration.","Defocus cues do double duty: they fix the global metric scale and offset, and they refine the relative depth map itself, since ablating the noise-latent optimization degrades all reported metrics.","The choice of PSF model matters: with a Gaussian PSF the method's average RMSE rises to 0.528, while the disc PSF gives 0.273, so physical fidelity of the forward model is a primary driver of accuracy.","The best blur strength is an intermediate F-stop (f/8 on the real data, f/13 in simulation); too little or too much blur makes the inverse problem ill-posed.","The approach is zero-shot in the sense that no weights are retrained, but it is not real-time: optimization takes roughly 3.5-4 minutes per scene on an A-40 GPU."],"supporting_citations":[{"why":"Supplies the pre-trained diffusion depth prior, Marigold, that the method optimizes at inference time.","marker":"[26]"},{"why":"Establishes the inference-time noise-optimization recipe of updating an initial latent with gradients from an auxiliary loss.","marker":"[37]"},{"why":"Shows that a distilled single-step diffusion model can be optimized this way, motivating the Marigold-LCM choice.","marker":"[36]"},{"why":"Provides the thin-lens circle-of-confusion camera model that defines the PSF radius.","marker":"[40]"},{"why":"Supplies the differentiable disc-PSF rendering formulation and CUDA implementation that the forward model builds on.","marker":"[60]"},{"why":"Provides the Gaussian-PSF depth-from-defocus baseline that the paper compares against and shows to be inferior.","marker":"[20]"},{"why":"Prior test-time optimization of Marigold with sparse metric depth, which the method extends by using defocus cues instead of depth input.","marker":"[58]"},{"why":"Baseline zero-shot metric depth model (MLPro / Depth Pro) used for quantitative comparison on the collected dataset.","marker":"[7]"},{"why":"Baseline zero-shot metric depth model (UniDepth) used for quantitative comparison.","marker":"[39]"},{"why":"Baseline zero-shot metric depth model (Metric3D) used for quantitative comparison.","marker":"[72]"}],"fun_headline_variants":["Defocus blur unlocks metric scale for Marigold depth","Two-aperture trick makes Marigold depth metric without training","Physics of blur calibrates depth from a diffusion model","Blur cue gives Marigold real-world depth scale at test time","Aperture pair turns zero-shot depth into metric depth"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The optimization relies on the photographed blur being well described by the thin-lens circle-of-confusion model: a depth-dependent, shift-invariant disc PSF, no occlusion, a truly all-in-focus small-aperture image, and radiometrically linear, energy-balanced raw captures; if the real lens's PSF, focus readout, occlusion, or scene radiance violate these assumptions, the gradient signal can be biased.","fun_headline_variants_meta":{"raw":{"variants":["Defocus blur unlocks metric scale for Marigold depth","Two-aperture trick makes Marigold depth metric without training","Physics of blur calibrates depth from a diffusion model","Blur cue gives Marigold real-world depth scale at test time","Aperture pair turns zero-shot depth into metric depth"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000562,"raw_usage":{"total_tokens":2662,"prompt_tokens":932,"completion_tokens":1730,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":548,"completion_tokens_details":{"reasoning_tokens":1645}},"tokens_in":548,"tokens_out":1730,"duration_ms":10657,"temperature":1.0,"reasoning_tokens":1645,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:49:58.174353+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Capture a new indoor scene containing a strong occlusion boundary and a textured foreground, run the two-aperture optimization, and compare against the RealSense ground truth at the boundary; if the predicted depth is systematically wrong exactly where the depth changes abruptly while the blur-rendering loss stays small, the shift-invariant no-occlusion forward model is fitting a wrong explanation. A second check is to use a lens with strong spherical aberration or a non-circular aperture: the paper's own Fig. 2 predicts model mismatch, so a larger-than-expected error drop versus the Gaussian-PSF baseline would confirm the disc-PSF assumption is load-bearing.","supporting_citations":[{"cited_title":"Repurposing diffusion-based image generators for monocular depth estimation","cited_arxiv_id":null,"evidence_quote":"Supplies the pre-trained diffusion depth prior, Marigold, that the method optimizes at inference time."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the inference-time noise-optimization recipe of updating an initial latent with gradients from an auxiliary loss."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows that a distilled single-step diffusion model can be optimized this way, motivating the Marigold-LCM choice."},{"cited_title":"A lens and aperture camera model for synthetic image generation","cited_arxiv_id":null,"evidence_quote":"Provides the thin-lens circle-of-confusion camera model that defines the PSF radius."},{"cited_title":"An implicit neural representation for the image stack: Depth, all in focus, and high dynamic range","cited_arxiv_id":null,"evidence_quote":"Supplies the differentiable disc-PSF rendering formulation and CUDA implementation that the forward model builds on."},{"cited_title":"Single image depth estimation trained via depth from defocus cues","cited_arxiv_id":null,"evidence_quote":"Provides the Gaussian-PSF depth-from-defocus baseline that the paper compares against and shows to be inferior."}],"review_version":1}