{"id":"7bdd4ec0-ebf5-4a79-bd90-488ef0e87f96","arxiv_id":"2412.00671","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"FiffDepth transforms a pre-trained diffusion image generator into a feed-forward monocular depth estimator that combines generative detail with DINOv2-based robustness.","lead":"FiffDepth turns a Stable Diffusion image generator into a fast, single-pass depth estimator, using a distilled DINOv2 signal to keep real-world depth robust. The model reports strong zero-shot accuracy and detail on common depth benchmarks while running in about 0.09 seconds per image.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The t=0/t=-1 feed-forward design is unvalidated: nothing shows these out-of-range timesteps are necessary, and the shared-weight filter could leak DAv2 blur into d0.","rationale":"The reader's weakest_assumption correctly identifies the unvalidated use of t=0 and t=-1, but the framing as 'outside the DDPM training range' is not itself fatal: these are trainable conditioning inputs, and the model is fine-tuned with direct losses at both timesteps. The load-bearing issue is sharper: the paper gives no evidence that these specific timesteps are necessary, that t=-1 acts as a low-pass filter, or that the shared-weight computation of d0 and d_{-1} does not cause the DAv2 distillation to degrade the detailed d0 output. The proposed test directly settles whether the extrapolated timesteps are required for the reported performance. Because the paper's benchmark results are plausible and consistent with the method as described, but the key mechanism is under-validated, the conditional verdict already issued remains appropriate.","tokens_in":12102,"tokens_out":10412,"duration_ms":107742,"concrete_test":"Retrain or minimally fine-tune the same pipeline with Eq. (1) evaluated at t=1 instead of t=0 and Eq. (4) evaluated at t=0 instead of t=-1, keeping all hyperparameters and data identical, and compare Table 1-2 metrics. If the in-range variant matches the paper's results within run-to-run variance, the extrapolated timesteps are not load-bearing; if it degrades materially, the t=0/t=-1 choices are empirically necessary. Additionally, compute the radial power spectrum of d_{-1} versus d0 on 100 held-out real images to verify the claimed high-frequency suppression.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on the premise that a U-Net trained for denoising at t=1..1000 can be evaluated at t=0 as a deterministic depth regressor and at t=-1 as a detail-suppressing filter. Eq. (1) and Eq. (4) are never compared against in-range alternatives, and the only support is the final benchmark tables. The paper asserts that d_{-1} removes high-frequency detail, but no spectral or correspondence analysis is reported. Because d0 and d_{-1} come from the same network, the loss on d_{-1} backpropagates through shared weights into the t=0 branch; the paper does not state whether d0 is detached when computing d_{-1}. Thus the claim that the filter transfers DAv2 robustness 'without interfering with the detailed features in d0' is unproven. If the filter is not actually low-pass, or if the t=-1 loss corrupts d0 through weight sharing, the main novelty fails even if the final numbers are good.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents FiffDepth, a monocular depth estimator obtained by fine-tuning a pre-trained Stable Diffusion U-Net so that it runs as a deterministic one-step feed-forward network. At inference, the model computes d0 = epsilon_theta(x0, t=0) from the image latent and decodes it with the SD VAE. Training preserves the original diffusion trajectory on a blended image-depth target (Eqs. (2)-(3)), adds depth losses on d0 from synthetic data, and applies a second evaluation d_-1 = epsilon_theta(d0, t=-1) supervised by Depth Anything v2-Large pseudo-labels on real images (Eqs. (4)-(7)). The paper reports state-of-the-art or near state-of-the-art zero-shot affine-invariant depth accuracy on NYUv2, KITTI, ETH3D, ScanNet, DIODE, and DA-2K, better boundary metrics than Depth Pro and DAv2 in Table 2, and faster inference than multi-step diffusion MDE methods.","tokens_in":12338,"tokens_out":9466,"duration_ms":84555,"significance":"If the results hold, FiffDepth is a useful practical contribution: it converts a text-to-image diffusion model into a single-step dense predictor with competitive accuracy and detail, uses far less training data than Depth Anything v2, and its deterministic inference is much cheaper than Marigold-style multi-step generative depth estimation. The boundary-metric results in Table 2 are independent evidence for the detail claim, and the release of source code is a further strength. However, the core mechanisms, namely extrapolating the denoiser to t=0 and t=-1, the learned-filter interpretation, and the generalization claim on DA-2K, are not yet rigorously validated, so the contribution is currently under-supported relative to the strength of the claims.","major_comments":[{"comment":"The central feed-forward design evaluates the denoiser at t=0 for d0 and at t=-1 for the filter, although the diffusion training objective Eq. (3) is defined only for t in {1,...,T}. The paper gives no evidence that the timestep embedding extrapolates sensibly to these values, and it does not compare against in-range alternatives such as t=1 for d0 or a conventional low-pass filter applied to d0 in place of Eq. (4). The claim that d_-1 is a learned low-pass filter rests on the qualitative Figure 3 and on benchmark numbers, which could be explained by other mechanisms. Please add a quantitative comparison that includes an in-range timestep and a non-learned filter baseline, and provide spectral or correspondence analysis of d_-1 if possible.","section":"Sections 3.1 and 3.3, Eqs. (1) and (4)"},{"comment":"The model is trained to regress Depth Anything v2-Large pseudo-labels at d_-1 (Eq. (4)), and DAv2-L itself scores 97.1 on DA-2K. Reporting the same 97.1 DA-2K accuracy is therefore not evidence of independent zero-shot generalization; it is partly a check of how faithfully the student imitates the teacher. Please either remove DA-2K from the generalization comparison, report it explicitly as a distillation-fidelity measure, or provide zero-shot evidence on benchmarks whose labels were not produced by DAv2-L and are not part of its training or evaluation distribution.","section":"Section 4.2, Table 1 (DA-2K column); Section 4.1"},{"comment":"Because d_-1 is obtained by applying the same network epsilon_theta to d0, gradients from the d_-1 losses can flow into d0 through the shared parameters unless a stop-gradient is inserted. The paper does not state whether d0 is detached, so the claim in Section 3.3 that the DAv2-L supervision transfers robustness without interfering with the detailed features in d0 is not supported. Please specify the gradient routing and include an ablation comparing training with and without detaching d0 in Eq. (4).","section":"Section 3.4, Eq. (7) with Eq. (4)"},{"comment":"The ablation evidence in Fig. 7 is qualitative only, even though the paper's main contributions are the trajectory-preserving training and the d_-1 filter distillation. Without quantitative metrics such as AbsRel, delta1, or boundary F1 for each removed component, the reader cannot determine which components drive the reported gains or whether Eq. (4) is necessary at all. Please add a quantitative ablation table covering w/o L_k, w/o blend, w/o DINO supervision, DINO supervision on d0, and replacement of Eq. (4) with a non-learned low-pass filter.","section":"Section 4.2, Ablation studies"}],"minor_comments":[{"comment":"Please specify the exact Stable Diffusion checkpoint and version, the VAE, whether the whole U-Net or a subset is fine-tuned, the optimizer, learning rate, batch size, resolution, number of iterations, and how the LAION-Art subset is sampled; this is needed to reproduce the 274K training-data claim.","section":"Section 4.1"},{"comment":"The roles of d and d* are swapped between the definition of the loss, where d is ground truth and d* is prediction, and the sentence after Eq. (7), where d*_0 is ground truth and d*_-1 is pseudo-label; please make the notation consistent.","section":"Equations (5)-(7)"},{"comment":"The teacher model is Depth Anything v2-Large, not DINOv2 itself; please use consistent terminology and clarify which component, DINOv2 features or DAv2 depth pseudo-labels, is actually being distilled.","section":"Sections 3.3 and 4.1"},{"comment":"The computation of F1 and boundary recall is only referenced to Depth Pro; please state the exact preprocessing, alignment, and threshold protocol in the paper or supplement.","section":"Table 2 and boundary-metric paragraph"},{"comment":"The running-time numbers should state the input resolution, GPU, and software versions used for each method, and whether all methods run at the same resolution with their default settings; otherwise the comparison is hard to interpret.","section":"Table 3"},{"comment":"Please fix typographical issues such as 'fintuned' in Section 3.3, 'performe' in the Table 3 caption, and clarify the 'blend' symbol in Figure 2, which is not defined in the caption.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The DA-2K row in Table 1 is likely to be seen as circular by reviewers and readers; I recommend requiring the authors to either re-label it or supply disjoint generalization evidence. The novelty over existing one-step diffusion-based depth predictors such as Lotus, GenPercept, and DepthFM should also be clarified. I recommend asking for quantitative ablations before acceptance, since the current qualitative Fig. 7 is not sufficient for the mechanistic claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"FiffDepth is a solid engineering contribution: it turns a pretrained Stable Diffusion model into a deterministic one-step depth predictor by evaluating the denoiser at t=0, preserves generative detail by blending image and depth latents along the diffusion trajectory, and uses a second 'filter' pass at t=-1 supervised by Depth Anything v2 pseudo-labels to transfer real-image robustness. The benchmark tables are competitive with DAv2-L, and the boundary metric results are genuinely stronger than Depth Pro on several datasets. The runtime improvement (0.092s vs 1.7s for LCM Marigold) is real.\n\nThe main soft spots are validation gaps rather than obvious errors. The t=0 and t=-1 evaluations are outside the DDPM training range (t=1..1000), and the paper gives no analysis of why the time embedding extrapolates well. More importantly, d0 and d_-1 share weights and d_-1 is computed from d0, so the loss on d_-1 can backpropagate into the t=0 branch. The paper asserts the filter transfers DAv2 robustness 'without interfering' with d0, but never tests this directly; a detach or a spectral analysis would settle it. The DA-2K accuracy of 97.1 exactly matches DAv2-L, which is unsurprising because DAv2-L is the teacher. That's not fatal, but the generalization claim should be qualified.\n\nThe biggest fixable issue is the ablation section: Figure 7 is qualitative only. A quantitative ablation table for the blend factor, trajectory loss, and filter would make the components credible. Error bars would also help, since the differences on some benchmarks are small.\n\nOverall, this is a plausible and useful paper. The central idea—using the diffusion model itself as a low-pass filter for distillation—is new and interesting, and the boundary results are independent evidence that the detail preservation actually works. I'd send it to review, with the expectation that the authors add ablations and address the weight-sharing question.","headline":"Solid engineering paper that turns Stable Diffusion into a fast one-step depth estimator; the core recipe is plausible but two unvalidated pieces (t=-1 filter, shared-weight interference) need attention.","tokens_in":12856,"tokens_out":2361,"would_cite":true,"duration_ms":21891,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FiffDepth claims that a pre-trained Stable Diffusion image generator, read at t=0 and filtered at t=-1, becomes a one-step depth estimator that matches or beats larger zero-shot models on detail and generalization.","keywords":["monocular depth estimation","diffusion priors","feed-forward depth estimation","Stable Diffusion","zero-shot generalization","DINOv2 distillation","depth boundaries","latent-space distillation"],"falsifier":"Run the released model with the timestep input clamped to the nearest in-range values and compare depth accuracy, detail, and boundary scores; if performance barely changes, the claimed $t=0$/$t=-1$ behavior is not what carries the result. Alternatively, compute the high-frequency energy of $d_0$ and $d_{-1}$ on unseen real images: if $d_{-1}$ is not actually a low-pass-filtered version of $d_0$, the learnable-filter distillation story fails.","tokens_in":11887,"feed_emoji":"📏","tokens_out":8399,"duration_ms":70765,"temperature":0.7,"pith_summary":"FiffDepth claims that a pre-trained Stable Diffusion image generator can be turned into a deterministic, one-step feed-forward depth estimator without losing the fine detail that makes generative models attractive. The key move is to evaluate the diffusion denoiser directly at time step $t=0$ for depth prediction and at $t=-1$ as a learnable filter, both outside its original training range, while keeping the original generative denoising trajectory alive during fine-tuning by blending image and depth latents. Because the $t=-1$ filter is supervised with pseudo-labels from a DINOv2-based model on unlabeled real images, the model learns DINOv2-style robustness on low-frequency depth while retaining generative high-frequency detail. If the claim holds, detailed and generalizable monocular depth can be obtained from roughly 274K training images in one fast inference pass.","feed_headline":"One t=0 forward pass turns diffusion into a depth estimator","feed_subtitle":"Reading the denoiser at t=0 and filtering at t=-1 yields detailed depth with DINOv2-style generalization.","key_machinery":"The central object is the pre-trained Stable Diffusion denoiser itself, reused as a feed-forward network at two nonstandard timesteps: $t=0$ gives the depth latent and $t=-1$ gives a learnable filter output matched to DINOv2 pseudo-labels. The trajectory-preserving objective diffuses a blended latent $b_0 = \\gamma x_0 + (1-\\gamma) d_0$ under the standard noise schedule and trains with v-prediction, so the model keeps generative image features while learning depth. The final objective combines latent-space MiDaS-style MAE and gradient-matching losses at $t=0$ (synthetic ground truth) and at $t=-1$ (DINOv2 pseudo-labels from real images) with the trajectory loss.","core_discovery":"On its own terms, the paper's discovery is that the denoising network of a latent diffusion model, evaluated at a single out-of-range timestep, is already a strong deterministic depth estimator, and that a second out-of-range evaluation acts as a learnable filter without adding a separate network. Equation (1) defines the depth latent as $d_0 = \\hat{\\epsilon}_\\theta(x_0, t=0)$, a single forward pass on the RGB image's latent, decoded by the Stable Diffusion VAE. Equation (4) defines $d_{-1} = \\hat{\\epsilon}_\\theta(d_0, t=-1)$, a filtered version that resembles DINOv2's low-frequency predictions; supervising only $d_{-1}$ with DINOv2 pseudo-labels transfers generalization to real images without erasing the details in $d_0$. The paper argues that this two-step scheme, together with the blended training target $b_0 = \\gamma x_0 + (1-\\gamma) d_0$ and a v-prediction trajectory loss, lets the model match or beat prior zero-shot depth estimators on NYUv2, KITTI, ScanNet, ETH3D, DIODE, and DA-2K, and outperform them on boundary accuracy.","pith_inferences":["A testable general principle suggested by the method: the timestep index of a diffusion denoiser can serve as a controllable detail dial, with out-of-range steps encoding coarser, more robust structure; the same $t=0$/$t=-1$ two-step scheme could transfer to surface normals, segmentation, or other dense tasks.","The blended target $\\gamma x_0 + (1-\\gamma) d_0$ interpolates between image and task latents, and the paper does not explore how varying $\\gamma$ changes the trade-off between generative fidelity and task-specific accuracy; a sweep would expose whether the choice $\\gamma=0.5$ is critical.","If the load-bearing extrapolation is correct, the feed-forward trick should work on other pre-trained latent diffusion models with minimal adaptation, giving a cheap way to probe which generative priors carry geometric structure."],"forward_implications":["Inference becomes a single deterministic forward pass, about 0.092 s per 512×512 image in the paper's timing table, roughly 10× faster than diffusion-sampling depth models.","Training uses only 74K synthetic images plus 0.2M unlabeled real images, versus about 62.6M images for the strongest comparison model, so cheap pseudo-labels may replace massive depth annotation.","Detail and generalization no longer trade off: the $t=0$ output keeps high-frequency structure while the $t=-1$ branch injects DINOv2-style low-frequency robustness on real images.","Boundary sharpness improves: the method reports higher F1 and boundary recall than prior state-of-the-art on Sintel, Spring, iBims, AM-2k, P3M, and DIS-5k.","The same diffusion backbone plays both predictor and filter, so the robustness transfer adds no extra network parameters."],"supporting_citations":[{"why":"supplies the pre-trained Stable Diffusion backbone and VAE that the method fine-tunes into a feed-forward depth network.","marker":"[35]"},{"why":"established the repurposing of diffusion generators for monocular depth and supplies the latent encoding and zero-shot evaluation protocol.","marker":"[19]"},{"why":"DINOv2 is the generalization source whose pseudo-labels supervise the t=-1 filter branch.","marker":"[29]"},{"why":"Depth Anything v2 is the pseudo-label teacher, the main benchmark comparison, and the source of the DA-2K evaluation set.","marker":"[50]"},{"why":"provides the MiDaS-style MAE and gradient-matching depth losses used in latent space and the zero-shot cross-dataset protocol.","marker":"[32]"},{"why":"supplies the boundary F1 and boundary recall metric and the Depth Pro baseline that FiffDepth compares against for sharpness.","marker":"[3]"},{"why":"one of the two synthetic datasets (Hypersim) providing dense ground truth depth for the t=0 supervision.","marker":"[34]"},{"why":"Virtual KITTI provides the other synthetic dataset for outdoor depth supervision during training.","marker":"[5]"}],"fun_headline_variants":["One t=0 forward pass: diffusion becomes depth estimator","Two out-of-range steps turn diffusion net into depth model","FiffDepth: feed-forward depth from a single denoiser call","Diffusion priors as deterministic depth: no extra network"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's load-bearing premise is that the fine-tuned denoiser remains well-behaved at timesteps $t=0$ and $t=-1$, which lie outside the $t \\in \\{1,\\ldots,1000\\}$ range it was trained on; the paper offers no separate evidence for this extrapolation beyond the final benchmark numbers.","fun_headline_variants_meta":{"raw":{"variants":["One t=0 forward pass: diffusion becomes depth estimator","Two out-of-range steps turn diffusion net into depth model","FiffDepth: feed-forward depth from a single denoiser call","Diffusion priors as deterministic depth: no extra network"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000272,"raw_usage":{"total_tokens":1658,"prompt_tokens":997,"completion_tokens":661,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":613,"completion_tokens_details":{"reasoning_tokens":591}},"tokens_in":613,"tokens_out":661,"duration_ms":6764,"temperature":1.0,"reasoning_tokens":591,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:07:31.924559+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the released model with the timestep input clamped to the nearest in-range values and compare depth accuracy, detail, and boundary scores; if performance barely changes, the claimed $t=0$/$t=-1$ behavior is not what carries the result. Alternatively, compute the high-frequency energy of $d_0$ and $d_{-1}$ on unseen real images: if $d_{-1}$ is not actually a low-pass-filtered version of $d_0$, the learnable-filter distillation story fails.","supporting_citations":[{"cited_title":"Repurpos- ing diffusion-based image generators for monocular depth estimation","cited_arxiv_id":null,"evidence_quote":"established the repurposing of diffusion generators for monocular depth and supplies the latent encoding and zero-shot evaluation protocol."},{"cited_title":"Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer","cited_arxiv_id":null,"evidence_quote":"provides the MiDaS-style MAE and gradient-matching depth losses used in latent space and the zero-shot cross-dataset protocol."},{"cited_title":"Hypersim: A photorealistic syn- thetic dataset for holistic indoor scene understanding","cited_arxiv_id":null,"evidence_quote":"one of the two synthetic datasets (Hypersim) providing dense ground truth depth for the t=0 supervision."}],"review_version":1}