REVIEW 3 major objections 4 minor 82 references
Monocular depth models can inherit multi-view scale accuracy when learnable Rectified Stereo Tokens stand in for the missing second view, keeping epipolar attention alive during single-image inference.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 22:48 UTC pith:O3IKY7ZQ
load-bearing objection Solid, well-written method paper that deserves a real referee, but the claimed geometric mechanism is under-evidenced and the headline DIODE gain rides on an anomalous baseline. the 3 major comments →
Geometric Distillation from Rectified Stereo: Leveraging Epipolar Cues for Monocular Depth
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that the 'architectural collapse' of multi-view transformers under single-view input—cross-view global attention degenerating into intra-frame self-attention and losing metric scale priors—can be prevented by a set of learnable Rectified Stereo Tokens (Trst). These tokens cross-attend to the reference image to form reference-guided tokens, which then participate in epipolar attention constrained strictly along horizontal scanlines, simulating a rectified stereo pair with an implicit virtual baseline. During training, a Depth-Guided Epipolar Attention uses ground-truth depth to compute the true correspondence and adds a Gaussian bias to the attention logits, producing a s
What carries the argument
The load-bearing mechanism is the pair formed by (1) Rectified Stereo Tokens, a learnable spatial token grid initialized at a fixed resolution and interpolated to feature resolution, which acts as a virtual source view and keeps cross-attention geometric rather than collapsing to self-attention; and (2) Depth-Guided Epipolar Attention, which during multi-view training biases cross-attention with a Gaussian centered on the ground-truth 3D correspondence (from depth plus relative pose), and whose predicted attention map is supervised by cross-entropy (L_attn). A token-level distillation loss (L_distill) aligns single-view features with multi-view epipolar-aggregated features. The rectified-ste
Load-bearing premise
The load-bearing premise is that the learned Rectified Stereo Tokens genuinely act as a geometric surrogate for a real second view, so the cross-attention peaks encode depth/disparity; if the tokens merely memorize dataset patterns, the performance gains come from extra parameters and training data, not from epipolar distillation.
What would settle it
One concrete test: train the full pipeline with the epipolar attention replaced by ordinary global cross-attention over the same reference-guided tokens and losses; if DIODE and ETH3D errors stay roughly the same, the horizontal-epipolar constraint is not the source of the gains. A second, direct check: on a held-out stereo image pair, measure whether the horizontal offset between each query token and its attention peak on the reference-guided tokens correlates with ground-truth disparity—absence of that inverse-depth correlation would falsify the geometric reading of the tokens.
If this is right
- Single-view depth models can reach multi-view-level metric scale accuracy, with UniDepthV2+EpiDistill cutting DIODE A.Rel from 0.716 to 0.396 and ETH3D A.Rel from 0.176 to 0.137 in zero-shot evaluation.
- The method is model-agnostic: it boosts both UniDepthV2 and DepthPro, suggesting the distillation and token mechanism can be attached to any ViT-based depth foundation model.
- Because the scale and structural components are decoupled, the method primarily fixes scale ambiguity while preserving or slightly improving relative structure—multi-view geometry is distilled into a globally consistent scale.
- The rectified stereo tokens double as efficient adapter parameters: fine-tuning only the tokens and cross-attention layer (keeping the rest frozen) further improves accuracy on target datasets with just a few epochs.
- The approach improves depth-intrinsic disentanglement, as shown by higher Pearson correlation on Dolly Zoom sequences where focal length varies while subject size stays constant.
Where Pith is reading between the lines
- Editor's inference: the learned attention peaks along the horizontal scanline are predicted to encode disparity; if the geometric interpretation holds, the peak offset for each query should scale inversely with depth. A reader could verify this by correlating peak offsets with ground-truth disparity on a held-out set.
- Editor's inference: the method depends on ground-truth depth to build the attention target, which limits training to datasets with annotated depth and poses; extending the Gaussian bias to use predicted or self-supervised depth could make the distillation pipeline applicable to unlabeled video.
- Editor's inference: the rectified-stereo simplification forces the model to learn a single implicit baseline. A possible boundary case is general camera geometries with vertical baselines or rotation (unrectified); whether the token mechanism generalizes beyond the horizontal scanline constraint is not addressed by the paper.
- Editor's inference: the reported gains could partly come from the added capacity of the DPT head, scale head, and token grid rather than from the geometric constraint itself; an ablation that swaps the epipolar constraint for a non-geometric peering mechanism would separate the two effects.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes EpiDistill, a framework that transfers scale-aware geometric priors from multi-view depth models into monocular depth foundation models. During training, a multi-view branch uses depth-guided epipolar attention with ground-truth depth to supervise attention along epipolar lines. At inference, the single-view branch replaces actual source views with learnable Rectified Stereo Tokens (T_rst) and applies a horizontally constrained cross-attention to simulate a rectified stereo setup. A compound loss distills multi-view token features into the single-view tokens, while lightweight heads predict a structural offset, a global scale factor, and focal-length residuals. Experiments on KITTI, DIODE, ETH3D, NYU, and other benchmarks claim consistent zero-shot metric depth improvements over UniDepthV2 and DepthPro, with ablations attributing the gains to the two proposed components.
Significance. The idea is timely and potentially impactful: it targets the well-known scale ambiguity of monocular metric depth by leveraging multi-view geometric cues without requiring multi-view input at inference. The epipolar geometry derivations in Eqs. (2) and (6) are standard and correctly presented, and the model-agnostic design is a strength. The evaluation is broad and the ablations cover the main loss terms. However, the central mechanism — that the learned tokens behave as a genuine surrogate view and that cross-attention preserves true epipolar geometry — is supported only by qualitative visualizations, and the headline DIODE improvement rests on an unexplained baseline value. If the geometric mechanism is quantitatively verified, this would be a solid contribution; as it stands, the causal claim is under-supported.
major comments (3)
- [§4.2, Fig. S1] The core claim is that T_rst acts as a surrogate source view and that the cross-attention respects true epipolar geometry. The only evidence is qualitative attention maps. Because T_rst is learned and the horizontal constraint is a fixed mask rather than a camera-derived epipolar line, the gains in Table 4 could stem from added parameters and direct scale/ray supervision. Please add a quantitative test: for each query, compare the attention-peak disparity to the value predicted from GT depth and focal length under a rectified stereo model (Eq. (3) with R=I and t=[b,0,0]). Also add an ablation with equal-capacity non-geometric tokens or unconstrained cross-attention. This is necessary to support the claimed mechanism.
- [Table 1, DIODE row] UniDepthV2's A.Rel of 0.716 on DIODE is anomalously high compared with UniDepthV1's 0.263 and with the other baselines in the same table. The claimed 44.7% improvement is measured from this point. The authors must clarify whether this follows UniDepthV2's official evaluation protocol; if yes, provide a reference for the expected value, and if not, explain the discrepancy. Without this, the headline result may be an artifact of a degraded baseline rather than a genuine improvement.
- [Table 4 and Table S3] The ablations do not isolate the geometric mechanism. The 'Global Attn.' baseline has 154.65M parameters while DGEA+RST has 180.98M, so the improvement could be capacity alone. The paper omits an RST-only variant and any comparison with a token-based module that does not enforce horizontal epipolar attention. A matched-parameter ablation, e.g., replacing the epipolar mask with unconstrained cross-attention using the same number of learned tokens, is required to attribute the gain to the epipolar constraint rather than to the added token stream and losses.
minor comments (4)
- [Fig. 5 caption] The caption contains a typo: 'A.RelUniDepthV2-A.RelUniDepthV23ED' should presumably be 'A.RelUniDepthV2 - A.RelUniDepthV2+ED'.
- [Eq. (4) and Table S3] Eq. (4) uses γ to denote the Gaussian bias, while Table S3 refers to 'Width' and 'Bias' as separate hyperparameters. Please clarify the relationship between γ, the sampling width, and the number of sampled points (16 in §3.1 of the supplement).
- [Tables 1–3] Please report variance or results over multiple seeds/runs for the main evaluation tables. Several improvements are small (e.g., NYU A.Rel 0.077→0.077 in Table 2; Booster 0.179→0.171), and single-run comparisons are difficult to interpret.
- [Reproducibility] Consider releasing code or detailed training scripts. The architecture has many interacting components (frozen encoder, shared attention layers, RST grid resolution, loss weights), and code would materially strengthen the evidence for the claimed improvements.
Circularity Check
No load-bearing circularity; the training pipeline uses external ground truth and benchmarks, with only minor non-load-bearing self-citations.
full rationale
The derivation chain is not circular. The teacher signal for the depth-guided epipolar attention (Eqs. 3-5) is computed from external ground-truth depth and camera poses, and the scale/ray losses (Eqs. 10-11) supervise predictions against ground truth on training data. The rectified stereo tokens are a new architectural component whose claimed geometric role is tested through ablations (Table 4, Table S3) and external zero-shot benchmarks, not restated from the inputs. The paper contains two self-citations ([11], [47]) but neither is load-bearing for the central claim. The main caveats are attribution and support, not circularity: the direct scale supervision Lscale could account for part of the metric-scale gain, and the claimed 'implicit virtual baseline' (Fig. S1) is only qualitatively supported. These are experimental/support gaps, not reductions of the prediction to its inputs by construction. No Eq. X = Eq. Y equivalence or fitted-parameter-renamed-as-prediction was found.
Axiom & Free-Parameter Ledger
free parameters (5)
- DGEA Gaussian bias and width =
bias=50, width=16
- Loss weights =
lambda_ray=0.3, lambda_distill=1.0, lambda_attn=0.1
- Rectified Stereo Token resolution =
37x37 tokens per layer
- Number of epipolar layers =
4 layers
- Minimum visual overlap and sampling window =
20% overlap, +/-20 frames
axioms (4)
- standard math Standard epipolar geometry: fundamental matrix relation (Eq. 2) and perspective projection (Eq. 3) hold for the given camera poses and intrinsics.
- standard math Rectified stereo setup assumes identity rotation and pure horizontal translation, so the epipolar line is horizontal (Eq. 6).
- domain assumption Ground-truth depth, camera intrinsics, and 6-DoF poses are accurate for all seven training datasets.
- ad hoc to paper Learnable rectified stereo tokens can substitute for genuine source views and maintain a meaningful epipolar structure at inference.
invented entities (1)
-
Rectified Stereo Tokens (Trst)
no independent evidence
read the original abstract
Monocular depth foundation models have demonstrated remarkable generalization capabilities across diverse environments. However, they continue to struggle with metric depth estimation in diverse environments. This limitation stems from the inherent scale ambiguity of single-view inference, leading to misaligned scale predictions even when the relative geometry is accurate. Conversely, recent multi-view foundation models leverage cross-view cues to learn robust scene-level geometry and consistent scale. Yet, these benefits typically vanish during single-image inference, as the absence of explicit geometric constraints causes performance to degrade. To bridge this gap, we propose a novel framework that transfers the scale-aware geometric priors of multi-view models into monocular depth foundation models. Specifically, we introduce an Epipolar Distillation (EpiDistill), an approach utilizing Rectified Stereo Tokens, which enables the single-view prediction model to retain epipolar attention patterns and maintain geometric consistency without requiring multi-view inputs at inference. Experimental results demonstrate that our method significantly improves zero-shot metric depth estimation, particularly on challenging datasets like ETH3D and DIODE where scale alignment is critical. Furthermore, our approach is model-agnostic, consistently boosting the performance of state-of-the-art ViT-based models, including UniDepthV2 and DepthPro.
Figures
Reference graph
Works this paper leans on
-
[1]
Proceedings of the Computer Vision and Pattern Recognition Conference , pages=
Vggt: Visual geometry grounded transformer , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=
-
[2]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Unidepth: Universal monocular metric depth estimation , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[3]
TPAMI , year=
Unidepthv2: Universal monocular metric depth estimation made simpler , author=. TPAMI , year=
-
[4]
arXiv preprint arXiv:2511.10647 , year=
Depth anything 3: Recovering the visual space from any views , author=. arXiv preprint arXiv:2511.10647 , year=
-
[5]
IJRR , year=
Vision meets robotics: The kitti dataset , author=. IJRR , year=
-
[6]
European conference on computer vision , pages=
Indoor segmentation and support inference from rgbd images , author=. European conference on computer vision , pages=. 2012 , organization=
2012
-
[7]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
3d packing for self-supervised monocular depth estimation , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
-
[8]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
nuscenes: A multimodal dataset for autonomous driving , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
-
[9]
Proceedings of the IEEE conference on computer vision and pattern recognition , pages=
Scannet: Richly-annotated 3d reconstructions of indoor scenes , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=
-
[10]
Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
Scannet++: A high-fidelity dataset of 3d indoor scenes , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
-
[11]
Proceedings of the IEEE/CVF international conference on computer vision , pages=
Metric3d: Towards zero-shot metric 3d prediction from a single image , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=
-
[12]
IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=
Metric3d v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 2024 , publisher=
2024
-
[13]
European Conference on Computer Vision , pages=
Rotary position embedding for vision transformer , author=. European Conference on Computer Vision , pages=. 2024 , organization=
2024
-
[14]
ICLR , year=
Depth Pro: Sharp Monocular Metric Depth in Less Than a Second , author=. ICLR , year=
-
[15]
arXiv preprint arXiv:2010.11929 , year=
An image is worth 16x16 words: Transformers for image recognition at scale , author=. arXiv preprint arXiv:2010.11929 , year=
Pith/arXiv arXiv 2010
-
[16]
ICCV , year=
Vision transformers for dense prediction , author=. ICCV , year=
-
[17]
arXiv preprint arXiv:2304.07193 , year=
Dinov2: Learning robust visual features without supervision , author=. arXiv preprint arXiv:2304.07193 , year=
-
[18]
arXiv preprint arXiv:2509.13414 , year=
Mapanything: Universal feed-forward metric 3d reconstruction , author=. arXiv preprint arXiv:2509.13414 , year=
-
[19]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[20]
NeurIPS , year=
MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Details , author=. NeurIPS , year=
-
[21]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
Depth anything: Unleashing the power of large-scale unlabeled data , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
-
[22]
Advances in Neural Information Processing Systems , volume=
Depth anything v2 , author=. Advances in Neural Information Processing Systems , volume=
-
[23]
1--a model zoo for robust monocular relative depth estimation , author=
Midas v3. 1--a model zoo for robust monocular relative depth estimation , author=. arXiv preprint arXiv:2307.14460 , year=
-
[24]
Advances in neural information processing systems , volume=
Depth map prediction from a single image using a multi-scale deep network , author=. Advances in neural information processing systems , volume=
-
[25]
2016 Fourth international conference on 3D vision (3DV) , pages=
Deeper depth prediction with fully convolutional residual networks , author=. 2016 Fourth international conference on 3D vision (3DV) , pages=. 2016 , organization=
2016
-
[26]
Proceedings of the IEEE conference on computer vision and pattern recognition , pages=
Multi-scale continuous crfs as sequential deep networks for monocular depth estimation , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=
-
[27]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
Adabins: Depth estimation using adaptive bins , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
-
[28]
Proceedings of the IEEE conference on computer vision and pattern recognition , pages=
Deep ordinal regression network for monocular depth estimation , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=
-
[29]
Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
Adaptive confidence thresholding for monocular depth estimation , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
-
[30]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
On the uncertainty of self-supervised monocular depth estimation , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
-
[31]
Proceedings of the IEEE/CVF international conference on computer vision , pages=
Self-supervised monocular depth hints , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=
-
[32]
IEEE Robotics and Automation Letters , year=
Unsupervised depth completion from visual inertial odometry , author=. IEEE Robotics and Automation Letters , year=
-
[33]
CVPR , year=
Unik3d: Universal camera monocular 3d estimation , author=. CVPR , year=
-
[34]
CVPR , year=
Dust3r: Geometric 3d vision made easy , author=. CVPR , year=
-
[35]
ECCV , year=
Grounding image matching in 3d with mast3r , author=. ECCV , year=
-
[36]
ICCV , year=
Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding , author=. ICCV , year=
-
[37]
WACV , year=
Eden: Multimodal synthetic dataset of enclosed garden scenes , author=. WACV , year=
-
[38]
ICCV , year=
Pointodyssey: A large-scale synthetic dataset for long-term point tracking , author=. ICCV , year=
-
[39]
CVPR , year=
The cityscapes dataset for semantic urban scene understanding , author=. CVPR , year=
-
[40]
arXiv preprint arXiv:2205.04565 , year=
Is my Depth Ground-Truth Good Enough? HAMMER--Highly Accurate Multi-Modal Dataset for DEnse 3D Scene Regression , author=. arXiv preprint arXiv:2205.04565 , year=
-
[41]
CoRR , year=
Diode: A dense indoor and outdoor depth dataset , author=. CoRR , year=
-
[42]
CVPR , year=
Open challenges in deep stereo: the booster dataset , author=. CVPR , year=
-
[43]
Computer Vision and Image Understanding , volume=
Comparison of monocular depth estimation methods using geometrically relevant metrics on the IBims-1 dataset , author=. Computer Vision and Image Understanding , volume=. 2020 , publisher=
2020
-
[44]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
Scalability in perception for autonomous driving: Waymo open dataset , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
-
[45]
arXiv preprint arXiv:1711.05101 , year=
Decoupled weight decay regularization , author=. arXiv preprint arXiv:1711.05101 , year=
-
[46]
Palazzolo and J
E. Palazzolo and J. Behley and P. Lottes and P. Gigu\`ere and C. Stachniss , title =. 2019 , url =
2019
-
[47]
Proceedings of the IEEE conference on computer vision and pattern recognition , pages=
A multi-view stereo benchmark with high-resolution images and multi-camera videos , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=
-
[48]
NeurIPS , year=
Evaluating Robustness of Monocular Depth Estimation with Procedural Scene Perturbations , author=. NeurIPS , year=
-
[49]
CVPR , year=
Vggt: Visual geometry grounded transformer , author=. CVPR , year=
-
[50]
CVPR , year=
Unidepth: Universal monocular metric depth estimation , author=. CVPR , year=
-
[51]
ICLR , year=
Depth anything 3: Recovering the visual space from any views , author=. ICLR , year=
-
[52]
ECCV , year=
Indoor segmentation and support inference from rgbd images , author=. ECCV , year=
-
[53]
CVPR , year=
3d packing for self-supervised monocular depth estimation , author=. CVPR , year=
-
[54]
CVPR , year=
nuscenes: A multimodal dataset for autonomous driving , author=. CVPR , year=
-
[55]
CVPR , year=
Scannet: Richly-annotated 3d reconstructions of indoor scenes , author=. CVPR , year=
-
[56]
ICCV , year=
Scannet++: A high-fidelity dataset of 3d indoor scenes , author=. ICCV , year=
-
[57]
ICCV , year=
Metric3d: Towards zero-shot metric 3d prediction from a single image , author=. ICCV , year=
-
[58]
TPAMI , year=
Metric3d v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation , author=. TPAMI , year=
-
[59]
ECCV , year=
Rotary position embedding for vision transformer , author=. ECCV , year=
-
[60]
ICLR , year=
An image is worth 16x16 words: Transformers for image recognition at scale , author=. ICLR , year=
-
[61]
3DV , year=
Mapanything: Universal feed-forward metric 3d reconstruction , author=. 3DV , year=
-
[62]
CVPR , year=
Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision , author=. CVPR , year=
-
[63]
CVPR , year=
Depth anything: Unleashing the power of large-scale unlabeled data , author=. CVPR , year=
-
[64]
NeurIPS , year=
Depth anything v2 , author=. NeurIPS , year=
-
[65]
NeurIPS , year=
Depth map prediction from a single image using a multi-scale deep network , author=. NeurIPS , year=
-
[66]
3DV , year=
Deeper depth prediction with fully convolutional residual networks , author=. 3DV , year=
-
[67]
CVPR , year=
Multi-scale continuous crfs as sequential deep networks for monocular depth estimation , author=. CVPR , year=
-
[68]
CVPR , year=
Adabins: Depth estimation using adaptive bins , author=. CVPR , year=
-
[69]
CVPR , year=
Deep ordinal regression network for monocular depth estimation , author=. CVPR , year=
-
[70]
ICCV , year=
Adaptive confidence thresholding for monocular depth estimation , author=. ICCV , year=
-
[71]
CVPR , year=
On the uncertainty of self-supervised monocular depth estimation , author=. CVPR , year=
-
[72]
ICCV , year=
Self-supervised monocular depth hints , author=. ICCV , year=
-
[73]
RAL , year=
Unsupervised depth completion from visual inertial odometry , author=. RAL , year=
-
[74]
CVIU , year=
Comparison of monocular depth estimation methods using geometrically relevant metrics on the IBims-1 dataset , author=. CVIU , year=
-
[75]
CVPR , year=
Scalability in perception for autonomous driving: Waymo open dataset , author=. CVPR , year=
-
[76]
ICLR , year=
Decoupled weight decay regularization , author=. ICLR , year=
-
[77]
Palazzolo and J
E. Palazzolo and J. Behley and P. Lottes and P. Gigu\`ere and C. Stachniss , title =. IROS , year =
-
[78]
CVPR , year=
A multi-view stereo benchmark with high-resolution images and multi-camera videos , author=. CVPR , year=
-
[79]
arXiv preprint arXiv:2507.00981 , year=
Evaluating Robustness of Monocular Depth Estimation with Procedural Scene Perturbations , author=. arXiv preprint arXiv:2507.00981 , year=
-
[80]
ICCV , year=
Towards zero-shot scale-aware monocular depth estimation , author=. ICCV , year=
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.