Pith. sign in

REVIEW 2 major objections 7 minor 56 references

Counterfactual Depth from a Single RGB Image

T0 review · 2 major / 7 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proposes counterfactual depth: from one RGB image and an object mask, a network predicts a dense, pixel-level depth map of the scene with that object removed.

desk verdict New task, plausible method, honest appendix—but the real-scene claim is only tested in the 0-5m band the model trained on, so the abstract overstates. read the letter →

arxiv 1909.00915 v1 pith:VPSOTP44 submitted 2019-09-03 cs.CV

Theerasit Issaranon , Chuhang Zou , David Forsyth This is my paper · ORCID
classification cs.CV
keywords counterfactualdepthsingle-imageestimationobjectremovalhiddengeometryscenecompletionsurfacenormallossmask-conditionedpredictionmonocular
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces counterfactual depth: given a single RGB image and a mask around an object, predict the dense depth map of the scene as it would look with that object removed. The claim is that hidden geometry behind the mask is recoverable at pixel precision from visible context alone, without RGB-D input or a voxel representation. The authors argue this works because indoor scenes are spatially coherent: the surfaces hidden behind objects are usually walls, floors, or simple planes whose depth can be extrapolated from surrounding pixels. If correct, the method gives a cheap way to reason about occluded free space from ordinary photographs, with direct uses in object insertion, manipulation, and scene editing.

What carries the argument

The load-bearing mechanism is a fully convolutional encoder-decoder in which the object mask is resized and concatenated to the feature maps at every up-projection layer of the decoder, where an up-projection layer is a learned upsampling block. The bottleneck compresses the image into long-range depth statistics; the mask tells the decoder which pixels to distrust and where to extrapolate. Training is stabilized by mask dropout, which randomly flips ten percent of mask pixels so the network learns to draw on nearby context, and by a weighted, quantized, smoothed surface-normal loss that keeps normals derived from predicted depth accurate on planes while preserving ridges. These components let the decoder fill in hidden depth rather than merely smooth the nearest visible depth.

What would settle it

Build a real test set where the region behind the masked object is deliberately non-coherent, such as a tangle of unrelated objects or dense vegetation rather than a wall or floor, and compare interior depth error against the do-nothing baseline on those images. If the network's interior error is no better than that baseline, the spatial-coherence premise fails, because the network would be reproducing its training prior rather than reading the scene.

Watch

Extended reading notes

Core claim

The central discovery is that a standard fully convolutional encoder-decoder, modified so a user-supplied object mask is concatenated into each up-projection stage of the decoder, can extrapolate depth into masked regions from monocular RGB input. Trained on synthetic indoor scenes rendered with objects physically removed, together with real indoor images paired with empty masks, the network predicts both visible and hidden depth. On the authors' evaluation dataset of 72 real-scene cases spanning object complexity, rarity, clutter, background, and distance, the method reports lower interior-region error than depth hole filling, image inpainting followed by depth estimation, and an auto-encoder baseline. The network also removes arbitrary combinations of objects through the supplied mask and produces consistent depth for non-removed areas.

Load-bearing premise

The load-bearing premise is that the depth hidden behind a masked object is reliably recoverable from the visible surrounding scene because indoor geometry is spatially coherent, and that training on synthetic removals plus real images with no object removed transfers to real masked objects. The paper itself notes that the synthetic and real depth distributions differ, so the transfer half of this premise is only partially tested.

Editorial extensions

If this is right

  • With a single RGB image and an object mask, the network produces dense pixel-level depth for the hidden region, so occluded free space can be interrogated at higher resolution than a coarse voxel grid.
  • Because the mask is an input rather than a fixed output category, the same trained network removes any object or combination of objects in one forward pass.
  • The method beats the natural baselines of depth hole filling, image inpainting plus depth prediction, and a depth auto-encoder on the authors' synthetic and real evaluation sets, especially inside the masked region.
  • The counterfactual depth map can support downstream image editing tasks such as object insertion and manipulation without requiring RGB-D sensors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: because the mask is user-supplied, the same architecture could be repurposed to predict depth under other hypothetical edits, such as moving an object, opening a door, or revealing a region behind a foreground surface, by training on corresponding synthetic edits.
  • Editorial extension: the reported sensitivity to depth range suggests a testable fix: train with synthetic scenes whose depth ranges match the target domain, or apply domain adaptation to align depth distributions, which would likely close the real-data gap the paper concedes in Appendix B.
  • Editorial extension: counterfactual depth could serve as a self-supervised consistency target for video or multi-view systems, where agreement between predicted hidden depth across frames could provide training signal without ground-truth removal data.
  • Editorial extension: the ANOVA findings on the authors' 72-case dataset, where far objects with cluttered backgrounds and simple rare objects tend to increase error, imply a prioritization rule for downstream robotics: trust hidden-depth predictions most near walls and floors, and least behind complex clutter.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. The paper introduces the task of counterfactual depth estimation: given a single RGB image and an object mask, predict a dense pixel-level depth map of the scene with the masked object removed. The proposed network is an encoder-decoder (ResNet-50 encoder with up-projection decoder) in which the mask is concatenated at each decoder stage; training uses a weighted sum of a smoothed surface-normal loss, an average-depth loss, and the berHu loss. Training data combine AI2-THOR synthetic scenes with one object removed and NYUv2 real images with empty masks. Experiments on a held-out AI2-THOR split and on a newly collected 72-case real RGBD dataset compare the method against do-nothing, Poisson depth editing, DepthComp, image inpainting, and a vanilla auto-encoder. The paper reports superior interior-region metrics for the proposed method, ablations for mask dropout and the normal loss, and an ANOVA of factors affecting error. An appendix analyzes the depth-distribution mismatch between AI2-THOR (0-5 m) and NYUv2 (0-10 m) and concedes that the mixed training biases the model toward shallow depths.

Significance. If validated, the paper makes a useful contribution by defining and attacking a well-specified new task: pixel-level counterfactual depth from monocular RGB and a mask. The controlled real-world evaluation set is a valuable asset, and the quantitative gains over the strongest baselines (interior RMSE of 0.310 versus 0.394 for Poisson and 0.353 for the auto-encoder on the real set) are concrete and credible. The paper is also commendably transparent about the synthetic-to-real depth-range mismatch in Appendix B. The main significance limitation is that the real evidence is confined to near-field masks, so the 'better than baselines' claim is established only within a restricted depth regime.

major comments (2)
  1. [Section 4.1, Appendix B, Section 4.2/Table 3] The general claim that a single RGB image and mask suffice to predict counterfactual depth in real scenes is not supported beyond the trained depth range. The only supervised object-removal examples in training come from AI2-THOR, whose rendered depths are capped at 5 m, while the real NYUv2 images contribute only empty-mask supervision. Appendix B explicitly concedes that mixing these distributions biases the network toward shallow depths and that the model makes larger errors above 5 m on NYUv2. The real quantitative evaluation (Section 4.2, Table 3) places objects at 1.5 m and 2.0 m, so the revealed backgrounds almost always lie within the trained 0-5 m band. Consequently, the paper's abstract and conclusions are not established for typical room-scale geometry beyond 5 m, where the manuscript itself anticipates failure. Please either add real-world evaluations with objects and backgrounds spanning and exceeding 5 m, or explicitly restrict the claims to near-field removals within the trained depth range.
  2. [Section 5.2 / Table 3] The central quantitative claim of superiority over baselines on real images is supported only by point estimates on 72 cases; no confidence intervals, paired significance tests, or per-factor error bars are reported. The interior-RMSE advantage (0.310 versus 0.394 for Poisson and 0.310 versus 0.353 for the auto-encoder) is substantial, but without interval estimates the reader cannot assess its stability across the controlled factor levels, especially because Appendix E reports that some factors significantly increase error. Please report paired bootstrap confidence intervals or equivalent significance tests, and ideally per-factor breakdowns, to make the 'better than baselines' claim statistically grounded.
minor comments (7)
  1. [Section 5, metric definition] The definition of the rel metric uses the predicted depth in the denominator: (1/Q) sum |d_p - d_hat_p| / d_hat_p. The standard relative error for depth estimation divides by the ground-truth depth d_p. Please correct the definition and recompute the reported rel values if the tables used the nonstandard form.
  2. [Related Work] There is a typo: 'conterfactual depth prediction' should be 'counterfactual depth prediction'.
  3. [Section 4.1] The phrase 'standard derivation (std)' should read 'standard deviation (std)'.
  4. [Tables 2 and 3] The mask-dropout ablation is inconsistent across datasets: on the synthetic AI2-THOR test set, removing mask dropout improves interior RMSE (0.569 versus 0.592) and delta1 (80.2 versus 78.9), while on the real set mask dropout is essential (interior RMSE 0.517 versus 0.310). The text says mask dropout 'helps stabilize performance' but does not address the synthetic reversal; please qualify the claim or provide an explanation.
  5. [Tables 2 and 3, footnote] The footnote that Poisson and DepthComp 'does not produce pixels in this region' is imprecise: these methods do produce exterior depth values identical to the input depth map, so their exterior error coincides with the 'Do Nothing' baseline. Please restate the meaning of the asterisk accordingly.
  6. [Appendix E] The ANOVA uses 72 images with 15 regression terms (five main effects and ten interactions) and reports an adjusted R-squared of 0.882 for the proposed method. This is surprisingly high for such a small sample; please report the fitted coefficients, standard errors, and effective number of observations to rule out overfitting.
  7. [Appendix A] The claimed advantage of the weighted quantized surface-normal computation over plane fitting is demonstrated only on synthetic noise. A comparison on real sensor data or an end-to-end depth metric would strengthen the claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the counterfactual-depth prediction is a supervised network output, not a by-construction restatement of its inputs; the only self-citation is a non-load-bearing related-work reference.

full rationale

The paper does not claim a first-principles derivation. It defines counterfactual depth as the depth map of the scene with a masked object removed, and obtains ground-truth tuples externally by rendering AI2-THOR scenes with objects removed (Sec. 4.1). The network is trained with the loss in Eq. 1 on RGB-mask-depth triples, and the reported predictions on held-out AI2-THOR scenes and on the separately collected real dataset (Tables 2-3) are outputs of that trained network. No fitted parameter is renamed as a prediction, and no equation reduces to its own input. The only self-citation (Hedau, Hoiem and Forsyth, ref. [16]) appears in related work as an example of early room-box priors and is not load-bearing. The decision to give all baselines the same initial depth from the authors' network is a controlled-comparison design; even if it limits external benchmarking, it does not make the headline claim equivalent to the method's inputs. Appendix B's admission of a depth-range mismatch between AI2-THOR and NYUv2 is an external-validity limitation, not circularity. Accordingly, no circular step is present.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central claim depends on hand-set hyperparameters (loss weights, mask dropout, normal smoothing constants, augmentation ranges) and on domain assumptions about the simplicity and transferability of indoor scene geometry. No new physical entities are introduced; counterfactual depth is a task definition, not an entity.

free parameters (5)
  • Loss weights w1, w2, w3 = 1, 0.5, 1
    Hand-set weights for surface normal, average depth, and berHu losses in Eq. 1; no ablation or selection criterion reported.
  • berHu cutoff c = 0.2 x max per-batch error
    Cut-off in reverse Huber loss, taken from prior work and not tuned for this task.
  • Mask dropout probability = 0.1 per pixel
    Chosen after initial experiments; ablation in Tables 2 and 3 shows large effect, so the final result depends on this choice.
  • Normal smoothing bins and beta = 16 latitudes, 4 azimuths, beta=8
    Parameters of weighted quantized normal smoothing in Section 3.2; manual and no sensitivity analysis.
  • Data augmentation ranges = crop alpha in [2/3,1], rotation [-5,5], color gain [0.8,1.2]
    Manual augmentation ranges shape training distribution; not optimized.
assumptions (3)
  • domain assumption Indoor scene depth fields have simple, spatially coherent structure that makes hidden depth behind an object predictable from visible context.
    Stated in the Introduction and Section 3.1 as the reason counterfactual depth is learnable; no proof or measure of this prior is given.
  • domain assumption Depth and appearance statistics of AI2-THOR synthetic rooms, with depth capped at 5m, transfer to real indoor scenes well enough to learn object removal.
    Section 4.1 trains removal only on AI2-THOR; Appendix B admits depth distribution mismatch and NYU performance degradation beyond 5m.
  • domain assumption Training on NYUd v2 depth maps with empty masks, combined with synthetic removal data, yields a network that removes objects in real images without seeing real removal ground truth.
    Section 4.1 mixes the two datasets; there is no real counterfactual depth ground truth used in training, only synthetic.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Counterfactual Depth from a Single RGB Image." pith.science (2026). https://pith.science/paper/VPSOTP44

@misc{pith2026190900915,
  author       = {Pith},
  title        = {Pith review of: Counterfactual Depth from a Single RGB Image},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VPSOTP44}},
  note         = {Machine review of arXiv:1909.00915}
}
read the original abstract

We describe a method that predicts, from a single RGB image, a depth map that describes the scene when a masked object is removed - we call this "counterfactual depth" that models hidden scene geometry together with the observations. Our method works for the same reason that scene completion works: the spatial structure of objects is simple. But we offer a much higher resolution representation of space than current scene completion methods, as we operate at pixel-level precision and do not rely on a voxel representation. Furthermore, we do not require RGBD inputs. Our method uses a standard encoder-decoder architecture, and with a decoder modified to accept an object mask. We describe a small evaluation dataset that we have collected, which allows inference about what factors affect reconstruction most strongly. Using this dataset, we show that our depth predictions for masked objects are better than other baselines.

Figures

Figures reproduced from arXiv: 1909.00915 by the authors.

Figure 1
Figure 1. Illustration. Given an image of a scene (left), our goal is to predict the depth map for that scene with the object removed (right): e.g. the image depth without the microwave (outlined in green). Our system predicts depth directly from a single RGB image, offering a representation of the free space behind an object, even though it cannot see what lies there. These predictions are possible because indoor depth maps … view at source ↗
Figure 2
Figure 2. Network architecture. Our network takes as input a single RGB image and a 2D object mask. The network follows an encoder and decoder strategy. The final output is the predicted depth of the scene with the object removed: we predict the depth of layouts behind the chair, and the depth of other non-removed objects, e.g. the small table in front of the chair. We also show the surface normal derived from our predicted d… view at source ↗
Figure 3
Figure 3. Surface normal derived from depth v.s. our weighted quantized smoothed normal. We show: RGB image (top left) and the ground truth depth (bottom left), ground truth surface normal which is the first-order derivatives of the ground truth depth (middle top) and our weighted quantized smooth ground truth normal (middel down). Top right is the normal direction field. Note that lighter pixel indicates that the surface nor… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Image samples from the dataset we use. Left to right: NYUd v2 [40] (real dataset), AI2-THOR [23] (synthetic dataset), our collected dataset (real dataset). AI2-THOR and our collected dataset has ground truth depth with object removed. Best viewed in color. Factor varia…
Figure 5
Figure 5. Figure 5: Qualitative results of depth estimation with the object [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Qualitative results of depth estimation with the object [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Qualitative results of depth estimation with multiple objects [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Illustration of the different depth statistic between AI2- [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Qualitative results of depth estimation with object [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: More qualitative results of depth estimation with the object [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Qualitative results of depth estimation with the object [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Qualitative results of depth estimation with the object [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 53 canonical work pages

  1. [1]

    Atapour-Abarghouei and T

    A. Atapour-Abarghouei and T. P. Breckon. Depthcomp: real- time depth image completion based on prior semantic scene segmentation. 2017. 2, 5

    work page 2017

  2. [2]

    Atapour-Abarghouei and T

    A. Atapour-Abarghouei and T. P. Breckon. Real-time monocular depth estimation using synthetic data with do- main adaptation via image style transfer. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2800–2810, 2018. 2

    work page 2018

  3. [3]

    Badrinarayanan, A

    V . Badrinarayanan, A. Kendall, and R. Cipolla. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE transactions on pattern analysis and machine intelligence, 39(12):2481–2495, 2017. 5

    work page 2017

  4. [4]

    Ballester, M

    C. Ballester, M. Bertalmio, V . Caselles, G. Sapiro, and J. Verdera. Filling-in by joint interpolation of vector fields and gray levels. IEEE transactions on image processing , 10(8):1200–1211, 2001. 2

    work page 2001

  5. [5]

    Barnes, E

    C. Barnes, E. Shechtman, A. Finkelstein, and D. B. Gold- man. Patchmatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics (ToG), 28(3):24, 2009. 2

    work page 2009

  6. [6]

    J. T. Barron and J. Malik. Intrinsic scene properties from a single rgb-d image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 17–24,

    work page

  7. [7]

    Bertalmio, A

    M. Bertalmio, A. L. Bertozzi, and G. Sapiro. Navier-stokes, fluid dynamics, and image and video inpainting. In Pro- ceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) , vol- ume 1, pages I–I. IEEE, 2001. 2

    work page 2001

  8. [8]

    Chauve, P

    A.-L. Chauve, P. Labatut, and J.-P. Pons. Robust piecewise- planar 3d reconstruction and completion from large-scale un- structured point data. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 1261–1268. IEEE, 2010. 2, 3

    work page 2010

Show all 56 references
  1. [9]

    W. Chen, Z. Fu, D. Yang, and J. Deng. Single-image depth perception in the wild. In Advances in Neural Information Processing Systems, pages 730–738, 2016. 2

    2016

  2. [10]

    W. Chen, D. Xiang, and J. Deng. Surface normals in the wild. In Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy, pages 22–29, 2017. 2

    2017

  3. [11]

    Eigen and R

    D. Eigen and R. Fergus. Predicting depth, surface normals and semantic labels with a common multi-scale convolu- tional architecture. In Proceedings of the IEEE International Conference on Computer Vision, pages 2650–2658, 2015. 2, 11

    2015

  4. [12]

    Eigen, C

    D. Eigen, C. Puhrsch, and R. Fergus. Depth map prediction from a single image using a multi-scale deep network. In Advances in neural information processing systems , pages 2366–2374, 2014. 2, 11

    2014

  5. [13]

    Firman, O

    M. Firman, O. Mac Aodha, S. Julier, and G. J. Bros- tow. Structured prediction of unobserved voxels from a sin- gle depth image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 5431– 5440, 2016. 1, 2

    2016

  6. [14]

    Godard, O

    C. Godard, O. Mac Aodha, and G. J. Brostow. Unsupervised monocular depth estimation with left-right consistency. In CVPR, volume 2, page 7, 2017. 2

    2017

  7. [15]

    Hays and A

    J. Hays and A. A. Efros. Scene completion using millions of photographs. In ACM Transactions on Graphics (TOG), volume 26, page 4. ACM, 2007. 2

    2007

  8. [16]

    Hedau, D

    V . Hedau, D. Hoiem, and D. Forsyth. Thinking inside the box: Using appearance models and context based on room geometry. In European Conference on Computer Vision , pages 224–237. Springer, 2010. 2

    2010

  9. [17]

    Huang, S

    J.-B. Huang, S. B. Kang, N. Ahuja, and J. Kopf. Image com- pletion using planar structure guidance. ACM Transactions on graphics (TOG), 33(4):129, 2014. 2, 3

    2014

  10. [18]

    Iizuka, E

    S. Iizuka, E. Simo-Serra, and H. Ishikawa. Globally and locally consistent image completion. ACM Transactions on Graphics (TOG), 36(4):107, 2017. 2, 5

    2017

  11. [19]

    Karsch, C

    K. Karsch, C. Liu, and S. B. Kang. Depth transfer: Depth extraction from video using non-parametric sampling. IEEE transactions on pattern analysis and machine intelligence , 36(11):2144–2158, 2014. 2

    2014

  12. [20]

    Kendall and Y

    A. Kendall and Y . Gal. What uncertainties do we need in bayesian deep learning for computer vision? In Advances in neural information processing systems, pages 5574–5584,

  13. [21]

    Y . Kim, H. Jung, D. Min, and K. Sohn. Deep monocular depth estimation via integration of global and local predic- tions. IEEE Transactions on Image Processing, 27(8):4131– 4144, 2018. 2

    2018

  14. [22]

    Kinga and J

    D. Kinga and J. B. Adam. A method for stochastic optimiza- tion. In International Conference on Learning Representa- tions (ICLR), volume 5, 2015. 5

    2015

  15. [23]

    Kolve, R

    E. Kolve, R. Mottaghi, D. Gordon, Y . Zhu, A. Gupta, and A. Farhadi. Ai2-thor: An interactive 3d environment for vi- sual ai. arXiv preprint arXiv:1712.05474, 2017. 4, 5

    2017 arXiv

  16. [24]

    Kuznietsov, J

    Y . Kuznietsov, J. St ¨uckler, and B. Leibe. Semi-supervised deep learning for monocular depth map prediction. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6647–6655, 2017. 2

    2017

  17. [25]

    Laina, C

    I. Laina, C. Rupprecht, V . Belagiannis, F. Tombari, and N. Navab. Deeper depth prediction with fully convolutional residual networks. In Fourth International Conference on 3D Vision (3DV), pages 239–248. IEEE, 2016. 2, 4, 11, 12

    2016

  18. [26]

    Li and N

    Z. Li and N. Snavely. Megadepth: Learning single-view depth prediction from internet photos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 2041–2050, 2018. 2

    2018

  19. [27]

    C. Liu, P. Kohli, and Y . Furukawa. Layered scene decompo- sition via the occlusion-crf. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, pages 165–173, 2016. 2

    2016

  20. [28]

    G. Liu, F. A. Reda, K. J. Shih, T.-C. Wang, A. Tao, and B. Catanzaro. Image inpainting for irregular holes using par- tial convolutions. arXiv preprint arXiv:1804.07723 , 2018. 2

    2018 arXiv

  21. [29]

    J. Liu, X. Gong, and J. Liu. Guided inpainting and filter- ing for kinect depth maps. In Proceedings of the 21st In- ternational Conference on Pattern Recognition (ICPR2012), pages 2055–2058. IEEE, 2012. 2

    2012

  22. [30]

    M. Liu, X. He, and M. Salzmann. Building scene models by completing and hallucinating depth and semantics. In European Conference on Computer Vision, pages 258–274. Springer, 2016. 2

    2016

  23. [31]

    M. Liu, M. Salzmann, and X. He. Discrete-continuous depth estimation from a single image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 716–723, 2014. 2

    2014

  24. [32]

    Mal and S

    F. Mal and S. Karaman. Sparse-to-dense: Depth predic- tion from sparse depth samples and a single image. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 1–8. IEEE, 2018. 2, 11

    2018

  25. [33]

    A. B. Owen. A robust hybrid of lasso and ridge regression. Contemporary Mathematics, 443(7):59–72, 2007. 3

    2007

  26. [34]

    Pathak, P

    D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2536–2544, 2016. 2

    2016

  27. [35]

    Pauly, N

    M. Pauly, N. J. Mitra, J. Giesen, M. H. Gross, and L. J. Guibas. Example-based 3d scan completion. In Symposium on Geometry Processing, number CONF, 2005. 2

    2005

  28. [36]

    P ´erez, M

    P. P ´erez, M. Gangnet, and A. Blake. Poisson image edit- ing. ACM Transactions on graphics (TOG), 22(3):313–318,

  29. [37]

    D. Ron, K. Duan, C. Ma, N. Xu, S. Wang, S. Hanumante, and D. Sagar. Monocular depth estimation via deep structured models with ordinal constraints. In 2018 International Con- ference on 3D Vision (3DV), pages 570–577. IEEE, 2018. 2

    2018

  30. [38]

    Saxena, S

    A. Saxena, S. H. Chung, and A. Y . Ng. Learning depth from single monocular images. In Advances in neural information processing systems, pages 1161–1168, 2006. 2, 11

    2006

  31. [39]

    Shen and S.-C

    J. Shen and S.-C. S. Cheung. Layer depth denoising and completion for structured-light rgb-d cameras. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1187–1194, 2013. 2

    2013

  32. [40]

    Silberman, D

    N. Silberman, D. Hoiem, P. Kohli, and R. Fergus. Indoor segmentation and support inference from rgbd images. In European Conference on Computer Vision, pages 746–760. Springer, 2012. 3, 4, 5, 6, 8, 11, 13, 14

    2012

  33. [41]

    S. Song, S. P. Lichtenberg, and J. Xiao. Sun rgb-d: A rgb- d scene understanding benchmark suite. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pages 567–576, 2015. 5

    2015

  34. [42]

    S. Song, F. Yu, A. Zeng, A. X. Chang, M. Savva, and T. Funkhouser. Semantic scene completion from a single depth image. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 190–198. IEEE, 2017. 1, 2

    2017

  35. [43]

    L. Wang, H. Jin, R. Yang, and M. Gong. Stereoscopic in- painting: Joint color and depth completion from stereo im- ages. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1–8. IEEE, 2008. 2

    2008

  36. [44]

    C. Yang, X. Lu, Z. Lin, E. Shechtman, O. Wang, and H. Li. High-resolution image inpainting using multi-scale neural patch synthesis. In The IEEE Conference on Computer Vi- sion and Pattern Recognition (CVPR) , volume 1, page 3,

  37. [45]

    Zhang and T

    Y . Zhang and T. Funkhouser. Deep depth completion of a sin- gle rgb-d image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 175–185,

  38. [46]

    Accuracy

    T. Zhou, M. Brown, N. Snavely, and D. G. Lowe. Unsu- pervised learning of depth and ego-motion from video. In CVPR, volume 2, page 7, 2017. 2 Ours Gradient-based plane fitting [40] Accuracy 0.926 0.898 0.898 Speed (s) 0.424 0.013 78.175 Table 4. Ground truth surface normal comp...

    2017

  39. [47]

    numbers of objects close by

  40. [48]

    background (objects) behind

  41. [49]

    We analyze on our approach and the two baselines: im- age inpainting and poisson inpainting

    object’s distance to the camera The interaction terms are the 2-combination of the five individual terms, resulting in (5 2 ) = 10 terms. We analyze on our approach and the two baselines: im- age inpainting and poisson inpainting. E.1. Our method For our method, only 3 of 10 in...

  42. [50]

    objects far from the camera with cluttered back- grounds (mild increase in error rate)

  43. [51]

    simple objects that are far from the camera (mild increase in error rate)

  44. [52]

    simple objects that are rare (mild increase in error rate). Of the individual terms, rarity, objects behind and dis- tance to the camera have effects, with common objects, cluttered or empty space behind, and objects far from the camera are each associated with an increase in ...

  45. [55]

    rare objects that are far from the camera (mild de- crease in error rate). Of the individual terms, complexity, rarity, objects be- hind and size have effects, with simple objects, common objects, cluttered or empty backgrounds, and objects far from the camera are each associa...

  46. [56]

    objects far from the camera with two other objects close by (mild decrease in error rate)

  47. [57]

    simple objects that are far from the camera (mild decrease in error rate)

  48. [58]

    Note that the above effects are the same as for the inpaint- ing baseline

    rare objects that are far from the camera (mild de- crease in error rate). Note that the above effects are the same as for the inpaint- ing baseline. Of the individual terms, complexity, rarity, objects behind and size have effects, with simple objects, common objects, clutter...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.