REVIEW 4 major objections 5 minor 25 references
EdgeNet: Semantic Scene Completion from a Single RGB-D Image
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read EdgeNet fuses RGB edges with depth to improve 3D semantic scene completion, reporting 70.3 average IoU on SUNCG and matching a two-stage rival on NYU.
desk verdict EdgeNet makes a plausible architectural contribution, but its central claim that F-TSDF-encoded edges help rests on single-run gaps that need multi-seed verification. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the flipped truncated signed distance function (F-TSDF) applied to a projected Canny edge volume. F-TSDF is defined as $\mathrm{sign}(\mathrm{TSDF})\cdot(1-|\mathrm{TSDF}|)$, which turns the surface into a sharp discontinuity and gives dense gradients in the volume; the paper's move is to treat RGB edges as a second binary surface so that the same encoding can be applied to them. The second component is EdgeNet itself, a U-Net style 3D CNN with ResNet modules and dilated convolutions, evaluated in early, middle and late fusion schemes with a fixed channel budget so that memory use stays constant. The machinery works by giving the 3D CNN two complementary dense distance fields: one for geometry from depth and one for appearance boundaries from colour.
What would settle it
Re-run VVNetR-120 and TNetFuse with EdgeNet's re-rendered SUNCG test images and its NYU ground-truth voxelisation and evaluation code. If they reproduce their reported scores, EdgeNet's 70.3 and 33.7 IoU results hold; if their scores move, the claimed gains are protocol artefacts. A further check is to feed EdgeNet a random binary volume with the same density as the projected edges: if the IoU gain over EdgeNet-D persists, the improvement is not specifically due to RGB edges.
Extended reading notes
Core claim
The central claim is that previous attempts to add colour to semantic scene completion failed because colour is sparse after projection into a voxel grid, and that binarising appearance into edges before projection removes this sparsity. Because an edge volume is binary, the same F-TSDF encoding used for depth surfaces can be applied to it, producing a dense signed-distance field around projected edges. EdgeNet takes the depth F-TSDF volume and the edge F-TSDF volume, fuses them at one of three points in a 3D encoder-decoder built from ResNet modules with dilated convolutions, and trains the whole network with a batch-balanced categorical cross-entropy loss. The authors report that this is the first end-to-end approach in which RGB information improves semantic scene completion over depth only, and that the improvement is largest for classes such as TVs and generic objects.
Load-bearing premise
The load-bearing premise is that the previously published scores for SSCNet, VVNetR-120, TNetFuse and the other baselines were computed under exactly the same test split, ground-truth voxelisation, and evaluation script as EdgeNet; the paper extracts those numbers from the original papers rather than running a shared evaluation.
Editorial extensions
If this is right
- On SUNCG, EdgeNet-EF reaches 70.3 average semantic IoU, surpassing the best depth-only method VVNetR-120 by 3.3 points.
- Feeding F-TSDF edges into the older SSCNet architecture (SSCNet-E) improves over the depth-only SSCNet*, so the edge representation helps beyond EdgeNet itself.
- On NYUDv2 with SUNCG pretraining and fine-tuning, EdgeNet-MF reaches 33.7 average IoU, within 0.7 points of TNetFuse while using a single end-to-end training stage.
- The gains are concentrated in hard-to-detect classes such as TVs, pictures and generic objects, which are nearly invisible to depth after down-sampling.
- The offline F-TSDF preprocessing cuts training time from 7 to 4 days on SUNCG and from 30 to 6 hours on NYUDv2 compared with the original SSCNet pipeline.
Reading between the lines
- The paper only tests Canny edges, but its transfer argument implies that any binary 3D cue that can be F-TSDF encoded, such as object boundaries, keypoints, or learned edge maps, could be fused the same way; that is an extension the authors did not run.
- Because the paper shows that naive RGB projection fails while edge projection succeeds, a natural follow-up is to compare F-TSDF edge volumes against F-TSDF applied to dense semantic class probabilities, which would test whether the benefit comes from binarisation or from the edge cue specifically.
- The NYU ground-truth errors the paper documents (missing windows, pictures, shelves) penalise accurate models, so a real-world benchmark with cleaner annotations would likely show a larger EdgeNet margin than the reported 33.7.
- The fixed-memory fusion comparison suggests that late fusion loses accuracy because duplicated branches force channel cuts; a multi-scale attention mechanism could capture late-fusion benefits without that memory penalty.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. EdgeNet proposes a semantic scene completion method from a single RGB-D image. It extracts Canny edges from the RGB image, projects them to 3D using the depth map, voxelizes them, and encodes both the depth surface and the edge points with flipped TSDF (F-TSDF). The two volumes are then fed into a U-Net-like 3D CNN built from ResNet modules with dilated convolutions, and the paper evaluates three fusion schemes: early (EdgeNet-EF), middle (EdgeNet-MF), and late (EdgeNet-LF). Training uses offline F-TSDF preprocessing, weighted categorical cross-entropy, and a one-cycle learning-rate schedule. On SUNCG, EdgeNet-EF reports 70.3 average semantic IoU, above the reported VVNetR-120 value of 66.7; on NYUDv2, EdgeNet-MF reports 33.7 after fine-tuning on SUNCG plus NYU, close to the reported TNetFuse value of 34.4. The paper also reports ablations isolating the training pipeline (SSCNet*), the architecture (EdgeNet-D), and the edge input (SSCNet-E and the EdgeNet fusion variants).
Significance. If the reported gains are reproducible, the paper makes a useful contribution: binary edge volumes encoded with F-TSDF provide a simple and transferable RGB cue for 3D semantic scene completion without requiring a separately trained 2D segmentation network. The ablation design is a genuine strength: SSCNet* and EdgeNet-D isolate the training-pipeline and architecture contributions from the edge contribution, and the three fusion schemes explore a well-defined design space while keeping memory constant. However, the central claim that edges consistently improve depth-only performance is currently supported by single-run differences smaller than one IoU point on SUNCG, there is no depth-only EdgeNet baseline on NYU, and the state-of-the-art comparisons are made with numbers extracted from other papers under unspecified evaluation protocols. If the robustness and comparability issues are addressed, the contribution would be significant for the semantic scene completion community.
major comments (4)
- [IV-D1, Table I] The evidence that F-TSDF encoded edges improve semantic scene completion rests on single training runs. On SUNCG the paired edge-vs-depth gaps are +0.5 (SSCNet-E vs SSCNet*) and +0.8 (EdgeNet-EF vs EdgeNet-D), while EdgeNet-LF is actually 2.7 points below EdgeNet-D (66.8 vs 69.5). These margins are comparable to typical stochasticity in 3D CNN training, and no variance or number of seeds is reported. The authors should provide mean and standard deviation over at least three runs for at least the paired ablations, or otherwise demonstrate that the edge advantage is not due to chance.
- [IV-E, Table II] On NYUDv2 the paper reports only EdgeNet variants with depth and edges; there is no EdgeNet-D depth-only baseline trained under the same protocols. The claim that edges help on real data therefore rests on comparisons to the original SSCNet, which differs in architecture and training pipeline. Without an NYU-trained EdgeNet-D, the edge contribution on real data is confounded with architecture and pipeline changes. The authors should add the depth-only EdgeNet baseline on NYU.
- [IV-A and IV-D, Table I] The SUNCG comparisons to previous methods are not apples-to-apples. The paper states that because the original SUNCG train/test sets did not include RGB images, the authors re-rendered new depth and RGB images and regenerated ground-truth volumes from the scene meshes. The numbers for SSCNet, DCRF, and VVNetR-120 in Table I were, however, extracted from the original papers, which used a different rendering and possibly different ground-truth generation. The large improvement of SSCNet* over the reported SSCNet (65.2 vs 46.4) may therefore be partly an artifact of the new data, not only of the new training pipeline. The authors should either evaluate prior methods on the same re-rendered data or provide evidence that their renderings and regenerated volumes reproduce the original evaluation protocol.
- [IV-D, Table II] The real-data comparisons are made with numbers 'extracted from the original papers' without shared evaluation code or a verified common ground-truth voxelization. The paper uses ground truth based on [20] and [21], but it does not establish that prior work used the same voxelization, test split, and evaluation script. Since the paper's stated contribution includes achieving state-of-the-art or equivalent performance on NYUDv2, this is load-bearing: small numerical differences such as 33.7 vs 34.4 cannot be interpreted as equivalence or near-equivalence unless the evaluation protocol is identical. The authors should release their evaluation code and ground-truth generation pipeline, or run official implementations of prior methods under that same pipeline.
minor comments (5)
- [IV-D1] The text refers to 'SSNet*' in the description of the baseline retrained with the new pipeline, but Table I and the rest of the paper use 'SSCNet*'; the notation should be made consistent.
- [III-B, Figure 4] Figure 4 shows the input volume as '240x144x200' in the network diagram, while Section III-C and the rest of the text specify a 240x144x240 volume; please correct the inconsistency.
- [Abstract and Section IV-E] The abstract claims an improvement of '6.9% over the state-of-the-art result on real data, for end-to-end approaches,' but no table directly supports this specific figure; please specify the exact comparison and how it is computed, or rephrase the claim to match the reported numbers.
- [III-B] The description of the loss-weighting scheme is difficult to follow: the definition of the ratio r and the construction of the random tensor randoccl should be clarified, including how the weights are normalized across the batch, since this affects all reported results.
- [V] There is a repeated typo 'approachs' in the first paragraph of the Discussion, and Section IV-D1 uses 'Ednet' instead of 'EdgeNet'; these should be corrected.
Circularity Check
No significant circularity: EdgeNet's claims are empirical and self-contained; the one self-citation is a non-load-bearing baseline reference.
full rationale
The paper does not present a derived chain that reduces to its inputs. Its central contribution, F-TSDF-encoded projected RGB edges fused with depth, is an architectural and representational proposal whose value is established by held-out test-set experiments, not by construction. The loss is a standard weighted categorical cross-entropy with weights derived from occupancy statistics, and no target metric (e.g., IoU) is fitted into the model or used to define the representation. The ablations compare SSCNet-E vs SSCNet* and EdgeNet-EF vs EdgeNet-D on SUNCG test data, which are empirical measurements, not quantities that equal the training objective by definition. The only self-citation is Guedes et al. [8], an author-overlapping preliminary study of naive RGB projection; it is used as a failed baseline and is not load-bearing for the claimed edge-fusion advantage, which is demonstrated by the paper's own paired experiments. Comparative results 'extracted from the original papers' are external evidence whose validation protocol is unverified, but that is a correctness/evaluation concern, not circularity. Overall, no step in the paper's argument reduces to its own input or to a self-citation chain.
Assumptions & free parameters
free parameters (6)
- Voxel grid resolution and room volume dimensions =
0.02 m voxels; 240x144x240 over 4.8 x 2.88 x 4.8 m
- TSDF truncation distance =
0.24 m
- Base learning rate and One Cycle schedule =
base 0.01, peak 0.1 at epoch 10, decay to 0.0005 by epoch 30
- Training batch size =
3 samples
- Loss class-balance ratio r =
r = 2 * sum(occu) / sum(occl)
- Canny edge detector thresholds =
not disclosed
assumptions (6)
- domain assumption Canny edge detection on the RGB image yields 2D boundaries that, after projection to 3D, are a meaningful cue for semantic scene completion.
- domain assumption Edges transfer from synthetic SUNCG to real NYU without domain adaptation.
- domain assumption Scenes satisfy the Manhattan-world assumption and can be aligned with gravity and room orientation.
- domain assumption The NYUDv2 ground truth, obtained by voxelizing mesh annotations from Guo et al. [20] with category mapping from Handa et al. [21], is adequate for benchmarking, despite acknowledged severe errors.
- domain assumption Numbers reported for other methods in Tables I and II are directly comparable to the authors' own evaluation protocol.
- ad hoc to paper F-TSDF encoding of projected binary edges produces a signal that 3D CNNs can exploit as effectively as F-TSDF of depth surfaces.
Cite this review
Pith. "Pith review of EdgeNet: Semantic Scene Completion from a Single RGB-D Image." pith.science (2026). https://pith.science/paper/JRHIOFGB
@misc{pith2026190802893,
author = {Pith},
title = {Pith review of: EdgeNet: Semantic Scene Completion from a Single RGB-D Image},
year = {2026},
howpublished = {\url{https://pith.science/paper/JRHIOFGB}},
note = {Machine review of arXiv:1908.02893}
}
read the original abstract
Semantic scene completion is the task of predicting a complete 3D representation of volumetric occupancy with corresponding semantic labels for a scene from a single point of view. Previous works on Semantic Scene Completion from RGB-D data used either only depth or depth with colour by projecting the 2D image into the 3D volume resulting in a sparse data representation. In this work, we present a new strategy to encode colour information in 3D space using edge detection and flipped truncated signed distance. We also present EdgeNet, a new end-to-end neural network architecture capable of handling features generated from the fusion of depth and edge information. Experimental results show improvement of 6.9% over the state-of-the-art result on real data, for end-to-end approaches.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[20]
Predicting complete 3D models of indoor scenes,
R. Guo, C. Zou, and D. Hoiem, “Predicting complete 3D models of indoor scenes,” CoRR arXiv , vol. 1504.02437, 2015. [Online]. Available: http://arxiv.org/abs/1504.02437
arXiv 2015
-
[21]
SceneNet: Understanding real world indoor scenes with synthetic data,
A. Handa, V . Patraucean, V . Badrinarayanan, S. Stent, and R. Cipolla, “SceneNet: Understanding real world indoor scenes with synthetic data,” CoRR arXiv , vol. 1511.07041, 2015. [Online]. Available: http://arxiv.org/abs/1511.07041
arXiv 2015
-
[1]
D. Marr, Vision: A Computational Investigation into the Human Repre- sentation and Processing of Visual Information . MIT Press, 1982
work page 1982
-
[2]
Perceptual organization and recog- nition of indoor scenes from RGB-D images,
S. Gupta, P. Arbelez, and J. Malik, “Perceptual organization and recog- nition of indoor scenes from RGB-D images,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2013, pp. 564– 571
work page 2013
-
[3]
3D graph neural networks for RGBD semantic segmentation,
X. Qi, R. Liao, J. Jia, S. Fidler, and R. Urtasun, “3D graph neural networks for RGBD semantic segmentation,” in IEEE International Conference on Computer Vision (ICCV) , 10 2017, pp. 5209–5218
work page 2017
-
[4]
Rgb-(d) scene labeling: Features and algorithms,
X. Ren, L. Bo, and D. Fox, “Rgb-(d) scene labeling: Features and algorithms,” in IEEE Conference on Computer Vision and Pattern Recognition, June 2012, pp. 2759–2766
work page 2012
-
[5]
Structured prediction of unobserved voxels from a single depth image,
M. Firman, O. M. Aodha, S. Julier, and G. J. Brostow, “Structured prediction of unobserved voxels from a single depth image,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2016, pp. 5431–5440
work page 2016
-
[6]
A field model for repairing 3D shapes,
D. T. Nguyen, B. Hua, M. Tran, Q. Pham, and S. Yeung, “A field model for repairing 3D shapes,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016, pp. 5676–5684
work page 2016
Show all 25 references
-
[7]
Semantic Scene Completion from a Single Depth Image,
S. Song, F. Yu, A. Zeng, A. X. Chang, M. Savva, and T. Funkhouser, “Semantic Scene Completion from a Single Depth Image,” inConference on Computer Vision and Pattern Recognition (CVPR) , 2017
2017
-
[8]
Semantic scene completion combining colour and depth: preliminary experiments,
A. B. S. Guedes, T. E. de Campos, and A. Hilton, “Semantic scene completion combining colour and depth: preliminary experiments,” CoRR arXiv , vol. 1802.04735, 2018. [Online]. Available: http: //arxiv.org/abs/1802.04735
2018 arXiv
-
[9]
Two stream 3D semantic scene completion,
M. Garbade, J. Sawatzky, A. Richard, and J. Gall, “Two stream 3D semantic scene completion,” CoRR arXiv , vol. 1804.03550, 2018. [Online]. Available: http://arxiv.org/abs/1804.03550
2018 arXiv
-
[10]
See and think: Disentangling semantic scene completion,
S. Liu, Y . HU, Y . Zeng, Q. Tang, B. Jin, Y . Han, and X. Li, “See and think: Disentangling semantic scene completion,” in Conference on Neural Information Processing Systems (NeurIPS), S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds. Cu...
2018
-
[11]
Fully convolutional networks for semantic segmentation,
J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2015, pp. 3431–3440
2015
-
[12]
Semantic scene completion with dense CRF from a single depth image,
L. Zhang, L. Wang, X. Zhang, P. Shen, M. Bennamoun, G. Zhu, S. A. A. Shah, and J. Song, “Semantic scene completion with dense CRF from a single depth image,” Neurocomputing, vol. 318, pp. 182–195, Nov. 2018. [Online]. Available: https://linkinghub.elsevier. com/retrieve/pii/S0...
2018
-
[13]
View-V olume Network for Semantic Scene Completion from a Single Depth Image,
Y . Guo and X. Tong, “View-V olume Network for Semantic Scene Completion from a Single Depth Image,” in Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence ((IJCAI)), Stockholm, Sweden, Jul. 2018, pp. 726–732. [Online]. Available: https:/...
2018
-
[14]
DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs,
L. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 40, no. 4, pp. 834–848, April 2018
2018
-
[15]
Deep residual learning for image recognition,
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inIEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016, pp. 770–778
2016
-
[16]
A computational approach to edge detection,
J. Canny, “A computational approach to edge detection,” IEEE Trans- actions on Pattern Analysis and Machine Intelligence (PAMI) , vol. 8, no. 6, pp. 679–698, Nov 1986
1986
-
[17]
U-Net: Convolutional net- works for biomedical image segmentation,
O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional net- works for biomedical image segmentation,” in Medical Image Comput- ing and Computer-Assisted Intervention (MICCAI), N. Navab, J. Horneg- ger, W. M. Wells, and A. F. Frangi, Eds. Cham: Springer International Pu...
2015
-
[18]
Convolutional neural networks at constrained time cost,
K. He and J. Sun, “Convolutional neural networks at constrained time cost,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015
2015
-
[19]
Indoor segmentation and support inference from rgbd images,
N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, “Indoor segmentation and support inference from rgbd images,” in European Conference in Computer Vision (ECCV) . Berlin, Heidelberg: Springer Berlin Heidelberg, 2012, pp. 746–760
2012
-
[22]
A disciplined approach to neural network hyper- parameters: Part 1 - learning rate, batch size, momentum, and weight decay,
L. N. Smith, “A disciplined approach to neural network hyper- parameters: Part 1 - learning rate, batch size, momentum, and weight decay,” CoRR arXiv , vol. 1803.09820, 2018. [Online]. Available: http://arxiv.org/abs/1803.09820
2018 arXiv
-
[23]
Curriculum learning,
Y . Bengio, J. Louradour, R. Collobert, and J. Weston, “Curriculum learning,” in Proceedings of the 26th Annual International Conference on Machine Learning (ICML) . New York, NY , USA: ACM, 2009, pp. 41–48. [Online]. Available: http://doi.acm.org/10.1145/1553374. 1553380
2009 doi
-
[24]
Aarts and J
E. Aarts and J. Korst, Simulated Annealing and Boltzmann Machines: A Stochastic Approach to Combinatorial Optimization and Neural Computing. New York, NY , USA: John Wiley & Sons, Inc., 1989
1989
-
[25]
Don’t decay the learning rate, increase the batch size,
S. Smith, P. jan Kindermans, C. Ying, and Q. V . Le, “Don’t decay the learning rate, increase the batch size,” in Sixth International Conference on Learning Representations (ICLR) , 2018
2018
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.