REVIEW 3 major objections 5 minor 37 references
Exploring Active Learning for Label-Efficient Training of Semantic Neural Radiance Field
T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Active learning cuts semantic NeRF annotation cost by half
desk verdict Useful empirical study of active learning for semantic NeRF, but the novel spatial-diversity term is not statistically supported; Entropy alone already delivers the headline >2x cost reduction. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the max-min diversification objective with a combined distance function $d'(x, y) = \frac{1}{2}(u(x) + u(y)) + d(x, y)$, where $u$ is prediction entropy and $d(x, y) = d_f(x, y) + d_s(x, y)$ is the sum of feature L2 distance and 3D spatial L2 distance between superpixels. The 3D coordinates for the spatial term come from volume-rendered depth (Equation 5): each pixel's depth is the expectation along its ray, giving a 3D point per superpixel. Selection is solved greedily with a 2-approximation algorithm (Algorithm 1), keeping complexity $O(n_b \cdot n_u \cdot f_{\text{dim}})$, comparable to CoreSet's k-center greedy.
What would settle it
Retrain the proposed selection on a scene while replacing the predicted depth in Equation 5 with ground-truth depth from the RGB-D sensors used by Replica and ScanNet; if the predicted-depth version does not lose noticeable mIoU relative to the ground-truth-depth version, or if the advantage over Entropy shrinks, then the 3D spatial term is not doing the work claimed.
Extended reading notes
Core claim
On the paper's terms, the central discovery is that a hybrid active-learning criterion combining prediction entropy, feature diversity, and a newly added 3D spatial-diversity term consistently outperforms Entropy, CoreSet, ViewAL, and random sampling for training Semantic-NeRF on Replica and ScanNet indoor scenes across annotation budgets. Measured by the budget needed to match random sampling at 20% annotation, the proposed method needs only about 9.3 to 9.5 percent of regions, a more than two-fold reduction in annotation cost. The paper also reports that superpixel-level selection is more cost-effective than image-level selection, and that ViewAL's viewpoint-consistency criterion is ineffective for NeRF because NeRF predictions are multi-view consistent by construction.
Load-bearing premise
The 3D spatial-diversity term assumes the volume-rendered depth from the current Semantic-NeRF model is accurate enough to place superpixels at their true 3D locations, even in early active-learning rounds when the model was trained on only five percent of labelled regions.
Editorial extensions
If this is right
- Annotation budgets for semantic NeRF can be cut by more than half while keeping the same segmentation quality.
- Region-level (superpixel) active selection is more cost-effective and more stable than image-level selection for Semantic-NeRF.
- Hybrid uncertainty-plus-diversity selection beats either criterion alone across scenes with different characteristics.
- Viewpoint-consistency cues such as those in ViewAL add little for NeRF because the semantic branch is already viewpoint invariant.
Reading between the lines
- The 3D spatial-diversity term could generalize to other NeRF-based label-efficient tasks, such as panoptic segmentation or open-vocabulary segmentation, where geometry is also available from the rendering pipeline.
- Because the spatial term depends on predicted depth, a testable extension is to weight the spatial term by estimated depth confidence or to switch to ground-truth depth when available; the paper does not ablate this dependence.
- The same max-min diversification framing could be applied to other 3D scene representations, for example 3D Gaussian splatting, without re-deriving the greedy solver.
- The two-fold reduction is measured against random sampling at a fixed 20% budget on four indoor scenes; the claim generalizes only if the trend holds across more scenes and tighter budgets.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This manuscript investigates active learning for Semantic-NeRF, comparing selection at image and superpixel granularity and several selection strategies (Random, Entropy, CoreSet, ViewAL, and a proposed hybrid). The proposed hybrid adds a 3D spatial diversity term, computed from volume-rendered depths, to entropy and feature diversity, and solves selection via a greedy diversification algorithm claimed to be a 2-approximation. Experiments on Replica and ScanNet scenes over four annotation batches, with three runs each, report that active learning reduces annotation cost by more than 2x relative to random sampling, and the hybrid is claimed to consistently outperform all baselines.
Significance. The paper addresses a timely and underexplored problem with a reasonable experimental design: standard datasets, retraining from scratch each round, and repeated runs with error bars. Its clearest finding—that active learning, including simple entropy, substantially reduces the annotation budget for Semantic-NeRF—is useful to the community. The specific novel contribution, 3D spatial diversity, is not statistically supported, and the theoretical guarantee is under-justified. With strengthened evidence or appropriately narrowed claims, the study could be a worthwhile contribution.
major comments (3)
- [Section IV-C, Table I, Table III] The claim that the proposed Entropy+Feature+Spatial strategy 'consistently outperforms' Entropy is not supported by the reported numbers. In Table I, the required annotation budgets for Ours vs. Entropy differ by only 0.06, 0.18, 0.28, and 0.09 percentage points across the four scenes; in Table III, adding the spatial term to Entropy+Feature improves mIoU by 0.32, 0.39, and 0.02 at batches 1–3 on Room0, while standard deviations across three runs reach 0.76. No significance tests, confidence intervals, or paired analysis are reported, so the differences are within run-to-run noise. This matters because Entropy alone already achieves the headline 'more than 2x reduction' (9.4–9.6% vs. 20%), so the paper does not demonstrate that the proposed component contributes to the central claim.
- [Section III-A, Eq. (3), Algorithm 1] The claim that the greedy algorithm in Algorithm 1 provides a 2-approximation for the combined max-min objective in Eq. (2) is asserted without proof. The reference [31] addresses a specific diversification objective, and it is not immediate that the distance redefinition in Eq. (3) preserves the approximation guarantee when unary uncertainty terms are mixed with pairwise distances. Please provide a formal proof or a precise statement of the applicable theorem with conditions.
- [Section III-B, Eq. (5)–(6)] The spatial diversity term relies on 3D coordinates C(x) obtained from volume-rendered depths of the current Semantic-NeRF, which is trained with very sparse labels in early batches. The paper does not evaluate depth accuracy or its influence on selection quality. Since the effectiveness of the spatial term is a central claim, this omission leaves the small observed gains unexplained: they could stem from geometry errors rather than from meaningful 3D diversity.
minor comments (5)
- [Section I] The phrase 'The most close work' should be 'The closest work.'
- [Section III-B] The normalization of d_f and d_s to [0,1] is not described; please specify how the scaling is computed (e.g., per-batch min-max normalization) and whether it affects the metric property claimed in Eq. (6).
- [Section IV-C, Table I] Table I reports interpolated budgets but does not state the interpolation method or provide confidence bounds; please add this information.
- [Section IV-E, Table III] The ablation table would be clearer with a column for batch 0 and with explicit differences between variants rather than only absolute means and standard deviations.
- [Appendix B] The supplementary plots in Fig. 6 include error bars only in the text description; please ensure the captions and figures clearly indicate the number of runs and the meaning of the bars.
Circularity Check
No significant circularity: the active-learning loop and its evaluation are measured against held-out labels, and co-author self-citations are not load-bearing.
full rationale
The paper's derivation chain is a standard active-learning loop: train Semantic-NeRF on labelled superpixels, score unlabelled superpixels with entropy and feature/spatial distances (Eqs. 4-6), select via max-min diversification (Eqs. 1-3), relabel, retrain, and evaluate mIoU on held-out test frames. The selection objective in Eq. (2) does not contain the evaluation target or the final annotation-cost number; it is computed from the current model's outputs and rendered geometry. There is no step where a claimed prediction is defined in terms of the fitted quantity or where a fitted parameter is later renamed as a prediction. The 'more than 2x reduction' claim is an interpolated reading of the measured active-learning curves in Table I, not a fitted quantity used as evidence for itself. The citations to co-author Lile Cai's prior work (UWE [17], superpixel active learning [20], region-based spatial diversity [21]) appear in related work and as background; the proposed 3D spatial-diversity term is evaluated on external Replica and ScanNet benchmarks and does not depend on those papers for its validity. The skeptic's observation that plain Entropy nearly matches the hybrid method in Table I and that Table III differences are within run-to-run noise is a statistical-strength concern, not a circularity concern: even if the novel spatial term contributed little, the derivation would still be non-circular. No load-bearing reduction to inputs or self-citation chain was found.
Assumptions & free parameters
free parameters (3)
- Weight of feature diversity relative to spatial diversity in distance function =
1 (equal weighting, unoptimized)
- Number of superpixels per image =
300
- Per-batch annotation budget increment =
5% of superpixels
assumptions (4)
- domain assumption Volume rendering depth (Eq. 5) from the current Semantic-NeRF model provides accurate enough 3D coordinates for spatial diversity.
- domain assumption L2 distance in 3D space correlates with semantic/appearance diversity.
- ad hoc to paper The combined distance d' = (u(x)+u(y))/2 + d(x,y) permits the 2-approximation greedy algorithm from [31] for the max-min diversification objective Eq. (2).
- domain assumption Each superpixel is label-homogeneous and annotation cost is uniform per superpixel.
Cite this review
Pith. "Pith review of Exploring Active Learning for Label-Efficient Training of Semantic Neural Radiance Field." pith.science (2026). https://pith.science/paper/JYD7K3Y3
@misc{pith2026250717351,
author = {Pith},
title = {Pith review of: Exploring Active Learning for Label-Efficient Training of Semantic Neural Radiance Field},
year = {2026},
howpublished = {\url{https://pith.science/paper/JYD7K3Y3}},
note = {Machine review of arXiv:2507.17351}
}
read the original abstract
Neural Radiance Field (NeRF) models are implicit neural scene representation methods that offer unprecedented capabilities in novel view synthesis. Semantically-aware NeRFs not only capture the shape and radiance of a scene, but also encode semantic information of the scene. The training of semantically-aware NeRFs typically requires pixel-level class labels, which can be prohibitively expensive to collect. In this work, we explore active learning as a potential solution to alleviate the annotation burden. We investigate various design choices for active learning of semantically-aware NeRF, including selection granularity and selection strategies. We further propose a novel active learning strategy that takes into account 3D geometric constraints in sample selection. Our experiments demonstrate that active learning can effectively reduce the annotation cost of training semantically-aware NeRF, achieving more than 2X reduction in annotation cost compared to random sampling.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[31]
An axiomatic approach for result diversification,
Sreenivas Gollapudi and Aneesh Sharma, “An axiomatic approach for result diversification,” in Proceedings of the 18th international conference on World wide web , 2009, pp. 381–390. 3
work page 2009
-
[1]
Nerf: Representing scenes as neural radiance fields for view synthesis,
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,” Communications of the ACM, vol. 65, no. 1, pp. 99–106, 2021. 1, 2
work page 2021
-
[2]
Nerf: Neural radiance field in 3d vision, a comprehensive review,
Kyle Gao, Yina Gao, Hongjie He, Dening Lu, Linlin Xu, and Jonathan Li, “Nerf: Neural radiance field in 3d vision, a comprehensive review,” arXiv preprint arXiv:2210.00379 , 2022. 1
arXiv 2022
-
[3]
Nerf++: Analyzing and improving neural radiance fields,
Kai Zhang, Gernot Riegler, Noah Snavely, and Vladlen Koltun, “Nerf++: Analyzing and improving neural radiance fields,” arXiv preprint arXiv:2010.07492, 2020. 1, 2
arXiv 2010
-
[4]
pixelnerf: Neural radiance fields from one or few images,
Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa, “pixelnerf: Neural radiance fields from one or few images,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 4578–4587. 1, 2
work page 2021
-
[5]
In-place scene labelling and understanding with implicit scene representation,
Shuaifeng Zhi, Tristan Laidlow, Stefan Leutenegger, and Andrew J Davison, “In-place scene labelling and understanding with implicit scene representation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 15838–15847. 1, 2
work page 2021
-
[6]
A survey of deep active learning,
Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po-Yao Huang, Zhihui Li, Brij B Gupta, Xiaojiang Chen, and Xin Wang, “A survey of deep active learning,” ACM computing surveys (CSUR) , vol. 54, no. 9, pp. 1–40,
-
[7]
Viewal: Active learning with viewpoint entropy for semantic segmentation,
Yawar Siddiqui, Julien Valentin, and Matthias Nießner, “Viewal: Active learning with viewpoint entropy for semantic segmentation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 9433–9443. 1, 2, 4
work page 2020
Show all 37 references
-
[8]
Encoder-decoder with atrous separable convolution for semantic image segmentation,
Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam, “Encoder-decoder with atrous separable convolution for semantic image segmentation,” in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 801–818. 1
2018
-
[9]
Unsupervised continual semantic adaptation through neural rendering,
Zhizheng Liu, Francesco Milano, Jonas Frey, Roland Siegwart, Hermann Blum, and Cesar Cadena, “Unsupervised continual semantic adaptation through neural rendering,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 3031–3040. 2
2023
-
[10]
Panoptic nerf: 3d- to-2d label transfer for panoptic urban scene segmentation,
Xiao Fu, Shangzhan Zhang, Tianrun Chen, Yichong Lu, Lanyun Zhu, Xiaowei Zhou, Andreas Geiger, and Yiyi Liao, “Panoptic nerf: 3d- to-2d label transfer for panoptic urban scene segmentation,” in 2022 International Conference on 3D Vision (3DV) . IEEE, 2022, pp. 1–11. 2
2022
-
[11]
Weakly supervised 3d open-vocabulary segmentation,
Kunhao Liu, Fangneng Zhan, Jiahui Zhang, Muyu Xu, Yingchen Yu, Abdulmotaleb El Saddik, Christian Theobalt, Eric Xing, and Shijian Lu, “Weakly supervised 3d open-vocabulary segmentation,” Advances in Neural Information Processing Systems , vol. 36, pp. 53433–53456,
-
[12]
ilabel: Interactive neural scene labelling,
Shuaifeng Zhi, Edgar Sucar, Andre Mouton, Iain Haughton, Tristan Laidlow, and Andrew J Davison, “ilabel: Interactive neural scene labelling,” arXiv preprint arXiv:2111.14637 , 2021. 2
2021 arXiv
-
[13]
Scene-generalizable interactive segmentation of radiance fields,
Songlin Tang, Wenjie Pei, Xin Tao, Tanghui Jia, Guangming Lu, and Yu-Wing Tai, “Scene-generalizable interactive segmentation of radiance fields,” in Proceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 6744–6755. 2
2023
-
[14]
The power of ensembles for active learning in image classifi- cation,
William H Beluch, Tim Genewein, Andreas N ¨urnberger, and Jan M K¨ohler, “The power of ensembles for active learning in image classifi- cation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 9368–9377. 2
2018
-
[15]
Active learning for convolutional neu- ral networks: A core-set approach,
Ozan Sener and Silvio Savarese, “Active learning for convolutional neu- ral networks: A core-set approach,” arXiv preprint arXiv:1708.00489 ,
-
[16]
Deep batch active learning by diverse, uncertain gradient lower bounds,
Jordan T Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal, “Deep batch active learning by diverse, uncertain gradient lower bounds,” arXiv preprint arXiv:1906.03671 , 2019. 2
1906 arXiv
-
[17]
Hybrid active learning with uncertainty-weighted embeddings,
Yinan He, Lile Cai, Jingyi Liao, and Chuan-Sheng Foo, “Hybrid active learning with uncertainty-weighted embeddings,” Transactions on Machine Learning Research , 2024. 2
2024
-
[18]
Active self-semi- supervised learning for few labeled samples,
Ziting Wen, Oscar Pizarro, and Stefan Williams, “Active self-semi- supervised learning for few labeled samples,” Neurocomputing, p. 128772, 2024. 2
2024
-
[19]
Reinforced active learning for image segmentation,
Arantxa Casanova, Pedro O Pinheiro, Negar Rostamzadeh, and Christo- pher J Pal, “Reinforced active learning for image segmentation,” arXiv preprint arXiv:2002.06583, 2020. 2
2002 arXiv
-
[20]
Revisiting superpixels for active learning in semantic segmentation with realistic annotation costs,
Lile Cai, Xun Xu, Jun Hao Liew, and Chuan Sheng Foo, “Revisiting superpixels for active learning in semantic segmentation with realistic annotation costs,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2021, pp. 10988–10997. 2
2021
-
[21]
Exploring spatial diversity for region-based active learning,
Lile Cai, Xun Xu, Lining Zhang, and Chuan-Sheng Foo, “Exploring spatial diversity for region-based active learning,” IEEE Transactions on Image Processing , vol. 30, pp. 8702–8712, 2021. 2
2021
-
[22]
Entropy-based active learning for object detection with progressive diversity constraint,
Jiaxi Wu, Jiaxin Chen, and Di Huang, “Entropy-based active learning for object detection with progressive diversity constraint,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 9397–9406. 2
2022
-
[23]
Box-level active detection,
Mengyao Lyu, Jundong Zhou, Hui Chen, Yijie Huang, Dongdong Yu, Yaqian Li, Yandong Guo, Yuchen Guo, Liuyu Xiang, and Guiguang Ding, “Box-level active detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 23766–23775. 2
2023
-
[24]
Box-level class-balanced sampling for active object detection,
Jingyi Liao, Xun Xu, Chuan-Sheng Foo, and Lile Cai, “Box-level class-balanced sampling for active object detection,” in 2024 IEEE International Conference on Image Processing (ICIP) . IEEE, 2024, pp. 701–707. 2
2024
-
[25]
Employing feature mixture for active learning of object detection,
Licheng Zhang, Siew-Kei Lam, Dingsheng Luo, and Xihong Wu, “Employing feature mixture for active learning of object detection,” Neurocomputing, vol. 594, pp. 127883, 2024. 2
2024
-
[26]
Learning loss for active learning,
Donggeun Yoo and In So Kweon, “Learning loss for active learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 93–102. 2
2019
-
[27]
Influence selection for active learning,
Zhuoming Liu, Hao Ding, Huaping Zhong, Weijia Li, Jifeng Dai, and Conghui He, “Influence selection for active learning,” in Proceedings of the IEEE/CVF international conference on computer vision , 2021, pp. 9274–9283. 2
2021
-
[28]
Hal: Hybrid active learning for efficient labeling in medical domain,
Xing Wu, Cheng Chen, Mingyu Zhong, and Jianjia Wang, “Hal: Hybrid active learning for efficient labeling in medical domain,” Neurocomput- ing, vol. 456, pp. 563–572, 2021. 2
2021
-
[29]
A mathematical theory of communication,
Claude Elwood Shannon, “A mathematical theory of communication,” The Bell system technical journal , vol. 27, no. 3, pp. 379–423, 1948. 2, 4
1948
-
[30]
Activenerf: Learning where to see with uncertainty estimation,
Xuran Pan, Zihang Lai, Shiji Song, and Gao Huang, “Activenerf: Learning where to see with uncertainty estimation,” in European Conference on Computer Vision . Springer, 2022, pp. 230–246. 2
2022
-
[32]
Randomness is the root of all evil: more reliable evaluation of deep active learning,
Yilin Ji, Daniel Kaestner, Oliver Wirth, and Christian Wressnegger, “Randomness is the root of all evil: more reliable evaluation of deep active learning,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2023, pp. 3943–3952. 3
2023
-
[33]
Semantically-aware neural radiance fields for visual scene understanding: A comprehensive review,
Thang-Anh-Quan Nguyen, Amine Bourki, M ´aty´as Macudzinski, An- thony Brunel, and Mohammed Bennamoun, “Semantically-aware neural radiance fields for visual scene understanding: A comprehensive review,” arXiv preprint arXiv:2402.11141 , 2024. 3
2024 arXiv
-
[34]
The replica dataset: A digital replica of indoor spaces,
Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J. Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, Anton Clarkson, Mingfei Yan, Brian Budge, Yajie Yan, Xiaqing Pan, June Yon, Yuyang Zou, Kimberly Leon, Nigel Carter, Jesus Briales, Tyler Gi...
2019
-
[35]
Scannet: Richly-annotated 3d reconstructions of indoor scenes,
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner, “Scannet: Richly-annotated 3d reconstructions of indoor scenes,” 2017. 4
2017
-
[36]
Adam: A method for stochastic optimization,
Diederik P Kingma and Jimmy Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 , 2014. 4
2014 arXiv
-
[37]
Seeds: Superpixels extracted via energy-driven sampling,
Michael Van den Bergh, Xavier Boix, Gemma Roig, and Luc Van Gool, “Seeds: Superpixels extracted via energy-driven sampling,” 2013. 4 APPENDIX A. Qualitative Results We visualize the regions selected by various methods for scene Room0 of the Replica dataset in Fig. 5. In the se...
2013
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.