REVIEW 3 major objections 5 minor 24 references
Standard cross-entropy with uniform weighting matches specialized imbalance-aware losses within 0.8-3.3% mIoU in point-based 3D point cloud segmentation.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 08:28 UTC pith:5P53SVIH
load-bearing objection Useful empirical benchmark on class-imbalance losses for 3D point cloud segmentation, but the landscape-topology 'why' is confounded by dataset differences — temper the causal claims. the 3 major comments →
Loss Landscape Topology Reveals Why Simple Baselines are Competitive at 3D Point Cloud Segmentation Under Class Imbalance
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central discovery is that the optimization landscape for point-based 3D segmentation is largely determined by the dataset's imbalance profile, not by the loss function. On DALES (641:1), the decision-boundary variability relative to uniform weighting correlates negatively with performance (Spearman ρ=−0.874): methods that stay close to cross-entropy's boundary reach 80–81% mIoU, while large deviations drop to about 68%. On S3DIS (56:1), all methods cluster within 62.9–64.8% mIoU regardless of boundary variability (ρ=0.768), indicating a flat landscape. Hessian eigenvalue spectra and weight-perturbation sensitivity are similar across all methods within each dataset but differ between data
What carries the argument
The argument is carried by three complementary measurements applied to converged models: (1) decision-boundary variability (DB-var), the fraction of test points labeled differently by a method versus uniform cross-entropy; (2) loss-landscape flatness via filter-normalized weight perturbations in K=20 random directions; and (3) the top-10 Hessian eigenvalue spectrum computed with Lanczos. The central object is the 'topology' of the loss landscape at the solution—narrow basin versus flat plateau—inferred from these local curvature proxies and tied to imbalance severity. The confusion-matrix decomposition (majority-to-minority vs minority-to-majority errors) supplies the precision-recall mechan
Load-bearing premise
The causal claim that imbalance severity, rather than the many other differences between DALES and S3DIS (domain, class count, input features, training length), produces the narrow-basin versus flat-plateau topology rests on a two-dataset comparison with those confounds entangled; if the confounds drive the geometry, the mechanism would not follow.
What would settle it
Train the same architecture on a single dataset resampled to multiple imbalance severities (e.g., subsample majority classes of DALES to 100:1, 200:1, 400:1, 641:1) and measure whether the Hessian spectrum transitions from flat/isotropic to narrow/anisotropic, and whether uniform CE's edge over specialized losses grows with severity. If the topology and the performance gap do not track severity within a fixed dataset, the mechanism is falsified. Alternatively, a single run where a specialized loss beats uniform CE by more than 5% mIoU on a point-based 3D segmenter would contradict the paper's
If this is right
- Practitioners can adopt uniform cross-entropy as a default loss for point-based 3D segmentation; specialized reweighting and loss functions offer at most 0.8–3.3% mIoU and can cost over 12 points if poorly tuned (e.g., inverse frequency or balanced softmax on DALES).
- Loss-level imbalance mitigation in this setting primarily trades minority recall for majority precision; gains on rare classes are offset by contamination of majority classes, so targeting imbalance through the loss alone has a ceiling.
- The topology claim implies that architecture choice (KPConv vs RandLA-Net) changes the magnitude of gains but not the qualitative pattern, so the result is likely to hold for other point-based networks.
- Dataset imbalance severity, not the loss formulation, sets the geometry of convergence; therefore comparisons of imbalance methods should report landscape proxies (e.g., Hessian spectra, perturbation sensitivity) alongside mIoU.
- A well-tuned specialized method may still be worth a few points on extreme-imbalance outdoor LiDAR, but the risk of degradation with poor tuning is substantial.
Where Pith is reading between the lines
- The data are consistent with a simpler interpretation: the two datasets differ not only in imbalance ratio but in domain, class count, and input features (RGB vs geometry-only). The causal claim that 'imbalance severity shapes topology' isolates one factor from several confounds; a cleaner test would vary imbalance within a single dataset or use synthetic resampling of one dataset to multiple seve
- If the narrow-basin mechanism generalizes, then in extreme-imbalance regimes the effective solution space is sharply limited, implying that data-level interventions (oversampling rare classes, augmentations that densify minority geometry) may be more controllable than loss-level changes, because they alter the landscape rather than the search path through it.
- The paper's proposed explanation—that neighborhood aggregation in point-based architectures provides implicit class balancing because minority classes form dense local clusters—is testable: one could ablate neighborhood size or aggregation radius and check whether the landscape flatness and CE-competitiveness shift accordingly.
- A concrete extension: compute the same landscape proxies on voxel- or transformer-based 3D segmenters; if the topology is truly a property of point-based geometry processing, other architectures should show different CE-competitiveness patterns.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper systematically evaluates 11 class-imbalance mitigation methods (six reweighting schemes and five loss functions) for point-based 3D semantic segmentation on two datasets with contrasting imbalance ratios: DALES (641:1, extreme) and S3DIS (56:1, moderate), using two architectures (KPConv and RandLA-Net). The central empirical finding is that standard cross-entropy with uniform weighting is competitive, typically within 0.8-3.3% mIoU of specialized methods, and specialized methods sometimes degrade accuracy substantially. To explain this, the paper analyzes confusion matrices, decision-boundary variability relative to CE, and loss-landscape geometry via weight-perturbation sensitivity and Hessian eigenvalue spectra. The authors conclude that imbalance severity shapes the topology of the optimization landscape, producing narrow favorable basins under extreme imbalance and flat plateaus under moderate imbalance, which constrains the benefit of loss-level modifications.
Significance. If the empirical pattern holds, the paper provides useful practical guidance and challenges routine transfer of 2D long-tail losses to 3D point cloud segmentation. The confusion-matrix precision/recall analysis is a credible explanation of why aggregate mIoU changes little even when minority recall improves. The evaluation is broad (11 methods, two architectures), and the supplements provide seed-variability evidence and per-class results. However, the paper's headline contribution is the mechanistic 'why' claim, and that claim is currently not supported by the evidence: the landscape differences are observed between two datasets that differ in many ways besides imbalance severity, and local-curvature measurements are used to infer global basin topology. These issues do not undermine the empirical comparison but are load-bearing for the paper's central narrative.
major comments (3)
- [Section 4.6, Figs. 4-5] The central mechanistic claim that 'imbalance severity shapes the topology' rests entirely on comparing DALES (641:1, outdoor aerial LiDAR, XYZ-only, 8 classes) with S3DIS (56:1, indoor, XYZ+RGB, 13 classes). These datasets differ simultaneously in domain, class count, input features, and training configuration (batch size, epochs, learning rate schedule; see Supplement S1). The statement in Section 4.6 that 'This dataset-level distinction supports our hypothesis of topological differences' is not a test of the imbalance-severity hypothesis; any of the confounds could produce the anisotropic-vs-isotropic Hessian spectra and perturbation differences. To support the causal attribution, the authors need a controlled comparison within a fixed dataset/domain, e.g., subsampling the majority classes to create multiple imbalance ratios, or matching datasets in all other respects. Without such ev
- [Section 3.4/4.6, Eq. (3)] The characterization of 'narrow basin' vs 'flat plateau' is based on K=20 random perturbation directions and top-10 Hessian eigenvalues at a single converged checkpoint per method. These quantities describe local curvature at one point, not basin topology. Terms such as 'narrow basin', 'flat plateau', and 'all methods must navigate these geometric constraints' imply global properties of the loss surface that are not measured. To infer that CE's solution lies in a narrow favorable basin on DALES, one would need analysis along optimization trajectories, mode connectivity, or multiple initialization basins, not just local sensitivity at convergence. This mismatch between the evidence and the topological language is load-bearing for the paper's mechanistic contribution and should be either fixed with additional experiments or substantially weakened in the claims.
- [Sections 4.4-4.6 and Conclusions] All mechanistic analyses (confusion-matrix precision/recall, decision-boundary variability, weight perturbation, Hessian spectra) are performed only with KPConv. RandLA-Net evidence consists of per-class performance tables (Tables S2-S3) and aggregate mIoU, with no decision-boundary or landscape analysis. The abstract and conclusion nevertheless state that the findings are 'validated across two representative architectures' and claim a general mechanism for point-based 3D segmentation. The performance replication on RandLA-Net is a strength, but the causal mechanism is not validated there. Either the mechanistic analysis should be extended to RandLA-Net, or the abstract/conclusion should explicitly restrict the mechanism claim to KPConv and present the RandLA-Net results only as evidence that the empirical pattern is not architecture-specific.
minor comments (5)
- [Supplement S2.3] The imbalance ratios are inconsistent: the main text and S2.1 say DALES is 641:1 and S3DIS is 56:1, while S2.3 states 643:1 and 55:1. Please harmonize.
- [Abstract vs Sections 1 and 5] The margin by which CE is competitive is reported as 'within 0.8-3.3% mIoU' in the abstract, '0.6-3.3%' in the introduction, and '0.8-1.8%' (KPConv) / '2.4-3.3%' (RandLA-Net) in the conclusion. Please reconcile these numbers.
- [Section 4.5] The S3DIS Spearman correlation (rho=0.768, p=0.009) is described as 'weak positive'; with 11 points, rho=0.77 is better described as a moderate-to-strong correlation. The qualitative conclusion is unaffected, but the wording is misleading.
- [Eq. (1)] The symbol N is used for the number of test points in Eq. (1), but earlier in Eq. (4) and Section 3.1 N denotes the number of points in the dataset. Please introduce a distinct notation (e.g., N_test) for clarity.
- [Section 4.4] The phrase 'inflate majority-to-minority errors by 10x (1.58%, 1.50%)' is confusing because the two percentages refer to invf and BalSoft, but the sentence could be read as listing two error percentages for invf. Consider rewriting to make the method-to-value mapping explicit.
Circularity Check
No circularity: the paper reports measured comparisons and external mechanistic analyses; central claims are not folded into the inputs.
full rationale
The paper is an empirical evaluation and mechanistic measurement study, not a derivation. The central claim that uniform cross-entropy is competitive with specialized imbalance losses is supported by directly measured mIoU tables (Tables 1-2, S2-S3) and seed-variability checks (Table S1); no parameter is fitted to this outcome and no 'prediction' is constructed from the target result. Hyperparameters such as β=0.9, γ=1, and τ=0.3 are taken from the cited original methods rather than tuned to force the comparison. The mechanistic analyses (decision-boundary variability Eq. (1), weight-perturbation sensitivity Eqs. (2)-(3), and Hessian eigenvalue spectra) are external measurements with stated definitions; they do not define the outcome in terms of the explanation. The only fragile point is that the causal attribution 'imbalance severity shapes the topology' is inferred from comparing DALES with S3DIS, which differ in domain, class count, and features. That is a confound/validity limitation, not a circularity: the inference does not reduce to the inputs by construction, and the paper explicitly labels the mechanistic extension to RandLA-Net as future work. No self-citation is load-bearing, and no known result is renamed as a derivation. Therefore no specific circular step can be exhibited, and the appropriate score is 0.
Axiom & Free-Parameter Ledger
free parameters (6)
- class-balanced beta =
0.9
- inverse-power gamma =
0.1
- focal loss gamma =
1
- logit adjustment tau =
0.3
- LDAM margin scale =
not stated
- number of perturbation directions K =
20
axioms (5)
- ad hoc to paper DALES vs S3DIS differ mainly in imbalance severity; other dataset/architecture differences do not explain landscape differences.
- domain assumption Flatness of the loss around a converged solution (weight perturbation, top-10 Hessian eigenvalues) characterizes the optimization landscape relevant to method comparison.
- domain assumption Decision-boundary variability against the uniform-CE model is a meaningful measure of how much a loss changes the learned partition.
- domain assumption Single-seed runs for most methods are representative; the three-seed check for two methods suffices.
- ad hoc to paper Point-based architectures implicitly perform class balancing through local neighborhood aggregation.
read the original abstract
Semantic segmentation of 3D point clouds faces severe class imbalance, yet the effectiveness of specialized imbalance-aware methods from 2D computer vision remains unclear in 3D contexts. We systematically evaluate 11 imbalance mitigation approaches across datasets with extreme (641:1) and moderate (56:1) imbalance ratios, revealing a surprising finding: standard cross-entropy with uniform weighting achieves competitive performance, typically within 0.8-3.3% mIoU of specialized methods across architectures and datasets. Through multifaceted mechanistic analysis of error patterns, decision boundaries, and the geometry of the optimization landscape, our analyses suggest that imbalance severity shapes the topology, creating narrow solution basins under extreme imbalance and flat plateaus under moderate imbalance. This appears to constrain the effectiveness of loss-level modifications, as all methods must navigate these geometric constraints. Our findings offer practical guidance; standard cross-entropy provides a robust baseline, with specialized methods offering modest improvements (0.8-3.3% mIoU) that vary by architecture and dataset but risk substantial degradation if poorly tuned. This work provides the first mechanistic explanation for why techniques proven effective in 2D do not readily transfer to point-based 3D point cloud segmentation, validated across two representative architectures.
Figures
Reference graph
Works this paper leans on
-
[1]
In: 2016 IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR)
Armeni, I., Sener, O., Zamir, A.R., Jiang, H., Brilakis, I., Fischer, M., Savarese, S.: 3D Semantic Parsing of Large-Scale Indoor Spaces. In: 2016 IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR). pp. 1534–1543 (2016). https://doi.org/10.1109/CVPR.2016.170 14 A. Savva et al
-
[2]
In: 2019 IEEE/CVF International Conference on Computer Vi- sion (ICCV)
Behley, J., Garbade, M., Milioto, A., Quenzel, J., Behnke, S., Stachniss, C., Gall, J.: SemanticKITTI: A Dataset for Semantic Scene Understanding of Li- DAR Sequences. In: 2019 IEEE/CVF International Conference on Computer Vi- sion (ICCV). pp. 9296–9306 (2019). https://doi.org/10.1109/ICCV.2019.00939
arXiv 2019
-
[3]
Journal of Hydrology617, 128975 (2023)
Bodoque, J.M., Aroca-Jiménez, E., Eguibar, M.A., García, J.A.: Developing Reli- able Urban Flood Hazard Mapping from LiDAR Data. Journal of Hydrology617, 128975 (2023). https://doi.org/https://doi.org/10.1016/j.jhydrol.2022.128975
arXiv 2023
-
[4]
In: Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., Garnett, R
Cao, K., Wei, C., Gaidon, A., Arechiga, N., Ma, T.: Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss. In: Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neu- ral Information Processing Systems. vol. 32. Curran Associates, Inc. (2019)
2019
-
[5]
In: 2017 IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR)
Charles, R.Q., Su, H., Kaichun, M., Guibas, L.J.: PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. In: 2017 IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR). pp. 77–85 (2017). https://doi.org/10.1109/CVPR.2017.16
-
[6]
In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Cui, Y., Jia, M., Lin, T.Y., Song, Y., Belongie, S.: Class-Balanced Loss Based on Effective Number of Samples. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 9260–9269 (2019). https://doi.org/10.1109/CVPR.2019.00949
arXiv 2019
-
[7]
In: 2019 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR)
Gupta, A., Dollár, P., Girshick, R.: LVIS: A Dataset for Large Vocab- ulary Instance Segmentation. In: 2019 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR). pp. 5351–5359 (2019). https://doi.org/10.1109/CVPR.2019.00550
arXiv 2019
-
[8]
Information Fusion115, 102722 (2025)
He, Y., Yu, H., Liu, X., Yang, Z., Sun, W., Anwar, S., Mian, A.: Deep Learning based 3D Segmentation in Computer Vision: A Survey. Information Fusion115, 102722 (2025). https://doi.org/https://doi.org/10.1016/j.inffus.2024.102722
arXiv 2025
-
[9]
In: 2020 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR)
Hu, Q., Yang, B., Xie, L., Rosa, S., Guo, Y., Wang, Z., Trigoni, N., Markham, A.: RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds. In: 2020 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR). pp. 11105–11114 (2020). https://doi.org/10.1109/CVPR42600.2020.01112
arXiv 2020
-
[10]
IEEE Transactions on Neural Networks and Learning Systems36(1), 1533–1544 (2025)
Lei, S., He, F., Yuan, Y., Tao, D.: Understanding Deep Learning via Decision Boundary. IEEE Transactions on Neural Networks and Learning Systems36(1), 1533–1544 (2025). https://doi.org/10.1109/TNNLS.2023.3326654
arXiv 2025
-
[11]
In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R
Li, H., Xu, Z., Taylor, G., Studer, C., Goldstein, T.: Visualizing the Loss Land- scape of Neural Nets. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 31. Curran Associates, Inc. (2018)
2018
-
[12]
In: 2017 IEEE International Conference on Computer Vision (ICCV)
Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal Loss for Dense Object Detection. In: 2017 IEEE International Conference on Computer Vision (ICCV). pp. 2999–3007 (2017). https://doi.org/10.1109/ICCV.2017.324
-
[13]
In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Liu, Z., Miao, Z., Zhan, X., Wang, J., Gong, B., Yu, S.X.: Large-scale long-tailed recognition in an open world. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 2532–2541 (2019). https://doi.org/10.1109/CVPR.2019.00264
arXiv 2019
-
[14]
Menon, A.K., Jayasumana, S., Rawat, A.S., Jain, H., Veit, A., Kumar, S.: Long-tail learning via logit adjustment (2021), https://arxiv.org/abs/2007.07314
Pith/arXiv arXiv 2021
-
[15]
IEEE Trans- actions on Intelligent Transportation Systems24(7), 6955–6970 (2023)
Pan, Y., Xie, F., Zhao, H.: Understanding the challenges when 3d se- mantic segmentation faces class imbalanced and ood data. IEEE Trans- actions on Intelligent Transportation Systems24(7), 6955–6970 (2023). https://doi.org/10.1109/TITS.2023.3256442 Loss Landscape Topology in 3D Segmentation 15
arXiv 2023
-
[16]
https://doi.org/https://arxiv.org/abs/2304.13216
Prakash, S., Shah, P., Agrawal, A.: Exploiting CNNs for Semantic Segmentation with Pascal VOC (2023). https://doi.org/https://arxiv.org/abs/2304.13216
Pith/arXiv arXiv 2023
-
[17]
Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: deep hierarchical feature learn- ing on point sets in a metric space (2017), https://arxiv.org/abs/1706.02413
Pith/arXiv arXiv 2017
-
[18]
In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H
Ren, J., Yu, C., Sheng, S., Ma, X., Zhao, H., Yi, S., Li, H.: Balanced Meta-Softmax for Long-Tailed Visual Recognition. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H. (eds.) Advances in Neural Information Processing Systems. vol. 33, pp. 4175–4186. Curran Associates, Inc. (2020)
2020
-
[19]
In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., Guibas, L.: KPConv: Flexible and Deformable Convolution for Point Clouds. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). pp. 6410–6419 (2019). https://doi.org/10.1109/ICCV.2019.00651
arXiv 2019
-
[20]
In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recog- nition
Van Horn, G., Mac Aodha, O., Song, Y., Cui, Y., Sun, C., Shepard, A., Adam, H., Perona, P., Belongie, S.: The iNaturalist Species Classification and Detection Dataset. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recog- nition. pp. 8769–8778 (2018). https://doi.org/10.1109/CVPR.2018.00914
arXiv 2018
-
[21]
In: 2020 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition Workshops (CVPRW)
Varney, N., Asari, V.K., Graehling, Q.: DALES: A Large-scale Aerial LiDAR Data Set for Semantic Segmentation. In: 2020 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition Workshops (CVPRW). pp. 717–726 (2020). https://doi.org/10.1109/CVPRW50498.2020.00101
arXiv 2020
-
[22]
In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Wang, J., Zhang, W., Zang, Y., Cao, Y., Pang, J., Gong, T., Chen, K., Liu, Z., Loy, C.C., Lin, D.: Seesaw Loss for Long-Tailed Instance Segmentation. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 9690–9699 (2021). https://doi.org/10.1109/CVPR46437.2021.00957
arXiv 2021
-
[23]
In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR)
Wu, W., Qi, Z., Fuxin, L.: PointConv: Deep Convolutional Networks on 3D Point Clouds. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR). pp. 9613–9622 (2019). https://doi.org/10.1109/CVPR.2019.00985
arXiv 2019
-
[24]
IEEE Transactions on Pattern Analysis and Machine Intelligence45(9), 10795–10816 (2023)
Zhang, Y., Kang, B., Hooi, B., Yan, S., Feng, J.: Deep long-tailed learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence45(9), 10795–10816 (2023). https://doi.org/10.1109/TPAMI.2023.3268118 Loss Landscape Topology in 3D Segmentation 1 Loss Landscape Topology Reveals Why Simple Baselines are Competitive at 3D Point Cloud Segmen...
arXiv 2023
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.