D-PLS: Decoupled Semantic Segmentation for 4D-Panoptic-LiDAR-Segmentation

Laurenz Reichardt; Maik Steinhauser; Nikolas Ebert; Oliver Wasenm\"uller

REVIEW 2 major objections 5 minor 18 references

D-PLS: Decoupled Semantic Segmentation for 4D-Panoptic-LiDAR-Segmentation

T0 review · 2 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read D-PLS claims 4D panoptic LiDAR segmentation improves by running single-scan semantic segmentation first and feeding those labels as a prior into a frozen instance-segmentation branch, reaching LSTQ 70.49 on SemanticKITTI with N=4 scans.

desk verdict Useful modular idea with an untested decoupling claim—the gain over baseline likely owes more to the 2DPASS semantic backbone than to decoupling itself. read the letter →

arxiv 2501.15870 v1 pith:I3VRM27B submitted 2025-01-27 cs.CV cs.AI

Maik Steinhauser , Laurenz Reichardt , Nikolas Ebert , Oliver Wasenmüller This is my paper · ORCID

classification cs.CVcs.AI

keywords 4DpanopticsegmentationLiDARpointcloudssemanticinstancedecoupledarchitecturetemporalaggregationKITTILSTQmetric

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

D-PLS sets out to show that 4D panoptic LiDAR segmentation does not need to be a monolithic joint task: it can be split into single-scan semantic segmentation followed by temporal instance segmentation, with the semantic labels acting as a coarse pre-clustering prior. On SemanticKITTI validation, the decoupled pipeline reaches LSTQ 70.49 when four scans are aggregated, compared with 66.40 for the 4D-StOP baseline retrained under the same conditions, and it outperforms all compared methods except Mask4D. If the argument holds, panoptic segmentation becomes a bolt-on module that automatically inherits progress from the active single-scan semantic segmentation community. The authors also claim the design avoids the information loss of approaches that mask out background points before instance clustering.

What carries the argument

The machinery is a two-stage pipeline whose load-bearing object is the augmented 4D point cloud $P = \mathbb{R}^{N \times [x,y,z,f,S]}$: a frozen single-scan semantic network (a 2DPASS-trained SPVCNN variant) labels each scan, those labels are appended to the point coordinates and features, and $N$ scans are fused with ego-motion poses. The instance branch then processes this cloud with a KPConv encoder-decoder that predicts per-point offsets $\Delta p$ toward instance centers, followed by the 4D-StOP proposal module (farthest-point sampling, radius grouping, a PointNet-style MLP for refined centers, radii, and boxes) and DBSCAN clustering into final masks. Semantic labels enter as input features and as a mask on the loss for background points, but they never delete points, preserving geometry. The one-hot semantic prior is the 'coarse clustering' that the offset and proposal machinery refines into spatio-temporally consistent instances.

What would settle it

Retrain 4D-StOP under the same point budget and training schedule with the same frozen 2DPASS one-hot semantic labels appended to every aggregated point, but keep its joint semantic-instance architecture. If that augmented 4D-StOP matches or beats D-PLS's LSTQ 70.49 with N=4 scans, the decoupling claim is falsified and the measured gain belongs to the semantic backbone.

Watch

Extended reading notes

Core claim

The central claim is that feeding one-hot semantic predictions from a frozen, pre-trained single-scan segmenter into a temporally aggregated 4D point cloud, without masking any points, improves both the semantic and the association components of 4D panoptic segmentation. In the N=2 ablation on SemanticKITTI validation, adding one-hot semantics lifts LSTQ from 58.01 (baseline 4D-StOP) to 69.50, with the classification score Scls rising from 51.38 to 66.17 and the association score Sassoc rising from 65.50 to 73.00. With N=4 scans, D-PLS reaches LSTQ 70.49 (Sassoc 74.87, Scls 66.36), ahead of 4D-StOP, 4D-DS-Net, Eq-4D-StOP, and Mask4Former, and behind only Mask4D at 71.40. The paper attributes part of the classification gain to the single-scan semantic backbone itself, and supports the decoupling hypothesis by showing that an auxiliary semantic loss inside the instance stage hurts rather than helps.

Load-bearing premise

The load-bearing premise is that the gap over 4D-StOP comes from decoupling rather than from the stronger single-scan semantic backbone, because the baseline is not given the same semantic prior features.

Editorial extensions

If this is right

Because the semantic stage is frozen and modular, any improvement in single-scan semantic segmentation can be dropped into D-PLS, so 4D panoptic quality should track single-scan progress without retraining the instance branch.
Preserving all points when the prior is wrong avoids the failure mode of methods that mask out things or background, where a misclassified point loses its geometric information before instance clustering.
The N=2 ablation's best configuration uses one-hot semantics without an auxiliary semantic loss, so the authors' decoupling claim implies jointly supervising semantics and instances is less effective than separating them.
With N=4 aggregated scans, D-PLS exceeds every compared method except Mask4D while remaining modular, implying the gap to Mask4D could be closed by pairing a newer semantic backbone with the same decoupled instance branch.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's own abstract concedes that part of the gain is 'due to advancements in single-scan semantic segmentation'; since the 4D-StOP baseline was not given the same 2DPASS semantic features, the causal role of decoupling alone remains untested. A direct augmentation of 4D-StOP with the identical semantic prior would isolate that effect.
Because the semantic prior is one-hot labels rather than embeddings, D-PLS treats all confident and uncertain predictions alike; a calibrated confidence-weighted prior could either improve borderline classes or dilute the one-hot advantage seen in the ablation.
The modularity suggests a practical deployment recipe: keep the instance branch fixed and refresh only the semantic segmenter as newer models appear, something monolithic mask-transformer systems require retraining to do.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: the paper has a clean, useful idea and a messy causal claim. D-PLS takes a frozen single-scan semantic net, appends its one-hot predictions to the point cloud, aggregates over time, and feeds that to an instance proposal module. It's genuinely modular—you can swap semantic backbones without retraining the instance stage—and the SemanticKITTI numbers are competitive (70.49 LSTQ at N=4, just under Mask4D). The authors also deserve credit for avoiding the masking trick from 4D-DS-Net; they keep all points in the instance branch, which is a sensible design choice.

The soft spot is exactly what the stress-test note says. The ablation compares D-PLS (2DPASS semantics + instance branch) against a retrained 4D-StOP baseline with no semantic input. Those two differ in both architecture and semantic source. A jump from 58 to 69 LSTQ could be mostly the stronger 2DPASS backbone, which the abstract itself hints at. The auxiliary-semantic-loss experiment changes only the loss, not the semantic input, so it doesn't isolate decoupling. The missing control is obvious: feed the same 2DPASS one-hot predictions into the 4D-StOP joint architecture and see whether decoupling still wins. Without that, the claim 'decoupling is what helps' is not established.

The paper still has value. The modularity claim is verified in the sense that a frozen semantic net can be bolted on top of a proposal module and produce competitive results. And the comparison to other SOTA is apples-to-apples on the same validation set, even if the baseline is retrained with a smaller point budget. No error bars, and no code, but for a niche method paper that's tolerable.

Who's this for? People working on 4D panoptic LiDAR segmentation who want a low-cost way to ride single-scan semantic improvements. It's a workshop/short-conference tier paper. I'd send it to peer review: the idea is concrete, the weakness is fixable with one controlled experiment, and the community would benefit from seeing the corrected comparison. I'd ask the authors to add that control and to soften the conclusion accordingly.

Referee Report

2 major / 5 minor

Summary. The paper proposes D-PLS, a two-stage method for 4D panoptic LiDAR segmentation. It first computes single-scan semantic predictions with a frozen 2DPASS network, aggregates semantically labeled scans over time using ego-motion poses, and then trains an instance segmentation branch on the aggregated point cloud with semantic one-hot or confidence features as additional input. The instance branch uses a KP-Conv offset predictor plus the 4D-StOP proposal and DBScan clustering modules. On SemanticKITTI, the method reports an LSTQ of 69.50 at N=2 versus a retrained 4D-StOP baseline at 58.01, and an LSTQ of 70.49 at N=4, which the paper claims outperforms most prior methods except Mask4D.

Significance. If the central claim holds, D-PLS would be a useful modular recipe for injecting rapid progress in single-scan semantic segmentation into 4D panoptic LiDAR segmentation without architectural changes or retraining of the semantic model. The design is simple, the semantic prior is used as an additional feature rather than a mask, and the paper demonstrates a large gain over its particular baseline. The significance is conditional, however, because the experiments do not isolate the effect of decoupling from the choice of the semantic backbone, and one of the headline comparison claims is contradicted by the paper's own table.

major comments (2)

[Section 4.1, Table 1; also Abstract] The ablation varies two factors at once: the architectural decoupling and the semantic source (2DPASS vs. the baseline's own learned semantic head). The retrained 4D-StOP baseline receives no semantic prior, while D-PLS receives strong, single-scan 2DPASS one-hot predictions. The observed LSTQ gain from 58.01 to 69.50 at N=2 therefore cannot be attributed to decoupling alone; the abstract itself states that the method 'surpasses the baseline due to advancements in single-scan semantic segmentation.' A control that feeds the same 2DPASS semantic predictions into the joint 4D-StOP architecture is needed to establish that decoupling, rather than a stronger semantic backbone, is responsible for the improvement.
[Section 5, Table 2] The text claims that 'our approach ... outperforms the state of the art, except for Mask4D,' but Table 2 lists Mask4Former with LSTQ 70.50 and 'ours' with 70.49. Since 70.49 < 70.50, the claim is factually incorrect. Additionally, the comparison uses the published 4D-StOP result (66.40) rather than the reduced-budget retrained baseline from Section 4.1, so the conclusion's statement of a 'substantial improvement over the baseline 4D-StOP' rests on two different baselines; the N=2 baseline from the ablation is not the one used in the state-of-the-art comparison.

minor comments (5)

[Section 4.1] The ablation reports a single number per configuration without error bars or multiple seeds. Given that the N=4 comparison is separated by 0.01 points from Mask4Former and by 0.39 points from Eq-4D-StOP, the robustness of the ranking is not established.
[Section 3.1] The description of the semantic prior is underspecified: the paper does not state whether the semantic labels are frozen during training of the instance branch, how the one-hot vectors are constructed (19 classes plus void?), or how confidence scores are normalized when used instead of one-hot labels.
[Section 3, text after 'Our approach addresses these challenges'] There is a typo: 'In a second step wen pass' should read 'we pass'.
[References] In the reference list, 'Lidas-based 4d panoptic segmentation' should likely be 'LiDAR-based 4D panoptic segmentation'.
[Section 4.1, Table 1] Table 1 includes an auxiliary semantic loss configuration, but the text does not explain what the auxiliary loss is or how it is supervised; without this detail the reader cannot interpret the small differences between the 'Auxiliary Semantic Loss' rows.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: D-PLS is evaluated against an external benchmark using a frozen pretrained semantic prior; no equation or fitted parameter reduces to the target result.

full rationale

The paper's derivation chain is empirical rather than definitional. D-PLS takes a frozen, pretrained single-scan semantic segmentation network (2DPASS) to generate one-hot semantic priors, aggregates scans using provided ego poses, and trains an instance branch (KP-Conv offset prediction plus the 4D-StOP proposal module) with supervised losses against SemanticKITTI ground truth. The semantic prior is neither fitted to the panoptic target nor derived from the instance branch's outputs; the instance branch is trained on ground-truth instance centers. There is no equation in which an output is defined in terms of the target, and no fitted parameter is relabeled as a prediction. The cited self-works (Reichardt et al. 2023/2024, Rishav et al. 2020) appear only in the future-work sentence and are not load-bearing. The comparison with 4D-StOP changes both the architecture and the semantic source, which is a potential experimental confound, but that is an evaluation-design concern, not circular reasoning. The main result is a benchmark number on SemanticKITTI, and the improvement is externally verifiable; the paper even explicitly attributes part of the gain to advances in single-scan segmentation. Consequently, no circular step can be exhibited.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or mathematical entities. Its free parameters are standard deep learning hyperparameters, and its assumptions are the typical dataset and model assumptions common to LiDAR segmentation research. The most consequential assumption is that the stronger semantic backbone is not the sole cause of the observed improvements.

free parameters (4)

Number of aggregated scans N = 2 for ablation, 4 for final comparison
Chosen by the authors to balance temporal context and computational cost; affects the results.
Number of sampled instance centers K = Not specified in the text
Used in farthest point sampling in the refinement module; selected by hand but not reported.
Grouping radius for instance proposals = Predefined but not specified
Determines which points are grouped around predicted centers; a key hyperparameter for clustering.
Training iteration counts = 400K for coarse stage, 150K for refinement
Sets the training budget; chosen without a sensitivity analysis.

assumptions (4)

domain assumption SemanticKITTI ground truth annotations are accurate and the evaluation metric LSTQ correctly measures panoptic quality.
Standard benchmark assumption; the paper relies on this to validate its method.
domain assumption Ego-motion poses provided by SemanticKITTI are accurate enough for temporal aggregation.
Required to fuse multiple scans into a consistent 4D point cloud (Section 3.2).
domain assumption The pretrained 2DPASS model provides reliable semantic predictions on the same dataset.
The semantic prior is generated by this frozen network; its accuracy directly influences the instance branch inputs.
domain assumption KP-Conv and PointNet architectures are appropriate for the instance segmentation task.
Borrowed from prior work (Thomas et al., Qi et al.); the paper assumes their effectiveness without re-deriving.

how reviews work

0 comments

Cite this review

Pith. "Pith review of D-PLS: Decoupled Semantic Segmentation for 4D-Panoptic-LiDAR-Segmentation." pith.science (2026). https://pith.science/paper/I3VRM27B

@misc{pith2026250115870,
  author       = {Pith},
  title        = {Pith review of: D-PLS: Decoupled Semantic Segmentation for 4D-Panoptic-LiDAR-Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I3VRM27B}},
  note         = {Machine review of arXiv:2501.15870}
}

read the original abstract

This paper introduces a novel approach to 4D Panoptic LiDAR Segmentation that decouples semantic and instance segmentation, leveraging single-scan semantic predictions as prior information for instance segmentation. Our method D-PLS first performs single-scan semantic segmentation and aggregates the results over time, using them to guide instance segmentation. The modular design of D-PLS allows for seamless integration on top of any semantic segmentation architecture, without requiring architectural changes or retraining. We evaluate our approach on the SemanticKITTI dataset, where it demonstrates significant improvements over the baseline in both classification and association tasks, as measured by the LiDAR Segmentation and Tracking Quality (LSTQ) metric. Furthermore, we show that our decoupled architecture not only enhances instance prediction but also surpasses the baseline due to advancements in single-scan semantic segmentation.

Figures

Figures reproduced from arXiv: 2501.15870 by the authors.

**Figure 2.** Our method D-PLS decomposes the task of 4D-Panoptic-LiDAR-Segmentation into two modular branches: the sin [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 17 canonical work pages

[1]

Ayg¨un, M., Osep, A., Weber, M., Maximov, M., Stachniss, C., Behley, J., and Leal-Taixe, L. (2021). 4D Panoptic Segmentation. In Conference on Computer Vision and Pattern Recognition (CVPR)

work page 2021
[2]

Behley, J., Garbade, M., Milioto, A., Quenzel, J., Behnke, S., Stachniss, C., and Gall, J. (2019). SemanticKITTI: A Dataset for Semantic Scene Understanding of Li- DAR Sequences. In International Conference on Computer Vision (ICCV)

work page 2019
[3]

Ester, M., Kriegel, H.-P., Sander, J., Xu, X., et al. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. In KDD

work page 1996
[4]

Hong, F., Zhou, H., Zhu, X., Li, H., and Liu, Z. (2021). Lidar-based panoptic segmentation via dynamic shift- ing network. In Conference on Computer Vision and Pattern Recognition (CVPR)

work page 2021
[5]

Hong, F., Zhou, H., Zhu, X., Li, H., and Liu, Z. (2022). Lidas-based 4d panoptic segmentation via dynamic shifting network. arXiv preprint arXiv:2203.07186

arXiv 2022
[6]

E., Mahadevan, S., Engelmann, F., and Leibe, B

Kreuzberg, L., Zulfikar, I. E., Mahadevan, S., Engelmann, F., and Leibe, B. (2022). 4d-stop: Panoptic segmenta- tion of 4d lidar using spatio-temporal object proposal generation and aggregation. In European Conference on Computer Vision (ECCV)

work page 2022
[7]

Marcuzzi, R., Nunes, L., Wiesmann, L., Marks, E., Behley, J., and Stachniss, C. (2023). Mask4d: End-to-end mask-based 4d panoptic segmentation for lidar se- quences. Robotics and Automation Letters (RA-L)

work page 2023
[8]

Marcuzzi, R., Nunes, L., Wiesmann, L., Vizzo, I., Behley, J., and Stachniss, C. (2022). Contrastive instance association for 4d panoptic segmentation using se- quences of 3d lidar scans. Robotics and Automation Letters (RA-L)

work page 2022

Show all 18 references

[9]

R., Su, H., Mo, K., and Guibas, L

Qi, C. R., Su, H., Mo, K., and Guibas, L. J. (2017). Pointnet: Deep learning on point sets for 3d classification and segmentation. In Conference on Computer Vision and Pattern Recognition

2017
[10]

Reichardt, L., Ebert, N., and Wasenm¨uller, O. (2023). 360◦ from a single camera: A few-shot approach for lidar segmentation. In International Conference on Com- puter Vision Workshop (ICCVW)

2023
[11]

Reichardt, L., Uhr, L., and Wasenm ¨uller, O. (2024). Text3daug – prompted instance augmentation for lidar perception. In International Conference on Intelligent Robots and Systems (IROS)

2024
[12]

Stricker, D. (2020). Deeplidarflow: A deep learning architecture for scene flow estimation using monocu- lar camera and sparse lidar. InIEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

2020
[13]

Tang, H., Liu, Z., Zhao, S., Lin, Y ., Lin, J., Wang, H., and Han, S. (2020). Searching efficient 3d architec- tures with sparse point-voxel convolution. In Euro- pean Conference on Computer Vision (ECCV)

2020
[14]

Goulette, F., and Guibas, L. J. (2019). Kpconv: Flex- ible and deformable convolution for point clouds. In International Conference on Computer Vision (ICCV)

2019
[15]

Ouyang, W., He, T., and Zhao, H. (2024). Point trans- former v3: Simpler faster stronger. In Conference on Computer Vision and Pattern Recognition (CVPR)

2024
[16]

Yan, X., Gao, J., Zheng, C., Zheng, C., Zhang, R., Cui, S., and Li, Z. (2022). 2dpass: 2d priors assisted seman- tic segmentation on lidar point clouds. In European Conference on Computer Vision (ECCV)

2022
[17]

Yilmaz, K., Schult, J., Nekrasov, A., and Leibe, B. (2024). Mask4former: Mask transformer for 4d panoptic seg- mentation. In International Conference on Robotics and Automation (ICRA)

2024
[18]

Porikli, F. (2023). 4d panoptic segmentation as invari- ant and equivariant field prediction. In International Conference on Computer Vision (ICCV)

2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Ayg¨un, M., Osep, A., Weber, M., Maximov, M., Stachniss, C., Behley, J., and Leal-Taixe, L. (2021). 4D Panoptic Segmentation. In Conference on Computer Vision and Pattern Recognition (CVPR)

work page 2021

[2] [2]

Behley, J., Garbade, M., Milioto, A., Quenzel, J., Behnke, S., Stachniss, C., and Gall, J. (2019). SemanticKITTI: A Dataset for Semantic Scene Understanding of Li- DAR Sequences. In International Conference on Computer Vision (ICCV)

work page 2019

[3] [3]

Ester, M., Kriegel, H.-P., Sander, J., Xu, X., et al. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. In KDD

work page 1996

[4] [4]

Hong, F., Zhou, H., Zhu, X., Li, H., and Liu, Z. (2021). Lidar-based panoptic segmentation via dynamic shift- ing network. In Conference on Computer Vision and Pattern Recognition (CVPR)

work page 2021

[5] [5]

Hong, F., Zhou, H., Zhu, X., Li, H., and Liu, Z. (2022). Lidas-based 4d panoptic segmentation via dynamic shifting network. arXiv preprint arXiv:2203.07186

arXiv 2022

[6] [6]

E., Mahadevan, S., Engelmann, F., and Leibe, B

Kreuzberg, L., Zulfikar, I. E., Mahadevan, S., Engelmann, F., and Leibe, B. (2022). 4d-stop: Panoptic segmenta- tion of 4d lidar using spatio-temporal object proposal generation and aggregation. In European Conference on Computer Vision (ECCV)

work page 2022

[7] [7]

Marcuzzi, R., Nunes, L., Wiesmann, L., Marks, E., Behley, J., and Stachniss, C. (2023). Mask4d: End-to-end mask-based 4d panoptic segmentation for lidar se- quences. Robotics and Automation Letters (RA-L)

work page 2023

[8] [8]

Marcuzzi, R., Nunes, L., Wiesmann, L., Vizzo, I., Behley, J., and Stachniss, C. (2022). Contrastive instance association for 4d panoptic segmentation using se- quences of 3d lidar scans. Robotics and Automation Letters (RA-L)

work page 2022