Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

How Do Images Align and Complement LiDAR? Towards a Harmonized Multi-modal 3D Panoptic Segmentation

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Keeping camera images and LiDAR points synchronized through augmentation, token fusion, and query initialization, IAL achieves state-of-the-art 3D panoptic segmentation—82.3% PQ on nuScenes, 63.1% on SemanticKITTI—with no clustering…

desk verdict Solid SOTA accuracy with honest ablations, but the end-to-end speed claim in the main text is contradicted by the paper's own supplementary table and needs correction. read the letter →

arxiv 2505.18956 v2 pith:QMPPNEBD submitted 2025-05-25 cs.CV cs.AIcs.LGcs.MM

Yining Pan , Qiongjie Cui , Xulei Yang , Na Zhao This is my paper · ORCID
classification cs.CVcs.AIcs.LGcs.MM
keywords multi-modal3DpanopticsegmentationLiDAR-camerafusionmodality-synchronizedaugmentationcylindricalvoxelizationtransformerdecoderprior-basedquerygenerationscale-awarepositionalembeddingnuScenes
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that camera images can reliably fill the gaps where LiDAR is sparse—small or distant objects—provided the two modalities are kept in lockstep at every stage: augmentation, token construction, and query initialization. The proposed framework, Image-Assists-LiDAR (IAL), cuts synchronized 'pie slices' from cylindrical voxels and from the corresponding image patches (PieAug), fuses voxel and image features under a shared scale-aware position embedding built from each voxel's eight corners (GTF), and seeds instance queries from geometric priors, texture priors, and learnable fallbacks (PQG). The payoff is that a transformer decoder directly outputs panoptic predictions, eliminating the clustering post-processing that earlier multi-modal methods required. On the nuScenes and SemanticKITTI validation sets, IAL reports 82.3% PQ and 63.1% PQ, exceeding the prior multi-modal baselines by 2.5 and 4.1 points.

What carries the argument

The load-bearing objects are three. First, the PieAug 'pie cut': a binary mask $S(r,\theta,z)$ over the cylindrical voxel grid (radius, azimuth, height) that marks voxels to swap from another scan, while the same mask selects the corresponding image rectangles, so a single operator covers instance pasting and scene swapping for both modalities at once and subsumes existing LiDAR augmentations. Second, the scale-aware positional embedding (SPE): the eight corner points of each cylindrical voxel give a scale feature, added to a mixed-parameterized centroid embedding in Cartesian and polar space, and this shared embedding is attached to both the LiDAR and image sides of every fused token so that large far-away voxels and small near voxels are encoded consistently. Third, the three query groups of PQG: geometric-prior queries from a class-agnostic BEV center heatmap with NMS, texture-prior queries from 2D masks lifted into 3D frustums and grouped by DBSCAN, and no-prior learnable queries; this division of labor lets the decoder handle easy instances, hard small or distant ones, and cases with no usable prior, all without post-processing.

What would settle it

Disable the texture-prior branch alone—keep the geometric-prior and no-prior queries but drop the Grounding-DINO and SAM mask proposals—and re-evaluate on the nuScenes validation set with results split by object distance, such as instances beyond 30 meters versus nearer ones. The paper's claim predicts the PQ drop concentrates on small, distant things; if the loss is uniform or negligible, the image pathway is not doing the work attributed to it and the reported result does not depend on the 2D proposers.

Watch

Extended reading notes

Core claim

The central claim is that misalignment, not missing information, has been blocking LiDAR-plus-image fusion for outdoor panoptic segmentation, and that keeping the two modalities synchronized at each design point turns image texture into a genuine assistant for LiDAR geometry. IAL realizes this with three coordinated choices: PieAug applies the same cylindrical 'pie' mask to both the point cloud and the images, so training never shows the model a mismatched pair; GTF projects every physical point of a voxel (not its centroid) into the image and adds a scale-aware positional embedding derived from the voxel's eight corners, so image features land where the voxel actually perceives; and PQG initializes instance queries from LiDAR center heatmaps, from lifted 2D mask proposals produced by Grounding-DINO and SAM, and from learnable parameters for cases neither modality resolves. With those inputs, a transformer decoder predicts masks and class labels directly, and the measured gains concentrate exactly where LiDAR alone struggles: distant objects, clustered objects, and ambiguous classes.

Load-bearing premise

The load-bearing premise is that the pre-trained 2D proposers behind the texture-prior queries—Grounding-DINO and SAM—reliably find the small and distant objects LiDAR misses; the authors themselves flag these models as generic rather than task-specific, so when the proposers miss an object or use a category that does not match the target classes, the lifted texture-prior queries add noise and the 1.2% PQ gain credited to the PQG module shrinks or reverses.

Editorial extensions

If this is right

  • Other multi-modal methods that augment only the LiDAR side are leaving measurable accuracy behind: in the paper's ablations, PieAug's synchronized operators add 2.7% PQ over LiDAR-only augmentation on nuScenes.
  • Direct panoptic prediction through a transformer decoder replaces the semantic-then-cluster pipeline: IAL runs at 4.0 FPS in its core configuration, over twice LCPS's 1.7 FPS, with no post-processing step.
  • Images help most where geometry degrades: the multi-modal gain over the LiDAR-only branch is concentrated in thing classes (+7.5% PQth on nuScenes), and the full model beats the LiDAR branch on the nighttime and rain splits by 7.3% and 8.1% PQ.
  • Query initialization matters as much as token fusion: modality-prior queries add 1.2% PQ on top of the fusion module, whereas prior-only query sets without a learnable fallback can slightly hurt performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The transferable idea is the symmetry rule: every operation applied to the LiDAR side should be mirrored on the image side; that rule should carry over to other LiDAR-camera tasks such as 3D detection or online mapping, where augmentation and query seeding are still done per modality.
  • A testable consequence the paper does not state: the PQ gap between IAL and its LiDAR-only branch should widen at longer ranges, so a distance-stratified evaluation (for example, objects beyond 30 meters) would sharpen the claim that images specifically rescue distant objects.
  • Because the texture-prior branch inherits the vocabularies and training domains of Grounding-DINO and SAM, IAL's ceiling is partly set by those models; a shift that breaks the 2D proposers, such as heavy snow or lens flare, should force the no-prior queries to absorb the loss—a robustness ceiling worth probing.
  • The scale-aware position embedding treats each cylindrical voxel as a box, and at large radii one box can straddle several objects, so averaging image features inside it may blend distinct instances; that error mode is unanalyzed in the paper and is a natural place for a finer representation next.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes IAL, a transformer-based LiDAR-camera fusion framework for 3D panoptic segmentation. It introduces three main contributions: PieAug, a modality-synchronized data augmentation strategy that pairs cylindrical voxels with corresponding image regions; a Geometric-guided Token Fusion (GTF) module that aligns image and LiDAR features using point-set projection and scale-aware positional embeddings; and a Prior-based Query Generation (PQG) module that initializes instance queries from LiDAR geometric priors, from image-derived masks produced by Grounding-DINO and SAM, and from learnable no-prior queries. The transformer decoder directly predicts panoptic results without the clustering post-processing used by previous methods. The authors report state-of-the-art PQ on nuScenes (82.3 validation, 82.0 test) and SemanticKITTI (63.1 validation), with ablations attributing gains to each module. The supplementary material includes robustness experiments on nighttime and rain splits, as well as a speed and parameter comparison with LCPS.

Significance. If the reported accuracy is reproducible, IAL is a meaningful advance: it is one of the first transformer-based end-to-end multi-modal 3D panoptic segmentation methods, the code and models are released, the validation and test results are consistent, the ablations in Tables 5, 6, and 8 are controlled, and the main results are reported without test-time augmentation. The adverse-condition analysis in Supplementary Table 10 is a useful and comparatively rare addition. However, the paper's efficiency claims are overstated: the full IAL pipeline is substantially slower and larger than LCPS, and the texture-prior contribution is inherited in part from large pre-trained 2D models. The core accuracy result appears sound, but the framing needs correction before publication.

major comments (3)
  1. [Supplementary Sec. C, Table 9] The speed comparison is internally inconsistent and overstates IAL's efficiency. The text says IAL achieves "over 2× faster inference" and "remains comparable in speed" when mask generation is included, but Table 9 shows full IAL at 0.9 FPS with 859.9M parameters versus LCPS at 1.7 FPS with 77.7M parameters: the full system is about 1.9× slower and roughly 11× larger. The 4.0 FPS figure belongs to IAL*, which is explicitly defined as excluding Grounding-DINO and SAM preprocessing, yet it is reported with the same PQ (82.3) as the full model. This is only meaningful if the removed components do not affect the output, which contradicts the PQG ablation, or if the PQ is measured on a different configuration. The introduction's motivation that IAL eliminates the "inefficient" post-processing bottleneck is therefore not supported at the system level. Please correct the speed statements, report the full-pipeline FPS and parameter count as the primary efficiency numbers, and clarify exactly what IAL* measures.
  2. [Sec. 3.3, Table 6, Supp. Sec. F] The contribution of texture-prior queries is not cleanly separated from the external pre-trained models. Table 6 attributes about 1.2 PQ to the complete PQG module, but the implementation details for Grounding-DINO and SAM are missing: whether they are frozen or fine-tuned, what text prompts or class vocabulary are used, and how their outputs are filtered before DBSCAN clustering. Because the paper's claim is that images complement LiDAR, the authors should report a controlled comparison with geometric and no-prior queries alone under the same query budget, and state explicitly how much of the final gain is inherited from pre-trained 2D models. The acknowledged limitation in Supp. Sec. F confirms this dependency; it should be addressed in the main text or supported with an experiment that removes the texture-prior component.
  3. [Sec. 3.1, Table 7] The claim that PieAug is a general multi-modal augmentation strategy is not directly supported by the comparison in Table 7. The baselines shown (PolarMix and LaserMix) are LiDAR-only augmentation methods, whereas the related work explicitly mentions LaserMix++ and MSeg3D as existing multi-modal augmentation approaches. Without comparing against these methods, Table 7 does not establish that synchronized multi-modal augmentation is superior to existing multi-modal augmentation. Please add these baselines or state clearly why they are omitted.
minor comments (5)
  1. [Sec. 4.1] The heading contains a typo: "Datasts" should read "Datasets."
  2. [Sec. 3.2, Eq. (7)] In Eq. (7), the phrase "and || · ||is L2 norm" should be rewritten, and the averaging operation Avg(·) should be defined before it is used.
  3. [Sec. 4.3, Table 6] The row labels in Table 6 are ambiguous because the checkmark patterns are not explicitly named; please label each row with the configuration name (e.g., Geo+Tex, Geo+NP, Tex+NP, all three) so the ablation is immediately clear.
  4. [Sec. 3.3] The allocation of the 256 total queries for the full three-group configuration is not specified; for row 5 of Table 6, please state the per-group numbers for geometric, texture, and no-prior queries.
  5. [Supplementary Sec. F] Supplementary Sec. F contains a duplicated sentence structure that praises the results twice in close succession; please tighten the wording.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: IAL's accuracy claims are grounded in external benchmarks and controlled ablations; self-citations are background only, and the acknowledged reliance on pre-trained 2D models is an external dependency rather than a circular derivation.

full rationale

This paper is an empirical systems paper. It does not derive its headline panoptic quality numbers from equations fitted to the target data: the nuScenes (82.3% PQ) and SemanticKITTI (63.1% PQ) results are benchmark measurements compared against external methods such as LCPS and Panoptic-FusionNet, and the module ablations in Tables 5, 6, and 8 are controlled comparisons on held-out validation frames with fixed hyperparameters. The methodological components are defined by explicit equations: Eq. (1)-(5) define projection and the synchronized augmentation mask, Eq. (6) aggregates image features by projecting physical points, Eq. (7) defines the scale-aware positional embedding, and Eq. (8) forms the fused token. None of these equations takes the final PQ metric as an input, and none of the reported gains is obtained by renaming a fitted quantity as a prediction. The texture-prior queries in PQG use pre-trained Grounding-DINO and SAM proposals, which are external, large-scale models; the paper itself candidly acknowledges in Sec. F that these are generic pre-trained models rather than task-specific components. That is an external dependency and a robustness caveat, not a circular step, because the proposals are not fit to the benchmark targets and the final gains are still measured on unseen data. The paper contains minor self-citations (Zhao et al., 2021; 2022; Li & Zhao, 2024; Xu et al., 2023b), but these appear only in related-work background and are not used to justify any architectural choice or to forbid alternative designs. No uniqueness theorem from the authors' prior work is invoked. The efficiency claim in Supplementary Sec. C and Table 9 appears overstated because the full model with mask generation runs at 0.9 FPS versus LCPS at 1.7 FPS, while the text claims over 2x faster inference; however, overstatement of a side claim is a correctness and presentation issue, not circular reasoning about the derivation of the accuracy result. Overall, the central claim is self-contained against external benchmarks and public code, so no circular step is present.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical entities or forces. The free parameters are hyperparameters and unspecified architectural details; they are not fitted to explain data in a circular way. The main assumptions are domain-level: calibration accuracy, suitability of the voxel representation, reliability of external 2D models, and the transformer paradigm.

free parameters (5)
  • Number of prior-based and no-prior instance queries (lpr=llt) = 128
    Chosen by hand; affects decoder capacity and the number of candidate instances. The paper sets both to 128 for all experiments.
  • Augmentation application ratios (instance pasting, scene swap height, scene swap angle) = 0.4:0.05:0.05
    Tuned on validation; controls diversity vs. distortion. These ratios are specific to the method and are likely dataset-dependent.
  • Number of slices for scene swapping = random in [3,4,5]
    Randomly sampled during training; a design choice that affects the granularity of augmentation.
  • Transformer decoder layers = not reported
    The number of decoder layers is not specified in the paper, yet it is a key architectural hyperparameter that could affect results.
  • Heatmap confidence threshold and range radius for NMS = not reported
    The geometric-prior query generation uses confidence scores and a range radius threshold, but the exact values are not given, hindering exact reproduction.
assumptions (4)
  • domain assumption LiDAR-camera calibration matrices (intrinsic K and extrinsic T) are known and accurate.
    Used in Eq. (1) for projecting points to image; if calibration is off, the fusion and augmentation alignment break. Datasets provide these but the method assumes their correctness.
  • domain assumption The cylindrical voxel representation with fixed binning (480x360x32) is an appropriate discretization for the task.
    Adopted from prior work; the binning affects voxel sizes and thus the scale-aware embeddings. This is a standard representation for LiDAR panoptic segmentation.
  • domain assumption Pre-trained Grounding-DINO and SAM provide reliable mask proposals for objects in outdoor scenes.
    These models are used without fine-tuning; their outputs are assumed to be good enough for texture-prior queries. The paper discloses this in Section F as a limitation.
  • domain assumption The transformer decoder and query-based prediction paradigm from P3Former is a valid basis for panoptic segmentation.
    The paper builds on P3Former's architecture. This paradigm is established in the literature, but the paper does not independently verify its suitability beyond benchmarks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of How Do Images Align and Complement LiDAR? Towards a Harmonized Multi-modal 3D Panoptic Segmentation." pith.science (2026). https://pith.science/paper/QMPPNEBD

@misc{pith2026250518956,
  author       = {Pith},
  title        = {Pith review of: How Do Images Align and Complement LiDAR? Towards a Harmonized Multi-modal 3D Panoptic Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QMPPNEBD}},
  note         = {Machine review of arXiv:2505.18956}
}
read the original abstract

LiDAR-based 3D panoptic segmentation often struggles with the inherent sparsity of data from LiDAR sensors, which makes it challenging to accurately recognize distant or small objects. Recently, a few studies have sought to overcome this challenge by integrating LiDAR inputs with camera images, leveraging the rich and dense texture information provided by the latter. While these approaches have shown promising results, they still face challenges, such as misalignment during data augmentation and the reliance on post-processing steps. To address these issues, we propose Image-Assists-LiDAR (IAL), a novel multi-modal 3D panoptic segmentation framework. In IAL, we first introduce a modality-synchronized data augmentation strategy, PieAug, to ensure alignment between LiDAR and image inputs from the start. Next, we adopt a transformer decoder to directly predict panoptic segmentation results. To effectively fuse LiDAR and image features into tokens for the decoder, we design a Geometric-guided Token Fusion (GTF) module. Additionally, we leverage the complementary strengths of each modality as priors for query initialization through a Prior-based Query Generation (PQG) module, enhancing the decoder's ability to generate accurate instance masks. Our IAL framework achieves state-of-the-art performance compared to previous multi-modal 3D panoptic segmentation methods on two widely used benchmarks. Code and models are publicly available at <https://github.com/IMPL-Lab/IAL.git>.

Figures

Figures reproduced from arXiv: 2505.18956 by the authors.

Figure 1
Figure 1. The architecture overview of our Image-Assists-LiDAR (IAL) framework. We first voxelize the point cloud into cylindrical voxels. In PieAug, we synchronize augmentation by pairing cylindrical and image grids, mixing original and new scans in different modes. Using a transformer-based structure, we design token fusion (GTF) and query initialization (PQG) modules to align and complement both modalities. In GTF, feature… view at source ↗
Figure 2
Figure 2. Motivation and implementation variants of PieAug. Each column illustrates the motivation for LiDAR-image syn￾chronized augmentation. Each row displays a different pie-cut strategy. Point clouds are projected on camera images, with colors indicating semantic labels or data sources. Best viewed in color. Here B(·) is the operator that fits a bounding rectangle en￾closing a set of pixels. During augmentation, we determ… view at source ↗
Figure 3
Figure 3. , which leverages the rich geometric information from LiDAR to guide alignment and enable effective fusion. Specifically, we align features at the voxel level by project￾ing all physical points within a voxel vi onto the image plane and averaging their corresponding image features to create an aggregated representation: F˜ 2D i = 1 Ni XNi j F 2D(π(pj )). (6) We refer to the paired voxel-wise LiDAR feature F 3D i and… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison of our method with the preliminary multi-modal panoptic segmentation baseline, LCPS. To highlight the differences, we mark false positive and false negative predictions, which affect recognition quality, as well as well-matched and mismatch point…
Figure 5
Figure 5. Figure 5: Visualization of instance predictions. Red circles high￾light instances where the LiDAR branch fails to segment correctly, but our multi-modal method succeeds. Best viewed in color. 4.5. Qualitative Results and Discussion We present qualitative evaluations on nuScenes …
Figure 6
Figure 6. Figure 6: Qualitative comparison of the ablation study for GTF and PQG modules. To emphasize the differences, we mark false positive and false negative predictions, which affect recognition quality, as well as well-matched and mismatch points in true positive predictions, which …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. UP-Fuse: Uncertainty-guided LiDAR-Camera Fusion for 3D Panoptic Segmentation

    cs.CV 2026-02 conditional novelty 6.0 of 10

    UP-Fuse learns to predict which camera features are unreliable and down-weights them during LiDAR-camera fusion, improving 3D panoptic segmentation under sensor degradation.

Reference graph

Works this paper leans on

59 extracted references · 45 canonical work pages · cited by 1 Pith paper

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Multimodality helps few-shot 3d point cloud semantic segmentation, 2025

    An, Z., Sun, G., Liu, Y., Li, R., Wu, M., Cheng, M.-M., Konukoglu, E., and Belongie, S. Multimodality helps few-shot 3d point cloud semantic segmentation, 2025. URL https://arxiv.org/abs/2410.22489

    arXiv 2025

  3. [3]

    4d-former: Multimodal 4d panoptic segmentation

    Athar, A., Li, E., Casas, S., and Urtasun, R. 4d-former: Multimodal 4d panoptic segmentation. In Tan, J., Toussaint, M., and Darvish, K. (eds.), Proceedings of The 7th Conference on Robot Learning, volume 229 of Proceedings of Machine Learning Research, pp.\ 2151--2164. PMLR, 06--09 Nov 2023. URL https://proceedings.mlr.press/v229/athar23a.html

    work page 2023

  4. [4]

    Transfusion: Robust lidar-camera fusion for 3d object detection with transformers

    Bai, X., Hu, Z., Zhu, X., Huang, Q., Chen, Y., Fu, H., and Tai, C.-L. Transfusion: Robust lidar-camera fusion for 3d object detection with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 1090--1099, 2022

    work page 2022

  5. [5]

    SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences

    Behley, J., Garbade, M., Milioto, A., Quenzel, J., Behnke, S., Stachniss, C., and Gall, J. SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences . In Proc. of the IEEE/CVF International Conf. on Computer Vision (ICCV), 2019

    work page 2019

  6. [6]

    Towards 3D LiDAR-based semantic scene understanding of 3D point cloud sequences: The SemanticKITTI Dataset

    Behley, J., Garbade, M., Milioto, A., Quenzel, J., Behnke, S., Gall, J., and Stachniss, C. Towards 3D LiDAR-based semantic scene understanding of 3D point cloud sequences: The SemanticKITTI Dataset . The International Journal on Robotics Research, 40 0 (8-9): 0 959--967, 2021. doi:10.1177/02783649211006735

    work page doi:10.1177/02783649211006735 2021

  7. [7]

    H., Vora, S., Liong, V

    Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., and Beijbom, O. nuscenes: A multimodal dataset for autonomous driving. In CVPR, 2020

    work page 2020

  8. [8]

    Pasco: Urban 3d panoptic scene completion with uncertainty awareness

    Cao, A.-Q., Dai, A., and de Charette, R. Pasco: Urban 3d panoptic scene completion with uncertainty awareness. In CVPR, 2024

    work page 2024

Show all 59 references
  1. [9]

    Futr3d: A unified sensor fusion framework for 3d detection

    Chen, X., Zhang, T., Wang, Y., Wang, Y., and Zhao, H. Futr3d: A unified sensor fusion framework for 3d detection. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 172--181, 2023

    2023

  2. [10]

    A density-based algorithm for discovering clusters in large spatial databases with noise

    Ester, M., Kriegel, H.-P., Sander, J., and Xu, X. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, KDD'96, pp.\ 226–231. AAAI Press, 1996

    1996

  3. [11]

    K., Mohan, R., Hurtado, J

    Fong, W. K., Mohan, R., Hurtado, J. V., Zhou, L., Caesar, H., Beijbom, O., and Valada, A. Panoptic nuscenes: A large-scale benchmark for lidar panoptic segmentation and tracking. In ICRA, 2022

    2022

  4. [12]

    Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite

    Geiger, A., Lenz, P., and Urtasun, R. Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite . In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp.\ 3354--3361, 2012

    2012

  5. [13]

    Maskrange: A mask-classification model for range-view based lidar segmentation

    Gu, Y., Huang, Y., Xu, C., and Kong, H. Maskrange: A mask-classification model for range-view based lidar segmentation. ArXiv, abs/2206.12073, 2022. URL https://api.semanticscholar.org/CorpusID:250048842

    2022 arXiv

  6. [15]

    Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. CoRR, abs/1412.6980, 2014. URL https://api.semanticscholar.org/CorpusID:6628106

    2014 arXiv

  7. [16]

    B., Rother, C., and Doll \'a r, P

    Kirillov, A., He, K., Girshick, R. B., Rother, C., and Doll \'a r, P. Panoptic segmentation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 9396--9405, 2018. URL https://api.semanticscholar.org/CorpusID:4853375

    2019

  8. [19]

    Lasermix for semi-supervised lidar semantic segmentation

    Kong, L., Ren, J., Pan, L., and Liu, Z. Lasermix for semi-supervised lidar semantic segmentation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 21705--21715, 2023 b

    2023

  9. [20]

    T., and Liu, Z

    Kong, L., Xu, X., Ren, J., Zhang, W., Pan, L., Chen, K., Ooi, W. T., and Liu, Z. Multi-modal data-efficient 3d scene understanding for autonomous driving, 2024. URL https://arxiv.org/abs/2405.05258

    2024

  10. [21]

    Fuseseg: Lidar point cloud segmentation fusing multi-modal data

    Krispel, G., Opitz, M., Waltner, G., Possegger, H., and Bischof, H. Fuseseg: Lidar point cloud segmentation fusing multi-modal data. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp.\ 1874--1883, 2020

    2020

  11. [22]

    Cpseg: Cluster-free panoptic segmentation of 3d lidar point clouds

    Li, E., Razani, R., Xu, Y., and Liu, B. Cpseg: Cluster-free panoptic segmentation of 3d lidar point clouds. pp.\ 8239--8245, 2023 a . URL https://api.semanticscholar.org/CorpusID:240419848

    2023

  12. [23]

    Panoptic-phnet: Towards real-time and high-precision lidar panoptic segmentation via clustering pseudo heatmap

    Li, J., He, X., Wen, Y., Gao, Y., Cheng, X., and Zhang, D. Panoptic-phnet: Towards real-time and high-precision lidar panoptic segmentation via clustering pseudo heatmap. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 11799--11808, 2022 a . UR...

    2022

  13. [24]

    Mseg3d: Multi-modal 3d semantic segmentation for autonomous driving

    Li, J., Dai, H., Han, H., and Ding, Y. Mseg3d: Multi-modal 3d semantic segmentation for autonomous driving. In CVPR, pp.\ 21694--21704, 2023 b

    2023

  14. [25]

    and Zhao, N

    Li, L. and Zhao, N. End-to-end semi-supervised 3d instance segmentation with pcteacher. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp.\ 5352--5358. IEEE, 2024

    2024

  15. [26]

    and Chen, D

    Li, X. and Chen, D. A survey on deep learning-based panoptic segmentation. Digital Signal Processing, 120: 0 103283, 2022. ISSN 1051-2004. doi:https://doi.org/10.1016/j.dsp.2021.103283. URL https://www.sciencedirect.com/science/article/pii/S1051200421003225

    2022

  16. [27]

    Center focusing network for real-time lidar panoptic segmentation

    Li, X., Zhang, G., Wang, B., Hu, Y., and Yin, B. Center focusing network for real-time lidar panoptic segmentation. pp.\ 13425--13434, 06 2023 c . doi:10.1109/CVPR52729.2023.01290

    2023

  17. [28]

    W., Meng, T., Caine, B., Ngiam, J., Peng, D., Shen, J., Lu, Y., Zhou, D., Le, Q

    Li, Y., Yu, A. W., Meng, T., Caine, B., Ngiam, J., Peng, D., Shen, J., Lu, Y., Zhou, D., Le, Q. V., et al. Deepfusion: Lidar-camera deep fusion for multi-modal 3d object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 17182...

    2022

  18. [29]

    Fully sparse fusion for 3d object detection

    Li, Y., Fan, L., Liu, Y., Huang, Z., Chen, Y., Wang, N., and Zhang, Z. Fully sparse fusion for 3d object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  19. [30]

    Bevfusion: A simple and robust lidar-camera fusion framework

    Liang, T., Xie, H., Yu, K., Xia, Z., Lin, Z., Wang, Y., Tang, T., Wang, B., and Tang, Z. Bevfusion: A simple and robust lidar-camera fusion framework. volume 35, pp.\ 10421--10434, 2022

    2022

  20. [31]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Li, C., Yang, J., Su, H., Zhu, J., et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499, 2023 a

    2023 arXiv

  21. [32]

    Uniseg: A unified multi-modal lidar segmentation network and the openpcseg codebase

    Liu, Y., Runnan, C., Li, X., Kong, L., Yang, Y., Xia, Z., Bai, Y., Zhu, X., Ma, B., Li, Y., Qiao, Y., and Hou, Y. Uniseg: A unified multi-modal lidar segmentation network and the openpcseg codebase. pp.\ 21605--21616, 10 2023 b . doi:10.1109/ICCV51070.2023.01980

    2023

  22. [33]

    BEV-Guided Multi-Modality Fusion for Driving Perception

    Man, Y., Gui, L.-Y., and Wang, Y.-X. BEV-Guided Multi-Modality Fusion for Driving Perception . In CVPR, 2023

    2023

  23. [34]

    Mask-Based Panoptic LiDAR Segmentation for Autonomous Driving

    Marcuzzi, R., Nunes, L., Wiesmann, L., Behley, J., and Stachniss, C. Mask-Based Panoptic LiDAR Segmentation for Autonomous Driving . 8 0 (2): 0 1141--1148, 2023. doi:10.1109/LRA.2023.3236568. URL https://www.ipb.uni-bonn.de/wp-content/papercite-data/pdf/marcuzzi2023ral.pdf

    2023

  24. [35]

    CenterLPS : Segment Instances by Centers for LiDAR Panoptic Segmentation

    Mei, J., Yang, Y., Wang, M., Li, Z., Hou, X., Ra, J., Li, L., and Liu, Y. CenterLPS : Segment Instances by Centers for LiDAR Panoptic Segmentation . In Proceedings of the 31st ACM International Conference on Multimedia , MM '23, pp.\ 1884--1894, New York, NY, USA, 2023. Associ...

    2023

  25. [36]

    and Šegvić, S

    Oršić, M. and Šegvić, S. Efficient semantic segmentation with pyramidal fusion. Pattern Recognition, 110: 0 107611, 2021. ISSN 0031-3203. doi:https://doi.org/10.1016/j.patcog.2020.107611. URL https://www.sciencedirect.com/science/article/pii/S0031320320304143

    2021

  26. [37]

    R., Colovic, A., and Kontschieder, P

    Porzi, L., Bul \`o , S. R., Colovic, A., and Kontschieder, P. Seamless scene segmentation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 8269--8278, 2019. URL https://api.semanticscholar.org/CorpusID:145047913

    2019

  27. [38]

    M., Ren, Y., and Liu, B

    Razani, R., Cheng, R., Li, E., Taghavi, E. M., Ren, Y., and Liu, B. Gp-s3net: Graph-based panoptic sparse semantic segmentation network. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp.\ 16056--16065, 2021. URL https://api.semanticscholar.org/CorpusID:237213370

    2021

  28. [39]

    Efficientlps: Efficient lidar panoptic segmentation

    Sirohi, K., Mohan, R., Buscher, D., Burgard, W., and Valada, A. Efficientlps: Efficient lidar panoptic segmentation. IEEE Transactions on Robotics, 38: 0 1894--1914, 2021. URL https://api.semanticscholar.org/CorpusID:231933783

    1914

  29. [40]

    Panoptic-fusionnet: Camera-lidar fusion-based point cloud panoptic segmentation for autonomous driving

    Song, H., Cho, J., Ha, J., Park, J., and Jo, K. Panoptic-fusionnet: Camera-lidar fusion-based point cloud panoptic segmentation for autonomous driving. Expert Syst. Appl., 251 0 (C), October 2024. ISSN 0957-4174. doi:10.1016/j.eswa.2024.123950. URL https://doi.org/10.1016/j.es...

    2024

  30. [41]

    PUPS : Point Cloud Unified Panoptic Segmentation , February 2023

    Su, S., Xu, J., Wang, H., Miao, Z., Zhan, X., Hao, D., and Li, X. PUPS : Point Cloud Unified Panoptic Segmentation , February 2023. URL http://arxiv.org/abs/2302.06185. arXiv:2302.06185 [cs]

    2023 arXiv

  31. [42]

    Joint semantic segmentation using representations of lidar point clouds and camera images

    Wu, Y., Liu, J., Gong, M., Miao, Q., Ma, W., and Xu, C. Joint semantic segmentation using representations of lidar point clouds and camera images. Information Fusion, 108: 0 102370, 2024

    2024

  32. [43]

    Polarmix: a general data augmentation technique for lidar point clouds

    Xiao, A., Huang, J., Guan, D., Cui, K., Lu, S., and Shao, L. Polarmix: a general data augmentation technique for lidar point clouds. 2022

    2022

  33. [44]

    C., Lin, D., and Pang, J

    Xiao, Z., Zhang, W., Wang, T., Loy, C. C., Lin, D., and Pang, J. Position- Guided Point Cloud Panoptic Segmentation Transformer . International Journal of Computer Vision, 133 0 (1): 0 275--290, January 2025. ISSN 1573-1405. doi:10.1007/s11263-024-02162-z. URL https://doi.org/...

    2025 doi

  34. [45]

    Sparse cross-scale attention network for efficient lidar panoptic segmentation

    Xu, S., Wan, R., Ye, M., Zou, X., and Cao, T. Sparse cross-scale attention network for efficient lidar panoptic segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp.\ 2920--2928, 2022

    2022

  35. [46]

    Aop-net: All-in-one perception network for lidar-based joint 3d object detection and panoptic segmentation

    Xu, Y., Fazlali, H., Ren, Y., and Liu, B. Aop-net: All-in-one perception network for lidar-based joint 3d object detection and panoptic segmentation. In 2023 IEEE Intelligent Vehicles Symposium (IV), pp.\ 1--7, 2023 a . doi:10.1109/IV55152.2023.10186743

    2023

  36. [47]

    Xu, Y., Hu, C., Zhao, N., and Lee, G. H. Generalized few-shot point cloud segmentation via geometric words. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 21506--21515, 2023 b

    2023

  37. [48]

    Cross modal transformer: Towards fast and robust 3d object detection

    Yan, J., Liu, Y., Sun, J., Jia, F., Li, S., Wang, T., and Zhang, X. Cross modal transformer: Towards fast and robust 3d object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 18268--18278, 2023

    2023

  38. [49]

    2dpass: 2d priors assisted semantic segmentation on lidar point clouds

    Yan, X., Gao, J., Zheng, C., Zheng, C., Zhang, R., Cui, S., and Li, Z. 2dpass: 2d priors assisted semantic segmentation on lidar point clouds. In European Conference on Computer Vision, pp.\ 677--695. Springer, 2022

    2022

  39. [50]

    4d panoptic scene graph generation

    Yang, J., Cen, J., Peng, W., Liu, Shuai amd Hong, F., Li, X., Zhou, K., Chen, Q., and Liu, Z. 4d panoptic scene graph generation. In NeurIPS, 2023

    2023

  40. [51]

    Dqformer: Toward unified lidar panoptic segmentation with decoupled queries for large-scale outdoor scenes

    Yang, Y., Mei, J., Du, S., Xiao, Y., Wu, H., Xu, X., and Liu, Y. Dqformer: Toward unified lidar panoptic segmentation with decoupled queries for large-scale outdoor scenes. IEEE Transactions on Geoscience and Remote Sensing, 63: 0 1--15, 2025. doi:10.1109/TGRS.2025.3558951

    2025

  41. [52]

    Lidarmultinet: towards a unified multi-task network for lidar perception

    Ye, D., Zhou, Z., Chen, W., Xie, Y., Wang, Y., Wang, P., and Foroosh, H. Lidarmultinet: towards a unified multi-task network for lidar perception. In Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applicat...

    2023 doi

  42. [53]

    Is-fusion: Instance-scene collaborative fusion for multimodal 3d object detection

    Yin, J., Shen, J., Chen, R., Li, W., Yang, R., Frossard, P., and Wang, W. Is-fusion: Instance-scene collaborative fusion for multimodal 3d object detection. In CVPR, 2024

    2024

  43. [54]

    a henb \

    Yin, T., Zhou, X., and Kr \"a henb \"u hl, P. Center-based 3d object detection and tracking. CVPR, 2021

    2021

  44. [55]

    SparseLIF: High-Performance Sparse LiDAR-Camera Fusion for 3D Object Detection, pp.\ 109--128

    Zhang, H., Liang, L., Zeng, P., Song, X., and Wang, Z. SparseLIF: High-Performance Sparse LiDAR-Camera Fusion for 3D Object Detection, pp.\ 109--128. 09 2024. ISBN 978-3-031-72760-3. doi:10.1007/978-3-031-72761-0_7

    2024 doi

  45. [56]

    Lidar-camera panoptic segmentation via geometry-consistent and semantic-aware alignment

    Zhang, Z., Zhang, Z., Yu, Q., Yi, R., Xie, Y., and Ma, L. Lidar-camera panoptic segmentation via geometry-consistent and semantic-aware alignment. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp.\ 3662--3671, October 2023

    2023

  46. [57]

    Unimix: Towards domain adaptive and generalizable lidar semantic segmentation in adverse weather

    Zhao, H., Zhang, J., Chen, Z., Zhao, S., and Tao, D. Unimix: Towards domain adaptive and generalizable lidar semantic segmentation in adverse weather. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 14781--14791, June 2024

    2024

  47. [58]

    Zhao, N., Chua, T.-S., and Lee, G. H. Few-shot 3d point cloud semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 8873--8882, 2021

    2021

  48. [59]

    Zhao, Y., Zhao, N., and Lee, G. H. Synthetic-to-real domain generalized semantic segmentation for 3d indoor point clouds. arXiv preprint arXiv:2212.04668, 2022

    2022 arXiv

  49. [60]

    Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentation

    Zhou, Z., Zhang, Y., and Foroosh, H. Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentation . In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 13189--13198, Los Alamitos, CA, USA, June 2021. IEEE Computer Society. doi:10.110...

    2021

  50. [61]

    Cylindrical and asymmetrical 3d convolution networks for lidar segmentation

    Zhu, X., Zhou, H., Wang, T., Hong, F., Ma, Y., Li, W., Li, H., and Lin, D. Cylindrical and asymmetrical 3d convolution networks for lidar segmentation. arXiv preprint arXiv:2011.10033, 2020

    2011 arXiv

  51. [62]

    Perception-aware multi-sensor fusion for 3d lidar semantic segmentation

    Zhuang, Z., Li, R., Jia, K., Wang, Q., Li, Y., and Tan, M. Perception-aware multi-sensor fusion for 3d lidar semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 16280--16290, 2021

    2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.