Pith. sign in

REVIEW 5 major objections 6 minor 41 references

Self-Supervised Multimodal NeRF for Autonomous Driving

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A self-supervised multimodal NeRF jointly synthesizes LiDAR point clouds and camera images from one 4D representation, with no 3D labels.

desk verdict A modest but real engineering extension of LiDAR4D with one strong LiDAR result, one overclaimed camera comparison, and a missing verification of the self-supervised flow that needs to be answered in revision. read the letter →

arxiv 2506.19615 v2 pith:ZOSUMP3U submitted 2025-06-24 cs.CV

Gaurav Sharma , Ravi Kothari , Josef Schmid This is my paper
classification cs.CV
keywords neuralradiancefieldsnovelviewsynthesisself-supervisedlearningLiDARpointcloudmultimodalscenerepresentationdynamicscenesautonomousdrivingflow
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that a single neural scene representation can synthesize novel LiDAR point clouds and camera images for a driving scene, including moving objects, without any 3D annotations. The proposed framework, NVSF, learns a 4D spatio-temporal radiance field from raw LiDAR and camera logs, using a scene-flow module to separate static background from dynamic foreground in a self-supervised way. If true, this matters because high-fidelity synthetic sensor data could be generated from unlabeled logs, reducing the cost of annotation and enabling sensor-configuration experiments. The paper reports best or near-best LiDAR point-cloud metrics against LiDAR-NeRF and LiDAR4D on KITTI-360, with the largest gains on dynamic foreground objects, and also produces camera views.

What carries the argument

The central machinery is a 4D hybrid hash-plane encoding built from multi-resolution hash grids and K-Planes-style 2D plane factorisations, which encodes spatial and temporal coordinates. A Scene Flow MLP trained with Chamfer distance on adjacent LiDAR frames separates static background from moving foreground, while separate 4D encodings for LiDAR and Camera feed a shared NeRF MLP that predicts density and global features. Three radiance heads then produce LiDAR intensity and raydrop as well as camera RGB, with a U-Net refining raydrop globally. Two training-side mechanisms carry much of the reported gain: a heuristic multinomial pixel sampler that prefers high-loss pixels, and a double-gradient clipping mask that directs structural regularization toward geometrically structured regions including distant sparse ones.

What would settle it

Train NVSF on a KITTI-360 sequence with a fast-moving, partially occluded vehicle and compare its synthesized foreground LiDAR points with the sequence's ground-truth foreground points; if the point-to-point error rises well above the reported 0.111 toward the 0.255 level of LiDAR4D, the self-supervised scene-flow separation is not carrying the claimed dynamic-scene improvement.

Watch

Extended reading notes

Core claim

The paper's central claim is that multimodal novel-view synthesis for autonomous driving does not require 3D supervision: a NeRF-based network can jointly learn space and time from LiDAR and camera data and, in doing so, reconstruct both sensor modalities at new viewpoints. NVSF couples separate 4D hash-plane encoders for LiDAR and camera with a shared density MLP, and relies on a scene-flow MLP trained by Chamfer distance between adjacent LiDAR frames to split static and dynamic content. On KITTI-360, it reports the best overall point-cloud Chamfer distance (0.091 vs 0.114 for LiDAR-NeRF and 0.129 for LiDAR4D), and the largest margin on foreground dynamic points (0.111 vs 0.255 and 0.758), alongside improved intensity, raydrop, and camera metrics. The paper presents this as evidence that a multimodal self-supervised NeRF can replace label-hungry dynamic NeRFs without sacrificing synthesis quality.

Load-bearing premise

The load-bearing assumption is that a module trained only to match consecutive laser scans can tell moving objects from the static background without any object labels; if that separation is unreliable for distant, occluded, or fast-moving objects, the claimed gains on dynamic scenes do not transfer.

Editorial extensions

If this is right

  • Raw LiDAR and camera logs, without 3D labels, can be turned into a component that renders new viewpoints in both modalities, lowering the annotation barrier for sensor-simulation training data.
  • A single canonical 4D scene representation can output geometry, intensity, raydrop, and RGB, so sensor intrinsics like beam pattern or camera pose can be varied while reusing one learned scene.
  • Dynamic foreground reconstruction is where the gap is largest: foreground Chamfer distance drops from 0.255 for LiDAR4D and 0.758 for LiDAR-NeRF to 0.111, and foreground F-score rises to 0.720.
  • Both proposed training-side mechanisms contribute: the heuristic pixel sampler and the double-gradient mask each improve point-cloud and intensity metrics in the ablations, with the mask giving the larger individual gain.
  • The current formulation confines synthesis to the ego-vehicle trajectory, so editing or manipulating elements outside that trajectory is not supported until the static and dynamic components are decoupled more freely.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the scene-flow decomposition transfers to other raw urban logs, unlabeled LiDAR-camera data could become a source of synthetic dynamic scenes with free labels, effectively bootstrapping downstream perception training.
  • Editorial inference: the motion-based static/dynamic split that emerges from the 4D encoding could be probed as an unsupervised objectness cue for tracking or segmentation, a use the paper does not explore.
  • Editorial inference: the finding that separate per-modality encoders outperform shared or fused encoders suggests a general design rule for multimodal radiance fields, which could be tested on other sensor pairs such as radar and camera.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes NVSF, a NeRF-based framework for joint novel view synthesis of LiDAR point clouds and camera images in dynamic autonomous-driving scenes. The method combines a 4D hybrid hash-plane encoding with a scene-flow MLP for self-supervised static/dynamic decomposition, a heuristic pixel-sampling scheme based on reconstruction error, and a double-gradient clipping mask for structural LiDAR loss. The framework is trained on KITTI-360 and compared against LiDAR-NeRF and LiDAR4D, reporting lower Chamfer distance and higher F-score for LiDAR synthesis in Table III, as well as camera-image metrics in Table IV. The central claim is that a self-supervised multimodal dynamic NeRF can achieve state-of-the-art LiDAR and camera synthesis without 3D labels.

Significance. If the claims are fully supported, the paper would be a valuable contribution: it demonstrates joint LiDAR-camera NeRF synthesis without 3D annotations on a public driving dataset, releases code, and reports gains over two recent LiDAR NeRF baselines. The strongest evidence is the LiDAR evaluation in Table III, where NVSF improves foreground Chamfer distance from 0.255 (LiDAR4D) and 0.758 (LiDAR-NeRF) to 0.111. However, the significance is tempered by three gaps: the camera evaluation has no baseline, the scene-flow mechanism that underlies the self-supervised dynamic decomposition is not verified, and no uncertainty estimates are provided for any metric. These issues affect the reproducibility and the strength of the central claims, but they are addressable with additional experiments and reporting.

major comments (5)
  1. [§IV.C, Table IV] The camera evaluation is reported without any baseline, yet the abstract claims 'best performance on both LiDAR and Camera domain.' Table IV lists only NVSF metrics. The comparison with NeuRAD's PSNR 27 on KITTI MOT (§IV.C) is not a controlled baseline because the dataset, split, and evaluation protocol differ. Please add at least one image-NeRF baseline (e.g., NeuRAD, a dynamic NeRF, or a static NeRF using the same backbone) evaluated on the same KITTI-360 scenes and train/test split, and report per-scene results with variance. Without this, the camera-domain claim is unsupported.
  2. [§III.C, Eq. (5)] The self-supervised dynamic scene decomposition relies entirely on the scene-flow loss Lflow, but the paper provides no evidence that the learned flow is nonzero or accurate on foreground points. In KITTI-360 driving scenes, static background points vastly outnumber moving-object points, so a zero-flow solution already minimizes the Chamfer distance on static regions. The paper should include: (a) quantitative flow evaluation (e.g., EPE or accuracy on moving-object points), (b) qualitative flow visualizations, or (c) an ablation with λ1=0 to demonstrate that the flow term is necessary for the foreground results in Table III. Without such evidence, the foreground CD improvements cannot be attributed to temporal object binding rather than per-frame memorization of the 60 training frames.
  3. [§IV.B, Table III] All metrics in Table III are reported as single numbers without error bars, multiple seeds, or per-scene breakdowns. Some improvements are small (e.g., PCD CD 0.091 vs 0.114, intensity RMSE 0.115 vs 0.117) and may be within run-to-run variability. Please report mean±std over at least three training runs or over the 10 individual scenes, and state the number of runs. In addition, define the 'F-score' and 'Acc.' columns (threshold, matching radius, and whether they are computed on the full point cloud or foreground only); currently these are not described.
  4. [§IV.A, Table II] The ablation study is run on an unspecified 'subset of scenes' and the components are not defined. HS, FA, and ML are not explained in the text; in particular, 'Multimodal Ray sampling' (ML) is never described in the method section. The ablation also omits camera metrics and does not compare the proposed double-gradient mask against the single-gradient mask used by LiDAR-NeRF and LiDAR4D, even though the double-gradient novelty is a stated contribution. Please specify the scenes, define each component, include camera metrics, and add a single-gradient-mask baseline to isolate the effect of the double-gradient mask.
  5. [§III.B] The heuristic sampling method is said to be 'given the Equation 4,' but Eq. (4) is the Chamfer distance; no equation for the multinomial sampling distribution based on reconstruction error is provided. The alternation schedule between heuristic and random sampling is also described only verbally. This is one of the three claimed contributions, and the missing formula makes it non-reproducible. Please provide the exact probability definition (including how reconstruction error is normalized), the jittering procedure, and the epoch-level alternation schedule, or give a precise reference to the released code.
minor comments (6)
  1. [§III.B] Equation cross-referencing is inconsistent: the heuristic sampling is referenced as 'Equation 4' but Eq. (4) is the Chamfer distance, and the depth rendering equation is labeled Eq. (3) while earlier equations are labeled Eq. (1) and Eq. (2). Please renumber or correct the references.
  2. [References] Reference [39] for 'PCGen' cites Allen and Baldwin (1980), a FORTRAN IV program for paired-comparison stimuli, which appears unrelated to the point-cloud generation context. Please verify and replace with the correct PCGen reference.
  3. [References] Reference [40] is cited as 'KITTI MOT dataset' in §IV.C, but the reference is the MOTS paper (Voigtlaender et al.). Please correct the citation to the actual KITTI MOT or MOTS dataset paper.
  4. [§II, Table III] The text says 'Higher values of PSNR suggest that the synthesised point cloud has less noise,' but PSNR is not a standard point-cloud metric and this claim is not self-evident. Please clarify how PSNR is computed for point clouds (e.g., on range images) and what it measures.
  5. [§IV.D/Implementation] The implementation states 'batch size of 2048 rays per image'; this is ambiguous because LiDAR pano images and camera images have different ray counts. Please specify the total number of LiDAR rays and camera rays per iteration, and the sampling ratio between modalities.
  6. [Throughout] There are several typos and grammatical slips, e.g., 'A V' for 'AV' in the introduction, 'Pano image' capitalization inconsistencies, and 'the proposed framework has reported best performance' in the abstract. A careful proofread is recommended.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: NVSF's contributions are evaluated against external baselines and its components are borrowed transparently, with no fitted parameter renamed as a prediction.

full rationale

The paper's central claims are supported by comparisons to external baselines, LiDAR-NeRF [17] and LiDAR4D [4], whose codebases are publicly available and not authored by the present authors. The method explicitly borrows the scene-flow network, 4D hybrid encoding, and U-Net refinement from LiDAR4D and K-Planes; this is transparent reuse of prior work, not a self-citation chain. The loss functions in Equations 4-10 optimize reconstruction consistency directly against LiDAR and camera observations; no fitted parameter is later renamed as a prediction, and no equation is defined in terms of the result it is supposed to establish. The foreground evaluation uses KITTI-360 3D annotations only for metric separation, not as training supervision, so the reported foreground numbers are not forced by construction. The ablation in Table II and the encoding-strategy comparison in Table I are empirical searches over design choices, not circular derivations. The paper's stated limitation about foreground/background binding restricting synthesis to the ego-vehicle trajectory is a scope limitation, not a circularity. Although the baseline models were retrained by the same authors, that is standard evaluation practice and does not reduce any claim to its own input. No circular step could be identified with quotable evidence.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical entities. Its central claims rest on three domain assumptions: one-surface LiDAR rendering, Chamfer-trained scene flow for unsupervised dynamic decomposition, and implicit multimodal fusion. The loss weights and thresholds are free parameters set without sensitivity analysis.

free parameters (3)
  • Loss weights lambda1-lambda6 = 1.0, 1.0, 0.1, 0.01, 0.01, 1.0
    Set by hand (Section IV.D) to balance flow, depth, intensity, raydrop, gradient, and RGB terms; the reported performance depends on this weighting.
  • Double gradient threshold tau = not reported
    Threshold in Equation 2 controls how many regions receive gradient loss; no value or sensitivity analysis is given in the paper.
  • Heuristic sampling alternation schedule = alternating epochs, exact schedule not specified
    The framework alternates between heuristic and random sampling across training epochs; the schedule is a free choice affecting convergence and final metrics.
assumptions (3)
  • domain assumption Volumetric rendering with expected depth (Eq. 3) is a valid model for LiDAR range, treating each beam's first return as the dominant surface.
    The method renders depth by accumulating transmittance along rays, which assumes opaque, single-surface returns; multi-surface effects and beam divergence are not modeled.
  • domain assumption Scene flow between adjacent frames, trained only with Chamfer distance (Eqs. 4-5), is sufficient to disentangle static background from dynamic foreground without object labels.
    This is the load-bearing premise for self-supervised dynamic decomposition; if flow is inaccurate for distant or occluded objects, foreground reconstruction quality collapses.
  • domain assumption Separate 4D hash-planar encodings for LiDAR and camera, feeding a shared NeRF MLP, can align the two modalities without explicit geometric calibration or learned fusion.
    Section III.C reports this encoding choice performed best, but no theoretical or calibration-based justification is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Self-Supervised Multimodal NeRF for Autonomous Driving." pith.science (2026). https://pith.science/paper/ZOSUMP3U

@misc{pith2026250619615,
  author       = {Pith},
  title        = {Pith review of: Self-Supervised Multimodal NeRF for Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZOSUMP3U}},
  note         = {Machine review of arXiv:2506.19615}
}
read the original abstract

In this paper, we propose a Neural Radiance Fields (NeRF) based framework, referred to as Novel View Synthesis Framework (NVSF). It jointly learns the implicit neural representation of space and time-varying scene for both LiDAR and Camera. We test this on a real-world autonomous driving scenario containing both static and dynamic scenes. Compared to existing multimodal dynamic NeRFs, our framework is self-supervised, thus eliminating the need for 3D labels. For efficient training and faster convergence, we introduce heuristic-based image pixel sampling to focus on pixels with rich information. To preserve the local features of LiDAR points, a Double Gradient based mask is employed. Extensive experiments on the KITTI-360 dataset show that, compared to the baseline models, our framework has reported best performance on both LiDAR and Camera domain. Code of the model is available at https://github.com/gaurav00700/Selfsupervised-NVSF

Figures

Figures reproduced from arXiv: 2506.19615 by the authors.

Figure 1
Figure 1. Joint Novel View Synthesis for Camera and LiDAR [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The proposed network consists of Scene flow MLP for learning the temporal features, 4D hybrid hash-planes encoding [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Compared to a single-gradient approach (a), our [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Figure shows the Heuristic based pixel sampling [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Comparison of error map of synthesised PCD by [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 33 canonical work pages

  1. [1]

    CARLA: An open urban driving simulator,

    A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V . Koltun, “CARLA: An open urban driving simulator,” in Conference on robot learning. PMLR, 2017, pp. 1–16

    work page 2017

  2. [2]

    LiDARsim: Realistic LiDAR simulation by leveraging the real world,

    S. Manivasagam, S. Wang, K. Wong, W. Zeng, M. Sazanovich, S. Tan, B. Yang, W.-C. Ma, and R. Urtasun, “LiDARsim: Realistic LiDAR simulation by leveraging the real world,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 11 167–11 176

    work page 2020

  3. [3]

    NeRF: Representing scenes as neural radiance fields for view synthesis,

    B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoor- thi, and R. Ng, “NeRF: Representing scenes as neural radiance fields for view synthesis,” Communications of the ACM , vol. 65, no. 1, pp. 99–106, 2021

    2021

  4. [4]

    LiDAR4D: Dynamic Neural Fields for Novel Space-time View LiDAR Synthesis

    Z. Zheng, F. Lu, W. Xue, G. Chen, and C. Jiang, “LiDAR4D: Dynamic Neural Fields for Novel Space-time View LiDAR Synthesis,” arXiv preprint arXiv:2404.02742, 2024

    work page Pith review arXiv 2024

  5. [5]

    K-planes: Explicit radiance fields in space, time, and appearance,

    S. Fridovich-Keil, G. Meanti, F. R. Warburg, B. Recht, and A. Kanazawa, “K-planes: Explicit radiance fields in space, time, and appearance,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 12 479–12 488

    work page 2023

  6. [6]

    Neural Scene Graphs for Dynamic Scenes,

    J. Ost, F. Mannan, N. Thuerey, J. Knodt, and F. Heide, “Neural Scene Graphs for Dynamic Scenes,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 2856–2865

    work page 2021

  7. [7]

    Mip-NeRF 360: Unbounded anti-aliased neural radiance fields,

    J. T. Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan, and P. Hedman, “Mip-NeRF 360: Unbounded anti-aliased neural radiance fields,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 5470–5479

    work page 2022

  8. [8]

    Zip-NeRF: Anti-aliased grid-based neural radiance fields,

    ——, “Zip-NeRF: Anti-aliased grid-based neural radiance fields,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 19 697–19 705

    work page 2023

Show all 41 references
  1. [9]

    Instant neural graphics primitives with a multiresolution hash encoding,

    T. M ¨uller, A. Evans, C. Schied, and A. Keller, “Instant neural graphics primitives with a multiresolution hash encoding,” ACM transactions on graphics (TOG) , vol. 41, no. 4, pp. 1–15, 2022

    2022

  2. [10]

    Tensorf: Tensorial ra- diance fields,

    A. Chen, Z. Xu, A. Geiger, J. Yu, and H. Su, “Tensorf: Tensorial ra- diance fields,” in European conference on computer vision. Springer, 2022, pp. 333–350

    2022

  3. [11]

    Plenoxels: Radiance fields without neural networks,

    S. Fridovich-Keil, A. Yu, M. Tancik, Q. Chen, B. Recht, and A. Kanazawa, “Plenoxels: Radiance fields without neural networks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 5501–5510

    2022

  4. [12]

    Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction,

    C. Sun, M. Sun, and H.-T. Chen, “Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction,” in Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 5459–5469

    2022

  5. [13]

    Efficient geometry-aware 3d generative adversarial networks,

    E. R. Chan, C. Z. Lin, M. A. Chan, K. Nagano, B. Pan, S. De Mello, O. Gallo, L. J. Guibas, J. Tremblay, S. Khamis, et al. , “Efficient geometry-aware 3d generative adversarial networks,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022,...

    2022

  6. [14]

    Tri- MipRF: Tri-Mip representation for efficient anti-aliasing neural radi- ance fields,

    W. Hu, Y . Wang, L. Ma, B. Yang, L. Gao, X. Liu, and Y . Ma, “Tri- MipRF: Tri-Mip representation for efficient anti-aliasing neural radi- ance fields,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 19 774–19 783

    2023

  7. [15]

    LidaRF: Delving into LiDAR for neural radiance field on street scenes,

    S. Sun, B. Zhuang, Z. Jiang, B. Liu, X. Xie, and M. Chandraker, “LidaRF: Delving into LiDAR for neural radiance field on street scenes,” arXiv preprint arXiv:2405.00900 , 2024

    2024 arXiv

  8. [16]

    Block-NeRF: Scalable large scene neural view synthesis,

    M. Tancik, V . Casser, X. Yan, S. Pradhan, B. Mildenhall, P. P. Srinivasan, J. T. Barron, and H. Kretzschmar, “Block-NeRF: Scalable large scene neural view synthesis,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 8248–8258

    2022

  9. [17]

    LiDAR-NeRF: Novel LiDAR view synthe- sis via neural radiance fields,

    T. Tao, L. Gao, G. Wang, Y . Lao, P. Chen, H. Zhao, D. Hao, X. Liang, M. Salzmann, and K. Yu, “LiDAR-NeRF: Novel LiDAR view synthe- sis via neural radiance fields,” arXiv preprint arXiv:2304.10406, 2023

    2023 arXiv

  10. [18]

    NeRF-LiDAR: Gener- ating Realistic LiDAR Point Clouds with Neural Radiance Fields,

    J. Zhang, F. Zhang, S. Kuang, and L. Zhang, “NeRF-LiDAR: Gener- ating Realistic LiDAR Point Clouds with Neural Radiance Fields,” in AAAI Conference on Artificial Intelligence (AAAI) , 2024

    2024

  11. [19]

    Urban radiance fields,

    K. Rematas, A. Liu, P. P. Srinivasan, J. T. Barron, A. Tagliasacchi, T. Funkhouser, and V . Ferrari, “Urban radiance fields,” in Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 12 932–12 942

    2022

  12. [20]

    Neural fields meet explicit geometric representations for inverse rendering of urban scenes,

    Z. Wang, T. Shen, J. Gao, S. Huang, J. Munkberg, J. Hasselgren, Z. Gojcic, W. Chen, and S. Fidler, “Neural fields meet explicit geometric representations for inverse rendering of urban scenes,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...

    2023

  13. [21]

    NeuRAD: Neural rendering for autonomous driving,

    A. Tonderski, C. Lindstr ¨om, G. Hess, W. Ljungbergh, L. Svensson, and C. Petersson, “NeuRAD: Neural rendering for autonomous driving,” arXiv preprint arXiv:2311.15260 , 2023

    2023 arXiv

  14. [22]

    UniSim: A Neural Closed-Loop Sensor Simulator,

    Z. Yang, Y . Chen, J. Wang, S. Manivasagam, W.-C. Ma, A. J. Yang, and R. Urtasun, “UniSim: A Neural Closed-Loop Sensor Simulator,” in CVPR, 2023

    2023

  15. [23]

    Neural LiDAR fields for novel view synthesis,

    S. Huang, Z. Gojcic, Z. Wang, F. Williams, Y . Kasten, S. Fidler, K. Schindler, and O. Litany, “Neural LiDAR fields for novel view synthesis,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 18 236–18 246

    2023

  16. [24]

    NeRF-LiDAR: Gen- erating realistic LiDAR point clouds with neural radiance fields,

    J. Zhang, F. Zhang, S. Kuang, and L. Zhang, “NeRF-LiDAR: Gen- erating realistic LiDAR point clouds with neural radiance fields,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 7, 2024, pp. 7178–7186

    2024

  17. [25]

    Neural Scene Flow Prior,

    X. Li, J. Kaesemodel Pontes, and S. Lucey, “Neural Scene Flow Prior,” in Advances in Neural Information Processing Systems , M. Ranzato, A. Beygelzimer, Y . Dauphin, P. Liang, and J. W. Vaughan, Eds., vol. 34. Curran Associates, Inc., 2021, pp. 7838–

    2021

  18. [26]

    MARS: An instance-aware, modular and realistic simulator for autonomous driving,

    Z. Wu, T. Liu, L. Luo, Z. Zhong, J. Chen, H. Xiao, C. Hou, H. Lou, Y . Chen, R. Yang, et al. , “MARS: An instance-aware, modular and realistic simulator for autonomous driving,” in CAAI International Conference on Artificial Intelligence . Springer, 2023, pp. 3–15

    2023

  19. [27]

    Non-uniform sampling strate- gies for NeRF on 360° images

    T. Otonari, S. Ikehata, and K. Aizawa, “Non-uniform sampling strate- gies for NeRF on 360° images.” in BMVC, 2022, p. 344

    2022

  20. [28]

    Depth-supervised NeRF: Fewer Views and Faster Training for Free,

    K. Deng, A. Liu, J.-Y . Zhu, and D. Ramanan, “Depth-supervised NeRF: Fewer Views and Faster Training for Free,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2022, pp. 12 872–12 881

    2022

  21. [29]

    AlignMiF: Geometry-Aligned Multimodal Implicit Field for LiDAR- Camera Joint Synthesis,

    T. Tang, G. Wang, Y . Lao, P. Chen, J. Liu, L. Lin, K. Yu, and X. Liang, “AlignMiF: Geometry-Aligned Multimodal Implicit Field for LiDAR- Camera Joint Synthesis,” arXiv preprint arXiv:2402.17483 , 2024

    2024 arXiv

  22. [30]

    Balanced multimodal learning via on-the-fly gradient modulation,

    X. Peng, Y . Wei, A. Deng, D. Wang, and D. Hu, “Balanced multimodal learning via on-the-fly gradient modulation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 8238–8247

    2022

  23. [31]

    Learning to balance the learning rates between various modalities via adaptive tracking factor,

    Y . Sun, S. Mai, and H. Hu, “Learning to balance the learning rates between various modalities via adaptive tracking factor,” IEEE Signal Processing Letters, vol. 28, pp. 1650–1654, 2021

    2021

  24. [32]

    What makes training multi- modal classification networks hard?

    W. Wang, D. Tran, and M. Feiszli, “What makes training multi- modal classification networks hard?” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 12 695–12 705

    2020

  25. [33]

    Neural scene flow prior,

    X. Li, J. Kaesemodel Pontes, and S. Lucey, “Neural scene flow prior,” Advances in Neural Information Processing Systems , vol. 34, pp. 7838–7851, 2021

    2021

  26. [34]

    Fast Neural Scene Flow,

    X. Li, J. Zheng, F. Ferroni, J. K. Pontes, and S. Lucey, “Fast Neural Scene Flow,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , October 2023, pp. 9878– 9890

    2023

  27. [35]

    NeuralPCI: Spatio-temporal Neural Field for 3D Point Cloud Multi-frame Non- linear Interpolation,

    Z. Zheng, D. Wu, R. Lu, F. Lu, G. Chen, and C. Jiang, “NeuralPCI: Spatio-temporal Neural Field for 3D Point Cloud Multi-frame Non- linear Interpolation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 909–918

    2023

  28. [36]

    DynIBaR: Neural dynamic image-based rendering,

    Z. Li, Q. Wang, F. Cole, R. Tucker, and N. Snavely, “DynIBaR: Neural dynamic image-based rendering,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 4273–4284

    2023

  29. [37]

    KITTI-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d,

    Y . Liao, J. Xie, and A. Geiger, “KITTI-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 45, no. 3, pp. 3292–3310, 2022

    2022

  30. [38]

    Adam: A method for stochastic optimiza- tion,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimiza- tion,” arXiv preprint arXiv:1412.6980 , 2014

    2014 arXiv

  31. [39]

    PCGen: A FORTRAN IV program to generate paired-comparison stimuli,

    G. A. Allen and L. M. Baldwin, “PCGen: A FORTRAN IV program to generate paired-comparison stimuli,” Behavior Research Methods & Instrumentation, vol. 12, no. 3, pp. 383–384, 1980

    1980

  32. [40]

    MOTS: Multi-Object Tracking and Segmen- tation,

    P. V oigtlaender, M. Krause, A. Osep, J. Luiten, B. B. G. Sekar, A. Geiger, and B. Leibe, “MOTS: Multi-Object Tracking and Segmen- tation,” in Conference on Computer Vision and Pattern Recognition (CVPR), 2019

    2019

  33. [7851]

    Available: https://proceedings.neurips.cc/paper files/ paper/2021/file/41263b9a46f6f8f22668476661614478-Paper.pdf

    [Online]. Available: https://proceedings.neurips.cc/paper files/ paper/2021/file/41263b9a46f6f8f22668476661614478-Paper.pdf

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.