{"id":"d6af3ed2-a308-4437-a9f1-c86d58452f1a","arxiv_id":"2411.16816","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A 3D Gaussian Splatting method renders dynamic driving scenes for both cameras and lidar in real time, using spherical-coordinate lidar rasterization and rolling shutter compensation.","lead":"SplatAD renders both camera images and lidar point clouds from a 3D Gaussian Splatting model of a driving scene, at real-time speeds. It is the first Gaussian-splatting method to handle both sensor types, reporting state-of-the-art image quality with roughly ten to twenty times faster rendering than NeRF-based simulators.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Constant-velocity assumption is baked into lidar GT preprocessing (Appendix A.2), so the 0.01 m depth error may measure consistency with the linear model, not physical accuracy.","rationale":"The reader's weakest assumption correctly points to the constant-velocity rolling-shutter model. I sharpen this concern: the same linear-motion assumption is used to preprocess the ground-truth lidar points (Appendix A.2), so the NVS lidar evaluation is not an independent test of the model's physical accuracy. The 0.01 m depth error on PandaSet is an internal-consistency score between the renderer and a linearized reference; any high-acceleration error is removed from both sides before the metric is computed. This is more load-bearing than a purely forward-modeling approximation because it can systematically optimize the evaluation metric. The proposed test, re-un-compensating with higher-rate poses, would settle the concern. If it lands, the lidar-side claims in Tables 2 and 3 would need rescoping, though the camera-side PSNR gains and the speedup are likely unaffected because they do not depend on this GT processing. The reader's verdict of CONDITIONAL remains appropriate: the methodological novelty and camera results are credible, but the lidar evaluation must be re-validated. Hence I recommend no change to the verdict.","tokens_in":19267,"tokens_out":14976,"duration_ms":140299,"concrete_test":"Re-run the lidar evaluation using higher-rate sensor poses (e.g., 100 Hz IMU/GPS data or cubic-spline interpolation of the provided pose stream) to un-compensate the point cloud, instead of the linear motion model in Appendix A.2. Recompute SplatAD's median depth error, intensity error, and Chamfer distance on the same 10 sequences per dataset, for each frame. If depth error grows above the reported 0.01 m (e.g., above 0.05 m) on frames where ego or actor acceleration exceeds 2 m/s², the linear GT preprocessing is materially biasing the result. A complementary check: use a synthetic or controlled scenario with a known high-acceleration braking profile, render SplatAD's range, and compare to the true known range rather than to the linearized GT.","verdict_should_be":"UNCHANGED","load_bearing_attack":"SplatAD's lidar evaluation uses ground-truth points that have been un-compensated from the dataset's ego-motion-compensated frame using the same linear-velocity model that the renderer uses. Appendix A.2 states: 'We begin by removing any ego-motion compensation by expressing the lidar points’ coordinates relative to the sensor pose at the time of capture. For this, we assume a linear motion during the lidar scan’s capture.' Thus the GT point positions are derived via constant-velocity interpolation, exactly the assumption used in the renderer's rolling-shutter compensation (Eq. 11 and Eq. 12). Any acceleration or braking during the 100 ms sweep is erased from both the rendered range and the reference range. The reported 0.01 m depth error therefore reflects how well SplatAD reproduces the linear-model approximation, not how accurately it renders physical scenes under real vehicle dynamics. This is a circular evaluation for the rolling-shutter component and is the most load-bearing point: the central lidar claim (Table 2) may be inflated by the GT preprocessing. The ablation in Table 5 (row f) shows depth error rising from 0.02 m to 0.05 m when the renderer-side lidar rolling-shutter compensation is removed, but even that residual is measured against the linearized GT, so it does not bound the bias. The method may still be accurate in low-acceleration conditions, but the published evaluation cannot distinguish that.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"SplatAD proposes a 3D Gaussian Splatting framework that renders both camera images and lidar point clouds from a unified scene representation for dynamic autonomous driving scenes. The method extends 3DGS with per-Gaussian learnable features, a scene-graph decomposition for rigid dynamic actors, rolling-shutter compensation in image and spherical sensor coordinates, a custom tile-based lidar rasterizer, and lightweight CNN/MLP decoders for image color, lidar intensity, and ray-drop probability. Experiments on PandaSet, Argoverse 2, and nuScenes compare against UniSim, NeuRAD, PVG, Street Gaussians, and OmniRe. The paper reports state-of-the-art image novel-view-synthesis metrics, lidar depth and intensity errors matching NeuRAD, and an order-of-magnitude rendering speedup over NeRF-based baselines.","tokens_in":19562,"tokens_out":8157,"duration_ms":80081,"significance":"The paper addresses an important practical gap: previous 3DGS methods for autonomous driving render only cameras, while NeRF-based sensor simulators are too slow for large-scale closed-loop testing. SplatAD's unified representation, custom CUDA kernels for spherical lidar rasterization, and explicit modeling of rolling shutter, intensity, and ray dropouts are valuable contributions. The authors use official baseline implementations, provide ablations for each design choice, and make their approach built on open-source frameworks. However, the quantitative claims about lidar rendering quality are weakened by methodological issues in the evaluation protocol, specifically the circular use of the linear-motion assumption in the ground-truth construction and the unequal point sets used for comparison. If these issues are addressed, the paper would provide a solid basis for real-time multi-modal sensor simulation.","major_comments":[{"comment":"The lidar ground truth used for evaluation is constructed by un-compensating the dataset's ego-motion-compensated point cloud using the same linear-velocity assumption that the renderer applies in Eq. (11)-(12). Appendix A.2 states that when removing ego-motion compensation, the authors assume a linear motion during the lidar scan's capture; Eq. (12) compensates the rendered range by the range-component of the relative velocity times the scan time offset. Both the reference and the prediction are therefore expressed in a coordinate frame defined by the linear motion model. The reported 0.01 m depth error on PandaSet measures agreement with that model, not physical accuracy under real vehicle dynamics such as acceleration or braking during the 100 ms sweep. The ablation in Table 5, row (f), compares against the same linearized reference and thus does not bound the potential bias. Please provide an evaluation against a reference that does not inherit this assumption, for example the dataset's original compensated point cloud expressed in the sweep's reference frame, or per-point poses from an external high-rate odometry or IMU source. In addition, report the depth error separately on segments with high longitudinal or lateral acceleration.","section":"Appendix A.2, Eq. (11)-(12), Table 2"},{"comment":"The lidar comparison is not made on identical point sets. The 3DGS baselines (PVG, Street-GS, OmniRe) are evaluated only on existing lidar points because drivestudio cannot infer missing points, whereas SplatAD is evaluated on the full ray set after filtering with its predicted ray-drop probability. Since the dropout predictor is learned, SplatAD may discard difficult rays, making per-point depth error and Chamfer distance favorable. This undermines the headline claim of matching NeuRAD's depth error. Please add a table row in which SplatAD is evaluated on exactly the same non-missing point set as the baselines without dropout filtering, and, if feasible, also evaluate the baselines on the same full ray set after applying their own or a fixed dropout criterion.","section":"Sec. 4, Table 2, Appendix C"},{"comment":"During training, lidar points whose un-compensated coordinates produce more than 256 points in a tile are shuffled and discarded. The paper itself notes that this situation occurs when the linear motion assumption is violated. Consequently, the training distribution is censored with respect to the very events (heavy acceleration, braking, rapid turns) that would stress-test the constant-velocity approximation used for rolling-shutter compensation. The evaluation does not quantify how many points are discarded, nor whether the held-out frames contain such events. Please report the fraction of discarded points per dataset and analyze a subset of frames with aggressive maneuvers, or justify why the censoring is negligible for the claimed realism.","section":"Appendix A.2"}],"minor_comments":[{"comment":"The main quantitative results are reported as single-run averages without variance. Since the claims of state-of-the-art performance often hinge on differences of 0.5-1.5 PSNR or 0.01 m in depth error, please report standard deviations or confidence intervals across the 10 sequences per dataset.","section":"Tables 1-3"},{"comment":"Ablations are averaged over all three datasets, which can hide dataset-specific effects of rolling-shutter compensation, dropout handling, and EWA antialiasing. A per-dataset breakdown would make it clearer whether the modest quantitative effects are consistent across sensors with different beam counts and scan durations.","section":"Table 5"},{"comment":"The unit 'MR/s' for lidar rendering speed is introduced without definition; please define it as megarais per second (or equivalent) and clarify that it refers to the number of rendered lidar points per second, to avoid confusion with the camera 'MP/s' unit.","section":"Sec. 3.3 and Table 2"},{"comment":"There is a typo in the acknowledgements: 'valueable feedback' should be 'valuable feedback'.","section":"Acknowledgements"},{"comment":"The notation λ_MCMC in Eq. (13) is later decomposed into λ_o and λ_Σ in Appendix B; consider stating this decomposition in the main text to avoid confusion about the total loss weighting.","section":"Eq. (13) and Appendix B"},{"comment":"The claim of being the first 3DGS-based method for lidar rendering is categorical. Please soften it to 'to our knowledge' and, in the final version, cite any concurrent works that appeared since the first arXiv posting to keep the novelty statement accurate.","section":"Abstract and Intro"}],"recommendation":"major_revision","confidential_remarks":"The paper is a good fit for the journal and the camera-side results appear solid, with clear evidence for the speed advantage and reasonable image quality. The load-bearing issue is the lidar evaluation: the circular use of the linear-motion assumption in the ground truth, combined with unequal evaluation point sets relative to the baselines, makes the current lidar accuracy numbers uninterpretable as physical accuracy. I believe these issues are fixable without rearchitecting the method: evaluate SplatAD on the same non-missing point set as the baselines, compare against a non-linearized reference, and quantify discarded points. I therefore recommend major revision rather than rejection. I also note that the authors' close connection to the NeuRAD baseline (both from Zenseact) does not itself create circularity, since they use the official NeuRAD code and the lidar-rendering claim is independent of NeuRAD's correctness."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"SplatAD is worth a careful read: it is the first 3DGS method that actually renders lidar, and it does so at an order-of-magnitude speedup over NeRF-based approaches. The spherical-coordinate rasterizer, the unified camera+lidar representation, and the sensor-specific modeling (rolling shutter, intensity, dropout probability) are real engineering contributions. The ablation study shows each component matters, and the results across three AD datasets are strong and consistent. The paper is also honest about its limitation to rigid actors.\n\nThe main soft spot is the lidar depth evaluation. Appendix A.2 un-compensates the ground-truth lidar points using a linear-motion assumption during the sweep, and the renderer's rolling-shutter compensation uses exactly the same linear model. So the reported 0.01 m depth error on PandaSet measures how well SplatAD reproduces the implicit linear approximation, not physical accuracy under real acceleration or braking. This doesn't break the core contribution—the speedup and the first 3DGS lidar rasterizer are valuable on their own—but it means the rolling-shutter component is not validated against raw physical measurements. The authors should acknowledge this and ideally evaluate against non-compensated ranges or logs with known dynamics.\n\nOther concerns are more standard: no error bars, no code release, and the lidar comparison is slightly uneven (baselines evaluate only existing points, while SplatAD filters by predicted dropout probability). The self-citation to NeuRAD is fine, since they use the official implementation and the central claim is independent.\n\nFor a reader, this paper is most useful to people working on sensor simulation for autonomous driving and on extending 3DGS to non-camera modalities. It deserves a serious referee—the novelty holds up, and the methodological caveat is addressable in revision. I would bring it to a reading group and would cite it when discussing lidar rendering with 3DGS.","headline":"A genuine first for lidar rendering with 3DGS, but the rolling-shutter evaluation is self-consistent rather than physically validated.","tokens_in":20102,"tokens_out":4660,"would_cite":true,"duration_ms":48444,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SplatAD is the first 3D Gaussian Splatting method that renders both camera images and lidar point clouds in real time for dynamic driving scenes, reporting quality that matches or beats slower neural-radiance-field baselines.","keywords":["3D Gaussian Splatting","lidar rendering","camera rendering","autonomous driving simulation","novel view synthesis","rolling shutter","sensor-realistic simulation","dynamic scene rendering"],"falsifier":"Train SplatAD on a driving log that contains a hard deceleration or emergency lane change inside a lidar sweep, then compare the rendered median depth and ray-drop accuracy on held-out frames against the real lidar points on those braking frames; if the depth error jumps well beyond the reported 0.01–0.02 m and ray-drop accuracy drops below the NeRF baseline specifically in the accelerating portion of the sweep, the constant-velocity rolling-shutter assumption is the failing part.","tokens_in":19076,"feed_emoji":"🚗","tokens_out":9858,"duration_ms":83294,"temperature":0.7,"pith_summary":"SplatAD proposes that one explicit 3D Gaussian scene representation can serve as a digital twin for an autonomous-driving log, rendering both camera images and lidar point clouds in real time. The paper's central claim is that the sensor-specific effects that previously pushed lidar simulation into slow ray-tracing pipelines—rolling shutter, beam divergence, laser intensity, and rays that return nothing—can be folded into a rasterizer and absorbed by trainable per-Gaussian features. If that claim is right, closed-loop simulation for driving tests becomes about ten times cheaper, because the same scene can feed camera-based and lidar-based perception stacks at interactive speed. On three public driving datasets, SplatAD reports up to +2 PSNR on novel-view images and +3 PSNR on reconstruction, with lidar depth error matching the slower ray-tracing baseline.","feed_headline":"Lidar joins camera in real-time 3D Gaussian rendering","feed_subtitle":"SplatAD matches slower neural simulators on driving-scene quality while running an order of magnitude faster","key_machinery":"The central object is the 3D Gaussian scene graph: static background plus rigid dynamic actors, with Gaussian means and covariances expressed in each actor's local frame and composed into the world at render time. The argument is carried by three mechanisms. First, per-Gaussian learnable features replace spherical harmonics and are decoded by a small CNN for images and a lightweight MLP for lidar intensity and ray-drop probability, letting one representation serve both modalities. Second, rolling-shutter compensation is done in sensor space: each Gaussian's mean is shifted by a pixel velocity or spherical velocity that combines sensor motion with actor motion, scaled by the capture-time offset of the row or lidar point. Third, lidar rasterization tiles follow the sensor's own non-equidistant elevation channels and fixed azimuth resolution, so it splats directly in spherical coordinates instead of projecting into dense depth images and avoids wasted computation in sparse regions.","core_discovery":"On the paper's own account, SplatAD is the first 3D Gaussian Splatting method to render lidar and camera from the same dynamically composed scene. The scene is a set of 3D Gaussians split into a static background and rigid actors, each Gaussian carrying a base color and a 13-dimensional learnable feature; each sensor also gets a small learned embedding to absorb appearance differences. Cameras are rendered with the standard 3DGS tile rasterizer, modified to handle rolling shutter by shifting each Gaussian's image-space mean according to its velocity and the pixel's capture time. Lidar is rendered by a custom spherical-coordinate rasterizer with tiles aligned to the sensor's elevation channels and azimuth resolution, producing expected range, median range, intensity, and ray-drop probability. The paper reports that this matches the lidar quality of a ray-tracing NeRF baseline (0.01 m median depth error on PandaSet) while rendering lidar at 19.5 million points per second versus that baseline's 1.1, and beats all compared methods on image novel-view synthesis.","pith_inferences":["Beyond the paper: the same differentiable spherical lidar renderer could serve as a sensor proxy for gradient-based tuning of perception stacks or extrinsic calibration, a use the paper does not discuss.","Beyond the paper: the constant-velocity motion model is the most likely failure point, so a natural stress test is a sequence with mid-sweep braking; the method's depth error should rise exactly on those frames.","Beyond the paper: because appearance is partly carried by per-sensor learned embeddings, swapping embeddings at inference could edit exposure or sensor look without retraining, something the paper leaves implicit."],"forward_implications":["A single trained scene model can feed both camera-based and lidar-based perception stacks in closed-loop simulation, with no separate depth-image renderer for lidar.","Modeled ray dropouts yield realistic sparse point clouds, so downstream lidar perception sees missing returns rather than artificially dense geometry.","The reported order-of-magnitude speedup over NeRF-based simulators lets the same compute budget cover more scenarios or longer test drives.","The rolling-shutter compensation works in sensor space, so it transfers to any rolling-shutter camera with known row timing and to lidar sweeps of different durations.","Rendering lidar directly in spherical coordinates instead of through depth images removes the line-of-sight errors that projection-based 3DGS baselines suffer."],"supporting_citations":[{"why":"Supplies the base 3D Gaussian representation and tile-based camera rasterizer that SplatAD adapts.","marker":"[14]"},{"why":"Defines the lidar evaluation metrics, the rolling-shutter-aware ray-tracing baseline, and the dataset split protocol used for comparison.","marker":"[35]"},{"why":"Established joint camera-lidar neural simulation that SplatAD aims to match in quality while making faster.","marker":"[45]"},{"why":"Provides image-space rolling-shutter compensation for Gaussians that SplatAD extends to dynamic actors.","marker":"[31]"},{"why":"Replaces the original densification with MCMC, which the paper reports improves far-field rendering and stabilizes training.","marker":"[16]"},{"why":"Supplies the EW A alpha-blending formulation used in SplatAD's rasterization.","marker":"[47]"},{"why":"Represents the depth-image-based 3DGS autonomous-driving baselines that SplatAD outperforms in lidar rendering.","marker":"[7]"},{"why":"PandaSet is the primary dataset for the reconstruction and main comparison results.","marker":"[41]"},{"why":"Argoverse2 is the dataset used for generalization and cross-dataset evaluation.","marker":"[38]"},{"why":"nuScenes is the third dataset used to demonstrate generalizability.","marker":"[5]"}],"fun_headline_variants":["First 3D Gaussian splatting for real-time lidar and camera rendering","SplatAD: real-time lidar and camera rendering via 3D Gaussian splatting","3D Gaussian splatting now renders lidar and camera in real time","SplatAD: 3D Gaussian splatting for real-time lidar and camera","Real-time lidar and camera rendering with 3D Gaussian splatting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the ego vehicle, its sensors, and every dynamic actor keep constant linear and angular velocities during a capture window, so a 100 ms lidar sweep that includes hard braking or acceleration falls outside the model's rolling-shutter compensation.","fun_headline_variants_meta":{"raw":{"variants":["First 3D Gaussian splatting for real-time lidar and camera rendering","SplatAD: real-time lidar and camera rendering via 3D Gaussian splatting","3D Gaussian splatting now renders lidar and camera in real time","SplatAD: 3D Gaussian splatting for real-time lidar and camera","Real-time lidar and camera rendering with 3D Gaussian splatting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00082,"raw_usage":{"total_tokens":3629,"prompt_tokens":1022,"completion_tokens":2607,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":638,"completion_tokens_details":{"reasoning_tokens":2498}},"tokens_in":638,"tokens_out":2607,"duration_ms":16697,"temperature":1.0,"reasoning_tokens":2498,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:59:57.759950+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train SplatAD on a driving log that contains a hard deceleration or emergency lane change inside a lidar sweep, then compare the rendered median depth and ray-drop accuracy on held-out frames against the real lidar points on those braking frames; if the depth error jumps well beyond the reported 0.01–0.02 m and ray-drop accuracy drops below the NeRF baseline specifically in the accelerating portion of the sweep, the constant-velocity rolling-shutter assumption is the failing part.","supporting_citations":[{"cited_title":"3d gaussian splatting for real-time radiance field rendering","cited_arxiv_id":null,"evidence_quote":"Supplies the base 3D Gaussian representation and tile-based camera rasterizer that SplatAD adapts."},{"cited_title":"Neurad: Neural rendering for autonomous driving","cited_arxiv_id":null,"evidence_quote":"Defines the lidar evaluation metrics, the rolling-shutter-aware ray-tracing baseline, and the dataset split protocol used for comparison."},{"cited_title":"Unisim: A neural closed-loop sensor simulator","cited_arxiv_id":null,"evidence_quote":"Established joint camera-lidar neural simulation that SplatAD aims to match in quality while making faster."},{"cited_title":"Gaussian splatting on the move: Blur and rolling shutter compensation for natural camera mo- tion","cited_arxiv_id":null,"evidence_quote":"Provides image-space rolling-shutter compensation for Gaussians that SplatAD extends to dynamic actors."},{"cited_title":"3d gaussian splat- ting as markov chain monte carlo","cited_arxiv_id":null,"evidence_quote":"Replaces the original densification with MCMC, which the paper reports improves far-field rendering and stabilizes training."},{"cited_title":"Mip-splatting: Alias-free 3d gaussian splat- ting","cited_arxiv_id":null,"evidence_quote":"Supplies the EW A alpha-blending formulation used in SplatAD's rasterization."},{"cited_title":"Pandaset: Ad- vanced sensor suite dataset for autonomous driving","cited_arxiv_id":null,"evidence_quote":"PandaSet is the primary dataset for the reconstruction and main comparison results."},{"cited_title":"Argoverse 2: Next generation datasets for self-driving perception and fore- casting","cited_arxiv_id":null,"evidence_quote":"Argoverse2 is the dataset used for generalization and cross-dataset evaluation."},{"cited_title":"nuscenes: A mul- timodal dataset for autonomous driving","cited_arxiv_id":null,"evidence_quote":"nuScenes is the third dataset used to demonstrate generalizability."}],"review_version":1}