{"id":"d9ab716b-525e-4794-bb3d-16196b2c7055","arxiv_id":"2412.00592","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A pipeline that removes and inserts vehicles in real LiDAR scans, using spherical voxelization and generative inpainting, to create controllable synthetic data with a small domain gap.","lead":"LiDAR-EDIT edits real-world LiDAR scans by removing objects from the scene and inserting new ones at user-specified positions, while keeping the original background. The goal is to generate realistic synthetic driving data with controllable object layouts for training and testing self-driving perception systems.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Background inpainting is only evaluated on artificial object-free masks; real de-occlusion masks at inference may be structurally different, leaving the realism of every edited scene unverified.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing concern: the inpainting model's training and evaluation masks differ from the de-occlusion masks used at inference, and no experiment directly tests this transfer. This concern is central because background inpainting is a required step for every edited scene; if the inpainting does not produce realistic backgrounds under real occlusion masks, the method's central claim of 'realistic LiDAR scenes' collapses regardless of the insertion quality. The paper's Table I is a surrogate that does not exercise the real mask distribution, and Table II only checks that objects are not present, not that the inpainted content is realistic. I agree with the reader's assessment that the paper is promising but conditionally accepted; the evidence gap is substantial but not a demonstrated internal inconsistency. My proposed concrete test would close this gap by measuring inpainting performance across the realistic range of de-occlusion mask shapes and sizes, providing a direct check on the weakest link in the pipeline.","tokens_in":9597,"tokens_out":5160,"duration_ms":56627,"concrete_test":"Evaluate the trained inpainting model on a sweep of de-occlusion masks generated from real ground-truth vehicle bounding boxes across the nuScenes validation set, varying distance (5–40 m), box dimensions per the dataset distribution, and orientation, while still placing masks in object-free sectors to retain ground truth. Compare FSVD, FPVD, JSD, and MMD against the current Table I protocol; if metrics degrade significantly with mask area or shape, or if qualitative artifacts appear for masks resembling real objects, the generalization assumption fails and the realism claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of realism rests on the background inpainting after object removal (Sec IV-C2). The model is trained on masks applied to 'random object-free voxels' and evaluated in Sec V-A using a single nominal bounding box of average size at 10 m placed in an object-free sector. At inference, the model must inpaint de-occlusion masks produced by real objects at arbitrary distances, sizes, and orientations (Sec IV-C1), which are larger, more complex, and include radial occlusion patterns through the object region. The paper provides no evaluation of inpainting quality on these real masks; the object-removal experiment (Tab II) only measures residual detected objects, not the fidelity of the inpainted background. If the model does not generalize to real de-occlusion masks, the background in every edited scene is unrealistic, which would invalidate the primary realism claim and weaken the downstream value argument. The paper does not report the distribution of training masks (sizes, shapes, contiguity), so the mismatch could be substantial.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LiDAR-EDIT, a pipeline that edits real LiDAR scans by removing objects, inpainting the occluded background with a MaskGIT-style generative model, completing partial object scans into a library, and re-inserting objects at user-specified poses using spherical voxelization to enforce LiDAR projective geometry. The method is evaluated on nuScenes in three ways: background inpainting quality on synthetic masks, object removal effectiveness via a pretrained detector, and object-insertion realism via domain-gap metrics and a downstream detection pretraining experiment. The paper claims small domain gap and practical value for autonomous-driving tasks.","tokens_in":9804,"tokens_out":3966,"duration_ms":40543,"significance":"The paradigm itself is a useful and reasonably novel contribution: editing real scans rather than generating from scratch preserves background realism and offers controllable counterfactual object layouts. The spherical-voxelization formulation is clean and gives occlusion handling and LiDAR-consistent resampling by construction, which is a genuine strength. The paper also provides open-sourced code and uses standard perceptual/statistical metrics. However, the evidence supporting the central realism claim is incomplete: the inpainting model is trained and evaluated on masks that differ structurally from the de-occlusion masks used at inference, and the downstream experiment confounds extra training epochs with synthetic-data value. With additional targeted evaluation the contribution could be solid, but as it stands the claims are not fully supported.","major_comments":[{"comment":"The background inpainting evaluation uses only artificially created masks: a nominal bounding box of average size at 10 m placed in object-free sectors (Sec V-A). Meanwhile, training uses masks on \"random object-free voxels\" (Sec IV-C2) and inference uses de-occlusion masks generated from real objects (Sec IV-C1), which are larger, more contiguous, and have radial occlusion patterns behind the object. The paper provides no evaluation of inpainting fidelity on these real de-occlusion masks; the object-removal experiment in Tab. II only counts residual detected objects and does not measure whether the inpainted background is realistic. Because the realism of every edited scene depends on background inpainting after real object removal, this train/test mask mismatch is load-bearing and leaves the primary realism claim unverified.","section":"V-A"},{"comment":"The downstream detection comparison confounds the effect of synthetic pretraining with the effect of additional training epochs: the baseline model is trained on real nuScenes for 40 epochs, while the pretrained model is trained on synthetic data for 15 epochs and then on real data for 40 more epochs, for a total of 55 epochs. Without a control that trains on real data for 55 epochs (or otherwise matches total compute), the observed mAP improvement from 0.361 to 0.368 cannot be attributed to the synthetic data. This is a central claim of the paper, so the experiment needs to be re-run with matched training budgets.","section":"V-B.2"},{"comment":"All quantitative results are reported as single numbers without error bars, multiple seeds, or statistical significance tests. The key claims of a small domain gap (mAP 0.361 vs. 0.340) and downstream improvement (mAP 0.361 vs. 0.368) involve small differences that could easily arise from training stochasticity. The paper should report mean and variance over at least three seeds, especially for the detector-based evaluations in Tabs. II and III.","section":"Tables I-III"}],"minor_comments":[{"comment":"Typo: \"background poins\" should be \"background points\".","section":"V-A"},{"comment":"The inpainting baseline (copy-tile of a neighboring object-free sector) is very weak; while the task has no established prior method, a stronger baseline (e.g., a range-image diffusion model or UltraLidar-style generation restricted to the mask) would make the quantitative improvements more convincing.","section":"V-A"},{"comment":"The paper does not report the distribution of training masks (sizes, shapes, contiguity, distances) used for background inpainting, which makes it difficult to assess how different the inference-time de-occlusion masks are. A figure or table summarizing training mask statistics would help.","section":"IV-C2"},{"comment":"The object library relies on AnchorFormer, but the paper does not state what dataset this completion network was pretrained on or how its domain gap to nuScenes objects is handled. This should be clarified.","section":"IV-D"},{"comment":"The higher car AP on synthetic data (0.785 vs. 0.672) is acknowledged as possibly due to simpler, ray-drop-free inserted objects, but the paper should quantify this (e.g., by comparing insertion point density or detection confidence distributions) rather than leaving it as a speculation.","section":"V-B.1"},{"comment":"The problem formulation assumes every object in the target set appears somewhere in the training data; the paper should discuss the coverage of the object library and possible failure modes for rare or unseen categories.","section":"III"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of a computer-vision venue and the central idea is promising. The main risk is not novelty but evaluation sufficiency: the inpainting evaluation avoids the realistic mask setting, and the downstream experiment is confounded by training budget. These are fixable with additional experiments rather than fundamental flaws. I would support a major revision with the expectation that the authors add real-mask inpainting evaluation, matched-epoch downstream controls, and variance reporting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"LiDAR-EDIT is a genuinely different approach to synthetic LiDAR: instead of rendering assets or generating point clouds from scratch, it edits real scans by removing objects, inpainting the background, and inserting completed object shapes at user-specified poses. The spherical voxelization for ray-consistent resampling and occlusion is a clean idea, and the authors deserve credit for building a complete pipeline around it. The paper is clearly written, and the code is claimed to be open-sourced, which is a plus.\n\nWhat is new: the task itself—layout editing of real LiDAR scans—and the use of spherical voxelization to enforce projective geometry by construction are not in the prior literature. The decomposition into de-occlusion masking, generative background inpainting, and object completion/insertion is sensible and makes controllability explicit. The experiments show the approach is plausible: inpainting beats the copy-tile baseline on all metrics, downstream detection after pretraining improves slightly, and the qualitative results look natural.\n\nThe soft spots are mostly in the evidence, not the idea. The stress-test concern about inpainting evaluation is fair. The model is trained on random object-free voxel masks and evaluated on a single artificial mask—an average-sized box at 10 meters placed in object-free sectors. At inference, the de-occlusion masks come from real objects at arbitrary distances, sizes, and orientations, which are structurally different. The paper never evaluates inpainting quality on those real masks; the object-removal experiment only counts residual detections, which is not a fidelity measure. If the inpainting doesn't generalize, the background in every edited scene is wrong, so this is a real gap in the central claim. Also, all quantitative results are single-run with no error bars, the baselines are weak (copy-tile; no comparison against other generative data methods), and the paper itself notes AnchorFormer is less reliable on other categories, so only cars are demonstrated even though 'type' is claimed as a controllable attribute.\n\nNone of this is fatal. The mechanism is coherent, and the spherical voxelization argument is sound. The paper would benefit from an inpainting evaluation on real de-occlusion masks, multiple seeds, and at least one alternative generative baseline. I'd send it to peer review, with those requests. It's a useful contribution for the autonomous-driving data generation community, and it will be citable for the spherical voxelization idea alone.","headline":"A genuinely novel LiDAR editing paradigm with a clean spherical-voxelization core, but the inpainting evidence skips the real de-occlusion masks that matter most.","tokens_in":10312,"tokens_out":2852,"would_cite":true,"duration_ms":26683,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Editing real LiDAR scans produces controllable synthetic data that improves detector training.","keywords":["LiDAR scene editing","synthetic LiDAR generation","background inpainting","spherical voxelization","object insertion","autonomous driving","point cloud completion","discrete latent transformer"],"falsifier":"Apply the full removal-and-inpainting pipeline to scans where the area behind a real vehicle is later observed from a nearby second LiDAR pass, then compare the inpainted points with the actually observed points: a large discrepancy in geometry or density would falsify the realism claim for object removal.","tokens_in":9376,"feed_emoji":"🚗","tokens_out":4978,"duration_ms":47550,"temperature":0.7,"pith_summary":"LiDAR-EDIT proposes to generate synthetic LiDAR data not by rendering from scratch but by editing real scans: it removes vehicles, fills the occluded background with a generative inpainting model, and inserts point-cloud objects from a library at user-chosen poses. The paper argues this preserves real-world background realism, gives full control over object count, type, and pose, and produces object labels for free, avoiding the domain gap of simulators and the limited controllability of end-to-end generation. The load-bearing mechanism is spherical voxelization, which discretizes space by range, azimuth, and elevation so occlusion and scan density follow real LiDAR projective geometry by construction. Experiments on a standard driving dataset show that detectors pretrained on edited scenes and then fine-tuned on real data outperform detectors trained on real data alone, with a small measured domain gap.","feed_headline":"Editing real LiDAR scans improves detector training","feed_subtitle":"Removing and reinserting objects with generative inpainting keeps backgrounds realistic and lifts detector average precision.","key_machinery":"Spherical voxelization, which discretizes space around the sensor into $(r,\\theta,\\phi)$ bins so that each LiDAR ray is a column of voxels, is the core object; it makes occlusion and beam-pattern consistency explicit, so removal masks and insertion resampling are computed by simple ray-column operations. Around it sit a vector-quantized variational autoencoder and a masked bidirectional transformer trained on azimuth-radius bird's-eye-view latent maps for background inpainting, an object library built with a pretrained point-cloud completion network, and an insertion routine that aligns objects to the ground and removes occluded background voxels.","core_discovery":"The paper's central claim is that realistic, usable LiDAR point clouds can be manufactured by surgical editing of real scans rather than by full-scene synthesis. Object removal is treated as de-occlusion masking plus generative background inpainting; object insertion is completed-shape placement with spherical-voxel resampling and occlusion resolution. The result is a synthetic scan with the original background, a newly specified object layout, and labels inherited from the object library. The authors support the claim with detector-based evaluation: the same detector's mean average precision is similar on real and edited validation sets, and pretraining on edited data improves downstream detection over real-only training.","pith_inferences":["If the realism claim is right, a direct stress test would be to evaluate background inpainting on real object de-occlusion masks instead of the average-size synthetic masks used in the paper; this is the point where the training and inference distributions differ most.","The same editing pipeline could be pointed at safety-critical counterfactual queries, such as moving a pedestrian into the ego lane, once the object library and completion models support non-car categories reliably.","Modeling secondary returns and ray dropping, which the paper lists as future work, should close the remaining gap between inserted cars and real cars and likely raise the car-category average precision further.","Because object labels are known by construction, edited data could serve as free annotated training and testing data for detection, tracking, and motion forecasting without manual labeling."],"forward_implications":["Users can specify how many objects, of which types, and at which poses appear in a scan while keeping the original real environment.","Generated scans come with object labels, which end-to-end LiDAR generators generally do not provide.","Pretraining an object detector on edited data before fine-tuning on real data improves mean average precision over training on real data alone.","The method can create counterfactual scenes with object layouts that differ greatly from the recorded scene, enabling controlled comparisons.","Because only modified regions are synthesized, the approach scales to existing large datasets without physics engines or full-scene reconstruction."],"supporting_citations":[{"why":"Supplies the two-stage masked-token generation architecture that the background inpainting model adapts to LiDAR scene editing.","marker":"[19]"},{"why":"Defines the masked generative transformer training procedure used for iterative prediction of latent background tokens.","marker":"[20]"},{"why":"Provides the discrete latent representation that encodes point clouds for the generative inpainting pipeline.","marker":"[26]"},{"why":"Completes partial vehicle scans into full object shapes that populate the insertion library.","marker":"[27]"},{"why":"Is the real-world driving dataset used to train and evaluate the removal, insertion, and detection experiments.","marker":"[28]"},{"why":"Is the object detector used to measure the domain gap and to test the downstream value of pretraining on edited data.","marker":"[32]"}],"fun_headline_variants":["LiDAR-EDIT: Edit real scans, control object layouts","Surgical LiDAR editing: realism-preserving synthetic data","Boost detection with edited LiDAR scenes, not full synthesis","Inpaint and insert objects in real LiDAR scans","Object-level control in LiDAR generation via real-scene editing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The inpainting network is trained on masks over random object-free voxels and is assumed to generalize to the larger, structurally different de-occlusion masks left by actual objects; if it does not, every edited scene inherits unrealistic background artifacts.","fun_headline_variants_meta":{"raw":{"variants":["LiDAR-EDIT: Edit real scans, control object layouts","Surgical LiDAR editing: realism-preserving synthetic data","Boost detection with edited LiDAR scenes, not full synthesis","Inpaint and insert objects in real LiDAR scans","Object-level control in LiDAR generation via real-scene editing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000907,"raw_usage":{"total_tokens":3850,"prompt_tokens":848,"completion_tokens":3002,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":464,"completion_tokens_details":{"reasoning_tokens":2919}},"tokens_in":464,"tokens_out":3002,"duration_ms":78382,"temperature":1.0,"reasoning_tokens":2919,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:12:16.499058+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Apply the full removal-and-inpainting pipeline to scans where the area behind a real vehicle is later observed from a nearby second LiDAR pass, then compare the inpainted points with the actually observed points: a large discrepancy in geometry or density would falsify the realism claim for object removal.","supporting_citations":[{"cited_title":"Learning compact representations for lidar completion and generation,","cited_arxiv_id":null,"evidence_quote":"Supplies the two-stage masked-token generation architecture that the background inpainting model adapts to LiDAR scene editing."},{"cited_title":"Maskgit: Masked generative image transformer,","cited_arxiv_id":null,"evidence_quote":"Defines the masked generative transformer training procedure used for iterative prediction of latent background tokens."},{"cited_title":"Neural discrete representation learning,","cited_arxiv_id":null,"evidence_quote":"Provides the discrete latent representation that encodes point clouds for the generative inpainting pipeline."},{"cited_title":"Anchorformer: Point cloud completion from discriminative nodes,","cited_arxiv_id":null,"evidence_quote":"Completes partial vehicle scans into full object shapes that populate the insertion library."},{"cited_title":"nuscenes: A multimodal dataset for autonomous driving,","cited_arxiv_id":null,"evidence_quote":"Is the real-world driving dataset used to train and evaluate the removal, insertion, and detection experiments."},{"cited_title":"V oxelnext: Fully sparse voxelnet for 3d object detection and tracking,","cited_arxiv_id":null,"evidence_quote":"Is the object detector used to measure the domain gap and to test the downstream value of pretraining on edited data."}],"review_version":1}