{"id":"75d24745-3617-4e1f-8846-192202160bc3","arxiv_id":"2412.19458","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"DriveEditor uses depth-aware 3D bounding box projection and single-reference appearance cues to reposition, replace, remove, and insert objects in driving videos with a single diffusion framework.","lead":"DriveEditor is a diffusion-based system that moves, replaces, deletes, or inserts objects in driving videos, guided by a 3D bounding box and a single reference image. It matters because it lets autonomous driving teams generate diverse training data for perception models from existing recordings, including rare scenarios and new object placements.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 3D-IFM ablation in Table 3 is confounded with 40K extra training steps, so the paper's central claim that SV3D priors improve editing is not yet established.","rationale":"The reader's weakest assumption identifies the SVD/SV3D latent-space compatibility as the load-bearing risk. I agree this is the right region of the argument, but the more precise and actionable issue is that the only supporting experiment is confounded by unequal training budgets. The paper's ablation in Table 3 does not hold total iterations constant, so the incremental improvement cannot be attributed to the 3D-IFM or the frozen SV3D features. This does not require rejecting the paper; it requires an additional control experiment. The reader's other noted issue, the translation-error discrepancy between prose and Table 2, appears to be a wording ambiguity rather than a contradiction: the prose numbers (0.26, 0.24, 0.32) are the differences between each task's mATE and the oracle mATE (0.68-0.42=0.26, 0.66-0.42=0.24, 0.74-0.42=0.32), so the underlying table is internally consistent. The downstream augmentation result (NDS 0.569 to 0.581) and the qualitative results provide some independent support for the framework's overall utility, but they do not isolate the 3D-prior contribution. A controlled ablation with matched training steps or a noise-replacement 3D-IFM would settle the concern definitively. Hence CONDITIONAL remains the appropriate verdict, unchanged by this stress-test.","tokens_in":15509,"tokens_out":10408,"duration_ms":98145,"concrete_test":"Run the second-stage training for the same 40K iterations with the 3D-IFM input replaced by shape-matched Gaussian noise (or by features from a frozen, unrelated encoder), keeping all other settings identical. Compare FID, FVD, PSNR, LPIPS, mRecall, mATE, and mAOE on the reconstruction validation set against the reported '+3D-IFM' row of Table 3. If the random-feature model matches the reported numbers, the benefit comes from extra training, not from SV3D priors; if it is clearly worse, the 3D prior is empirically validated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's key novelty is the 3D Prior Incorporation via the 3D Information Fusion Module (3D-IFM), and its only quantitative support is the ablation in Table 3. However, the training protocol in 'Method' states: 'In the first stage... train the video model and Position Controller without 3D information fusion... In the second stage, we reintroduce the 3D-IFM and the frozen SV3D model...' with 60K iterations in stage 1 and 40K in stage 2. Therefore the '+3D-IFM' row of Table 3 has 40K more gradient updates than the 'PC+DP' row. The observed FVD drop from 22.11 to 18.82, and mAOE drop from 0.061 to 0.043, could be due to continued fine-tuning rather than to the SV3D features themselves. No control is reported: the paper does not train a 'PC+DP' model for 100K total steps, nor a 3D-IFM variant fed with random or mismatched features. Consequently, the reader's concern about SVD/SV3D latent-space compatibility remains unresolved: the one empirical test that could support it is not cleanly attributable to the 3D prior. This weakness is load-bearing because the title, the method design, and contribution 3 all hinge on 3D information guiding appearance; if the 3D prior is inert or harmful, the framework's novelty is substantially reduced even though position control might still work.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"DriveEditor proposes a unified diffusion-based framework for object-level editing in driving videos, supporting repositioning, replacement, deletion, and insertion using a shared network. The position control module projects 3D bounding boxes while preserving depth information and injects these features into the video model's ResBlocks. The appearance maintenance module combines cut-and-paste, CLIP cross-attention, and SV3D-derived 3D priors via a 3D Information Fusion Module (3D-IFM). The method is trained on a reconstruction task using nuScenes data and evaluated on editing quality, position controllability, and downstream object-detection improvement. The paper reports favorable FID/FVD scores against text-based and inpainting baselines, position metrics close to the oracle, and an NDS improvement from 0.569 to 0.581 for StreamPETR trained with generated augmentations.","tokens_in":15767,"tokens_out":4417,"duration_ms":36670,"significance":"If the results are valid, DriveEditor provides a practical and novel framework for controllable object editing in autonomous driving data, with a useful combination of 3D box control and appearance preservation from a single reference image. The downstream task improvement is a strong positive result, as it demonstrates real utility beyond visual quality metrics. The position control method using depth-aware projected box faces is technically interesting and appears well motivated. However, the paper's headline contribution—the 3D prior from SV3D—is supported almost exclusively by an ablation that is confounded with additional training iterations, which weakens the current evidence for the central claim.","major_comments":[{"comment":"The paper claims that SV3D and SVD share the same architecture and latent space, enabling 'seamless' integration of intermediate features. This is a load-bearing assumption, since the 3D-IFM is designed to fuse features across these two models. The only supporting evidence is the confounded ablation in Table 3; no direct measurement of feature-space alignment is provided. The fixed azimuth set in the appendix limits viewpoint coverage, and it is not demonstrated that the selected SV3D features actually correspond to the video frame's viewpoint in a way that improves the video model's representation. Please provide either a quantitative compatibility test (e.g., feature distance between SVD and SV3D encodings for the same object) or an ablation with mismatched 3D features to show that the specific SV3D priors, rather than any injected signal, are responsible for the improvement.","section":"Section 3D Prior Incorporation, Eq. (4)"},{"comment":"The prose reports translation errors of 0.26 m, 0.24 m, and 0.32 m for repositioning, insertion, and replacement, respectively, with an average of 0.27 m. However, Table 2 reports mATE values of 0.68, 0.66, and 0.74 for these tasks. These two sets of numbers are inconsistent by roughly a factor of 2.5. Please correct the prose or clarify what quantity the 0.26/0.24/0.32 values represent, since mATE is a standard nuScenes metric and the table values are much larger than the prose claims.","section":"Main Results, Position Control Ability paragraph"}],"minor_comments":[{"comment":"The abbreviation 'T2V' is used for both Tune-A-Video and Text2Video-Zero in the baseline description, which is confusing. In Table 1 the names appear as 'T2V' and 'TA V', but in the text both are introduced as T2V. Please use distinct abbreviations throughout (e.g., TAV for Tune-A-Video and T2V-Zero for Text2Video-Zero).","section":"Experimental Setups and Table 1"},{"comment":"The appendix contains a garbled line of '/uni000000...' characters immediately before Figure 8, which appears to be a LaTeX compilation artifact. This should be cleaned up before publication.","section":"Appendix, Figure 8 caption area"},{"comment":"The paper states that the fixed azimuth set achieves an average angular difference of only 1.08 degrees, but this statistic is only for the training distribution. Given the zero-shot generalization experiments on Waymo, it would be helpful to also report the angular difference for validation or unseen data, since the fixed set may not cover the viewpoint range of all driving videos.","section":"Section 3D Prior Incorporation and Appendix 'Azimuth Angles'"},{"comment":"The conclusion mentions iterative editing for long videos, but this is not described in the method or evaluated quantitatively. The qualitative results in the appendix are suggestive, but a brief description of how iterative editing is performed and its limitations would improve completeness.","section":"Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The core methodological issue is the confounded ablation for the 3D-IFM, which is the paper's main novelty. The authors must retrain or report a control without 3D-IFM for the full 100K iterations and ideally add a feature-compatibility test. The metric inconsistency in the main results is also a credibility problem. With these fixed, the paper could be a strong contribution to the driving video editing community. The scope is appropriate for the venue, and the downstream task improvement is a valuable result."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: DriveEditor is a real attempt at a hard problem—unified object-level editing in driving videos—and the architecture is plausible. The genuinely new ideas are the depth-preserving projection of 3D box faces for position control and the three-level appearance module that injects SV3D features. But the paper's key ablation for the 3D prior is confounded with extra training time, so the central claim is weaker than it looks.\n\nWhat the paper does well: the depth-aware projection that treats each 3D box face separately, assigns depth as pixel intensity, and injects into spatial and temporal ResBlocks via adapters is a clean way to condition on 3D layout. The three-level appearance control (cut-and-paste, CLIP cross-attention, SV3D priors) is a sensible design, and the downstream augmentation experiment is a nice practical validation—NDS on StreamPETR goes from 0.569 to 0.581, modest but real.\n\nThe soft spots: the main issue is the 3D-IFM ablation in Table 3. Training is split into 60K iterations without 3D-IFM and 40K with it, so the row that adds 3D-IFM also gets 40K more gradient updates than the PC+DP row. There is no control—no 100K run without 3D-IFM, no random-feature 3D-IFM—so the observed FVD drop from 22.11 to 18.82 could come from continued fine-tuning rather than from SV3D features. This matters because the title and contribution 3 hinge on 3D information guiding appearance. It is not fatal; the rest of the pipeline stands on its own, but the evidence for the flagship module is not clean.\n\nAlso, there is an internal inconsistency: the prose reports translation errors of 0.26, 0.24, and 0.32 meters, but Table 2 lists mATE values of 0.68, 0.66, and 0.74—roughly a 2.6x gap. The numbers need to be reconciled. And there are no error bars or significance tests anywhere, which makes it hard to judge the gaps between ablation rows.\n\nOverall, this is a solid engineering paper with a genuine problem and a workable solution. The 3D-prior contribution is plausible but under-evidenced. A serious referee should ask for a controlled ablation and a correction of the numeric inconsistency. Worth sending to peer review; with a careful revision it could be a solid publication.","headline":"DriveEditor is a serious engineering effort for object-level editing in driving videos; its 3D-prior module is plausible but the key ablation is confounded, so treat the headline claims with care.","tokens_in":16378,"tokens_out":3070,"would_cite":true,"duration_ms":25218,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DriveEditor claims a single diffusion network can reposition, replace, delete, or insert objects in driving videos.","keywords":["driving video editing","diffusion models","3D bounding box control","object repositioning","object replacement","object deletion","object insertion","autonomous driving data augmentation"],"falsifier":"Run the video-editing pipeline with the 3D-prior fusion switched off and on while measuring reconstruction quality on a held-out set; if the gap reported in the paper's ablation (FVD 18.82 with fusion versus 22.11 without) does not reproduce, the central role of the 3D prior is not established. A sharper test is to measure how similar the two models' internal features actually are at matched layers; if the similarity is no higher than a random projection baseline, the claimed seamless integration is not supported.","tokens_in":15249,"feed_emoji":"🚗","tokens_out":10254,"duration_ms":77739,"temperature":0.7,"pith_summary":"DriveEditor is a diffusion-based framework for editing objects in driving videos. Its central claim is that four operations—repositioning, replacement, deletion, and insertion—can be handled by one network whose only task-specific inputs are a masked video, a target 3D bounding box, and a single reference image. Position is controlled by projecting the 3D box onto the image plane while encoding depth, and appearance is enforced at three levels: pasted low-level detail, CLIP semantic conditioning, and 3D priors from a frozen novel-view-synthesis model. The paper reports that edited videos stay close to ground truth in 3D detection metrics, and that using such edits as training augmentation improves a downstream detector's NDS from 0.569 to 0.581. A sympathetic reader would care because this points to a cheap route to diversify autonomous-driving data without new collection or manual annotation.","feed_headline":"One network moves, swaps, deletes, or inserts objects in driving video","feed_subtitle":"A depth-aware 3D box sets position; one reference image sets appearance; edited data boosts a 3D detector's NDS.","key_machinery":"The depth-aware projection of the 3D bounding box is the core position mechanism: each face of the box is interpolated, projected to the image plane with its depth value as pixel intensity, and assembled into a six-channel pose image that carries position and orientation without face occlusion. A Position Controller encodes this image and injects features through adapters into every spatial and temporal ResBlock of the video U-Net. On the appearance side, the 3D Information Fusion Module (3D-IFM) is the load-bearing new component: it takes intermediate features from a frozen SV3D U-Net at matched azimuth views, transforms them to the object's scale and position in the video frame, applies zero-convolution layers so the pretrained video model is not corrupted at initialization, and adds the result to the corresponding block outputs. The paper argues this is what lets a single reference image provide multi-view appearance information.","core_discovery":"The paper's central discovery is that object position and object appearance can be separately controlled within a single latent video diffusion model, so that the same trained weights perform all four editing tasks. The position side encodes a depth-preserving projection of the 3D bounding box into a six-channel pose image, feeds it through a ResNet-style controller, and injects the resulting multi-scale features into both spatial and temporal ResBlocks of the video U-Net. The appearance side conditions the model on one reference image through cut-and-paste of the object, CLIP cross-attention for semantic content, and a 3D Information Fusion Module that aligns and adds intermediate features from a frozen SV3D U-Net to the video model's blocks. Trained once on a reconstruction task (mask an object, recover it from the box and reference image), the network switches operations by changing only its inputs. The quantitative case is that 3D-detection metrics on edited videos remain close to the unedited oracle (mRecall 0.93–0.94 vs 0.99; mAOE 0.043–0.044 vs 0.037) and that augmenting StreamPETR training data with generated edits raises NDS from 0.569 to 0.581.","pith_inferences":["A testable extension: because the position and appearance modules are decoupled, the same network should edit any annotated 3D object category, not just the five tested, by swapping only the object bank and reference images.","The fixed azimuth schedule exploits the small viewpoint changes of driving videos (average 5.67 degrees); in scenes with sharp turns or fast ego motion, the angular matching error would grow, and the 3D prior could begin to distort rather than preserve appearance. Measuring FVD as a function of viewpoint-change magnitude would test this directly.","The downstream gain suggests a closed-loop data-augmentation strategy the paper does not explore: generate edits aimed at a detector's failure cases, retrain, and repeat. The reported NDS improvement is a one-shot result, not a loop.","Because reference images come from segmentation, editing quality is upper-bounded by segmentation quality; the paper's own limitation notes blur in low-contrast scenes, which implies segmentation improvements would directly improve repositioning."],"forward_implications":["Four editing tasks collapse into one trained model: repositioning, replacement, deletion, and insertion differ only in the mask, bounding box, and reference image supplied at inference.","Position control is tight enough for a pretrained 3D detector to treat edited objects almost like real ones: mRecall stays within 0.05 of the oracle and orientation error within about 0.007 radians.","Generated edits transfer to downstream perception: adding repositioned and replaced objects to half the nuScenes training set improves StreamPETR's NDS from 0.569 to 0.581.","Deletion and replacement quality surpass text- and inpainting-based baselines on temporal consistency, with FVD dropping from 457 to 228 (deletion) and from 151 to 39 (replacement).","The fixed 21-view azimuth schedule holds average angular error to 1.08 degrees, and iterative conditioning extends 10-frame editing to 39-frame videos."],"supporting_citations":[{"why":"Stable Video Diffusion, the pretrained video U-Net that DriveEditor adapts and controls.","marker":"Blattmann et al. 2023"},{"why":"SV3D, the frozen novel-view-synthesis model whose intermediate features supply the 3D appearance prior.","marker":"Voleti et al. 2024"},{"why":"nuScenes, the dataset used to build training clips, validation edits, and the downstream detection benchmark.","marker":"Caesar et al. 2020"},{"why":"Segment Anything, used to segment objects and build the reference images and object bank.","marker":"Kirillov et al. 2023"},{"why":"CLIP image encoder, whose features carry the high-level semantic conditioning injected by cross-attention.","marker":"Radford et al. 2021"},{"why":"StreamPETR, the 3D detector used for position-controllability metrics and for the downstream augmentation experiment.","marker":"Wang et al. 2023a"},{"why":"Stable Diffusion inpainting, the baseline DriveEditor compares against on the deletion task.","marker":"Rombach et al. 2022"},{"why":"ProPainter, the video-inpainting baseline for deletion.","marker":"Zhou et al. 2023"},{"why":"Tune-A-Video, the one-shot text-based video-editing baseline for replacement.","marker":"Wu et al. 2023"},{"why":"Text2Video-Zero, the zero-shot text-based baseline for replacement.","marker":"Khachatryan et al. 2023"}],"fun_headline_variants":["Depth-aware 3D box plus one image edits driving objects","One diffusion model repositions, swaps, deletes, and inserts objects","Box sets position, image sets look: one model does all edits","Unified diffusion framework edits driving objects with 3D control","One reference image and a 3D box control all driving object edits"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework assumes the internal representations of the frozen 3D-view generator and the video generator are compatible enough that adding one model's mid-level features to the other improves the object's look instead of distorting it; the only support is an indirect quality ablation, not a direct check of that compatibility.","fun_headline_variants_meta":{"raw":{"variants":["Depth-aware 3D box plus one image edits driving objects","One diffusion model repositions, swaps, deletes, and inserts objects","Box sets position, image sets look: one model does all edits","Unified diffusion framework edits driving objects with 3D control","One reference image and a 3D box control all driving object edits"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000588,"raw_usage":{"total_tokens":2819,"prompt_tokens":1065,"completion_tokens":1754,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":681,"completion_tokens_details":{"reasoning_tokens":1663}},"tokens_in":681,"tokens_out":1754,"duration_ms":11766,"temperature":1.0,"reasoning_tokens":1663,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T00:35:05.732109+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the video-editing pipeline with the 3D-prior fusion switched off and on while measuring reconstruction quality on a held-out set; if the gap reported in the paper's ablation (FVD 18.82 with fusion versus 22.11 without) does not reproduce, the central role of the 3D prior is not established. A sharper test is to measure how similar the two models' internal features actually are at matched layers; if the similarity is no higher than a random projection baseline, the claimed seamless integration is not supported.","supporting_citations":[{"cited_title":"H.; Vora, S.; Liong, V","cited_arxiv_id":null,"evidence_quote":"nuScenes, the dataset used to build training clips, validation edits, and the downstream detection benchmark."},{"cited_title":"W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; Krueger, G.; and Sutskever, I","cited_arxiv_id":null,"evidence_quote":"CLIP image encoder, whose features carry the high-level semantic conditioning injected by cross-attention."},{"cited_title":"C.; and Loy, C","cited_arxiv_id":null,"evidence_quote":"ProPainter, the video-inpainting baseline for deletion."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Text2Video-Zero, the zero-shot text-based baseline for replacement."}],"review_version":1}