{"id":"33c599e0-b352-4ef1-9ecd-a0aaf3412f1d","arxiv_id":"2412.12877","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"PRIMEdit edits multiple video objects independently using per-object masks and captions, and contributes a benchmark dataset and a leakage metric.","lead":"This paper presents a method to edit several objects in a video separately, each according to its own text description. It also introduces a new 200-video benchmark dataset and a metric that detects when an edit leaks into the wrong object.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Hyperparameters tuned on the same 200 MIVE videos used for the headline comparison may inflate PRIMEdit's advantage over default-configured baselines; a held-out tuning split is required to support 'significantly outperforms'.","rationale":"The reader's weakest assumption identifies exactly the load-bearing concern: the evaluation is inbred because the authors constructed the dataset, defined the CIA metric, and tuned the method's core hyperparameters on the same 200 videos used in the headline comparison. This is a real threat to the central claim of significant outperformance, because the comparison pits a heavily tuned method against baselines with default settings. A held-out tuning split is the direct fix and would settle whether the gap is genuine. The paper has independent strengths—the user study, extensive ablations, and efficiency analysis—but those do not remove the tuning asymmetry. I agree with the reader's conditional verdict: the claim is likely defensible but needs released artifacts, a held-out split, and statistical reporting before unconditional acceptance. I recommend no change to the CONDITIONAL verdict, so 'UNCHANGED' is appropriate.","tokens_in":31625,"tokens_out":3757,"duration_ms":34452,"concrete_test":"Hold out 50 of the 200 MIVE videos, retune the four main hyperparameters (λ, λ_r, IPR step fraction, SNS fraction) on the remaining 150, and recompute Table 2 on the held-out 50. If PRIMEdit's margins over VideoGrain and GA V shrink to within noise or reverse, the headline claim is not supported. Optionally, also recompute CIA using a soft score (mean diagonal LTF minus mean off-diagonal per row) to check whether the argmax formulation drives the gap.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that PRIMEdit significantly outperforms SOTA in faithfulness, accuracy, and leakage prevention—rests on Table 2, which compares against baselines run with their default settings. However, the paper's own ablations (Supp. D.2, D.3) show that λ, λ_r, IPR step fraction, SNS fraction, and re-inversion steps were all selected on the full 200-video MIVE set used for the headline table. The same set is also used for the reported CIA and user study numbers. This creates a test-set-tuning asymmetry: PRIMEdit is effectively optimized on the evaluation set, while baselines are not, so the observed margins (e.g., CIA 0.6705 vs 0.5868) may overstate the method's real advantage. An additional confound is the CIA metric itself, an author-defined argmax-over-rows score; if two target captions are similar in CLIP space, even a faithful edit can receive a zero, and this may differentially penalize baselines that produce more ambiguous outputs. The user study is a mitigating signal, but it evaluates the same tuned model on the same dataset. This does not invalidate the method, but it means the quantitative claim is not yet established for general videos.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PRIMEdit, a zero-shot framework for multi-instance video editing that combines two modules: Instance-centric Probability Redistribution (IPR), which manipulates cross-attention probabilities to localize edits within instance masks, and Disentangled Multi-instance Sampling (DMS), which separates per-instance denoising to reduce attention leakage. The authors also introduce the MIVE dataset (200 videos with per-instance masks and captions) and the Cross-Instance Accuracy (CIA) score. Quantitative comparisons against seven baselines, extensive ablations, and a user study are reported, with the claim that PRIMEdit significantly outperforms recent state-of-the-art methods in faithfulness, accuracy, and leakage prevention.","tokens_in":31922,"tokens_out":6197,"duration_ms":54922,"significance":"The task of multi-instance video editing is timely and the proposed IPR and DMS designs are conceptually simple and potentially useful. If the empirical claims are robust, PRIMEdit would be a strong zero-shot method and the MIVE dataset and CIA score could provide a useful evaluation resource. The ablation study is thorough, covering the IPR and DMS components and several hyperparameters. However, the evaluation as presented does not yet establish the headline claim of significant superiority: the method is tuned on the same videos used in the headline comparison, results lack uncertainty quantification, the CIA metric needs further scrutiny, and the code and dataset are not released.","major_comments":[{"comment":"The headline comparison in Table 2 is performed on the same 200-video MIVE set that was used to select all key hyperparameters: the IPR step fraction (10%), λ and λ_r (both 0.5), the SNS step fraction (40%), and the re-inversion steps l (2) are chosen by ablations on the full MIVE set (Supp. D.2–D.3). Baselines are run with their default settings, so PRIMEdit is effectively optimized on the test set while the baselines are not. This asymmetry can inflate the observed margins (e.g., CIA 0.6705 vs 0.5868) and does not support the abstract's claim of \"significantly outperforms\" for general videos. I request a held-out validation split or a separate test set for hyperparameter selection, and a report of the robustness of the results across different splits.","section":"Sec. 5 Implementation details; Supp. D.2–D.3, Tables 9–10"},{"comment":"The formula for λ_S is degenerate: min(mean(A_{I,j=S}), min(A_{I,j=S})) simplifies to min(A_{I,j=S}), and the warm-up value W is introduced without a precise definition of its role in the expression. As written, the equation is not a well-defined dynamic schedule, and the IPR method cannot be reproduced from the text. Please provide a corrected formulation, a clear definition of W, and pseudo-code for the IPR update.","section":"Eq. (2), Sec. 3.2"},{"comment":"The CIA score assigns a binary 1 to the maximum similarity in each row of the n×n similarity matrix. If two instance target captions are semantically similar in CLIP space, a perfectly faithful edit may not receive the largest similarity for its own crop, producing a 0 and penalizing correct behavior. Thus, the metric conflates attention leakage with caption ambiguity, and it may differentially favor methods that produce overly discriminative or artifact-laden crops. Please provide an analysis of how CIA behaves under caption similarity, or consider a soft version based on normalized ranks or thresholds.","section":"Sec. 4.2, Eq. (10)"},{"comment":"No error bars, multiple random seeds, or statistical significance tests are reported. The improvements over the strongest baselines are small in several local metrics (e.g., LTF 0.2048 vs 0.2026 for VideoGrain), and the user-study temporal-consistency win rate of PRIMEdit (33.65%) is close to FLATTEN's (32.60%). Without uncertainty quantification, the claim of \"significantly outperforms\" is not statistically established. Please report the mean and standard deviation over at least three random seeds per method and add paired significance tests (e.g., bootstrap or Wilcoxon) for the quantitative and user-study comparisons.","section":"Table 2, Table 5, Sec. 5.1"},{"comment":"The paper introduces a new dataset (MIVE) and a new metric (CIA), but neither the dataset nor the code or evaluation scripts are released. The project page (https://kaist-viclab.github.io/primedit-site/) shows demos but not the dataset or implementation. Without releasing these resources, the central empirical claims cannot be reproduced or independently verified, and the proposed benchmark cannot serve the community as intended. Please make the dataset, code, and metric implementation publicly available.","section":"Sec. 1, Data availability"}],"minor_comments":[{"comment":"There are several typos and grammatical issues, including \"addtition\" (Sec. 3.2), \"ommitted\" (Supp. B.1), and \"we ourperform\" (Supp. C.3). The manuscript would benefit from a careful proofread.","section":"Throughout"},{"comment":"The user study's temporal-consistency result for PRIMEdit is close to that of FLATTEN; please add a statistical significance test or temper the wording related to temporal consistency.","section":"Sec. 5.1, User study"},{"comment":"In the full quantitative results, PRIMEdit ranks fourth in Global Temporal Consistency and Global Textual Faithfulness; the main text should acknowledge this more explicitly rather than focusing only on local scores.","section":"Supp. Table 5"},{"comment":"The limitations section only discusses reflection consistency; it should also mention the evaluation limitations raised here (test-set tuning, lack of error bars) and how they are addressed.","section":"Supp. G, Limitations"},{"comment":"The caption-generation pipeline in Fig. 8 suggests a level of automation that contradicts the described \"Manual Fix and Annotation\" step; please clarify the human effort involved in producing the MIVE captions and masks.","section":"Sec. B.1, Supp."}],"recommendation":"major_revision","confidential_remarks":"The paper is technically interesting but the evaluation protocol has significant test-set-tuning and reproducibility issues. The MIVE dataset is derived from VIPSeg with LLM-generated captions and manual refinement, so its novelty as a benchmark is moderate. I would support a major revision if the authors can address the statistical validity of the comparisons, release the dataset and code, and correct the specification of Eq. (2)."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know about PRIMEdit is that it's a genuine systems contribution: zero-shot multi-instance video editing from per-object masks and captions, with a new 200-video benchmark (MIVE) and a leakage metric (CIA) that fills a real gap. The IPR idea -- redistributing cross-attention probability from the S token to text/end tokens only inside masks -- is simple and the ablations suggest it works. The DMS sampling with series-then-parallel noise and re-inversion is a reasonable way to keep instances disentangled. The user study gives independent signal, and the paper is honest about its global metric disadvantage and about the reflection limitation in Sec. G. The soft spots are real. The biggest one is the inbred evaluation: the authors built the dataset, defined the metric, and tuned every important hyperparameter (lambda, lambda_r, IPR step fraction, SNS fraction, re-inversion steps) on the same 200 videos used for the headline comparison, while baselines ran with default settings. The CIA gap (0.67 vs 0.59) could easily shrink on a held-out split with tuned baselines. No code or dataset is released. No error bars or multiple seeds, so small differences like LTC 0.9552 vs 0.9551 are meaningless. Eq. (2) has an apparent typo (min(mean(...), min(...)) -- likely should be max), which needs fixing. The stress-test concern is on the mark. The user study mitigates the leakage concern somewhat, but it evaluates the same tuned model on the same dataset. I would not accept the 'significantly outperforms' claim as-is. Who is this for? Researchers in zero-shot video editing or benchmark design. It deserves a serious referee: the method is plausible, the benchmark is a contribution, and the evaluation issues are fixable. A revision with released code and data, a held-out tuning split, and statistical reporting would make this solid. My recommendation: send it to peer review, but with a clear request for those fixes.","headline":"PRIMEdit is a solid zero-shot multi-instance video editing method with a useful benchmark, but the evaluation is inbred enough that 'significantly outperforms' is not yet established.","tokens_in":653,"tokens_out":745,"would_cite":false,"duration_ms":25496,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PRIMEdit claims a training-free way to edit several objects in one video at once, keeping each edit inside its target mask.","keywords":["multi-instance video editing","zero-shot video editing","diffusion models","cross-attention modulation","attention leakage","instance masks","video editing benchmark","CLIP-based evaluation"],"falsifier":"Run PRIMEdit with the paper's fixed hyperparameters on a held-out set of videos that were not used for any tuning, compute the Cross-Instance Accuracy and per-instance textual faithfulness, and compare against the strongest mask-based multi-instance baseline; if PRIMEdit does not lead on leakage prevention or faithfulness on that held-out set, the central claim fails.","tokens_in":31428,"feed_emoji":"🎬","tokens_out":5215,"duration_ms":49667,"temperature":0.7,"pith_summary":"The paper tries to establish that a zero-shot, training-free framework can edit multiple distinct objects in a single video simultaneously, with each edit staying inside its own mask and not bleeding into other objects. It argues that existing global or single-object editors fail exactly when several instances need different prompts at once, either because a global caption cannot specify who gets edited or because bounding-box conditions leak across overlapping regions. PRIMEdit's answer is to reshape cross-attention probabilities inside each mask and to sample each instance separately before fusing, which the paper reports as more faithful and leakage-free than recent methods on a new 200-video benchmark. A reader should care because this is the capability behind practical text-driven video fix-ups: change one person's outfit, another's prop, and the background's color in a single pass without retraining.","feed_headline":"One pass edits every object in a video by its own prompt","feed_subtitle":"PRIMEdit confines each change to its mask with redrawn attention and decoupled sampling, plus a 200-video test set.","key_machinery":"The load-bearing object is the cross-attention map between image features and text tokens in a diffusion U-Net. IPR treats each row of that map as probability mass and redistributes it among the start-of-sequence, text, and end tokens, with the mask deciding which side of the boundary receives the redistribution; a dynamic strength decays over sampling steps and avoids padding-token manipulation. DMS is the second mechanism: Series Noise Sampling edits instances sequentially against inverted background latents, latent fusion combines the per-instance results, re-inversion harmonizes them, and Parallel Noise Sampling keeps the captions separate in the remaining steps. The Cross-Instance Accuracy score is the evaluation machinery: a per-row winner-take-all over vision-language similarities between cropped instances and instance captions, averaged down the diagonal.","core_discovery":"On its own terms, the paper claims that multi-instance video editing can be decomposed into per-instance probability redistribution plus disentangled sampling. Instance-centric Probability Redistribution (IPR) works only in cross-attention layers: outside the mask, attention to text and end tokens is moved to the start-of-sequence token so no edit appears; inside the mask, probability is moved from the start token to text and end tokens so the described object actually appears there. Disentangled Multi-instance Sampling (DMS) then edits each instance independently with its own caption while the background is pinned to the inverted input, fuses the instance latents, re-inverts briefly, and finishes in a parallel mode that keeps captions separate. With those two mechanisms, the paper claims edits are faithful, temporally consistent, and free of attention leakage; it also contributes the MIVE dataset of 200 annotated videos and the Cross-Instance Accuracy (CIA) score, which asks whether each cropped instance matches its own caption better than any other caption.","pith_inferences":["If the benchmark's hyperparameters were tuned on the same 200 videos used in the headline comparison, the size of the advantage over untuned baselines may shrink on arbitrary user videos; an out-of-benchmark test with fixed settings would settle that.","The winner-take-all binarization in CIA discards how decisively one caption wins; a soft variant that rewards large margins could be more sensitive to partial leakage.","The mask-only interface suggests a natural extension to video object tracking: given masks for one frame, propagate masks and edit consistently through the clip without user re-annotation.","Reflective surfaces are a known gap; combining the approach with reflection-aware masking could extend it to scenes with mirrors and polished floors."],"forward_implications":["Users can give each object its own prompt and get edits confined to that object's mask, without writing one long global caption or retraining the model.","Overlapping or nearby objects no longer need bounding-box separation, since mask-based probability redistribution replaces box conditions.","The MIVE dataset gives the field a common testbed with many instances per object class, small-to-large masks, and instance-level captions, plus the CIA metric for leakage.","Because IPR only touches cross-attention and DMS avoids spatio-temporal attention modulation, the method runs faster and uses less memory than the mask-based baseline it is compared against.","The same framework handles a single instance and partial-instance edits, not only dense multi-object scenes."],"supporting_citations":[{"why":"The pre-trained text-to-image diffusion model whose U-Net cross-attention IPR modifies; it supplies the base generative backbone for the whole pipeline.","marker":"[46]"},{"why":"The motion module added to the base model to give temporal consistency, which PRIMEdit inherits as its video backbone.","marker":"[21]"},{"why":"The deterministic inversion and denoising steps that PRIMEdit uses to reconstruct and edit the input video latents.","marker":"[51]"},{"why":"The conditioning network that provides depth-based spatial control during sampling, used by PRIMEdit and some baselines.","marker":"[71]"},{"why":"The attention-modulation method that inspires IPR and serves as the main IPR ablation baseline.","marker":"[32]"},{"why":"The multi-instance, bounding-box-based baseline whose overlapping-box leakage PRIMEdit targets, and a key comparison method.","marker":"[28]"},{"why":"The mask-based multi-instance video editing baseline that PRIMEdit compares against and outperforms in efficiency.","marker":"[65]"},{"why":"The instance-level image generation method that inspires the multi-instance sampling design and supplies the local textual faithfulness metric.","marker":"[55]"},{"why":"The video panoptic segmentation source from which the MIVE dataset's 200 videos are selected and annotated.","marker":"[39]"}],"fun_headline_variants":["One video, many prompts: PRIMEdit edits each object without leaks","Zero-shot multi-instance video edit: redraw attention, decouple sampling","Stop editing leakage: PRIMEdit's IPR and DMS keep each object faithful","Multi-object video editing made precise: PRIMEdit localizes and isolates","New metric and dataset for multi-instance editing leaks: PRIMEdit leads"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's reported edge over other editors presumes that the MIVE benchmark is a fair, representative test and that tuning the framework's hyperparameters on those same videos does not inflate its scores.","fun_headline_variants_meta":{"raw":{"variants":["One video, many prompts: PRIMEdit edits each object without leaks","Zero-shot multi-instance video edit: redraw attention, decouple sampling","Stop editing leakage: PRIMEdit's IPR and DMS keep each object faithful","Multi-object video editing made precise: PRIMEdit localizes and isolates","New metric and dataset for multi-instance editing leaks: PRIMEdit leads"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000227,"raw_usage":{"total_tokens":1501,"prompt_tokens":1002,"completion_tokens":499,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":618,"completion_tokens_details":{"reasoning_tokens":398}},"tokens_in":618,"tokens_out":499,"duration_ms":5063,"temperature":1.0,"reasoning_tokens":398,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:39:09.783362+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run PRIMEdit with the paper's fixed hyperparameters on a held-out set of videos that were not used for any tuning, compute the Cross-Instance Accuracy and per-instance textual faithfulness, and compare against the strongest mask-based multi-instance baseline; if PRIMEdit does not lead on leakage prevention or faithfulness on that held-out set, the central claim fails.","supporting_citations":[{"cited_title":"High-resolution image syn- thesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"The pre-trained text-to-image diffusion model whose U-Net cross-attention IPR modifies; it supplies the base generative backbone for the whole pipeline."},{"cited_title":"Animatediff: Animate your personalized text-to- image diffusion models without specific tuning","cited_arxiv_id":null,"evidence_quote":"The motion module added to the base model to give temporal consistency, which PRIMEdit inherits as its video backbone."},{"cited_title":"Denois- ing diffusion implicit models","cited_arxiv_id":null,"evidence_quote":"The deterministic inversion and denoising steps that PRIMEdit uses to reconstruct and edit the input video latents."},{"cited_title":"Adding conditional control to text-to-image diffusion models","cited_arxiv_id":null,"evidence_quote":"The conditioning network that provides depth-based spatial control during sampling, used by PRIMEdit and some baselines."},{"cited_title":"Dense text-to-image generation with attention modulation","cited_arxiv_id":null,"evidence_quote":"The attention-modulation method that inspires IPR and serves as the main IPR ablation baseline."},{"cited_title":"Ground-a-video: Zero- shot grounded video editing using text-to-image diffusion models","cited_arxiv_id":null,"evidence_quote":"The multi-instance, bounding-box-based baseline whose overlapping-box leakage PRIMEdit targets, and a key comparison method."},{"cited_title":"Videograin: Modulating space-time attention for multi- grained video editing","cited_arxiv_id":null,"evidence_quote":"The mask-based multi-instance video editing baseline that PRIMEdit compares against and outperforms in efficiency."},{"cited_title":"Large-scale video panoptic segmen- tation in the wild: A benchmark","cited_arxiv_id":null,"evidence_quote":"The video panoptic segmentation source from which the MIVE dataset's 200 videos are selected and annotated."}],"review_version":1}