{"id":"28ac277a-3772-4290-8bf0-844db314349e","arxiv_id":"2501.07530","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"IP-FaceDiff edits facial attributes in videos with text prompts by injecting identity-preserving self-attention features from one fine-tuned diffusion branch into a separate localized-editing diffusion branch.","lead":"This paper presents a system that edits faces in videos from a text prompt, such as adding a beard or changing hair color, while trying to keep the person's identity unchanged. It claims an 80% reduction in editing time compared with prior face-video editing methods.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1 reports identical R@1 and MRR values across three distinct face-recognition backbones, which is implausible and undermines the central quantitative claim of superior identity preservation.","rationale":"The stress-test pass confirms the reader's rejection verdict. The most load-bearing concern is not the theoretical feature-compatibility assumption, but the concrete data integrity problem in Table 1. Identical retrieval metrics across three distinct face-recognition backbones are effectively impossible under the described protocol, so the table cannot support the claim that the method 'consistently outperforms existing techniques.' The reader's rationale already mentions this inconsistency, but the weakest_assumption field points to the feature-drift concern. Both are real, but the data inconsistency is more fundamental because it casts doubt on the entire empirical basis of the paper. A single concrete test—recomputing the metrics per backbone—can settle whether the table is a faithful report. If it is not, the central quantitative claim is unsupported; if it is (which would require a remarkable coincidence or a flawed protocol), the feature-drift concern would then become the primary risk to validate. The paper does have independent merit in its engineering setup (using pretrained T2I models, keyframe propagation, and attention substitution), but the evidence presented is insufficient for acceptance. No code or data is released, and the evaluation lacks statistical rigor. Thus the verdict remains REJECT, unchanged.","tokens_in":11750,"tokens_out":2486,"duration_ms":25376,"concrete_test":"Recompute Table 1 by running the retrieval protocol separately for each face-recognition backbone (VGGFace, CosFace, FaceNet) on the same evaluation videos and the CelebV-HQ retrieval database. For each method, compute R@1, MRR, and cosine similarity independently per backbone. If the R@1 or MRR values differ across backbones for any method, the printed table is confirmed as inaccurate; if they remain identical, provide the exact retrieval rankings for each backbone to establish that the equality is genuine rather than a copy-paste artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim of 'consistently outperforming' existing techniques rests heavily on Table 1, which reports identity preservation metrics (R@1, MRR, Cosine distance) using three face-recognition models: VGGFace, CosFace, and FaceNet. For every method, the R@1 and MRR values are numerically identical across all three backbones (e.g., DVA: 0.76/0.794 for all three; Ours with ID guidance: 0.96/0.97 for all three). This is statistically implausible: each backbone produces different embeddings, so the retrieval ranking over the CelebV-HQ database (35,000 videos) should yield different ranks and hence different R@1 and MRR values. The near-certainty of divergence means the table as printed cannot be a faithful report of three independent evaluations. This is load-bearing because it directly invalidates the quantitative evidence for identity preservation, which is the paper's main claimed advantage. The reader's chosen weakest assumption—compatibility of self-attention features between independently fine-tuned branches—is a legitimate theoretical concern, but it is secondary: even if features are incompatible, the empirical table was the intended demonstration that the mechanism works, and that demonstration is not credible. Other issues (small non-random evaluation set, missing TokenFlow baseline, no error bars) compound the problem, but the identical metrics are the single most decisive red flag.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes IP-FaceDiff, a facial video editing framework built on two fine-tuned Stable Diffusion 2.1 branches: one fine-tuned with a reconstruction plus ArcFace identity loss (ε1), and one fine-tuned with a directional CLIP loss for localized edits (ε2). During inference, the method performs DDIM inversion with ε1, extracts self-attention features, and injects them into ε2's self-attention layers while editing keyframes and propagating via the TokenFlow correspondence mechanism. The authors claim consistent identity preservation, faster inference (roughly 80% reduction), temporal consistency, and multi-attribute editing, and report quantitative comparisons against DVA, STIT, and LTFE.","tokens_in":11999,"tokens_out":5569,"duration_ms":49764,"significance":"If the empirical claims were reliable, the contribution would be valuable: leveraging pretrained T2I diffusion models for facial video editing with identity preservation and lower inference cost is a practically important goal. The proposed feature-injection idea is well motivated by PnP-style editing, and the paper includes a useful ablation of the two fine-tuning losses. However, the quantitative evidence—especially the identity-preservation table—is not credible as reported, and the evaluation is too thin (25 videos, no error bars, missing a directly relevant baseline) to support the abstract's claims of consistent superiority.","major_comments":[{"comment":"The R@1 and MRR values for DVA and Ours (with ID guidance) are numerically identical across all three face-recognition backbones (VGGFace, CosFace, FaceNet): DVA reports 0.76/0.794 in all three columns and Ours reports 0.96/0.97 in all three columns. Because each backbone produces different embeddings, exact agreement across backbones is extremely unlikely on a 35,000-video retrieval database, which raises a serious question about whether three independent evaluations were actually performed. Since Table 1 is the main quantitative support for the paper's central identity-preservation claim, this is a load-bearing issue that must be resolved.","section":"Section 5.3, Table 1"},{"comment":"The evaluation uses only 25 edited videos per method (from 50 original videos), with no error bars, confidence intervals, or significance tests. The reported improvements, such as the FaceNet cosine distance of 0.221 for Ours versus 0.256 for DVA, could be within noise at this sample size; the paper should provide variance estimates and a statistical comparison. Additionally, Section 5.5 and Table 3 report results for '20 ten-second videos', while Section 5.2 says the evaluation dataset is 25 edited videos; this inconsistency needs to be corrected.","section":"Section 5.2, Tables 1–3"},{"comment":"The identity-preservation mechanism is trained with an ArcFace cosine-distance loss (Eq. 5) and evaluated with cosine distances and retrieval metrics computed from face-recognition embeddings (Table 1). Although the evaluation uses CosFace, VGGFace, and FaceNet rather than ArcFace itself, all are face-recognition embedding spaces of the same family, so the reported identity scores are partially a fit to the training objective. The authors should discuss this circularity and include a human perceptual identity-judgment study or a metric not derived from face-recognition embeddings.","section":"Section 4.2, Eq. (5), Table 1"},{"comment":"The central novelty is substituting ε1's self-attention features into ε2 during editing, but the manuscript provides no analysis of whether the independently fine-tuned branches' feature distributions remain aligned. If the distributions drift, the swap could either corrupt the edit or fail to transfer identity. At minimum, the paper should show a comparative visualization or a quantitative test (e.g., feature-distance histograms) demonstrating that the injected features are compatible at the selected layers.","section":"Section 4.1, Fig. 2"},{"comment":"The method is built directly on TokenFlow (Geyer et al. [14]), which is the relevant state-of-the-art for consistent video editing with T2I models, yet TokenFlow is neither included as a quantitative baseline nor evaluated on the facial-video benchmark. The paper states in Section 4.2 that TokenFlow 'suffers from this issue' but provides no numbers; adding this baseline is necessary to support the claim of outperforming existing techniques.","section":"Section 5.2, 5.9"}],"minor_comments":[{"comment":"Several typos and missing spaces appear, such as 'DV A' in Table 1 entries, 'modelϵ1' in Section 4.1, and 'networkϵ2' in Section 4.1; these should be corrected for readability.","section":"Throughout"},{"comment":"The text refers to 'pre-trained SDE models', but Stable Diffusion is a latent diffusion model, not an SDE in the sense used here; the terminology should be consistent ('pre-trained T2I models').","section":"Section 5.8"},{"comment":"The MOS is reported on a 1–10 scale, but the caption does not explicitly state the scale, and no per-video variance or inter-rater agreement is provided; please clarify and add these details.","section":"Section 5.5, Table 3"},{"comment":"The hyperparameters λ1=0.3 and λ2=0.7 are listed, but no sensitivity analysis or justification for these values is provided; the free parameters of the loss and training (30,000 iterations, batch size 16) are not ablated.","section":"Section 5.1"}],"recommendation":"reject","confidential_remarks":"The identical retrieval numbers across backbones in Table 1 are a serious data-integrity red flag. If the authors cannot provide the per-backbone raw rankings and code that reproduces the three independent evaluations, the paper should not be considered. Even setting that aside, the evaluation lacks the statistical power and baseline coverage needed for the claims. I recommend rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is an engineering combination of known pieces: TokenFlow keyframe propagation, PNP-style self-attention feature injection, an ArcFace-fine-tuned branch for identity, and a directional-CLIP-fine-tuned branch for localized edits. That combination, applied to facial video editing, is new as far as I can tell, and the speed gain from keyframe-only editing is real and practically useful. The qualitative figures show genuinely localized edits on in-the-wild videos, and the method makes sense as a recipe.\n\nThe problem is the numbers. Table 1 reports identical R@1 and MRR values across VGGFace, CosFace, and FaceNet for every method. That cannot happen with three different embedding spaces over 25 queries and a 35k-video database; the ranks would diverge somewhere. The identical values across backbones mean the table is not a faithful report. That is load-bearing because identity preservation is the paper's main claim. The reader's feature-compatibility concern is legitimate but secondary—even if the mechanism is sound, the evidence meant to demonstrate it isn't credible.\n\nThe rest of the evaluation is thin: 25 edited videos, no error bars, no significance tests, no code or data, and no TokenFlow baseline in the quantitative comparison despite the method being built on it. The ablation is qualitative only. The MOS study is under-described—50 participants but no confidence intervals. None of this would be fatal on its own, but combined with Table 1 it leaves the central claim unsupported.\n\nI would tell the authors: re-run the identity evaluation, show per-backbone numbers with variance, add TokenFlow or explain its exclusion, and release code. The identity-loss/evaluation circularity is real but mild—ArcFace-trained branches evaluated with face-recognition metrics are common in this area, and the retrieval metrics are still meaningful.\n\nVerdict: reject as-is, but not a waste of time. The pipeline is worth a second look if the numbers are fixed. If this crosses my desk again with clean tables and code, I would send it out.","headline":"A plausible engineering pipeline for facial video editing with a real speed gain, but the identity-preservation table is internally implausible and the central claim lacks credible support.","tokens_in":12543,"tokens_out":2257,"would_cite":false,"duration_ms":22063,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that facial video editing can preserve a person's identity and follow text prompts by injecting attention features from an identity-fine-tuned diffusion branch into an edit-fine-tuned branch, while cutting editing time…","keywords":["facial video editing","diffusion models","identity preservation","text-guided editing","self-attention feature injection","keyframe editing","directional CLIP loss","ArcFace"],"falsifier":"Take a held-out set of face videos, run the method, and at each substituted layer compute a distribution-shift metric (such as cosine distance or a feature-space distance) between the self-attention features produced by ε1 and ε2 for the same inverted latents. If the shift is large at layers that are nonetheless swapped, or if ablating the ArcFace fine-tuning of ε1 does not measurably reduce identity preservation, the central mechanism would be falsified.","tokens_in":11531,"feed_emoji":"🎬","tokens_out":7972,"duration_ms":66655,"temperature":0.7,"pith_summary":"The paper proposes a facial video editing framework that claims to preserve the subject's identity while following arbitrary text prompts, and to cut editing time by roughly 80%. It does so by fine-tuning two copies of a pre-trained text-to-image diffusion model: one copy learns to reconstruct faces faithfully using an ArcFace identity loss, the other learns localized, prompt-aligned edits using a directional CLIP loss. At edit time, self-attention features from the identity branch are injected into the editing branch, and edits are applied to keyframes and propagated to the rest of the video. The paper reports that this approach beats three established baselines on identity-retrieval metrics, temporal consistency, and human opinion scores, and works on in-the-wild videos.","feed_headline":"Face-video edits preserve identity and run 80% faster","feed_subtitle":"Two fine-tuned diffusion branches swap attention features so edits follow text prompts without changing who the person is.","key_machinery":"The load-bearing mechanism is cross-branch self-attention feature substitution during editing. Two U-Nets share the same pre-trained Stable Diffusion initialization but are fine-tuned independently: ε1 with an identity loss based on ArcFace, ε2 with a directional CLIP loss. For each frame, DDIM inversion produces noisy latents; ε1 generates self-attention features at every layer, and during editing these features replace the self-attention features of ε2 at corresponding layers. The edit direction therefore comes from the CLIP-fine-tuned branch while identity and head pose come from the ArcFace-fine-tuned branch. The same feature space also supports joint keyframe editing: only a subset of frames is edited, and the edits are propagated to the other frames via nearest-neighbor matching in the latent feature space, which provides temporal consistency and the reported speedup.","core_discovery":"The central claim is that identity preservation and text-driven edit quality can be decoupled into two independently fine-tuned diffusion branches and recombined at inference without retraining per video. The identity branch ε1 is fine-tuned on face images to minimize both pixel reconstruction error and ArcFace cosine distance to the ground-truth identity; the editing branch ε2 is fine-tuned with a directional CLIP loss to make localized changes that follow the prompt. During editing, the self-attention features that ε1 computes from DDIM-inverted frames are substituted into the corresponding layers of ε2, so the editing branch receives the prompt direction while the identity branch supplies global facial structure. Joint keyframe editing propagates the edit across frames, yielding temporal consistency and an ~80% reduction in inference time compared with frame-wise baselines. The paper claims this consistently outperforms prior facial video editors across identity-retrieval metrics (CosFace, VGGFace, FaceNet), temporal loss, and mean opinion scores.","pith_inferences":["A natural extension would be to measure per-layer distribution shift between the two branches' self-attention features on held-out data; layers with large drift could be excluded or reweighted rather than blindly swapped.","The same decoupling may transfer to newer backbones such as SDXL or video diffusion models, provided both branches share architecture and the compatibility assumption holds.","The reported 80% speedup depends on how many keyframes are sampled; a systematic sweep of keyframe density against identity and temporal quality would reveal the practical operating range."],"forward_implications":["Facial video editing could become a single-pass, text-driven operation on one GPU, with no per-video optimization.","Identity preservation would transfer to unseen faces and in-the-wild footage because the identity branch is fixed after fine-tuning.","Multiple edits (for example, adding glasses, bangs, and a beard) could be applied in one forward pass without extra computational cost.","The two-branch attention-injection design may extend to other subject-critical video edits, such as preserving an object's or a body's consistency."],"supporting_citations":[{"why":"Supplies the joint keyframe editing and nearest-neighbor feature propagation that give temporal consistency and the ~80% speedup.","marker":"[14]"},{"why":"Supplies the ArcFace face-recognition network whose cosine-distance loss fine-tunes the identity branch ε1.","marker":"[9]"},{"why":"Supplies the directional CLIP loss used to fine-tune the editing branch ε2 for localized, prompt-aligned edits.","marker":"[13]"},{"why":"The prior work showing self-attention feature replacement can control generation, which the identity-injection scheme extends.","marker":"[45]"},{"why":"Provides the pre-trained text-to-image diffusion model that both branches start from, giving the rich latent space and generalization.","marker":"[35]"},{"why":"Provides the DDIM inversion procedure used to obtain noisy latents and per-layer self-attention features for identity extraction.","marker":"[43]"},{"why":"Supplies the face-image dataset (CelebA-HQ, cited via TediGAN) used for fine-tuning both branches.","marker":"[50]"},{"why":"Baseline diffusion-based facial video editor that the paper compares against on identity, temporal consistency, and speed.","marker":"[26]"},{"why":"Baseline GAN-based facial video editor that the paper compares against on identity, temporal consistency, and speed.","marker":"[46]"},{"why":"Baseline StyleGAN-based facial video editor that the paper compares against on identity, temporal consistency, and speed.","marker":"[52]"}],"fun_headline_variants":["80% faster face video editing with identity intact","Two diffusion branches keep identity, drive text edits","Face-video edits: identity locked, time cut 80%","Text-driven video editing that never changes who you are"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The identity-preservation mechanism assumes that self-attention features computed by the ArcFace-fine-tuned branch stay compatible with the editing branch at every layer where they are injected, even though the two branches were fine-tuned independently; if their feature distributions drift apart, the swap could corrupt the edit or fail to hold identity.","fun_headline_variants_meta":{"raw":{"variants":["80% faster face video editing with identity intact","Two diffusion branches keep identity, drive text edits","Face-video edits: identity locked, time cut 80%","Text-driven video editing that never changes who you are"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000648,"raw_usage":{"total_tokens":2970,"prompt_tokens":933,"completion_tokens":2037,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":549,"completion_tokens_details":{"reasoning_tokens":1973}},"tokens_in":549,"tokens_out":2037,"duration_ms":14822,"temperature":1.0,"reasoning_tokens":1973,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:38:54.664813+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of face videos, run the method, and at each substituted layer compute a distribution-shift metric (such as cosine distance or a feature-space distance) between the self-attention features produced by ε1 and ε2 for the same inverted latents. If the shift is large at layers that are nonetheless swapped, or if ablating the ArcFace fine-tuning of ε1 does not measurably reduce identity preservation, the central mechanism would be falsified.","supporting_citations":[{"cited_title":"Tokenflow: Consistent diffusion features for consistent video editing","cited_arxiv_id":null,"evidence_quote":"Supplies the joint keyframe editing and nearest-neighbor feature propagation that give temporal consistency and the ~80% speedup."},{"cited_title":"Arcface: Additive angular mar- gin loss for deep face recognition.IEEE Transactions on Pat- tern Analysis and Machine Intelligence , 44(10):5962–5979, Oct","cited_arxiv_id":null,"evidence_quote":"Supplies the ArcFace face-recognition network whose cosine-distance loss fine-tunes the identity branch ε1."},{"cited_title":"Bermano, Gal Chechik, and Daniel Cohen-Or","cited_arxiv_id":null,"evidence_quote":"Supplies the directional CLIP loss used to fine-tune the editing branch ε2 for localized, prompt-aligned edits."},{"cited_title":"Plug-and-play diffusion features for text-driven image-to-image translation","cited_arxiv_id":null,"evidence_quote":"The prior work showing self-attention feature replacement can control generation, which the identity-injection scheme extends."},{"cited_title":"Sdxl: Improving latent diffusion models for high-resolution image synthesis","cited_arxiv_id":null,"evidence_quote":"Provides the pre-trained text-to-image diffusion model that both branches start from, giving the rich latent space and generalization."},{"cited_title":"Denois- ing diffusion implicit models","cited_arxiv_id":null,"evidence_quote":"Provides the DDIM inversion procedure used to obtain noisy latents and per-layer self-attention features for identity extraction."},{"cited_title":"Tedigan: Text-guided diverse face image generation and ma- nipulation","cited_arxiv_id":null,"evidence_quote":"Supplies the face-image dataset (CelebA-HQ, cited via TediGAN) used for fine-tuning both branches."},{"cited_title":"Diffusion video autoencoders: Toward temporally consistent face video editing via disen- tangled video encoding","cited_arxiv_id":null,"evidence_quote":"Baseline diffusion-based facial video editor that the paper compares against on identity, temporal consistency, and speed."},{"cited_title":"Stitch it in time: Gan-based facial editing of real videos","cited_arxiv_id":null,"evidence_quote":"Baseline GAN-based facial video editor that the paper compares against on identity, temporal consistency, and speed."},{"cited_title":"A latent transformer for disentangled face editing in im- ages and videos","cited_arxiv_id":null,"evidence_quote":"Baseline StyleGAN-based facial video editor that the paper compares against on identity, temporal consistency, and speed."}],"review_version":1}