{"id":"9c997c03-6860-45be-98af-9e5a224af301","arxiv_id":"2412.06578","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"MoViE distills a diffusion-based video editor into a single-step mobile model, achieving 12 fps on a Snapdragon 8 Gen 3 phone with modest quality loss.","lead":"MoViE is a phone-sized video editing model that edits about 12 frames per second on a Snapdragon 8 Gen 3 device. It reaches that speed by shrinking the image editing network, using a tiny autoencoder, and distilling the diffusion process to a single step.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 12 FPS headline rests on single-frame denoising latency only; the 9.6 s figure equals 120×0.08 s and excludes VAE encode/decode and anchor overhead, so end-to-end editing throughput is unverified.","rationale":"The reader's weakest assumption correctly targeted the single-frame denoising note in Section 4.1 and worried about cross-frame attention/anchor encoding. My stress test agrees that the measurement protocol is the load-bearing weak point, but sharpens it: the 9.6 s number is exactly 120 × 0.08 s, so the headline speed figure is arithmetically derived from the denoising-only latency. The more direct omission is the VAE/TAESD encode-decode stage, which the paper itself counts in end-to-end FLOPs but does not include in the phone-latency column. This is not a contradiction internal to the methods — the staged optimizations and ablations are coherent — but it means the central '12 fps video editing' claim is not yet supported by the reported data. A single end-to-end timing experiment on the target phone would resolve it. Since the reader already issued CONDITIONAL and my concern reinforces rather than overturns that verdict, I recommend UNCHANGED. No ad hominem is intended; this is a measurement-scope issue that could be fixed by re-profiling and re-reporting.","tokens_in":13920,"tokens_out":4747,"duration_ms":49003,"concrete_test":"Measure full-pipeline wall-clock time on the Xiaomi-14 Pro for a 120-frame video at 512x384, starting from raw input frames and ending with decoded edited frames, using the released MoViE model (or, absent code, a reimplementation with the same TAESD and cross-frame attention): include TAESD encode, middle-frame anchor key/value computation, per-frame cross-frame-attention denoising at 1 step, and TAESD decode, with no overlap or caching of per-frame autoencoder work. Compare the total to 9.6 s and the per-frame average to 83.3 ms. As a control, run the same pipeline with the autoencoder removed to isolate UNet-only latency and verify whether 0.08 s is reproduced. If the end-to-end time exceeds 9.6 s by more than the 12 fps tolerance, the headline claim should be revised to 'denoising at 12 fps' or the autoencoder cost should be quantified and included.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that MoViE edits a 120-frame 512x384 video in 9.6 s at 12 fps (Section 1) is directly tied to the final phone latency entry in Table 1: 120 x 0.08 s = 9.6 s. But Section 4.1 states 'Reported latencies correspond to a single-frame denoising,' and Table 1 labels the final row 0.08 s under 'Latency(Phone)' for the single-step model. Thus the headline number covers only the denoising UNet forward pass. It omits (a) TAESD encoding of every source frame and decoding of every edited frame—which the paper itself counts as part of end-to-end FLOPs (the 0.76 TFLOPs per frame in Table 1 includes autoencoder cost)—and (b) any additional work for cross-frame attention with the middle-frame anchor described in Section 3.1, such as computing the anchor's keys/values and the extra attention traffic for Q[K_anchor; K_self]. If TAESD encode/decode alone takes, say, 15–25 ms on the Snapdragon NPU, the per-frame budget left for denoising is 58–68 ms instead of 83 ms, and 12 fps would drop to roughly 10 fps or below. Because the abstract and introduction promote exactly this 12 fps / 9.6 s figure, the measurement protocol leaves the paper's central quantitative claim unverified. This is a reporting/completeness issue, not a failure of the optimization stack; the staged ablations and FLOPs reductions are plausible, but they do not by themselves establish the headline throughput.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents MoViE, a mobile-oriented video editing pipeline built from InstructPix2Pix. The authors apply four optimizations: (1) a mobile-friendly UNet with attention layers removed at the highest resolutions, (2) a lightweight TAESD autoencoder, (3) multimodal guidance distillation that folds the three classifier-free-guidance forward passes into one student pass conditioned on the image and text guidance scales, and (4) adversarial step distillation that reduces sampling to a single step while conditioning on the guidance scales to preserve controllability. The claims include on-device video editing at 12 fps on a Snapdragon 8 Gen 3 mobile platform, with quality comparable to state-of-the-art video editing models on the TGVE benchmark and DAVIS videos. The experiments are organized as sequential stages, with per-stage FLOPs and latency measurements, qualitative comparisons, and an appendix human evaluation.","tokens_in":14269,"tokens_out":5354,"duration_ms":55216,"significance":"If the headline throughput claim is substantiated, this would be a noteworthy practical milestone: the first demonstration of an on-device video-to-video diffusion editing model, with a reported 120-frame edit in 9.6 seconds. The staged optimization story is transparent and useful, and the two methodological components—multimodal guidance distillation and guidance-preserving adversarial distillation—are well motivated and could transfer to other editing pipelines. The paper also provides quantitative comparisons on a standard benchmark (TGVE) and qualitative results on DAVIS, with per-stage ablations. However, the central quantitative claim currently rests on a denoising-only latency measurement, and several quality claims are supported by small, underpowered evaluations. The overall engineering contribution is credible, but the headline needs to be either re-measured or reworded.","major_comments":[{"comment":"The headline number \"12 fps\" and the associated 9.6-second video-editing time are not supported by the reported measurement protocol. Section 4.1 states \"Reported latencies correspond to a single-frame denoising,\" and Table 1's final row gives 0.08 s phone latency; 120 × 0.08 s = 9.6 s. This measurement excludes the TAESD encoder/decoder cost for every source and edited frame, which the paper itself counts as part of the end-to-end FLOPs (0.76 TFLOPs per frame in Table 1), and it also does not appear to include the cross-frame attention overhead with the middle-frame anchor described in Section 3.1. The abstract and introduction promote the 9.6 s / 12 fps figure as the central contribution. The authors should either report a measured end-to-end phone latency that includes autoencoding and anchor processing, or clearly label the headline as denoising-only throughput.","section":"§4.1, Table 1, §1"},{"comment":"The human evaluation is explicitly underpowered by the authors' own admission: \"due to the limited sample size, we cannot draw definitive conclusions.\" Yet Section 4.2 builds on this study to claim \"maintaining good quality edits\" and competitiveness with SOTA methods. Since the quality-preservation claim is central to the trade-off narrative, either the human study must be expanded (with more participants or pairs, and appropriate statistical testing), or the claims in the main text must be correspondingly weakened.","section":"Appendix A.1, §4.2"},{"comment":"The ablations for the adversarial distillation design choices report only CLIP-Image similarity on the InstructPix2Pix validation set, with no variance, confidence intervals, or significance tests. The observed differences are small (0.759 vs. 0.769 vs. 0.765 for noise distributions; 0.781 vs. 0.786 for guidance conditioning on discriminator heads) and could be within run-to-run noise. This weakens the support for the specific design choices (mean = −1, std = 1; guidance conditioning on heads). The authors should add standard deviations or multiple seeds, and ideally report directional CLIP or a second metric.","section":"Table 2, §4.3"},{"comment":"The paper's claim that adversarial distillation \"preserves controllability\" is supported only by the qualitative Figure 4, which shows varying outputs for a few guidance-scale combinations. There is no quantitative measurement of how edit strength responds to sI and sT in the single-step student, nor a comparison of the guidance-response curve before and after distillation. Since preserving controllability is one of the main stated advantages over LADD [51] and SF-V [67], this needs a more rigorous demonstration, such as a plot of a relevant similarity metric versus each guidance scale.","section":"§3.4, Fig. 4"},{"comment":"There is a numerical inconsistency in the reported frame rate: the abstract and introduction state 12 fps, Section 4.2 also says 12 frames per second, but the conclusion states 12.5 frames per second. This must be harmonized, and the exact computation underlying the chosen value should be stated explicitly.","section":"§5 vs. §1, §4.2"}],"minor_comments":[{"comment":"The row labeled \"RA VE\" should be \"RAVE,\" and the table would be more readable if every row reported the same set of columns; currently Fairy and EVE have missing latency entries, and the footnote about source of metric values is easy to miss.","section":"Table 1"},{"comment":"The sentence \"Frames are generated in 10 diffusion steps, with the middle frame used as the sole anchor\" refers to the base model, but the final MoViE pipeline uses one step; the wording should identify which model variant is being described to avoid confusion.","section":"§3.1"},{"comment":"In Step 4, the notation x0 = M(ϵ, C) implies a single call to the teacher, yet the appendix explains that the teacher is evaluated with 5 LCM sampling steps; the algorithm should either show the sampling loop or clarify that M represents the full multi-step sampling procedure.","section":"Algorithm 2"},{"comment":"The CLIP metric curves in Figure 5 are plotted as single lines without variance estimates; adding error bars or multiple runs would make the \"minimum quality drop\" claim more convincing.","section":"Fig. 5"},{"comment":"The latency is measured on 512×384 frames while the FLOPs and GPU latencies are reported for 480×480 resolution; the authors should state explicitly why the resolution difference does not affect the comparison, or report all numbers at the same resolution.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid engineering contribution with a clear staged development. The main risk is overclaiming the throughput result; asking for an end-to-end phone latency measurement is essential. I did not find a damaging circularity: distilling a student from one's own teacher is standard practice. The controllability-by-construction point is real but should be quantified, not just asserted. The significance is moderate-to-high for mobile video editing, contingent on the corrected measurement."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper delivers what it promises at the level of engineering: a video editing pipeline that runs on a phone, with plausible staged optimizations and a clear ablation path. Multimodal guidance distillation and guidance-preserving adversarial distillation are natural extensions of previous work, and the first on-device video editing model is a real milestone. I buy the qualitative claims and the FLOPs reductions; the staged numbers in Table 1 are internally consistent and the ablations are appropriately targeted.\n\nThat said, the central speed claim is not quite what it appears. The 12 fps and 9.6 s figures come straight from Table 1's final row: 120 × 0.08 s. But the paper tells you in Section 4.1 that reported latencies are single-frame denoising only. The TFLOPs per frame in the same table include the autoencoder, so the latency and FLOPs are measuring different things. The anchor-based cross-frame attention from Section 3.1 is also not counted. So the real end-to-end throughput is unverified, and the 9.6 s number in the abstract is almost certainly optimistic. This is a reporting/completeness problem, not a sign of a broken method — but it is the headline, and it needs fixing.\n\nOther soft spots are less severe. Quality metrics are point estimates without error bars or significance tests; the human evaluation is small and the authors admit it is underpowered. No code or checkpoints are released, which matters for a system paper like this. There is also a minor internal inconsistency: the abstract says 12 fps, the conclusion says 12.5 fps. None of these undermine the basic contribution.\n\nThe citation pattern is fine. The distillation targets are the authors' own teacher, which is standard practice and not circular. Self-citations are to relevant prior work.\n\nWho is this for? Anyone working on efficient diffusion or mobile generative AI will find the recipe useful. It deserves a serious referee. A revision should report a true end-to-end latency measurement on device — including VAE encode/decode and anchor processing — plus error bars and ideally release the checkpoints. If those come, this is a strong systems paper.","headline":"Solid mobile-editing engineering with a real capability, but the headline 12 fps is single-frame denoising only, not end-to-end editing.","tokens_in":14781,"tokens_out":2253,"would_cite":false,"duration_ms":24138,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that instruction-based video editing can run on a mobile phone at 12 frames per second, and shows a system that reaches this speed by pruning the denoising network, collapsing classifier-free guidance to one pass, and…","keywords":["mobile video editing","diffusion models","adversarial distillation","classifier-free guidance","on-device inference","instruction-based editing","single-step sampling","efficient video editing"],"falsifier":"Measure the wall-clock time of the complete MoViE pipeline—input frame encoding, anchor encoding, cross-frame attention, single-step denoising, and decoding—on a 120-frame, 512×384 video running on a Snapdragon 8 Gen 3 phone, and compare it with the claimed 9.6 seconds. If the full run takes longer, the central on-device 12 fps claim fails for complete videos.","tokens_in":13752,"feed_emoji":"📱","tokens_out":7182,"duration_ms":63487,"temperature":0.7,"pith_summary":"This paper attempts to establish that instruction-based video editing can run in near-real time on a mobile phone, not just on server GPUs. The authors introduce MoViE, a video-editing diffusion pipeline built on InstructPix2Pix with cross-frame attention, and claim it edits a 120-frame video at 512×384 resolution in 9.6 seconds on a Snapdragon 8 Gen 3 device, i.e., about 12 frames per second. They achieve this through three optimizations: pruning attention layers in the denoiser and swapping in a lightweight autoencoder, distilling the three-pass multi-modal classifier-free guidance into a single forward pass, and distilling the multi-step denoiser into a single adversarial step that still accepts text and image guidance scales. If the claim holds, common video-editing tasks such as style transfer and attribute changes could be done locally and privately on consumer hardware without a cloud round-trip.","feed_headline":"On-device diffusion model edits video at 12 frames per second","feed_subtitle":"A three-stage optimization cuts per-frame compute to 0.76 TFLOPs while preserving text and image control.","key_machinery":"Three components carry the result. Mobile-Pix2Pix is a pruned InstructPix2Pix U-Net with the lightweight deterministic TAESD autoencoder, which together cut the per-frame FLOPs of network and autoencoder substantially. Multimodal guidance distillation trains a student network that takes the image guidance scale and text guidance scale as sinusoidal embeddings, injected into ResNet blocks like timestep embeddings, so that one forward pass reproduces the three-way combination of conditional and unconditional predictions that classifier-free guidance normally requires. Guidance-preserving adversarial step distillation borrows the LADD recipe, using a frozen teacher that generates clean latents, a student that denoises a noisy version in one step, and a discriminator with frozen feature extractor plus trainable spatial heads; because the student and discriminator both receive the guidance scales, the distilled model keeps the user's ability to control edit strength at inference. These mechanisms reduce the number of forward evaluations per diffusion step by a factor of three and the number of denoising steps by a factor of ten.","core_discovery":"The central discovery is a compound optimization recipe that makes on-device video-to-video diffusion practical. Starting from the InstructPix2Pix image editor with a middle-frame cross-frame attention anchor, the paper replaces the heavy U-Net with Mobile-Pix2Pix, which drops self- and cross-attention at the highest resolution and uses the TAESD autoencoder; it then trains a student denoiser to reproduce the combined output of the three-pass classifier-free guidance in a single pass by conditioning on the image guidance scale and text guidance scale, and finally replaces the ten-step denoising loop with one adversarial step, carried out by a discriminator whose spatial heads are conditioned on the same guidance signals. The paper reports that a full 120-frame 512×384 edit takes 9.6 seconds on a Xiaomi 14 Pro and that editing quality, measured by CLIP-based scores and a small human preference study, stays close to the original multi-step model. It argues that this is the first on-device video-to-video diffusion model, with per-frame cost of 0.76 TFLOPs versus tens to over a hundred TFLOPs for recent server-side methods.","pith_inferences":["The reported 12 fps is a single-frame denoising latency, not a measured end-to-end video edit; anchor encoding, cross-frame attention traffic, and autoencoder time are not separately reported, so the true per-video throughput could be lower than the headline number.","The same three-stage recipe (architectural pruning, multimodal CFG distillation, guidance-conditioned adversarial distillation) likely transfers to other instruction-based editors and to text-to-image models, since none of the steps is specific to video.","The distilled model inherits the limitations of its teacher and of InstructPix2Pix: prompts or edits the base model cannot represent will not appear in the output.","A direct way to verify the headline is to time the whole 120-frame pipeline on the same phone; if wall-clock time exceeds 9.6 seconds, the fps claim is not valid for full videos."],"forward_implications":["A 120-frame, 512×384 video can be edited on a phone in 9.6 seconds instead of requiring a multi-GPU server.","Per-frame compute drops to 0.76 TFLOPs, more than an order of magnitude below recent video-editing baselines, so the model can run on low-power hardware.","Video length no longer forces a memory blow-up: frames are generated with cost linear in the number of frames, and the only anchor is the middle frame.","Text and image guidance scales remain controllable after distillation, so users can trade edit strength against fidelity at inference.","Local editing removes the need to upload video to the cloud, which changes the privacy and connectivity requirements of video-editing tools."],"supporting_citations":[{"why":"Supplies the InstructPix2Pix base model whose three-pass classifier-free guidance and editing pipeline MoViE optimizes.","marker":"[2]"},{"why":"SnapFusion contributes the mobile-friendly architectural recipe, specifically dropping attention layers at the highest resolution, which Mobile-Pix2Pix adopts.","marker":"[30]"},{"why":"LADD provides the adversarial distillation framework that MoViE extends with guidance-preserving conditioning.","marker":"[51]"},{"why":"Fairy defines the cross-frame attention baseline and the server-side latency comparison point that MoViE aims to beat.","marker":"[60]"},{"why":"TAESD is the lightweight autoencoder that replaces the InstructPix2Pix VAE and produces most of the FLOPs savings.","marker":"[1]"},{"why":"Guided-distillation literature supplies the idea of distilling classifier-free guidance into a single forward pass, which the paper extends to two modalities.","marker":"[39]"},{"why":"TokenFlow represents the inversion-based video editing family whose high cost motivates the on-device optimization.","marker":"[11]"},{"why":"InsV2V is a state-of-the-art comparison method and the source of many of the reported baseline quality scores.","marker":"[7]"},{"why":"TGVE is the benchmark dataset used for evaluating edit quality and consistency against prior methods.","marker":"[62]"}],"fun_headline_variants":["On-device diffusion edits video at 12 fps on phones","Mobile diffusion brings video editing to 12 fps on-device","First on-device diffusion video editor runs at 12 fps","Video editing at 12 fps on mobile with diffusion model","Diffusion video editing now works on phones at 12 fps"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline speed figure rests on a single-frame denoising measurement, and the paper does not report an end-to-end video-edit timing that includes anchor encoding, cross-frame attention, and autoencoder overhead; if those costs are substantial, real throughput will fall short of 12 frames per second.","fun_headline_variants_meta":{"raw":{"variants":["On-device diffusion edits video at 12 fps on phones","Mobile diffusion brings video editing to 12 fps on-device","First on-device diffusion video editor runs at 12 fps","Video editing at 12 fps on mobile with diffusion model","Diffusion video editing now works on phones at 12 fps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000177,"raw_usage":{"total_tokens":1282,"prompt_tokens":921,"completion_tokens":361,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":537,"completion_tokens_details":{"reasoning_tokens":276}},"tokens_in":537,"tokens_out":361,"duration_ms":3651,"temperature":1.0,"reasoning_tokens":276,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:30:05.574732+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the wall-clock time of the complete MoViE pipeline—input frame encoding, anchor encoding, cross-frame attention, single-step denoising, and decoding—on a 120-frame, 512×384 video running on a Snapdragon 8 Gen 3 phone, and compare it with the claimed 9.6 seconds. If the full run takes longer, the central on-device 12 fps claim fails for complete videos.","supporting_citations":[{"cited_title":"Snap- fusion: Text-to-image diffusion model on mobile devices within two seconds","cited_arxiv_id":null,"evidence_quote":"SnapFusion contributes the mobile-friendly architectural recipe, specifically dropping attention layers at the highest resolution, which Mobile-Pix2Pix adopts."},{"cited_title":"Fairy: Fast parallelized instruction-guided video-to-video synthesis","cited_arxiv_id":null,"evidence_quote":"Fairy defines the cross-frame attention baseline and the server-side latency comparison point that MoViE aims to beat."},{"cited_title":"Tiny autoencoder for stable diffusion","cited_arxiv_id":null,"evidence_quote":"TAESD is the lightweight autoencoder that replaces the InstructPix2Pix VAE and produces most of the FLOPs savings."},{"cited_title":"On distillation of guided diffusion models","cited_arxiv_id":null,"evidence_quote":"Guided-distillation literature supplies the idea of distilling classifier-free guidance into a single forward pass, which the paper extends to two modalities."},{"cited_title":"Consistent video- to-video transfer using synthetic dataset, 2023","cited_arxiv_id":null,"evidence_quote":"InsV2V is a state-of-the-art comparison method and the source of many of the reported baseline quality scores."},{"cited_title":"Cvpr 2023 text guided video editing competition, 2023","cited_arxiv_id":null,"evidence_quote":"TGVE is the benchmark dataset used for evaluating edit quality and consistency against prior methods."}],"review_version":1}