{"id":"816da3be-85ab-4812-a84c-ad0398c4534a","arxiv_id":"2501.09960","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A codebook-prediction transformer with cross-frame mean and variance modulation improves blind face video restoration quality and temporal coherence on synthetic and in-the-wild benchmarks.","lead":"This paper introduces a face video restoration model that predicts high-quality visual tokens from a learned codebook and adjusts their cross-frame statistics to reduce flicker. It reports improved restoration quality and temporal consistency over several image and video restoration baselines on synthetic and real-world benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation (7) cancels the variance term it claims to apply, so the printed motion statistics modulation does not implement variance matching; the central temporal-coherence claim rests on a mechanism the paper does not actually describe.","rationale":"The reader's weakest-assumption analysis identified the same Eq. (7) cancellation, and I agree that this is the most load-bearing point. The paper's central novelty is the claim that discrete motion priors in terms of cross-frame mean and variance make the predicted content statistics match those of real videos. Equation (7) is the only place where variance matching is supposed to happen, and as printed it cancels the rescaling. This directly undermines the abstract and the contribution statement. The issue is not merely stylistic: if the implementation follows the printed equation, the variance branch is inert and the reported IFD improvements must come from the mean shift or from other architectural components, not from the stated variance matching. If the implementation uses the corrected AdaIN form, the paper still has an unstated discrepancy between text and code, and the actual contribution of the variance branch remains unquantified. The reader's verdict of CONDITIONAL is appropriate: the concern is addressable by correcting Eq. (7) and by running the nullification experiment, but the central claim should not be accepted as written. I therefore recommend no change to the reader's verdict, and I set agreement_with_reader to 'agree' because the reader and I converge on the same load-bearing assumption.","tokens_in":11679,"tokens_out":4790,"duration_ms":47683,"concrete_test":"Clone the released repository and inspect the motion modulation forward pass. If the code contains σ'/(σ'+ε) exactly as in Eq. (7), reproduce the VFHQ-Test-Deg row of Table 2 with the variance branch nullified by setting σ' = 1 while keeping μ'; if IFD remains near 3.80 and PSNR/FID are unchanged, the variance-matching component contributes nothing to the reported temporal coherence. If, instead, the code divides by the input feature's standard deviation (true AdaIN), then Eq. (7) is a typo; in that case, rerun the same nullification to measure the variance branch's actual contribution to IFD and confirm whether the stated mechanism is responsible for the reported improvement.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing issue is the variance branch of the motion statistics modulation. Equation (7) reads z''_{f,w,h} = σ'_{f,w,h} (z'_{f,w,h} − μ_{f,w,h}) / (σ'_{f,w,h} + ε) + μ'_{f,w,h}. For ε much smaller than σ', the factor σ'/(σ'+ε) is approximately 1, so the output is approximately z' − μ + μ'. The predicted feature's own standard deviation is not divided out, and the retrieved variance σ'^2 is never used to rescale the feature. The operation therefore reduces to a mean shift toward the retrieved μ', and the abstract's claim that 'the statistics of the predicted content can match with that of real videos over time' is unsupported for the variance component. A correct statistics-matching modulation would use the input standard deviation in the denominator, e.g. z'' = σ'(z' − μ)/σ + μ'. This is not a cosmetic typo: it is exactly the operation that realizes the proposed variance matching, and the paper provides no other mechanism by which variance statistics are matched. The ablation evidence does not rescue the claim: Table 1 row (c), motion modulation alone, gives PSNR 12.55 and FID 210.12, so the module by itself severely degrades quality, while adding it to the S&T predictor changes IFD only from 3.92 to 3.80, a small margin that could be produced by the mean-shift component alone. Because the paper promises released code, the issue is checkable, but as printed the central mechanism does not perform the stated variance matching.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces DP-TempCoh, a blind face video restoration method built around two discrete priors: a vision bank for spatial-temporal-aware content prediction and a motion statistics bank for cross-frame mean/variance modulation. The content prediction module predicts bank indices from degraded video tokens using self-attention, and the motion modulation module adjusts predicted features by retrieving nearest-neighbor statistics from a bank built from high-quality face videos. The method is evaluated on VFHQ-Test-Deg, HDTF-Deg, and YTF-Medium/Hard, with extensive comparisons to image- and video-level restoration baselines, and reports state-of-the-art results on all reported metrics, including temporal coherence measured by IFD. The paper also provides ablations, visualizations, and a user study.","tokens_in":12033,"tokens_out":3721,"duration_ms":37330,"significance":"If the proposed mechanisms work as described, the paper would make a solid empirical contribution to blind face video restoration, combining dictionary-style visual priors with temporally modulated statistics, and it would be notable for reporting consistent improvements across quality, identity, and temporal-coherence metrics on both synthetic and in-the-wild benchmarks. The authors promise code release, which strengthens the reproducibility potential. However, the significance is currently undermined by a central technical issue in the stated modulation mechanism and by ambiguity in the training objective, so the empirical results cannot be attributed to the method as written without clarification or correction.","major_comments":[{"comment":"Equation (7) as printed multiplies and divides by the same target standard deviation: z''_{f,w,h} = σ'_{f,w,h} (z'_{f,w,h} − μ_{f,w,h})/(σ'_{f,w,h} + ε) + μ'_{f,w,h}. For ε much smaller than σ', the factor σ'/(σ'+ε) is approximately 1, so the output is approximately z' − μ + μ'. The input feature's own standard deviation is never divided out, and the retrieved variance σ'^2 is never used to rescale the feature. The operation therefore reduces to a mean shift toward the retrieved μ', contradicting the abstract's claim that 'the statistics of the predicted content can match with that of real videos over time.' A correct variance-matching operation would use the input standard deviation in the denominator, e.g., z'' = σ'(z' − μ)/σ + μ'. This is load-bearing because the proposed temporal-coherence mechanism is explicitly built on variance matching, and Table 1 does not rescue the claim: row (c) shows that motion modulation alone severely degrades quality (PSNR 12.55, FID 210.12), and row (d) shows only a small IFD change (3.92 to 3.80), which could be produced by the mean-shift component alone.","section":"Motion Prior-based Statistics Modulation, Eq. (7)"},{"comment":"The bank-index prediction loss L_bank in Eq. (11) uses ground-truth labels z^gt 'derived from the pre-trained encoder E and vision bank T,' while Eq. (12) jointly optimizes E, C, and G. If E is updated during training, the label distribution z^gt changes as E's tokenization changes, so L_bank can be minimized by adapting the encoder rather than by learning to predict meaningful high-quality indices from the degraded input. The paper does not state whether E is frozen after bank construction, whether labels are recomputed periodically, or whether a stop-gradient is applied. Without this specification, the content-prediction training objective is not well defined, and the reported contribution of the content prediction module is ambiguous.","section":"Model Training, Eqs. (11)-(12)"},{"comment":"The construction of the motion statistics bank M is not described. Equation (6) retrieves entries from a bank of size 16,384, but the paper does not specify how these entries were collected from high-quality videos, what each entry represents (per-frame scalar statistics or per-pixel maps), or how the nearest-neighbor search is performed and trained. The notation in Eqs. (4)-(7) is also internally inconsistent: Eqs. (4)-(5) compute per-pixel channel-wise mean and variance over the channel dimension, yielding W×H values per frame, whereas the text describes 'cross-frame mean and variance vectors whose components are the channel mean and variance corresponding to each frame.' This ambiguity makes Eq. (7) impossible to reproduce and impedes assessment of whether the proposed modulation actually operates on frame-level or pixel-level statistics.","section":"Motion Prior-based Statistics Modulation and Implementation Details"}],"minor_comments":[{"comment":"There are several typos and inconsistencies that should be corrected: 'detial' (Introduction), 'donotes' (Experiments), 'tenporal' (User Study), 'labeld' (Content Prediction), 'IDF' for IFD in Table 2 and its caption, 'TYF-Hard' for YTF-Hard in Table 2, 'BVFR' for BFVR in Related Work, and 'DiffFace' where the method is named 'DifFace.'","section":"Throughout"},{"comment":"The evaluation protocol states that video clips are 8 frames long, but it does not clarify whether the temporal statistics in Eqs. (4)-(7) are computed within each 8-frame clip or over longer sequences; this affects the interpretation of the motion bank and the reported IFD values.","section":"Experiments, Evaluation Protocol"},{"comment":"The degradation ranges in Eq. (13) are specified as b' ∈ [b−1, b+1] with b ∈ {2:32}, which allows a downsampling factor as low as 1 (no downsampling) for b=2; please clarify whether this is intended and how the range is sampled in practice.","section":"Eq. (13)"},{"comment":"The convergence comparison in Figure 3 reports that 'S-aware' requires 5.6 times more iterations, but the figure itself does not show the loss curves or the exact convergence criterion; please add the curves and define the criterion so the claim is verifiable.","section":"Content Prediction, Figure 3"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The headline: this is a useful, moderately novel face-video restoration paper, but the central statistics-matching mechanism is misstated in the printed equations. The architecture is plausible and the empirical work is broad; the issues are checkable because code is promised.\n\nWhat is actually new: combining spatial-temporal codebook prediction for video tokens with a learned motion-statistics bank. That combination is not in CodeFormer, RestoreFormer, or the video baselines cited. The experiments are extensive: VFHQ, HDTF, YTF-Medium/Hard, multiple baselines, a user study, and an ablation table. On the reported numbers the full model does well, especially on IFD/FID/IDS. This is incremental but honest engineering, and if the mechanism is corrected, the contribution is real.\n\nNow the soft spots, in proportion. The load-bearing one is Eq. (7). As printed it multiplies and divides by the same retrieved standard deviation, so the factor is about 1 and the variance scaling cancels. The operation reduces to a mean shift toward the retrieved mu. The abstract's claim that the predicted content's statistics match real videos is therefore unsupported for the variance component. The likely fix is to divide by the input feature's own standard deviation, not the retrieved one, but as written the mechanism does not do what the paper says. This is not cosmetic: it is exactly the operation that realizes variance matching.\n\nThe second issue is the bank-index label in Eq. (11). The labels are said to be derived from the pre-trained encoder E and vision bank T, while Eq. (12) optimizes E. If E is updated during training, the label distribution shifts and the index prediction can track the encoder. The paper needs to state explicitly whether the label-generating encoder is frozen or whether labels are refreshed.\n\nThird, the motion bank construction is not described, and Eq. (6)'s nearest-neighbor lookup is non-differentiable with a 16,384-entry bank. That may be acceptable, but the details matter for reproducibility.\n\nFourth, there is a real ablation-text discrepancy: the text says the motion-only model (c) drops IFD from 9.86 to 3.92, but Table 1 reports 5.59. Also, motion alone degrades quality severely, and the combined gain is small (3.92 to 3.80). So the temporal-coherence benefit of the module is weaker than the prose suggests.\n\nThese problems are addressable and none kills the empirical claim outright, but as printed the central mechanism does not do what is claimed. I would send this to peer review rather than desk reject, and ask the authors to correct Eq. (7), clarify the label-generation setup, describe the motion bank, and fix the ablation narrative. If the released code confirms the intended variance normalization, the results likely stand.","headline":"Plausible, useful face-video restoration architecture whose printed motion-statistics modulation is not correctly stated: Eq. (7) cancels the variance term it claims to apply, so the central temporal-coherence mechanism is unsupported as written.","tokens_in":12568,"tokens_out":2815,"would_cite":false,"duration_ms":30384,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-bank discrete prior mechanism predicts face video content and modulates it with motion statistics to produce coherent, identity-stable restoration.","keywords":["blind face video restoration","discrete visual priors","codebook lookup transformer","temporal coherence","motion statistics modulation","video restoration","face identity preservation","in-the-wild degradation"],"falsifier":"Measure the channel-wise mean and variance of DP-TempCoh's restored clips frame by frame and compare them with the nearest retrieved bank vectors and with ground-truth video statistics; if the output statistics do not match the bank vector, the stated mechanism is not what produces the reported IFD gains. A direct arithmetic check of the modulation formula on a synthetic tensor would reveal whether the output variance equals the retrieved variance, since the retrieved variance appears as both multiplier and denominator in the printed formula.","tokens_in":11473,"feed_emoji":"🎥","tokens_out":15977,"duration_ms":138249,"temperature":0.7,"pith_summary":"Blind face video restoration has to recover lost detail while keeping the restored face from changing identity or flickering between frames. This paper claims that both can be handled with two discrete priors: a vision bank of 1,024 high-quality face tokens for content and a motion-statistics bank of 16,384 cross-frame mean and variance vectors for temporal coherence. The proposed DP-TempCoh predicts codebook indices from degraded video tokens using spatial-temporal self-attention, retrieves the corresponding high-quality features, and then modulates those features toward the retrieved motion statistics before decoding. Across two synthetic benchmarks and two in-the-wild subsets, the paper reports the best values on every metric it lists, including PSNR, IDS, LPIPS, FID, and IFD on the synthetic sets and FID and IFD on the wild sets. If correct, the framework provides a deterministic alternative to diffusion-based face video restoration, avoiding the per-frame sampling randomness that causes flicker.","feed_headline":"Two prior banks beat image and video baselines for face clips","feed_subtitle":"A spatial-temporal codebook predicts content; a motion-statistics bank matches real video dynamics.","key_machinery":"Two discrete banks carry the argument. The visual prior bank, built by vector quantization during high-quality face reconstruction, holds 1,024 visual tokens; a spatial-temporal-aware transformer predicts one bank index per degraded video token and extracts the matching entry as the restored content feature. The motion prior bank holds 16,384 cross-frame mean and variance vectors; the modulation module concatenates each predicted feature's frame-wise mean and variance, retrieves the nearest stored vector, and adjusts the predicted content so its temporal statistics align with the stored high-quality statistics. A cross-attention transformer then takes the original predicted content as the query and the modulated content as key and value, and a generator with 3D residual blocks and frame attention decodes the fused representation into a video clip. The training objective combines a cross-entropy loss on the predicted bank indices, a pixel and perceptual consistency loss, and an adversarial loss.","core_discovery":"DP-TempCoh's central claim is that high-quality face video content can be synthesized by replacing degraded video tokens with entries from a discrete visual prior bank, and then made temporally coherent by matching each frame's channel-wise mean and variance to statistics retrieved from a bank learned from high-quality face videos. The content prediction module applies self-attention over tokens with learned spatial-temporal position embeddings, so the codebook index predicted for one frame is informed by neighboring frames rather than by that frame alone. The motion statistics modulation module then retrieves the nearest stored cross-frame mean and variance vector and adjusts the predicted content so that, in the paper's words, the statistics of the predicted content match those of real videos over time. In the reported experiments, the full model reports the best PSNR, IDS, LPIPS, FID, and IFD among all compared methods on VFHQ-Test-Deg and HDTF-Deg, including an IFD of 3.80 versus 5.11 for the second-best video method on VFHQ-Test-Deg and an IDS of 0.7721 versus 0.7115 for the best image method. On the in-the-wild YTF-Medium and YTF-Hard subsets, it reports the best FID values of 51.86 and 55.50, with IFD values of 5.51 and 7.38.","pith_inferences":["If the printed modulation formula is read literally, the retrieved variance appears as both a multiplier and a denominator in the same term, making the operation a mean shift rather than a variance match; zeroing the variance component should then leave IFD essentially unchanged.","The nearest-neighbor retrieval in the motion bank is non-differentiable, so that bank cannot be trained end-to-end with the rest of the network; shrinking or pruning the bank should measurably degrade temporal coherence if the bank's coverage is load-bearing.","The ablation table shows that spatial-temporal prediction alone lowers IFD from 9.86 to 3.92, while adding motion modulation lowers it only to 3.80; freezing the spatial-temporal module and re-training the modulation alone would isolate how much coherence each component actually contributes.","Applying the same two-bank design to non-face video restoration and measuring IFD would test whether the motion-statistics bank encodes face-specific dynamics or a general temporal-coherence principle."],"forward_implications":["On VFHQ-Test-Deg, the reported inter-frame difference drops to 3.80, below the best image method (9.86) and the best video method (5.11), so discrete statistics modulation can suppress flicker in the compared setting.","The reported identity similarity of 0.7721 versus 0.7115 for the nearest competitor suggests that shared codebook content keeps identity stable across frames better than per-frame generative priors.","Because inference does not require diffusion sampling, restored videos avoid the per-frame randomness that produces flicker in sampling-based methods.","The reported FID values of 51.86 and 55.50 on YTF-Medium and YTF-Hard indicate the two-bank design carries over to naturally degraded videos, not only to synthetic degradations.","The separation of a content bank and a motion bank offers a template for other video-to-video tasks where content fidelity and temporal smoothness compete."],"supporting_citations":[{"why":"CodeFormer supplies the codebook-lookup face restoration paradigm that the visual prior module extends to video with spatial-temporal context.","marker":"(Zhou et al. 2022b)"},{"why":"VQFR establishes vector-quantized dictionary reconstruction, the basis for building the vision bank through high-quality image reconstruction.","marker":"(Gu et al. 2022)"},{"why":"VFHQ is the high-quality face video dataset used to train and test the model and to learn the banks.","marker":"(Xie et al. 2022)"},{"why":"Its degradation model defines the synthetic LQ/HQ training pairs used in Eq. (13).","marker":"(Yang et al. 2021)"},{"why":"BasicVSR++ is the recurrent video restoration baseline on the temporal coherence metric IFD that the paper claims to beat.","marker":"(Chan et al. 2022)"},{"why":"FMA-Net is the flow-guided video restoration baseline used in the comparisons on synthetic and in-the-wild data.","marker":"(Youk, Oh, and Kim 2024)"},{"why":"CosFace supplies the face recognition model used to compute identity similarity IDS, a key claimed advantage.","marker":"(Wang et al. 2018)"}],"fun_headline_variants":["Two prior banks restore face videos with temporal coherence","Codebook and motion statistics modulate face video restoration","DP-TempCoh: temporal-coherent face video prediction","Match real video dynamics with discrete motion priors","Two banks: visual codebook and motion statistics for face video"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that retrieving nearest-neighbor cross-frame mean and variance statistics from a pre-built bank of high-quality videos and applying them to predicted content is a sufficient and reliable way to enforce temporal coherence; the bank's construction and the non-differentiable retrieval are not described, and as printed the variance scaling in the modulation formula cancels itself.","fun_headline_variants_meta":{"raw":{"variants":["Two prior banks restore face videos with temporal coherence","Codebook and motion statistics modulate face video restoration","DP-TempCoh: temporal-coherent face video prediction","Match real video dynamics with discrete motion priors","Two banks: visual codebook and motion statistics for face video"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000771,"raw_usage":{"total_tokens":3440,"prompt_tokens":995,"completion_tokens":2445,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":611,"completion_tokens_details":{"reasoning_tokens":2368}},"tokens_in":611,"tokens_out":2445,"duration_ms":19260,"temperature":1.0,"reasoning_tokens":2368,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T19:30:05.625904+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the channel-wise mean and variance of DP-TempCoh's restored clips frame by frame and compare them with the nearest retrieved bank vectors and with ground-truth video statistics; if the output statistics do not match the bank vector, the stated mechanism is not what produces the reported IFD gains. A direct arithmetic check of the modulation formula on a synthetic tensor would reveal whether the output variance equals the retrieved variance, since the retrieved variance appears as both multiplier and denominator in the printed formula.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"VFHQ is the high-quality face video dataset used to train and test the model and to learn the banks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Its degradation model defines the synthetic LQ/HQ training pairs used in Eq. (13)."},{"cited_title":"C.; Zhou, S.; Xu, X.; and Loy, C","cited_arxiv_id":null,"evidence_quote":"BasicVSR++ is the recurrent video restoration baseline on the temporal coherence metric IFD that the paper claims to beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"FMA-Net is the flow-guided video restoration baseline used in the comparisons on synthetic and in-the-wild data."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CosFace supplies the face recognition model used to compute identity similarity IDS, a key claimed advantage."}],"review_version":1}