REVIEW 4 major objections 4 minor 37 references
A diffusion model that restores damaged film frame-by-frame at high resolution while preserving scene structure, trained on real restorations and requiring only one 24GB-VRAM GPU.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 13:12 UTC pith:ZZGNNDZQ
load-bearing objection A coherent architecture write-up and a plausible dataset idea, but the paper's central empirical claim is entirely unrepresented — empty experiment sections and no numbers — so the contribution cannot currently be evaluated. the 4 major comments →
HaineiFRDM: Structure-Preserving Diffusion for Film Restoration under Fast Motion and Diverse Defects
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Central claim: scene-structure damage in restored film comes not from the defects themselves but from faulty motion modeling, so a restoration model should not rely on optical flow. Instead, HaineiFRDM lets a pretrained video diffusion model use its intrinsic content-understanding to recognize and remove degradations while leaving clean content untouched. To scale to 2K/4K, the authors introduce patch-wise training and inference: each patch is restored with the help of whole-frame visual and textual information—via a position-aware global-frame fusion module and a global-prompt fusion module—so local patches stay consistent with the global scene. A patch-consistent inference framework first
What carries the argument
The load-bearing object is the Restoration-Guidance Network, a trainable side branch that injects learned residuals into a frozen pretrained video-diffusion U-Net, steering denoising toward the restored frame. Four mechanisms carry the argument. Global-frame fusion resizes the whole degraded frame, encodes it with a learned image encoder, adds the patch's bounding-box coordinates through a Fourier embedder, and injects this position-aware global feature via cross-attention, so a local patch can distinguish defect from content. Global-prompt fusion repeats this for a whole-frame text caption, letting the model exploit scene understanding. A texture-reconstruction module works in the 3D-FFT fr
Load-bearing premise
The real-degraded half of the training set is trustworthy: the before-after restoration videos are pixel-aligned and the defect masks, computed by frame differencing and filtered frame-by-frame by hand, mark actual defects rather than valid film content.
What would settle it
On a held-out real restoration video with fast motion, run the model and have human annotators flag any restored area where scene content (limbs, faces, text) changed shape or vanished; if the erasure rate matches or exceeds that of optical-flow baselines, the central content-understanding claim fails. A cheaper check: sample the training masks, compare them to human-labeled defects, and see whether frame-differencing masks have high precision—low precision would mean the model was trained to delete valid content.
If this is right
- Fast-motion shots should no longer show limb disappearance or structural distortion, because no optical-flow warping is used at any stage.
- A single 24GB-VRAM consumer GPU can restore high-resolution (2K/4K) frames, since the diffusion model runs on patches with features reused across patches.
- Defect restoration is decoupled from global color and deblur adjustments, so the restored film preserves the director's intended look rather than being aggressively 'enhanced'.
- The combination of real restored films and synthetic defect templates provides a training set that should transfer to unseen films better than purely synthetic data.
- The global-residual inference framework should suppress blocking artifacts when full frames are assembled from patches.
Where Pith is reading between the lines
- If the content-understanding claim holds, the method should extend to non-scratch degradations (dye fading, frame loss, flicker) by retraining or fine-tuning on paired data, because the architecture contains no explicit defect detector that would need redesign.
- The dataset construction bottleneck suggests that real-world performance is gated by the quality and scale of before-after restoration pairs; collecting more pairs with verifiable masks is the most direct test of generalization.
- The patch-consistent inference scheme, with global-residual fusion and cross-patch feature reuse, is a general recipe that could transfer to other high-resolution video restoration tasks, such as VHS or archival footage.
- A testable extension: measure restoration consistency on scenes where content repeats across shots; the position-aware fusion should keep repeated objects identical, a failure point for optical-flow baselines.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes HaineiFRDM, a patch-based latent diffusion framework for high-resolution film restoration. The method combines a ControlNet-style restoration-guidance network with global prompt/frame fusion modules, a frequency-domain texture-reconstruction module, and a patch-consistent inference procedure that fuses global residuals according to a cosine weighting schedule (Eqs. 1-2). The authors also introduce a dataset containing real-degraded film clips paired with restored frames and synthetic degradation generated from human-labeled defect templates. The abstract and introduction make strong empirical claims: superior restoration quality over existing open-source methods, strong structural consistency under fast motion, and training/inference on a single 24GB-VRAM consumer GPU.
Significance. If the empirical claims were supported, the paper would offer a meaningful step toward practical diffusion-based film restoration on consumer hardware. The architecture description is coherent and several design choices—position-aware global fusion, frequency-based texture conditioning, and global-residual inference weighting—are technically plausible. The dataset proposal, including categorized defect templates and real restoration-laboratory footage, is potentially valuable to the community. However, because the experimental section is almost entirely empty, no conclusion about the method's actual performance can be drawn. The contribution is therefore currently an unvalidated system description rather than a demonstrated advance.
major comments (4)
- [§6.2, §6.3, abstract] The central claim of 'superior restoration quality' and 'comprehensive experimental results conclusively demonstrate superiority' is unsupported: both quantitative subsections (6.2, 6.3) are empty headings. No PSNR, SSIM, LPIPS, BRISQUE, or FVD values are reported anywhere, and no comparison against RTN, DeepRemaster, TAPE, or Mamba-based baselines appears. Without any numeric results, the reader cannot verify the abstract's claims, and the claim itself is load-bearing for the paper's contribution.
- [§6.4, §6.5] The qualitative evaluation (6.4) and ablation study (6.5) are placeholder text, not results. Section 6.4 merely states 'show pathe newsreel results here'; no figures or qualitative comparisons are included. Section 6.5 lists ablation topics (6.5.1-6.5.5) but provides no metrics, no visual comparisons, and no discussion of the effect of removing components. The only quantitative statement, 'to reach PSNR=20dB... global prompt uses 12w iterations... enlarged fixed grid prompt costing 20w iterations,' is a training-convergence observation, not a restoration-quality metric, and it cannot support the claim that the proposed modules improve output quality.
- [§5 and §3 (real-degraded data)] The real-degraded training set is constructed by frame-differencing 'before-after' comparison videos and computing binary defect masks from paired frames. The text concedes a 'pixel-mismatch problem' due to deflickering/color toning and relies on manual frame-by-frame filtering to obtain a 5-minute training set. No independent validation of mask quality is provided. If misaligned pairs or incorrect masks remain, the model would be trained to erase valid content—the exact limb-disappearance failure attributed to optical-flow methods. Given that the paper's real-world generalization claims rest partly on this data, the absence of any mask-quality analysis is a substantive gap.
- [§7] The limitations listed in the conclusion directly qualify the paper's headline claims. The model 'still has some trouble recognizing and repainting for some constant lines scratches,' the synthetic pipeline 'may still suffer from generating LQ and GT pair with color distortion,' generation is 'slow,' and there is 'loss in exquisite details.' These are not minor caveats; together they undermine the advertised 'strong structural consistency' and 'superior restoration quality.' The conclusions should be calibrated to what is actually demonstrated, which—given the empty experimental sections—is nothing quantitative.
minor comments (4)
- [Throughout] There are numerous typos and formatting issues: 'dateset' (Sec. 3), 'LIPIPS' (Sec. 6.1.2), 'radom-sampled' (Eq. 3), 'freuqency' (Fig. 1 caption), and 'emplyed' (Sec. 1). Figures 4 and 5 are labeled only 'Example of a short caption' and do not show actual restored outputs.
- [§3 and §5] Sections 3 and 5 are nearly identical in content; this duplication should be removed or one section should be renamed and made the definitive dataset description.
- [§4.3] The loss weights α_p=1 and α_d=81 are introduced without any sensitivity analysis; a brief justification or ablation would be helpful (though not decisive given the larger experimental gap).
- [References] Reference [7] is truncated ('with no $'), and reference [19] mixes the RVRT paper (Liang et al.) with a shifted-window citation; the numbering is inconsistent with the citation context.
Circularity Check
No circular derivation found; central claims are unverified but not circular.
full rationale
I inspected the claimed derivation chain: the method is a ControlNet-style diffusion model with global fusion, frequency modules, and a patch-consistent inference scheme. Nothing in the paper defines a quantity in terms of the target prediction, fits a parameter and then reports it as a prediction, or imports a load-bearing result from the authors' own prior work (there are no self-citations). The training losses (Eq. 3-6) are standard supervised losses; the defect masks are generated from paired degraded/restored frames or from synthetic templates, so using them in the defect loss is supervised training rather than circular prediction. The inference-time global residual in Sec. 4.2 is produced by the same model from downsampled frames and then used as a reference; this is a coarse-to-fine bootstrapping design, not a reduction of the final output to its own input. The real circularity-adjacent concerns are evidentiary, not definitional: the quantitative sections are empty headings, the evaluation would run on the authors' own synthetic and self-collected data, and the Section 7 limitations undercut the empirical claims. These are serious credibility and validity problems, but they do not satisfy the required standard of exhibiting a specific equation or fitted parameter that is equivalent to its input by construction. Accordingly, the honest circularity finding is 0.
Axiom & Free-Parameter Ledger
free parameters (3)
- alpha_p (preprocess loss weight) =
1
- alpha_d (defect loss weight) =
81
- global-residual weight coefficient in GRFM =
0.25
axioms (5)
- domain assumption A pretrained video diffusion model's content prior transfers to film-defect restoration through ControlNet-style residual injection.
- domain assumption CLIP image/text embeddings of the full degraded frame, fused with Fourier-encoded patch coordinates, provide sufficient global context to resolve patch-level ambiguity.
- domain assumption Binary defect masks computed by paired-frame differencing on real restoration videos are accurate after manual filtering.
- domain assumption Hand-labeled defect templates sampled by appearance probability approximate the distribution of real film defects.
- standard math 3D FFT/iFFT of latent features is a valid invertible domain for texture-consistency constraints.
read the original abstract
Existing film-restoration methods frequently fail under fast motion, producing limb disappearance and structural distortion due to inaccurate motion modeling. Moreover, high-resolution restoration under spatially-persistent and mixed defects remains insufficiently studied. We propose HaineiFRDM, a Film Restoration Diffusion Model that leverages the content modeling capability of diffusion models for content-aware restoration, removing defects while preserving scene structure.To enable scalable high-resolution restoration, we adopt a patch-wise strategy with position-aware global fusion modules to maintain cross-patch coherence. We further introduce a frequency-based module to enhance texture consistency and a patch-consistent inference framework to alleviate blocking artifacts introduced by patch-based processing.We also construct a film restoration dataset comprising categorized defect templates, professionally restored films, and realistic synthetic degradations.Extensive experiments demonstrate our superior restoration quality with strong structural consistency. Our design also reduces memory requirements, enabling high-resolution restoration on a single 24GB-VRAM GPU.Code and the dataset will be released at https://anonymous.4open.science/r/HaineiFRDM.
Figures
Reference graph
Works this paper leans on
-
[1]
https : / / www
Diamant film restoration software. https : / / www . hs - art . com / index . php / research - main / deeprestore-menu. 2, 3, 6
-
[2]
https://mtifilm.com/ restoration/
Mti film restoration software. https://mtifilm.com/ restoration/. 3, 6
-
[3]
Reference-based restoration of digitized analog videotapes
Lorenzo Agnolucci, Leonardo Galteri, Marco Bertini, and Alberto Del Bimbo. Reference-based restoration of digitized analog videotapes. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1659– 1668, 2024. 2
2024
-
[4]
Multidiffusion: Fusing diffusion paths for controlled image generation
Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel. Multidiffusion: Fusing diffusion paths for controlled image generation. 2023. 3, 6
2023
-
[5]
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023. 2
Pith/arXiv arXiv 2023
-
[6]
Basicvsr: The search for essential compo- nents in video super-resolution and beyond
Kelvin CK Chan, Xintao Wang, Ke Yu, Chao Dong, and Chen Change Loy. Basicvsr: The search for essential compo- nents in video super-resolution and beyond. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4947–4956, 2021. 2
2021
-
[7]
Demofusion: Democratising high- resolution image generation with no $
Ruoyi Du, Dongliang Chang, Timothy Hospedales, Yi-Zhe Song, and Zhanyu Ma. Demofusion: Democratising high- resolution image generation with no $. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6159–6168, 2024. 3, 6
2024
-
[8]
Mamba: Linear-time sequence mod- eling with selective state spaces
Albert Gu and Tri Dao. Mamba: Linear-time sequence mod- eling with selective state spaces. InFirst conference on lan- guage modeling, 2024. 2
2024
-
[9]
Scalecrafter: Tuning-free higher- resolution visual generation with diffusion models
Yingqing He, Shaoshu Yang, Haoxin Chen, Xiaodong Cun, Menghan Xia, Yong Zhang, Xintao Wang, Ran He, Qifeng Chen, and Ying Shan. Scalecrafter: Tuning-free higher- resolution visual generation with diffusion models. InThe Twelfth International Conference on Learning Representa- tions, 2023. 3
2023
-
[11]
Cogvlm2: Visual language models for image and video understanding.arXiv preprint arXiv:2408.16500,
Wenyi Hong, Weihan Wang, Ming Ding, Wenmeng Yu, Qing- song Lv, Yan Wang, Yean Cheng, Shiyu Huang, Junhui Ji, Zhao Xue, et al. Cogvlm2: Visual language models for image and video understanding.arXiv preprint arXiv:2408.16500,
-
[12]
Fouriscale: A frequency perspective on training-free high-resolution image synthesis
Linjiang Huang, Rongyao Fang, Aiping Zhang, Guanglu Song, Si Liu, Yu Liu, and Hongsheng Li. Fouriscale: A frequency perspective on training-free high-resolution image synthesis. InEuropean conference on computer vision, pages 196–212. Springer, 2024. 3
2024
-
[13]
Deepremaster: tem- poral source-reference attention networks for comprehensive video enhancement.ACM Transactions on Graphics (TOG), 38(6):1–13, 2019
Satoshi Iizuka and Edgar Simo-Serra. Deepremaster: tem- poral source-reference attention networks for comprehensive video enhancement.ACM Transactions on Graphics (TOG), 38(6):1–13, 2019. 2
2019
-
[14]
Latent space super-resolution for higher-resolution image generation with diffusion models
Jinho Jeong, Sangmin Han, Jinwoo Kim, and Seon Joo Kim. Latent space super-resolution for higher-resolution image generation with diffusion models. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 2355–2365, 2025. 3
2025
-
[15]
Deepenhancer: Temporally consistent focal transformer for comprehensive video enhancement
Qin Jiang, Qinglin Wang, Lihua Chi, Wentao Ma, Feng Li, and Jie Liu. Deepenhancer: Temporally consistent focal transformer for comprehensive video enhancement. InPro- ceedings of the 2024 International Conference on Multimedia Retrieval, pages 969–977, 2024. 2
2024
-
[16]
When fast fourier transform meets transformer for image restoration
Xingyu Jiang, Xiuhui Zhang, Ning Gao, and Yue Deng. When fast fourier transform meets transformer for image restoration. InEuropean Conference on Computer Vision, pages 381–402. Springer, 2024. 5
2024
-
[17]
Efficient optimization of inpainting scheme and line scratch detection for old film restoration
Seong-Whan Kim and Ki-Hong Ko. Efficient optimization of inpainting scheme and line scratch detection for old film restoration. InPacific Rim International Conference on Artifi- cial Intelligence, pages 623–631. Springer, 2006. 2
2006
-
[18]
On missing data treatment for degraded video and film archives: a survey and a new bayesian ap- proach.IEEE Transactions on Image Processing, 13(3):397– 415, 2004
Anil C Kokaram. On missing data treatment for degraded video and film archives: a survey and a new bayesian ap- proach.IEEE Transactions on Image Processing, 13(3):397– 415, 2004. 2
2004
-
[19]
Recurrent video restoration trans- former with guided deformable attention.Advances in Neural Information Processing Systems, 35:378–393, 2022
Jingyun Liang, Yuchen Fan, Xiaoyu Xiang, Rakesh Ranjan, Eddy Ilg, Simon Green, Jiezhang Cao, Kai Zhang, Radu Timofte, and Luc V Gool. Recurrent video restoration trans- former with guided deformable attention.Advances in Neural Information Processing Systems, 35:378–393, 2022. 4
2022
-
[20]
Restoring degraded old films with recursive recurrent transformer networks
Shan Lin and Edgar Simo-Serra. Restoring degraded old films with recursive recurrent transformer networks. InProceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6718–6728, 2024. 2
2024
-
[21]
Accdiffusion: An accurate method for higher-resolution im- age generation
Zhihang Lin, Mingbao Lin, Meng Zhao, and Rongrong Ji. Accdiffusion: An accurate method for higher-resolution im- age generation. InEuropean Conference on Computer Vision, pages 38–53. Springer, 2025. 3
2025
-
[22]
Swin transformer: Hierarchical vision transformer using shifted windows
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 2
2021
-
[23]
Subject- diffusion: Open domain personalized text-to-image genera- tion without test-time fine-tuning
Jian Ma, Junhao Liang, Chen Chen, and Haonan Lu. Subject- diffusion: Open domain personalized text-to-image genera- tion without test-time fine-tuning. InACM SIGGRAPH 2024 Conference Papers, pages 1–12, 2024. 5
2024
-
[24]
Making old film great again: Degradation-aware state space model for old film restoration
Yudong Mao, Hao Luo, Zhiwei Zhong, Peilin Chen, Zhi- jiang Zhang, and Shiqi Wang. Making old film great again: Degradation-aware state space model for old film restoration. InProceedings of the Computer Vision and Pattern Recogni- tion Conference, pages 28039–28049, 2025. 2 8
2025
-
[25]
Learning transferable visual models from natural language supervi- sion
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021. 5
2021
-
[26]
High-resolution image synthesis with latent diffusion models
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2
2022
-
[27]
Image processing for restoration of old film se- quences
Takahiro Saito, Takashi Komatsu, Tomohisa Hoshi, and Toshi- aki Ohuchi. Image processing for restoration of old film se- quences. InProceedings 10th International Conference on Image Analysis and Processing, pages 709–714. IEEE, 1999. 2
1999
-
[28]
Hierarchical patch diffusion mod- els for high-resolution video generation
Ivan Skorokhodov, Willi Menapace, Aliaksandr Siarohin, and Sergey Tulyakov. Hierarchical patch diffusion mod- els for high-resolution video generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7569–7579, 2024. 3
2024
-
[29]
Athanasios Tragakis, Marco Aversa, Chaitanya Kaul, Roder- ick Murray-Smith, and Daniele Faccio. Is one gpu enough? pushing image generation at higher-resolutions with founda- tion models.arXiv preprint arXiv:2406.07251, 2(3):5, 2024. 3
Pith/arXiv arXiv 2024
-
[30]
Bring- ing old films back to life
Ziyu Wan, Bo Zhang, Dongdong Chen, and Jing Liao. Bring- ing old films back to life. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17694–17703, 2022. 2, 7
2022
-
[31]
Svfr: A unified framework for gener- alized video face restoration
Zhiyao Wang, Xu Chen, Chengming Xu, Junwei Zhu, Xi- aobin Hu, Jiangning Zhang, Chengjie Wang, Yuqi Liu, Yiyi Zhou, and Rongrong Ji. Svfr: A unified framework for gener- alized video face restoration. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 7406–7415,
-
[32]
Fastcomposer: Tuning-free multi- subject image generation with localized attention.Interna- tional Journal of Computer Vision, 133(3):1175–1194, 2025
Guangxuan Xiao, Tianwei Yin, William T Freeman, Fr ´edo Durand, and Song Han. Fastcomposer: Tuning-free multi- subject image generation with localized attention.Interna- tional Journal of Computer Vision, 133(3):1175–1194, 2025. 5
2025
-
[33]
Focal attention for long-range interactions in vision transformers.Advances in Neural Information Processing Systems, 34:30008–30022,
Jianwei Yang, Chunyuan Li, Pengchuan Zhang, Xiyang Dai, Bin Xiao, Lu Yuan, and Jianfeng Gao. Focal attention for long-range interactions in vision transformers.Advances in Neural Information Processing Systems, 34:30008–30022,
-
[34]
Pixel-aware stable diffusion for realistic image super-resolution and personalized stylization
Tao Yang, Rongyuan Wu, Peiran Ren, Xuansong Xie, and Lei Zhang. Pixel-aware stable diffusion for realistic image super-resolution and personalized stylization. InEuropean conference on computer vision, pages 74–91. Springer, 2024. 6
2024
-
[35]
Motion- guided latent diffusion for temporally consistent real-world video super-resolution
Xi Yang, Chenhang He, Jianqi Ma, and Lei Zhang. Motion- guided latent diffusion for temporally consistent real-world video super-resolution. InEuropean conference on computer vision, pages 224–242. Springer, 2024. 2
2024
-
[36]
Inf- dit: Upsampling any-resolution image with memory-efficient diffusion transformer
Zhuoyi Yang, Heyang Jiang, Wenyi Hong, Jiayan Teng, Wendi Zheng, Yuxiao Dong, Ming Ding, and Jie Tang. Inf- dit: Upsampling any-resolution image with memory-efficient diffusion transformer. InEuropean Conference on Computer Vision, pages 141–156. Springer, 2025. 3, 6
2025
-
[37]
Adding conditional control to text-to-image diffusion models
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847, 2023. 4
2023
-
[38]
Upscale-a-video: Temporal-consistent diffusion model for real-world video super-resolution
Shangchen Zhou, Peiqing Yang, Jianyi Wang, Yihang Luo, and Chen Change Loy. Upscale-a-video: Temporal-consistent diffusion model for real-world video super-resolution. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2535–2545, 2024. 2 9
2024
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.