REVIEW 3 major objections 5 minor 45 references
Predicting the Original Appearance of Damaged Historical Documents
T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read The paper claims that a diffusion-based network, DiffHDR, can predict the original appearance of damaged historical documents when given the damaged image plus a content image and mask of the missing characters, and that it outperforms…
desk verdict A genuinely useful benchmark dataset and a strong conditional inpainter, but the title and abstract promise autonomous reconstruction the method doesn't deliver. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is an 8-channel conditional diffusion denoiser that concatenates the noised image, the damaged image, the content image, and the mask image, and outputs the repaired image in a single forward pass. The content image supplies the semantics of what should appear and the mask supplies the spatial location, so the model does not need to infer damaged content from pixels alone. Randomly dropping these conditions during training enables classifier-free guidance at inference, with separate guidance scales for the damaged image and for the semantic/spatial pair. The character perceptual loss, computed as a masked multi-scale VGG feature distance between the repaired and target regions, focuses training on producing correct character content and style within the repaired area.
What would settle it
Feed DiffHDR real damaged document patches with severe ink erosion or large missing regions while suppressing the content and mask cues (as in the paper's limitation figure) and measure the repaired-region character recognition accuracy; the paper's own limitation predicts it will remain near chance. A complementary test is to supply a deliberately wrong transcript for one damaged character: if the model faithfully renders the wrong character, that confirms the repair is copying the supplied cue rather than inferring the original content, which is exactly the conditional behavior the paper claims.
Extended reading notes
Core claim
The central claim is that a conditional diffusion process can reconstruct the original appearance of damaged historical documents when it is told what the missing characters say and where they sit. Formally, the model learns $x_r = F(x_d, x_c, x_m)$, where $x_d$ is the damaged image, $x_c$ is a content image encoding the transcript of the damaged characters, and $x_m$ is a mask marking the damaged regions. DiffHDR's denoiser predicts the repaired image directly, rather than the added noise, and is trained with a diffusion loss plus a character perceptual loss that penalizes feature mismatches inside the repaired regions. The authors report that this design outperforms the compared methods on the HDR28K benchmark and that the model trained only on synthetic degradations still repairs real damaged documents, edits text content, and generates new text blocks in the document's style.
Load-bearing premise
At test time, someone must already know and supply the content and exact location of every damaged character, and if the damage is severe the method cannot repair the image without those cues.
Editorial extensions
If this is right
- HDR28K gives future work a common benchmark of 28,552 damaged-repaired pairs with character-level annotations, so new repair methods can be trained and compared on the same ground.
- A model trained only on synthetic degradations transfers to real damaged scans, which means large-scale synthetic data can substitute for scarce real damaged-repaired pairs.
- The same conditional diffusion framework performs document editing and text block font generation, so one model can rewrite or extend a historical document in its own style.
- Repaired-region character recognition accuracy is substantially higher with DiffHDR than with the compared baselines, implying fewer hallucinated or wrong characters in the output.
Reading between the lines
- The method's reliance on user-provided transcripts means an end-to-end repair tool would need an upstream module that reads partially damaged text; the paper names this as future work but does not implement it.
- Because HDR28K is built from Chinese historical documents and three synthetic damage types, transfer to other scripts, inks, and damage modes is untested; analogous datasets for other traditions would be a direct extension.
- The conditional formulation suggests a controlled-use application: by deliberately altering the content image, the same model could serve as a text-editing or reconstruction tool for digital heritage, a capability the paper only begins to explore.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Historical Document Repair (HDR), a task of reconstructing the original appearance of damaged historical document images. It contributes HDR28K, a dataset of 28,552 synthetic damaged-repaired pairs with character-level annotations, and DiffHDR, a diffusion-based model conditioned on the damaged image xd, a content image xc, and a mask image xm, together with a character perceptual loss. Experiments on HDR28K compare against nine baselines and report lower FID/LPIPS and higher character-recognition accuracy, along with qualitative results on real damaged documents and extensions to document editing and text-block generation.
Significance. The dataset and the guided-repair formulation are potentially useful contributions: HDR28K is large, multi-style, and accompanied by character-level annotations; the held-out evaluation is reasonably designed with no same-document leakage between train and test; the comparison is broad; and the announced code and data release would support reproducibility. The reported gains on the guided task are substantial. However, the significance as currently framed is overstated: the method does not predict unknown damaged content but renders user-specified content into user-specified locations. If repositioned as an interactive repair tool, the work is solid; the autonomous 'predicting the original appearance' claim requires an unconditional model or a demonstration without user-supplied cues.
major comments (3)
- [Abstract; Eq. (2); Section 'DiffHDR: Diffusion-based HDR Network' (Condition Parsing); Limitation] The central claim of predicting the original appearance of damaged documents is not supported for the autonomous setting. The task is defined in Eq. (1) as xr = FHDR(xd), but the proposed method in Eq. (2) is xr = F(xd, xc, xm), where xc and xm are parsed from user-provided content and location. The condition-parsing paragraph explicitly states that 'the user provides the content and location of the damaged characters,' and the Limitation section concedes that without semantic and spatial information, severe damage cannot be repaired. The abstract and title should be qualified, or the paper should report results without xc and xm and quantify the gap.
- [Evaluation Metrics; Table 1] Rec-ACC is measured with xc supplied, because the recognizer is applied to repaired regions that were generated from the ground-truth content image. This metric therefore measures how faithfully the model renders given content, not how well it recovers unknown content. The 'character correctness' claim in Table 1 should be stated in conditional terms; an unconditional evaluation (for example, recognition accuracy on outputs produced without xc) is needed to support the autonomous framing.
- [Real Damaged Document Image Repair (Figure 9)] The real-document experiment does not report what inputs were provided: whether xc and xm were supplied and, if so, whether they came from manual transcription or an automated recognizer. Without this information, the qualitative results support only guided repair, not autonomous damage recovery. Please state the protocol and, if feasible, include a failure analysis when the cues are withheld.
minor comments (5)
- [Related Work heading] The heading 'Historcial Document Image Processing' contains a typo; it should be 'Historical.'
- [Eq. (2) explanation] The word 'senmantic' in the explanation of xc should be 'semantic.'
- [Eq. (6)] The notation 'sc.m' should be 'sc,m' to match the text describing the guidance scales.
- [Figure 10 caption] The caption 'without semantic and visual cues' is ambiguous because the first row shows successful repair; the text clarifies this, but the figure should distinguish the moderate case from the severe failure case.
- [Attribute-Sensitive Repair Strategy] The sentence describing the 8% dropout is ambiguous about the probabilities for the three conditions; please specify the exact sampling schedule (for example, each condition with 8% independently or jointly).
Circularity Check
No circularity: the model is a supervised conditional repair network whose user-provided content and mask inputs are disclosed, the evaluation is held-out and compares against baselines receiving the same cues, and the limitation that severe damage cannot be repaired without cues is explicitly acknowledged.
full rationale
The derivation chain is self-contained and does not reduce to its inputs. The paper defines the HDR task in Eq. (1) as xr = FHDR(xd), but the proposed method in Eq. (2) explicitly conditions on user-supplied semantic and spatial cues: xr = F(xd, xc, xm). The condition-parsing section states 'the user provides the content and location of the damaged characters and we parse them out to obtain the content image xc and the mask image xm.' This means the system is a guided inpainter rather than an autonomous content predictor, but this is disclosed in the method and reiterated in the Limitation section ('when the damage is severe, our method is unable to repair the image in the absence of semantic and spatial information'). The held-out test set is constructed from source-dataset test images with no patch overlap with training, and all compared methods are given the same concatenated inputs xd, xc, xm, so the quantitative gains are not forced by giving DiffHDR extra information. Rec-ACC is measured with xc supplied, so it evaluates rendering fidelity to given content rather than recovery of unknown content; that is a scope limitation, not a circular derivation. The citations to prior same-group works (MTHv2, M5HisDoc, UPOCR, FontDiffuser) are used as dataset sources, baselines, or related work, not as load-bearing unverified theorems, and no uniqueness claim is imported from the authors. The title and abstract phrase 'predict the original appearance' is broader than the implemented conditional repair, but the paper itself acknowledges the narrower capability, so the gap is a correctness or scope concern, not circularity.
Assumptions & free parameters
free parameters (3)
- Guidance scale s_d =
1.2
- Guidance scale s_c,m =
1.5
- Conditional dropout probability =
8%
assumptions (4)
- domain assumption Synthetic degradations (character missing via LaMa, paper damage masks, ink erosion via genalog) faithfully simulate real historical document damage.
- domain assumption MTHv2 and M5HisDoc character annotations provide accurate ground-truth content for all damaged regions.
- domain assumption Pretrained VGG features are a suitable perceptual space for character content preservation.
- standard math The standard DDPM forward process equations hold.
Cite this review
Pith. "Pith review of Predicting the Original Appearance of Damaged Historical Documents." pith.science (2026). https://pith.science/paper/MZQFXBSS
@misc{pith2026241211634,
author = {Pith},
title = {Pith review of: Predicting the Original Appearance of Damaged Historical Documents},
year = {2026},
howpublished = {\url{https://pith.science/paper/MZQFXBSS}},
note = {Machine review of arXiv:2412.11634}
}
read the original abstract
Historical documents encompass a wealth of cultural treasures but suffer from severe damages including character missing, paper damage, and ink erosion over time. However, existing document processing methods primarily focus on binarization, enhancement, etc., neglecting the repair of these damages. To this end, we present a new task, termed Historical Document Repair (HDR), which aims to predict the original appearance of damaged historical documents. To fill the gap in this field, we propose a large-scale dataset HDR28K and a diffusion-based network DiffHDR for historical document repair. Specifically, HDR28K contains 28,552 damaged-repaired image pairs with character-level annotations and multi-style degradations. Moreover, DiffHDR augments the vanilla diffusion framework with semantic and spatial information and a meticulously designed character perceptual loss for contextual and visual coherence. Experimental results demonstrate that the proposed DiffHDR trained using HDR28K significantly surpasses existing approaches and exhibits remarkable performance in handling real damaged documents. Notably, DiffHDR can also be extended to document editing and text block generation, showcasing its high flexibility and generalization capacity. We believe this study could pioneer a new direction of document processing and contribute to the inheritance of invaluable cultures and civilizations. The dataset and code is available at https://github.com/yeungchenwa/HDR.
Figures
Figures from the paper (13 more)
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Amin, J.; Siddiqi, I.; and Moetesum, M. 2023. Reconstruction of Broken Writing Strokes in Greek Papyri. In International Conference on Document Analysis and Recognition, 253--266. Springer
work page 2023
-
[4]
Assael, Y.; Sommerschield, T.; Shillingford, B.; Bordbar, M.; Pavlopoulos, J.; Chatzipanagiotou, M.; Androutsopoulos, I.; Prag, J.; and de Freitas, N. 2022. Restoring and attributing ancient texts using deep neural networks. Nature, 603(7900): 280--283
work page 2022
-
[5]
Brooks, T.; Holynski, A.; and Efros, A. A. 2023. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18392--18402
2023
-
[6]
Chen, L.; Chu, X.; Zhang, X.; and Sun, J. 2022. Simple baselines for image restoration. In European Conference on Computer Vision, 17--33. Springer
work page 2022
-
[7]
Chen, Z.; Wu, J.; Wang, W.; Su, W.; Chen, G.; Xing, S.; Zhong, M.; Zhang, Q.; Zhu, X.; Lu, L.; et al. 2024. InternVL : Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 24185--24198
work page 2024
-
[8]
Cui, Y.; Ren, W.; Cao, X.; and Knoll, A. 2023. Focal Network for Image Restoration. In Proceedings of the IEEE/CVF international conference on computer vision, 13001--13011
work page 2023
Show all 45 references
-
[9]
Das, S.; Ma, K.; Shu, Z.; Samaras, D.; and Shilkrot, R. 2019. Dewarpnet: Single-image document unwarping with stacked 3d and 2d regression networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 131--140
2019
-
[10]
E.-A.; and Cheriet, M
Ech-Cherif, M. E.-A.; and Cheriet, M. 2022. Frank-Wolfe-based multi-task learning for historical document restoration. In 2022 26th International Conference on Pattern Recognition (ICPR), 3900--3907. IEEE
2022
-
[11]
Hedjam, R.; and Cheriet, M. 2013. Historical document image restoration using multispectral imaging system. Pattern Recognition, 46(8): 2297--2312
2013
-
[12]
Hertlein, F.; and Naumann, A. 2023. Template-Guided Illumination Correction for Document Images with Imperfect Geometric Reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 904--913
2023
-
[13]
Heusel, M.; Ramsauer, H.; Unterthiner, T.; Nessler, B.; and Hochreiter, S. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30
2017
-
[14]
Ho, J.; Jain, A.; and Abbeel, P. 2020. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33: 6840--6851
2020
-
[15]
Ho, J.; and Salimans, T. 2022. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598
2022 arXiv
-
[16]
Isola, P.; Zhu, J.-Y.; Zhou, T.; and Efros, A. A. 2017. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 1125--1134
2017
-
[17]
Jiang, X.; Long, R.; Xue, N.; Yang, Z.; Yao, C.; and Xia, G.-S. 2022. Revisiting document image dewarping by grid regularization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4543--4552
2022
-
[18]
Kong, Y.; Luo, C.; Ma, W.; Zhu, Q.; Zhu, S.; Yuan, N.; and Jin, L. 2022. Look closer to supervise better: one-shot font generation via component-based discriminator. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 13482--13491
2022
-
[19]
Li, H.; Wu, X.; Chen, Q.; and Xiang, Q. 2023 a . Foreground and Text-lines Aware Document Image Rectification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 19574--19583
2023
-
[20]
Li, Y.; Fan, Y.; Xiang, X.; Demandolx, D.; Ranjan, R.; Timofte, R.; and Van Gool, L. 2023 b . Efficient and explicit modelling of image hierarchies for image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18278--18289
2023
-
[21]
Li, Z.; Chen, X.; Pun, C.-M.; and Cun, X. 2023 c . High-Resolution Document Shadow Removal via A Large-Scale Real-World Dataset and A Frequency-Aware Shadow Erasing Net. arXiv preprint arXiv:2308.14221
2023 arXiv
-
[22]
Lin, Y.-H.; Chen, W.-C.; and Chuang, Y.-Y. 2020. Bedsr-net: A deep shadow removal network from a single document image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12905--12914
2020
-
[23]
Lu, C.; Zhou, Y.; Bao, F.; Chen, J.; Li, C.; and Zhu, J. 2022. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arXiv:2211.01095
2022 arXiv
-
[24]
Ma, W.; Zhang, H.; Jin, L.; Wu, S.; Wang, J.; and Wang, Y. 2020. Joint layout analysis, character detection and recognition for historical document digitization. In 2020 17th International Conference on Frontiers in Handwriting Recognition (ICFHR), 31--36. IEEE
2020
-
[25]
C.; Nguyen, C
Nguyen, K. C.; Nguyen, C. T.; Hotta, S.; and Nakagawa, M. 2019. A character attention generative adversarial network for degraded historical document restoration. In 2019 International Conference on Document Analysis and Recognition (ICDAR), 420--425. IEEE
2019
-
[26]
Peng, D.; Yang, Z.; Zhang, J.; Liu, C.; Shi, Y.; Ding, K.; Guo, F.; and Jin, L. 2023. UPOCR: Towards unified pixel-level ocr interface. In Forty-first International Conference on Machine Learning
2023
-
[27]
Raha, P.; and Chanda, B. 2019. Restoration of historical document images using convolutional neural networks. In 2019 IEEE region 10 symposium (TENSYMP), 56--61. IEEE
2019
-
[28]
Ronneberger, O.; Fischer, P.; and Brox, T. 2015. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention--MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part II...
2015
-
[29]
Shi, Y.; Liu, C.; Peng, D.; Jian, C.; Huang, J.; and Jin, L. 2023. M5HisDoc: A Large-scale Multi-style Chinese Historical Document Analysis Benchmark. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track
2023
-
[30]
Simonyan, K.; and Zisserman, A. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
2014 arXiv
-
[31]
Suvorov, R.; Logacheva, E.; Mashikhin, A.; Remizova, A.; Ashukha, A.; Silvestrov, A.; Kong, N.; Goka, H.; Park, K.; and Lempitsky, V. 2022. Resolution-robust large mask inpainting with fourier convolutions. In Proceedings of the IEEE/CVF winter conference on applications of co...
2022
-
[32]
Wadhwani, M.; Kundu, D.; Chakraborty, D.; and Chanda, B. 2021. Text extraction and restoration of old handwritten documents. Digital Techniques for Heritage Presentation and Preservation, 109--132
2021
-
[33]
Wang, C.; Zhou, M.; Ge, T.; Jiang, Y.; Bao, H.; and Xu, W. 2023. CF-Font: Content Fusion for Few-shot Font Generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1858--1867
2023
-
[34]
Wang, P.; Bai, S.; Tan, S.; Wang, S.; Fan, Z.; Bai, J.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; et al. 2024. Qwen2-VL : Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191
2024 arXiv
-
[35]
Wang, Y.; Zhou, W.; Lu, Z.; and Li, H. 2022 a . Udoc-gan: Unpaired document illumination correction with background light prior. In Proceedings of the 30th ACM International Conference on Multimedia, 5074--5082
2022
-
[36]
Wang, Z.; Cun, X.; Bao, J.; Zhou, W.; Liu, J.; and Li, H. 2022 b . Uformer: A general u-shaped transformer for image restoration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 17683--17693
2022
-
[37]
Xue, C.; Tian, Z.; Zhan, F.; Lu, S.; and Bai, S. 2022. Fourier document restoration for robust document dewarping and recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4573--4582
2022
-
[38]
Yang, M.; and Xu, S. 2023. A novel Degraded Document Binarization model through vision transformer network. Information Fusion, 93: 159--173
2023
-
[39]
Yang, Z.; Liu, B.; Xxiong, Y.; Yi, L.; Wu, G.; Tang, X.; Liu, Z.; Zhou, J.; and Zhang, X. 2023 a . DocDiff: Document enhancement via residual diffusion models. In Proceedings of the 31st ACM International Conference on Multimedia, 2795--2806
2023
-
[40]
Yang, Z.; Peng, D.; Kong, Y.; Zhang, Y.; Yao, C.; and Jin, L. 2023 b . FontDiffuser: One-Shot Font Generation via Denoising Diffusion with Multi-Scale Content Aggregation and Style Contrastive Learning. arXiv preprint arXiv:2312.12142
2023 arXiv
-
[41]
W.; Arora, A.; Khan, S.; Hayat, M.; Khan, F
Zamir, S. W.; Arora, A.; Khan, S.; Hayat, M.; Khan, F. S.; and Yang, M.-H. 2022. Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 5728--5739
2022
-
[42]
M.; and Shum, H.-Y
Zhang, H.; Li, F.; Liu, S.; Zhang, L.; Su, H.; Zhu, J.; Ni, L. M.; and Shum, H.-Y. 2022. DINO : Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605
2022 arXiv
-
[43]
A.; Shechtman, E.; and Wang, O
Zhang, R.; Isola, P.; Efros, A. A.; Shechtman, E.; and Wang, O. 2018. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, 586--595
2018
-
[44]
Zhu, J.-Y.; Park, T.; Isola, P.; and Efros, A. A. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, 2223--2232
2017
-
[45]
Zhu, S.; Xue, H.; Nie, N.; Zhu, C.; Liu, H.; and Fang, P. 2024. Reproducing the Past: A Dataset for Benchmarking Inscription Restoration. In Proceedings of the 32nd ACM International Conference on Multimedia, 7714--7723
2024
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.