REVIEW 3 major objections 5 minor 46 references
Editing Text in the Wild
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A style retention network decomposes scene text editing into conversion, inpainting, and fusion.
desk verdict First real word-level scene text editing network with a sensible modular decomposition, but the "in the wild" realism claim outruns the evidence: synthetic-only quantitative metrics and legibility-only real-world tests. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the style retention network (SRNet), a generator decomposed into three sub-networks whose modular separation is itself the main design mechanism: a text conversion module transfers foreground style while a skeleton response block, trained with dice loss, preserves the target glyph structure; a background inpainting module erases text and restores texture using U-Net-style skip connections and an adversarial discriminator; and a fusion module reconnects the background decoder features to the fusion decoder so that restored texture is carried into the final output. Two PatchGAN discriminators provide adversarial supervision at the background and fusion stages, and a VGG-based perceptual and style loss regularizes the fusion output. The claim carried by this machinery is that decomposing the task into style transfer, erasure, and fusion makes each subproblem learnable enough that synthetic-only training generalizes to real scene text.
What would settle it
Take a set of real signboard photos containing ornate fonts, strong shadows, and perspective distortion, run the published SRNet on them, and inspect whether the original word's shadow remains visible or the replacement glyphs bend and break. If a substantial fraction of such images keeps ghost strokes or distorted characters, the claim that synthetic-only training produces realistic word-level edits in the wild is falsified.
Extended reading notes
Core claim
SRNet takes a source style image and a rendered target text image and produces an edited image in which the target word appears with the source word's visual style and the original background is intact. The text conversion module transfers style under a skeleton-guided loss that keeps glyph structure readable; the background inpainting module, built with U-Net skip connections and a PatchGAN discriminator, erases the original strokes and fills them with plausible texture; the fusion module combines both streams using adversarial loss plus VGG perceptual and style losses. On synthetic test data the network reaches 0.014 L2 error, 21.12 PSNR, and 0.79 SSIM, all better than a pix2pix baseline, and on ICDAR 2013 real images it reaches 0.827 sequence recognition accuracy. Trained only on synthetic data, the same model also performs word-level text erasure (lowering text detection F-measure to 4.64 percent) and English-to-Chinese editing, which the authors present as evidence that the decomposition transfers to real-world conditions.
Load-bearing premise
The load-bearing premise is that synthetic training pairs—text rendered in random fonts and colors onto background photos—are representative enough of real street text that a network trained only on them will edit real-world images convincingly.
Editorial extensions
If this is right
- Word-level text editing becomes an end-to-end generative task solvable by modular decomposition, rather than a per-character replacement problem.
- The background inpainting branch yields word-level text erasure as a byproduct, so one network serves both editing and selective text removal.
- Cross-language editing, demonstrated from English to Chinese, becomes feasible when fonts are mapped by stroke similarity.
- The network handles target words whose lengths differ from the original, because the conversion module adaptively places the new glyphs.
- Quantitative evaluation on real data can proceed through recognition accuracy and text-detection drop, giving unpaired real images a proxy for visual fidelity.
Reading between the lines
- If the modular decomposition is what carries the performance, the text conversion module could be replaced by a stronger few-shot font transfer model without retraining the fusion and background modules, since those see only the converted foreground and inpainted background.
- The failure cases the authors report—remaining shadows, complex spatial structures, and untransferred borders—suggest that extending the synthetic training distribution with shadowed, perspective-distorted, and ornate fonts is a direct way to test how far the central claim scales.
- The same erasure capability could be used as a pre-processing step for privacy redaction or for generating clean training data for text detectors, since it removes selected words rather than entire images.
- Recognition accuracy as a metric rewards legibility more than style identity, so future benchmarks may need paired human or perceptual judgments to verify that 'realistic look' is fully captured.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SRNet, an end-to-end trainable network for word-level scene text editing. Given a source image containing text and a target text string, the network aims to replace the text while preserving the original text style and background texture. The architecture decomposes the task into three modules: a skeleton-guided text conversion module that transfers the source text style to the target text, a background inpainting module that erases the original text and fills the region, and a fusion module that combines the foreground and background outputs. Training is done on synthetic image pairs generated with an improved Gupta et al. pipeline; evaluation is performed on the synthetic test set with MSE, PSNR, and SSIM, and on real ICDAR 2013 images with sequence recognition accuracy plus qualitative examples. The paper also reports ablations (removing skeleton guidance, removing the decomposition, removing the VGG loss) and comparisons with pix2pix, and demonstrates applications such as cross-language editing and word-level text erasure.
Significance. If the central claim holds, this is a useful contribution: it is an early end-to-end learnable approach to word-level scene text editing, and the modular decomposition with skeleton supervision is a sensible design that the ablations suggest is beneficial. The authors provide machine-checked-style details of the loss functions and training protocol, and the qualitative results on ICDAR 2013 illustrate plausible edited images. However, the significance is currently limited by the evidence gap between the stated goal of producing visually indistinguishable real-world edits and the reported experiments, which mainly measure pixel similarity on synthetic data and legibility on real data. The paper also positions itself as the first word-level editor, but does not quantitatively compare against the closest prior work, STEFANN, which weakens the novelty claim.
major comments (3)
- [Sec. 4.3, Eq. (11), Table 1] The central claim is that SRNet edits natural scene text while preserving text style and background texture so that the result is visually indistinguishable from the source. However, the only real-world quantitative metric is sequence recognition accuracy, which measures whether the target string is legible, not whether font, stroke, color, shadow, perspective, or background texture are preserved. The MSE, PSNR, and SSIM numbers in Table 1 are computed only on the synthetic test set, which is generated by the same pipeline as the training set (Sec. 4.1), so they do not provide evidence about transfer to real scenes. The qualitative figures are selected examples and Sec. 4.8 acknowledges failures on complex structures and rare fonts. To support the in-the-wild realism claim, the paper should include a perceptual study on real images (e.g., human raters comparing style and background preservation) or a realism-oriented metric measured on real data.
- [Table 1 and Sec. 4.4] No error bars, variances, or significance tests are reported for any quantitative result. The differences between SRNet (ℓ2=0.014, PSNR=21.12, SSIM=0.79) and the ablations (e.g., without skeleton: 0.025, 20.08, 0.64; without VGG loss: 0.022, 20.39, 0.74) are reported as single-run numbers on a 500-image synthetic test set. Without run-to-run statistics, the claimed benefits of the skeleton module, the decomposition, and the VGG loss are not established beyond a single run. Please report multiple seeds with standard deviations and, where feasible, significance tests, or state plainly that these are single-run results.
- [Sec. 4.5 and Sec. 2.3] The comparison with prior work is limited to pix2pix. The most directly related prior work, STEFANN [24], is described as character-level and as ignoring background consistency, but no quantitative comparison is provided. Given that the paper claims to be the first word-level scene text editor, this positioning needs stronger support: either run the proposed method and STEFANN on the same protocol, or provide a more detailed justification of why a quantitative comparison is not feasible. As it stands, the novelty claim is not adequately validated against the closest existing method.
minor comments (5)
- [Eq. (2)] The word 'pixell' is a typo; it should be 'pixels'. Also, N is the number of pixels, not the number of pixel 'l', and the dice loss index i should be defined over pixels.
- [Eq. (4)] The notation E(Tb, Is)[log DB(Tb, Is)] is non-standard and should be written as E_{Tb,Is}[log DB(Tb, Is)]; the second term EIs log[1-DB(Ob, Is)] similarly needs bracket placement for clarity.
- [Sec. 3.5] The description of inference says the network is applied to patches cropped according to bounding box annotations and the results are pasted back. This implies that ground-truth bounding boxes are required at test time; please clarify whether this is a limitation and how the method would be used without them.
- [Sec. 4.3] There is a typo: 'PSNR, which computes the the ratio of peak signal to noise' should read 'the ratio'.
- [Sec. 4.1] The sentence 'We synthesize the image pairs with similar style except text as our training data' is grammatically ambiguous; it should say 'similar style but different text' or similar.
Circularity Check
No circularity found: the central claim is an empirical architecture result trained on synthetic pairs and tested on external real-world data.
full rationale
SRNet's claimed contribution is an end-to-end network for replacing text in natural images while retaining text style and background texture. The derivation chain is empirical rather than analytic: the model is trained with supervised image pairs synthesized by an improved version of Gupta et al.'s pipeline, with explicit ground-truth background, foreground text, and skeleton maps, and the losses (L1, adversarial, VGG perceptual/style, skeleton dice loss) supervise those intermediate predictions directly. The central real-world claim is evaluated on ICDAR 2013 test crops, and the paper states explicitly, 'we only train our model on synthetic data, and all real-world data is used for testing only.' The MSE/PSNR/SSIM scores in Table 1 are computed on a synthetic test set drawn from the same generation pipeline, so those numbers partly measure self-consistency rather than transfer to natural scenes; that is an evaluation limitation, not a derivation-level circularity, because no parameter is fitted on real data and then renamed as a prediction. References to prior work by overlapping authors (e.g., the Aster recognizer used for seq_acc) are standard tools and are not load-bearing justifications of the architecture. I therefore cannot exhibit any specific equation or construction in which a claimed prediction reduces to its inputs, and the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (5)
- alpha (skeleton loss weight) =
1.0
- beta (background L1 weight) =
10
- theta1 (fusion L1 weight) =
10
- theta2 (perceptual loss weight) =
1
- theta3 (style loss weight) =
500
assumptions (4)
- domain assumption Synthetic data generated by the improved text synthesis method of Gupta et al. is representative of natural scene text.
- domain assumption Recognition accuracy on ICDAR 2013 is a valid proxy for text editing quality when paired ground truth is unavailable.
- domain assumption Ground truth foreground, background, and skeleton maps can be cleanly separated from synthetic text images.
- domain assumption The adversarial training procedure converges to a stable solution with the stated losses and spectral normalization.
Cite this review
Pith. "Pith review of Editing Text in the Wild." pith.science (2026). https://pith.science/paper/HGXNZCNS
@misc{pith2026190803047,
author = {Pith},
title = {Pith review of: Editing Text in the Wild},
year = {2026},
howpublished = {\url{https://pith.science/paper/HGXNZCNS}},
note = {Machine review of arXiv:1908.03047}
}
read the original abstract
In this paper, we are interested in editing text in natural images, which aims to replace or modify a word in the source image with another one while maintaining its realistic look. This task is challenging, as the styles of both background and text need to be preserved so that the edited image is visually indistinguishable from the source image. Specifically, we propose an end-to-end trainable style retention network (SRNet) that consists of three modules: text conversion module, background inpainting module and fusion module. The text conversion module changes the text content of the source image into the target text while keeping the original text style. The background inpainting module erases the original text, and fills the text region with appropriate texture. The fusion module combines the information from the two former modules, and generates the edited text images. To our knowledge, this work is the first attempt to edit text in natural images at the word level. Both visual effects and quantitative results on synthetic and real-world dataset (ICDAR 2013) fully confirm the importance and necessity of modular decomposition. We also conduct extensive experiments to validate the usefulness of our method in various real-world applications such as text image synthesis, augmented reality (AR) translation, information hiding, etc.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[24]
Prasun Roy, Saumik Bhattacharya, Subhankar Ghosh, and Umapada Pal. 2019. STEFANN: Scene Text Editor using Font Adaptive Neural Network.arXiv preprint arXiv:1903.01192 (2019)
work page Pith review arXiv 2019
-
[1]
Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. 2016. Learning to Compose Neural Networks for Question Answering. In NAACL-HLT. 1545– 1554
work page 2016
-
[2]
Samaneh Azadi, Matthew Fisher, Vladimir G Kim, Zhaowen Wang, Eli Shechtman, and Trevor Darrell. 2018. Multi-content gan for few-shot font style transfer. In CVPR. 7564–7573
work page 2018
-
[3]
Guha Balakrishnan, Amy Zhao, Adrian V Dalca, Fredo Durand, and John Guttag
-
[4]
Shancheng Fang, Hongtao Xie, Zheng-Jun Zha, Nannan Sun, Jianlong Tan, and Yongdong Zhang. 2018. Attention and Language Ensemble for Scene Text Recogni- tion with Convolutional Sequence Modeling. InACM Multimedia. ACM, 248–256
work page 2018
-
[5]
Victor Fragoso, Steffen Gauglitz, Shane Zamora, Jim Kleban, and Matthew Turk
-
[6]
Leon A Gatys, Alexander S Ecker, and Matthias Bethge. 2016. Image style transfer using convolutional neural networks. In CVPR. 2414–2423
work page 2016
-
[7]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In NeurIPS. 2672–2680
work page 2014
Show all 46 references
-
[8]
Ankush Gupta, Andrea Vedaldi, and Andrew Zisserman. 2016. Synthetic data for text localisation in natural images. In CVPR. 2315–2324
2016
-
[9]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR. 770–778
2016
-
[10]
Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In ICML. 448–456
2015
-
[11]
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to- image translation with conditional adversarial networks. In CVPR. 1125–1134
2017
-
[12]
Jaderberg, K
M. Jaderberg, K. Simonyan, A. Vedaldi, and A. Zisserman. 2014. Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition.arXiv preprint arXiv:1406.2227 (2014)
2014 arXiv
-
[13]
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. In ECCV. Springer, 694–711
2016
-
[14]
Dimosthenis Karatzas, Faisal Shafait, Seiichi Uchida, Masakazu Iwamura, Lluis Gomez i Bigorda, Sergi Robles Mestre, Joan Mas, David Fernandez Mota, Jon Almazan Almazan, and Lluis Pere De Las Heras. 2013. ICDAR 2013 robust reading competition. In ICDAR. IEEE, 1484–1493
2013
-
[15]
Diederik P Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Opti- mization. In ICLR. 13
2015
-
[16]
Shangbang Long, Xin He, and Cong Yao. 2018. Scene Text Detection and Recog- nition: The Deep Learning Era. arXiv preprint arXiv:1811.04256 (2018)
2018 arXiv
-
[17]
Pengyuan Lyu, Xiang Bai, Cong Yao, Zhen Zhu, Tengteng Huang, and Wenyu Liu. 2017. Auto-encoder guided gan for chinese calligraphy synthesis. In ICDAR, Vol. 1. IEEE, 1095–1100
2017
-
[18]
Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. 2016. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In IC3DV. IEEE, 565–571
2016
-
[19]
Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)
2014 arXiv
-
[20]
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. 2018. Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957 (2018)
2018 arXiv
-
[21]
Toshiki Nakamura, Anna Zhu, Keiji Yanai, and Seiichi Uchida. 2017. Scene text eraser. In ICDAR, Vol. 1. IEEE, 832–837
2017
-
[22]
Alec Radford, Luke Metz, and Soumith Chintala. 2016. Unsupervised repre- sentation learning with deep convolutional generative adversarial networks. In ICLR
2016
-
[23]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In MICCAI. Springer, 234–241
2015
-
[25]
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al
-
[26]
Baoguang Shi, Xiang Bai, and Cong Yao. 2017. An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE TPAMI 39, 11 (2017), 2298–2304
2017
-
[27]
Baoguang Shi, Mingkun Yang, Xinggang Wang, Pengyuan Lyu, Cong Yao, and Xiang Bai. 2018. Aster: An attentional scene text recognizer with flexible rectifi- cation. IEEE TPAMI (2018)
2018
-
[28]
Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. In ICLR
2015
-
[29]
Danyang Sun, Tongzheng Ren, Chongxun Li, Hang Su, and Jun Zhu. 2017. Learn- ing to write stylized chinese characters by reading a handful of examples. IJCAI (2017)
2017
-
[30]
Zhou Wang, Alan C Bovik, Hamid R Sheikh, Eero P Simoncelli, et al. 2004. Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing 13, 4 (2004), 600–612
2004
-
[31]
Shuai Yang, Jiaying Liu, Zhouhui Lian, and Zongming Guo. 2017. Awesome typography: Statistics-based text effects transfer. In NeurIPS. 7464–7473
2017
-
[32]
Shuai Yang, Jiaying Liu, Wenjing Wang, and Zongming Guo. 2019. Tet-gan: Text effects transfer via stylization and destylization. In AAAI, Vol. 33. 1238–1245
2019
-
[33]
Shuai Yang, Jiaying Liu, Wenhan Yang, and Zongming Guo. 2018. Context-Aware Unsupervised Text Stylization. In ACM Multimedia. ACM, 1688–1696
2018
-
[34]
Chengquan Zhang, Borong Liang, Zuming Huang, Mengyi En, Junyu Han, Errui Ding, and Xinghao Ding. 2019. Look More Than Once: An Accurate Detector for Text of Arbitrary Shapes. In CVPR. 10552–10561
2019
-
[35]
Shuaitao Zhang, Yuliang Liu, Lianwen Jin, Yaoxiong Huang, and Songxuan Lai
-
[36]
TY Zhang and Ching Y Suen. 1984. A fast parallel algorithm for thinning digital patterns. Commun. ACM 27, 3 (1984), 236–239
1984
-
[37]
Yexun Zhang, Ya Zhang, and Wenbin Cai. 2018. Separating style and content for generalized style transfer. In CVPR. 8447–8455
2018
-
[38]
Zheng Zhang, Chengquan Zhang, Wei Shen, Cong Yao, Wenyu Liu, and Xiang Bai. 2016. Multi-oriented text detection with fully convolutional networks. In CVPR. 4159–4167
2016
-
[39]
Xinyu Zhou, Cong Yao, He Wen, Yuzhi Wang, Shuchang Zhou, Weiran He, and Jiajun Liang. 2017. EAST: an efficient and accurate scene text detector. In CVPR. 5551–5560
2017
-
[40]
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In ICCV. 2223–2232
2017
-
[41]
Zhen Zhu, Tengteng Huang, Baoguang Shi, Miao Yu, Bofei Wang, and Xiang Bai
-
[46]
Progressive Pose Attention Transfer for Person Image Generation. InCVPR. 2347–2356
-
[2011]
In W ACV
TranslatAR: A mobile augmented reality translator. In W ACV. IEEE, 497– 502
-
[2015]
IJCV 3, 115 (2015), 211–252
ImageNet Large Scale Visual Recognition Challenge. IJCV 3, 115 (2015), 211–252
2015
-
[2018]
Synthesizing images of humans in unseen poses. In CVPR. 8340–8348
-
[2019]
In AAAI, Vol
Ensnet: Ensconce text in the wild. In AAAI, Vol. 33. 801–808
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.