Pith. sign in

REVIEW 3 major objections 5 minor 35 references

Image Formation Model Guided Deep Image Super-Resolution

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper shows that a deep super-resolution network can be made to obey the known blur-and-downsample image formation model by replacing, at each cascade stage, the pixel values at un-decimated positions with the corresponding…

desk verdict Solid SR paper with a real but modest empirical gain; the 'strictly satisfies' claim only holds for the assumed kernel, which does not match the benchmark degradation. read the letter →

arxiv 1908.06444 v3 pith:ZRQQSXHQ submitted 2019-08-18 eess.IV cs.CV

Jinshan Pan , Yang Liu , Deqing Sun , Jimmy Ren , Ming-Ming Cheng , Jian Yang , Jinhui Tang This is my paper · ORCID
classification eess.IVcs.CV
keywords imagesuper-resolutionformationmodelpixelsubstitutiondeepconvolutionalneuralnetworkcascadedrefinementblurkernelPSNRSSIM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a deep super-resolution network produces better high-resolution images when the known relationship between high- and low-resolution images—blur then downsample—is imposed as a hard constraint rather than learned implicitly. The constraint is applied by a pixel substitution step: the intermediate high-resolution image is blurred with a known kernel, and at the positions that survive downsampling, the blurred values are replaced by the observed low-resolution pixel values. The corrected image is then fed through the same network again, and the whole cascade is trained end-to-end. On standard benchmarks, the method reports higher PSNR/SSIM than strong baselines including EDSR, RDN, and DBPN, and it runs about three times faster than the feedback DBPN. The reason to care is that this offers an architecture-agnostic way to inject domain structure into any feed-forward SR network.

What carries the argument

The load-bearing object is the pixel substitution rule together with the selection matrix D^T D derived from the downsampling operator D. At each stage, the network estimates an intermediate HR image I; the machinery blurs it to B = KI, then overwrites B at un-decimated positions (x = ds) with the corresponding values from the upsampled LR image D^T L. This makes the updated image strictly satisfy the formation model at those pixels before the next network pass. It converts a global constraint into a local, differentiable data-injection step.

What would settle it

Take a test image, degrade it with a Gaussian kernel whose width is half the paper's setting, upsample by 4, and run the method with its default kernel: if the PSNR of the pixel-substitution result is lower than the same network run without substitution, then the method actively hurts when the kernel is slightly wrong, confirming that the reported gains are strictly conditional on kernel fidelity.

Watch

Extended reading notes

Core claim

The central discovery is that the image formation constraint L = DKI can be enforced pointwise during a deep-network SR pipeline by substituting observed low-resolution values into the blurred intermediate estimate at exactly the pixel positions that the downsampling operator keeps. Writing D^T L = D^T DKI, the matrix D^T D selects the un-decimated positions; the paper's update rule sets B̂(x) = D^T L(x) at those positions and B(x) elsewhere. Because the substituted image satisfies the equality at the selected positions by construction, the cascaded refinement has a self-consistent anchor that feed-forward regression lacks. The paper further shows that this hard substitution outperforms both an unconstrained cascade of the same network and a soft image-formation loss added to the training objective.

Load-bearing premise

The paper's whole benefit rests on the assumption that the blur kernel used in pixel substitution matches the degradation that actually produced the low-resolution image; if it does not—as the paper itself shows for JPEG-compressed inputs—the substituted pixels inject inconsistent values and can worsen artifacts.

Editorial extensions

If this is right

  • Any existing feed-forward SR architecture can be wrapped in this cascade and, according to the paper's experiments, gain reconstruction quality on standard benchmarks without changing the network's internal design.
  • The method's outputs satisfy the image formation model far more tightly than feedback baselines: regenerating LR images from its HR outputs yields PSNR around 72 dB on Set5 (×2), versus about 61 dB for DBPN.
  • The performance gain comes from the constraint itself, not from added capacity: a plain cascade of the same network without pixel substitution gives almost no improvement, and a soft image-formation loss does not consistently help.
  • The hard-substitution loop is computationally cheap relative to feedback networks, running about three times faster than DBPN and only modestly slower than the feed-forward EDSR baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test this, but the residual between the blurred network output and the observed LR at un-decimated positions is a ready-made diagnostic for kernel mismatch; plotting this residual across a test set would show where the assumed Gaussian diverges from the true degradation.
  • A natural extension the paper leaves implicit is blind SR: estimate the blur kernel from the LR image, then use it in the substitution step; the paper's fixed Gaussian would then be a special case and its reported gains a lower bound for well-estimated kernels.
  • Because the substitution rule depends only on a selection matrix and observed pixel values, the same idea should transfer to other inverse problems with known forward models, such as demosaicing, inpainting, or burst fusion, but the paper does not demonstrate this.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript proposes a cascaded single-image super-resolution (SR) framework that combines a deep feed-forward network, similar to EDSR, with a hard image-formation constraint. At each stage, the network estimates an intermediate HR image; this image is blurred by a Gaussian kernel, and the blurred image's values at the un-decimated positions are replaced by the corresponding LR values (Eq. 5), producing a substituted image that is fed to the next stage. Training is performed stage by stage with an L1 loss, and inference uses T = 3 stages. The paper reports PSNR/SSIM comparisons on Set5, Set14, B100, Urban100, Manga109, and DIV2K (validation), along with ablations, a robustness test to noise, and a self-consistency check of the formation constraint. The source code and trained models are publicly released.

Significance. The idea of enforcing the image formation model by pixel substitution inside a deep SR pipeline is simple and potentially useful; if the mechanism is confirmed, it could serve as an architecture-agnostic refinement step for existing SR networks. The paper has several strengths: it releases public code and models, evaluates on standard benchmarks, includes an ablation that separates the pixel-substitution effect from mere network cascading, and explicitly discusses limitations. However, the empirical gains over strong baselines are small (for example, +0.07 dB on Set5 at x2, +0.04 dB on DIV2K at x2, and +0.01 to +0.02 dB on B100), and the central mechanistic claim is not validated under the degradation model actually used to generate the benchmark data. The contribution is interesting but not yet fully substantiated.

major comments (3)
  1. [Section 5.1 vs. Section 3] The formation model in Eq. (1) assumes a blur kernel K, and Eq. (5) imposes the constraint L = D K I at the un-decimated positions. However, Section 5.1 states that the LR images used for training and testing are generated by bicubic downsampling, while K is chosen as a Gaussian kernel 'with the same settings used in Shan et al. 2008'. Thus the hard constraint enforced by pixel substitution does not correspond to the actual degradation of the benchmark data. To support the claim that the algorithm uses known blur kernels and that enforcing the true formation model produces the reported gains, the authors should add experiments in which the LR images are generated with the same Gaussian blur followed by downsampling (matching K), and compare those results with the bicubic-degraded setting. Without this comparison, the reported gains may reflect an interaction between the Gaussian substitution and bicubic inputs rather than correct enforcement of the formation model.
  2. [Abstract and Section 4.2, Eq. (5)] The claim that the 'output of the pixel substitution process strictly satisfies the image formation model' is stronger than what is implemented. Eq. (5) only constrains the substituted blurred intermediate image at the un-decimated positions; it does not guarantee that the final high-resolution estimate I_T, after the cascaded network refinements, satisfies the global constraint D K I = L. The conclusion in Section 7 phrases this more carefully, but the abstract and Section 4.2 should be qualified. If the intended claim is about the intermediate substituted image, that should be stated explicitly; if it is about the final output, a proof or direct verification on the final output is needed.
  3. [Section 6, Table 4] The self-consistency check in Table 4 is performed relative to the assumed Gaussian formation model, not the bicubic degradation that produced the benchmark LR images. Moreover, because pixel substitution directly places the LR values into the un-decimated positions, regenerating LR images via the same Gaussian model is partly a check of the substitution construction itself. The authors should report the formation-constraint error on the non-substituted positions and, more importantly, measure consistency with the actual bicubic degradation (for example, D_bicubic(I_T) versus L). Without these numbers, Table 4 does not establish that the estimated HR images satisfy the image formation model of the data.
minor comments (5)
  1. [Table 2] The entry 'MemNe' should be 'MemNet', and the VDSR x4 SSIM value '0.0726' is clearly a typo; the value should be around 0.883. Please correct these errors.
  2. [Section 6] The word 'downsmapling' appears in the first paragraph of Section 6 and should be 'downsampling'.
  3. [Section 5.2] The word 'featrues' appears in the comparison with DBPN and should be 'features'.
  4. [References] The reference list contains two 2008 Shan et al. entries with different co-author sets; the in-text citations do not disambiguate which one is the source of the pixel substitution idea, making attribution hard to verify.
  5. [Equation (4)] The notation x = y = ds in Eq. (4) is ambiguous because d is a vector-valued index; please use explicit coordinate notation to clarify the set of un-decimated positions.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: pixel substitution is a constructive constraint, and evaluation is against external benchmarks with standard degradation.

full rationale

The paper's derivation chain is self-contained rather than circular. The image formation model is defined in Eqs. (1)-(3), and the pixel substitution rule in Eq. (5) is an algebraic consequence of that model, not a fitted parameter masquerading as a prediction. The claim that the substituted intermediate image 'strictly satisfies the image formation model' is a construction built from the model equations, which the paper explicitly presents as the mechanism, not as an empirical discovery. The Gaussian blur kernel is a stated modeling choice adopted from Shan et al. 2008 (Section 5.1), and the benchmark LR images are generated by standard bicubic downsampling (Section 5.1), so the comparisons in Table 2 are external and not forced by the paper's own assumptions. The self-citation to Pan et al. 2018 appears only in related work and is not load-bearing for the proposed algorithm. The limitation discussion (Section 6, Figure 8) explicitly acknowledges that the assumed formation model can mismatch real degradation such as JPEG compression, which is a robustness caveat and not a circular step. Table 4 verifies consistency under the assumed Gaussian model; although this is a self-consistency check rather than a fully independent test, the evaluated output is the cascaded network result, not merely the substituted image, so it is not a definitional identity. No fitted input is renamed as a prediction, no uniqueness theorem is imported from the authors' prior work, and no known result is merely renamed. The central empirical claim is therefore supported by independent benchmark evaluation.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a small number of assumptions: the linear blur-and-downsample formation model, a known kernel that matches the degradation, and empirical convergence of the cascade. The only fitted or hand-chosen numbers that matter are the number of stages T=3 and the Gaussian kernel settings. No new physical entities are introduced.

free parameters (2)
  • Number of cascade stages T = 3
    Set empirically as a trade-off between accuracy and speed (Section 5.1). Performance changes measurably across stages (Table 3), so T is a hand-chosen parameter that affects the central result.
  • Gaussian blur kernel settings = not specified, inherited from Shan et al. 2008
    The kernel in Eq. (3) is a Gaussian chosen with settings from Shan et al. 2008 (Section 5.1), not fit to the data. It is a free modeling choice that determines which positions are treated as consistent with the formation model.
assumptions (4)
  • domain assumption The LR image is generated by a convolution with a blur kernel followed by downsampling: L = DKI.
    Invoked in Eq. (1)-(2) and used as the basis for the pixel substitution constraint.
  • domain assumption The blur kernel used in the network is known and approximates the true degradation.
    Section 5.1 uses a Gaussian kernel from Shan et al. 2008 while benchmark LR images are created by bicubic downsampling; Section 6 acknowledges the limitation when the approximation fails, e.g., JPEG compression.
  • domain assumption The cascaded pixel-substitution iteration converges to better HR estimates.
    Section 4.2 states 'Empirically, we find that the approximation scheme for image formation process converges well'; no convergence proof is given.
  • standard math D^T D is a selection matrix and the substitution (5) enforces D B_hat = L.
    Section 3 Eq. (4) defines the selection matrix; the substitution follows from the definition of D^T.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Image Formation Model Guided Deep Image Super-Resolution." pith.science (2026). https://pith.science/paper/ZRQQSXHQ

@misc{pith2026190806444,
  author       = {Pith},
  title        = {Pith review of: Image Formation Model Guided Deep Image Super-Resolution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZRQQSXHQ}},
  note         = {Machine review of arXiv:1908.06444}
}
read the original abstract

We present a simple and effective image super-resolution algorithm that imposes an image formation constraint on the deep neural networks via pixel substitution. The proposed algorithm first uses a deep neural network to estimate intermediate high-resolution images, blurs the intermediate images using known blur kernels, and then substitutes values of the pixels at the un-decimated positions with those of the corresponding pixels from the low-resolution images. The output of the pixel substitution process strictly satisfies the image formation model and is further refined by the same deep neural network in a cascaded manner. The proposed framework is trained in an end-to-end fashion and can work with existing feed-forward deep neural networks for super-resolution and converges fast in practice. Extensive experimental results show that the proposed algorithm performs favorably against state-of-the-art methods.

Figures

Figures reproduced from arXiv: 1908.06444 by the authors.

Figure 1
Figure 1. Super-resolution result (×4). Our algorithm uses the image formation of super-resolution to constrain a deep neural network via pixel substitution, which generates the images satisfying the image formation model and better re￾covers structural details. methods by large margins. However, as the SR problem is highly ill-posed, using feed-forward networks may not be sufficient to estimate the LR-to-HR mapping. In parti… view at source ↗
Figure 2
Figure 2. An overview of the proposed method. The image formation constraint is enclosed in the dotted red box, which is used [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Visual comparisons for 4× SR from the Urban100 dataset. The proposed algorithm generates much better results with fine detailed structures [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Results on real example (×4). The proposed method recovers much clearer images with better detailed structures. and use the Set5 (Bevilacqua et al. 2012) as the validation test set. Then, we evaluate the effectiveness of our algorithm when LR images are obtained with d…
Figure 6
Figure 6. Figure 6: Comparisons of the results by different back [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Effectiveness of the proposed stage-dependent al [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: The proposed algorithm is less effective when the [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 33 canonical work pages

  1. [1]

    Bevilacqua, M.; Roumy, A.; Guillemot, C.; and Alberi - Morel, M. 2012. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In BMVC , 1--10

    work page 2012

  2. [2]

    Bulat, A.; Yang, J.; and Tzimiropoulos, G. 2018. To learn image super-resolution, use a GAN to learn how to do image degradation first. In ECCV , 187--202

    work page 2018

  3. [3]

    C.; He, K.; and Tang, X

    Dong, C.; Loy, C. C.; He, K.; and Tang, X. 2014. Learning a deep convolutional network for image super-resolution. In ECCV , 184--199

    work page 2014

  4. [4]

    C.; and Tang, X

    Dong, C.; Loy, C. C.; and Tang, X. 2016. Accelerating the super-resolution convolutional neural network. In ECCV , 391--407

    work page 2016

  5. [5]

    Han, W.; Chang, S.; Liu, D.; Yu, M.; Witbrock, M.; and Huang, T. S. 2018. Image super-resolution via dual-state recurrent networks. In CVPR , 1654--1663

    work page 2018

  6. [6]

    Haris, M.; Shakhnarovich, G.; and Ukita, N. 2018. Deep back-projection networks for super-resolution. In CVPR , 1664--1673

    work page 2018

  7. [7]

    He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Deep residual learning for image recognition. In CVPR , 770--778

    work page 2016

  8. [8]

    Huang, J.-B.; Singh, A.; and Ahuja, N. 2015. Single image super-resolution from transformed self-exemplars. In CVPR , 5197--5206

    work page 2015

Show all 35 references
  1. [9]

    Irani, M., and Peleg, S. 1991. Improving resolution by image registration. CVGIP: Graphical Model and Image Processing 53(3):231--239

    1991

  2. [10]

    K.; and Lee, K

    Kim, J.; Lee, J. K.; and Lee, K. M. 2016a. Accurate image super-resolution using very deep convolutional networks. In CVPR , 1646--1654

  3. [11]

    K.; and Lee, K

    Kim, J.; Lee, J. K.; and Lee, K. M. 2016b. Deeply-recursive convolutional network for image super-resolution. In CVPR , 1637--1645

  4. [12]

    P., and Ba, J

    Kingma, D. P., and Ba, J. 2014. Adam: A method for stochastic optimization. CoRR abs/1412.6980

    2014 arXiv

  5. [13]

    Lai, W.-S.; Huang, J.-B.; Ahuja, N.; and Yang, M.-H. 2017. Deep laplacian pyramid networks for fast and accurate super-resolution. In CVPR , 624--632

    2017

  6. [14]

    P.; Tejani, A.; Totz, J.; Wang, Z.; and Shi, W

    Ledig, C.; Theis, L.; Huszar, F.; Caballero, J.; Cunningham, A.; Acosta, A.; Aitken, A. P.; Tejani, A.; Totz, J.; Wang, Z.; and Shi, W. 2017. Photo-realistic single image super-resolution using a generative adversarial network. In CVPR , 105--114

    2017

  7. [15]

    Lim, B.; Son, S.; Kim, H.; Nah, S.; and Lee, K. M. 2017. Enhanced deep residual networks for single image super-resolution. In CVPR , 1132--1140

    2017

  8. [16]

    R.; Fowlkes, C

    Martin, D. R.; Fowlkes, C. C.; Tal, D.; and Malik, J. 2001. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In ICCV , 416--425

    2001

  9. [17]

    Matsui, Y.; Ito, K.; Aramaki, Y.; Fujimoto, A.; Ogawa, T.; Yamasaki, T.; and Aizawa, K. 2017. Sketch-based manga retrieval using manga109 dataset. Multimedia Tools Appl. 76(20):21811--21838

    2017

  10. [18]

    Pan, J.; Liu, Y.; Dong, J.; Zhang, J.; Ren, J. S. J.; Tang, J.; Tai, Y.-W.; and Yang, M.-H. 2018. Physics-based generative adversarial models for image restoration and beyond. CoRR abs/1808.00605

    2018 arXiv

  11. [19]

    Sajjadi, M. S. M.; Scholkopf, B.; and Hirsch, M. 2017. Enhancenet: Single image super-resolution through automated texture synthesis. In ICCV , 4491--4500

    2017

  12. [20]

    Shan, Q.; Li, Z.; Jia, J.; and Tang, C. 2008. Fast image/video upsampling. ACM TOG 27(5):153:1--153:7

    2008

  13. [21]

    Shan, Q.; Jia, J.; and Agarwala, A. 2008. High-quality motion deblurring from a single image. ACM TOG 27(3):73:1--73:10

    2008

  14. [22]

    P.; Bishop, R.; Rueckert, D.; and Wang, Z

    Shi, W.; Caballero, J.; Huszar, F.; Totz, J.; Aitken, A. P.; Bishop, R.; Rueckert, D.; and Wang, Z. 2016. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In CVPR , 1874--1883

    2016

  15. [23]

    Tai, Y.; Yang, J.; Liu, X.; and Xu, C. 2017. Memnet: A persistent memory network for image restoration. In ICCV , 4539--4547

    2017

  16. [24]

    Tai, Y.; Yang, J.; and Liu, X. 2017. Image super-resolution via deep recursive residual network. In CVPR , 3147--3155

    2017

  17. [25]

    V.; Yang, M.; Zhang, L.; and et al

    Timofte, R.; Agustsson, E.; Gool, L. V.; Yang, M.; Zhang, L.; and et al. 2017. NTIRE 2017 challenge on single image super-resolution: Methods and results. In CVPR Workshops , 1110--1121

    2017

  18. [26]

    D.; and Gool, L

    Timofte, R.; Smet, V. D.; and Gool, L. J. V. 2014. A+: adjusted anchored neighborhood regression for fast super-resolution. In ACCV , 111--126

    2014

  19. [27]

    Tong, T.; Li, G.; Liu, X.; and Gao, Q. 2017. Image super-resolution using dense skip connections. In CVPR , 4799--4807

    2017

  20. [28]

    Wang, Z.; Liu, D.; Yang, J.; Han, W.; and Huang, T. 2015. Deep networks for image super-resolution with sparse prior. In CVPR , 370--378

    2015

  21. [29]

    Yang, C.-Y.; Ma, C.; and Yang, M. 2014. Single-image super-resolution: A benchmark. In ECCV , 372--386

    2014

  22. [30]

    Zeyde, R.; Elad, M.; and Protter, M. 2010. On single image scale-up using sparse-representations. In The 7th International Conference on Curves and Surfaces , 711--730

    2010

  23. [31]

    Zhang, K.; Zuo, W.; Gu, S.; and Zhang, L. 2017. Learning deep CNN denoiser prior for image restoration. In CVPR , 2808--2817

    2017

  24. [32]

    Zhang, Y.; Li, K.; Li, K.; Wang, L.; Zhong, B.; and Fu, Y. 2018a. Image super-resolution using very deep residual channel attention networks. In ECCV , 286--301

  25. [33]

    Zhang, Y.; Li, K.; Li, K.; Wang, L.; Zhong, B.; and Fu, Y. 2018b. Image super-resolution using very deep residual channel attention networks. In ECCV , 294--310

  26. [34]

    Zhang, Y.; Tian, Y.; Kong, Y.; Zhong, B.; and Fu, Y. 2018c. Residual dense network for image super-resolution. In CVPR , 2472--2481

  27. [35]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.