Enforcing Perceptual Consistency on Generative Adversarial Networks by Using the Normalised Laplacian Pyramid Distance

Alexander Hepburn; Raul Santos-Rodriguez; Ryan McConville; Valero Laparra

REVIEW 4 major objections 5 minor 19 references

Enforcing Perceptual Consistency on Generative Adversarial Networks by Using the Normalised Laplacian Pyramid Distance

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Replacing the L1 regulariser with the Normalised Laplacian Pyramid Distance (NLPD) in a cGAN objective produces more perceptually realistic generated images, as measured by segmentation accuracy, no-reference quality metrics, and human…

desk verdict A plausible drop-in perceptual loss for cGANs that is weakened by overclaimed conclusions and statistics that do not support them. read the letter →

arxiv 1908.04347 v2 pith:YIYATMAC submitted 2019-08-09 cs.CV cs.LGeess.IVstat.ML

Alexander Hepburn , Valero Laparra , Ryan McConville , Raul Santos-Rodriguez This is my paper · ORCID

classification cs.CVcs.LGeess.IVstat.ML

keywords conditionalgenerativeadversarialnetworksimage-to-imagetranslationperceptualimagequalityassessmentNormalisedLaplacianPyramidDistanceregulariserL1lossFCNsegmentationno-referencemetrics

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a way to put human perception into the training objective of conditional generative adversarial networks (cGANs) for image-to-image translation. It replaces the standard L1 reconstruction regulariser with the Normalised Laplacian Pyramid Distance (NLPD), a multi-scale, locally normalised distance derived from the early stages of the human visual system. The claim is that this swap makes generated images more perceptually realistic: on the Cityscapes, Maps, and Facades benchmarks, the NLPD-regularised model improves FCN segmentation accuracy and no-reference image quality scores, and human raters prefer its output over L1-regularised output. If true, the result matters because it shows that a principled perceptual distance can be used directly as a learning signal, rather than being reserved for evaluation.

What carries the argument

The central object is the Normalised Laplacian Pyramid Distance (NLPD), a perceptual image distance defined in the paper's foundational reference [8] and used here as a cost term. The image is filtered and downsampled through several stages; at each stage each coefficient is divided by a local estimate of the amplitude or energy, mimicking contrast normalisation in the human visual system. The distance between two images is the root-mean-square error between their transformed representations across $N$ stages. In this paper the transform is appended to the cGAN computation graph as extra convolutional layers, so its gradient flows into the generator and enforces realistic local and global contrast during training.

What would settle it

Re-run the Cityscapes training with NLPD normalisation filters recomputed from the target dataset's own statistics rather than taken from the precomputed monochromatic set; if FCN per-pixel accuracy does not rise above the L1 baseline, the transfer of the normalisation filters is the load-bearing component.

Watch

Extended reading notes

Core claim

The paper's central claim is that optimising a cGAN with the objective $L_{cGAN}(G,D) + \lambda L_{NLPD}$ instead of $L_{cGAN}(G,D) + \lambda L_{L1}$ yields images that are closer to human perception of natural images. The NLPD term computes the RMS distance between the generator output and the ground-truth image in a 'perceptual space' built from a Laplacian pyramid with local amplitude normalisation at each scale. The authors report that this regulariser produces more realistic local and global contrast, improves per-pixel accuracy and class IOU of an FCN segmentation network trained on ground-truth Cityscapes, and lowers BRISQUE and NIQE scores on Cityscapes and Maps. In a human survey asking which image looks more natural, NLPD-regularised images were preferred above chance for Maps and Cityscapes and at chance for Facades.

Load-bearing premise

The load-bearing premise is that the NLPD normalisation filters, precomputed on a monochromatic image set, remain valid for colour target datasets when used as a gradient signal during GAN training; if these filters do not transfer, the regulariser could push the generator toward artifacts instead of perceptual quality.

Editorial extensions

If this is right

On Cityscapes, FCN per-pixel accuracy rises from $0.71 \pm 0.15$ with L1 to $0.74 \pm 0.09$ with NLPD, and class IOU from $0.18 \pm 0.04$ to $0.19 \pm 0.04$, suggesting that NLPD-generated images retain more features of the original dataset.
BRISQUE and NIQE scores for the NLPD model move closer to ground-truth values on Cityscapes and Maps, indicating more natural-looking outputs according to no-reference metrics.
Human raters choose NLPD-generated images over L1-generated images in 52.37% of Maps comparisons and 56.16% of Cityscapes comparisons, while Facades is at 50.04%.
The NLPD regulariser's computational overhead is modest relative to GAN training, because the pyramid transform can be expressed as extra convolutional layers in the computation graph.
The results support the broader principle that perceptual metrics validated on human judgment can serve as training objectives, not just evaluation tools.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the NLPD regulariser is not specific to cGANs; any differentiable image-generation objective with a reconstruction term could be tested with it, including super-resolution or style transfer, although the gradient behaviour of the normalisation filters would need re-checking.
Editorial inference: the authors fix the NLPD normalisation filters from a monochromatic image set and apply them per colour channel; relearning these filters on each target dataset, or jointly with the generator, is a direct testable extension that could improve transfer.
Editorial inference: a natural extension is to test whether combining NLPD with a small L1 term preserves pixel fidelity while adding perceptual quality, since NLPD might behave differently from L1 under strong regularisation.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is the first application of NLPD as a GAN regulariser, replacing L1 in the pix2pix objective. The idea is sensible and the implementation is straightforward, but the paper's own numbers do not support the conclusion that NLPD is better across all evaluations.

What is genuinely new is narrow: NLPD was previously an image quality metric and a rendering regulariser; using it as a cGAN training loss is a new combination. The authors use standard datasets and splits, the same architecture as pix2pix, and they honestly report three different evaluation strategies. That is more than many GAN papers do.

The soft spots are real. In the FCN table every metric overlaps between L1 and NLPD when you account for standard deviation; per-pixel accuracy is 0.71±0.15 against 0.74±0.09. The BRISQUE/NIQE table has point estimates with no variance, so the Cityscapes and Maps differences could be seed noise. The Facades row is identical for both losses, and the AMT preference is 50.04% on Facades, which is chance. Yet the conclusion states 'better performance in all the evaluation procedures' and that users preferred the NLPD images. That internal inconsistency is load-bearing, because without the unqualified conclusion the evidence reduces to 'sometimes better, sometimes indistinguishable.'

I do not think the central idea is wrong. NLPD is psychophysically motivated, its normalisation weights were fit on McGill rather than the target datasets, and it has been externally validated as an IQM, so the self-citation concern is minor. But the authors need to provide error bars or significance tests, and they need to soften the conclusion to match the data. The cross-dataset transfer of the normalisation filters is also assumed rather than tested; a small ablation would help.

Who should read this: people working on perceptual losses for conditional GANs, and anyone teaching evaluation of generative models. I would send it to a competent referee, not desk-reject it; the question is answerable and the method is already public. I would not cite it as evidence of improvement until the statistical issues are addressed.

Referee Report

4 major / 5 minor

Summary. The paper proposes replacing the L1 reconstruction regularizer in the pix2pix conditional GAN objective with the Normalised Laplacian Pyramid Distance (NLPD), a perceptual metric inspired by the early human visual system. The authors evaluate this modification on three image-to-image translation datasets (Facades, Cityscapes, Maps) using FCN segmentation accuracy, the no-reference image quality metrics BRISQUE and NIQE, and an Amazon Mechanical Turk preference experiment. They report that the NLPD-regularised model achieves better results than the L1 baseline across these evaluations and conclude that the proposed objective shows better performance in all evaluation procedures.

Significance. If the empirical claim were statistically established, the paper would make a useful contribution: it demonstrates that a biologically motivated, differentiable perceptual distance can serve as a practical training loss for conditional GANs, with a computational overhead that is negligible relative to GAN training. The experimental design covers multiple datasets and standard evaluation protocols, and the motivation for moving beyond pixel-level losses is well argued. However, as presented, the quantitative evidence is not strong enough to support the central claim of consistent perceptual improvement, so the paper's current value lies mainly in the proposal and the experimental setup rather than in the demonstrated gains.

major comments (4)

[Section 4.3, Table 1] The BRISQUE and NIQE scores are reported as point estimates without any measure of variance or statistical test; the Facades row shows identical scores of 30.1 (5.2) for both methods, and the Cityscapes and Maps differences (2.1 and 1.6 BRISQUE points) could easily arise from seed-to-seed variability. Without error bars or significance tests, the claim that NLPD improves no-reference IQM scores is not established.
[Section 4.3, Table 2] The FCN segmentation metrics show overlapping standard deviations between cGAN+L1 and cGAN+NLPD: per-pixel accuracy 0.71 ± 0.15 versus 0.74 ± 0.09, per-class accuracy 0.25 ± 0.05 versus 0.25 ± 0.04, and class IOU 0.18 ± 0.04 versus 0.19 ± 0.04. Since no significance test or confidence interval is reported, the claimed improvement in segmentation accuracy is statistically indistinguishable from noise.
[Section 4.3, AMT; Section 5] The AMT preference results are 50.04% for Facades and 52.37% for Maps, both compatible with chance given 500 decisions per dataset; only Cityscapes (56.16%) is suggestive. The conclusion in Section 5 that 'the suggested objective shows better performance in all the evaluation procedures' and that 'users preferred' the NLPD images is therefore not supported by the presented numbers. This is a load-bearing internal inconsistency because the central claim of perceptual improvement rests on these evaluations.
[Section 4.1] The NLPD normalization filters are precomputed on the McGill dataset from monochromatic images, and the paper provides no analysis of whether these weights remain valid for the target datasets when the NLPD is used as a training loss under GAN optimization. A concrete test would be to compare performance with filters re-estimated on the target data or to validate the transferred metric on human-rated images from the target domain; without this evidence, the risk that the regularizer induces artifacts specific to domain mismatch remains unaddressed.

minor comments (5)

[Author byline] The author byline contains a typo: 'Santos-Rodgriguez' should likely be 'Santos-Rodriguez'.
[Section 4.3] The phrase 'above chance' should be accompanied by a confidence interval or a binomial test; as written, it is not a statistical claim.
[Table 1 caption] The caption 'BRISQUE (NIQE) Scores' is ambiguous; it should clarify that both BRISQUE and NIQE scores are reported.
[Section 4.3, Figures 1 and 2] The qualitative descriptions of texture and contrast differences between methods are not quantified; adding a small-scale quantitative comparison or a separate perceptual study would strengthen these observations.
[Section 4.1] The paper does not state whether code or trained models will be released, which would aid reproducibility of the proposed regulariser.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: NLPD is imported as an existing metric, the regularizer comparison is tested against external benchmarks, and the co-authored NLPD citation is independent prior validation rather than a load-bearing self-citation.

full rationale

The paper's claim is that replacing L1 with NLPD in a cGAN objective yields more perceptually realistic images. The derivation chain is: (1) take NLPD from Laparra et al. [8] as a fixed, pre-trained perceptual distance; (2) insert it as an additive regularizer in the standard cGAN objective (Eq. 4); (3) train pix2pix with this objective and compare with cGAN+L1. None of these steps defines the target result in terms of the input. The NLPD weights were fit on the unrelated McGill dataset, not on Facades/Cityscapes/Maps, so the regularizer is not fitted to the evaluation data. The hyperparameters lambda=15 and N=6 are set manually for comparable loss scale, not optimized against the reported metrics. The evaluation uses independent external benchmarks: FCN segmentation, BRISQUE, NIQE, and AMT human preference; no evaluation metric is the NLPD itself, and the human AMT test directly tests the perceptual claim without relying on the co-authored NLPD validation. The only author-overlap citation is [8] (Laparra is a co-author), but the paper does not derive NLPD here, and the claim that NLPD is a valid perceptual metric was externally validated in prior human-rated database experiments; moreover the present paper's AMT experiment independently tests perceived naturalness. Thus the self-citation is not load-bearing. Concerns about overlapping error bars and near-chance AMT results are statistical-evidence issues, not circularity, and do not change this verdict.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new entities. Its free parameters are the NLPD regularizer weight and pyramid stages, both chosen by hand. The main assumptions are the transferability of the precomputed NLPD normalization filters and the validity of NLPD as a perceptual loss for GAN training.

free parameters (2)

lambda for NLPD = 15
Chosen so that the NLPD loss is a similar order of magnitude to the GAN loss; not tuned per dataset. Section 4.1.
Pyramid stages N = 6
Set by the user without a sensitivity study. Section 4.1.

assumptions (3)

domain assumption NLPD is a valid perceptual distance that correlates with human judgments.
The paper relies on the external validation in [8] and does not revalidate NLPD on generated images.
domain assumption Normalisation filters optimised on the McGill dataset transfer to Facades, Cityscapes, and Maps.
Section 4.1 states the filters were found using monochromatic McGill images; the paper assumes this transfer without testing.
domain assumption Gradient descent on the composite loss (Eq. 4) is well-behaved and the chosen lambda balances the losses.
The paper selects lambda=15 by order of magnitude and does not analyze sensitivity or convergence.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Enforcing Perceptual Consistency on Generative Adversarial Networks by Using the Normalised Laplacian Pyramid Distance." pith.science (2026). https://pith.science/paper/YIYATMAC

@misc{pith2026190804347,
  author       = {Pith},
  title        = {Pith review of: Enforcing Perceptual Consistency on Generative Adversarial Networks by Using the Normalised Laplacian Pyramid Distance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YIYATMAC}},
  note         = {Machine review of arXiv:1908.04347}
}

read the original abstract

In recent years there has been a growing interest in image generation through deep learning. While an important part of the evaluation of the generated images usually involves visual inspection, the inclusion of human perception as a factor in the training process is often overlooked. In this paper we propose an alternative perceptual regulariser for image-to-image translation using conditional generative adversarial networks (cGANs). To do so automatically (avoiding visual inspection), we use the Normalised Laplacian Pyramid Distance (NLPD) to measure the perceptual similarity between the generated image and the original image. The NLPD is based on the principle of normalising the value of coefficients with respect to a local estimate of mean energy at different scales and has already been successfully tested in different experiments involving human perception. We compare this regulariser with the originally proposed L1 distance and note that when using NLPD the generated images contain more realistic values for both local and global contrast. We found that using NLPD as a regulariser improves image segmentation accuracy on generated images as well as improving two no-reference image quality metrics.

Figures

Figures reproduced from arXiv: 1908.04347 by the authors.

**Figure 2.** Images generated from the (a) Maps and (b) Facades datasets at a resolution of 256 [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 18 canonical work pages

[1]

H. B. Barlow. Possible principles underlying the transformation of sensory messages. Sensory Com- munication, pages 217–234, 1961

work page 1961
[2]

Burt and E

P. Burt and E. Adelson. The laplacian pyramid as a compact image code. IEEE Transactions on communications, 31(4):532–540, 1983. 5 (a) Maps (b) Facades Figure 2: Images generated from the (a) Maps and (b) Facades datasets at a resolution of 256 × 256

work page 1983
[3]

Cordts, M

M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. The cityscapes dataset for seman- tic urban scene understanding. In IEEE CVPR , pages 3213–3223, 2016

work page 2016
[4]

Dosovitskiy and T

A. Dosovitskiy and T. Brox. Generating images with perceptual similarity metrics based on deep networks. In NIPS, pages 658–666, 2016

work page 2016
[5]

Goodfellow, J

I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In NIPS, pages 2672–2680, 2014

work page 2014
[6]

Hertzmann, C

A. Hertzmann, C. E. Jacobs, N. Oliver, B. Curless, and D. H. Salesin. Image analogies. In ACMCGIT, pages 327–340. ACM, 2001

work page 2001
[7]

Isola, J

P. Isola, J. Zhu, T. Zhou, and A. Efros. Image- to-image translation with conditional adversarial networks. In IEEE CVPR , 2017

work page 2017
[8]

Laparra, J

V. Laparra, J. Ball´ e, A. Berardino, and E. P. Si- moncelli. Perceptual image quality assessment us- ing a normalized laplacian pyramid. Electronic Imaging, 2016(16):1–6, 2016

work page 2016

Show all 19 references

[9]

Laparra, A

V. Laparra, A. Berardino, J. Ball´ e, and E. P. Si- moncelli. Perceptually optimized image rendering. JOSA, 2017

2017
[10]

J. Long, E. Shelhamer, and T. Darrell. Fully con- volutional networks for semantic segmentation. In IEEE CVPR , pages 3431–3440, 2015

2015
[11]

Mittal, A

A. Mittal, A. K. Moorthy, and A. C. Bovik. No- reference image quality assessment in the spatial domain. IEEE TIP , 21(12):4695–4708, 2012

2012
[12]

Mittal, R

A. Mittal, R. Soundararajan, and A. C. Bovik. Making a” completely blind” image quality ana- lyzer. IEEE Signal Process. Lett. , 20(3):209–212, 2013

2013
[13]

Olmos and F

A. Olmos and F. A. A. Kingdom. A biologically inspired algorithm for the recovery of shading and reﬂectance images. Perception, 33(12):1463–1473, 2004

2004
[14]

Radford, L

A. Radford, L. Metz, and S. Chintala. Unsu- pervised representation learning with deep convo- lutional generative adversarial networks. CoRR, abs/1511.06434, 2016

2016 arXiv
[15]

Theis, Land Oord and M

A. Theis, Land Oord and M. Bethge. A note on the evaluation of generative models. ICLR, 2015

2015
[16]

Tyleˇ cek and R.ˇS´ ara

R. Tyleˇ cek and R.ˇS´ ara. Spatial pattern templates for recognition of objects with regular structure. In GCPR, pages 364–374. Springer, 2013

2013
[17]

T. Wang, M. Liu, J. Zhu, A. Tao, J. Kautz, and B. Catanzaro. High-resolution image synthesis and semantic manipulation with conditional gans. In IEEE CVPR , pages 8798–8807, 2018

2018
[18]

Z. Wang, E. P. Simoncelli, and A. C. Bovik. Mul- tiscale structural similarity for image quality as- sessment. In ACSSC, volume 2, pages 1398–1402. Ieee, 2003

2003
[19]

Zhang, P

R. Zhang, P. Isola, A. Efros, E. Shechtman, and O. Wang. The unreasonable eﬀectiveness of deep features as a perceptual metric. In IEEE CVPR , pages 586–595, 2018. 6

2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

H. B. Barlow. Possible principles underlying the transformation of sensory messages. Sensory Com- munication, pages 217–234, 1961

work page 1961

[2] [2]

Burt and E

P. Burt and E. Adelson. The laplacian pyramid as a compact image code. IEEE Transactions on communications, 31(4):532–540, 1983. 5 (a) Maps (b) Facades Figure 2: Images generated from the (a) Maps and (b) Facades datasets at a resolution of 256 × 256

work page 1983

[3] [3]

Cordts, M

M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. The cityscapes dataset for seman- tic urban scene understanding. In IEEE CVPR , pages 3213–3223, 2016

work page 2016

[4] [4]

Dosovitskiy and T

A. Dosovitskiy and T. Brox. Generating images with perceptual similarity metrics based on deep networks. In NIPS, pages 658–666, 2016

work page 2016

[5] [5]

Goodfellow, J

I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In NIPS, pages 2672–2680, 2014

work page 2014

[6] [6]

Hertzmann, C

A. Hertzmann, C. E. Jacobs, N. Oliver, B. Curless, and D. H. Salesin. Image analogies. In ACMCGIT, pages 327–340. ACM, 2001

work page 2001

[7] [7]

Isola, J

P. Isola, J. Zhu, T. Zhou, and A. Efros. Image- to-image translation with conditional adversarial networks. In IEEE CVPR , 2017

work page 2017

[8] [8]

Laparra, J

V. Laparra, J. Ball´ e, A. Berardino, and E. P. Si- moncelli. Perceptual image quality assessment us- ing a normalized laplacian pyramid. Electronic Imaging, 2016(16):1–6, 2016

work page 2016