pith. sign in

arxiv: 1907.08924 · v1 · pith:LTMDF4LSnew · submitted 2019-07-21 · 📡 eess.IV · cs.CV

Validation of Modulation Transfer Functions and Noise Power Spectra from Natural Scenes

Pith reviewed 2026-05-24 18:35 UTC · model grok-4.3

classification 📡 eess.IV cs.CV
keywords modulation transfer functionnoise power spectrumnatural scenesscene-dependent systemsnon-linear imagingdead leaves chartimage quality measurement
0
0 comments X

The pith

New scene-and-process-dependent MTF and NPS measures better suit non-linear imaging systems than test charts.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Traditional test charts produce inaccurate MTF and NPS values for imaging systems that apply non-linear, content-aware processing because those charts use signals unlike natural scenes. The paper validates SPD-MTF and SPD-NPS variants that measure performance for a single scene, average performance across many scenes, or the degree of scene dependency. It also shows versions derived from dead leaves charts. These measures are presented as robust and preferable for real systems that change behavior according to image content.

Core claim

SPD-MTF and SPD-NPS measures derived from natural scenes characterize system sharpness and noise either for one scene, averaged over many scenes, or the level of scene-dependency, and all proposed measures are robust and preferable for scene-dependent systems than current measures.

What carries the argument

Scene-and-process-dependent modulation transfer function (SPD-MTF) and noise power spectrum (SPD-NPS), which use natural scene signals to isolate performance in non-linear systems.

If this is right

  • System performance can be reported for specific individual scenes rather than generic test signals.
  • Average real-world performance can be estimated by averaging SPD measures across a set of natural scenes.
  • The degree of scene-dependency in a given imaging pipeline can be quantified directly.
  • Dead leaves charts can be repurposed to generate SPD-MTF and SPD-NPS without requiring new capture hardware.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Camera testing standards could move from fixed charts toward curated natural-scene libraries for more predictive quality assessment.
  • Image-processing algorithms might be tuned by minimizing measured scene-dependency rather than optimizing only average metrics.
  • The same scene-signal approach could extend to other system properties such as tone mapping or color reproduction.

Load-bearing premise

Natural scene signals provide a more representative and accurate basis for MTF/NPS measurement in non-linear systems than traditional test charts.

What would settle it

If side-by-side tests on held-out natural images show that traditional chart-based MTF/NPS predict actual captured sharpness and noise better than the SPD versions, the preference for the new measures would be refuted.

Figures

Figures reproduced from arXiv: 1907.08924 by Edward W. S. Fry, John R. Jarvis, Ralph E. Jacobson, Robin B. Jenkin, Sophie Triantaphillidou.

Figure 1
Figure 1. Figure 1: Illustration of theoretical properties of the MTF or NPS (𝐹 [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Uniform patches provide ideal input conditions for these algorithms. Thus, when captured, the former are generally less noisy than real scenes, [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 2
Figure 2. Figure 2: Noise images generated from a non-linear camera simulation pipeline at SNR 10 [3], for the following: uniform patch (left), Branca et al. [18] ‘People’ image (center) and ‘Architecture’ image (right). Noise image contrast was increased to emphasize scene-dependent noise variation. The Modulation Transfer Function (MTF) The MTF, and related Spatial Frequency Response (SFR), characterize the reproduction of … view at source ↗
Figure 3
Figure 3. Figure 3: The SPD-NPS measurement framework, adapted from [3]. The framework accounts for system noise scene-dependency. It is computationally complex, however, since many replicates must be captured (10 in this paper); using fewer replicates underestimates system noise. The framework does not account for demosaicing artefacts of a fixed pattern, [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Artefacts of fixed pattern caused by demosaicing the Branca et al. [18] ‘People’ image (left), and detail of it (right); image contrast was enhanced. We define the four proposed SPD-NPS measures below. i) The dead leaves SPD-NPS is derived from the dead leaves chart using the SPD-NPS framework ( [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The SPD-MTF measurement framework, adapted from [3] [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: 2D DFT log luminance spectra (c) and (d) for the Branca et al. [18] ‘People’ image (a) and (b), before and after windowing, respectively, with (e) [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Luminance NPSs derived from the dead lea [PITH_FULL_IMAGE:figures/full_fig_p018_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Pictorial image SPD-NPSs (grey lines), mean pictorial image SPD-NPSs (black lines), SPD￾NPS standard deviations (black dotted lines), and dead leaves SPD-NPSs (red lines) of luminance noise at different stages of processing at SNRs 40 and 5 [PITH_FULL_IMAGE:figures/full_fig_p019_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Dead leaves SPD-MTFs (red lines) and direct dead leaves MTFs (black lines) and at different stages of processing, at SNR 40 (a) – (f) and SNR 5 (g) – (l). Windowing was not applied for these measurements [PITH_FULL_IMAGE:figures/full_fig_p023_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Pictorial image SPD-MTFs (grey lines), mean pictorial image SPD-MTFs (black lines), SPD-MTF standard deviations (black dotted lines) and dead leaves SPD-MTFs (red lines), at different stages of processing at SNRs 40 and 5. All scenes and test charts were windowed for these measurements [PITH_FULL_IMAGE:figures/full_fig_p024_10.png] view at source ↗
read the original abstract

The Modulation Transfer Function (MTF) and the Noise Power Spectrum (NPS) characterize imaging system sharpness/resolution and noise, respectively. Both measures are based on linear system theory but are applied routinely to systems employing non-linear, content-aware image processing. For such systems, MTFs/NPSs are derived inaccurately from traditional test charts containing edges, sinusoids, noise or uniform tone signals, which are unrepresentative of natural scene signals. The dead leaves test chart delivers improved measurements, but still has limitations when describing the performance of scene-dependent systems. In this paper, we validate several novel scene-and-process-dependent MTF (SPD-MTF) and NPS (SPD-NPS) measures that characterize, either: i) system performance concerning one scene, or ii) average real-world performance concerning many scenes, or iii) the level of system scene-dependency. We also derive novel SPD-NPS and SPD-MTF measures using the dead leaves chart. We demonstrate that all the proposed measures are robust and preferable for scene-dependent systems than current measures.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes and validates novel scene-and-process-dependent MTF (SPD-MTF) and NPS (SPD-NPS) measures derived from natural scenes (and the dead leaves chart) to characterize imaging system performance. These measures aim to handle non-linear, content-aware processing by quantifying either single-scene performance, average real-world performance across scenes, or the degree of scene-dependency, claiming robustness and superiority over traditional test-chart-based MTF/NPS for scene-dependent systems.

Significance. If the SPD measures are shown to recover classical MTF/NPS on linear systems without residual scene variation or bias while correctly isolating scene-dependency in non-linear cases, the work would meaningfully advance evaluation of modern imaging pipelines beyond linear-system assumptions. The derivation of SPD variants from the dead leaves chart is a constructive addition if empirically supported.

major comments (2)
  1. [Validation results (likely §4 or §5)] The central claim of robustness and preferability requires that SPD-MTF/SPD-NPS recover traditional MTF/NPS (from edges/sinusoids) on linear, scene-independent systems without introducing scene-to-scene variation or bias. No section or result in the manuscript reports this necessary sanity check; without it the isolation of scene-dependency is not demonstrated to be clean.
  2. [Methods and results sections] The abstract and methods assert that the proposed measures are 'robust' for non-linear systems, yet the manuscript provides no quantitative error analysis, confidence intervals, or cross-validation against ground-truth linear-system responses to support the robustness claim.
minor comments (2)
  1. [Introduction or §2] Notation for SPD-MTF and SPD-NPS should be defined explicitly with equations early in the paper to avoid ambiguity when comparing to classical definitions.
  2. [Figures] Figure captions for any example natural scenes or dead-leaves results should include the exact imaging conditions and processing pipeline used.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments on our manuscript. We address each major comment below and indicate where revisions will be made to strengthen the validation of the SPD-MTF and SPD-NPS measures.

read point-by-point responses
  1. Referee: [Validation results (likely §4 or §5)] The central claim of robustness and preferability requires that SPD-MTF/SPD-NPS recover traditional MTF/NPS (from edges/sinusoids) on linear, scene-independent systems without introducing scene-to-scene variation or bias. No section or result in the manuscript reports this necessary sanity check; without it the isolation of scene-dependency is not demonstrated to be clean.

    Authors: We agree that a direct sanity check demonstrating recovery of classical MTF/NPS values on linear systems, with minimal scene-to-scene variation, is necessary to fully isolate and validate the scene-dependency component. Our current demonstrations focus on non-linear, content-aware processing using natural scenes and the dead leaves chart, where traditional measures are known to be inaccurate. We will add a new subsection in the results (likely §4 or §5) that applies the SPD measures to simulated linear systems (e.g., using edge and sinusoidal inputs) and reports the recovered MTF/NPS values alongside traditional calculations, including any residual variation across scenes. revision: yes

  2. Referee: [Methods and results sections] The abstract and methods assert that the proposed measures are 'robust' for non-linear systems, yet the manuscript provides no quantitative error analysis, confidence intervals, or cross-validation against ground-truth linear-system responses to support the robustness claim.

    Authors: The manuscript supports the robustness claim through comparative examples and visual/qualitative agreement with expected behavior on non-linear pipelines, but we acknowledge the absence of formal quantitative error metrics, confidence intervals, or explicit cross-validation against ground-truth linear responses. We will revise the methods and results sections to include quantitative error analysis (e.g., mean squared deviation from reference measures where applicable), bootstrap-derived confidence intervals for the SPD estimates, and cross-validation results on both linear and non-linear test cases. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation chain is self-contained

full rationale

The paper introduces novel SPD-MTF and SPD-NPS measures derived from natural scene statistics and the dead leaves chart to characterize scene-dependent performance in non-linear systems. These are presented as new constructions validated against traditional methods, with no equations or steps shown that define a quantity in terms of itself, fit parameters to a subset and relabel the output as a prediction, or rely on self-citation chains for uniqueness or ansatz smuggling. The abstract and described claims treat the measures as independent outputs from scene data, making the central validation claims externally falsifiable rather than tautological.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review provides no details on free parameters, axioms, or invented entities; none can be identified from the given text.

pith-pipeline@v0.9.0 · 5733 in / 1040 out tokens · 19748 ms · 2026-05-24T18:35:14.037505+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

19 extracted references · 19 canonical work pages

  1. [1]

    Noise image contrast was increased to emphasize scene-dependent noise variation

    ‘People’ image (center) and ‘Architecture’ image (right). Noise image contrast was increased to emphasize scene-dependent noise variation. The Modulation Transfer Function (MTF) The MTF, and related Spatial Frequency Response (SFR), characterize the reproduction of modulation (image contrast) versus spatial frequency. The 1D MTF is defined as the modulus ...

  2. [2]

    slanted-edge method is most common; it has been shown to describe well the subjective sharpness of non-linear systems [12]. However, signals from edge test charts are generally denoised less and sharpened more than natural 8 scene signals by the non-linear, content aware de-noising and sharpening algorithms incorporated in many contemporary capturing syst...

  3. [3]

    IEEE Standards Association Board of Governors,

    IEEE Std 1858TM-2016: IEEE Standard for Camera Phone Image Quality. IEEE Standards Association Board of Governors,

  4. [4]

    Evaluation of the Effect of Noise on Subjective Image Quality,

    P. G. J. Barten, “Evaluation of the Effect of Noise on Subjective Image Quality,” in Proc. SPIE 1453, Human Vision, Visual Processing, and Digital Display II, 1991, vol. 1453, pp. 2–15

  5. [5]

    International Organization for Standardization (ISO),

    International Organisation for Standardization, ISO 12233:2017: Photography — Electronic still picture imaging — resolution and spatial frequency responses. International Organization for Standardization (ISO),

  6. [6]

    Verification of a Method to Estimate the Wiener Kernel Transforms of a nonlinear system,

    P. D. Burns and J. Koplowitz, “Verification of a Method to Estimate the Wiener Kernel Transforms of a nonlinear system,” in Proc. 9th Pittsburgh Conference on Modelling and Simulation, 1978, pp. 1189–1194

  7. [7]

    Texture-based measurement of spatial frequency response using the dead leaves target: extensions, and application to real camera systems,

    J. McElvain, S. P. Campbell, J. Miller, and E. W. Jin, “Texture-based measurement of spatial frequency response using the dead leaves target: extensions, and application to real camera systems,” in Proc. SPIE 7537, Digit. Photog. VI, 2010, pp. 75370D–1

  8. [8]

    Fields of Experts: a framework for learning image priors,

    S. Roth and M. J. Black, “Fields of Experts: a framework for learning image priors,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005, p. 10.1109/CVPR.2005.160

  9. [9]

    From learning models of natural image patches to whole image restoration,

    D. Zoran and Y. Weiss, “From learning models of natural image patches to whole image restoration,” 2011 Int. Conf. Comput. Vis., pp. 479–486,

  10. [10]

    Restoration for weakly blurred and strongly noisy images,

    X. Zhu and P. Milanfar, “Restoration for weakly blurred and strongly noisy images,” 2011 IEEE Work. Appl. Comput. Vis., pp. 103–109,

  11. [11]

    Edge-preserving decompositions for multi-scale tone and detail manipulation,

    Z. Farbman, R. Fattal, D. Lischinski, and R. Szeliski, “Edge-preserving decompositions for multi-scale tone and detail manipulation,” Trans. Graph. - Proc. ACM SIGGRAPH 2008, vol. 27, no. 3, p. 1,

  12. [12]

    Available: https://support.apple.com/kb/sp705?locale=en_GB

    [Online]. Available: https://support.apple.com/kb/sp705?locale=en_GB. [Accessed: 03-Dec-2016]

  13. [13]

    Available: http://uk.mathworks.com/help/images/ref/imnoise.html

    [Online]. Available: http://uk.mathworks.com/help/images/ref/imnoise.html. [Accessed: 07-Apr-2018]

  14. [14]

    Demosaicking by alternating projections: Theory and fast one-step implementation,

    Y. M. Lu, M. Karzand, and M. Vetterli, “Demosaicking by alternating projections: Theory and fast one-step implementation,” IEEE Trans. Image Process., vol. 19, no. 8, pp. 2085–2098,

  15. [15]

    High-quality linear interpolation for demosaicing of Bayer-patterned color images,

    H. S. Malvar, L. He, and R. Cutler, “High-quality linear interpolation for demosaicing of Bayer-patterned color images,” 2004 IEEE Int. Conf. Acoust. Speech, Signal Process., vol. 3, pp. 5–8,

  16. [16]

    World Wide Web Consortium (WC3),

    ISO/IEC 15948:2003 (E): Information technology — Computer graphics and image processing — Portable Network Graphics (PNG): Functional specification., Second. World Wide Web Consortium (WC3),

  17. [17]

    Available: http://live.ece.utexas.edu/research/quality

    [Online]. Available: http://live.ece.utexas.edu/research/quality. [Accessed: 13-Jun-2017]

  18. [18]

    Perceptibility and acceptability of JPEG 2000 compressed images of various scene types,

    E. Allen, S. Triantaphillidou, and R. E. Jacobson, “Perceptibility and acceptability of JPEG 2000 compressed images of various scene types,” Proc. SPIE 9016, Image Qual. Syst. Perform. XI,

  19. [19]

    MTF and NPS Scene-Dependency in Simulated Image Capture Systems,

    E. W. S. Fry, S. Triantaphillidou, R. B. Jenkin, R. E. Jacobson, and J. R. Jarvis, “MTF and NPS Scene-Dependency in Simulated Image Capture Systems,” in IS&T International Symposium on Electronic Imaging 2020: Image Quality and System Performance XVII (ready for submission),