REVIEW 5 major objections 4 minor 22 references
Spatial Flow-Field Approximation Using Few Thermodynamic Measurements Part II: Uncertainty Assessments
T0 review · 5 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read The paper claims that the standard sampling-uncertainty metric overstates engine temperature uncertainty by a factor of about three, and proposes two propagation-based metrics—spatial sampling uncertainty and measurement imprecision—to…
desk verdict The motivating observation is solid and the Gaussian propagation is textbook, but the two new uncertainty metrics are not consistently defined and the paper's core claim about rigorous derivation does not hold in its current form. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the multivariate harmonic least-squares model $T(r,\theta) = v^T(r) U X^T a(\theta)$, with Fourier matrix $A$ encoding the chosen harmonic pair; uncertainty is propagated by treating the measurements as $B \sim \mathcal{N}(\mu_B, \Sigma_B)$ and pushing the covariance through the pseudoinverse $P = (A^T A)^{-1} A^T$ into coefficient covariance $\Sigma_X = (I_M \otimes P)\Sigma_B (I_M \otimes P)^T$ and field covariance $\Sigma_F$. The load-bearing identity is that, when $\Sigma_B = \sigma_b^2 I$, the quantity $NM\,\epsilon_p^2/\sigma_b^2$ follows a non-central chi-square distribution with degrees of freedom $g = \text{rank}(\Sigma_R)$ and non-centrality parameter $\phi = \text{vec}(\mu_R)^T \Sigma_R^- \text{vec}(\mu_R)$; equations (19)–(21) give the mean and variance. This identity converts the vague notion of “sampling uncertainty” into a computable variance, and yields the two proposed metrics: $\epsilon_p^2$ for sampling and $\epsilon_m^2 = \mu(\epsilon_p^2) - \epsilon_p^2$ for measurement imprecision.
What would settle it
On a real engine extract with a known temperature field containing harmonics beyond the chosen pair, compute $\epsilon_p^2$ from equation (25) while also measuring the true reconstruction error against a dense traverse; if $\epsilon_p^2$ stays near zero while the reconstruction error is large, the metric is not actually capturing sampling deficiency.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that sampling uncertainty cannot be defined without a spatial model: the same perfectly captured single-harmonic pattern yields standard-deviation sampling uncertainties of 0.64 K, 0.91 K, and nonzero even with 300 rakes, so the conventional metric tracks the non-uniformity of the field rather than the adequacy of the sample. The paper therefore defines spatial sampling uncertainty as $\epsilon_p^2 = \frac{1}{NM-1}\|AX-\mu_B\|_2^2$, the squared norm of the residual between the harmonic model evaluated at the chosen coefficients and the measured mean, and measurement imprecision uncertainty as the difference between the total expected squared error and that sampling term. It then proves that under Gaussian uncorrelated noise the total error is a non-central chi-square variable, so both the mean and variance of $\epsilon_p^2$ have closed forms. Numerical experiments on engine extracts show that temperature measurement uncertainty dominates rake-position uncertainty, and that correlated measurement chains reduce spatial temperature uncertainty compared with uncorrelated ones; the standard root-sum-square sampling uncertainty is about three times larger than the proposed framework's bound.
Load-bearing premise
The load-bearing premise is that the measurement noise is Gaussian with equal variance across probes and no correlations, and that the chosen harmonic pair is the true model that generated the data; if either fails, the analytical chi-square expressions and the clean interpretation of $\epsilon_p^2$ as pure sampling uncertainty no longer hold.
Editorial extensions
If this is right
- For a fixed harmonic pair and uncorrelated Gaussian probe noise, an engineer can compute the mean and variance of the spatial reconstruction error in closed form, without Monte Carlo.
- The proposed $\epsilon_p^2$ will be large when the harmonic frequencies are poorly chosen or too few, giving a direct, model-based criterion for adding rakes or harmonics.
- Because measurement uncertainty dominates probe-position uncertainty, effort spent improving probe calibration and characterizing correlations should reduce spatial temperature uncertainty more than tightening rake positioning.
- Correlated temperature measurement chains shrink the spatial uncertainty compared with independent probes, so calibration strategy becomes part of the uncertainty budget.
- The conventional standard-deviation sampling uncertainty should not be used as a standalone bound; the paper shows it can overstate uncertainty by roughly a factor of three on a four-harmonic synthetic temperature profile.
Reading between the lines
- If the harmonic model is misspecified—the real field contains harmonics not in $A$—then $\epsilon_p^2$ will absorb the resulting bias, so the “sampling uncertainty” label conflates sampling error with model error; a separate model-bias diagnostic would be needed.
- The same quadratic-form machinery could be applied with a whitening transformation to handle correlated Gaussian noise analytically, rather than falling back to Monte Carlo, at the cost of introducing the inverse covariance matrix into the chi-square parameters.
- The finding that positive correlations reduce spatial temperature uncertainty mirrors the efficiency-level result the paper reproduces from its reference [16], suggesting a general principle: common-mode calibration error moves uncertainty from the spatial pattern to the overall level, which matters for absolute temperature but not for pattern shape.
- A natural testable extension is to compute $\epsilon_p^2$ and $\epsilon_m^2$ on a rig where a dense traverse gives the truth, and compare the proposed sampling metric against actual reconstruction error—this would validate whether the metric can serve as a stopping rule for rake count.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper develops a frequentist uncertainty propagation framework for the multivariate linear least-squares temperature-field model introduced in Part I. Measurements are treated as Gaussian with known mean and covariance, probe positions may also be uncertain, and the paper derives moments for the squared residual and for area-averaged predictions. On this basis it proposes two new metrics: a 'spatial sampling uncertainty' epsilon_p^2 and a 'measurement imprecision uncertainty' epsilon_m^2. The numerical sections apply the framework to engine extracts, compare correlated and uncorrelated measurement noise, propagate rake-position uncertainty by Monte Carlo, and argue that the standard AGARD/PTC sampling-uncertainty formula overstates the uncertainty for harmonic flow fields. The paper concludes that measurement imprecision, rather than probe placement, dominates the uncertainty in the reconstructed field.
Significance. The paper addresses a practically important problem: how to decide whether a few rakes suffice to characterize a circumferentially non-uniform temperature field. It gives several useful pieces: the explicit propagation of Gaussian measurement covariance through the pseudoinverse, formulas for the variance of area-averaged temperature, a clear demonstration that the standard sampling-uncertainty formula (1) is misleading for harmonic fields, and a Monte Carlo scheme for probe-position uncertainty. If the proposed metrics were valid, they would be a useful addition to turbomachinery test practice. However, the central metric definitions are internally inconsistent: epsilon_p^2 is defined differently in Eq. (18) and Eq. (25), the variance formula in Eq. (21) is dimensionally wrong, and epsilon_m^2 in Eq. (26) is not a well-defined variance. These are not presentation issues; they invalidate the two headline contributions as stated.
major comments (5)
- [Sec. 4.3, Eqs. (18) and (25)] The symbol epsilon_p^2 is defined twice with different meanings. Equation (18) defines epsilon_p^2 = (1/NM)||AX-B||_2^2 with B a Gaussian random matrix, so epsilon_p^2 is a random variable whose moments are then derived in Eqs. (19)-(21). Equation (25) defines epsilon_p^2 = (1/(NM-1))||AX-mu_B||_2^2, a deterministic quantity computed from the mean data, with a different denominator and with B replaced by mu_B. Both objects are called the spatial sampling uncertainty and both are used in Eqs. (26)-(27). The moment formulas of Sec. 4.1 apply to the Eq. (18) object, while the metric used in Table 2 and Algorithm 1 is the Eq. (25) object. The paper never reconciles these two definitions, so the claim that the sampling-uncertainty metric is rigorously derived is not supported.
- [Sec. 4.3, Eq. (26)] The measurement imprecision metric epsilon_m^2 = mu(epsilon_p^2) - epsilon_p^2 is not a valid variance. If epsilon_p^2 is the random variable of Eq. (18), then E[epsilon_m^2] = 0 and the quantity carries no information about uncertainty. If epsilon_p^2 is instead the deterministic residual of Eq. (25), then epsilon_m^2 is a difference between an expectation and a realized residual, and it is not guaranteed to be nonnegative. A quantity proposed as a variance that can be negative is inadmissible, and the paper provides no other statistical interpretation for epsilon_m^2.
- [Sec. 4.1, Eq. (21)] Equation (21) is dimensionally inconsistent. The quantity epsilon_p^2 is a squared temperature residual and therefore has units K^2; its variance sigma^2(epsilon_p^2) must have units K^4. The right-hand side as printed, (sigma_b^2/(NM))(2g+4phi), has units K^2 because sigma_b^2 has units K^2 and g, phi, and NM are dimensionless. The correct expression for the variance of (1/NM)||AX-B||^2 under B ~ N(mu_B, sigma_b^2 I) is sigma_b^4/(NM)^2(2g+4phi). Consequently the values in Table 2, described as sigma^2(epsilon_p^2), cannot be the values obtained from Eq. (21) as written, and the numerical results in Sec. 5 do not validate the printed formula.
- [Sec. 4.3, Eq. (27)] The limit in Eq. (27) does not follow from the preceding definitions. Under Eq. (21), lim_{sigma_b->0} mu(epsilon_p^2) = ||(I-H)mu_B||^2/(NM), where H = AP is the hat matrix. Under Eq. (25), epsilon_p^2 = ||AX-mu_B||^2/(NM-1) = ||(I-H)mu_B||^2/(NM-1). The difference is -||(I-H)mu_B||^2/[NM(NM-1)], which is zero only if the harmonic model interpolates the mean data. Thus the stated claim that epsilon_m^2 vanishes as sigma_b -> 0 is generally false.
- [Sec. 5.2 and Algorithm 1] The reported accuracy of the selected harmonic pairs is partly a selection artifact. Algorithm 1 searches over harmonic pairs and returns those that minimize mu(epsilon_p^2) evaluated on the same data used to fit the model, and Eq. (25) defines epsilon_p^2 as the in-sample least-squares residual. In-sample residuals are biased downward as estimates of out-of-sample error, and Sec. 5.1 explicitly states that no distinction is made between testing and training data. In addition, when the Fourier model is misspecified, the residual ||AX-mu_B||^2 contains model bias as well as sampling variation, so the proposed 'spatial sampling uncertainty' is not a pure sampling uncertainty. The paper does not discuss either of these limitations when presenting the metric in Sec. 4.3.
minor comments (4)
- [Sec. 2.1, Eq. (6)] The variance expansion in Eq. (6) appears to contain a factor error: the printed expression '= 2 sum_i sum_j ... cov' gives twice the correct total variance and mixes the variance and covariance terms incorrectly; the standard form is sum_i (partial eta/partial z_i)^2 sigma_i^2 + 2 sum_{i<j} (partial eta/partial z_i)(partial eta/partial z_j) cov(z_i,z_j).
- [Algorithms 1 and 2] Both algorithms use a while-loop condition ||Xhat||_2 >= beta, but beta is never defined, and lambda is set to a vector on line 4 while line 9 references lambda_i without making the loop index explicit. This makes the regularization procedure difficult to reproduce.
- [Fig. 3 caption and Sec. 2.3] The text refers to 'Pearson rank correlation', but Pearson correlation is not a rank correlation; the authors should use either 'Pearson correlation' or 'Spearman rank correlation' consistently.
- [Eqs. (1) and (30)] The symbol K denotes the number of probes in Eq. (1) and then the total number of measurements NM = 42 in Eq. (30). Reusing K for different quantities in the same uncertainty discussion is confusing and should be fixed.
Circularity Check
The two new uncertainty metrics are not rigorously derived: the measurement-imprecision metric mixes two incompatible definitions of eps_p^2, and the spatial-sampling metric is minimized in-sample by the harmonic selection algorithm.
-
self definitional
[Section 4.3, Eqs. (18), (21), (25)-(27)]
"ε²_p = 1/NM‖AX−B‖²_2, (18) ... µ(ε²_p) = σ_b²/NM (g + φ), (21) ... ε²_p = 1/NM− 1‖AX−µB‖²_2, (25) ... ε²_m = µ(ε²_p) − ε²_p, (26) ... limit_{σb→0} µ(ε²_p) = ε²_p, (27)"
Eq. (18) defines ε_p² as the random variable (1/NM)||AX−B||², and Eq. (21) gives its expectation under B∼N(µ_B,σ_b²I). Eq. (25) reuses the same symbol for the deterministic residual (1/(NM−1))||AX−µ_B||². Eq. (26) then subtracts the latter from the former's expectation, so ε_m² is not a well-defined variance of any single quantity. If ε_p² is the Eq.-(18) random variable, E[ε_m²]=0, so it cannot measure imprecision. If ε_p² is the Eq.-(25) object, then E[ε_m²]=σ_b²(2k+1)/(NM−1)+||(I−H)µ_B||²/(NM−1) − [σ_b²(NM−2k−1)+||(I−H)µ_B||²]/NM, which can be negative and does not tend to zero as σ_b→0 unless the model interpolates µ_B. Thus Eq. (27)'s stated limit fails and the 'rigorously derived' metric is internally inconsistent.
-
fitted input called prediction
[Section 5.2, Algorithm 1; Section 5.1; Fig. 7 discussion]
"Algorithm 1 ... 1: Set ω=(ω1,ω2) ... 3: Solve Xˆ=argmin‖AX−B‖²_2 ... 9: Compute µR,ΣR and then µ(ε²_p) 10: return µ(ε²_p) values. ... As we only have 6 rakes, we do not discern between testing and training data ... The frequency pairs that yield low values of µ(ε²_p) are the same as identified in part I."
Algorithm 1 selects the harmonic pair by computing µ(ε²_p) on the same data used to fit X, and Section 5.1 explicitly says there is no testing/training split. The low µ(ε²_p) values reported for the selected pairs are therefore in-sample residuals of the least-squares fit, not independent out-of-sample validations. Because the harmonic pair is chosen by minimizing the same metric that is then presented as the 'spatial sampling uncertainty,' the numerical demonstration is forced by construction: it reports the training error of the best-fitting model as evidence for the metric's rigor. No external or held-out benchmark separates sampling error from model-selection overfitting.
full rationale
Most of the uncertainty propagation in Section 4 is standard linear algebra and Gaussian quadratic-form theory, and the paper's reliance on Part I is a normal self-citation: Algorithm 1 is reproduced in the present paper, and the harmonic model is carried over from prior work. The circularity lies in the two proposed metrics. First, ε_p² is used both as a random variable (Eq. 18) with moments (Eq. 21) and as a deterministic in-sample residual (Eq. 25); Eq. (26) subtracts these incompatible objects, so ε_m² is not a genuine uncertainty and Eq. (27)'s limit is false unless the fitted model already interpolates the mean data. Second, the numerical support for the metrics is in-sample: the harmonic pair that minimizes µ(ε_p²) is selected on the same six rakes used to evaluate it, so the low reported values are partly an artifact of the fitting and selection procedure. These issues affect the paper's central claim that the metrics are 'rigorously derived'; however, the underlying propagation framework has independent content, so a score of 6 rather than higher is appropriate.
Assumptions & free parameters
free parameters (5)
- Measurement noise standard deviation sigma_b =
0.51 K and 1.02 K
- Rake position standard deviation sigma_theta =
0.51 degrees and 5.1 degrees
- Harmonic pair omega =
(1, 4) for Engine A
- Regularization threshold beta and lambda sequence =
lambda = (0.0001, 0.001, 0.1, 10); beta not specified
- Correlation matrix for correlated measurement case =
Block structure with 0.0 or 1.0 entries
assumptions (5)
- domain assumption The measurement vector B follows a multivariate Gaussian distribution with known mean mu_B and covariance Sigma_B.
- domain assumption The true temperature field is exactly representable by the chosen small set of Fourier harmonics.
- domain assumption For the analytical derivations, the measurement uncertainties are uncorrelated and have equal variance, Sigma_B = sigma_b^2 I.
- domain assumption The measurement noise standard deviation sigma_b is known exactly.
- domain assumption The Fourier matrix A is known and its entries, the rake positions, are either exact or sampled from a known distribution with specified covariance.
Cite this review
Pith. "Pith review of Spatial Flow-Field Approximation Using Few Thermodynamic Measurements Part II: Uncertainty Assessments." pith.science (2026). https://pith.science/paper/IOBLCBMM
@misc{pith2026190802934,
author = {Pith},
title = {Pith review of: Spatial Flow-Field Approximation Using Few Thermodynamic Measurements Part II: Uncertainty Assessments},
year = {2026},
howpublished = {\url{https://pith.science/paper/IOBLCBMM}},
note = {Machine review of arXiv:1908.02934}
}
read the original abstract
In this second part of our two-part paper, we provide a detailed, frequentist framework for propagating uncertainties within our multivariate linear least squares model. This permits us to quantify the impact of uncertainties in thermodynamic measurements---arising from calibrations and the data acquisition system---and the correlations therein, along with uncertainties in probe positions. We show how the former has a much larger effect (relatively) than uncertainties in probe placement. We use this non-deterministic framework to demonstrate why the well-worn metric for assessing spatial sampling uncertainty falls short of providing an accurate characterization of the effect of a few spatial measurements. In other words, it does not accurately describe the uncertainty associated with sampling a non-uniform pattern with a few circumferentially scattered rakes. To this end, we argue that our data-centric framework can offer a more rigorous characterization of this uncertainty. Our paper proposes two new uncertainty metrics: one for characterizing spatial sampling uncertainty and another for capturing the impact of measurement imprecision in individual probes. These metrics are rigorously derived in our paper and their ease in computation permits them to be widely adopted by the turbomachinery community for carrying out uncertainty assessments.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
Spatialflow-fieldap- proximationusingfewthermodynamicmeasurementsPartI:formulationandareaaveraging
Seshadri,P.,Simpson,D.,Thorne,G.,Duncan,A.,andParks,G.,2019. “Spatialflow-fieldap- proximationusingfewthermodynamicmeasurementsPartI:formulationandareaaveraging”. Submitted to ASME Journal of Turbomachinery. 19
work page 2019
-
[2]
InternationalOrganization for Standardization, Geneva
International Organization for Standardization, 2008.Uncertainty of Measurement–Part 3: GuidetotheexpressionofUncertaintyinMeasurement ,Vol.98-3. InternationalOrganization for Standardization, Geneva
work page 2008
-
[3]
Dieck, R., Steele, W., and Osolsobe, G., 2005. “Test uncertainty”.American Society of Mechanical Engineers, PTC 19.1
work page 2005
-
[4]
EA Laboratory Committee, 2013.Evaluation of the uncertainty of measurement in calibra- tion, Vol. EA-4/02. European Cooperation for Accreditation
work page 2013
-
[5]
Handbook, Uncertainty in Gas Turbine Measurements
Abernathy,R.B.,Powell,B.D.,Colbert,D.L.,Sanders,D.G.,andThompsonJr,J.W.,1973. Handbook, Uncertainty in Gas Turbine Measurements. Tech. rep., Arnold Air Force Station, Texas
work page 1973
-
[6]
H., 2007.Measurement uncertainty: methods and applications
Dieck, R. H., 2007.Measurement uncertainty: methods and applications. International So- ciety of Automation
work page 2007
-
[7]
C., 2013.Uncertainty quantification: theory, implementation, and applications, Vol
Smith, R. C., 2013.Uncertainty quantification: theory, implementation, and applications, Vol. 12. SIAM
work page 2013
-
[8]
Abernethy, R., Benedict, R., and Dowdell, R., 1985. “ASME measurement uncertainty”. Journal of Fluids Engineering,107(2), pp. 161–164
work page 1985
Show all 22 references
-
[9]
B., 2013.Monte Carlo theory, methods and examples
Owen, A. B., 2013.Monte Carlo theory, methods and examples
2013
-
[10]
Suitable Averaging Techniques in Non-Uniform Internal Flows
Pianko, M., and Wazelt, F., 1983. Suitable Averaging Techniques in Non-Uniform Internal Flows. Tech. rep., AGARD-AR-182
1983
-
[11]
Recommended Practices for Measurement of Gas Path Pressures andTemperaturesforPerformanceAssessmentofAircraftTurbineEnginesandComponents
Saravanmuttoo, H., 1990. Recommended Practices for Measurement of Gas Path Pressures andTemperaturesforPerformanceAssessmentofAircraftTurbineEnginesandComponents. Tech. rep., AGARD-AR-245
1990
-
[12]
Engineering application of experimental uncer- tainty analysis
Coleman, H. W., and Steele, W. G., 1995. “Engineering application of experimental uncer- tainty analysis”.AIAA Journal, 33(10), pp. 1888–1896
1995
-
[13]
Do Rosenblatt and Nataf isoprobabilistic transformations really differ?
Lebrun, R., and Dutfoy, A., 2009. “Do Rosenblatt and Nataf isoprobabilistic transformations really differ?”.Probabilistic Engineering Mechanics,24(4), pp. 577–584
2009
-
[14]
Copulamodeling: anintroductionforpractition- ers
Trivedi, P.K., Zimmer, D.M., etal., 2007. “Copulamodeling: anintroductionforpractition- ers”.Foundations and TrendsR©in Econometrics, 1(1), pp. 1–111
2007
-
[15]
Generalized accept-reject sampling schemes
Casella, G., Robert, C. P., and Wells, M. T., 2004. “Generalized accept-reject sampling schemes”.Lecture Notes-Monograph Series, Institute of Mathematical Statistics, pp. 342– 347
2004
-
[16]
Temperature measurement system for low pressure ratio turbine testing
Vázquez, R., and Sánchez, J. M., 2003. “Temperature measurement system for low pressure ratio turbine testing”. In ASME Turbo Expo 2003: Turbine Technical Conference and Expo- sition, American Society of Mechanical Engineers
2003
-
[17]
Vleghert, J. P. K., 1989. Measurement Uncertainty within the Uniform Engine Testing Pro- gram. Tech. rep., AGARD-AG-307. 20
1989
-
[18]
Combination probes for stagnation pressure and temperature measurements in gas turbine engines
Bonham, C., Thorpe, S. J., Erlund, M. N., and Stevenson, R., 2017. “Combination probes for stagnation pressure and temperature measurements in gas turbine engines”.Measurement Science and Technology,29(1), p. 015002
2017
-
[19]
Gas Temperature Measurement
Moffat, R. J., 1962. “Gas Temperature Measurement”. In Temperature; Its Measurement and Control in Science and Industry, Volume 2, A. I. Dahl and C. M. Herzfeld, eds., Reinhold Publishing Corporation, pp. 553–571
1962
-
[20]
TheElementsofStatisticalLearning ,2nded
Hastie,T.,Tibshirani,R.,andFriedman,J.,2009. TheElementsofStatisticalLearning ,2nded. Series in Statistics. Springer, New York, NY
2009
-
[21]
J., 2016
Faraway, J. J., 2016. Linear Models with R. Texts in Statistical Science. Chapman and Hall/CRC, Boca Raton, FL
2016
-
[22]
Generalized inverse of a matrix and its applications
Rao, C. R., and Mitra, S. K., 1972. “Generalized inverse of a matrix and its applications”. In Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Theory of Statistics, University of California Press, pp. 601–620. 21
1972
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.