Pith. sign in

REVIEW 3 major objections 5 minor 24 references

A Neural Quality Metric for BRDF Models

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read The paper proposes BRDF-NQM, a compact neural network that predicts the perceptual quality of BRDF approximations directly from BRDF samples, without rendering, and reports an average Spearman correlation of 0.67 with human judgments.

desk verdict The 0.67 correlation is inflated because the JOD calibration is fitted on the test materials, so the central claim doesn't hold, though the idea is worth a revised attempt. read the letter →

arxiv 2508.02131 v1 pith:B4FBQV2P submitted 2025-08-04 cs.CV

classification cs.CV
keywords BRDFqualitymetricperceptualneuralnetworkjust-objectionable-differencematerialappearancefittingimage-spaceRusinkiewiczcoordinates
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces BRDF-NQM, a compact neural network that predicts how much a fitted BRDF model will deviate perceptually from the reference material, using only samples of the two BRDFs rather than rendered images. The authors train a four-layer MLP on paired reference and approximated BRDFs from measured datasets, using just-objectionable-difference (JOD) scores as targets; for most of the data the JODs are pseudo-labels obtained by fitting $\Delta E_{\mathrm{ITP}}$ color-difference values to the perceptual scores from an earlier study. On the 180 test instances with human ground truth, the metric reaches an average Spearman correlation of 0.67 with subjective judgments, clearly above the 0.38 to 0.55 range of traditional BRDF-space error metrics. The practical payoff is a quality evaluation that runs in milliseconds and needs no rendering, though the authors report that using the same network as a loss for BRDF fitting still produces color artifacts.

What carries the argument

The load-bearing object is the learned mapping from BRDF samples to perceived quality, implemented as a four-layer MLP with layer normalization, GELU activations, dropout, and a sigmoid-scaled output. Its inputs are 500 reflectance samples per BRDF, paired as reference and approximation, selected in the Rusinkiewicz coordinate system with denser sampling near the half-vector origin to capture specular structure. The training targets come from a fitted function $\mathrm{JOD} = f(\Delta E_{\mathrm{ITP}})$ with parameters $b_1, b_2, b_3$ estimated on the perceptual-study materials, making the image-space metric $\Delta E_{\mathrm{ITP}}$ the teacher that transfers human judgments into BRDF-space labels. This combination of perceptual pseudo-labels, a subsampled angular representation, and a small MLP is what lets the metric run in 0.04 ms per prediction without rendering.

What would settle it

Retrain or re-evaluate with strict material separation: fit the $\Delta E_{\mathrm{ITP}}$-to-JOD parameters on one set of materials and test on disjoint materials with human ratings. If the Spearman correlation drops toward the 0.4 to 0.6 baseline range, the reported 0.67 advantage would be shown to depend on the label-fitting overlap rather than on a generalizable perceptual mapping.

Watch

Extended reading notes

Core claim

The central claim is that the perceptual quality of a BRDF approximation can be learned and predicted directly in BRDF space. BRDF-NQM takes 500 paired samples of reference and approximated BRDFs, transformed to a cube-root, logarithmic, whitened representation in Rusinkiewicz coordinates, and outputs a scalar JOD value. Trained with pseudo-labels derived from the perceptually validated $\Delta E_{\mathrm{ITP}}$ metric, the network achieves the highest correlation with human judgments among all compared BRDF-space metrics, averaging 0.67 Spearman versus a top baseline of 0.55. The paper also reports that using BRDF-NQM as a loss function for BRDF fitting yields partially accurate parameters but introduces a red color shift in the estimated diffuse component, so predictive quality and optimization-loss quality do not automatically coincide.

Load-bearing premise

The load-bearing premise is that the JOD pseudo-labels are valid for every material, even though the $\Delta E_{\mathrm{ITP}}$-to-JOD function is fitted on the same 20 BRDF materials that are later used as the test set, so the test labels are not independent of that fitting step.

Editorial extensions

If this is right

  • BRDF fit quality can be checked in real time (0.04 ms per prediction) without rendering, making perceptually informed evaluation practical in iterative workflows.
  • Traditional BRDF-space metrics such as RMSE, MAE, and their cube-root or logarithmic variants are outperformed in correlation with human judgment, suggesting representation-space error alone is not enough.
  • The same learned quality signal does not yet serve as a reliable loss function, since fitting with it produced a red color shift in the diffuse component; predictive accuracy does not guarantee optimization usefulness.
  • The success of transferring image-space perceptual labels into BRDF space suggests future BRDF-space loss functions can be designed from perceptual data rather than from ad hoc distance measures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the test materials overlap the materials used to fit the label-generation curve, the headline 0.67 is best read as an upper bound; a fully held-out material evaluation would likely give a lower, more realistic generalization estimate.
  • The same architecture could be adapted to other appearance representations, such as SVBRDF or BTF, by replacing the sampling scheme, provided perceptual pseudo-labels are available.
  • If the training-data color imbalance is corrected and the whitening step is made more chromatic-sensitive, the observed red shift as a loss function might be mitigated, turning the metric into a gradient source for perceptually guided fitting.
  • A testable extension would be to use angularly correlated noise for low-JOD augmentation, which could improve sensitivity to structured distortions instead of per-direction independent noise.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes BRDF-NQM, a compact multi-layer perceptron that predicts a just-objectionable-difference (JOD) score directly from 500 paired samples of a reference and an approximated BRDF, without rendering. The network is trained on pseudo-labels produced by a calibrated image-space metric, ΔE_ITP, applied to a dataset of 159 measured BRDFs and augmented with noisy BRDF pairs. The authors report that BRDF-NQM achieves an average Spearman correlation of 0.67 with human perceptual judgments on a 20-material test subset, outperforming eight BRDF-space baseline metrics. They also report that using the metric as a loss function for BRDF fitting yields limited results due to color shifts.

Significance. The underlying idea—a rendering-free, neural quality metric operating directly in BRDF space—is potentially useful for BRDF fitting, model selection, and compression, and the paper includes an open-source implementation and a compact architecture with fast inference. However, the central empirical claim of substantially improved correlation with human judgments is compromised by the evaluation protocol: the calibration function used to generate training labels is fitted on the same subjective data that later constitutes the test set. As presented, the reported 0.67 Spearman correlation does not establish an independent perceptual alignment, and the comparison against uncalibrated baselines is not like-for-like. The concept is interesting, but the validation needs to be substantially reworked.

major comments (3)
  1. [Section 3.1, Eq. (1) and Section 4] The calibration of ΔE_ITP to JOD values uses the Levenberg–Marquardt algorithm on "the subset of materials used in the perceptual experiment"—the same 20 materials that Section 3.1 later reserves for testing. The fitted parameters (b1, b2, b3) in Eq. (1) are then used to compute pseudo-labels for all training data, and at test time the network's predictions on the 180 held-out instances are effectively approximations of this calibrated function. The reported Spearman correlation of 0.67 therefore partly reflects the goodness of fit of the calibration curve to the test responses, not an independent measure of perceptual alignment. To support the claim, the authors must use a disjoint split for calibration and testing (e.g., leave-one-out or k-fold over the 20 materials), and should also calibrate the baseline metrics with the same regression procedure to ensure a fair comparison.
  2. [Section 3.1, augmentation paragraph] The paper does not specify how JOD pseudo-labels are assigned to the 3,340 augmented BRDF pairs generated by adding Gaussian noise with σ=0.01. If ΔE_ITP values for these pairs are computed by rendering the noisy BRDFs, the rendering pipeline and its cost should be described; if instead the JOD values are inferred without rendering, it is unclear how Eq. (1) applies to noise-perturbed BRDF samples. This information is necessary for reproducibility and for assessing the validity of the training targets.
  3. [Section 4, Figure 3] The paper reports only a single average Spearman correlation (0.67) without error bars, confidence intervals, or a significance test against the best baseline (0.55). Given the small test set (180 instances from 20 materials), the claim of "significantly higher correlation" requires a statistical analysis, such as a permutation test or bootstrapping across materials, to establish that the improvement is not within sampling variability.
minor comments (5)
  1. [Equation (1)] The expression includes `max(ΔE_ITP, 0)^b3` with a negative exponent b3 = −0.21; for ΔE_ITP = 0 this term is undefined. Please clarify the domain and any numerical safeguards used.
  2. [Abstract] The phrase "perceptually validated image-space metric" is vague; specify that ΔE_ITP was selected based on its previously reported correlation with human judgments.
  3. [Section 3.1, sampling strategy] The description of selecting 500 BRDF samples says samples are chosen based on reflectance magnitude while also using uniform angular sampling; please clarify how these two criteria are combined.
  4. [Figure 3] The order of baseline metrics in the figure legend is not consistently aligned with the text in Section 4; ensure the labels match the described definitions.
  5. [General] There are typographical errors in author names (e.g., "Rafa l" should be "Rafał") and in the acknowledgments ("Sk lodowska"); these should be corrected in the final version.

Circularity Check

1 steps flagged · score 6.0 of 10

The reported 0.67 Spearman correlation is partly circular: the ΔE_ITP-to-JOD calibration (Eq. 1) is fitted on the same 20 materials later used for testing, and those parameters generate the training pseudo-labels, so BRDF-NQM's test predictions inherit a mapping already optimized to the test judgments.

  1. fitted input called prediction [Section 3.1 (Eq. 1 and dataset split) and Section 4 (evaluation, Figure 3)]
    "For the subset of materials used in the perceptual experiment, where ground-truth JOD values are available, we fitted the metric predictions to the corresponding JOD scores using the Levenberg–Marquardt algorithm ... Once the parameters were obtained, we used them to compute JOD values for all materials in the dataset based on their ∆E ITP values ... The dataset was randomly split into training (80%) and validation (20%) sets, while the BRDFs used in the perceptual experiment were reserved for testing."

    b1–b3 in Eq. (1) are estimated against human JODs of exactly the 20 materials later 'reserved for testing.' Those same parameters generate the pseudo-labels used to train BRDF-NQM, so the network is trained to approximate a calibrated function of ΔE_ITP whose free parameters were optimized on the test responses. At evaluation, the reported Spearman 0.67 compares predictions on those 20 materials against the same human judgments used for the fit; the test is therefore not independent of label construction. The comparison with uncalibrated BRDF-space baselines is unfair because BRDF-NQM's teacher was fitted to the test set while the baselines were not. The headline prediction is thus partially a restatement of the calibration fit.

full rationale

BRDF-NQM is a learned surrogate trained on pseudo-labels derived from ΔE_ITP, which is a legitimate knowledge-distillation setup and not circular by itself. The circularity is in the evaluation protocol: Eq. (1)'s parameters are estimated on the same 20-material subset that is later used as the test set, and the resulting fitted function generates the training targets. Consequently, the test correlation of 0.67 is partly a reflection of the calibration fit rather than an independent measure of the neural metric's perceptual validity. The self-citation to prior work [10] is load-bearing for the dataset and the choice of ΔE_ITP, but [10] is an external user study, so it does not by itself constitute circularity. The monotonicity of Eq. (1) limits the direct inflation of rank correlation, but the protocol still leaks test information into the label generator, making the comparison against uncalibrated baselines not a controlled test. Overall score 6: the central evaluation claim partially reduces to a test-fitted label generator.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim depends on the fitted mapping of ΔE_ITP to JOD (Eq. 1) and on the assumption that this mapping generalizes beyond the 20-material subset. The augmentation parameters and sampling design are hand-chosen and introduce additional degrees of freedom that are not fitted to human data.

free parameters (5)
  • b1 = -14.11
    Fitted parameter in the logistic function mapping ΔE_ITP to JOD (Eq. 1), fitted to the 20-material perceptual data using Levenberg-Marquardt.
  • b2 = -0.47
    Fitted parameter in Eq. (1), fitted to the perceptual data.
  • b3 = -0.21
    Fitted parameter in Eq. (1), fitted to the perceptual data.
  • noise sigma (σ) = 0.01
    Chosen by hand for data augmentation to generate low-JOD pairs.
  • scaling range for augmentation = [0.95, 1.05]
    Chosen by hand for the second augmentation step.
assumptions (4)
  • domain assumption The fitted logistic function (Eq. 1) accurately maps ΔE_ITP to perceptual JOD for all materials, including those not in the perceptual study.
    This is assumed when generating pseudo-labels for the full dataset; the parameters are fitted on only 20 materials and then applied to all 159.
  • domain assumption ΔE_ITP is a perceptually valid image-space metric for BRDF rendering quality, after fitting.
    The paper relies on the prior study's conclusion that ΔE_ITP has the highest correlation with human judgments.
  • domain assumption A 500-sample subsampling of the BRDF preserves the perceptual quality information needed for the metric.
    The paper reduces each BRDF to 500 samples without evidence that this is sufficient for perceptual quality prediction.
  • domain assumption The synthetic noise augmentation produces BRDF pairs whose JOD labels are meaningful and representative of real distortions.
    Section 3.1 describes adding Gaussian noise and computing labels, presumably via the same fitted mapping.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Neural Quality Metric for BRDF Models." pith.science (2026). https://pith.science/paper/B4FBQV2P

@misc{pith2026250802131,
  author       = {Pith},
  title        = {Pith review of: A Neural Quality Metric for BRDF Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B4FBQV2P}},
  note         = {Machine review of arXiv:2508.02131}
}
read the original abstract

Accurately evaluating the quality of bidirectional reflectance distribution function (BRDF) models is essential for photo-realistic rendering. Traditional BRDF-space metrics often employ numerical error measures that fail to capture perceptual differences evident in rendered images. In this paper, we introduce the first perceptually informed neural quality metric for BRDF evaluation that operates directly in BRDF space, eliminating the need for rendering during quality assessment. Our metric is implemented as a compact multi-layer perceptron (MLP), trained on a dataset of measured BRDFs supplemented with synthetically generated data and labelled using a perceptually validated image-space metric. The network takes as input paired samples of reference and approximated BRDFs and predicts their perceptual quality in terms of just-objectionable-difference (JOD) scores. We show that our neural metric achieves significantly higher correlation with human judgments than existing BRDF-space metrics. While its performance as a loss function for BRDF fitting remains limited, the proposed metric offers a perceptually grounded alternative for evaluating BRDF models.

Figures

Figures reproduced from arXiv: 2508.02131 by the authors.

Figure 1
Figure 1. (a) Relationship between JOD values and ∆E [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Network architecture: The input layer is the concatenation of reference and model RGB BRDFs [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Average Spearman correlation between subjective JOD scores and various BRDF-space metrics. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 22 canonical work pages

  1. [1]

    Layer Normalization

    Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. “Layer Normalization”. In: (July 2016). arXiv: 1607.06450 [stat.ML] (cit. on p. 4)

  2. [2]

    An Adaptive BRDF Fitting Metric

    J. Bieron and P. Peers. “An Adaptive BRDF Fitting Metric”. In: Computer Graphics Forum39.4 (2020), pp. 59–74. doi: 10.1111/cgf.14054 (cit. on pp. 1, 2)

  3. [3]

    genBRDF: discovering new analytic BRDFs with genetic programming

    Adam Brady et al. “genBRDF: discovering new analytic BRDFs with genetic programming”. In: ACM Trans. Graph.33.4 (July 2014). issn: 0730-0301. doi: 10.1145/2601097.2601193 (cit. on p. 2)

  4. [4]

    Acquisition and Validation of Spectral Ground Truth Data for Predictive Rendering of Rough Surfaces

    O. Clausen, R. Marroquim, and A. Fuhrmann. “Acquisition and Validation of Spectral Ground Truth Data for Predictive Rendering of Rough Surfaces”. In:Computer Graphics Forum37.4 (2018), pp. 1–12. doi: 10.1111/cgf.13470 (cit. on p. 2)

  5. [5]

    An Adaptive Parameterization for Efficient Material Acqui- sition and Rendering

    Jonathan Dupuy and Wenzel Jakob. “An Adaptive Parameterization for Efficient Material Acqui- sition and Rendering”. In: Transactions on Graphics (Proceedings of SIGGRAPH Asia)37.6 (Nov. 2018), 274:1–274:18. doi: 10.1145/3272127.3275059 (cit. on pp. 1, 3)

  6. [6]

    Material Fingerprinting: Identifying and Predicting Perceptual Attributes of Material Appearance

    Jiri Filip et al. Material Fingerprinting: Identifying and Predicting Perceptual Attributes of Material Appearance. 2024. arXiv: 2410.13615 [cs.CV] (cit. on p. 2)

  7. [7]

    Toward a Perceptually Based Metric for BRDF Modeling

    Adri` a For´ es, James Ferwerda, and Jinwei Gu. “Toward a Perceptually Based Metric for BRDF Modeling”. In: Final Program and Proceedings - IS and T/SID Color Imaging ConferenceCIC’12 (Jan. 2012), pp. 142–148. doi: 10.2352/CIC.2012.20.1.art00025 (cit. on p. 2)

  8. [8]

    Perceptually Motivated BRDF Comparison using Single Image

    V. Havran, J. Filip, and K. Myszkowski. “Perceptually Motivated BRDF Comparison using Single Image”. In: Computer Graphics Forum 35.4 (2016), pp. 1–12. doi: 10.1111/cgf.12944 (cit. on pp. 1, 2)

Show all 24 references
  1. [9]

    Gaussian Error Linear Units (GELUs)

    Dan Hendrycks and Kevin Gimpel. “Gaussian Error Linear Units (GELUs)”. In: arXiv preprint arXiv:1606.08415 (2016) (cit. on p. 4)

  2. [10]

    Perceived Quality of BRDF Models

    Behnaz Kavoosighafi et al. “Perceived Quality of BRDF Models”. In: Computer Graphics Forum (2025). issn: 1467-8659. doi: 10.1111/cgf.70162 (cit. on pp. 1–3, 5, 6)

  3. [11]

    Adam: A Method for Stochastic Optimization

    Diederik Kingma and Jimmy Ba. “Adam: A Method for Stochastic Optimization”. In: International Conference on Learning Representations(Dec. 2014) (cit. on p. 5)

  4. [12]

    A similarity measure for material appearance

    Manuel Lagunas et al. “A similarity measure for material appearance”. In: ACM Trans. Graph. 38.4 (July 2019). issn: 0730-0301. doi: 10.1145/3306346.3323036 (cit. on p. 2)

  5. [13]

    Perceptual quality of BRDF approximations: dataset and metrics

    Guillaume Lavou´ e et al. “Perceptual quality of BRDF approximations: dataset and metrics”. In: Computer Graphics Forum40.2 (2021), pp. 327–338. doi: 10.1111/cgf.142636 (cit. on pp. 2, 4)

  6. [14]

    A method for the solution of certain nonlinear problems in least squares

    Kenneth Levenberg. “A method for the solution of certain nonlinear problems in least squares”. In: Quarterly of Applied Mathematics2.2 (1944), pp. 164–168 (cit. on p. 3)

  7. [15]

    Image-Difference Prediction: From Grayscale to Color

    Ingmar Lissner et al. “Image-Difference Prediction: From Grayscale to Color”. In: IEEE Transac- tions on Image Processing22.2 (2013), pp. 435–446. doi: 10.1109/TIP.2012.2216279 (cit. on pp. 1, 2)

  8. [16]

    BRDF models for accurate and efficient rendering of glossy surfaces

    Joakim L¨ ow et al. “BRDF models for accurate and efficient rendering of glossy surfaces”. In: ACM Trans. Graph.31.1 (Feb. 2012). issn: 0730-0301. doi: 10.1145/2077341.2077350 (cit. on pp. 2, 4)

  9. [17]

    HDR-VDP-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions

    Rafa l Mantiuk et al. “HDR-VDP-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions”. In: 30.4 (2011), pp. 1–14 (cit. on p. 2)

  10. [18]

    An algorithm for least-squares estimation of nonlinear parameters

    Donald W. Marquardt. “An algorithm for least-squares estimation of nonlinear parameters”. In: Journal of the Society for Industrial and Applied Mathematics11.2 (1963), pp. 431–441 (cit. on p. 3)

  11. [19]

    A Data-Driven Reflectance Model

    Wojciech Matusik et al. “A Data-Driven Reflectance Model”. In: ACM Trans. Graph.22.3 (July 2003), pp. 759–769. issn: 0730-0301. doi: 10.1145/882262.882343 (cit. on pp. 1, 3)

  12. [20]

    Experimental Analysis of BRDF Models

    Addy Ngan, Fr´ edo Durand, and Wojciech Matusik. “Experimental Analysis of BRDF Models”. In: Eurographics Symposium on Rendering (2005). Ed. by Kavita Bala and Philip Dutre. The Eurographics Association, 2005. isbn: 3-905673-23-1. doi: 10.2312/EGWR/EGSR05/117-126 (cit. on pp. 2, 3)

  13. [21]

    Geometrical Considerations and Nomenclature for Reflectance

    Fred E. Nicodemus et al. “Geometrical Considerations and Nomenclature for Reflectance”. In: 1977 (cit. on p. 1)

  14. [22]

    On Optimal, Minimal BRDF Sampling for Reflectance Acquisition

    Jannik Boll Nielsen, Henrik Wann Jensen, and Ravi Ramamoorthi. “On Optimal, Minimal BRDF Sampling for Reflectance Acquisition”. In: ACM Transactions on Graphics (TOG)34.6 (Nov. 2015), 186:1–186:11. doi: 10.1145/2816795.2818085 (cit. on p. 3)

  15. [23]

    A New Change of Variables for Efficient BRDF Representation

    Szymon M. Rusinkiewicz. “A New Change of Variables for Efficient BRDF Representation”. In: Rendering Techniques ’98. Ed. by George Drettakis and Nelson Max. Vienna: Springer Vienna, 1998, pp. 11–22. isbn: 978-3-7091-6453-2 (cit. on p. 3)

  16. [24]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang et al. “Image quality assessment: from error visibility to structural similarity”. In: IEEE Transactions on Image Processing13.4 (2004), pp. 600–612. doi: 10.1109/TIP.2003.819861 (cit. on p. 2)

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.