Chord: Chain of Rendering Decomposition for PBR Material Estimation from Generated Texture Images

Boxiang Rong; Jingyu Wang; Maoyuan Xu; Zhi Ying

REVIEW 5 major objections 6 minor 4 references

Chord: Chain of Rendering Decomposition for PBR Material Estimation from Generated Texture Images

T0 review · 5 major / 6 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proposes a two-stage generate-and-estimate pipeline in which a diffusion model synthesizes a tileable, shaded texture and a rendering-aware chained estimator recovers a complete SVBRDF material, reporting higher quality and much f

desk verdict Chord is a solid, well-ablated chained-decomposition pipeline for PBR estimation from generated textures; the glossy-surface limit is real and acknowledged, and the eval has protocol gaps, but the core claim holds for its target domain. read the letter →

arxiv 2509.09952 v1 pith:RQWCZVP6 submitted 2025-09-12 cs.GR cs.CV

Zhi Ying , Boxiang Rong , Jingyu Wang , Maoyuan Xu This is my paper

classification cs.GRcs.CV

keywords SVBRDFestimationPBRmaterialgenerationdiffusionmodelsimage-conditioneddensepredictioninverserenderingtexturesynthesisintrinsicdecompositionchain-of-rendering

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that single-image PBR material estimation is best solved not by predicting all maps at once, but by a chain of rendering-aware decomposition steps: first basecolor, then normals from an irradiance estimate, then roughness and metalness from a renderer-based grid search. Each step is a single-step image-conditional diffusion model whose inputs are built from previous outputs, so the model sees physical cues rather than raw RGB alone. If correct, this gives a practical generate-and-estimate pipeline: users control a texture-generation stage with text, sketches, depth, or masks, and the Chord stage turns that texture into a full set of relightable material maps with quality that the authors report as surpassing existing methods.

What carries the argument

The Chain of Rendering Decomposition (Chord) is the central mechanism: a three-step sequential prediction in which previous outputs are converted into physically grounded conditioning signals (an irradiance approximation and a per-pixel roughness/metalness grid search) and fed to a single-step image-conditional latent diffusion model. The supporting mechanism is LEGO-conditioning, which gives each target channel its own first down-block, last up-block, and final convolution while sharing the middle blocks of the U-Net, avoiding modality-weight interference. The single-step phase sets the diffusion timestep to T and trains with image-space losses (L1 plus VGG perceptual plus render loss sampl

What would settle it

Render a glossy PBR material under the paper's single directional light with a strong highlight crossing the tile boundary, run Chord, and compare predicted basecolor, roughness, and metalness to ground truth. The paper's own discussion predicts that circular-padding wrap-around will corrupt the estimates; a clean recovery would show the assumption is stronger than claimed, while a clear failure would confirm the load-bearing lighting limitation.

Watch

Extended reading notes

Core claim

The central claim is that the under-constrained problem of single-image SVBRDF estimation becomes tractable when the estimator is organized around the rendering equation. Chord predicts basecolor first because its distribution is closest to the input RGB; it then forms an approximate irradiance map I_IRR = I_RGB / b and conditions normal prediction on it; finally it estimates the lighting direction and grid-searches roughness and binary metalness per pixel to form an RM map that conditions the last prediction. LEGO-conditioning assigns per-modality first-down-block, last-up-block, and output-convolution weights while sharing intermediate U-Net blocks, and single-step fine-tuning with pixel,

Load-bearing premise

The whole chain rests on the assumption that the input RGB is well approximated by a Cook-Torrance renderer under a single directional light with negligible specular error, so dividing by the predicted basecolor yields a clean irradiance map; the paper states this fails for glossy tileable surfaces.

Editorial extensions

If this is right

A single shaded texture image can be converted into a complete, relightable PBR material in about two seconds, roughly an order of magnitude faster than iterative diffusion-based estimation.
The two-stage design turns any control available to text-to-image generation (text, reference images, depth, line art, masks, inpaint edits) into a control for the final material, including localized edits.
Estimating channels sequentially with rendering-derived conditioning improves per-channel quality compared with joint prediction, especially for basecolor and normal, while preserving spatial alignment.
The method extends beyond generated textures to real top-down photographs with consistent lighting, so the estimation stage works as a standalone inverse-rendering tool.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An immediate testable extension is swapping the single directional light assumption for two or more lights; the chain's irradiance step would show whether the model learns to average lighting or fails, isolating the true source of the gains.
The paper's acknowledged glossy-surface failure suggests a concrete next design: generate separate tileable and non-tileable images, route specular detail through the non-tileable branch, and fuse them before estimation; this is a low-cost experiment the authors did not run.
Because LEGO-conditioning decouples modalities with minimal added parameters, the same sequential-conditioning strategy could transfer to other multi-modal dense prediction tasks (for instance depth, normal, and albedo from one image) where one modality can physically condition the next.
The single-step fine-tuning trade-off between sharpness and in-the-wild generalization implies that a few-step rectified-flow variant could retain speed while improving robustness; the paper itself identifies this direction as future work.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Dear colleague,

This is a real contribution, and it deserves a serious referee. The new thing is the chained decomposition: instead of estimating all SVBRDF channels at once, they predict basecolor, then normal using I_RGB/b as irradiance conditioning, then roughness/metalness via a rendering-consistent searched guidance image. That ordering is derived from the renderer, and the LEGO-conditioning (separate first/last U-Net blocks per target channel) is a sensible way to avoid weight conflicts between modalities. The single-step fine-tuning gives an 11x speedup over RGB->X while improving PSNR and LPIPS on most channels. The ablation is honest: each addition is tested, and some components (the RM grid search) actually hurt roughness and metalness, but the full system wins. I believe the central claim—state of the art on the tested benchmarks—holds.

The soft spots are mostly in the evaluation protocol. No error bars, no significance tests, and the Substance test set is "curated" without stated criteria. The model uses private data mixed into training, so numbers are not reproducible until code and data land. They retrain baselines on their dataset, which is fair, but I would like to see author-provided-weight comparisons flagged more clearly.

The stress-test point about Eq. 3 is legitimate, in proportion. I_RGB = b * irradiance treats shading as Lambertian and ignores specular; for glossy and metallic materials that conditioning image is contaminated. The paper waves at this with "we empirically verify the error is negligible" but gives no quantitative verification, and Section 7 concedes the pipeline fails on glossy surfaces. That means the general-SOTA claim is overbroad: it is SOTA for the diffuse-dominant test distribution, and there is no glossiness breakdown to show otherwise. This is a real limitation, but it is not a hidden fit—the grid search is disclosed, and the ablation shows it can even hurt.

Who is this for? Researchers and engineers working on automated material creation from text or images. It is a practical, incremental improvement on RGB->X with a clean architectural idea. I would accept it for peer review expecting a request for error bars, dataset details, and either a glossiness breakdown or a more bounded claim. Worth citing if you work in inverse rendering.

Referee Report

5 major / 6 minor

Summary. The paper proposes a two-stage PBR material generation and estimation framework. Stage 1 fine-tunes SDXL to generate tileable, directionally lit texture images I_RGB. Stage 2 introduces Chord, a chain that estimates basecolor, normal (+height), then roughness/metalness, using intermediate representations I_IRR = I_RGB / b (Eq. 3) and I_RM from a per-pixel grid search (Eqs. 4-6). The diffusion backbone uses LEGO-conditioning (Eq. 2) and single-step fine-tuning with an image-space loss (Eq. 9). Evaluations compare against Material Palette, SurfaceNet, MatFusion, RGB→X, and single-modality methods on MatSynth and a curated Substance set, with reported SOTA on several channels and an 11× speedup over RGB→X.

Significance. If the measurements hold, the work is a useful advance in material estimation/generation: it makes a coherent, renderer-aware chain for SVBRDF decomposition, uses a disclosed grid-search conditioning (not a hidden fit), and reports meaningful speedups over RGB→X. The paper is clearly written and the ablation is staged. However, the load-bearing empirical evidence has important gaps: no error bars, a curated test set with unspecified criteria, private training data, and an acknowledged failure mode for glossy surfaces that affects the core conditioning step. These issues do not invalidate the approach but currently make the SOTA claim broader than the evidence supports.

major comments (5)

[§4.2.2, Eq. (3), and §7] Eq. (3) defines I_IRR = I_RGB / b, which is exact only when the observed radiance contains no specular term. The text claims 'We empirically verify that the error introduced from specular terms is negligible,' but no measurement is reported. Section 7 explicitly concedes that the framework fails for glossy surfaces because tileable edge highlights bleed across the image. This is load-bearing: glossy and metallic materials are a core part of the SVBRDF domain, and the central decomposition step is invalid on them. Please add a quantitative breakdown of per-channel error as a function of roughness or material category, report the glossiness distribution of the MatSynth test split and the curated Substance set, and state the range of conditions under which Eq. (3) is a valid approximation. Without this, the general SOTA claim in Section 8 is not supported.
[§5.3] The Substance test set is described as 'curated' but no curation criteria are given; the reader cannot rule out selection that avoids the known glossy failure mode. Please state the exact curation rule, the proportion of glossy (low-roughness) and metallic materials, and whether such materials were excluded or proportionally represented. Also provide material IDs or hashes so the set can be audited. This is necessary to interpret Tables 3 and 4 and the cross-dataset generalization claim.
[Tables 2–5] No error bars, standard deviations, or significance tests are reported, and the advantage is not uniform. For example, roughness LPIPS is worse for the proposed method than for RGB→X (Table 2: 0.532 vs 0.512; Table 3: 0.550 vs 0.536). The main text states the method 'significantly outperforms' baselines (Section 8), but a 0.02 LPIPS difference on one channel, with no variance estimate, cannot be called significant. Add repeated-run statistics or bootstrap intervals for all metrics, and temper the significance claim accordingly.
[Table 5 and §5.4] The ablation narrative overstates the final components. In Table 5, the '+RM Grid Search' row has roughness PSNR 17.91 and LPIPS 0.594; adding '+Render Loss' decreases roughness PSNR to 16.97 and increases LPIPS to 0.633; '+Pretraining' recovers only to 17.48/0.550, still below the '+Approx Irradiance' row's 18.50/0.576. Metalness LPIPS also degrades slightly when render loss is added (0.076 to 0.077). The statement that 'including render loss and Pretraining Phase yields additional performance gains' is therefore only true for selected channels. Please discuss this trade-off explicitly or revise the claim.
[§5.1] The material estimation training set mixes MatSynth with unreleased private data ('28,344 materials after augmentation'), and the baseline methods are retrained on this same mix. This complicates external reproducibility and makes it hard to determine whether the reported advantage comes from the architecture/chain or from private data. Please release the private data or at least specify the split, augmentation recipe, and the exact filter used to construct the Substance test set; alternatively, re-run the key comparisons on a fully public subset.

minor comments (6)

[§4.2.2, Eq. (9)] The text says the lighting direction l is randomly sampled 8 times to generate 8 rendered pairs, but Eq. (9) writes only one l. Clarify the summation or state that the equation shows one sample.
[Fig. 10] The caption uses 'tillable' and 'non-tillable'; these should be 'tileable' and 'non-tileable'.
[Table 5] The final row is labeled 'Ours' while the preceding rows also include components; renaming to 'Ours (full)' would improve consistency with the text.
[§5.1] The generative stage is fine-tuned on '1,000 high-quality texture rendering images'; please specify how these images were selected and captioned, since this affects the reproducibility of Stage 1.
[§5.3.1] The comparison against StableNormal, Lotus, and E2E-FT involves methods predicting one modality versus four for the proposed method; the text acknowledges this, but Table 1 would benefit from a sentence interpreting the handicap more explicitly.
[References] The reference '[Miika Aittala, Timo Aila 2016]' appears to be a formatting error; the entry should be consistent with the two-shot SVBRDF reference above it.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the chained conditioning steps are algebraic preprocessing and a disclosed grid search, and the reported results are measured against external ground-truth benchmarks.

full rationale

The paper's load-bearing claims do not reduce to their inputs by construction. Eq. 3 (I_IRR = I_RGB / b) is an algebraic rearrangement of the diffuse shading term under the stated single-directional-light assumption; it is a conditioning transform on the predicted basecolor, not a fitted parameter used to report test metrics. Eqs. 4-6 are an explicit per-pixel discrete grid search over 41 roughness values and binary metalness using the disclosed renderer and an estimated light direction; the resulting I_RM is a conditioning feature. The final estimator is supervised with pixel, perceptual, and render losses against ground-truth SVBRDF maps (Eq. 9), and Tables 2-3 compare predictions to ground truth on the external MatSynth test split and a curated Substance test set. The ablation in Table 5 shows that adding the RM grid search can degrade roughness (17.91 vs 18.50 without it) and metalness (70.02 vs 80.14), so the final prediction is not simply copying I_RM. There are no load-bearing self-citations: the method builds on external works (RGB->X, Marigold, Lotus, SDXL) and the authors do not import a self-authored uniqueness theorem or ansatz. The limitations stated in Section 7 (failure on glossy surfaces, baked shadows, limited in-the-wild generalization) are candid scope restrictions and correctness concerns, not evidence of circularity. Overall, the derivation chain is self-contained and the evaluation is grounded in external benchmarks, so no circular step is identified.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The method's added value is the chain design; its correctness depends on a set of lighting and separability assumptions, not on new physical entities. Most axioms are standard rendering math or domain assumptions that the authors partially flag in Section 7.

free parameters (7)

Perceptual loss weight lambda = 0.005
Fixed coefficient on VGG-16 perceptual terms in Eq. 9; hand-chosen.
Roughness-metalness grid S = 41 roughness values in 25/255 to 225/255, binary metalness
Hand-defined space for per-pixel grid search in Eq. 5; determines the conditioning image for roughness and metalness prediction.
Render-loss lighting samples = 8 per iteration
Eq. 9 randomly samples 8 directional lights per iteration; training hyperparameter.
Fine-tuning schedule = 20 epochs single-step, optional 5 epochs pretraining
Reported in Section 5.4; no theoretical basis.
Texture-generation training set = 1,000 rendered texture images
SDXL fine-tuned on this size in Section 5.1; affects the distribution stage 1 produces.
Material estimation training mix = 28,344 materials after augmentation, MatSynth plus private data
Private, unreleased component; central to generalization claims.
Lighting-direction energy-decay heuristic = not disclosed in main text, supplementary A.2
Estimates l* for the grid search; a hand-designed heuristic.

assumptions (6)

domain assumption Input image I_RGB is a render of the material under a fixed top-down directional light using the same renderer R used for estimation.
Section 4.1 and 4.2; the entire inverse rendering chain relies on this. Section 7 admits glossy surfaces break it.
domain assumption Specular contribution to I_RGB / b is negligible, so I_IRR is diffuse irradiance.
Section 4.2.2 Normal Prediction; empirically asserted, fails for metals and glossy finishes.
ad hoc to paper Per-pixel search over the discrete set S produces a useful approximation of r and m conditioning.
Eqs. 5 and 6; the conditioning image I_RM encodes fitted roughness and metalness values, so the final model is partly refining its own rendered fit.
standard math Cook-Torrance BRDF with GGX NDF, Schlick-GGX geometry, and Schlick Fresnel is an adequate reflectance model.
Section 4.1, cited prior work; accepted in graphics.
standard math Height can be recovered from the normal map by Poisson integration.
Section 4.2.2, Simchony et al. 1990; standard technique.
domain assumption SVBRDF channels are separate modalities with separable latent structure, and basecolor is closest to RGB.
Fig. 2 t-SNE; motivates predicting basecolor first and using clean intermediates.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Chord: Chain of Rendering Decomposition for PBR Material Estimation from Generated Texture Images." pith.science (2026). https://pith.science/paper/RQWCZVP6

@misc{pith2026250909952,
  author       = {Pith},
  title        = {Pith review of: Chord: Chain of Rendering Decomposition for PBR Material Estimation from Generated Texture Images},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RQWCZVP6}},
  note         = {Machine review of arXiv:2509.09952}
}

read the original abstract

Material creation and reconstruction are crucial for appearance modeling but traditionally require significant time and expertise from artists. While recent methods leverage visual foundation models to synthesize PBR materials from user-provided inputs, they often fall short in quality, flexibility, and user control. We propose a novel two-stage generate-and-estimate framework for PBR material generation. In the generation stage, a fine-tuned diffusion model synthesizes shaded, tileable texture images aligned with user input. In the estimation stage, we introduce a chained decomposition scheme that sequentially predicts SVBRDF channels by passing previously extracted representation as input into a single-step image-conditional diffusion model. Our method is efficient, high quality, and enables flexible user control. We evaluate our approach against existing material generation and estimation methods, demonstrating superior performance. Our material estimation method shows strong robustness on both generated textures and in-the-wild photographs. Furthermore, we highlight the flexibility of our framework across diverse applications, including text-to-material, image-to-material, structure-guided generation, and material editing.

Figures

Figures reproduced from arXiv: 2509.09952 by the authors.

**Figure 1.** PBR materials generated by our method. Keywords in the text prompts used for generating each material (from left to right) are: fabric with flower embroidery, cobblestone, green marble, salmon fish meat, wood floor, colorful stone wall, cyberpunk iron wall, and wet mossy rock. Material creation and reconstruction are crucial for appearance modeling but traditionally require significant time and expertise from artist… view at source ↗

**Figure 2.** 2D t-SNE visualization of latent vectors. Texture RGB and basecolor modalities exhibit significant overlap, suggesting a strong bijective relationship. The normal modality forms a distinct cluster. Metalness vectors are concentrated near binary values (0 and 1), while roughness vectors are more evenly distributed across the (0, 1) range. LEGO-conditioning and single-step fine-tuning for efficient, high-quality mate… view at source ↗

**Figure 3.** Method Overview. Stage 1: Tileable texture image (𝐼RGB) generation using a fine-tuned diffusion model, controllable via user guidance (text prompts, reference images, or other control types). Stage 2: Material estimation predicts SVBRDF channels sequentially: (1) basecolor ˆ𝑏, (2) normal 𝑛ˆ (with height ℎˆ derived via normal integration), and (3) roughness 𝑟ˆ and metalness 𝑚ˆ . Each step’s input is computed from pre… view at source ↗

Figures from the paper (7 more)

**Figure 4.** Figure 4: Detailed illustration of LEGO-conditioning. Using the normal prediction step as an example, we illustrate the block-level architecture of the U-Net. We also highlight the corresponding active and inactive LEGOconditioning blocks in this step. (1) Pretraining Phase (op…

**Figure 6.** Figure 6: Qualitative normal estimation comparisons [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]

**Figure 5.** Figure 5: Material Generation Comparison with MatFuse [Vecchio et al. 2024b]. Prompts are prefixed with "A material of" for MatFuse and "texture of" for our method. We compare three conditioning scenarios: text only, text with sketch, and text with reference image. Reference ima…

**Figure 7.** Figure 7: Qualitative PBR estimation comparisons. We also include results from Substance 3D Sampler’s feature called "AI-Powered Image to Material" [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]

**Figure 8.** Figure 8: PBR material generated by our method. Texture RGB represents 𝐼RGB generated by texture generation stage, while Render represents re-rendered image from estimated material. Input Relit Basecolor Normal Roughness Metalness Render [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]

**Figure 9.** Figure 9: Material estimation from in-the-wild photographs. We demonstrate the robustness of our Chord pipeline using real-world top-down photographs as inputs. Photo from Unsplash. SA Conference Papers ’25, December 15–18, 2025, Hong Kong, Hong Kong [PITH_FULL_IMAGE:figures/fu…

**Figure 10.** Figure 10: More applications. SA Conference Papers ’25, December 15–18, 2025, Hong Kong, Hong Kong [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

4 extracted references · 2 linked inside Pith

[2020]

textureofdelicious spaghettiwithsauce, basilleaves

Joint SVBRDF Recovery and Synthesis From a Single Image using an Unsuper- vised Generative Adversarial Network. InEurographics Symposium on Rendering. Zibo Zhao, Zeqiang Lai, Qingxiang Lin, Yunfei Zhao, Haolin Liu, Shuhui Yang, Yifei Feng, Mingxin Yang, Sheng Zhang, Xianghui Yang, et al . 2025. Hunyuan3d 2.0: Scaling diffusion models for high resolution t...

arXiv 2025
[2021]

InIEEE Conf

High-Resolution Image Synthesis with Latent Diffusion Models. InIEEE Conf. Comput. Vis. Pattern Recog. (CVPR). Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. InMedical Image Computing and Computer-Assisted Intervention. Springer International Publishing. Tim Salimans and Jonathan ...

arXiv 2015
[2022]

MaterIA: Single Image High-Resolution Material Capture in the Wild.Comput. Graph. Forum(2022). Jaakko Lehtinen Miika Aittala, Timo Aila. 2016. Reflectance modeling by neural texture synthesis.ACM Trans. on Graphics(2016). Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. 2024. SDXL: Impro...

2022
[2024]

arXiv preprint arXiv:2411.17515(2024)

SuperMat: Physically Consistent PBR Material Estimation at Interactive Rates. arXiv preprint arXiv:2411.17515(2024). Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. InComputer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings,...

arXiv 2024

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.

[1] [2020]

textureofdelicious spaghettiwithsauce, basilleaves

Joint SVBRDF Recovery and Synthesis From a Single Image using an Unsuper- vised Generative Adversarial Network. InEurographics Symposium on Rendering. Zibo Zhao, Zeqiang Lai, Qingxiang Lin, Yunfei Zhao, Haolin Liu, Shuhui Yang, Yifei Feng, Mingxin Yang, Sheng Zhang, Xianghui Yang, et al . 2025. Hunyuan3d 2.0: Scaling diffusion models for high resolution t...

arXiv 2025

[2] [2021]

InIEEE Conf

High-Resolution Image Synthesis with Latent Diffusion Models. InIEEE Conf. Comput. Vis. Pattern Recog. (CVPR). Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. InMedical Image Computing and Computer-Assisted Intervention. Springer International Publishing. Tim Salimans and Jonathan ...

arXiv 2015

[3] [2022]

MaterIA: Single Image High-Resolution Material Capture in the Wild.Comput. Graph. Forum(2022). Jaakko Lehtinen Miika Aittala, Timo Aila. 2016. Reflectance modeling by neural texture synthesis.ACM Trans. on Graphics(2016). Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. 2024. SDXL: Impro...

2022

[4] [2024]

arXiv preprint arXiv:2411.17515(2024)

SuperMat: Physically Consistent PBR Material Estimation at Interactive Rates. arXiv preprint arXiv:2411.17515(2024). Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. InComputer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings,...

arXiv 2024