{"id":"009dfab5-91c4-47ac-bef8-23b04afceee5","arxiv_id":"2412.10004","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":11,"one_line_summary":"A NeRF-based representation plus latent patch-matching algorithm that synthesizes meso-structure textures from multi-view images and maps them onto new 3D shapes.","lead":"This paper presents a method to capture real-world textures that have 3D surface detail (meso-structure) using neural radiance fields, then synthesize larger versions of those textures for application to new 3D shapes. It matters because ordinary 2D textures look flat for objects like leaves, fabric, grass, and bark, and this approach preserves the 3D relief and view-dependent appearance.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The base-shape extraction step (Sec. 3.1.1) lacks a principled scale separation, so the central disentanglement into coarse mesh plus SDF/tangent-frame texture may be biased on exactly the real-world geometries the method targets.","rationale":"The reader's weakest_assumption identifies the same load-bearing dependency: the captured scene must be faithfully decomposed into a smooth base mesh plus a latent feature field, with SDF and tangent frame capturing all meso-structure. My stress-test agrees and sharpens the point: the problem is not just that inaccurate base extraction degrades quality, but that the extraction pipeline (Instant-NGP, Co-ACD, Laplacian smoothing) has no explicit scale-separation criterion and can systematically misplace the base surface for convex protrusions (durian thorns), thin structures (truss), or perforated topologies (Lego). The paper's own failure analysis in Sec. 5.1 confirms the method degrades on these cases, and because the representation, synthesis, and mapping all inherit the base mesh, any bias there propagates through the entire pipeline. The concern is real and load-bearing, but it is an acknowledged limitation rather than an internal inconsistency: for the demonstrated textures (grass, leaves, fabric, bark), the method produces plausible results, and the synthetic test I propose would clarify the regime of validity. Therefore the reader's conditional verdict is appropriate; my read does not change it. I also note the quantitative evaluation is thin (no error bars, no code/data), but the reader already accounted for that in the CONDITIONAL verdict. The synthetic test is the single check most likely to settle whether the base-shape concern actually undermines the central claim.","tokens_in":18606,"tokens_out":11141,"duration_ms":134319,"concrete_test":"Construct a synthetic scene with known ground truth: a planar base surface and a sinusoidal height displacement of amplitude A and wavelength lambda (e.g., A=0.5cm, lambda=2cm) rendered as multi-view images. Run the base-shape extraction pipeline (Instant-NGP density field -> Marching Cubes -> Co-ACD -> Laplacian smoothing/remeshing) to obtain the coarse mesh. Compute the mean absolute error (MAE) and 95th-percentile error between the extracted mesh and the true base plane, normalized by A. Then train the NeRF-texture model and measure the residual SDF values at the true surface: they should be zero if the base shape is correct. If the normalized MAE exceeds 0.2A or the residual SDF shows a systematic offset, the disentanglement is biased, and the central claim fails for even simple synthetic meso-structure. Repeating with A/lambda varied would map the regime where the method works.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim presumes that every captured scene can be decomposed into a smooth base mesh plus a latent feature field on that mesh, with signed distance and tangent frame capturing all meso-structure (Sec. 3.1.2-3.1.4). The base mesh is produced by Instant-NGP reconstruction, Co-ACD convex decomposition, and Laplacian smoothing/remeshing (Sec. 3.1.1). These operations are generic geometry-processing tools, not a principled scale-separation filter. For a durian with thorns, the convex hull may enclose the thorn tips rather than the smooth body; for a truss or perforated part, the coarse mesh may be too thin or topologically incorrect. If the extracted base surface lies off the true base, the signed distance s(x) is biased and the latent features attached to footpoints no longer encode the texture consistently. The paper's own challenge analysis (Sec. 5.1) concedes that 'limited coarse extents' and 'complex topology' cause the method to fail: the truss example produces artifacts and the Lego cockpit cannot be captured. This is not a peripheral limitation; it identifies the exact condition on which the entire capture-synthesis-mapping pipeline rests. When the base shape is wrong, the SDF values seen during training are systematically different from those used at application time on a new shape, and the learned decoder cannot transfer. Thus the load-bearing assumption is not merely 'a base shape exists' but 'the extraction procedure reliably returns the true scale-separated base shape,' which is unvalidated empirically and demonstrably false for several real-world geometries in Fig. 20-21.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents NeRF-Texture, a method to capture, model, synthesize, and re-render textures with meso-structure (geometric detail) and view-dependent appearance from multi-view images. The scene is disentangled into a coarse base mesh and a latent feature field defined on that mesh; query points are parameterized by signed distance and footpoint, and a small MLP decodes density, shading coefficients, and fine normals. Texture synthesis is performed by implicit patch matching of latent features over a planar domain or curved surfaces, with a clustering regularizer to align latent-space and content-space metrics. Experiments include qualitative results on real-world captures, quantitative view synthesis on DTU, comparison with 2D textures via SIFID, comparison with NeRF-Tex, ablations, and a challenge analysis.","tokens_in":19025,"tokens_out":8203,"duration_ms":91442,"significance":"If the central claims hold, this is a useful and timely contribution to neural texture synthesis: it is among the first to treat meso-structure textures as neural radiance fields that can be captured from real images, synthesized, and mapped to new shapes, while preserving view-dependent effects. The paper provides a fairly detailed description of the representation and synthesis pipeline, and it is commendable that the authors include a challenge analysis (Sec. 5) that candidly identifies failure modes for narrow structures and complex topology. However, the quantitative support for key claims is thinner than the breadth of the results suggests: the SIFID comparison is reported without error bars, the clustering ablation is qualitative only, and the base-shape extraction assumption is not validated quantitatively. With additional experiments and clarifications, the contribution would be solid.","major_comments":[{"comment":"The base shape extraction procedure (Instant-NGP + Co-ACD + Laplacian smoothing/remeshing) is a generic heuristic with no principled scale-separation criterion. The signed distance s(x) and tangent frame Tc(xc) are defined relative to this base; if the extracted base is inaccurate (as the paper's own examples of truss and Lego cockpit in Sec. 5.1 show), the learned representation is biased and the decoder cannot transfer to new shapes. The paper does not quantify under what geometric conditions the disentanglement assumption holds, nor does it provide a diagnostic for detecting base-shape failure before synthesis. The challenge analysis lists failure cases but does not analyze the sensitivity of the representation to base-shape error. Please add a quantitative sensitivity study (e.g., perturbing the base mesh and measuring synthesis quality) and a discussion of the geometric properties required for the assumption to be valid.","section":"Sec. 3.1.1 and Sec. 5.1"},{"comment":"The clustering loss Lclu is presented as a key contribution for improving patch matching, yet the ablation in Sec. 4.5 is purely qualitative: Fig. 17 shows PCA visualizations and a rendered comparison, but no quantitative metric. The claim that the clustering constraint 'reduces artifacts' is not substantiated with numbers. Please report a quantitative measure of synthesis quality with and without Lclu (e.g., SIFID on the synthesized textures, patch matching error, or a perceptual metric) across multiple textures, including error bars. This is load-bearing because the paper motivates Lclu as essential for matching performance.","section":"Sec. 4.5 and Eq. (4)"},{"comment":"The SIFID comparison against 2D textures is the main quantitative evidence for the advantage of the NeRF-based representation, but Table 2 reports only a single average per elevation angle with no standard deviation, number of textures/scenes, or statistical test. The protocol for cropping ground-truth regions and selecting 'closest viewing directions' is not fully specified, making the comparison hard to reproduce or interpret. Please provide per-scene/per-texture results with error bars, and describe the ground-truth selection procedure in enough detail that the evaluation is reproducible. As written, the table does not convincingly support the claim that the NeRF-based representation is 'more realistic than 2D textures.'","section":"Table 2 and Sec. 4.3"}],"minor_comments":[{"comment":"The formula for the coarse normal n~c(x) is ambiguous: the second term (x - v1)/(w||x - v1||^2) appears inside the summation in the typeset equation, which would add it K times. Please place the term outside the summation or clarify the intended expression, and define W unambiguously.","section":"Eq. (1)"},{"comment":"The comparison with NeRF-Tex is described only qualitatively, and the training data for NeRF-Tex is generated using the authors' own representation. Please provide more implementation details (e.g., bounding box size, number of views, lighting) so that the fairness of the comparison can be assessed, and consider reporting a quantitative metric.","section":"Sec. 4.4"},{"comment":"Figure references are out of order: the text mentions 'the second row of Fig. 21' before introducing Fig. 20, and the Lego example in Fig. 20 is referenced after Fig. 21. Please reorder the figures or the text so that the discussion matches the figure numbering.","section":"Sec. 5.1"},{"comment":"The claim that latent features are fetched in 'O(1) time complexity' is only true for the hash lookup itself; the overall pipeline includes KNN search and ray casting, which have higher complexity. Please qualify the statement to avoid overclaiming.","section":"Sec. 3.1.4"},{"comment":"There are several typos and formatting issues, including 'Standord Bunny' (Sec. 4.1), 'F r e c h e t' (Sec. 4.3), and the malformed reference [4] ('Bao and Yang, Z. Junyi...'). A careful proofread is needed.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is an extension of the authors' SIGGRAPH 2023 work, and the TPAMI version adds curved-surface synthesis and additional experiments. The novelty is incremental but within the scope of a journal extension. I would encourage the editor to ask the authors to release code and captured data to support reproducibility, as none is mentioned. Additionally, the claim of being 'the first work for NeRF texture synthesis' should be carefully checked against concurrent work such as [26] and NeRF-Tex; the distinction from NeRF-Tex (synthesis vs. repeated placement) is clear, but the phrasing should be tightened to avoid overclaiming."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this is a real method contribution. The paper takes NeRF-Tex from modeling to synthesis, and that is the part that is actually new. The representation—coarse base mesh plus hash-grid latent features on the mesh, with signed distance and tangent frame as the texture coordinate—is a sensible way to capture meso-structure. The latent-space patch matching with the clustering regularizer addresses a real problem, since raw latent distances don't necessarily correspond to perceptual similarity. The curved-surface extension is a useful addition. The paper is also honest: the challenge section (Sec. 5) names the failure modes, and the ablation on training views is a good idea.\n\nWhat I'd flag is not the core idea but the evaluation and reproducibility. Table 2 reports SIFID as a single average per elevation angle without error bars or per-sample spread. On 4 DTU scenes with 5 test images, the PSNR/SSIM/LPIPS comparison is illustrative but not decisive. No code or data are released. Some hyperparameters are in the text (lambda values, K=8, w=0.01, patch 128x128), but others—patch overlap width, cluster center count, kappa, atlas resolution, candidate count K' in the matching algorithm—are not specified. A reader cannot reproduce this without asking the authors.\n\nOn the stress-test concern about base shape extraction: it is a real soft spot. Co-ACD and Laplacian smoothing are not a principled scale-separation filter. For a durian or a truss, the extracted base may not sit where the true smooth base is, which biases the SDF and footpoints and breaks transfer to new shapes. But the paper's own challenge analysis already demonstrates exactly this: Fig. 20 shows the Lego cockpit failing, Fig. 21 shows the truss failing, Fig. 22 shows the semantic limits. So the concern is not a hidden flaw; it is a known boundary condition. It does mean the selling line 'capture any real-world texture' is too broad. The method works when the base shape is reasonably smooth and has enough spatial extent to sample patches. That should be stated as a scope limitation in the abstract and intro, not buried in Sec. 5.\n\nBottom line: this deserves a serious referee. The science is credible, the novelty is real, and the failure analysis makes the boundaries clear. Graphics and vision people working on neural materials, texture synthesis, or editable NeRF representations will get value from it. I'd send it to review and ask for code/data or at least a full hyperparameter appendix, plus error bars on the SIFID comparison. If those come, I'd be comfortable with the contribution.","headline":"Solid method paper that adds a genuinely missing capability—NeRF texture synthesis—with an honest failure analysis; the main weaknesses are evaluation rigor and under-specified reproducibility, not the core idea.","tokens_in":19541,"tokens_out":2372,"would_cite":true,"duration_ms":26816,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Real-world textures with fine 3D geometry can be captured as NeRF textures and synthesized to any size, then mapped onto new shapes.","keywords":["neural radiance fields","texture synthesis","meso-structure texture","view-dependent appearance","disentangled representation","latent patch matching","hash grid encoding","spherical harmonics"],"falsifier":"For a scene with a clean base shape, such as a flat wall of stones, render the synthesized texture from a grazing angle and measure SIFID against a real photograph: the paper reports 0.82 at 80 degrees elevation, so a substantially larger value would indicate that the view-dependent meso-structure is not actually preserved.","tokens_in":18421,"feed_emoji":"🧵","tokens_out":6533,"duration_ms":67679,"temperature":0.7,"pith_summary":"The paper proposes NeRF-Texture, a pipeline that captures, models, synthesizes, and re-renders real-world textures that have fine 3D geometry (meso-structure) and view-dependent appearance. Its central claim is that such textures can be disentangled from a coarse base shape and stored as latent features on that shape, then synthesized to arbitrary size by patch-matching the latent features rather than pixel colors. If correct, a user could shoot a short video of a material, synthesize arbitrarily large 3D textures from it, and map them onto new shapes while preserving the meso-structure and its lighting effects. The paper presents this as the first NeRF texture synthesis method, extending its earlier conference version to curved surfaces.","feed_headline":"Turn a phone video into a 3D texture for any shape","feed_subtitle":"Captures fur, grass, and weaving as view-dependent NeRF textures, then maps them onto new objects.","key_machinery":"The central object is the disentangled NeRF texture representation: 3D space is reparameterized through a differentiable projection onto a coarse base mesh, giving a footpoint and a signed distance, and hash grids store latent texture features indexed by the footpoint. The argument is carried by combining this representation with implicit patch matching: patches of latent features are sampled on the base shape, matched and quilted in feature space by the minimum-cost-cut procedure of image quilting, and a Student t-distribution clustering loss regularizes the feature distribution so that feature-space distance tracks content-space distance. A custom derivation rule for the projection, dxc/dx = I - nc^T nc and ds/dx = nc, lets gradients flow through the footpoint for normal estimation and camera optimization. For curved surfaces, an atlas UV map and a coarse-to-fine patch pyramid replace the fixed kd-tree acceleration used in planar synthesis.","core_discovery":"NeRF-Texture represents a textured scene as a smooth base mesh plus a NeRF texture: two hash-grid feature fields defined on the base mesh's surface, queried by the footpoint of each 3D sample, together with the sample's signed distance from the mesh. An MLP decoder turns these features into density, diffuse and specular Phong coefficients, glossiness, and the elevation and azimuth of a fine normal expressed in the local tangent frame; color is then computed with spherical-harmonic lighting. Synthesis is performed by extracting implicit patches of latent features on the base shape, then running the image-quilting approach of patch matching and minimum-cut stitching directly in latent space, with a clustering loss that makes latent distance correlate with reconstructed content similarity. The synthesized latent field can be mapped onto any new mesh, including curved surfaces via an atlas parameterization, and rendered in real time. The paper demonstrates applications on bark, durian, fabric, leaves, flowers, mirror balls, a metal bed, and curved targets such as a ring, a shark, and a tower.","pith_inferences":["If the latent metric after clustering truly tracks content similarity, the clustering regularization could transfer to other neural fields that need patch-based editing, such as relightable or animatable radiance fields.","A natural testable extension is to measure whether nearest-neighbor search in the clustered latent space returns patches whose rendered appearance matches human judgment, which would validate the metric-consistency argument directly.","The representation effectively turns a 3D texture into a reusable asset with its own lighting decomposition, so a library of captured NeRF textures could be built and shared much like 2D texture libraries.","Building on the paper's own suggestion, adding a generative completion model could extend the method to thin or semantically meaningful structures like railings and keycaps, where the current greedy patch matching is known to break continuity."],"forward_implications":["A single short video of a material can produce a reusable NeRF texture that is applied to many different shapes without re-training the model.","Synthesized NeRF textures preserve meso-structure occlusion and view-dependent reflection at high viewing angles, where 2D image textures visibly break down.","The method renders at around 84 FPS, making the synthesized textures practical for real-time applications rather than offline baking.","The same latent patch-matching pipeline extends to arbitrary curved surfaces through atlas parameterization with a coarse-to-fine matching strategy."],"supporting_citations":[{"why":"Supplies the hash-grid encoding and the coarse mesh extraction via marching cubes that the representation builds on.","marker":"[6]"},{"why":"Provides the patch matching and minimum-cut stitching algorithm adapted to latent feature patches.","marker":"[32]"},{"why":"The base neural radiance field formulation that the method extends and compares against for view synthesis.","marker":"[3]"},{"why":"Prior disentangled mesh-based representation whose vertex-stored codes require high-resolution meshes and cannot handle non-watertight meso-structure.","marker":"[4]"},{"why":"Prior UV-parameterized neural texture mapping that the paper contrasts with its own surface-based parameterization.","marker":"[5]"},{"why":"NeRF-Tex, the prior NeRF texture modeling approach on planar bounding boxes, which the method compares against and extends to real-world data.","marker":"[1]"},{"why":"Approximate convex decomposition used to smooth the extracted coarse mesh into a base shape.","marker":"[7]"},{"why":"Source of the clustering loss that regularizes the latent feature distribution for better patch matching.","marker":"[70]"},{"why":"Supplies the spherical-harmonic shading framework used to decompose lighting and material in the rendering module.","marker":"[59]"}],"fun_headline_variants":["NeRF-Texture: synthesize 3D textures from multi-view photos","Capture fur, grass, and weaving as NeRF textures onto new shapes","Photos to view-dependent NeRF textures for any mesh surface","Synthesize meso-structure textures and map them onto curved objects","From real images to NeRF textures: transferable to any 3D shape"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a real textured scene can be split cleanly into a smooth base shape plus latent features on that shape that carry all meso-structure and appearance, so the whole pipeline inherits the quality of the base shape extraction.","fun_headline_variants_meta":{"raw":{"variants":["NeRF-Texture: synthesize 3D textures from multi-view photos","Capture fur, grass, and weaving as NeRF textures onto new shapes","Photos to view-dependent NeRF textures for any mesh surface","Synthesize meso-structure textures and map them onto curved objects","From real images to NeRF textures: transferable to any 3D shape"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000357,"raw_usage":{"total_tokens":1960,"prompt_tokens":996,"completion_tokens":964,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":612,"completion_tokens_details":{"reasoning_tokens":869}},"tokens_in":612,"tokens_out":964,"duration_ms":10608,"temperature":1.0,"reasoning_tokens":869,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:27:29.423510+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For a scene with a clean base shape, such as a flat wall of stones, render the synthesized texture from a grazing angle and measure SIFID against a real photograph: the paper reports 0.82 at 80 degrees elevation, so a substantially larger value would indicate that the view-dependent meso-structure is not actually preserved.","supporting_citations":[{"cited_title":"Instant neural graphics primitives with a multiresolution hash encoding,","cited_arxiv_id":null,"evidence_quote":"Supplies the hash-grid encoding and the coarse mesh extraction via marching cubes that the representation builds on."},{"cited_title":"Image quilting for texture syn- thesis and transfer,","cited_arxiv_id":null,"evidence_quote":"Provides the patch matching and minimum-cut stitching algorithm adapted to latent feature patches."},{"cited_title":"Approximate convex decom- position for 3D meshes with collision-aware concavity and tree search,","cited_arxiv_id":null,"evidence_quote":"Approximate convex decomposition used to smooth the extracted coarse mesh into a base shape."},{"cited_title":"Unsupervised deep embed- ding for clustering analysis,","cited_arxiv_id":null,"evidence_quote":"Source of the clustering loss that regularizes the latent feature distribution for better patch matching."},{"cited_title":"A signal-processing frame- work for inverse rendering,","cited_arxiv_id":null,"evidence_quote":"Supplies the spherical-harmonic shading framework used to decompose lighting and material in the rendering module."}],"review_version":1}