Pith. sign in

REVIEW 4 major objections 4 minor 23 references

Hardware Accelerated Neural Block Texture Compression with Cooperative Vectors

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read Low-dynamic-range BC1 latents make neural texture compression viable—comparable quality at half the memory, decoded in 0.55 ms.

desk verdict BC1 latents are a genuinely useful twist on neural texture compression, but the anisotropic-filtering and timing claims overreach. read the letter →

arxiv 2506.06040 v1 pith:YAJITBZT submitted 2025-06-06 cs.GR

classification cs.GR
keywords neuraltexturecompressionblockBC1cooperativevectorsanisotropicfilteringmaterialshardwareaccelerationtile-basedrendering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper extends neural texture compression by replacing high-dynamic-range BC6 latent textures with low-dynamic-range BC1 latents. The central claim is that LDR block formats are sufficient containers for learned latent codes, so a texture set can use twice as many latent maps at the same memory footprint, yielding better quality at equal compression or a much higher compression ratio at modest quality loss. The paper also claims that anisotropic filtering works without any anisotropic training data, because the decoder learns that blending latent codes produces blended outputs. On the hardware side, a tile-based rendering pass and cooperative-vector matrix multiplication bring decoding of 4K PBR texture sets to 0.55 ms per frame on an Intel B580.

What carries the argument

The central object is BCF1: a stack of BC1-compressed latent textures at different resolutions (the VARA and VARB layouts), each texel encoded as a quantized blend of two endpoints, $T(uv) = (1-\alpha(uv))e_0(uv) + \alpha(uv)e_1(uv)$, with endpoints and $\alpha$ quantized through a sigmoid during quantization-aware training. The decoder is a small MLP $f(x)$ that maps concatenated latent samples to PBR channels. Around this, the runtime machinery is tile-based: 8x4 tiles are classified into no-neural, uniform, or mixed tiles, mixed tiles are repacked by MLP id, and MLP layers run as cooperative-vector matrix multiplications. The BC1 container is what permits hardware filtering of latents and the doubled latent count.

What would settle it

Render a trained BCF1 scene with 16x anisotropic filtering and compare each pixel against the reference: compute the MLP on each anisotropic tap separately, average the decoded results, and compare to decoding the hardware-filtered latent once. If the two diverge beyond a small PSNR margin, the paper's side-effect hypothesis fails. If the side effect holds, the 0.55 ms timing should be reproducible on the same hardware and scene.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims to make neural block texture compression viable with low-dynamic-range storage: BCF1 stores latent features as BC1 textures and decodes them with a small MLP, and this matches BCF6 quality at slightly better compression (VARA) or nearly doubles compression (VARB) with a reasonable PSNR drop. It further claims that standard bilinear-only training generalizes to anisotropic filtering, and that cooperative vectors accelerate matrix-vector products so the decoder runs in real time. The headline is that a 4K, nine-channel texture set can be rendered with anisotropic filtering at 1080p in 0.55 ms using 28 MB of VRAM per asset on an Intel B580.

Load-bearing premise

The load-bearing premise is that a network trained only on bilinearly filtered latents automatically handles anisotropic filtering—that because it learned to interpolate latents, any blend of latent codes will decode to the blend of the decoded outputs; if that does not hold for the many-sample blends of anisotropic filtering, the advertised rendering quality and timing are not supported.

Editorial extensions

If this is right

  • Neural materials can be stored and decoded from BC1 latents with quality comparable to BC6 at equal memory, so an asset's whole nine-channel 4K texture set fits in 28 MB of VRAM.
  • The VARA layout matches BCF6 quality with a small compression gain; the VARB layout roughly doubles the compression ratio with a moderate PSNR drop (about 1.5 dB in the reported example).
  • Anisotropic filtering requires no retraining or special data: the decoder extrapolates from bilinear training to anisotropic taps.
  • Cooperative-vector matrix multiplication accelerates MLP inference by 20-46x for wide layers, bringing 1080p material evaluation to 0.55 ms on an Intel B580.
  • Tile-based classification keeps the neural cost proportional to visible neural pixels, so the method fits into a deferred shading pipeline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, if the bilinear-to-anisotropic side effect is real, it should also cover other linear filter kernels; a stress test at high anisotropy ratios would separate genuine generalization from visual tolerance.
  • Beyond the paper, the viability of BC1 suggests other LDR block formats with different rate-quality tradeoffs (BC2, BC3, or ASTC LDR) deserve the same treatment.
  • Beyond the paper, a single shared scene-wide decoder—an option the authors defer—would make the tile-sorting step nearly unnecessary and is a natural fit for the same cooperative-vector hardware.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper extends the block-compressed features (BCF) method of Weinreich et al. by replacing BC6 latent textures with BC1 latents, introducing two latent-resolution schedules (VARA and VARB), and adding a quantization-aware training procedure. The runtime uses a visibility-buffer pipeline with a tile-based classification pass and cooperative-vector hardware matrix multiplication to accelerate the small MLP decoder. The authors report quality and compression-ratio comparisons against self-implemented BCF6 and NTC baselines, as well as performance measurements on an Intel B580 and an NVIDIA RTX 4090. They also claim that anisotropic filtering works as an unplanned side effect of training on filtered latents, even though no anisotropic data was used during training.

Significance. If the empirical claims hold, the paper makes a useful practical contribution: low dynamic range BC1 latents can halve the memory footprint of BC6 latents at similar quality, and the cooperative-vector tile-based implementation makes real-time neural material decoding plausible on current GPUs. Strengths include the concrete quantization-aware training description (Eqs. 1-3), the reported speedups of up to roughly 40x with cooperative vectors, and the candid admission that anisotropic filtering is a hypothesis rather than a trained behavior. The main open issues are that the anisotropic-filtering claim currently lacks quantitative support, and the headline timing of 0.55 ms is not traceable to the paper's main performance table.

major comments (4)
  1. [Section 3.3] The anisotropic-filtering claim rests on an explicitly stated hypothesis: the authors write that because the network learned bilinearly filtered latents, 'as a side effect it learned that blending latent codes results in blended outputs.' Figure 3 provides only a visual side-by-side comparison, and Section 3.2 states that training used trilinear filtering with no anisotropic samples. Since the runtime evaluates the MLP on a hardware-computed anisotropic sum of many bilinear taps, there is no architectural guarantee that the MLP is approximately affine over the set of latent vectors encountered under an anisotropic footprint. This is load-bearing because the abstract's 'with anisotropic filtering' claim and the 0.55 ms headline are tied to hardware anisotropic filtering in Section 4. I request a quantitative validation (e.g., PSNR/SSIM of filtered neural outputs against filtered references across multiple anisotropy levels and angles), or a revision that limits the claim to bilinear/trilinear filtering.
  2. [Abstract and Table 2] The abstract reports rendering at 0.55 ms on an Intel B580, but Table 2 reports 0.06 ms for the varA D=64 configuration with cooperative vectors, and the text explicitly states that Visibility and Lighting passes are skipped from those timings. The 0.55 ms value appears only in the Figure 1 caption. As written, a reader cannot tell whether 0.55 ms is a full-pipeline time, a different configuration, or a measurement on a different scene. Please report the full-pipeline timing in Table 2, or clarify what 0.55 ms includes, so that the headline number is reproducible.
  3. [Algorithm 1] In Algorithm 1, ClassificationB, line 54 writes to `_IndexedTilesBufferRW[tileGroupOffset * WORK_GROUP_SIZE + targetSlot]`, but `tileGroupOffset` is never defined in the listing; the variable defined two lines earlier is `mlpGroupOffset`, which is never used. As printed, the shader would not compile, and the tile-sorting logic cannot be checked. Please fix the variable name and specify the indexing scheme for `_MLPUsageBufferRW`.
  4. [Section 5 and Figure 5] The quality/compression-ratio comparison in Figure 5 relies entirely on self-implemented BCF6 and NTC baselines, and neither Table 1 nor Figure 5 reports error bars or the number of independent runs. Because the central claim of 'similar quality with a small gain in compression' is a comparative claim, the baselines must be trustworthy. Please validate the baselines against published numbers, provide confidence intervals over multiple training runs, or release the compression code so the comparison is independently checkable.
minor comments (4)
  1. [Equations (2)-(3)] Equations (2) and (3) both apply `quant(sigmoid(α), [5,6,5])`, but they should presumably apply it to `sigmoid(e0)` and `sigmoid(e1)`, respectively; this looks like a copy-and-paste typo.
  2. [Sections 3.3 and 7] There are several typos that should be corrected: 'anisotropicaly', 'anistropic', 'fitlering', 'surprizing', 'substancial', and 'reasonnable'.
  3. [References] The dataset reference [Z*] is incomplete: it lists only an author prefix and a URL, with no title, venue, or access date. Please complete it.
  4. [Table 2] Table 2's color key for B580 (blue) and RTX 4090 (green) is only in the caption; since the table uses slash-separated pairs, the text should explicitly state which position corresponds to which GPU.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the headline claims are empirical measurements or clearly labeled hypotheses, not derivations that reduce to their own inputs.

full rationale

I walked the paper's claimed derivation chain and found no circular step. The central claims are (1) BC1-stored latent textures make neural block texture compression viable, (2) this yields comparable quality at higher compression ratios than BCF6, and (3) cooperative vectors accelerate MLP evaluation. Each is supported by direct training/optimization experiments and by measured timings; no fitted parameter is renamed as a prediction, and no equation is defined in terms of the quantity it is supposed to predict. The only potentially concerning passage is Section 3.3, where anisotropic filtering is justified by an explicitly labeled hypothesis: 'we hypothetize that since the network has learned to infer the behaviour of bilinearly filtered latents, as a side effect it learned that blending latent codes results in blended outputs.' That is an unproven generalization assumption whose validation is visual only, and it is a legitimate correctness/support weakness, but it is not circular: the anisotropic-filtering claim is not used to define the training objective, nor is the hypothesis derived from the claim. Similarly, the BCF6 and NTC baselines are reimplemented in-house, which affects external benchmarking credibility but is not a circularity pattern under the stated rules. No self-citation is load-bearing, and no uniqueness theorem or prior-work ansatz is invoked to force the method's choices. The paper is self-contained against direct measurements, so the correct circularity score is 0.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The central quality and performance results are empirical and depend on many hand-chosen hyperparameters (listed above) and on domain assumptions inherited from neural material compression. No invented entities appear: cooperative vectors and BC1 are existing hardware and formats. The main unvalidated assumption is that training with bilinear filtering generalizes to anisotropic filtering.

free parameters (8)
  • Latent resolution schedule VARA = K=4 BC1 textures at WxH, WxH, W/2xH/2, W/2xH/2
    Targets about the same footprint as BCF6 by using two full-res and two quarter-res BC1 textures; chosen by hand in Section 3.1.
  • Latent resolution schedule VARB = WxH, W/2xH/2, W/4xH/4, W/8xH/8
    Doubles the compression ratio relative to VARA; chosen by hand in Section 3.1.
  • Number of latent textures K = 4
    Four latent textures per material is a design choice in Section 3.1; the paper does not test other counts.
  • MLP hidden dimension D = 64 default; tested 16, 32, 64
    Wider MLP improves PSNR in Table 1; D=64 is used for the headline quality and performance results.
  • MLP depth L = 1 hidden layer
    A single hidden layer is fixed in Section 3.1, following the prior BCF architecture.
  • Sub-pixel shift pattern = Half-texel shift on the 2nd and 4th latent textures
    Introduced in Section 3.1 to avoid alignment of BC1 blocks and reduce artifacts; chosen empirically.
  • Learning rates for MLP and latent textures = MLP lr=1e-3, latent BC1 lr=1e-2
    Section 3.2; these hyperparameters affect the trained quality of every reported model.
  • Tile size for classification = 8x4 pixel workgroups
    Used in Section 4 to match cooperative vector workgroup requirements and classification granularity; chosen for performance.
assumptions (5)
  • domain assumption Hardware texture units correctly decode BC1 blocks and perform bilinear and anisotropic filtering on the decoded values.
    The method stores latent features in BC1 and relies on the GPU texture unit for interpolation; Sections 3.1 and 3.3 depend on this.
  • ad hoc to paper An MLP trained on bilinearly filtered inputs extrapolates to anisotropic filtering.
    Section 3.3 states this as a side-effect hypothesis and validates only visually; no training data includes anisotropic kernels.
  • domain assumption Correlated PBR channels can be represented by a shared low-dimensional latent tensor.
    Core premise of neural material compression inherited from prior work; Section 3 uses one latent set for 9 channels.
  • standard math Straight-through estimators give usable gradients through BC1 quantization.
    Section 3.2 equations (1)-(3) let gradients pass through the quantization function; standard practice in quantization-aware training.
  • domain assumption Adam optimization with the chosen learning rates converges to a good local optimum on the Polyhaven textures.
    Section 3.2 sets lr=1e-3 and 1e-2; no convergence guarantees are given, so quality is empirical.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hardware Accelerated Neural Block Texture Compression with Cooperative Vectors." pith.science (2026). https://pith.science/paper/YAJITBZT

@misc{pith2026250606040,
  author       = {Pith},
  title        = {Pith review of: Hardware Accelerated Neural Block Texture Compression with Cooperative Vectors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YAJITBZT}},
  note         = {Machine review of arXiv:2506.06040}
}
read the original abstract

In this work, we present an extension to the neural texture compression method of Weinreich and colleagues [2024]. Like them, we leverage existing block compression methods which permit to use hardware texture filtering to store a neural representation of physically-based rendering (PBR) texture sets (including albedo, normal maps, roughness, etc.). However, we show that low dynamic range block compression formats still make the solution viable. Thanks to this, we show that we can achieve higher compression ratio or higher quality at fixed compression ratio. We improve performance at runtime using a tile based rendering architecture that leverage hardware matrix multiplication engine. Thanks to all this, we render 4k textures sets (9 channels per asset) with anisotropic filtering at 1080p using only 28MB of VRAM per texture set at 0.55ms on an Intel B580.

Figures

Figures reproduced from arXiv: 2506.06040 by the authors.

Figure 1
Figure 1. Rendering physically-based materials using neural block texture compression. We compress PBR texture sets using block compressed latent images decoded by a multi-layer perceptron (left). We show how to accelerate this architecture using the cooperative vectors extension to obtain up to 23× speedup compared to a compute based with fused multiply add (FMA) implementation (right). Abstract In this work, we present an e… view at source ↗
Figure 3
Figure 3. Isotropic and anisotropic fitlering. We validate that training with isotropic filtering permits to also extrapolate anisotropic filtering in our VARA. 4. Runtime & Hardware Acceleration Given that the neural texture set is stored in the BC1 format and leverages the texture unit, the main bottleneck in the runtime eval￾uation lies in how the MLP is evaluated and where in the graphics pipeline it is evaluated. Renderi… view at source ↗
Figure 4
Figure 4. Tile-based classification. We tile the output of the Visi￾bility pass in 8×4 pixels tiles. Each tile is classified as either non neural (gray), neural with a single MLP (green), and neural with mixed MLPs (red). To avoid divergence, we repack all mixed tiles to have 8×4 groups of pixels with the same MLP (orange, marron, blue). Once this sorting is done, we can perform our Lighting pass. Tile-based classification. F… view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Quality and compression ratio. We compare the qual￾ity of our variants, varA and varB with latents at full resolution (varA1, varB1) and half resolution (varA2, varB2) respectively. We also plot the quality obtain using Nvidia’s NTC (both 0.5 and 1.0 variants) and BCF …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

23 extracted references · 19 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry.original add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 i...

  3. [3]

    : Jpeg xl next-generation image compression architecture and coding tools

    Alakuijala J., Van Asseldonk R., Boukortt S., Bruse M., Comșa I.-M., Firsching M., Fischbacher T., Kliuchnikov E., Gomez S., Obryk R., et al. : Jpeg xl next-generation image compression architecture and coding tools. In Applications of digital image processing XLII (2019), vol. 11137, SPIE, pp. 112--124

  4. [4]

    A., Hunt W

    Burns C. A., Hunt W. A. : The visibility buffer: A cache-friendly approach to deferred shading. Journal of Computer Graphics Techniques (JCGT) 2, 2 (August 2013), 55--69. URL: http://jcgt.org/published/0002/02/04/

  5. [5]

    Ball \'e J., Laparra V., Simoncelli E. P. : End-to-end optimized image compression. In International Conference on Learning Representations (2017). URL: https://openreview.net/forum?id=rJxdQ3jeg

  6. [6]

    URL: https://devblogs.microsoft.com/directx/enabling-neural-rendering-in-directx-cooperative-vector-support-coming-soon/

    Enabling neural rendering in directx: Cooperative vector support coming soon, 2025. URL: https://devblogs.microsoft.com/directx/enabling-neural-rendering-in-directx-cooperative-vector-support-coming-soon/

  7. [7]

    A., Ebrahimi T., Skodras A

    Christopoulos C. A., Ebrahimi T., Skodras A. N. : Jpeg2000: the new still picture compression standard. In Proceedings of the 2000 ACM workshops on Multimedia (2000), pp. 45--49

  8. [8]

    : Image compression using block truncation coding

    Delp E., Mitchell O. : Image compression using block truncation coding. IEEE transactions on Communications 27, 9 (1979), 1335--1342

Show all 23 references
  1. [9]

    : Efficient graphics representation with differentiable indirection

    Datta S., Marshall C., Dong Z., Li Z., Nowrouzezahrai D. : Efficient graphics representation with differentiable indirection. In SIGGRAPH Asia 2023 Conference Papers (2023), pp. 1--10

  2. [10]

    : Neural texture block compression

    Fujieda S., Harada T. : Neural texture block compression. arXiv preprint arXiv:2407.09543 (2024)

  3. [11]

    : Neural graphics texture compression supporting random access

    Farhadzadeh F., Hou Q., Le H., Said A., Rauwendaal R., Bourd A., Porikli F. : Neural graphics texture compression supporting random access. In European Conference on Computer Vision (2024), Springer, pp. 412--429

  4. [12]

    : Compression of digital images by block truncation coding: a survey

    Fr \"a nti P., Nevalainen O., Kaukoranta T. : Compression of digital images by block truncation coding: a survey. The Computer Journal 37, 4 (1994), 308--332

  5. [13]

    : Extending in-game textures using cdns for 'call of duty: Modern warfare 2'

    Fowler C. : Extending in-game textures using cdns for 'call of duty: Modern warfare 2'. GDC'23, 2023

  6. [14]

    : Physically based shading in theory and practice

    Hill S., McAuley S., Belcour L., Earl W., Harrysson N., Hillaire S., Hoffman N., Kerley L., Patry J., Piek \'e R., et al. : Physically based shading in theory and practice. In ACM SIGGRAPH 2020 Courses. 2020, pp. 1--12

  7. [15]

    : Vk\_nv\_cooperative\_vector, 2024

    Jeff B. : Vk\_nv\_cooperative\_vector, 2024. URL: https://registry.khronos.org/vulkan/specs/latest/man/html/VK_NV_cooperative_vector.html

  8. [16]

    URL: https://docs.vulkan.org/spec/latest/appendices/compressedtex.html

    Khronos : Vulkan documentation: Compressed image formats, 2025. URL: https://docs.vulkan.org/spec/latest/appendices/compressedtex.html

  9. [17]

    : Moving frostbite to physically based rendering

    Lagarde S., de Rousiers C. : Moving frostbite to physically based rendering. In SIGGRAPH 2014 Courses. ACM, 2014

  10. [18]

    : Adaptive scalable texture compression

    Nystad J., Lassen A., Pomianowski A., Ellis S., Olson T. : Adaptive scalable texture compression. In Proceedings of the Fourth ACM SIGGRAPH/Eurographics Conference on High-Performance Graphics (2012), pp. 105--114

  11. [19]

    : PyTorch: an imperative style, high-performance deep learning library

    Paszke A., Gross S., Massa F., Lerer A., Bradbury J., Chanan G., Killeen T., Lin Z., Gimelshein N., Antiga L., Desmaison A., K\" o pf A., Yang E., DeVito Z., Raison M., Tejani A., Chilamkurthy S., Steiner B., Fang L., Bai J., Chintala S. : PyTorch: an imperative style, high-pe...

  12. [20]

    : Random-access neural compression of material textures

    Vaidyanathan K., Salvi M., Wronski B., Akenine-M \"o ller T., Ebelin P., Lefohn A. : Random-access neural compression of material textures. arXiv preprint arXiv:2305.17105 (2023)

  13. [21]

    Wallace G. K. : The jpeg still picture compression standard. IEEE transactions on consumer electronics 38, 1 (1992), xviii--xxxiv

  14. [22]

    : Real-time neural materials using block-compressed features

    Weinreich C., De Oliveira L., Houdard A., Nader G. : Real-time neural materials using block-compressed features. In Computer Graphics Forum (2024), vol. 43, Wiley Online Library, p. e15013

  15. [23]

    Zaal G., et al. :. URL: https://polyhaven.com/

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.