REVIEW 4 major objections 4 minor 23 references
Hardware Accelerated Neural Block Texture Compression with Cooperative Vectors
T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Low-dynamic-range BC1 latents make neural texture compression viable—comparable quality at half the memory, decoded in 0.55 ms.
desk verdict BC1 latents are a genuinely useful twist on neural texture compression, but the anisotropic-filtering and timing claims overreach. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is BCF1: a stack of BC1-compressed latent textures at different resolutions (the VARA and VARB layouts), each texel encoded as a quantized blend of two endpoints, $T(uv) = (1-\alpha(uv))e_0(uv) + \alpha(uv)e_1(uv)$, with endpoints and $\alpha$ quantized through a sigmoid during quantization-aware training. The decoder is a small MLP $f(x)$ that maps concatenated latent samples to PBR channels. Around this, the runtime machinery is tile-based: 8x4 tiles are classified into no-neural, uniform, or mixed tiles, mixed tiles are repacked by MLP id, and MLP layers run as cooperative-vector matrix multiplications. The BC1 container is what permits hardware filtering of latents and the doubled latent count.
What would settle it
Render a trained BCF1 scene with 16x anisotropic filtering and compare each pixel against the reference: compute the MLP on each anisotropic tap separately, average the decoded results, and compare to decoding the hardware-filtered latent once. If the two diverge beyond a small PSNR margin, the paper's side-effect hypothesis fails. If the side effect holds, the 0.55 ms timing should be reproducible on the same hardware and scene.
Extended reading notes
Core claim
On its own terms, the paper claims to make neural block texture compression viable with low-dynamic-range storage: BCF1 stores latent features as BC1 textures and decodes them with a small MLP, and this matches BCF6 quality at slightly better compression (VARA) or nearly doubles compression (VARB) with a reasonable PSNR drop. It further claims that standard bilinear-only training generalizes to anisotropic filtering, and that cooperative vectors accelerate matrix-vector products so the decoder runs in real time. The headline is that a 4K, nine-channel texture set can be rendered with anisotropic filtering at 1080p in 0.55 ms using 28 MB of VRAM per asset on an Intel B580.
Load-bearing premise
The load-bearing premise is that a network trained only on bilinearly filtered latents automatically handles anisotropic filtering—that because it learned to interpolate latents, any blend of latent codes will decode to the blend of the decoded outputs; if that does not hold for the many-sample blends of anisotropic filtering, the advertised rendering quality and timing are not supported.
Editorial extensions
If this is right
- Neural materials can be stored and decoded from BC1 latents with quality comparable to BC6 at equal memory, so an asset's whole nine-channel 4K texture set fits in 28 MB of VRAM.
- The VARA layout matches BCF6 quality with a small compression gain; the VARB layout roughly doubles the compression ratio with a moderate PSNR drop (about 1.5 dB in the reported example).
- Anisotropic filtering requires no retraining or special data: the decoder extrapolates from bilinear training to anisotropic taps.
- Cooperative-vector matrix multiplication accelerates MLP inference by 20-46x for wide layers, bringing 1080p material evaluation to 0.55 ms on an Intel B580.
- Tile-based classification keeps the neural cost proportional to visible neural pixels, so the method fits into a deferred shading pipeline.
Reading between the lines
- Beyond the paper, if the bilinear-to-anisotropic side effect is real, it should also cover other linear filter kernels; a stress test at high anisotropy ratios would separate genuine generalization from visual tolerance.
- Beyond the paper, the viability of BC1 suggests other LDR block formats with different rate-quality tradeoffs (BC2, BC3, or ASTC LDR) deserve the same treatment.
- Beyond the paper, a single shared scene-wide decoder—an option the authors defer—would make the tile-sorting step nearly unnecessary and is a natural fit for the same cooperative-vector hardware.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper extends the block-compressed features (BCF) method of Weinreich et al. by replacing BC6 latent textures with BC1 latents, introducing two latent-resolution schedules (VARA and VARB), and adding a quantization-aware training procedure. The runtime uses a visibility-buffer pipeline with a tile-based classification pass and cooperative-vector hardware matrix multiplication to accelerate the small MLP decoder. The authors report quality and compression-ratio comparisons against self-implemented BCF6 and NTC baselines, as well as performance measurements on an Intel B580 and an NVIDIA RTX 4090. They also claim that anisotropic filtering works as an unplanned side effect of training on filtered latents, even though no anisotropic data was used during training.
Significance. If the empirical claims hold, the paper makes a useful practical contribution: low dynamic range BC1 latents can halve the memory footprint of BC6 latents at similar quality, and the cooperative-vector tile-based implementation makes real-time neural material decoding plausible on current GPUs. Strengths include the concrete quantization-aware training description (Eqs. 1-3), the reported speedups of up to roughly 40x with cooperative vectors, and the candid admission that anisotropic filtering is a hypothesis rather than a trained behavior. The main open issues are that the anisotropic-filtering claim currently lacks quantitative support, and the headline timing of 0.55 ms is not traceable to the paper's main performance table.
major comments (4)
- [Section 3.3] The anisotropic-filtering claim rests on an explicitly stated hypothesis: the authors write that because the network learned bilinearly filtered latents, 'as a side effect it learned that blending latent codes results in blended outputs.' Figure 3 provides only a visual side-by-side comparison, and Section 3.2 states that training used trilinear filtering with no anisotropic samples. Since the runtime evaluates the MLP on a hardware-computed anisotropic sum of many bilinear taps, there is no architectural guarantee that the MLP is approximately affine over the set of latent vectors encountered under an anisotropic footprint. This is load-bearing because the abstract's 'with anisotropic filtering' claim and the 0.55 ms headline are tied to hardware anisotropic filtering in Section 4. I request a quantitative validation (e.g., PSNR/SSIM of filtered neural outputs against filtered references across multiple anisotropy levels and angles), or a revision that limits the claim to bilinear/trilinear filtering.
- [Abstract and Table 2] The abstract reports rendering at 0.55 ms on an Intel B580, but Table 2 reports 0.06 ms for the varA D=64 configuration with cooperative vectors, and the text explicitly states that Visibility and Lighting passes are skipped from those timings. The 0.55 ms value appears only in the Figure 1 caption. As written, a reader cannot tell whether 0.55 ms is a full-pipeline time, a different configuration, or a measurement on a different scene. Please report the full-pipeline timing in Table 2, or clarify what 0.55 ms includes, so that the headline number is reproducible.
- [Algorithm 1] In Algorithm 1, ClassificationB, line 54 writes to `_IndexedTilesBufferRW[tileGroupOffset * WORK_GROUP_SIZE + targetSlot]`, but `tileGroupOffset` is never defined in the listing; the variable defined two lines earlier is `mlpGroupOffset`, which is never used. As printed, the shader would not compile, and the tile-sorting logic cannot be checked. Please fix the variable name and specify the indexing scheme for `_MLPUsageBufferRW`.
- [Section 5 and Figure 5] The quality/compression-ratio comparison in Figure 5 relies entirely on self-implemented BCF6 and NTC baselines, and neither Table 1 nor Figure 5 reports error bars or the number of independent runs. Because the central claim of 'similar quality with a small gain in compression' is a comparative claim, the baselines must be trustworthy. Please validate the baselines against published numbers, provide confidence intervals over multiple training runs, or release the compression code so the comparison is independently checkable.
minor comments (4)
- [Equations (2)-(3)] Equations (2) and (3) both apply `quant(sigmoid(α), [5,6,5])`, but they should presumably apply it to `sigmoid(e0)` and `sigmoid(e1)`, respectively; this looks like a copy-and-paste typo.
- [Sections 3.3 and 7] There are several typos that should be corrected: 'anisotropicaly', 'anistropic', 'fitlering', 'surprizing', 'substancial', and 'reasonnable'.
- [References] The dataset reference [Z*] is incomplete: it lists only an author prefix and a URL, with no title, venue, or access date. Please complete it.
- [Table 2] Table 2's color key for B580 (blue) and RTX 4090 (green) is only in the caption; since the table uses slash-separated pairs, the text should explicitly state which position corresponds to which GPU.
Circularity Check
No significant circularity: the headline claims are empirical measurements or clearly labeled hypotheses, not derivations that reduce to their own inputs.
full rationale
I walked the paper's claimed derivation chain and found no circular step. The central claims are (1) BC1-stored latent textures make neural block texture compression viable, (2) this yields comparable quality at higher compression ratios than BCF6, and (3) cooperative vectors accelerate MLP evaluation. Each is supported by direct training/optimization experiments and by measured timings; no fitted parameter is renamed as a prediction, and no equation is defined in terms of the quantity it is supposed to predict. The only potentially concerning passage is Section 3.3, where anisotropic filtering is justified by an explicitly labeled hypothesis: 'we hypothetize that since the network has learned to infer the behaviour of bilinearly filtered latents, as a side effect it learned that blending latent codes results in blended outputs.' That is an unproven generalization assumption whose validation is visual only, and it is a legitimate correctness/support weakness, but it is not circular: the anisotropic-filtering claim is not used to define the training objective, nor is the hypothesis derived from the claim. Similarly, the BCF6 and NTC baselines are reimplemented in-house, which affects external benchmarking credibility but is not a circularity pattern under the stated rules. No self-citation is load-bearing, and no uniqueness theorem or prior-work ansatz is invoked to force the method's choices. The paper is self-contained against direct measurements, so the correct circularity score is 0.
Assumptions & free parameters
free parameters (8)
- Latent resolution schedule VARA =
K=4 BC1 textures at WxH, WxH, W/2xH/2, W/2xH/2
- Latent resolution schedule VARB =
WxH, W/2xH/2, W/4xH/4, W/8xH/8
- Number of latent textures K =
4
- MLP hidden dimension D =
64 default; tested 16, 32, 64
- MLP depth L =
1 hidden layer
- Sub-pixel shift pattern =
Half-texel shift on the 2nd and 4th latent textures
- Learning rates for MLP and latent textures =
MLP lr=1e-3, latent BC1 lr=1e-2
- Tile size for classification =
8x4 pixel workgroups
assumptions (5)
- domain assumption Hardware texture units correctly decode BC1 blocks and perform bilinear and anisotropic filtering on the decoded values.
- ad hoc to paper An MLP trained on bilinearly filtered inputs extrapolates to anisotropic filtering.
- domain assumption Correlated PBR channels can be represented by a shared low-dimensional latent tensor.
- standard math Straight-through estimators give usable gradients through BC1 quantization.
- domain assumption Adam optimization with the chosen learning rates converges to a good local optimum on the Polyhaven textures.
Cite this review
Pith. "Pith review of Hardware Accelerated Neural Block Texture Compression with Cooperative Vectors." pith.science (2026). https://pith.science/paper/YAJITBZT
@misc{pith2026250606040,
author = {Pith},
title = {Pith review of: Hardware Accelerated Neural Block Texture Compression with Cooperative Vectors},
year = {2026},
howpublished = {\url{https://pith.science/paper/YAJITBZT}},
note = {Machine review of arXiv:2506.06040}
}
read the original abstract
In this work, we present an extension to the neural texture compression method of Weinreich and colleagues [2024]. Like them, we leverage existing block compression methods which permit to use hardware texture filtering to store a neural representation of physically-based rendering (PBR) texture sets (including albedo, normal maps, roughness, etc.). However, we show that low dynamic range block compression formats still make the solution viable. Thanks to this, we show that we can achieve higher compression ratio or higher quality at fixed compression ratio. We improve performance at runtime using a tile based rendering architecture that leverage hardware matrix multiplication engine. Thanks to all this, we render 4k textures sets (9 channels per asset) with anisotropic filtering at 1080p using only 28MB of VRAM per texture set at 0.55ms on an Intel B580.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION fin.entry.original add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 i...
-
[3]
: Jpeg xl next-generation image compression architecture and coding tools
Alakuijala J., Van Asseldonk R., Boukortt S., Bruse M., Comșa I.-M., Firsching M., Fischbacher T., Kliuchnikov E., Gomez S., Obryk R., et al. : Jpeg xl next-generation image compression architecture and coding tools. In Applications of digital image processing XLII (2019), vol. 11137, SPIE, pp. 112--124
work page 2019
-
[4]
Burns C. A., Hunt W. A. : The visibility buffer: A cache-friendly approach to deferred shading. Journal of Computer Graphics Techniques (JCGT) 2, 2 (August 2013), 55--69. URL: http://jcgt.org/published/0002/02/04/
work page 2013
-
[5]
Ball \'e J., Laparra V., Simoncelli E. P. : End-to-end optimized image compression. In International Conference on Learning Representations (2017). URL: https://openreview.net/forum?id=rJxdQ3jeg
work page 2017
-
[6]
Enabling neural rendering in directx: Cooperative vector support coming soon, 2025. URL: https://devblogs.microsoft.com/directx/enabling-neural-rendering-in-directx-cooperative-vector-support-coming-soon/
work page 2025
-
[7]
Christopoulos C. A., Ebrahimi T., Skodras A. N. : Jpeg2000: the new still picture compression standard. In Proceedings of the 2000 ACM workshops on Multimedia (2000), pp. 45--49
work page 2000
-
[8]
: Image compression using block truncation coding
Delp E., Mitchell O. : Image compression using block truncation coding. IEEE transactions on Communications 27, 9 (1979), 1335--1342
work page 1979
Show all 23 references
-
[9]
: Efficient graphics representation with differentiable indirection
Datta S., Marshall C., Dong Z., Li Z., Nowrouzezahrai D. : Efficient graphics representation with differentiable indirection. In SIGGRAPH Asia 2023 Conference Papers (2023), pp. 1--10
2023
-
[10]
: Neural texture block compression
Fujieda S., Harada T. : Neural texture block compression. arXiv preprint arXiv:2407.09543 (2024)
2024 arXiv
-
[11]
: Neural graphics texture compression supporting random access
Farhadzadeh F., Hou Q., Le H., Said A., Rauwendaal R., Bourd A., Porikli F. : Neural graphics texture compression supporting random access. In European Conference on Computer Vision (2024), Springer, pp. 412--429
2024
-
[12]
: Compression of digital images by block truncation coding: a survey
Fr \"a nti P., Nevalainen O., Kaukoranta T. : Compression of digital images by block truncation coding: a survey. The Computer Journal 37, 4 (1994), 308--332
1994
-
[13]
: Extending in-game textures using cdns for 'call of duty: Modern warfare 2'
Fowler C. : Extending in-game textures using cdns for 'call of duty: Modern warfare 2'. GDC'23, 2023
2023
-
[14]
: Physically based shading in theory and practice
Hill S., McAuley S., Belcour L., Earl W., Harrysson N., Hillaire S., Hoffman N., Kerley L., Patry J., Piek \'e R., et al. : Physically based shading in theory and practice. In ACM SIGGRAPH 2020 Courses. 2020, pp. 1--12
2020
-
[15]
: Vk\_nv\_cooperative\_vector, 2024
Jeff B. : Vk\_nv\_cooperative\_vector, 2024. URL: https://registry.khronos.org/vulkan/specs/latest/man/html/VK_NV_cooperative_vector.html
2024
-
[16]
URL: https://docs.vulkan.org/spec/latest/appendices/compressedtex.html
Khronos : Vulkan documentation: Compressed image formats, 2025. URL: https://docs.vulkan.org/spec/latest/appendices/compressedtex.html
2025
-
[17]
: Moving frostbite to physically based rendering
Lagarde S., de Rousiers C. : Moving frostbite to physically based rendering. In SIGGRAPH 2014 Courses. ACM, 2014
2014
-
[18]
: Adaptive scalable texture compression
Nystad J., Lassen A., Pomianowski A., Ellis S., Olson T. : Adaptive scalable texture compression. In Proceedings of the Fourth ACM SIGGRAPH/Eurographics Conference on High-Performance Graphics (2012), pp. 105--114
2012
-
[19]
: PyTorch: an imperative style, high-performance deep learning library
Paszke A., Gross S., Massa F., Lerer A., Bradbury J., Chanan G., Killeen T., Lin Z., Gimelshein N., Antiga L., Desmaison A., K\" o pf A., Yang E., DeVito Z., Raison M., Tejani A., Chilamkurthy S., Steiner B., Fang L., Bai J., Chintala S. : PyTorch: an imperative style, high-pe...
2019
-
[20]
: Random-access neural compression of material textures
Vaidyanathan K., Salvi M., Wronski B., Akenine-M \"o ller T., Ebelin P., Lefohn A. : Random-access neural compression of material textures. arXiv preprint arXiv:2305.17105 (2023)
2023 arXiv
-
[21]
Wallace G. K. : The jpeg still picture compression standard. IEEE transactions on consumer electronics 38, 1 (1992), xviii--xxxiv
1992
-
[22]
: Real-time neural materials using block-compressed features
Weinreich C., De Oliveira L., Houdard A., Nader G. : Real-time neural materials using block-compressed features. In Computer Graphics Forum (2024), vol. 43, Wiley Online Library, p. e15013
2024
-
[23]
Zaal G., et al. :. URL: https://polyhaven.com/
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.