REVIEW 4 major objections 5 minor 42 references
RAVQ-HoloNet: Rate-Adaptive Vector-Quantized Hologram Compression
T0 review · 4 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash
Pith's one-line read RAVQ-HoloNet claims that a rate-adaptive vector-quantized autoencoder can compress phase-only holograms by a third of the bitrate of the prior best method, at higher PSNR, within a single network.
desk verdict Plausible rate-adaptive VQ for hologram compression, but the headline RD gain over DPRC is confounded by borrowing DPRC's phase initializer, and the abstract's SLM claim never appears in the text. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the rate-adaptive codebook: a learned VQ codebook whose size is adaptively transformed by a Seq2seq LSTM encoder-decoder, treating codebook entries like words in a sentence. This single mechanism lets the network generate codebooks of different sizes (powers of two from 512 to 4096) after one training run, so the sender and receiver only need to transmit index sequences. The rest of the architecture—hierarchical VQ-HoloNet encoders/decoders, deformable convolutions, and a band-limited angular-spectrum-method forward model in the loss—exists to make the phase-only reconstruction from discrete codes faithful, while Huffman coding of indices realizes the final bitrate
What would settle it
Run the full pipeline on intensity inputs with the borrowed phase-initializer replaced by a neutral phase retrieval method (such as Gerchberg-Saxton or a random initial phase) and recompute BD-Rate and BD-PSNR against the prior method; if the gap shrinks or disappears, the claimed advantage came from the borrowed component rather than the proposed compressor.
Extended reading notes
Core claim
RAVQ-HoloNet's claim is that vector quantization, rather than continuous-latent entropy coding, is the right compression primitive for holographic data. A complex-valued encoder maps a hologram to hierarchical latent maps; a VQ module discretizes them against a 4096-entry codebook; a Seq2seq LSTM adapter shrinks or enlarges that codebook (512-4096 entries) to set the bitrate at inference time; and a lightweight decoder emits the phase-only hologram. Because only codebook index sequences are transmitted and entropy-coded, bitrate drops without retraining. The paper reports that this system beats the prior state-of-the-art DPRC by -33.91% BD-Rate and +1.02 dB BD-PSNR in the low-bitrate domain,
Load-bearing premise
The reported rate-distortion edge over the prior method assumes that the borrowed phase-initializer U-Net from that prior method, used when the input is an intensity image, is not what actually produces the quality gain—so the comparison isolates the new compressor's own contribution.
Editorial extensions
If this is right
- One trained RA VQ-HoloNet serves a continuous range of bitrates; separate models per quality setting are unnecessary.
- At comparable visual quality, the low model uses roughly half the bitrate of the previous best method (1.787 vs 3.5 Bpp), and the ultra-low variant cuts bitrate by about 78% while losing about 13% PSNR.
- Because codebooks are precomputed and stored on both sender and receiver, real-time inference reduces to transmitting integer code indices, which are then Huffman-coded.
- The physics-based ASM loss lets the network learn phase-only outputs from complex or intensity-only inputs without ground-truth phase supervision, so the pipeline can be repurposed for other hologram formats.
Reading between the lines
- Because the codebook adapter decouples rate choices from network weights, the same idea could be dropped into other VQ-based codecs or applied to RGB-D and 3D holographic content without retraining the backbone for every rate.
- The non-uniform code-vector usage across test images suggests there is headroom in entropy coding or in learned index priors; measuring the gap between Huffman coding and an adaptive arithmetic codec on the index stream would quantify that headroom.
- A natural deployment variant is to apply the rate-adaptive codebook per spatial region, allocating larger codebooks to salient areas and smaller ones to periphery, which the paper leaves as future work.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents RAVQ-HoloNet, a hierarchical VQ-VAE for phase-only hologram compression with a Seq2seq module that adapts the codebook size to support multiple bitrates from a single trained model. The pipeline optionally starts from an intensity image, which is turned into a complex hologram using a pre-trained U-Net phase initializer taken from DPRC [15]; the complex hologram is then encoded into discrete latent vectors, quantized via EMA-updated codebooks, and decoded to a phase-only hologram. A differentiable ASM propagator is used in the loss to compare the reconstructed amplitude with the target. The authors report results on DIV2K: VQ-HoloNet-Low reaches 29.43 dB PSNR at 1.787 Bpp versus DPRC-High Quality at 29.67 dB and 3.5 Bpp, and claim a BD-Rate reduction of -33.91% and a BD-PSNR gain of +1.02 dB relative to DPRC in the low-bitrate domain. The abstract also states that SLM device experiments were performed.
Significance. If the reported comparisons are valid, the paper would make a useful contribution by demonstrating that a rate-adaptive VQ approach with discrete latents can outperform Ballé-style continuous-latent methods for hologram compression using a single model across multiple bitrates. The use of a public dataset (DIV2K), the per-layer codebook utilization analysis, and the inclusion of both generator-compressor and standalone-compressor modes are strengths. However, the central quantitative claim is undermined by a confounded comparison: the intensity-input path relies on DPRC's own phase-initializer U-Net, and no ablation isolates the contribution of the proposed compressor. In addition, the claimed SLM experiments do not appear in the paper, and all numerical results are point estimates without error bars. These issues must be addressed before the results can be taken as evidence for the paper's conclusions.
major comments (4)
- [§2.1, §3.6] The comparison with DPRC is confounded because the intensity-input variant uses 'a pre-trained U-Net from [15]' as the phase initializer. Since [15] is DPRC itself, RAVQ-HoloNet may inherit a phase prior co-adapted to DPRC's compression pipeline. The paper does not state whether the DPRC baseline uses the same initializer, and no ablation replaces the frozen U-Net with an independent initializer, random phase, or SGD phase retrieval. Thus the -33.91% BD-Rate and +1.02 dB BD-PSNR gains cannot be cleanly attributed to the proposed VQ-HoloNet compressor; an ablation with matched initializers is required.
- [Abstract, §3] The abstract claims 'experimental results on the SLM device show that our method achieves higher contrast and improved quality,' but no SLM or optical-hardware experiment appears anywhere in the full text. All reported evaluations are numerical reconstructions using ASM simulation. This claim must either be substantiated with the actual hardware results or removed from the abstract.
- [§3.6, Fig. 5] All quantitative results are point estimates from single runs with no error bars, confidence intervals, or statistical significance tests. The BD-Rate and BD-PSNR values are computed from RD curves, but the paper does not state how many rate points were used per method, how the curves were interpolated, or over which bitrate range the BD metrics were integrated. Because the central claim rests on these numbers, the authors should report variance across training seeds or a bootstrap over the test set and specify the exact BD computation procedure.
- [§3.4] The paper claims rate adaptation 'within a single network,' but §3.4 describes two distinct configurations, VQ-HoloNet Low and VQ-HoloNet Ultra-Low, with different numbers of filters, depth, and latent spatial resolutions. The Seq2seq module provides rate adaptation only within each configuration, not across the two architectures. The abstract's phrasing is therefore misleading, and the evaluation should clarify which configuration produced the reported BD-Rate/BD-PSNR numbers and whether the two configurations are meant to be viewed as one system or two separate models.
minor comments (5)
- [Title/Abstract] Inconsistent typography: 'RAVQ-HoloNet' appears in the abstract and body, while the title uses 'RA VQ-HoloNet'; the arXiv abstract also has 'RAVQ-HoloNet' with no space. Please standardize.
- [Table 1] Typo: 'VQ-HoloNet Ultera Low' should be 'Ultra Low'. Also, Table 1 reports a single Bpp for each method, but the RD curves in Fig. 5 imply multiple rate points; the relationship between Table 1 and the RD curves should be stated.
- [§3.5, Eq. (7)] Notation inconsistency: Eq. (7) uses a_MSE, a_MS-SSIM, a_WFFT, while the text defines w_MSE, w_MS-SSIM, w_WFFT. Use a single set of symbols. Also, the chosen values (1, 0.1, 0.025) are presented without any sensitivity analysis; at least a brief discussion or a sensitivity table would help.
- [Fig. 7 caption] The caption reads 'Neumerical reconstruction' — typo for 'Numerical reconstruction'. Additionally, panel (h) and (i) both say 'Low, codebook size = 4096', which appears to be a duplicate label.
- [§2.2.1] The text says the Seq2seq model converts the codebook 'from size A to size B', but in the loss formulation (Eq. 8) and the experimental setup, the Seq2seq output codebook size is constrained to powers of two during inference. Please clarify whether the training continuously samples sizes between 512 and 4096 and how the discrete inference sizes are selected.
Circularity Check
No significant circularity: the RD comparison is an empirical benchmark against an external baseline; the borrowed DPRC U-Net and the unsupported SLM claim are fairness/evidence concerns, not definitional circularity.
full rationale
RAVQ-HoloNet contains no derivation chain in which a claimed prediction is defined in terms of a fitted input or in which a load-bearing result is justified only by the authors' own prior work. The central claims—the -33.91% BD-Rate and +1.02 dB BD-PSNR improvements over DPRC—are empirical comparisons of measured rate-distortion points on DIV2K and a validation subset from [15]; no equation or construction makes these numbers equal to the training losses or codebook-fit statistics by definition. The rate-adaptive Seq2seq component is explicitly attributed to the external RAQ-VAE [22] and VQ-VAE-2 [23], not to a self-citation chain, and the VQ codebook/EMA/stagewise training is standard and disclosed. Two text passages deserve flagging but are not circular: Section 2.1 says the intensity-input mode generates an initial phase using 'a pre-trained U-Net from [15]', so the Section 3.6 full-system comparison conflates DPRC's phase-retrieval front-end with RAVQ-HoloNet's compressor and lacks an ablation of that component; and the abstract's claim of 'experimental results on the SLM device' is unsupported by any hardware experiment in the full text. These are attribution and evidence-quality problems, not instances where a result reduces to its own input, so they do not raise the circularity score.
Assumptions & free parameters
free parameters (6)
- Reconstruction loss weights w_MSE, w_MS-SSIM, w_WFFT =
1, 0.1, 0.025
- Commitment cost beta =
0.25
- EMA decay for codebook updates =
0.95
- Codebook size =
4096 base; inference sizes 512/1024/2048/4096
- Residual blocks R and channel depth =
4 residual blocks, 128 channels
- Region of interest =
700x1400
assumptions (6)
- domain assumption Band-limited Angular Spectrum Method is an adequate forward model for 20 cm propagation of RGB holograms at 6.4 µm pixel pitch.
- domain assumption A pre-trained U-Net from [15] provides a valid initial phase for intensity-only inputs.
- ad hoc to paper VQ-VAE-2-style hierarchical discrete latents with EMA codebook update and straight-through estimator train effectively with the combined MSE/MS-SSIM/Watson-DFT loss.
- ad hoc to paper Deformable convolutions improve phase-map reconstruction relative to standard convolutions.
- domain assumption Huffman coding of codebook indices is a fair proxy for entropy coding in reported Bpp.
- domain assumption DIV2K validation images (100 from [15]) are representative for holographic content.
Cite this review
Pith. "Pith review of RAVQ-HoloNet: Rate-Adaptive Vector-Quantized Hologram Compression." pith.science (2026). https://pith.science/paper/WGRFPJ4B
@misc{pith2026251121035,
author = {Pith},
title = {Pith review of: RAVQ-HoloNet: Rate-Adaptive Vector-Quantized Hologram Compression},
year = {2026},
howpublished = {\url{https://pith.science/paper/WGRFPJ4B}},
note = {Machine review of arXiv:2511.21035}
}
read the original abstract
Holography offers significant potential for AR/VR applications. However, its adoption is limited by the high demand for data compression. Existing deep learning approaches generally lack rate adaptivity within a single network and often require multiple models to cover different bandwidth requirements. We present RAVQ-HoloNet, a rate-adaptive vector quantization framework that integrates the rate-adaptive compression with the transformation of image data into phase-only hologram. RAVQ-HoloNet achieves high-fidelity reconstructions, outperforming current state-of-the-art methods implemented via two distinct architectural configurations: a standard model optimized for low bit rates and a deeper, extended variant tailored for ultra low bit rate setting. To evaluate these models, we utilized the DIV2K dataset as a benchmark for high-fidelity holographic reconstruction. Quantitative analysis in the simulation reveals that our approach significantly surpasses current benchmarks. Specifically, in the low bit rate domain, our method achieves a BD-Rate reduction of -33.91% and a BD-PSNR gain of 1.02dB relative to the state-of-the-art method. Additionally, experimental results on the SLM device show that our method achieves higher contrast and improved quality.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[15]
Joint neural phase retrieval and compression for energy-and computation-efficient holography on the edge,
Y. Wang, P. Chakravarthula, Q. Sun, and B. Chen, “Joint neural phase retrieval and compression for energy-and computation-efficient holography on the edge,”ACM Transactions on Graphics, vol. 41, no. 4, 2022
2022
-
[1]
Neural 3d holography: learning accurate wave propagation models for 3d holographic virtual and augmented reality displays,
S. Choi, M. Gopakumar, Y. Peng, J. Kim, and G. Wetzstein, “Neural 3d holography: learning accurate wave propagation models for 3d holographic virtual and augmented reality displays,”ACM Transactions on Graphics (TOG), vol. 40, no. 6, pp. 1–12, 2021
2021
-
[2]
Fast hologram synthesis methods for realistic 3D visualization,
A. Gilles, “Fast hologram synthesis methods for realistic 3D visualization,” Theses, INSA de Rennes, Sep. 2016. [Online]. Available: https: //theses.hal.science/tel-01392677
2016
-
[3]
Band-limited angular spectrum method for numerical simulation of free-space propagation in far and near fields,
K. Matsushima and T. Shimobaba, “Band-limited angular spectrum method for numerical simulation of free-space propagation in far and near fields,”Optics Express, vol. 17, no. 22, pp. 19 662–19 673, 2009
2009
-
[4]
Extremely high-definition full-parallax computer-generated hologram created by the polygon-based method,
K. Matsushima and S. Nakahara, “Extremely high-definition full-parallax computer-generated hologram created by the polygon-based method,”Ap- plied Optics, vol. 48, no. 34, pp. H54–H63, 2009
2009
-
[5]
Towards real-time photorealistic 3d holography with deep neural networks,
L. Shi, B. Li, C. Kim, P. Kellnhofer, and W. Matusik, “Towards real-time photorealistic 3d holography with deep neural networks,”Nature, vol. 591, no. 7849, pp. 234–239, 2021
2021
-
[6]
End-to-end learning of 3d phase-only holograms for holographic display,
L. Shi, B. Li, and W. Matusik, “End-to-end learning of 3d phase-only holograms for holographic display,”Light: Science & Applications, vol. 11, no. 1, p. 247, 2022
2022
-
[7]
Computer-generated double-phase holo- grams,
C.-K. Hsueh and A. A. Sawchuk, “Computer-generated double-phase holo- grams,”Applied Optics, vol. 17, no. 24, pp. 3874–3883, 1978
1978
Show all 42 references
-
[8]
Holographic near-eye displays for virtual and augmented reality,
A. Maimone, A. Georgiou, and J. S. Kollin, “Holographic near-eye displays for virtual and augmented reality,”ACM Transactions on Graphics (Tog), vol. 36, no. 4, pp. 1–16, 2017
2017
-
[9]
Neural holography with camera-in-the-loop training,
Y. Peng, S. Choi, N. Padmanaban, and G. Wetzstein, “Neural holography with camera-in-the-loop training,”ACM Transactions on Graphics (TOG), vol. 39, no. 6, pp. 1–14, 2020
2020
-
[10]
Deepcgh: 3d computer-generated holography using deep learning,
M. H. Eybposhet al., “Deepcgh: 3d computer-generated holography using deep learning,”Opt. Express, vol. 28, no. 18, pp. 26 636–26 650, Aug 20
-
[11]
Deep-learning-generated hologra- phy,
R. Horisaki, R. Takagi, and J. Tanida, “Deep-learning-generated hologra- phy,”Applied Optics, vol. 57, no. 14, pp. 3859–3863, 2018
2018
-
[12]
Three- dimensional deeply generated holography,
R. Horisaki, Y. Nishizaki, K. Kitaguchi, M. Saito, and J. Tanida, “Three- dimensional deeply generated holography,”Applied Optics, vol. 60, no. 4, pp. A323–A328, 2021
2021
-
[13]
A practical algorithm for the de- termination of phase from image and diffraction plane pictures,
R. W. Gerchberg and W. O. Saxton, “A practical algorithm for the de- termination of phase from image and diffraction plane pictures,”Optik, vol. 35, no. 2, pp. 237–246, 1972
1972
-
[14]
Jpeg pleno holography presents the numerical reconstruc- tion software for holograms: an excursion in holographic views,
B. Tobiaset al., “Jpeg pleno holography presents the numerical reconstruc- tion software for holograms: an excursion in holographic views,”Applied Optics, vol. 62, no. 10, pp. 2462–2469, 2023
2023
-
[16]
End-to-end compression-aware computer-generated holography,
M. Zhouet al., “End-to-end compression-aware computer-generated holography,”Opt. Express, vol. 31, no. 26, pp. 43 908–43 919, Dec
-
[17]
Deep learning-based compression for phase-only hologram,
H. Ko and H. Y. Kim, “Deep learning-based compression for phase-only hologram,”IEEE Access, vol. 9, pp. 79 735–79 751, 2021
2021
-
[18]
Vari- ational image compression with a scale hyperprior,
J. Ball´ e, D. Minnen, S. Singh, S. J. Hwang, and N. Johnston, “Vari- ational image compression with a scale hyperprior,”arXiv preprint arXiv:1802.01436, 2018
2018 arXiv
-
[19]
Neural compression for hologram images and videos,
L. Shi, R. Webb, L. Xiao, C. Kim, and C. Jang, “Neural compression for hologram images and videos,”Opt. Lett., vol. 47, no. 22, pp. 6013–6016, Nov 2022. [Online]. Available: https: //opg.optica.org/ol/abstract.cfm?URI=ol-47-22-6013
2022
-
[20]
Gaze-contingent efficient holo- gram compression for foveated near-eye holographic displays,
Z. Dong, Y. Ling, C. Xu, Y. Li, and Y. Su, “Gaze-contingent efficient holo- gram compression for foveated near-eye holographic displays,”Displays, vol. 79, p. 102464, 2023
2023
-
[21]
End-to-end optimized image compression,
J. Ball´ e, V. Laparra, and E. P. Simoncelli, “End-to-end optimized image compression,”arXiv preprint arXiv:1611.01704, 2016
2016 arXiv
-
[22]
Raq-vae: Rate-adaptive vector-quantized variational autoencoder,
J. Seo and J. Kang, “Raq-vae: Rate-adaptive vector-quantized variational autoencoder,”arXiv preprint arXiv:2405.14222, 2024
2024 arXiv
-
[23]
Generating diverse high-fidelity images with vq-vae-2,
R. Aliet al., “Generating diverse high-fidelity images with vq-vae-2,”Ad- vances in Neural Information Processing Systems, vol. 32, 2019. 21
2019
-
[24]
Deep compression network for enhanc- ing numerical reconstruction quality of full-complex holograms,
J. Seo, J. Lee, J. Lee, and H. Ko, “Deep compression network for enhanc- ing numerical reconstruction quality of full-complex holograms,”Optics Express, vol. 31, no. 15, pp. 24 573–24 597, 2023
2023
-
[25]
Neural discrete representation learning,
V. D. Oordet al., “Neural discrete representation learning,”Advances in Neural Information Processing Systems, vol. 30, 2017
2017
-
[26]
Long short-term memory,
S. Hochreiter and J. Schmidhuber, “Long short-term memory,”Neural com- putation, vol. 9, no. 8, pp. 1735–1780, 1997
1997
-
[27]
Sequence to sequence learning with neural networks,
I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,”Advances in Neural Information Processing Systems (NeurIPS), 2014
2014
-
[28]
Universal modeling and coding,
J. Rissanen and G. Langdon, “Universal modeling and coding,”IEEE Transactions on Information Theory, vol. 27, no. 1, pp. 12–23, 1981
1981
-
[29]
J. W. Goodman,Introduction to Fourier Optics, 3rd ed. Greenwood Vil- lage, CO: Roberts and Company Publishers, 2005
2005
-
[30]
Multiscale structural similarity for image quality assessment,
Z. Wang, E. Simoncelli, and A. Bovik, “Multiscale structural similarity for image quality assessment,” inThe Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, vol. 2, 2003, pp. 1398–1402 Vol.2
2003
-
[31]
A loss function for generative neural networks based on watson’s perceptual model,
S. Czolbe, O. Krause, I. Cox, and C. Igel, “A loss function for generative neural networks based on watson’s perceptual model,”Advances in Neural Information Processing Systems, vol. 33, pp. 2051–2061, 2020
-
[32]
Ntire 2017 challenge on single image super- resolution: Dataset and study,
E. Agustsson and R. Timofte, “Ntire 2017 challenge on single image super- resolution: Dataset and study,” inProceedings of the IEEE conference on computer vision and pattern recognition workshops, 2017, pp. 126–135
2017
-
[33]
Deformable convolutional networks,
J. Dai, H. Qi, Y. Xiong, Y. Li, G. Zhang, H. Hu, and Y. Wei, “Deformable convolutional networks,” inProceedings of the IEEE international confer- ence on computer vision, 2017, pp. 764–773
2017
-
[34]
Deformable convnets v2: More de- formable, better results,
X. Zhu, H. Hu, S. Lin, and J. Dai, “Deformable convnets v2: More de- formable, better results,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 9308–9316
2019
-
[35]
Image quality assessment: from error visibility to structural similarity,
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,”IEEE Transac- tions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004
2004
-
[36]
The unrea- sonable effectiveness of deep features as a perceptual metric,
R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unrea- sonable effectiveness of deep features as a perceptual metric,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 586–595
2018
-
[37]
Calculation of average psnr differences between rd- curves,
G. Bjontegaard, “Calculation of average psnr differences between rd- curves,”ITU-T SG16, Doc. VCEG-M33, 2001. 22
2001
-
[38]
Deep networks for saliency detection via local estimation and global search,
L. Wang, H. Lu, X. Ruan, and M.-H. Yang, “Deep networks for saliency detection via local estimation and global search,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3183–3192
2015
-
[39]
Holographic parallax improves 3d perceptual realism,
D. Kim, S.-W. Nam, S. Choi, J.-M. Seo, G. Wetzstein, and Y. Jeong, “Holographic parallax improves 3d perceptual realism,” ACM Trans. Graph., vol. 43, no. 4, Jul. 2024. [Online]. Available: https://doi.org/10.1145/3658168
2024 doi
-
[40]
Efficient hogel-based hologram synthe- sis method for holographic stereogram printing,
E. Dashdavaa, A. Khuderchuluun, H.-Y. Wu, Y.-T. Lim, C.-W. Shin, H. Kang, S.-H. Jeon, and N. Kim, “Efficient hogel-based hologram synthe- sis method for holographic stereogram printing,”Applied Sciences, vol. 10, no. 22, p. 8088, 2020. 23
2020
-
[2020]
Available: https://opg.optica.org/oe/abstract.cfm?URI= oe-28-18-26636
[Online]. Available: https://opg.optica.org/oe/abstract.cfm?URI= oe-28-18-26636
-
[2023]
Available: https://opg.optica.org/oe/abstract.cfm?URI= oe-31-26-43908
[Online]. Available: https://opg.optica.org/oe/abstract.cfm?URI= oe-31-26-43908
Reviewed August 3, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.