REVIEW 3 major objections 6 minor 10 references
Neural Network-based Information-Theoretic Transceivers for High-Order Modulation Schemes
T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read This paper argues that a trained bitwise NN demapper and a symbol-wise autoencoder-CNN transceiver match or beat conventional APP-based LLR receivers under AWGN, at lower per-symbol complexity.
desk verdict Competent but incremental neural-demapper/AE paper whose theoretical BER curves are wrong (Eq. 29), undermining the main validation claim; fixable but needs major revision. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The key machinery is the replacement of explicit likelihood summation by learned feedforward mappings. The APP LLR receiver sums over $M/2$ constellation points per bit, with complexity $O(M \log M)$ per symbol; the NN demapper replaces this with three fully connected layers that map the real and imaginary parts of a received symbol to $k$ LLR values. The AE-CNN extends the idea to joint optimization: a one-hot input goes through convolutional encoder layers to a normalized complex symbol, the AWGN channel acts as a differentiable bottleneck, and a convolutional decoder with softmax outputs symbol posteriors, trained end to end with categorical cross-entropy. The claimed complexity advantage is $O(k N C_{\mathrm{out}})$ for the AE-CNN versus $O(2^{2k} N)$ for a fully connected AE-DNN, which is why the CNN structure is presented as the practical route to higher modulation orders.
What would settle it
Recompute the theoretical SER and BER curves for the same square $M$-QAM constellations using the exact union-bound or standard Gray-coded bit-error expressions, overlay the proposed NN curves and the APP-based Monte Carlo baseline, and check whether the NN curves still track theory and the baseline; if they diverge, the validation claim fails. A sharper test: measure wall-clock inference time per symbol for the AE-CNN and the APP LLR receiver at $M=256$; failure to match or beat both error rate and complexity would falsify the superiority claim.
Extended reading notes
Core claim
The central discovery is that a trained feedforward network can approximate the bitwise log-likelihood-ratio outputs of an APP-based soft-decision demapper for square $M$-QAM, and that a convolutional autoencoder trained with symbol-wise cross-entropy can jointly optimize the transmitter and receiver so that the end-to-end system meets or beats the conventional baseline. The bitwise demapper is a regression model: it takes the real and imaginary parts of a received symbol and outputs $k$ sigmoid-adjusted LLR estimates, trained with binary cross-entropy against ground-truth bits. The AE-CNN transceiver instead treats transmission as a classification problem: a one-hot symbol index is mapped by the encoder to a normalized complex baseband signal, passed through an AWGN bottleneck, and decoded by a CNN whose softmax output estimates the posterior over all $M$ symbols, trained with categorical cross-entropy. The reported experiments show the NN-based curves tracking theoretical and Monte Carlo curves, with the AE-based systems outperforming the conventional receiver at low SNR and remaining comparable at high SNR, especially as $M$ grows.
Load-bearing premise
The load-bearing premise is that the theoretical validation is sound: Remark 1 uses a closed-form square $M$-QAM SER formula (Eq. 28) without showing its derivation, and Remark 2 converts SER to BER via $BER=(M/2)/(M-1)\,SER$ instead of the standard Gray-coded approximation $BER\approx SER/\log_2 M$, so if either formula is incorrect or misapplied, the claimed corroboration of the NN results by theory in Figs. 1–2 collapses.
Editorial extensions
If this is right
- A neural demapper can serve as a drop-in soft-decision front end for high-order QAM, delivering LLR outputs at feedforward cost instead of $O(M \log M)$ per-symbol likelihood sums.
- Jointly training the transmitter and receiver as a symbol-wise autoencoder gives BER/SER at least comparable to an APP-based LLR receiver, with gains at low SNR and high modulation orders.
- The SNR chosen during training is a performance-critical hyperparameter: models trained at different $E_b/N_0$ dominate in different inference SNR regimes.
- The reported agreement among NN-based, theoretical, and Monte Carlo curves indicates NN-based transceivers can act as digital twins of conventional information-theoretic systems under AWGN.
- Because the AE-CNN has lower per-symbol complexity than a fully connected AE, the CNN structure is the practical choice for scaling to higher modulation orders.
Reading between the lines
- If the training-SNR sensitivity observed here holds more generally, deployed systems should train at multiple SNR points or use SNR-averaged training; the paper's single-SNR training setup may understate the achievable gains.
- The BER conversion in Eq. (29) is not the standard Gray-coded QAM approximation, so before relying on the numerical corroboration, the figures should be re-checked with the usual $BER\approx SER/\log_2 M$ rule.
- The same AE-CNN structure could be trained on fading or nonlinear channels, where the learned encoder might reshape the constellation beyond standard QAM; the paper only tests AWGN.
- A direct wall-clock complexity benchmark at $M=256$ would make the claimed computational advantage concrete, since the paper's complexity comparison is analytic rather than measured.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes two NN-based receiver designs for uncoded AWGN channels with M-QAM: a bitwise fully-connected demapper trained with binary cross-entropy to approximate APP-based LLR soft outputs, and a symbol-wise AE-CNN (with an AE-DNN variant) end-to-end transceiver trained with categorical cross-entropy. It compares BER/SER against a Sionna APP-based LLR receiver and against closed-form formulas stated in Remarks 1 and 2, and it studies how the training Eb/N0 affects inference performance. The central claim is that the NN-based systems match or outperform conventional APP-based reception, especially at higher modulation orders, while offering lower computational complexity.
Significance. If the claims are established, the paper would provide a useful empirical demonstration that learned demappers and AE transceivers can approach conventional APP-based soft-decision receivers under AWGN without explicit noise-variance knowledge, and the public code release is a positive step for reproducibility. However, the validation is currently weakened by a likely incorrect theoretical BER benchmark and by omitted derivations and hyperparameter details, so the strength of the central claims is not yet established.
major comments (3)
- [Remark 1, Eq. (28) (Section IV-A)] The SER formula in Eq. (28) is presented without derivation and without definitions of P_s, B, and R, so the expression cannot be independently checked or reproduced from the manuscript. Since Section IV-B invokes this formula to claim that the numerical results are 'well corroborated by theoretical and Monte Carlo simulation results', the theoretical validation in Figs. 1 and 2 rests on an unauditable expression; the authors must either provide a full derivation or replace the benchmark with a standard, fully specified M-QAM SER expression.
- [Remark 2, Eq. (29) (Section IV-A)] The conversion BER = (M/2)/(M-1) * SER in Eq. (29) is the relation for M-ary orthogonal signaling under uniformly distributed symbol errors, not for Gray-coded square M-QAM, where symbol errors are concentrated on nearest neighbors and BER is approximately SER/log2(M). For 16-QAM this conversion overstates BER by roughly a factor of 2, and for 256-QAM by roughly a factor of 4. Because the theoretical BER curves in Fig. 1 and the derived 'well corroborated' claim rely on this conversion, the theoretical benchmark is likely wrong; the authors should redo the theoretical curves with the Gray-coded QAM bit-error approximation (or with Monte Carlo BER) and then re-examine the claimed agreement and the high-order superiority claims.
- [Section III-B-2 and Abstract] The abstract claims that the NN-based bitwise receiver 'improves computational efficiency', and Section III-B-2 gives a complexity comparison for AE-CNN versus AE-DNN, but the comparison is not demonstrated for the proposed NN demapper, and the AE complexity analysis is explicitly omitted ('In our analysis (which is omitted here)'). No measured runtime, parameter count, FLOPs, or latency results are reported, so the claimed computational-efficiency advantage is not supported by the presented evidence; this needs either a proper complexity derivation or experimental measurements.
minor comments (6)
- [Section IV-B] The statement that numerical NN results are 'well corroborated by theoretical and Monte Carlo simulation results' would be more convincing with a quantitative agreement metric or with confidence intervals, rather than only a qualitative visual comparison.
- [Equation (16)] The sentence around Eq. (16) says 'W_l is called the weight and b_l are called the weight and bias'; the first 'weight' should be 'weight matrix', and the phrasing should be corrected for clarity.
- [References] References [1] and [7] are the same paper by O'Shea and Hoydis; this duplication should be removed and the reference list checked for other inconsistencies.
- [Title and terminology] The title and several passages use 'information-theoretic', but the paper does not compute any information-theoretic quantity such as mutual information or capacity; consider rewording to avoid overclaiming.
- [Training details] Key training hyperparameters (layer widths, kernel counts, learning rate, batch size, and the exact training Eb/N0 values) are not given in the text and are deferred to the GitHub repository; for a journal version these settings should be summarized in a table or in the experimental section.
- [Figure 3 and robustness claim] The claim that the AE-based transceiver exhibits greater robustness than the NN-based demapper across varying SNR levels is deferred to the repository; this should be supported with explicit curves or a numerical comparison in the paper.
Circularity Check
No circularity: NN outputs are validated against independent external benchmarks; a questionable theoretical BER formula is a correctness risk, not a circular step.
full rationale
The paper contains no load-bearing circular step. The NN-based bitwise demapper is trained with BCE against ground-truth transmitted bits (Eq. 19), and the AE-based transceiver is trained with CCE against one-hot symbol labels (Eq. 27); their BER/SER is then measured by Monte Carlo comparison with the transmitted data. The conventional baseline is an APP-based LLR receiver implemented in the external Sionna library [9], and the theoretical benchmarks are closed-form SER/BER expressions (Eqs. 28-29). None of these benchmarks is produced by the NN itself, and no NN parameter is fitted to the benchmark curves. The theoretical SER expression in Remark 1 is asserted without derivation and the SER-to-BER conversion in Remark 2 is questionable for Gray-coded M-QAM, but that is an external-validity/correctness concern, not a case of a result reducing to its own inputs. References [1]-[9] are prior external works, not self-citations carrying the argument. The paper therefore does not exhibit any of the circularity patterns: its predictions and validations are structurally separate from its inputs.
Assumptions & free parameters
free parameters (2)
- Training Eb/N0 =
8 dB and 12 dB (per Fig. 3 discussion)
- Network hyperparameters (layer widths, kernel counts, learning rate, batch size) =
Not stated in the manuscript; deferred to GitHub repository
assumptions (5)
- domain assumption AWGN channel model with circularly symmetric complex Gaussian noise and known noise variance for the baseline.
- domain assumption Square M-QAM constellation with Gray labeling and unit average energy normalization.
- domain assumption Eqs. (28) and (29) are correct closed-form SER/BER benchmarks for square M-QAM.
- domain assumption Minimizing BCE and CCE with the stated network architectures yields a soft demapper and decoder that approximate MAP decisions.
- domain assumption The Sionna APP-based LLR implementation used as baseline is correct and configured to match the AWGN assumptions.
Cite this review
Pith. "Pith review of Neural Network-based Information-Theoretic Transceivers for High-Order Modulation Schemes." pith.science (2026). https://pith.science/paper/VTUQJMOY
@misc{pith2026250600368,
author = {Pith},
title = {Pith review of: Neural Network-based Information-Theoretic Transceivers for High-Order Modulation Schemes},
year = {2026},
howpublished = {\url{https://pith.science/paper/VTUQJMOY}},
note = {Machine review of arXiv:2506.00368}
}
read the original abstract
Neural network (NN)-based end-to-end (E2E) communication systems, in which each system component may consist of a portion of a neural network, have been investigated as potential tools for developing artificial intelligence (Al)-native E2E systems. In this paper, we propose an NN-based bitwise receiver that improves computational efficiency while maintaining performance comparable to baseline demappers. Building on this foundation, we introduce a novel symbol-wise autoencoder (AE)-based E2E system that jointly optimizes the transmitter and receiver at the physical layer. We evaluate the proposed NN-based receiver using bit-error rate (BER) analysis to confirm that the numerical BER achieved by NN-based receivers or transceivers is accurate. Results demonstrate that the AE-based system outperforms baseline architectures, particularly for higher-order modulation schemes. We further show that the training signal-to-noise ratio (SNR) significantly affects the performance of the systems when inference is conducted at different SNR levels.
Figures
Reference graph
Works this paper leans on
-
[1]
An introduction to deep learning for the physical layer,
T. O’shea and J. Hoydis, “An introduction to deep learning for the physical layer,” IEEE Transactions on Cognitive Communications and Networking, vol. 3, no. 4, pp. 563–575, 2017
2017
-
[2]
Deep learning based communication over the air,
S. D ¨orner, S. Cammerer, J. Hoydis, and S. Ten Brink, “Deep learning based communication over the air,” IEEE Journal of Selected Topics in Signal Processing, vol. 12, no. 1, pp. 132–143, 2017
work page 2017
-
[3]
Trainable communication systems: Concepts and prototype,
S. Cammerer, F. A. Aoudia, S. D ¨orner, M. Stark, J. Hoydis, and S. t. Brink, “Trainable communication systems: Concepts and prototype,”
-
[4]
"Machine LLRning": Learning to Softly Demodulate
O. Shental and J. Hoydis, “”machine llrning”: Learning to softly demodulate,” CoRR, vol. abs/1907.01512, 2019. [Online]. Available: http://arxiv.org/abs/1907.01512
work page Pith review arXiv 1907
-
[5]
S. Zheng, S. Chen, and X. Yang, “Deepreceiver: A deep learning-based intelligent receiver for wireless communications in the physical layer,” IEEE Transactions on Cognitive Communications and Networking, vol. 7, no. 1, pp. 5–20, 2020
work page 2020
-
[6]
Demodnet: Learning soft demodulation from hard information using convolutional neural network,
S. Zheng, X. Zhou, S. Chen, P. Qi, C. Lou, and X. Yang, “Demodnet: Learning soft demodulation from hard information using convolutional neural network,” in ICC 2022-IEEE International Conference on Com- munications. IEEE, 2022, pp. 1–6
work page 2022
-
[7]
An introduction to deep learning for the physical layer,
T. O’Shea and J. Hoydis, “An introduction to deep learning for the physical layer,” IEEE Transactions on Cognitive Communications and Networking, vol. 3, no. 4, pp. 563–575, 2017
2017
-
[8]
Joint transceiver optimization for wireless communication phy using neural network,
B. Zhu, J. Wang, L. He, and J. Song, “Joint transceiver optimization for wireless communication phy using neural network,” IEEE Journal on Selected Areas in Communications , vol. 37, no. 6, pp. 1364–1373, 2019
work page 2019
Show all 10 references
-
[9]
Sionna: An Open-Source Library for Next- Generation Physical Layer Research,
J. Hoydis et al., “Sionna: An Open-Source Library for Next- Generation Physical Layer Research,” 2022. [Online]. Available: https://arxiv.org/abs/2203.11854
2022 arXiv
-
[2019]
Available: https://arxiv.org/abs/1911.13055
[Online]. Available: https://arxiv.org/abs/1911.13055
1911 arXiv
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.