pith. sign in

arxiv: 2604.27689 · v1 · submitted 2026-04-30 · 📡 eess.SP

Bitwise Over-Parameterized Neural Polar Decoding: A Theoretical Performance Analysis

Hongzhi Zhu , Wei Xu , Xiaohu You This is my paper

Pith reviewed 2026-05-07 05:53 UTC · model grok-4.3

classification 📡 eess.SP
keywords polar codesneural network decodingover-parameterized networksMSE convergenceBER boundsBLER boundsGaussian approximationsuccessive cancellation
0
0 comments X

The pith

Over-parameterized neural networks enable explicit theoretical bounds on BER and BLER for polar code decoding via per-bit MSE convergence analysis.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes a bitwise over-parameterized neural network decoder for polar codes and builds a theoretical framework that links neural training dynamics to polar code reliability measures. It models each synthesized bit channel as an independent supervised regression task to keep the successive cancellation structure intact. Under over-parameterization the training trajectory stays near random initialization, and expressing empirical MSE convergence in the dB domain produces a per-iteration gain set by learning rate, bit-channel Gram spectrum, and training-set size. From this gain a population MSE bound is derived and converted, through posterior-margin structure and Gaussian approximation, into explicit BER and BLER expressions that show how hidden-layer width controls optimization and final performance.

Core claim

By treating polar decoding as a collection of bitwise supervised regressions inside an over-parameterized network, the empirical MSE convergence in the dB domain exhibits a per-iteration training gain determined by the learning rate, the bit-channel Gram spectrum, and the training-set size; local generalization analysis then yields a population MSE bound that converts, via the posterior-margin structure of the bitwise MAP target and Gaussian approximation on AWGN channels, into explicit bounds on bit error rate and block error rate.

What carries the argument

The bitwise over-parameterized neural network (ONN) decoder that models each synthesized message channel as an independent supervised regression task while preserving the successive structure of polar decoding, with the bit-channel Gram spectrum governing the per-iteration training gain.

If this is right

  • Increasing hidden-layer width improves both oracle-aided and sequential decoding performance.
  • Explicit BER and BLER bounds follow directly from the population MSE bound once the Gaussian approximation is applied.
  • The per-iteration gain supplies a quantitative link between learning rate, training-set size, and convergence speed.
  • Network-scale selection can be guided by how width affects the optimization and generalization terms in the analysis.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same bitwise regression decomposition could support theoretical performance analysis for neural decoders of other linear codes.
  • Gram-spectrum information might be used to pre-select learning rates before training begins.
  • Replacing the Gaussian approximation with other margin characterizations would extend the bounds to non-AWGN channels.
  • The framework suggests that over-parameterization provides a scalable route toward MAP-level performance in successive polar decoding.

Load-bearing premise

Each synthesized message channel can be treated as an independent supervised regression task while the successive structure of polar decoding is preserved, and the training trajectory remains close to random initialization under over-parameterization.

What would settle it

Direct measurement of the per-iteration dB-scale MSE gain in simulations and comparison against the value predicted from the learning rate, the eigenvalues of the bit-channel Gram matrix, and the training-set size, or checking whether the derived BER and BLER bounds are violated by the empirical error rates of the trained decoder.

Figures

Figures reproduced from arXiv: 2604.27689 by Hongzhi Zhu, Wei Xu, Xiaohu You.

Figure 1
Figure 1. Figure 1: End-to-end system model of the proposed bitwise ONN polar decoder. view at source ↗
Figure 3
Figure 3. Figure 3: Validation MSE on the most reliable synthesized message channel for view at source ↗
Figure 4
Figure 4. Figure 4: Maximum neuron-wise deviation of the first-layer weights from their view at source ↗
Figure 5
Figure 5. Figure 5: Empirical low-margin probability versus its GA-based prediction for view at source ↗
Figure 6
Figure 6. Figure 6: Empirical verification of the bitwise error decomposition on the view at source ↗
Figure 8
Figure 8. Figure 8: BLER performance of the proposed bitwise ONN decoder in the view at source ↗
Figure 9
Figure 9. Figure 9: BER performance of the proposed bitwise ONN decoder in the view at source ↗
Figure 10
Figure 10. Figure 10: BLER performance of the proposed bitwise ONN decoder in the view at source ↗
read the original abstract

This paper proposes a bitwise over-parameterized neural network (ONN) decoder for polar-coded transmission and develops a tractable theoretical performance analysis framework. By modeling each synthesized message channel as an individual supervised regression task, the proposed decoder preserves the successive structure of polar decoding while enabling a communication-oriented integration of neural-network learning theory and polar-code reliability analysis. Under over-parameterization, we first characterize the empirical convergence behavior of each bitwise ONN and show that the training trajectory remains close to the random initialization. By expressing the empirical MSE convergence in the dB domain, the result further reveals a per-iteration training gain determined by the learning rate, the bit-channel Gram spectrum, and the training-set size. Upon this observation, we then derive a population mean squared error (MSE) bound via local generalization analysis and convert it into a bitwise decoding error bound through the posterior-margin structure of the bitwise maximum a posteriori (MAP) target. Under additive white Gaussian noise (AWGN) channels, a Gaussian approximation (GA)-based characterization of the low-margin probability is further established, which leads to explicit bounds for the bit error rate (BER) and block error rate (BLER). The analysis clarifies how the hidden-layer width affects optimization, generalization, and the final decoding performance, thereby providing theoretical guidance for network-scale selection. Numerical results validate the main theoretical findings and show that increasing the network width consistently improves both oracle-aided and sequential decoding performance.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

4 major / 3 minor

Summary. The paper proposes a bitwise over-parameterized neural network (ONN) decoder for polar-coded transmission and develops a theoretical performance analysis framework. By modeling each synthesized message channel as an individual supervised regression task, the proposed decoder preserves the successive structure of polar decoding. Under over-parameterization, the empirical convergence behavior of each bitwise ONN is characterized with the training trajectory remaining close to random initialization. The empirical MSE convergence in the dB domain reveals a per-iteration training gain determined by the learning rate, the bit-channel Gram spectrum, and the training-set size. A population MSE bound is derived via local generalization analysis and converted into a bitwise decoding error bound through the posterior-margin structure of the bitwise MAP target. Under AWGN channels, a Gaussian approximation (GA)-based characterization of the low-margin probability leads to explicit bounds for BER and BLER. The analysis examines how hidden-layer width affects optimization, generalization, and decoding performance, with numerical results validating the main findings.

Significance. If the derivations hold, this work provides a valuable theoretical bridge between over-parameterized neural network learning theory and polar-code reliability analysis, offering explicit guidance on network-scale selection for neural decoders. The per-iteration gain expression involving the Gram spectrum and the MSE-to-BER/BLER conversion via posterior margins and GA are creative elements that could advance understanding of over-parameterization effects in communication systems. The preservation of successive cancellation structure while treating bits as independent regressions is a modeling choice with potential practical utility, provided the assumptions are rigorously justified.

major comments (4)
  1. The per-iteration training gain is expressed in terms of the bit-channel Gram spectrum and training-set size (as outlined in the convergence characterization). Clarify whether the Gram spectrum is computed theoretically from the channel model or estimated from the same training data used for the ONN; if the latter, this risks circularity in the subsequent population MSE bound and error-rate bounds, as they would depend on quantities derived from the training process itself.
  2. The modeling of each synthesized message channel as an independent supervised regression task while preserving the successive structure of polar decoding is load-bearing for the entire framework. Provide a detailed argument or theorem showing how independent per-bit training maintains the dependencies required for successive cancellation, as any violation would undermine the conversion from bitwise MSE bounds to overall decoding performance.
  3. The local generalization analysis leading to the population MSE bound, and its conversion to bitwise error bounds via the posterior-margin structure of the MAP target, requires explicit conditions and error terms. Specify how closeness to random initialization under over-parameterization enables the local bound and any approximation errors in the MAP conversion step.
  4. The GA-based characterization of the low-margin probability under AWGN, leading to explicit BER and BLER bounds, should include conditions for the approximation's validity and an analysis of its impact on the block error rate (which aggregates over multiple bits).
minor comments (3)
  1. Define all acronyms (ONN, MAP, GA, BER, BLER) on first use and ensure consistent notation for the Gram spectrum and related quantities throughout the manuscript.
  2. Numerical results should explicitly compare simulated performance against the derived theoretical bounds to allow readers to assess tightness and any gaps between analysis and practice.
  3. Add a brief discussion of related work on neural polar decoders and over-parameterized NN generalization bounds to better position the contributions.

Simulated Author's Rebuttal

4 responses · 0 unresolved

We thank the referee for the thorough and insightful review of our manuscript. The comments have prompted us to clarify several key aspects of the theoretical framework, and we have made revisions to address the concerns. Below, we provide point-by-point responses to the major comments.

read point-by-point responses

  1. Referee: The per-iteration training gain is expressed in terms of the bit-channel Gram spectrum and training-set size (as outlined in the convergence characterization). Clarify whether the Gram spectrum is computed theoretically from the channel model or estimated from the same training data used for the ONN; if the latter, this risks circularity in the subsequent population MSE bound and error-rate bounds, as they would depend on quantities derived from the training process itself.

    Authors: We appreciate this observation. The bit-channel Gram spectrum in our analysis is derived theoretically from the AWGN channel statistics and the polar code's bit-channel construction parameters, such as the synthetic channel capacities or Bhattacharyya bounds, which are known a priori and independent of the training data. The training data is used only to optimize the ONN weights, while the Gram spectrum characterizes the input feature covariance based on the channel model. This separation ensures no circularity in the population MSE and error-rate bounds. We have added a paragraph in Section III to explicitly distinguish these quantities and their sources. revision: yes

  2. Referee: The modeling of each synthesized message channel as an independent supervised regression task while preserving the successive structure of polar decoding is load-bearing for the entire framework. Provide a detailed argument or theorem showing how independent per-bit training maintains the dependencies required for successive cancellation, as any violation would undermine the conversion from bitwise MSE bounds to overall decoding performance.

    Authors: This is indeed central to our approach. The successive cancellation structure is preserved at the inference stage, where each bitwise ONN receives the channel observations and the hard decisions from previous bits as inputs, mirroring standard SC decoding. The independent training as regression tasks is justified because the target for each bit is its marginal MAP probability conditioned on the previous decisions being correct (as assumed in SC analysis). To formalize this, we have introduced a new lemma in the revised manuscript (Lemma 2 in Section IV) that bounds the difference between the jointly optimal decoder and the product of marginal regressions. The lemma shows that the total variation distance is controlled by the sum of previous bit error probabilities, allowing the bitwise MSE bounds to be propagated to the overall performance with an additive error term. This argument relies on the chain rule for the joint posterior and the fact that over-parameterized networks can approximate the marginals accurately. revision: yes

  3. Referee: The local generalization analysis leading to the population MSE bound, and its conversion to bitwise error bounds via the posterior-margin structure of the MAP target, requires explicit conditions and error terms. Specify how closeness to random initialization under over-parameterization enables the local bound and any approximation errors in the MAP conversion step.

    Authors: We agree that the conditions should be stated more explicitly. The closeness to random initialization is ensured by the over-parameterization regime, where the network width W satisfies W = Ω(n² / ε²) for training set size n and accuracy ε, leading to the training trajectory remaining in a neighborhood of radius O(1/√W) around initialization with high probability (as characterized in our convergence theorem). This enables the local generalization bound via standard Rademacher complexity arguments localized to that ball, yielding a population MSE bound of the form empirical_MSE + O(√(log W / n)) + O(1/W). For the conversion from MSE to bitwise error via posterior margins, the MAP target has a margin that is the difference in log-posteriors; the approximation error is bounded by the Lipschitz constant of the sigmoid function times the MSE, specifically |P(error) - MSE| ≤ √MSE under unit-variance normalization. We have expanded Section IV-B with these conditions, error terms, and a proof sketch for the local bound. revision: yes

  4. Referee: The GA-based characterization of the low-margin probability under AWGN, leading to explicit BER and BLER bounds, should include conditions for the approximation's validity and an analysis of its impact on the block error rate (which aggregates over multiple bits).

    Authors: The Gaussian approximation for the bit-channel LLR distributions is a standard tool in polar code analysis and holds with high accuracy when the bit-channel polarization is strong, specifically when the channel capacity I(W_i) is either close to 0 or 1 (e.g., |I(W_i) - 0.5| > 0.4), which is ensured by the polar code construction for sufficiently large block lengths N. We have added these validity conditions in the revised Section V, along with a reference to the literature on GA tightness. Regarding the impact on BLER, which is the probability of at least one information bit error, we derive the bound using a union bound over the K information bits: BLER ≤ ∑ BER_i. This is conservative but explicit; for tighter analysis, we note that under approximate independence of bit errors (valid for large N due to polarization), BLER ≈ 1 - ∏ (1 - BER_i). We have included a numerical comparison of the bound tightness in the updated simulation results and a remark discussing the aggregation effect. revision: partial

Circularity Check

0 steps flagged

Derivation chain remains self-contained; no reduction to inputs by construction

full rationale

The abstract and described framework begin with an empirical characterization of per-iteration MSE convergence in the dB domain under over-parameterization, expressed in terms of learning rate, bit-channel Gram spectrum, and training-set size. From this, a population MSE bound is derived via local generalization analysis, followed by conversion to bitwise error bounds using posterior-margin structure and Gaussian approximation for AWGN channels. These steps constitute forward theoretical derivations from stated modeling assumptions (independent bitwise regression tasks preserving successive cancellation) rather than any self-definitional loop, fitted parameter renamed as prediction, or load-bearing self-citation. The Gram spectrum is presented as a channel-derived quantity, not a post-training fit, and the numerical validation is positioned as separate confirmation rather than the source of the bounds. No quoted equation or step reduces the final BER/BLER expressions to the training data by construction.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 0 invented entities

The central claims rest on several modeling choices and approximations whose validity is asserted rather than independently verified in the abstract. The analysis treats the Gram spectrum and training-set size as given inputs to the gain formula; if these are estimated from data they function as free parameters. The Gaussian approximation for low-margin probability is introduced without stated error bounds.

free parameters (3)
  • learning rate
    Appears as a direct multiplier in the per-iteration training gain; its value is chosen by the designer and affects the derived bounds.
  • hidden-layer width
    The analysis shows its effect on optimization and generalization but does not derive an optimal value; it is a design parameter whose influence is characterized.
  • training-set size
    Enters the per-iteration gain formula; larger sets improve the bound but the exact scaling is part of the derived expression.
axioms (3)
  • domain assumption Each synthesized bit channel can be modeled as an independent supervised regression task while the overall decoder preserves the successive cancellation structure of polar decoding.
    Stated in the abstract as the modeling choice that enables the per-bit analysis.
  • domain assumption Under over-parameterization the training trajectory of each bitwise ONN remains close to random initialization.
    Central premise used to characterize empirical MSE convergence.
  • domain assumption A Gaussian approximation accurately characterizes the low-margin probability under AWGN.
    Invoked to obtain explicit BER and BLER bounds from the population MSE bound.

pith-pipeline@v0.9.0 · 5558 in / 1988 out tokens · 37165 ms · 2026-05-07T05:53:02.022870+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

25 extracted references · 25 canonical work pages

  1. [1]

    Channel polarization: A method for constructing capacity achieving codes for symmetric binary-input memoryless channels,

    E. Arikan, “Channel polarization: A method for constructing capacity achieving codes for symmetric binary-input memoryless channels,”IEEE Trans. Inf. Theory, vol. 55, no. 7, pp. 3051–3073, Jul. 2009

    work page 2009

  2. [2]

    6G: A welcome chance to unify channel coding?

    M. Geiselhart, F. Krieg, J. Clausius, D. Tandler, and S. ten Brink, “6G: A welcome chance to unify channel coding?”IEEE BITS Inf. Theory Mag., vol. 3, no. 1, pp. 67–80, Oct. 2023

    work page 2023

  3. [3]

    Channel coding for 6G extreme connectivity requirements, capabilities and fundamental tradeoffs,

    H. Zhang and W. Tong, “Channel coding for 6G extreme connectivity requirements, capabilities and fundamental tradeoffs,”IEEE BITS Inf. Theory Mag., vol. 3, no. 1, pp. 54–66, Oct. 2023

    work page 2023

  4. [4]

    Edge learning for B5G networks with distributed signal processing: Semantic communication, edge computing, and wireless sensing,

    W. Xu, Z. Yang, D. W. K. Ng, M. Levorato, Y . C. Eldar and M. Debbah, “Edge learning for B5G networks with distributed signal processing: Semantic communication, edge computing, and wireless sensing,”IEEE J. Sel. Top. Signal Process., vol. 17, no. 1, pp. 9–39, Jan. 2023

    work page 2023

  5. [5]

    On the rate of channel polarization,

    E. Arikan and E. Telatar, “On the rate of channel polarization,” inProc. IEEE Int. Symp. Inf. Theory, Seoul, Korea, Jul. 2009, pp. 1493–1495

    work page 2009

  6. [6]

    CRC-aided decoding of polar codes,

    K. Niu and K. Chen, “CRC-aided decoding of polar codes,”IEEE Commun. Lett., vol. 16, no. 10, pp. 1668–1671, Oct. 2012

    work page 2012

  7. [7]

    At the dawn of generative AI era: A tutorial-cum-survey on new frontiers in 6G wireless intelligence,

    A. Celik and A. M. Eltawil, “At the dawn of generative AI era: A tutorial-cum-survey on new frontiers in 6G wireless intelligence,”IEEE Open J. Commun. Soc., vol. 5, pp. 2433–2489, Feb. 2024

    work page 2024

  8. [8]

    Theoretical insights into the optimization landscape of over-parameterized shallow neural networks,

    M. Soltanolkotabi, A. Javanmard, and J. D. Lee, “Theoretical insights into the optimization landscape of over-parameterized shallow neural networks,”IEEE Trans. Inf. Theory, vol. 65, no. 2, pp. 742–769, Feb. 2019

    work page 2019

  9. [9]

    List decoding of polar codes,

    I. Tal and A. Vardy, “List decoding of polar codes,”IEEE Trans. Inf. Theory, vol. 61, no. 5, pp. 2213–2226, May 2015

    work page 2015

  10. [10]

    List successive cancellation decoding of polar codes,

    K. Chen, K. Niu, and J. R. Lin, “List successive cancellation decoding of polar codes,”Electron. Lett., vol. 48, no. 9, pp. 500–501, Apr. 2012

    work page 2012

  11. [11]

    Stack decoding of polar codes,

    K. Niu and K. Chen, “Stack decoding of polar codes,”Electron. Lett., vol. 48, no. 12, pp. 695–696, Jun. 2012

    work page 2012

  12. [12]

    Deep- learning-aided successive-cancellation decoding of polar codes,

    S. A. Hashemi, N. Doan, T. Tonnellier, and W. J. Gross, “Deep- learning-aided successive-cancellation decoding of polar codes,” inProc. Asilomar Conf. Signals, Syst., Comput., Pacific Grove, CA, USA, Nov. 2019, pp. 532–536

    work page 2019

  13. [13]

    Recent advances in deep learning for channel coding: A survey,

    T. Matsumine and H. Ochiai, “Recent advances in deep learning for channel coding: A survey,”IEEE Open J. Commun. Soc., vol. 5, pp. 6443–6481, Oct. 2024

    work page 2024

  14. [14]

    A new path to integrated learning and communication (ILAC): Large AI models leveraging hyperdimensional computing,

    W. Xu, Z. Yang, D. W. K. Ng, R. Schober, H. V . Poor, and Z. Zhang, “A new path to integrated learning and communication (ILAC): Large AI models leveraging hyperdimensional computing,”IEEE Trans. Commun., vol. 74, pp. 4948–4973, Feb. 2026

    work page 2026

  15. [15]

    Performance evaluation of a deep neural network joint equalizer-decoder in AWGN-ISI channels,

    Z. Joleini and A. Jamshidi, “Performance evaluation of a deep neural network joint equalizer-decoder in AWGN-ISI channels,” inProc. Int. Conf. Electr. Eng., Isfahan, Iran, May 2025, pp. 251–256

    work page 2025

  16. [16]

    On deep learning- based channel decoding,

    T. Gruber, S. Cammerer, J. Hoydis, and S. ten Brink, “On deep learning- based channel decoding,” inProc. Conf. Inf. Sci. Syst., Baltimore, MD, USA, Mar. 2017, pp. 1–6

    work page 2017

  17. [17]

    Deep learning aided BP-flip decoding of polar codes,

    Y . Lee, U. Lee, H. H. Fisseha, and M. H. Sunwoo, “Deep learning aided BP-flip decoding of polar codes,” inProc. IEEE Int. Conf. Artif. Intell. Circ. Syst., Incheon, Korea, Jun. 2022, pp. 114–117

    work page 2022

  18. [18]

    Mathematical models of overparam- eterized neural networks,

    C. Fang, H. Dong, and T. Zhang, “Mathematical models of overparam- eterized neural networks,”Proc. IEEE, vol. 109, no. 5, pp. 683–703, May 2021

    work page 2021

  19. [19]

    Gradient descent provably optimizes over-parameterized neural networks,

    S. S. Du, X. Zhai, B. Poczos, and A. Singh, “Gradient descent provably optimizes over-parameterized neural networks,” inProc. Int. Conf. Learn. Represent., New Orleans, Louisiana, USA, May 2019, pp. 1– 19

    work page 2019

  20. [20]

    Generalization bounds of stochastic gradient descent for wide and deep neural networks,

    Y . Cao and Q. Gu, “Generalization bounds of stochastic gradient descent for wide and deep neural networks,” inProc. Int. Conf. Neural Inf. Process. Syst., Vancouver, BC, Canada, Dec. 2019, pp. 10836–10846

    work page 2019

  21. [21]

    Generalization error bounds of gradient descent for learning over-parameterized deep ReLU networks,

    Y . Cao and Q. Gu, “Generalization error bounds of gradient descent for learning over-parameterized deep ReLU networks,” inProc. AAAI Conf. Artif. Intell., Hilton New York Midtown, New York, USA, Feb. 2020, pp. 3349–3356

    work page 2020

  22. [22]

    Deep learning: A statistical viewpoint,

    P. L. Bartlett, A. Montanari, and A. Rakhlin, “Deep learning: A statistical viewpoint,”Acta Numerica, vol. 30, pp. 87–201, 2021

    work page 2021

  23. [23]

    Efficient design and decoding of polar codes,

    P. Trifonov, “Efficient design and decoding of polar codes,”IEEE Trans. Commun., vol. 60, no. 11, pp. 3221–3227, Nov. 2012

    work page 2012

  24. [24]

    Rademacher and Gaussian complex- ities: Risk bounds and structural results,

    P. L. Bartlett and S. Mendelson, “Rademacher and Gaussian complex- ities: Risk bounds and structural results,”J. Mach. Learn. Res., vol. 3, pp. 463–482, Nov. 2002

    work page 2002

  25. [25]

    Local Rademacher complexities,

    P. L. Bartlett, O. Bousquet, and S. Mendelson, “Local Rademacher complexities,”Ann. Statist., vol. 33, no. 4, pp. 1497–1537, 2005

    work page 2005