REVIEW 3 major objections 4 minor 18 references
On the Worst-Case Complexity of Gibbs Decoding for Reed--Muller Codes
T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read The paper establishes that the Gibbs decoder has super-polynomial worst-case mixing time for Reed–Muller codes over the binary symmetric channel.
desk verdict A genuine worst-case slow-mixing result for Gibbs decoding of RM codes, but Theorem 3 as stated overclaims; the core bound survives with rate-dependent constants. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The proof is carried by two mechanisms. The Gibbs chain on the message space has stationary distribution $\mu(m)\propto \theta^{d_H(mG,y)}$ with $\theta=p/(1-p)<1$, so posterior mass is exponentially concentrated near codewords close to $y$; a step flips one uniformly chosen message bit with probability $\mu(m\oplus i)/(\mu(m)+\mu(m\oplus i))$. The bottleneck ratio (conductance) of a set $S$ is $\Phi(S)=\sum_{s_1\in S,s_2\notin S}\mu(s_1)P(s_1\to s_2)/\mu(S)$, and the standard bound $T_{\rm mix}\ge 1/(4\Phi(P))$ lets the proof use the single state $S=\{0\}$: its conductance is just $1-P(0\to 0)\le (1/k)\sum_i \theta^{\delta_i}$, where $\delta_i=d_H(e_iG,y)-d_H(0,y)$. The constructed polynomial $f_y$ makes $\delta_i\ge 2^{m-r-q+1}$ for every generator row, and the normal approximation to the binomial distribution converts the rate condition $R_j\to R$ into $r=m/2+c\sqrt{m}/2$, which turns $2^{m-r-q}$ into $\sqrt{n}\,\exp(-\sqrt{\log n})$ and hence, through the bottleneck lemma, into the mixing-time lower bound $\exp(\Omega(\sqrt{n}\,\exp(-\sqrt{\log n})))$.
What would settle it
Take the sequence $RM(0,m)$ with any $p<1/2$: the message space has only two states, the Gibbs transition matrix is $2\times 2$, and direct diagonalization gives $T_{\rm mix}=O(1)$ independent of $m$, contradicting the claimed super-polynomial lower bound. This pins the missing hypothesis as $R>0$ (equivalently $r\ge 1$).
Extended reading notes
Core claim
The central claim is that, for any crossover probability $p<1/2$ and any rate $R<1-H(p)$, every sequence of Reed–Muller codes with unbounded length and rate approaching $R$ admits received words for which the Gibbs decoder mixes in time $\Omega(\exp(\sqrt{n}\cdot \exp(-\sqrt{\log n})))$, which is super-polynomial in the blocklength $n=2^m$. The proof constructs the received word as the evaluation of the polynomial $f_y(z_1,\dots,z_m)=(\prod_{i=1}^{q-1}z_i+1)(z_m+1)$ with $q=\lceil\log_2(1/p)\rceil$. For this word, the all-zero message has posterior probability at most $1/2$, while flipping any one message bit changes the Hamming distance to $y$ by at least $2^{m-r-q+1}$; the Gibbs chain must cross this exponentially small escape probability, and the bottleneck-ratio bound converts it into the super-polynomial mixing-time lower bound. In the paper's own terms, this is a worst-case lower bound on the complexity of Gibbs decoding, not a statement about average received words.
Load-bearing premise
The proof assumes the normal approximation that turns the code rate $R$ into the code order $r$ is accurate enough, without quantifying the error, and the approximation fails at $R=0$, where the stated theorem would be false; the claim therefore implicitly requires $R>0$ (or $r\ge 1$).
Editorial extensions
If this is right
- For every $p<1/2$ and every rate below capacity, some conditionally typical received word makes the Gibbs chain take $\Omega(\exp(\sqrt{n}\,\exp(-\sqrt{\log n})))$ steps to mix, so no universal polynomial worst-case guarantee is possible for this decoder.
- The bad received words are not arbitrary adversarial noise; each lies in the conditional typical set for a codeword, so the slow mixing can occur on outputs the channel would actually produce.
- Because the only property used is an energy gap under single-bit flips, any MCMC decoder whose updates resample one message bit from the posterior will inherit the same exponential mixing lower bound for the same constructed words.
- The result does not settle average-case complexity: the constructed received word may have vanishing probability, and the paper explicitly leaves open the possibility that typical received sequences mix quickly on average.
Reading between the lines
- The bottleneck construction is likely not special to Reed–Muller structure: any linear code whose generator rows have weights growing like $2^{m-r}$ and that contains a codeword at distance $2^{m-q}$ from a suitable received word would yield the same lower bound through the same bottleneck argument.
- A direct check at rate zero shows the theorem needs an $R>0$ (or $r\ge 1$) hypothesis; replacing the normal approximation with an exact binomial tail bound would make the statement airtight for positive rates and quantify the constant in the exponent.
- Empirically, the bound may be loose: running the Gibbs chain on the constructed received word at moderate blocklengths would show whether the bottleneck at the all-zero message is as severe as the exponent suggests, and whether annealed or block-update variants escape it.
- If average-case mixing were later shown to be polynomial, then combining that with the posterior-sampling equivalence lemma would produce a polynomial-time capacity-achieving posterior-sampling decoder for Reed–Muller codes; the present worst-case result would not contradict it.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper studies the mixing time of the Gibbs decoder (Glauber dynamics on the message space) for Reed–Muller codes over the binary symmetric channel. The main result, Theorem 3, claims that for any crossover probability p < 1/2 and any asymptotic rate R < 1 - H(p), there exists a sequence of conditionally typical received sequences for which the mixing time of the Gibbs decoder is Ω(exp(√n · exp(−√log n))), i.e., super-polynomial. The proof combines a conductance-based bottleneck lemma (Lemma 3) with an explicit construction of a received vector y (Lemma 4) such that the all-zero message has posterior mass at most 1/2 and every single-bit flip increases the Hamming distance to y by at least 2^{m−r−q+1}, where q = ⌈log_2(1/p)⌉. The final asymptotic step converts r ≈ m/2 + c√m/2 into a lower bound on the mixing time using the normal approximation to the binomial distribution.
Significance. If the main theorem were correct as stated, it would be a notable worst-case lower bound: it would show that a natural MCMC decoder for RM codes can require super-polynomial time to converge to the posterior, in contrast to the capacity-achieving MAP decoding of RM codes. The construction in Lemma 4 is explicit and the conductance argument is self-contained. However, as written the theorem is not proven: the proof drops a rate-dependent exponent, and the statement is false for R = 0. The qualitative super-polynomial conclusion appears salvageable after restricting the rate range and carrying the rate-dependent constants, but the paper in its current form does not establish the stated uniform bound.
major comments (3)
- [Theorem 3, proof] The passage from the bound δ_i ≥ 2^{m−r−q+1} to the claimed log T_mix = Ω(√n exp(−√log n)) silently drops the dependence on c = Φ^{-1}(R). Substituting r = m/2 + (c/2)√m into the Lemmas 3–4 lower bound gives log T_mix = Ω(2^{m/2 − (c/2)√m − q + 1}) = Ω(√n · 2^{−q+1} · exp(−(c√ln 2 / 2)√log n)). The claimed uniform bound holds only when c√ln 2 / 2 ≤ 1, i.e., R ≤ Φ(2/√ln 2) ≈ 0.9918. For p small enough that 1 − H(p) > 0.9918, the theorem's assumptions allow rates in (0.9918, 1 − H(p)) for which the proven bound is exp(√n · exp(−C√log n)) with C > 1, which is not Ω(exp(√n · exp(−√log n))) because the ratio of the exponents tends to zero. Please either restrict the theorem to the regime c√ln 2 / 2 ≤ 1 or state the rate-dependent bound; the latter still yields the advertised super-polynomiality for every fixed R < 1 − H(p).
- [Theorem 3 (statement), Lemma 4] The theorem claims any rate R < 1 − H(p), which includes R = 0. For the sequence RM(0,m) (r = 0, k = 1), the message space has two states and the Gibbs chain has P(0→1) = µ(1), P(1→0) = µ(0); after one step the distribution is exactly the stationary distribution, so T_mix = 1, contradicting the claimed lower bound. The proof fails at Lemma 4(2): the codeword c = Eval(z_m + 1) used for the comparison message u is not in RM(0,m), so the posterior comparison µ(u) ≥ µ(0) is unavailable; indeed µ(0) > 1/2 in that case. The theorem and Lemma 4 must exclude r = 0 (or state a separate treatment).
- [§IV, Eq. (2)] Equation (2) treats the normal approximation to the binomial CDF as an exact identity and inverts it to obtain r = m/2 + (c/2)√m. The paper does not quantify the approximation error. Since the exponent of the mixing-time lower bound depends on r through √n · 2^{−r}, an unquantified O(1) error in r would change the constant in front of √log n, so the asymptotic conclusion is not rigorous as written. A Berry–Esseen bound (or a direct large-deviation estimate) should be invoked to justify the expansion for the fixed-rate sequences under consideration.
minor comments (4)
- [Lemma 4 statement] The statement says "there exists a non-zero vector y ∈ F_m^2"; this should read y ∈ F_2^{2^m} (or F_2^n).
- [Lemma 4 statement (3)] The notation "d/(2q−1)" is missing an exponent; it should be d/2^{q−1}.
- [Theorem 3 proof] The bound is written as δ_i ≥ 2^{m−r−q}, whereas Lemma 4 gives 2^{m−r−q+1}; the exponent is off by one. This does not change the asymptotics but should be corrected.
- [Theorem 3 proof] The formula "k = (m ≤ r)" in the proof of Theorem 3 is typeset incorrectly; it should be the binomial sum ∑_{i=0}^r \binom{m}{i}.
Circularity Check
No significant circularity: the mixing-time lower bound follows from standard conductance estimates and an explicit code/channel-output construction, with no fitted parameter renamed as a prediction.
full rationale
The paper's central claim is a worst-case lower bound on the Gibbs decoder's mixing time for Reed--Muller codes. The derivation chain is self-contained: Lemma 3 reduces slow mixing to a bottleneck ratio bound, using the standard Levin--Peres theorem Tmix >= 1/(4Phi(P)); Lemma 4 explicitly constructs the received word y = Eval(f_y) and computes the required quantities wt(y), dH(c,y), and wt(y+g_i)-wt(y) exactly from the polynomial structure. No step takes as input the conclusion it claims to derive, and no fitted parameter is later called a prediction. The cited Liu--Cuff--Verdú lemma on posterior sampling appears only as motivation for why sampling from the posterior is relevant; the proof of Theorem 3 does not depend on it. There are no load-bearing self-citations: the references to Huang's thesis and Huang--Kim are contextual comparisons, not justifications of the main theorem. The asymptotic step in Theorem 3 does use the binomial CLT to express r in terms of R, which is a legitimate (though approximate) input-to-parameter relation, not a circular reuse of the claimed bound. Separate concerns about the theorem as stated — the silent dropping of the rate-dependent coefficient c = Φ^{-1}(R) and the q factor, and the R=0 counterexample — are correctness issues, not circularity under the requested taxonomy. Therefore the circularity score is 0.
Assumptions & free parameters
assumptions (4)
- standard math Ergodic finite Markov chains converge to a unique stationary distribution (Theorem 1 in the paper).
- standard math Bottleneck ratio bound: Tmix ≥ 1/(4Φ(P)) (Theorem 2 in the paper).
- domain assumption Binomial CDF asymptotic normal approximation: Pr[Bin(m,0.5) ≤ r] ≈ Φ((r-m/2)/√(m/4)).
- domain assumption For any rate R in (0,1), there exists a sequence of RM codes RM(r_j,m_j) with m_j→∞ and rate → R.
Cite this review
Pith. "Pith review of On the Worst-Case Complexity of Gibbs Decoding for Reed--Muller Codes." pith.science (2026). https://pith.science/paper/JYYJXCAC
@misc{pith2026250500974,
author = {Pith},
title = {Pith review of: On the Worst-Case Complexity of Gibbs Decoding for Reed--Muller Codes},
year = {2026},
howpublished = {\url{https://pith.science/paper/JYYJXCAC}},
note = {Machine review of arXiv:2505.00974}
}
read the original abstract
Reed--Muller (RM) codes are known to achieve capacity on binary symmetric channels (BSC) under the Maximum a Posteriori (MAP) decoder. However, it remains an open problem to design a capacity achieving polynomial-time RM decoder. Due to a lemma by Liu, Cuff, and Verd\'u, it can be shown that decoding by sampling from the posterior distribution is also capacity-achieving for RM codes over BSC. The Gibbs decoder is one such Markov Chain Monte Carlo (MCMC) based method, which samples from the posterior distribution by flipping message bits according to the posterior, and can be modified to give other MCMC decoding methods. In this paper, we analyze the mixing time of the Gibbs decoder for RM codes. Our analysis reveals that the Gibbs decoder can exhibit slow mixing for certain carefully constructed sequences. This slow mixing implies that, in the worst-case scenario, the decoder requires super-polynomial time to converge to the desired posterior distribution.
Reference graph
Works this paper leans on
-
[1]
Application of boolean algebra to switchi ng circuit design and to error detection,
D. E. Muller, “Application of boolean algebra to switchi ng circuit design and to error detection,” Transactions of the I.R.E. Professional Group on Electronic Computers , vol. EC-3, no. 3, pp. 6–12, 1954
work page 1954
-
[2]
A class of multiple-error-correcting codes an d the decoding scheme,
I. Reed, “A class of multiple-error-correcting codes an d the decoding scheme,” Transactions of the IRE Professional Group on Information Theory, vol. 4, no. 4, pp. 38–49, 1954
1954
-
[3]
Reed–muller codes achieve capacity on erasure ch annels,
S. Kudekar, S. Kumar, M. Mondelli, H. D. Pfister, E. ¸ Sa¸ so ˇglu, and R. L. Urbanke, “Reed–muller codes achieve capacity on erasure ch annels,” IEEE Transactions on Information Theory , vol. 63, no. 7, pp. 4298– 4316, 2017
work page 2017
-
[4]
Reed–muller codes on bms chan nels achieve vanishing bit-error probability for all rates belo w capacity,
G. Reeves and H. D. Pfister, “Reed–muller codes on bms chan nels achieve vanishing bit-error probability for all rates belo w capacity,” IEEE Transactions on Information Theory , vol. 70, no. 2, pp. 920–949, 2024
2024
-
[5]
A proof that Reed-Muller codes achieve Shannon capacity on symmetric channels
E. Abbe and C. Sandon, “A proof that reed-muller codes ach ieve shannon capacity on symmetric channels,” 2023. [Online]. A vailable: https://arxiv.org/abs/2304.02509
work page Pith review arXiv 2023
-
[6]
R. R. Green, “A serial orthogonal decoder,” in JPL Space Programs Summary, vol. 37, 1966, pp. 247–253
work page 1966
-
[7]
Recursive projection-aggregation de coding of reed- muller codes,
M. Y e and E. Abbe, “Recursive projection-aggregation de coding of reed- muller codes,” in 2019 IEEE International Symposium on Information Theory (ISIT) , 2019, pp. 2064–2068
work page 2019
-
[8]
An Analysis of RPA Decoding of Reed-Muller Codes Over the BSC
V . A. Rameshwar and V . Lalitha, “An upper bound on the erro r probability of rpa decoding of reed-muller codes over the bs c,” 2024. [Online]. Available: https://arxiv.org/abs/2412.08129
work page Pith review arXiv 2024
Show all 18 references
-
[9]
Recursive decoding and its performance for lo w-rate reed- muller codes,
I. Dumer, “Recursive decoding and its performance for lo w-rate reed- muller codes,” IEEE Transactions on Information Theory , vol. 50, no. 5, pp. 811–823, 2004
2004
-
[10]
Decoding of second order reed-muller code s with a large number of errors,
B. Sakkour, “Decoding of second order reed-muller code s with a large number of errors,” in IEEE Information Theory W orkshop, 2005. , 2005, pp. 3 pp.–
2005
-
[11]
Comparing the bit-map and block-map decoding thresholds o f reed- muller codes on bms channels,
S. Kudekar, S. Kumar, M. Mondelli, H. D. Pfister, and R. Ur bankez, “Comparing the bit-map and block-map decoding thresholds o f reed- muller codes on bms channels,” in 2016 IEEE International Symposium on Information Theory (ISIT) . IEEE Press, 2016, p. 1755–1759. [Online]. Av...
2016
-
[12]
On α -decodability and α -likelihood decoder,
J. Liu, P . Cuff, and S. V erdú, “On α -decodability and α -likelihood decoder,” in 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton) , 2017, pp. 118–124
2017
-
[13]
Monte carlo decoding of LDPC codes,
R. M. Neal, “Monte carlo decoding of LDPC codes,” 2001, t alk given at ICTP Workshop on Statistical Physics and Capacity-Approaching Codes
2001
-
[14]
Estimating th e weight enu- merators of reed-muller codes via sampling,
S. Jain, V . A. Rameshwar, and N. Kashyap, “Estimating th e weight enu- merators of reed-muller codes via sampling,” in 2024 IEEE International Symposium on Information Theory (ISIT) , 2024, pp. 280–285
2024
-
[15]
Markov chain monte carlo methods for dete ction and decoding,
J.-T. Huang, “Markov chain monte carlo methods for dete ction and decoding,” PhD thesis, University of California San Diego, 2023
2023
-
[16]
D. A. Levin and Y . Peres, Markov chains and mixing times . American Mathematical Soc., 2017, vol. 107
2017
-
[17]
Parallel monte carlo markov chain decoding of linear codes,
J.-T. Huang and Y .-H. Kim, “Parallel monte carlo markov chain decoding of linear codes,” in 2023 IEEE International Symposium on Information Theory (ISIT) , 2023, pp. 2051–2056
2023
-
[18]
MCMC decoding of LDPC codes with BP preprocessing,
——, “MCMC decoding of LDPC codes with BP preprocessing, ” in GLOBECOM 2020 - 2020 IEEE Global Communications Conference , 2020, pp. 1–5
2020
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.