{"id":"a161bb56-ba52-43c0-baea-a00792dac9bb","arxiv_id":"2505.14635","paper_version":2,"verdict":"REJECT","confidence":"HIGH","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A theoretical framework claims that predictive coding performs block-coordinate descent on a two-part code objective and bounds true risk by empirical risk plus codelength divided by sample size.","lead":"This paper argues that layerwise predictive coding, a brain-inspired local learning rule, implicitly minimizes a minimum description length (MDL) objective that combines data fit and model complexity. It also derives a generalization bound that trades off empirical loss against codelength.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorems 2–3 prove descent for exact block-argmin PC, not for the local gradient PC update in §2; the codelength objective also changes between §3.1 and §3.2.","rationale":"The paper's advertised contribution is not the Occam bound itself, which is algorithm-independent and standard, nor the BCD convergence theorem, which is classical, but the identification of PC's update rule as minimizing the two-part codelength. Theorems 2 and 3 are the only results that attach to PC, and both replace the §2 local gradient update with an exact blockwise argmin. The reader's rejection is therefore well placed: absent a link from Δθ_l = -η(ε_l ⊙ f'(θ_l x_{l-1})) x_{l-1}^⊤ to exact block minimization, the paper proves properties of a different algorithm. The objective normalization slip (R̂ + L/N versus R̂ + L) is a second independent obstacle: even the quantity being descended is ambiguous. The proposed simulation is decisive because it tests the actual §2 algorithm on the same setup the paper uses for its empirical claim; if codelength is non-increasing there, the gap could be patched. Credit is due for the scalar and vector regression examples, which do show that exact-argmin PC on quadratic energies performs coordinate descent, and for the standard Occam bound; but those do not rescue the general claim. No formal verification or public code is provided, and the only experiment implicitly relies on the exact-argmin interpretation. Verdict unchanged.","tokens_in":19764,"tokens_out":5530,"duration_ms":52941,"concrete_test":"Use the exact §4 setup (two-layer linear net, N=100, η=0.01) but implement the actual PC update from §2: run inference to equilibrium, then Δθ_l = -η(ε_l ⊙ f'(θ_l x_{l-1})) x_{l-1}^⊤ for each layer in a sweep. Compute Ĉ(θ) = R̂(θ) + L(θ)/N after each of 100 sweeps across the same 1000 seeds. If any seed shows an increase in Ĉ after a full sweep, or if the final Ĉ exceeds the BP value, Theorem 2's monotone-decrease claim fails for the algorithm the paper calls PC. Alternatively, derive the η-threshold for the scalar regression in §B.1 and check whether η=0.01 satisfies it on all seeds; if the threshold is violated, exact-argmin and gradient PC diverge.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that 'each PC sweep monotonically decreases the empirical two-part codelength' (Theorem 2) and that repeated PC updates converge to a blockwise stationary point of the MDL objective (Theorem 3). Both proofs rely on θ_l = arg min_{θ_l} Σ_i c(θ; x_i) in Theorem 2 and on Assumption A3, 'Exact Layer Minimization', in Theorem 3. But the predictive coding algorithm defined in §2 is Δθ_l = -η(ε_l ⊙ f'(θ_l x_{l-1})) x_{l-1}^⊤ after inference on the value layers. No step in the paper shows that this finite-step-size gradient update equals, or even descends, the exact blockwise codelength minimizer. The appendices' linear examples solve the block minimization in closed form, but actual PC with finite η does not jump to that solution; for generic η the codelength can increase on a single sweep. A second, independent gap: §3.1 defines Ĉ = R̂ + L/N, while §3.2 defines c = L + ℓ and Ĉ = (1/N)Σ c = R̂ + L, so Theorem 2's descent statement and its Occam-bound right-hand side are not the same quantity. The inexact-BCD discussion in Appendix C asserts sufficient decrease from 'error-correcting feedback' but gives no proof from the §2 update. Thus the headline result is established only for a redefined algorithm.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a theoretical link between predictive coding (PC) and the minimum description length (MDL) principle. It derives an Occam-style generalization bound (Theorem 1), claims that each layerwise PC sweep monotonically reduces an empirical two-part codelength (Theorem 2), and claims that repeated PC updates converge to a blockwise stationary point of that codelength (Theorem 3). The appendices contain additional rate claims and a small two-layer linear simulation comparing PC and backpropagation. The central thesis is that PC should be understood as a compression-driven learning algorithm rather than merely an approximation to backpropagation.","tokens_in":20040,"tokens_out":7679,"duration_ms":78768,"significance":"If the central claims held, the paper would provide a useful new interpretation of predictive coding and would connect a biologically plausible learning rule to an information-theoretic generalization bound. Theorem 1 is a standard Occam/PAC-Bayes-type bound and is essentially correct under its stated assumptions. However, the paper's main results are not established for the predictive-coding algorithm defined in Section 2: Theorem 2 and Assumption A3 replace the local gradient update with an exact blockwise argmin, and the codelength objective changes between Sections 3.1 and 3.2. Because these issues affect the headline contributions, the significance of the paper as it stands is limited.","major_comments":[{"comment":"The theorem defines 'one full sweep of layerwise PC' as θ_l^{(l)} = arg min_{θ_l} Σ_i c(θ; x_i). This is not the PC learning update of Section 2, which is Δθ_l = -η(ε_l ⊙ f'(θ_l x_{l-1})) x_{l-1}^T. No result in the paper shows that this finite-step-size gradient update descends the blockwise codelength objective or even decreases it; Appendix C.1 asserts a sufficient-decrease condition 'via error-correcting feedback' but gives no proof from the Section 2 update. Consequently, Theorems 2 and 3 (through Assumption A3) apply to an exact block-coordinate minimization algorithm, not to the PC algorithm that is defined and simulated in the paper.","section":"Section 3.2, Theorem 2"},{"comment":"The empirical codelength is defined inconsistently. Section 3.1 defines ĀC(θ) = ĀR(θ) + L(θ)/N, while Section 3.2 defines per-sample c(θ; x) = L(θ) + ℓ(θ; x) and ĀC(θ) = (1/N)Σ_i c(θ; x_i) = L(θ) + ĀR(θ). These differ by a factor N in the complexity term. Theorem 2's conclusion R(θ_PC) ≤ ĀC(θ^{(0)}) + ln(1/δ)/N is therefore a statement about a different, much weaker objective, and it does not connect to the MDL objective used in the abstract and Section 3.1. The proof of Theorem 2 also writes ĀC as a sum without the 1/N factor, reinforcing the inconsistency.","section":"Sections 3.1 and 3.2"},{"comment":"The convergence proof is incomplete even if Assumption A3 is granted. Monotone convergence of the real sequence C(θ^{(t)}) does not imply convergence of the parameter sequence; the Bolzano-Weierstrass step requires boundedness of the iterates or of a level set, which is not stated among (A1)-(A3). Moreover, continuity and exact coordinate minimization do not by themselves guarantee that every limit point is a coordinate-wise minimizer of C; classical coordinate-descent convergence results require additional hypotheses on the objective and the update rule. The theorem as stated is therefore not proven.","section":"Section 3.3, Theorem 3"},{"comment":"The claimed norm contraction for backpropagation is not justified. From ‖θ_{t+1}‖² = ‖θ_t‖² + η²‖∇ĀR‖² - 2η⟨θ_t, ∇ĀR⟩, the proof derives a lower bound ‖θ_T‖² ≥ ‖θ_0‖² + η²G²(Σ ρ^{L-l†})². But Cauchy-Schwarz only bounds the absolute value of the inner product; without a sign or alignment assumption on ⟨θ_t, ∇ĀR⟩, the parameter norm can decrease. Thus the claimed strict Occam improvement of PC over backpropagation under equal compute is not established.","section":"Appendix A, Lemma 4 and Theorem 4"}],"minor_comments":[{"comment":"The theorem uses ln(1/δ) in the final bound, while Theorem 1 and Lemma 2 have ln(2/δ); the mismatch should be corrected.","section":"Theorem 2"},{"comment":"Figure 1 reports mean trajectories and shaded standard deviations but no statistical comparison; the claim that PC 'consistently' reaches lower MDL values across all seeds is not directly demonstrated.","section":"Section 4.2"},{"comment":"The perturbation percentages (2.5% vs 18.7%) are reported without a null model or confidence intervals, so their evidentiary value for blockwise stationarity is limited.","section":"Section 4.3"},{"comment":"The assertion that 'asymptotic regularity' Σ_t ‖θ^{(t+1)} - θ^{(t)}‖² < ∞ follows from a decreasing and bounded energy sequence is not valid without additional assumptions, such as a sufficient-descent condition that supplies a quantitative decrease.","section":"Appendix C.1"},{"comment":"The introductory 'Summary of Contributions' describes the bound as holding 'for every parameter realization' and any learner, which is fine, but the word 'novel' overstates what is a standard Occam/PAC-Bayes union-bound argument.","section":"Section 1"}],"recommendation":"reject","confidential_remarks":"The central contribution of the paper is not supported by the proofs: the algorithm analyzed in Theorems 2 and 3 is an idealized exact block-minimization algorithm, not the predictive-coding update defined in Section 2, and the codelength objective changes between Sections 3.1 and 3.2. These are load-bearing issues that cannot be addressed by minor revision. I recommend rejection, while noting that a substantially revised paper proving sufficient descent for the actual gradient-based PC update against a single consistently defined codelength objective could be a worthwhile contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about arXiv:2505.14635. The paper wants to show that predictive coding (PC) minimizes a two-part MDL objective and therefore has a compression-based generalization guarantee. But the theorem that does the work, Theorem 2, defines a 'PC sweep' as exact blockwise minimization of the codelength. That is not the algorithm described in Section 2, where updates are local gradient steps of the form Δθ = -η(ε ⊙ f')x^T. No argument shows that those gradient steps descend the codelength objective, let alone minimize each block. The appendices' linear examples solve the block minimization in closed form, but a real PC sweep with finite η does not jump to that solution. So the main claim is established only for a redefined algorithm, not for the subject of the paper.\n\nThe paper also has an internal inconsistency worth noting. Section 3.1 defines the empirical codelength as C = R̂ + L/N, which matches Theorem 1's bound. Section 3.2 redefines the per-sample cost as c = L + ℓ, making C = R̂ + L. Theorem 2's descent proof uses the second definition, but its risk bound only follows from Theorem 1 under the first. You cannot have both, so the theorem as written is false even for exact block minimization.\n\nWhat's genuinely good: Theorem 1 is a correctly stated Occam/PAC-Bayes bound, and the paper is honest in Appendix A that this is the standard result. The scalar and vector regression examples in Appendix B are clean. The idea of asking whether PC's local updates can be viewed as approximate BCD on a complexity-penalized objective is a reasonable research direction, and the paper points to the right classical theorems (Tseng, Beck-Tetruashvili). But the connection to the actual PC algorithm is the whole point, and that's where it breaks.\n\nThe experiments are synthetic simulations without code or data, and they compare PC to BP on the MDL objective that PC is assumed to minimize, so they don't compensate for the theoretical gap. Appendix C asserts that PC satisfies a sufficient-descent condition for inexact BCD, but no proof is given from the Section 2 update; it's an assertion.\n\nBottom line: this paper is for readers who want to know what would be needed to give PC an MDL-style generalization story. It doesn't deliver that story. The citation pattern is fine, but the load-bearing claim is not supported. I'd reject it, and I wouldn't send it to review as is. If the authors can prove a descent property for the actual local update—or honestly restrict the theorems to exact block minimization and fix the L/N inconsistency—a revision might be worth another look.","headline":"The central claim is about a redefined exact-argmin algorithm, and the codelength objective changes between sections; the actual PC update is never shown to descend the MDL objective.","tokens_in":20645,"tokens_out":6151,"would_cite":false,"duration_ms":48528,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","68Q32","90C26","94A17"],"pacs":[],"model":"deepseek-v4-flash","headline":"Predictive coding is a block-coordinate descent method on the MDL two-part codelength; each local sweep provably tightens a high-probability risk bound.","keywords":["predictive coding","minimum description length","two-part code","generalization bound","block-coordinate descent","model complexity","local learning rules","deep network theory"],"falsifier":"Run the standard predictive-coding update $\\Delta\\theta_l = -\\eta(\\varepsilon_l \\odot f'(\\theta_l x_{l-1})) x_{l-1}^T$ on a network and record $\\hat{C}(\\theta) = \\hat{R}(\\theta) + \\frac{1}{N} L(\\theta)$ after every sweep; if $\\hat{C}$ ever increases on a single sweep, the claimed monotone codelength decrease fails for the actual algorithm. At convergence, perturb one layer's weights by small noise and count how often $\\hat{C}$ decreases; the paper predicts this should essentially never happen, so a large fraction of decreases would falsify the blockwise-stationarity claim.","tokens_in":19469,"feed_emoji":"🧠","tokens_out":8839,"duration_ms":74904,"temperature":0.7,"pith_summary":"Predictive coding (PC) is a biologically inspired learning rule in which neurons reduce local prediction errors; this paper claims that, when run layer by layer, PC is exactly a block-coordinate descent algorithm on the minimum description length (MDL) two-part code objective. The central result is that one full sweep of layerwise PC updates monotonically decreases the empirical codelength Ĉ(θ) = R̂(θ) + L(θ)/N, where R̂ is the training loss and L(θ) = −log p(θ) is the parameter codelength under a factorized prefix-code prior. Under an exact-layer-minimization assumption, repeated sweeps converge to a blockwise stationary point of that codelength, so PC finds a local MDL optimum. If these theorems are correct, PC-trained networks inherit a high-probability risk bound R(θ) ≤ R̂(θ) + L(θ)/N + ln(1/δ)/N that tightens with every sweep, giving a generalization guarantee for PC that does not depend on approximating backpropagation.","feed_headline":"Cuts codelength per sweep: predictive coding tightens the risk bound","feed_subtitle":"The brain-inspired rule provably descends an MDL objective, so each sweep yields a tighter generalization guarantee.","key_machinery":"The load-bearing object is the empirical two-part codelength $\\hat{C}(\\theta) = \\hat{R}(\\theta) + \\frac{1}{N} L(\\theta)$, where $\\hat{R}$ is the average per-sample negative log-likelihood and $L(\\theta) = -\\log p(\\theta)$ is the code-length of the parameters under a factorized prefix-code prior. The proof machinery is the identification of a PC sweep with exact cyclic block-coordinate descent on $\\hat{C}$: at each layer, in the idealized update, the weights are chosen as the exact minimizer of the codelength with all other layers fixed. This identification turns the generalization bound $R(\\theta) \\le \\hat{R}(\\theta) + L(\\theta)/N + \\ln(1/\\delta)/N$ into an objective that PC actively descends, so each sweep both compresses the model and tightens the high-probability risk guarantee.","core_discovery":"The paper's core discovery is that the local, energy-based dynamics of predictive coding can be read as exact layerwise minimization of a two-part description length, not merely as a biologically plausible approximation to backpropagation. Formally, defining the empirical codelength $\\hat{C}(\\theta) = \\hat{R}(\\theta) + \\frac{1}{N} L(\\theta)$ with $L(\\theta) = -\\log p(\\theta)$, the paper proves that every block update in a PC sweep chooses the layer weights that minimize $\\hat{C}$ holding other layers fixed; therefore one sweep cannot increase $\\hat{C}$, and applying the concentration-based bound $R(\\theta) \\le \\hat{R}(\\theta) + L(\\theta)/N + \\ln(1/\\delta)/N$ to the updated parameters gives a tighter generalization guarantee. Repeating sweeps yields a nonincreasing, bounded codelength sequence whose limit point satisfies the block-coordinate stationarity condition $\\theta^\\infty_l = \\arg\\min_{\\theta_l} \\hat{C}(\\theta^\\infty_1, \\dots, \\theta_l, \\dots, \\theta^\\infty_L)$. The paper frames this as a convergence to an approximate MDL-optimal solution.","pith_inferences":["I infer a testable design principle: any local learning rule that can be shown to implement block-coordinate descent on a two-part codelength would inherit the same Occam-style risk bound, so other energy-based rules could be compared by measuring their codelength decrease per sweep.","A direct extension would replace the exact argmin update with the actual predictive-coding gradient step and measure how much codelength decreases per sweep; the theorem's exact-layer-minimization assumption is exactly what such an experiment would probe.","If the exact-argmin idealization fails, the monotone descent theorem need not hold, so the practical value of the framework hinges on whether the local PC update descends the empirical codelength on real losses, which the paper's simulation suggests but does not prove.","The bound's dependence on the prior suggests an experimental comparison: varying the Gaussian prior scale should change the generalization gap for PC-trained networks in a predictable way, since larger code-lengths worsen the right-hand side of the risk bound."],"forward_implications":["One full layerwise PC sweep never increases the empirical two-part codelength, so starting from any parameters, the high-probability risk bound improves after a single sweep.","Under exact layer minimization, repeated PC sweeps converge to a blockwise stationary point of the MDL objective, meaning no single layer can be adjusted alone to lower the codelength.","On a fixed compute budget, a PC sweep and a backpropagation pass have comparable cost, and the paper's budget analysis argues PC yields a strictly tighter risk bound than backpropagation under its depth-attenuated gradient assumption.","Because the generalization bound is algorithm-independent, the PC-specific contribution is a proof that this particular local learning rule actually minimizes the right-hand side of the bound."],"supporting_citations":[{"why":"Defines the minimum description length principle that supplies the two-part code objective.","marker":"[30]"},{"why":"Introduces predictive coding as a model of hierarchical cortical processing and gives the local error-based update rule analyzed here.","marker":"[29]"},{"why":"Formulates hierarchical predictive coding under an energy-style functional, the basis of the PCN energy used in the paper.","marker":"[7]"},{"why":"Provides the concentration inequality used to derive the high-probability uniform bound in Lemma 2.","marker":"[4]"},{"why":"Prior theoretical treatment of inference and learning in predictive coding networks whose lack of independent MDL-type guarantees motivates the paper.","marker":"[17]"},{"why":"Prior stability and convergence analysis of predictive coding dynamics that the present paper extends to the MDL objective.","marker":"[14]"},{"why":"Block-coordinate descent convergence results invoked to justify convergence of repeated PC sweeps to a blockwise stationary point.","marker":"[3]"},{"why":"Coordinate descent survey used to benchmark convergence rates of the PC sweep as exact cyclic block-coordinate descent.","marker":"[41]"}],"fun_headline_variants":["PC descends MDL objective, tightening risk bounds","Predictive coding provably minimizes two-part codelength","Each PC sweep shrinks codelength, boosts generalization","Brain-inspired learning converges to MDL-optimal point"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof needs each layer to be updated to the exact best value for that layer while the other layers stay fixed, but the actual predictive-coding rule takes a small local gradient step; the paper does not show these are equivalent.","fun_headline_variants_meta":{"raw":{"variants":["PC descends MDL objective, tightening risk bounds","Predictive coding provably minimizes two-part codelength","Each PC sweep shrinks codelength, boosts generalization","Brain-inspired learning converges to MDL-optimal point"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000155,"raw_usage":{"total_tokens":1232,"prompt_tokens":980,"completion_tokens":252,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":596,"completion_tokens_details":{"reasoning_tokens":187}},"tokens_in":596,"tokens_out":252,"duration_ms":5118,"temperature":1.0,"reasoning_tokens":187,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:30:36.005847+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the standard predictive-coding update $\\Delta\\theta_l = -\\eta(\\varepsilon_l \\odot f'(\\theta_l x_{l-1})) x_{l-1}^T$ on a network and record $\\hat{C}(\\theta) = \\hat{R}(\\theta) + \\frac{1}{N} L(\\theta)$ after every sweep; if $\\hat{C}$ ever increases on a single sweep, the claimed monotone codelength decrease fails for the actual algorithm. At convergence, perturb one layer's weights by small noise and count how often $\\hat{C}$ decreases; the paper predicts this should essentially never happen, so a large fraction of decreases would falsify the blockwise-stationarity claim.","supporting_citations":[{"cited_title":"Rissanen","cited_arxiv_id":null,"evidence_quote":"Defines the minimum description length principle that supplies the two-part code objective."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces predictive coding as a model of hierarchical cortical processing and gives the local error-based update rule analyzed here."},{"cited_title":"Friston and S","cited_arxiv_id":null,"evidence_quote":"Formulates hierarchical predictive coding under an energy-style functional, the basis of the PCN energy used in the paper."},{"cited_title":"Bernstein","cited_arxiv_id":null,"evidence_quote":"Provides the concentration inequality used to derive the high-probability uniform bound in Lemma 2."},{"cited_title":"Millidge, Y","cited_arxiv_id":null,"evidence_quote":"Prior theoretical treatment of inference and learning in predictive coding networks whose lack of independent MDL-type guarantees motivates the paper."},{"cited_title":"Tight Stability, Convergence, and Robustness Bounds for Predictive Coding Networks","cited_arxiv_id":"2410.04708","evidence_quote":"Prior stability and convergence analysis of predictive coding dynamics that the present paper extends to the MDL objective."},{"cited_title":"Beck and L","cited_arxiv_id":null,"evidence_quote":"Block-coordinate descent convergence results invoked to justify convergence of repeated PC sweeps to a blockwise stationary point."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Coordinate descent survey used to benchmark convergence rates of the PC sweep as exact cyclic block-coordinate descent."}],"review_version":1}