Normative Networks for Source Separation via Local Plasticity and Dendritic Computation

Bariscan Bozkurt; Efe Ali Gorguner; Francesco Innocenti; Rafal Bogacz

arxiv: 2605.19965 · v2 · pith:SLJ767SZnew · submitted 2026-05-19 · 💻 cs.LG · eess.SP

Normative Networks for Source Separation via Local Plasticity and Dendritic Computation

Bariscan Bozkurt , Efe Ali Gorguner , Francesco Innocenti , Rafal Bogacz This is my paper

Pith reviewed 2026-05-20 07:40 UTC · model grok-4.3

classification 💻 cs.LG eess.SP

keywords blind source separationlocal plasticityentropy maximizationdendritic computationpredictive codingHebbian learningneural networks

0 comments

The pith

Predictive Entropy Maximization derives a neural network for blind source separation that relies solely on local plasticity and dendritic error signals.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces Predictive Entropy Maximization to recover latent sources from sensory mixtures using only online, local weight updates. It approximates an entropy objective so that the resulting loss decomposes into interpretable terms that map directly onto feedforward error-driven synapses, Hebbian lateral inhibition, and simple output nonlinearities. A sympathetic reader cares because the approach relaxes the strong independence assumptions common in earlier biologically plausible BSS methods while still remaining competitive with exact global algorithms. The derivation supplies explicit spectral bounds that quantify when the approximation holds. Empirical tests confirm robustness to both source correlation and observation noise.

Core claim

Minimizing the surrogate objective produces a predictive architecture in which feedforward connections follow an error-driven rule realizable by dendritic mechanisms, lateral connections are updated by local Hebbian plasticity, and domain constraints appear as output nonlinearities; spectral bounds on the approximation error characterize the regimes where this local procedure remains faithful to the original entropy measure.

What carries the argument

The second-order surrogate for the entropy objective, whose minimization yields local Hebbian and error-driven plasticity rules together with output nonlinearities.

If this is right

Separation performance remains stable as source correlation and additive noise increase.
The network outperforms earlier local algorithms that impose stronger decorrelation or independence assumptions.
Domain constraints are enforced simply by choosing appropriate output nonlinearities.
Spectral error bounds predict the range of source statistics for which the approximation stays valid.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same local-objective construction could be applied to other unsupervised tasks that require separation of structured latent variables.
Biological circuits that already exhibit dendritic error computation and Hebbian lateral inhibition might be implementing an entropy-maximization objective without global communication.
Replacing the fixed output nonlinearities with learned ones would test whether the framework can discover domain constraints rather than receive them.

Load-bearing premise

The second-order approximation to the entropy measure stays sufficiently accurate across the source distributions the network is meant to separate.

What would settle it

An experiment in which the surrogate objective produces a measurably different separation solution than exact entropy maximization on the same data set would falsify the claim that the local rules achieve the normative goal.

Figures

Figures reproduced from arXiv: 2605.19965 by Bariscan Bozkurt, Efe Ali Gorguner, Francesco Innocenti, Rafal Bogacz.

**Figure 1.** Figure 1: Representative Predictive Entropy Maximization architectures for two source domains. (a) Antisparse architecture. Mixture inputs are mapped through feedforward weights W to local prediction compartments, each paired with an output unit yk. Prediction errors ek are computed as the differences between somatic and dendritic activity. The output layer is coupled through adaptive recurrent inhibitory interact… view at source ↗

**Figure 2.** Figure 2: Performance comparisons across source domains. (a) Mean component SNR versus correlation ρ for nonnegative antisparse sources. Predictive Entropy Maximization (PEM) and its unnormalized variant (u-PEM) remain robust and outperform online baselines relying on stronger independence or decorrelation assumptions. (b) Mean component SNR versus input SNR for sparse sources. The PEM models stay close to the batch… view at source ↗

**Figure 3.** Figure 3: Receptive fields learned by the sparse Predictive Entropy Maximization model from natural image patches exhibit localized and oriented structure characteristic of sparse sensory representations. Learning sparse receptive fields. We evaluate the sparse architecture illustrated in Figure 1b on prewhitened 12 × 12 natural image patches [4].2 The network is trained on vectorized patches in R 144. The learne… view at source ↗

**Figure 4.** Figure 4: Taylor-surrogate diagnostics. (a) Exact Taylor remainder versus the theoretical bound from (Eq. 8), over recorded runs and iterations, color-coded by source correlation ρ. The solid line denotes y = x. (b) Mean Taylor remainder and corresponding bound (Eq. 8) as functions of ρ. 5 Conclusion Summary. In this work, we introduced Predictive Entropy Maximization, an online determinantbased entropy maximizatio… view at source ↗

**Figure 5.** Figure 5: Representative auditory source-separation result. Temporal alignment between the ground-truth and recovered sources for one trial in the cocktail-party experiment described in Section 4. E.2 Transform invariance in auditory source separation In Section 4, we applied Predictive Entropy Maximization to audio mixtures in a sparse wavelet domain. The justification is straightforward. In the noise-free setting… view at source ↗

**Figure 6.** Figure 6: Additional performance comparisons. (a) Mean component SNR (mSNR) as a function of the source correlation level ρ for antisparse sources. (b) Mean component SNR as a function of the input SNR for nonnegative sparse sources. (c) Mean component SNR as a function of the input SNR for simplex sources. These complementary experiments confirm the same pattern observed in the main text: Predictive Entropy Maximiz… view at source ↗

**Figure 7.** Figure 7: Transient Taylor-surrogate diagnostics. Exact Taylor remainder and corresponding spectral upper bound over training iterations for two representative source-correlation levels. supplied, the feedforward matrix was initialized as W(0) = I + ξW , (E.3) where I denotes the rectangular identity and ξW has i.i.d. Gaussian entries with standard deviation 0.01. Likewise, if not specified explicitly, the running m… view at source ↗

**Figure 8.** Figure 8: Ablation with respect to the distribution of the mixing-matrix entries. Box plots summarize the distribution of mSNR over 30 seeds in the uncorrelated setting (ρ = 0) at input SNR 30 dB. All candidate mixing laws are centered and scaled to unit variance: N (0, 1), U(− √ 3, √ 3), Laplace(0, 1/ √ 2), Rad(±1), and p 3/5 t5. The performance of Predictive Entropy Maximization remains broadly stable across these… view at source ↗

**Figure 9.** Figure 9: Ablation with respect to the number of mixtures. Mean mSNR is plotted against the number of mixtures m ∈ {7, . . . , 13} while keeping the number of sources fixed at n = 5. The shaded envelopes show one standard error over 30 seeds. For each seed, a single Gaussian 13 × 5 mixing matrix is sampled and the experiment with m mixtures uses its first m rows, so the comparison isolates the effect of adding obser… view at source ↗

**Figure 10.** Figure 10: Evolution of the variance term in PEM during training. The evolution of the mean variance (averaged across the sources) is plotted across training samples for the PEM model. Solid lines and shaded bands indicate the mean and the 95% confidence interval over 30 seeds, respectively. For each seed, a single Gaussian 10 × 5 mixing matrix is sampled. These experiments demonstrate that the variance terms stabi… view at source ↗

read the original abstract

Blind source separation (BSS) is a natural framework for studying how latent causes may be recovered from sensory mixtures, but deriving online and biologically plausible algorithms for structured (i.e., constrained to known domains) and potentially correlated sources remains challenging. Recent work has derived neural networks for BSS from maximization of an entropy measure, yet its online implementations involve complex and nonlocal recurrent dynamics. Motivated by this perspective, we propose Predictive Entropy Maximization, which achieves competitive performance in BSS, using only local weight updates. The method employs a close approximation of an entropy measure, yielding an objective function with easily interpretable components. Minimizing this objective leads to a predictive neural architecture in which feedforward synapses follow an error-driven rule (that can be realized through dendritic mechanisms), lateral inhibitory connections are learned with local Hebbian plasticity, and source-domain constraints are enforced through simple output nonlinearities. We derive explicit spectral bounds on the surrogate error, characterizing when the approximation is accurate. Empirically, Predictive Entropy Maximization remains robust under increasing source correlation and observation noise, outperforms biologically plausible algorithms that rely on stronger independence or decorrelation assumptions, and remains competitive with exact determinant- and correlative-information-based baselines. These results show how local plasticity and adaptive lateral inhibition can emerge from maximizing a regularized second-order entropy over structured source domains. Our implementation code is available at https://github.com/BariscanBozkurt/Predictive-Entropy-Maximization.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

This paper gives local plasticity rules for blind source separation from a surrogate entropy objective, with empirical robustness to correlation and noise, but the spectral bounds look potentially too loose in exactly those regimes.

read the letter

Hi colleague, the headline on this paper is that it gives a biologically inspired local plasticity method for blind source separation based on approximating entropy maximization, and while the experiments look promising, the supporting spectral bounds may not hold up strongly enough in the high-correlation settings they emphasize. They do some solid work. The Predictive Entropy Maximization objective leads to a network where feedforward synapses use an error-driven rule realizable by dendritic computation, lateral inhibitory connections are updated locally with Hebbian plasticity, and source constraints come from output nonlinearities. They provide explicit spectral bounds on the error of this approximation to the true entropy. Empirically, the method is shown to be robust as source correlation and observation noise increase, outperforming other local algorithms with stricter assumptions and remaining competitive with exact determinant or correlative information methods. The GitHub code is a good move for verification. The soft spot, as the stress-test note suggests, is the tightness of those bounds. Since the bounds are characterized using source-domain properties that become less favorable with more correlation and noise, there is a risk that the approximation error grows in the regimes where robustness is most needed. If that happens, the local rules might not be fully justified by the entropy objective, and performance could be driven more by the nonlinearities. The paper does not seem to report specific numerical values for the bounds in the experimental conditions, which leaves some uncertainty. This paper is for folks in computational neuroscience and machine learning focused on online, local learning rules for tasks like source separation. A reader looking for new ideas in normative modeling with some empirical backing would find it worthwhile. I think it deserves to go through peer review. The attempt to link local mechanisms to an entropy objective with bounds and tests makes it a reasonable candidate for detailed feedback.

Referee Report

2 major / 2 minor

Summary. The paper proposes Predictive Entropy Maximization (PEM) for blind source separation, deriving a neural network from a close surrogate approximation to entropy maximization. This yields an objective with interpretable terms that translate into local plasticity rules: error-driven feedforward updates (via dendritic mechanisms), Hebbian lateral inhibition, and output nonlinearities enforcing source-domain structure. Explicit spectral bounds on the surrogate approximation error are derived, and experiments demonstrate competitive BSS performance that is robust to increasing source correlation and observation noise, outperforming other local biologically plausible methods while remaining competitive with exact baselines. Code is provided.

Significance. If the spectral bounds ensure the surrogate objective preserves the key properties of entropy maximization (i.e., its local minima recover the sources) even under moderate correlation, the work provides a valuable normative bridge between information-theoretic objectives and biologically plausible local mechanisms. Strengths include the explicit error bounds, the emergence of dendritic-style computation and Hebbian rules from a single objective, and public code for reproducibility. This could inform models of sensory processing that rely on structured rather than fully independent sources.

major comments (2)

[§4] §4 (Spectral Bounds on Surrogate Error): The derived bounds are expressed in terms of source-domain properties (e.g., eigenvalues of the covariance) that degrade exactly when source correlation rises; the manuscript does not show that the resulting error remains below the threshold at which the interpretable components (error-driven feedforward, Hebbian lateral) continue to enforce source recovery rather than relying primarily on the output nonlinearities.
[§5.3] §5.3 (Robustness Experiments): While performance is reported as robust under increasing correlation, the actual pointwise or integrated surrogate error on the test sets is not quantified; without this, it is unclear whether the empirical success validates the normative derivation or occurs despite the approximation becoming loose.

minor comments (2)

[§3] Notation for the dendritic error computation (around Eq. (12)) could be clarified by explicitly linking each term to the corresponding biological mechanism in a single summary table.
[Figure 4] Figure 4 caption should state the number of independent runs and whether error bars represent standard deviation or standard error.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive and detailed feedback on our manuscript. We address each major comment below and have incorporated revisions to clarify the role of the spectral bounds and to quantify the surrogate error in the experimental sections.

read point-by-point responses

Referee: [§4] §4 (Spectral Bounds on Surrogate Error): The derived bounds are expressed in terms of source-domain properties (e.g., eigenvalues of the covariance) that degrade exactly when source correlation rises; the manuscript does not show that the resulting error remains below the threshold at which the interpretable components (error-driven feedforward, Hebbian lateral) continue to enforce source recovery rather than relying primarily on the output nonlinearities.

Authors: We appreciate this observation regarding the dependence of the bounds on source covariance eigenvalues. The derived spectral bounds are intended as a theoretical characterization of the approximation error rather than a guarantee of a fixed error threshold; they explicitly delineate the regimes in which the surrogate remains close to the original entropy objective. In the revised manuscript we have added a discussion in §4 explaining how the local plasticity rules (error-driven feedforward and Hebbian lateral inhibition) retain their normative interpretation even as the bound loosens, supported by a new ablation experiment that isolates the contribution of these rules versus the output nonlinearities under increasing correlation. revision: partial
Referee: [§5.3] §5.3 (Robustness Experiments): While performance is reported as robust under increasing correlation, the actual pointwise or integrated surrogate error on the test sets is not quantified; without this, it is unclear whether the empirical success validates the normative derivation or occurs despite the approximation becoming loose.

Authors: We agree that reporting the surrogate error directly on the test sets would strengthen the connection between the theoretical analysis and the empirical results. In the revised §5.3 we now include both pointwise and integrated surrogate error values across the correlation and noise levels used in the robustness experiments. These additional results indicate that the error remains within moderate bounds in the regimes where Predictive Entropy Maximization continues to outperform other local methods, thereby supporting that the normative components contribute to the observed performance. revision: yes

Circularity Check

0 steps flagged

Derivation chain is self-contained with independent mathematical and empirical support

full rationale

The paper introduces Predictive Entropy Maximization as a close approximation to an entropy objective for blind source separation, derives spectral bounds on the resulting surrogate error directly from the approximation's properties, and demonstrates competitive performance via local plasticity rules through both the interpretable components of the objective and empirical tests across source correlations and noise levels. No step reduces by construction to its own inputs: the bounds are obtained via standard spectral analysis of the error term rather than being fitted or self-referential, the local update rules follow from minimizing the stated objective without tautological renaming, and performance claims rest on external validation rather than self-citation chains or ansatzes smuggled from prior author work. The derivation remains self-contained against the provided benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Based solely on the abstract, the method rests on the standard assumption that entropy maximization separates sources and on the validity of a close approximation whose error is bounded spectrally; no explicit free parameters or new invented entities are described.

axioms (1)

domain assumption Maximizing (an approximation to) entropy recovers independent or structured latent sources from linear mixtures.
Invoked as the normative starting point for deriving the local update rules.

pith-pipeline@v0.9.0 · 5803 in / 1249 out tokens · 34610 ms · 2026-05-20T07:40:32.054599+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

−log det(Ĉ_λ(t) + εI) = −∑ log(v̂_i(t) + ε) + ½ ∑_{i≠j} ĉ_ij(t)² / ((v̂_i+ε)(v̂_j+ε)) + R₂(t)
IndisputableMonolith/Foundation/AlphaCoordinateFixation.lean costAlphaLog_high_calibrated_iff unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

explicit spectral bounds on the surrogate error |R₂(t)| ≤ ∥B̂_λ,ε(t)∥_F² ∥B̂_λ,ε(t)∥₂³ / (1 + λ_min(B̂_λ,ε(t)))

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.