{"id":"c4e04bb8-55fa-4edc-b496-fa0b3dc7d209","arxiv_id":"2506.19726","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A von Mises-Fisher posterior on weight directions, matched to normalized networks, yields a per-layer effective noise parameter with a closed-form dimension-aware KL that improves calibration on CIFAR-10.","lead":"This paper proposes a Bayesian neural network method that models weight uncertainty only through directions, using a von Mises-Fisher distribution on the sphere, which reduces each layer to a single effective noise parameter. The method improves calibration on CIFAR-10 with a VGG16+BN network, but the empirical evidence is limited to one dataset and a weak baseline.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The vMF KL and the Gaussian post-BN noise are only moment-matched, so the objective is not shown to be a variational bound; the CIFAR-10 claim also rests on a single unseeded run.","rationale":"I read the paper in good faith and stress-tested the mathematical core. The derivations of the vMF covariance, the spherical-average identity (Eq 6), the compact interpolant D/(kappa+D), and the KL_approx formula (Eq 10) are internally consistent, and the synthetic validation at D=100 supports them. I checked the KL_approx against the exact vMF-vs-uniform KL in moderate regimes and found it tracks well; the reader's stated weakest assumption about intermediate-regime accuracy is a fair but not fatal concern. The more load-bearing gap is that the stochastic forward pass actually used in training adds independent Gaussian noise after BN, while the KL term is that of a vMF distribution over weight directions. These two are equated only through a first/second-moment argument, and the paper does not show that the resulting objective is a variational lower bound for the softmax classification model actually evaluated. This is a correctness risk for the 'Precise Bayesian' framing, not just for approximation accuracy. In addition, the empirical headline rests on a single CIFAR-10 run with a no-KL baseline, so even if the theory were fully clean, the practical claim would need seeds, error bars, and stronger baselines. Since the reader already assigned CONDITIONAL for the empirical deficiencies, my concern reinforces that verdict without moving it; I would keep CONDITIONAL but add the ELBO derivation or an explicit heuristic-reguralizer framing as a required condition.","tokens_in":7814,"tokens_out":26284,"duration_ms":279744,"concrete_test":"On a single normalized linear layer with fixed unit-norm mean weight mu, take a trained or synthetic softmax classifier and fix kappa (or sigma_eff) at a value observed in Fig 5. Estimate E_{w~vMF(mu,kappa)}[log p(y|BN(w^T x))] by Monte Carlo and compare it with E_epsilon[log p(y|BN(mu^T x)+sigma_eff epsilon)] on the same inputs. If the two expected log-likelihoods differ beyond Monte Carlo error, or if the full ELBO inequality fails with the vMF KL term, then the proposed objective is not the variational objective of the vMF posterior, and the Bayesian framing is not established. Separately, rerun the CIFAR-10 experiment with at least five seeds and include MC Dropout and mean-field VI baselines; if the ECE gain does not survive those comparisons, the headline calibration claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's load-bearing assertion is that the proposed objective is a valid variational objective for a vMF posterior over weight directions. But the forward pass replaces vMF weight sampling with independent additive Gaussian noise after BN, while the KL term is the vMF-vs-uniform KL expressed through sigma_eff. These ingredients are only matched 'in expectation' via first and second moments (Sec 3.3-3.4). For the Gaussian-likelihood student-teacher task, the moment match makes the expected NLL exact, but for the softmax classification used in the CIFAR-10 experiment, the expected log-likelihood depends on higher moments of the projected vMF noise, which are not Gaussian. The paper does not derive an ELBO for the actual model, so the 'variational' interpretation is unsupported; the method may simply be a KL-shaped regularizer. This matters because the central empirical result (Sec 6.4, Fig 5, ECE reduced ~5.6x) compares only against a no-KL baseline, without seeds, error bars, code, or standard BNN baselines (MC dropout, mean-field VI, ensembles). The theoretical machinery (Eq 6, Eq 10, Fig 3) is internally consistent; the gap is the missing link between the KL and the actual stochastic forward pass, plus the thinness of the empirical support.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a directional Bayesian neural network method for architectures with normalization layers. It places a von Mises-Fisher (vMF) posterior on the unit sphere of weight directions, introduces approximations sigma_u^2(kappa) ≈ D/(kappa+D) and sigma_eff = sigma_u / A_D(kappa), and derives a closed-form, dimension-aware KL approximation KL_approx(sigma_eff,D) = (D-1)/2 log(1 + D/(D-1) sigma_eff^{-2}). The training objective is NLL plus a beta-weighted sum of per-layer KL terms, with one learned scalar sigma_eff per normalized layer. The forward pass injects additive Gaussian noise after batch normalization. The paper validates the approximation on synthetic data at D=100, demonstrates student-teacher parameter recovery, and reports improved calibration on CIFAR-10 with VGG16+BN, claiming about a 5.6x reduction in ECE. Limitations are acknowledged regarding small input dimensions and early-training stationarity.","tokens_in":8080,"tokens_out":13460,"duration_ms":133860,"significance":"The closed-form directional KL is elegant and dimension-aware, and the asymptotic-matching argument in Section 5 and Appendix F is sound; the synthetic validation at D=100 is a useful check. If the variational interpretation were established and the empirical result replicated, the method would be a lightweight, practical addition to normalized architectures. However, the manuscript does not show that Eq. (11) is an ELBO for the stochastic forward pass actually trained, and the main empirical claim rests on a single unseeded run. The significance is therefore conditional: the paper presents an attractive and potentially useful framework, but its central 'Bayesian' and 'variational' claims are not yet supported.","major_comments":[{"comment":"The objective is not derived as a variational lower bound for the model actually trained. The forward pass uses fixed unit-norm weights and injects independent Gaussian noise after BN, whereas the KL term is the vMF-versus-uniform KL for a distribution over weight directions. The statement in Section 3.4 that Gaussian noise with scale sigma_eff 'matches the effect (in expectation)' of vMF sampling is a first- and second-moment match; it does not make Eq. (11) an ELBO. For the linear-Gaussian student-teacher task, moment matching suffices for expected NLL, but the CIFAR-10 experiment uses a softmax likelihood, whose expectation under the projected vMF noise depends on higher moments that the Gaussian approximation does not control. The authors should either derive an ELBO for the actual stochastic forward pass or explicitly reframe the method as a KL-regularized training scheme rather than a variational BNN.","section":"Sections 3.4 and 5, Eq. (11)"},{"comment":"The main empirical claim rests on a single unseeded run on one dataset and one architecture. There are no error bars, no reported seeds, and no comparisons with standard uncertainty baselines such as MC dropout, mean-field VI, or deep ensembles. The reported 5.6x ECE reduction is therefore not established. Please report repeated runs with variance, add at least one standard baseline, and state explicitly whether the no-KL baseline is otherwise identical in architecture, optimizer, and training schedule.","section":"Section 6.4 and Figure 5"},{"comment":"The interpolant D/(kappa+D) is not a close approximation to the spherical-average variance 1 - A_D(kappa)^2 in the moderate-kappa regime. Using the small-kappa expansion A_D(kappa) = kappa/D + O(kappa^3) from Appendix E, the exact variance is 1 - kappa^2/D^2 + O(kappa^4), while D/(kappa+D) = 1 - kappa/D + O(kappa^2); the linear term differs. For example, at D=100 and kappa=30 the interpolant gives about 0.77 while the leading exact term is about 0.91. Since sigma_eff and KL_approx are defined through this interpolant, the claim in Figure 3(A) that the approximation tracks ground truth needs a quantitative comparison over the kappa range actually visited during training, or the claim should be weakened.","section":"Eqs. (6)-(7) and Figure 3(A)"},{"comment":"The analytic vMF prediction Delta-L = kappa(1 - cos theta) requires a value of kappa, but the text does not state how kappa is determined. If kappa is fitted to the observed landscape, the agreement in Figure 6 is circular and cannot independently support the directional uncertainty model. Please report the fitting procedure and the resulting kappa value, or remove the claim that the figure validates the model.","section":"Figure 6"}],"minor_comments":[{"comment":"The interpolant in Eq. (4) is introduced before the exact spherical-average identity in Eq. (6), which could give the impression that Eq. (4) is exact; reordering or an explicit cross-reference would clarify the status of the approximation.","section":"Sections 3.3 and 4.1"},{"comment":"The CIFAR-10 implementation details are incomplete: the appendix states that optimizer and schedules follow standard practice without specifying them, and no code or data availability statement is provided. This limits reproducibility.","section":"Appendix A"},{"comment":"For convolutional layers, the notation D_l is ambiguous; it should be stated whether D_l is the fan-in, fan-in times kernel area, or the number of input channels, since the KL and sigma_eff depend directly on D_l.","section":"Sections 5 and A"},{"comment":"The Monte Carlo validation plots would benefit from error bars or a statement of the number of MC samples and seeds, since the text reports that measurements 'match' without quantifying the agreement.","section":"Figures 3 and 4"},{"comment":"The abstract's claim that the method is 'simultaneously principled, practical, and precise' is overstated in light of the missing ELBO derivation; 'principled' should be qualified until the objective is placed on a variational footing.","section":"Abstract and Section 7"}],"recommendation":"major_revision","confidential_remarks":"The paper is competently written and the directional-statistics machinery is attractive, but the 'variational' framing currently overclaims. I would ask the authors to either prove an ELBO-type statement for the actual stochastic process or explicitly reposition the contribution as a geometry-motivated regularizer. The empirical section also needs a minimal number of seeds and one standard uncertainty baseline before the calibration claim can be evaluated. The accuracy of the D/(kappa+D) interpolant in the moderate-kappa regime should be checked quantitatively against Eq. (6), since the current Figure 3(A) claim is not supported by the asymptotic expansions presented."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know before you read this. First, the central parameterization is genuinely nice: one scalar per normalized layer, sigma_eff, with a dimension-aware closed-form KL that matches the vMF-vs-uniform KL asymptotics. Second, the paper overclaims what that scalar buys you. The CIFAR-10 result rests on a single unseeded run against a no-KL baseline, and the 'variational' framing doesn't survive contact with the actual forward pass.\n\nWhat's actually new: the per-layer effective noise parameter, the interpolant sigma_u^2 ≈ D/(kappa+D), the closed-form KL_approx = (D-1)/2 log(1 + D/(D-1) sigma_eff^-2), and the regime inversions. These are useful and not in the cited radial/spherical BNN work. The synthetic checks at D=100 (Figure 3) show the approximations track Monte Carlo and the numerical KL across regimes, and the student-teacher recovery (Figure 4) is a clean sanity check. The math is standard vMF moment machinery, but put together in a way that is practical and implementation-friendly. The limitations paragraph is honest about early conv layers and warm-up.\n\nThe biggest gap is conceptual: the KL term is the vMF-vs-uniform KL, while the forward pass injects independent Gaussian noise after BN. These are only matched in first and second moments. For a Gaussian likelihood that makes the expected NLL exact, but for softmax classification the expected log-likelihood depends on higher moments, and the paper never derives an ELBO for the actual stochastic model. So the objective is at best a KL-shaped regularizer, not a variational bound. That doesn't kill the method, but it changes what the experiments can claim.\n\nThe empirical section is thinner than the headline suggests. One dataset, one architecture, one baseline without KL, no seeds, no error bars, no code. The reported ~5.6x ECE reduction is not something I'd trust as reported. Figure 6 is also unclear: if kappa is chosen to fit the empirical landscape, the comparison is circular. The paper doesn't say how the analytic kappa is set.\n\nAll that said, the theory is internally consistent and the approximations are checked against exact numerics, so this is a real contribution with a soft empirical base rather than a hand-wavy one.\n\nWho should read it: anyone working on variational inference for BNNs, especially with normalization layers. They'll get a neat parameterization and a useful closed-form KL. But they should treat the CIFAR-10 numbers as preliminary. It deserves peer review — the idea is worth referees' time — but I'd make the authors add baselines, seeds, error bars, code, and clarify the objective's status before publication.","headline":"Worth a serious look for its clean sigma_eff parameterization and closed-form KL, but the empirical claims are thin and the variational interpretation doesn't quite hold together as stated.","tokens_in":8588,"tokens_out":2896,"would_cite":false,"duration_ms":28214,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62H11","68T07","62F15"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that in normalized neural networks, weight uncertainty can be compressed into one per-layer effective noise scalar with a closed-form, dimension-aware KL, yielding calibrated predictions without accuracy loss.","keywords":["Bayesian neural networks","von Mises-Fisher","directional statistics","variational inference","batch normalization","calibration","effective noise","dimension-aware KL"],"falsifier":"Use a first-layer convolution with small input dimension, say $D=3$ for RGB images, compute the exact numerical vMF KL at intermediate concentrations, and compare it with $\\operatorname{KL}_{\\mathrm{approx}}$; if the gap changes the optimal $\\sigma_{\\mathrm{eff}}$ and ECE degrades on CIFAR-10 relative to the full pipeline, the claim that one closed-form scalar works across all normalized layers is falsified.","tokens_in":7544,"feed_emoji":"🎲","tokens_out":6988,"duration_ms":65843,"temperature":0.7,"pith_summary":"Bayesian neural networks usually place Gaussian uncertainty on every weight, which fights the geometry of networks that use normalization. This paper claims that after normalization only the direction of a weight vector matters, so uncertainty should live on the unit sphere. It reduces each layer's uncertainty to one scalar, the effective post-normalization noise $\\sigma_{\\mathrm{eff}}$, and derives a closed-form dimension-aware KL penalty for it. If correct, variational BNN training becomes a small modification of standard normalized networks and improves calibration without losing accuracy.","feed_headline":"One noise scalar per layer cuts calibration error ~5.6x","feed_subtitle":"One learned noise value per layer replaces fragile Gaussian KLs and cuts ECE by about 5.6x on VGG16+BN.","key_machinery":"The central object is the von Mises-Fisher posterior on the unit sphere $S^{D-1}$ for weight directions, the spherical analogue of a Gaussian. It is compressed through two formulas: the activation-variance interpolant $\\sigma_u^2(\\kappa) \\approx D/(\\kappa+D)$ and the effective noise $\\sigma_{\\mathrm{eff}} = \\sigma_u / A_D(\\kappa)$, where $A_D(\\kappa) = I_{D/2}(\\kappa)/I_{D/2-1}(\\kappa)$ is the mean resultant length. These yield one optimized scalar per normalized layer and the closed-form KL $\\operatorname{KL}_{\\mathrm{approx}}(\\sigma_{\\mathrm{eff}},D) = \\frac{D-1}{2}\\log\\left(1 + \\frac{D}{D-1}\\sigma_{\\mathrm{eff}}^{-2}\\right)$, whose $(D-1)/2$ prefactor makes the regularization scale with layer input dimensionality.","core_discovery":"On the paper's own terms, a von Mises-Fisher posterior over weight directions with a uniform spherical prior is the right variational family for normalized networks. The central quantitative claim is that this posterior is equivalent, in expectation, to injecting Gaussian noise $\\sigma_{\\mathrm{eff}}$ after normalization, with $\\sigma_{\\mathrm{eff}} = \\sigma_u / A_D(\\kappa)$ and $\\sigma_u^2 \\approx D/(\\kappa+D)$. The KL of the directional posterior against the uniform prior has the closed form $\\operatorname{KL}_{\\mathrm{approx}}(\\sigma_{\\mathrm{eff}},D) = \\frac{D-1}{2}\\log\\left(1 + \\frac{D}{D-1}\\sigma_{\\mathrm{eff}}^{-2}\\right)$, which matches the true vMF KL in both tight and broad regimes. Optimizing NLL plus a weighted sum of these per-layer KL terms improves calibration, reducing ECE by roughly a factor of $5.6\\times$ on CIFAR-10 with VGG16+BN, while keeping accuracy at least competitive and lowering NLL.","pith_inferences":["A testable extension is to replace $\\operatorname{KL}_{\\mathrm{approx}}$ with the exact numerical vMF KL only for the lowest-dimensional layers; if calibration holds and training cost stays flat, the recipe transfers to small-$D$ layers with a hybrid objective.","Because $\\sigma_{\\mathrm{eff}}$ is just additive Gaussian noise after BN, it could be interpreted as an alternative to dropout or label smoothing; running the same VGG16 baseline with tuned Gaussian noise but no KL would isolate how much of the calibration gain comes from the KL rather than the noise injection.","The structured per-layer $\\sigma_{\\mathrm{eff}}$ profile, larger at mid-depth, hints at a cheap diagnostic for layer difficulty or a target for layer-wise learning-rate schedules, though the paper does not pursue this.","If the dimension-aware KL is as accurate as claimed, the $(D-1)/2$ prefactor already sets the regularization scale, so wider or deeper models could be trained without any $\\beta$ search; this is a direct transfer prediction that future experiments could test."],"forward_implications":["A normalized network can be trained with one extra scalar per layer and a KL warm-up, with the KL penalty scaling automatically with layer input dimension.","Sampling vMF weight directions is replaced by adding Gaussian noise of size $\\sigma_{\\mathrm{eff}}$ after batch normalization, so the training forward pass uses only standard operations.","At test time the noise can be turned off, leaving a deterministic network, or a few Monte Carlo passes can provide predictive uncertainty.","The reported CIFAR-10 VGG16+BN result predicts that this recipe lowers expected calibration error by roughly a factor of $5.6\\times$ while keeping accuracy competitive and NLL lower.","The same geometric alignment should apply to other normalized architectures, though regression and non-BN pipelines may require minor adaptations."],"supporting_citations":[{"why":"Supplies the von Mises-Fisher distribution, its normalizer, and the mean resultant length $A_D(\\kappa)$ used throughout.","marker":"[14]"},{"why":"Provides hyperspherical clustering and vMF normalization machinery that the posterior builds on.","marker":"[1]"},{"why":"Batch normalization is the basis of the scale-invariance argument that only weight directions matter.","marker":"[9]"},{"why":"Weight normalization formalizes the direction/magnitude split that motivates a spherical posterior.","marker":"[20]"},{"why":"Scale-symmetry analyses of ReLU networks support treating weight direction as the informative degree of freedom.","marker":"[15, 18]"},{"why":"Supplies the vMF KL asymptotics that the closed-form $\\operatorname{KL}_{\\mathrm{approx}}$ is designed to match.","marker":"[21]"},{"why":"The reparameterization trick is used to backpropagate through the $\\sigma_{\\mathrm{eff}}$ noise in the forward pass.","marker":"[12]"},{"why":"Provides the expected calibration error metric and the calibration challenge that motivates the experiments.","marker":"[8]"}],"fun_headline_variants":["One noise scalar per layer replaces fragile KL, cuts ECE 5.6x","vMF posterior makes BNNs precise: one scalar per layer, 5.6x better calibration","Normalized BNNs: single scalar per layer yields 5.6x ECE cut","Directional posterior slashes calibration error 5.6x with one scalar per layer","Precious BNNs: one learned noise value per layer gives 5.6x calibration gain"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the compact formulas $\\sigma_u^2 \\approx D/(\\kappa+D)$ and the closed-form KL stay accurate for the whole training trajectory, including intermediate concentrations and small input dimensions; the paper validates them synthetically at $D=100$ and notes that early convolutional layers weaken the high-dimensional asymptotics.","fun_headline_variants_meta":{"raw":{"variants":["One noise scalar per layer replaces fragile KL, cuts ECE 5.6x","vMF posterior makes BNNs precise: one scalar per layer, 5.6x better calibration","Normalized BNNs: single scalar per layer yields 5.6x ECE cut","Directional posterior slashes calibration error 5.6x with one scalar per layer","Precious BNNs: one learned noise value per layer gives 5.6x calibration gain"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000747,"raw_usage":{"total_tokens":3343,"prompt_tokens":976,"completion_tokens":2367,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":592,"completion_tokens_details":{"reasoning_tokens":2246}},"tokens_in":592,"tokens_out":2367,"duration_ms":17509,"temperature":1.0,"reasoning_tokens":2246,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:25:52.713100+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Use a first-layer convolution with small input dimension, say $D=3$ for RGB images, compute the exact numerical vMF KL at intermediate concentrations, and compare it with $\\operatorname{KL}_{\\mathrm{approx}}$; if the gap changes the optimal $\\sigma_{\\mathrm{eff}}$ and ECE degrades on CIFAR-10 relative to the full pipeline, the claim that one closed-form scalar works across all normalized layers is falsified.","supporting_citations":[{"cited_title":"Mardia and Peter E","cited_arxiv_id":null,"evidence_quote":"Supplies the von Mises-Fisher distribution, its normalizer, and the mean resultant length $A_D(\\kappa)$ used throughout."},{"cited_title":"Dhillon, Joydeep Ghosh, and Suvrit Sra","cited_arxiv_id":null,"evidence_quote":"Provides hyperspherical clustering and vMF normalization machinery that the posterior builds on."},{"cited_title":"Ioffe and C","cited_arxiv_id":null,"evidence_quote":"Batch normalization is the basis of the scale-invariance argument that only weight directions matter."},{"cited_title":"Salimans and D","cited_arxiv_id":null,"evidence_quote":"Weight normalization formalizes the direction/magnitude split that motivates a spherical posterior."},{"cited_title":"Kingma and Max Welling","cited_arxiv_id":null,"evidence_quote":"The reparameterization trick is used to backpropagate through the $\\sigma_{\\mathrm{eff}}$ noise in the forward pass."},{"cited_title":"On calibration of modern neural networks.International Conference on Machine Learning, pp","cited_arxiv_id":null,"evidence_quote":"Provides the expected calibration error metric and the calibration challenge that motivates the experiments."}],"review_version":2}