{"id":"a63ac7af-9be0-4227-90c7-1ff218ab1300","arxiv_id":"2608.02991","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Joint Regularized Inverse (JRI) applies a capped inverse spectral map to the combined weight-bias momentum matrix and uses the transformed bias column as the physical bias update, yielding small consistent gains on BERT-mini/IMDb.","lead":"This paper tests whether spectral optimizers that reshape weight updates but leave biases to Adam are leaving performance on the table. On a four-layer BERT-mini sentiment model, sharing one spectral transform between weight and bias raises selected test accuracy from 84.90% to 85.74% and lowers test loss, with all five seeds improving.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"JRI gain over affine probe conflates joint-bias direction with reduced bias norm; the paper's own §6.2 admits no ablation isolates them, so the central mechanism claim is not yet supported.","rationale":"The paper is a careful, small-scale mechanism study, and the authors deserve credit for explicitly flagging the exact confound I would otherwise raise. The strongest claim is not that JRI works—that is well supported by five paired seeds plus a 22-seed stability run—but that joint affine spectral allocation, as opposed to a smaller bias update, is the cause. That causal attribution is the load-bearing part of the paper's title and conclusion, and it is not yet isolated. The comparison between JRI and affine probe is the only one that could separate coupling from norm, and it changes both. The paper's proposed future ablation in §6.2 is exactly the missing experiment; until it is run, 'joint spectral allocation' remains a hypothesis rather than a demonstrated mechanism. This does not undermine the scoped empirical contribution, and it does not require changing the conditional verdict. The reader identified the same weakness; agreement is total. I did not find an independent fatal flaw: exact-SVD and norm-matched controls rule out the obvious SVD-implementation confound, matched train-accuracy comparisons rule out a time-axis shift, and the one-sided cap is a sensible guard. The absence of a permanent repository is an availability concern, not a correctness one. Verdict: keep conditional acceptance, with the mechanism claim explicitly deferred until the norm/direction ablation is run.","tokens_in":12000,"tokens_out":9076,"duration_ms":97658,"concrete_test":"Run the decisive ablation the paper itself proposes in §6.2 on the same five paired seeds and protocol as §4.2. Store the JRI weight updates D_W from the full JRI run and hold them elementwise fixed; vary only the bias update: (1) d_b from Equation (18), (2) d_b rescaled to the Adam-reference norm (0.02095), (3) Adam-reference direction rescaled to JRI's norm (0.00301), (4) zero bias, and (5) fixed-ratio bias matched to JRI's module-level ratios. If condition (3) or (4) reproduces JRI's selected test loss 0.3291±0.0024 and accuracy 85.738±0.180, then reduced bias norm, not joint spectral direction, explains the gain. If only condition (1) reproduces it, the coupling claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central incremental claim—that joint spectral allocation, not merely a smaller bias update, drives JRI's gain—rests on a confounded comparison. JRI versus affine probe changes three things at once: bias direction (cosine with weight-induced boundary motion shifts from +0.030 to −0.137), bias norm (0.02095 to 0.00301), and the fact that both weight and bias emerge from one SVD. Because Table 4 shows the eligible weight-update norm is essentially identical (0.09595 vs 0.09600), all observable gain is mediated by the bias path. The paper is candid: Section 6.2 states that 'the data do not yet identify whether the improvement requires the exact joint-bias direction, its reduced norm, or both.' The fixed α in Equations (7)–(8) is an additional confound: it simultaneously controls bias participation in mode identification and physical bias-output scale, as admitted in Section 3.1. A simple scalar reduction of the Adam bias update—no spectral coupling, no joint SVD—could plausibly reproduce the lower boundary-function share and the loss reduction. The five-seed consistency and 22-seed replication establish that JRI is stably better than the tested controls, but they do not establish why. The central mechanism claim is therefore not yet supported, even though the empirical finding is credible and honestly scoped.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies whether the common practice of spectrally shaping only weight matrices while optimizing bias vectors with Adam is functionally neutral for affine layers. It defines a joint momentum matrix A=[M_W, alpha*m_b] and applies a capped regularized-inverse spectral map to the whole matrix, then uses the reconstructed last column as the physical bias update (JRI). The authors run a strict five-seed, four-method ablation on BERT-mini/IMDb: exact-SVD Muon, weight-only inverse shaping, affine-probe inverse shaping (bias enters the SVD but is discarded), and JRI. They report that inverse weight shaping improves on exact Muon, that the affine probe does not improve on weight-only inverse shaping, and that JRI improves further on the probe with all five seeds, with a separate 22-seed JRI-only replication matching the five-seed numbers. The paper interprets the results as evidence that joint mode identification and joint physical bias output work together, while explicitly flagging several open confounds.","tokens_in":12248,"tokens_out":5711,"duration_ms":58856,"significance":"If the mechanism claim were established, the paper would make a modest but conceptually useful contribution: it would show that spectral optimizers should treat affine layers as joint allocation units rather than optimizing bias separately. The empirical work has genuine strengths: five-seed paired comparisons with shared initialization and data order, exact SVD rather than approximate orthogonalization, matched weight-update norms, a probe condition that isolates SVD participation from physical bias output, functional diagnostics on shape versus boundary motion, and a machine-readable data package. The authors are also unusually candid about limitations, including the confound analyzed below. The central incremental claim, however, is not yet supported because the JRI-versus-probe contrast changes bias direction and bias norm simultaneously while the weight update is held essentially fixed.","major_comments":[{"comment":"The incremental claim that 'joint affine spectral allocation' drives the JRI gain rests on a confounded contrast. JRI versus affine probe changes the bias direction (cosine from +0.030 to -0.137), the bias-update norm (0.02095 to 0.00301), and the fact that both updates share one SVD, while the eligible weight-update norm is held essentially constant (Table 4). The paper itself states in §6.2 that 'the data do not yet identify whether the improvement requires the exact joint-bias direction, its reduced norm, or both.' Because all of the observable gain is mediated by the bias path under a fixed weight norm, a scalar reduction of the Adam bias update, with no joint SVD, could plausibly reproduce the improvement. The ablation proposed in §6.2 (joint direction with joint norm, joint direction rescaled to Adam norm, zero bias, fixed-ratio bias, Adam bias, with the weight update held elementwise fixed) is exactly what is needed; until it is run, the paper's central mechanism conclusion is not supported, even though the empirical ranking is credible and honestly scoped.","section":"§5.3, §5.5, §6.2; Table 4"},{"comment":"The fixed factor alpha = sqrt(n) in Eq. (8) is a second confound. Because the transformed bias column is not divided by alpha after reconstruction, alpha simultaneously controls how strongly the bias participates in mode identification and the physical scale of the output bias update. The affine-probe condition uses the same alpha but discards the output, so the probe-versus-JRI comparison cannot separate changes in joint direction from changes in effective bias step size. The authors acknowledge this in §3.1 and propose the cleaner parameterization A=[M_W, alpha_probe*m_b], Delta_b = -eta*gamma_out*d_b in Eq. (23), but the experiments do not vary alpha_probe and gamma_out independently. A minimal control that renormalizes the JRI bias output to the Adam-reference norm, or vice versa, would determine whether the gain survives when the bias norm is matched.","section":"§3.1, Eqs. (7)–(8), (23)"}],"minor_comments":[{"comment":"The column header 'Pairedt p d z' is ambiguous: after the 'Wins' column the rows show only two numeric values, so it is unclear whether the reported quantities are the paired t statistic, the p-value, Cohen's d, or the effect-size z. Please label the columns explicitly or add a table note explaining the statistics and their degrees of freedom.","section":"Table 3"},{"comment":"Figure 5 mixes the five strict-ablation seeds with the independent 22-seed JRI-only runs in a single scatter panel; the legend does not make clear that the 22-seed points come from a different, JRI-only harness. A panel split or explicit marker legend would prevent readers from inferring that the 22-seed points participated in the four-way paired comparison.","section":"Figure 5"},{"comment":"References [10], [11], and [12] contain placeholder '???' for publisher locations; these should be completed before publication, as should the missing acknowledgements, funding statement, code repository URL, and competing-interest statement noted in the text.","section":"References"},{"comment":"The sentence describing the bias reduction as 'approximately 14.4% as large' is correct but could be stated more directly as a factor of roughly 6.96; consider giving both forms for readability.","section":"§5.5"}],"recommendation":"major_revision","confidential_remarks":"The empirical package and the authors' explicit honesty about their own confounds are strengths; the missing control, however, is not a presentation issue but the decisive experiment for the paper's central claim. The proposed §6.2 ablation is well specified and should be feasible within the same harness. If the authors run it and the gain persists under a matched bias norm, the contribution would be solid. I would also encourage the editor to consider whether a single dataset/model configuration is sufficient for the journal's scope, though the authors already acknowledge this and frame the work as a mechanism study."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a controlled, honest mechanism study on one small model, and the empirical comparison is run carefully. What is not settled is why JRI wins: the paper's own §6.2 admits the joint-bias direction and its reduced norm are entangled.\n\nThe four-way ablation is the real strength. Exact SVD everywhere, weight-update norms matched, five paired seeds with shared initialization and data order, checkpoints selected on validation loss, and a separate 22-seed stability run. The affine-probe control is genuinely informative: bias can substantially alter the joint SVD without improving over weight-only inverse, so the simple \"bias as probe\" story is rejected. JRI then gives a consistent, small gain (0.25 pp accuracy over the probe, all five seeds; the 22-seed replication lands at 85.743, nearly identical). The paper is also unusually upfront about its scope and limitations, and it ships machine-readable tables.\n\nThe soft spot is the central mechanism claim. JRI versus the affine probe changes three things at once: bias direction, bias norm (0.021 to 0.003), and the fact that both weight and bias come from one SVD. The paper states in §6.2 that the data do not identify whether the improvement requires the exact joint-bias direction, its reduced norm, or both. That is the right caveat, but it means the headline interpretation—joint affine spectral allocation as a distinct mechanism—is not yet supported. A simple scalar reduction of the Adam bias update could plausibly reproduce the gain. The fixed α also couples bias participation in mode identification with physical bias scale (admitted in §3.1), and there is no sensitivity analysis for λ, g_max, or α. The lack of a public repository is minor but annoying; the processed CSVs are at least provided.\n\nThese are addressable issues, not fatal flaws. The empirical finding—that JRI is stably better than the tested controls—is credible and honestly scoped. If you work on spectral optimizers, this is worth a careful read; the negative probe result and the norm confound are genuinely useful. It deserves a serious referee and, in my view, conditional acceptance, with the decisive ablation (joint direction rescaled to Adam norm, zero bias, fixed-ratio bias) requested before the mechanism claim is accepted.","headline":"A carefully run, honestly scoped empirical study of joint weight-bias spectral shaping, but the central mechanism claim is confounded with bias-norm reduction and needs a decisive ablation.","tokens_in":12811,"tokens_out":2021,"would_cite":true,"duration_ms":23167,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper shows that applying a capped regularized-inverse spectral map to the joint weight-bias momentum matrix, and using the transformed bias column as the actual bias update, yields a small but consistent gain over weight-only…","keywords":["matrix spectral optimization","affine layer coupling","bias update","regularized inverse spectrum","Muon","weight-bias allocation","language model training","SVD-based optimizer"],"falsifier":"Hold the weight update elementwise fixed and compare four bias treatments: the JRI joint direction at its joint norm, the same direction rescaled to the Adam norm, a fixed-ratio scalar bias, and zero bias. If a small scalar bias or zero bias matches the reported $85.738\\%$ accuracy and $0.3291$ loss, then the improved boundary allocation is not required and the spectral-coupling claim is falsified.","tokens_in":11767,"feed_emoji":"📈","tokens_out":9171,"duration_ms":84191,"temperature":0.7,"pith_summary":"Standard matrix spectral optimizers reshape the singular spectrum of weight updates but delegate vector biases to a separate optimizer; this paper asks whether that separation is neutral. It claims it is not: treating an affine layer as a joint momentum matrix $[M_W,\\alpha m_b]$ under one capped regularized-inverse spectral map, and using the transformed bias column as the physical bias update, gives a small but consistent gain. In five-seed paired runs on a BERT-mini classifier trained from scratch on IMDb, the joint regularized inverse reaches $85.738\\pm 0.180\\%$ selected test accuracy and $0.3291$ selected test loss, with every seed improving over the probe baseline. Letting bias alter the joint SVD while still updating bias with Adam gives no reliable improvement over weight-only inverse shaping, so the effect is located in joint allocation rather than in mode identification alone. If correct, the result says spectral optimizers should treat affine layers, not just weight matrices, as the unit of update-budget allocation.","feed_headline":"Joint spectral update of weights and bias beats weight-only Muon","feed_subtitle":"Using the transformed bias column lifts BERT-mini/IMDb test accuracy to 85.738% and cuts loss to 0.3291.","key_machinery":"The load-bearing object is the augmented affine momentum matrix $A=[M_W,\\alpha m_b]\\in\\mathbb{R}^{m\\times(n+1)}$, where $M_W$ is the Nesterov-style weight momentum, $m_b$ is the bias momentum, and $\\alpha=\\sqrt{n}$ is a fixed participation scale. A thin SVD $A=U\\mathrm{diag}(\\sigma_i)V^\\top$ is normalized by the leading singular value, and each mode is reweighted by the capped regularized inverse $h(q_i)=\\min(g_{\\max},1/(q_i+\\lambda))$; the whole matrix is then Frobenius-normalized to the polar-factor target and split into a weight block $D_W$ and a bias column $d_b$, giving physical updates $W\\leftarrow W-\\eta D_W$ and $b\\leftarrow b-\\eta d_b$. Muon, the baseline, is the matrix optimizer that replaces the momentum spectrum with the polar factor $UV^\\top$. The map and the joint matrix together make weight and bias share one modal basis and one update budget; the cap and norm matching make the practical behavior close to suppressing a few dominant modes while keeping the tail flat.","core_discovery":"The paper's central claim is that coupling weight and bias momenta in a single spectral allocation unit, and retaining the transformed bias column as the physical bias update, is a real extension of weight-only spectral optimization. With exact SVD and matched weight-update norms, replacing the flat Muon polar factor on the weight momentum $M_W$ by a capped regularized-inverse map raises selected test accuracy from $84.903\\pm 0.242\\%$ to $85.562\\pm 0.308\\%$. Letting the scaled bias momentum $\\alpha m_b$ alter the joint SVD but discarding the transformed bias (the affine-probe condition) does not improve over weight-only inverse shaping ($-0.074$ points). Only when the reconstructed bias column is used as the actual bias update does performance rise further, to $85.738\\pm 0.180\\%$ accuracy and $0.3291$ loss, with all five seeds improving on four metrics and an independent 22-seed JRI run giving $85.743\\pm 0.203\\%$. The mechanism diagnostics show JRI preserves the eligible weight-update norm, cuts the bias-update norm from $0.02095$ to $0.00301$, lowers boundary-function share from $86.58\\%$ to $78.97\\%$, and flips the cosine between weight-induced boundary motion and explicit bias from $+0.030$ to $-0.137$, so the explicit bias becomes mildly compensatory.","pith_inferences":["Editorial extension: because almost all inverse gains saturate to the cap, a cheap implementation that suppresses only the leading few singular modes and keeps the rest flat could approximate JRI at much lower cost than a full per-step SVD; the paper notes this possibility but does not test it.","Editorial extension: the joint affine object is defined only for the inverse map, so the same coupling could be tried with positive, fractional, or scheduled spectral powers, and the size or sign of the bias benefit may change in those regimes.","Editorial extension: the paper interprets JRI as reducing sample-independent boundary motion, which connects to function-space orthogonalization ideas; a testable question is whether explicit functional orthogonalization plus joint spectral shaping compounds or duplicates this gain.","Editorial extension: the module-level selectivity suggests that a per-module spectral participation scale, rather than one global $\\alpha$, could tune the effect further."],"forward_implications":["Exact-SVD Muon is outperformed by inverse spectral allocation on this task: with matched weight-update norms, selected test accuracy rises from $84.903\\pm 0.242\\%$ to $85.562\\pm 0.308\\%$, all five seeds.","Affine-probe conditioning alone is insufficient: letting bias alter the joint SVD while keeping Adam bias changes selected test accuracy by $-0.074$ points with mixed seed directions.","JRI's incremental gain persists at matched training progress: at 95% train accuracy, JRI's test accuracy is 85.63%, versus 85.44% for weight-only inverse and 84.46% for exact-SVD Muon.","JRI improves solution quality and calibration more than peak classification accuracy, since it has the lowest test-loss AUC and minimum test loss while the final-step accuracy difference is smaller.","Joint allocation is not a uniform bias learning-rate reduction: per-module bias norms relative to the Adam reference range from about 6% to above 1 depending on the module."],"supporting_citations":[{"why":"Provides the Muon polar-factor baseline that every spectral variant is compared against.","marker":"[1]"},{"why":"Supplies the dynamic spectral shaping family $U\\Sigma^pV^\\top$ that motivates choosing an inverse rather than flat spectrum.","marker":"[2]"},{"why":"Establishes the broader Schatten-norm spectral family and the precedent of non-flat, inverted spectra.","marker":"[3]"},{"why":"Gives fractional-power spectral variants, used as context that inverse spectra are an established option.","marker":"[4]"},{"why":"Provides spike-suppression spectral methods whose cap-like behavior approximates the saturated inverse map.","marker":"[5]"},{"why":"Defines the Adam optimizer used for the reference bias update that JRI is compared against.","marker":"[7]"},{"why":"Formalizes the hidden boundary motion $\\Delta W\\mu$ that motivates treating the affine layer jointly.","marker":"[8]"},{"why":"Shows the precedent of jointly handling weight and bias through augmented activations, the conceptual contrast for JRI.","marker":"[10]"},{"why":"Supplies the IMDb dataset and the train/validation/test split used in all experiments.","marker":"[12]"},{"why":"Supplies the BERT architecture that the four-layer BERT-mini model is based on.","marker":"[13]"}],"fun_headline_variants":["Joint spectral update of weights and bias trumps weight-only Muon","Bias-inclusive spectral shaping raises BERT-mini accuracy to 85.7%","Coupling bias momentum in SVD improves beyond weight-only inverse","New joint spectral method outperforms weight-only Muon on IMDb","Spectral coupling of weight and bias updates yields 85.74% accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The improvement of the joint method over the condition that uses bias only to identify spectral directions is attributed to joint spectral allocation; however, the joint method changes the bias direction and shrinks the bias-update norm at the same time, and the data do not separate whether the gain needs the exact joint-bias direction, its reduced norm, or both.","fun_headline_variants_meta":{"raw":{"variants":["Joint spectral update of weights and bias trumps weight-only Muon","Bias-inclusive spectral shaping raises BERT-mini accuracy to 85.7%","Coupling bias momentum in SVD improves beyond weight-only inverse","New joint spectral method outperforms weight-only Muon on IMDb","Spectral coupling of weight and bias updates yields 85.74% accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000589,"raw_usage":{"total_tokens":2887,"prompt_tokens":1190,"completion_tokens":1697,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":806,"completion_tokens_details":{"reasoning_tokens":1601}},"tokens_in":806,"tokens_out":1697,"duration_ms":14097,"temperature":1.0,"reasoning_tokens":1601,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T04:20:59.763530+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Hold the weight update elementwise fixed and compare four bias treatments: the JRI joint direction at its joint norm, the same direction rescaled to the Adam norm, a fixed-ratio scalar bias, and zero bias. If a small scalar bias or zero bias matches the reported $85.738\\%$ accuracy and $0.3291$ loss, then the improved boundary allocation is not required and the spectral-coupling claim is falsified.","supporting_citations":[{"cited_title":"DynMuon: A Dynamic Spectral Shaping View of Muon","cited_arxiv_id":"2605.17109","evidence_quote":"Supplies the dynamic spectral shaping family $U\\Sigma^pV^\\top$ that motivates choosing an inverse rather than flat spectrum."},{"cited_title":"In: Interna- tional Conference on Learning Representations (2015) 17","cited_arxiv_id":null,"evidence_quote":"Defines the Adam optimizer used for the reference bias update that JRI is compared against."},{"cited_title":"Hidden Boundary Motion in Transformer Optimization: Function-Space Orthogonalization of Affine Weight and Bias Updates","cited_arxiv_id":"2607.22927","evidence_quote":"Formalizes the hidden boundary motion $\\Delta W\\mu$ that motivates treating the affine layer jointly."},{"cited_title":"In: Proceedings of the 32nd International Conference on Machine Learning","cited_arxiv_id":null,"evidence_quote":"Shows the precedent of jointly handling weight and bias through augmented activations, the conceptual contrast for JRI."},{"cited_title":"In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp","cited_arxiv_id":null,"evidence_quote":"Supplies the IMDb dataset and the train/validation/test split used in all experiments."}],"review_version":1}