{"id":"1244c808-7a0e-4b23-905d-0f8c528ca80e","arxiv_id":"2607.09266","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Per-layer LionVote shows Lion's prescribed rate is 2–2.8× too high on ViT layer types and lifts ViT-Tiny/CIFAR-100 accuracy to 69.7% vs Lion's 69.0%.","lead":"LionVote adapts Lion's learning rate per layer using two derived diagnostics and a validation-loss tiebreaker, finding Lion runs 2–2.8× too hot on ViT layers with a 32% type gap. The method beats Lion on ViT-Tiny/CIFAR-100 but not on uniform CNNs or CIFAR-10, so value tracks architecture and task.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The 2.6–2.8\times miscalibration factors are still produced by LionVote’s own state machine, so the headline quantification remains mechanism-dependent even after the paper’s controls.","rationale":"The reader correctly isolates the single load-bearing soft spot: the numerical miscalibration factors are generated by the very voting loop whose utility is being claimed. The paper’s controls (raw diagnostics, static multipliers, lower base LR, WD sweep) are honest and partially mitigate the circularity, which is why CONDITIONAL with medium correctness risk remains the right call; they do not fully close it. No stronger internal inconsistency or unaddressed statistical flaw rises above this concern. The concrete test above is cheap (two already-ablated variants) and would settle whether the 2.6–2.8\times numbers are robust or artifactual. Scope, dormant Vote 2, and CIFAR-10 degradation are already flagged by the authors and do not overturn the verdict.","tokens_in":28609,"tokens_out":590,"duration_ms":19833,"concrete_test":"Re-train the 8-seed ViT-Tiny/CIFAR-100 suite under the symmetric update rule (already shown to reach 70.54 %) and under a pure additive vote accumulator without the asymmetric reset-to-zero; report mean compound levels and attn/norm multiplier ratio at epoch 300. If either variant shifts the reported 2.6–2.8\times / 2\times factors by more than ~20 % (or erases the 32 % disparity), the headline quantification is confirmed to be rule-specific.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim (Table 3 / §5.1) is that Lion’s prescribed rate is 2.6–2.8\times too high for attention/MLP and ~2\times too high for normalisation (32 % disparity). Those multipliers are exactly the compound levels s_i that LionVote itself generates via the asymmetric reset rule, cadence-c voting, Vote-1/2 thresholds, and the global validation-loss tiebreaker. Raw Lion diagnostics (App. B.16) confirm only the qualitative ordering (attn/mlp more unstable than norm); they do not independently recover the numerical factors. The static-type and lower-global-LR controls (App. B.15) show that those particular numbers cannot be replaced by a constant schedule, but they do not demonstrate that a different adaptive rule would have settled at the same 0.36–0.50 multipliers. Because the compound multiplier also scales weight decay, the joint LR+WD interpretation inherits the same dependence. Thus the strongest scientific number in the abstract is still an output of the mechanism being evaluated rather than an external measurement of Lion.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes LionVote, a stateful per-layer learning-rate mechanism for the Lion optimizer. Each parameter tensor holds an integer compound level that multiplies the base LR (and decoupled weight decay) by exp(s_i * β_1/2); levels are updated every c epochs by two diagnostics—epoch-mean gradient cosine alignment and momentum-to-gradient norm ratio—resolved by a global validation-loss tiebreaker when the local votes conflict or abstain. Thresholds for the two votes are derived from Sheppard’s sign-agreement identity, the EMA time-constant identity β^τ_2 = e^{-1}, and a noise-floor argument for cross-entropy; cadence and max level are structurally bounded and then ablated. On ViT-Tiny/CIFAR-100 (8 seeds) LionVote at c=8 reaches 69.71 % vs Lion 68.95 % (p=0.017) and AdamW 68.75 %; the same architecture on CIFAR-10 and both WideResNet configurations show no consistent gain, and tuned SGD remains dominant on the CNNs. Compound-level trajectories are used to claim that Lion’s prescribed rate is 2.6–2.8\times too high for attention/MLP and ~2\times too high for normalisation (32 % cross-type disparity). Ablations (v1-only, v2-only, no-tiebreaker, symmetric update) and controls (lower global LR, static type multipliers, WD sweep) are reported.","tokens_in":28888,"tokens_out":1495,"duration_ms":13490,"significance":"If the central claims hold, the work supplies three reusable pieces: (i) a derivation methodology for per-layer voting thresholds that is largely free of free parameters (Appendix A.1–A.6), (ii) a quantified, architecture-aware miscalibration statement for a sign-based optimizer that complements Zhao et al.’s qualitative observation on language models, and (iii) evidence that the value of per-layer adaptation tracks architectural heterogeneity and is task-dependent. Strengths that raise the bar relative to typical optimizer papers include 8-seed replication with Welch tests, explicit ablations of each vote and of the update rule, and control experiments that rule out simple global LR or WD fixes and static type multipliers. The portability analysis (Table 18) that separates optimizer-agnostic from Lion-specific components is also useful for follow-on work.","major_comments":[{"comment":"The headline numerical claim in the abstract and §5.1 (Table 3)—that Lion’s effective scale is 2.6–2.8\times too high for attention/MLP and ~2\times too high for normalisation—is produced by LionVote’s own compound-level trajectories (asymmetric reset, cadence-c voting, Vote-1/2 thresholds, and the validation-loss tiebreaker). Appendix B.16 shows that raw Lion gradient cosines already exhibit the qualitative attn/mlp < norm ordering, and Appendix B.15 shows that static type multipliers and a uniformly lower global LR fail to recover the accuracy gain; those controls establish that the numbers cannot be replaced by a constant schedule, but they do not establish that a different adaptive rule would settle at the same 0.36–0.50 multipliers. Because the compound multiplier also scales weight decay, the joint LR+WD interpretation inherits the same dependence. The abstract and §5.1 should theref","section":null},{"comment":"Cadence c=8 is selected by ablation from {4,6,8} (§4.2, §5.3, Appendix A.6, B.3). The primary accuracy comparison on ViT-Tiny/CIFAR-100 is therefore partially post-hoc. The paper correctly notes that the c=8 vs Lion p-value (0.017) is marginal after Holm–Bonferroni and that the control comparisons (lower LR, static multipliers) are the stronger evidence; those controls should be elevated to the main text and the abstract should not present the 69.7 % figure as if cadence were fixed a priori.","section":null},{"comment":"The validation-loss tiebreaker is the dominant decision path (~59 % of per-parameter votes, Appendix B.6) and is computed on the same validation set used for reporting. §5.3 flags the issue and suggests a held-out split for future work; given that the entire accuracy gain on ViT/C100 is attributed to the tiebreaker (Table 2: full system vs notie +1.07 pp, p=0.003), a held-out confirmation (or an explicit statement that the reported numbers use the training-time validation signal) is load-bearing for the claim that the mechanism improves generalisation rather than merely fitting the monitored split.","section":null}],"minor_comments":[{"comment":"Table 1 reports LionVote (c8) 69.71 % while the abstract rounds to 69.7 % and Lion to 69.0 %; keep one consistent rounding convention.","section":null},{"comment":"Figure 2 caption says “~33 % difference in effective scale” while the text and Table 3 use 32 %; align the numbers.","section":null},{"comment":"Appendix A.1 Remark notes that the coordinate-isotropy assumption is approximate; a short sentence in the main text (§3.3) would help readers who do not read the appendix.","section":null},{"comment":"The symmetric-update variant reaches the study’s highest single accuracy (70.54 %) but is tested on only 2 of 4 configurations; either expand or clearly mark it as exploratory in the abstract/conclusion.","section":null},{"comment":"Typographical consistency: “normalisation” vs “normalization”, “V ote” spacing artefacts in several places, and “LionV ote” vs “LionVote”.","section":null}],"recommendation":"major_revision","confidential_remarks":"The manuscript is carefully written and the experimental hygiene (8 seeds, controls, ablations) is above the median for optimizer papers. The main risk is over-claiming the numerical miscalibration factors as external measurements of Lion rather than as the operating point of this particular adaptive rule. If the authors reframe those numbers and elevate the controls, the paper becomes a solid, reusable contribution; if they insist on the stronger reading, the circularity concern remains. Scope (CIFAR + Tiny models only) is acknowledged and is acceptable given the replication budget, but the journal should expect the authors not to over-generalise the accuracy numbers."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful core is a stateful per-layer rate for Lion (compound levels + two diagnostics + validation-loss tiebreaker) whose thresholds are mostly derived rather than grid-searched, plus a quantified claim that Lion’s prescribed scale is systematically too high on ViT-Tiny, with a ~32% attention-vs-norm disparity a single global rate cannot match.\n\nWhat is actually new is the combination: LARS/LAMB are per-layer but stateless, schedule-free methods stay global, Muon hardcodes layer types, LANTON adapts instantaneously. The paper maps that gap cleanly. The derivations (Sheppard sign-agreement for Vote 1, EMA time-constant for Vote 2, cosh degradation for the LR exponent) are explicit and reusable even if you never adopt the full system. Eight seeds, Welch tests, ablations (v1/v2/tiebreaker/symmetric), and controls (lower global LR, static type multipliers, WD sweep) are better than average for this literature. Raw Lion diagnostics already show the attn/mlp < norm ordering without voting, so the qualitative signal is not pure circularity.\n\nSoft spots, in proportion: the headline 2.6–2.8× factors are still outputs of LionVote’s own state machine (asymmetric reset, cadence, tiebreaker). Controls show those numbers cannot be replaced by a constant schedule, but they do not prove a different adaptive rule would settle at the same multipliers; the joint LR+WD reading inherits that dependence. Cadence c=8 is ablated, so the main accuracy comparison is partly post-hoc and only marginally significant after the authors’ own multiple-comparison caution. Vote 2 is nearly dormant. Gains reverse on CIFAR-10; SGD still wins on WideResNet. Scope is deliberately CIFAR + Tiny models. No code artifact.\n\nThis is for people who care about sign-based optimizers, layer-type heterogeneity, or principled hyperparameter design. It is not a default training upgrade. The math and tables are readable, the limitations section is honest, and the controls are real. I would send it to peer review; a serious referee can decide how much of the quantification survives. Worth engaging if you work in this corner.","headline":"Solid methods paper: derived per-layer voting for Lion plus a real (if mechanism-tied) layer-type miscalibration signal on small ViTs; gains are small, scope is narrow, but the work is careful enough to read and referee.","tokens_in":29544,"tokens_out":561,"would_cite":true,"duration_ms":6363,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"At Lion's prescribed learning rate on ViT-Tiny, attention and MLP layers run 2.6–2.8× too hot and normalisation layers about 2× too hot—a 32% gap no single global rate can fix.","keywords":["Lion optimizer","per-layer learning rates","vision transformers","adaptive optimizers","gradient direction stability","momentum health","compound level","sign-based optimization"],"falsifier":"On the same ViT-Tiny/CIFAR-100 setup, run a static per-layer-type schedule that freezes the epoch-300 multipliers from the first step (or a global rate lowered to match the average effective rate) and check whether either closes the accuracy gap to LionVote; the paper's own controls already show they do not, so a successful reverse would falsify the dynamic-adaptation claim.","tokens_in":29440,"feed_emoji":"🦁","tokens_out":814,"duration_ms":6711,"temperature":0.7,"pith_summary":"The paper argues that Lion's single global learning rate systematically mis-scales different layer types. On a small vision transformer trained on CIFAR-100, per-layer compound levels that the method itself discovers show attention and MLP parameters receive an effective scale 2.6–2.8 times too high, while normalisation layers sit about 2 times too high—a 32% cross-type disparity that no uniform rate can reproduce. LionVote turns that measurement into a mechanism: each parameter tensor keeps a persistent integer level updated every few epochs by two diagnostics (gradient-direction stability and momentum health) that a validation-loss tiebreaker resolves when they conflict. The voting thresholds are derived from geometric identities, the EMA time constant, and a noise-floor estimate rather than grid search. On the same ViT/CIFAR-100 workload the method reaches 69.7% top-1 accuracy against Lion's 69.0% (statistically significant over eight seeds) and AdamW's 68.8%. The value of the adaptation is not universal: it tracks architectural heterogeneity and is task-dependent, and on uniform residual CNNs tuned SGD with cosine annealing remains strongest.","feed_headline":"Lion runs attention layers 2.6–2.8× too hot","feed_subtitle":"A per-layer vote derived from geometry and the EMA time constant recovers a 0.76-point gain on ViT/CIFAR-100","key_machinery":"The compound level: a persistent integer per parameter tensor that multiplies the base learning rate by exp(s · β1/2). It is updated every c epochs by two derived votes (gradient-direction cosine stability and momentum-to-gradient norm health) resolved by a validation-loss tiebreaker, with an asymmetric reset rule that revokes accumulated level on a single opposing vote.","core_discovery":"At Lion's prescribed learning rate on ViT-Tiny/CIFAR-100, the effective scale is 2.6–2.8× too high for attention and MLP parameters and roughly 2× too high for normalisation layers—a 32% cross-layer-type disparity that a single global rate cannot reproduce. LionVote's per-layer compound-level trajectories measure that miscalibration and convert it into a statistically significant accuracy gain (69.71% vs Lion 68.95%, p=0.017 over eight seeds).","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Lion runs attention and MLP layers 2.6–2.8× too hot","LionVote fixes 32% cross-layer scale gap on ViT/CIFAR-100","Per-layer vote reveals Lion over-scales attention by 2.6–2.8×","Lion's global rate leaves norm layers ~2× too high on ViT","LionVote recovers 0.7-pt gain from Lion layer miscalibration"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"The headline 2.6–2.8× miscalibration factors rest on the compound levels that LionVote's own voting loop produces; if those levels are an artifact of the vote rules, cadence, and tiebreaker rather than a true property of Lion, the quantitative claim collapses.","fun_headline_variants_meta":{"raw":{"variants":["Lion runs attention and MLP layers 2.6–2.8× too hot","LionVote fixes 32% cross-layer scale gap on ViT/CIFAR-100","Per-layer vote reveals Lion over-scales attention by 2.6–2.8×","Lion's global rate leaves norm layers ~2× too high on ViT","LionVote recovers 0.7-pt gain from Lion layer miscalibration"]},"model":"grok-4.5","effort":"low","cost_usd":0.004102,"raw_usage":{"total_tokens":1289,"prompt_tokens":812,"num_sources_used":0,"completion_tokens":95,"cost_in_usd_ticks":41020000,"prompt_tokens_details":{"text_tokens":812,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":382,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":812,"tokens_out":95,"duration_ms":3761,"temperature":1.0,"reasoning_tokens":382,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-13T04:16:45.284081+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On the same ViT-Tiny/CIFAR-100 setup, run a static per-layer-type schedule that freezes the epoch-300 multipliers from the first step (or a global rate lowered to match the average effective rate) and check whether either closes the accuracy gap to LionVote; the paper's own controls already show they do not, so a successful reverse would falsify the dynamic-adaptation claim.","supporting_citations":[],"review_version":1}