{"id":"6b9f0e09-1aa9-451c-9e5a-5c47df405c6b","arxiv_id":"2607.04531","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Monotone Lyapunov projection of layerwise hidden-state energy suppresses two's-complement overflow under wrapping fixed-point QAT/PTQ, recovering 86.55% MNIST accuracy where unconstrained models collapse to chance.","lead":"Lyapunov energy projection stops fixed-point overflow wrapping from destroying low-precision neural nets during training and inference. This matters for reliable AI on tiny chips and FPGAs where two's-complement wraparound silently flips activation signs.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The global V_max + radial projection may leave individual activations outside the fixed-point range, so the overflow-suppression claim is not guaranteed by the stated construction.","rationale":"The reader correctly flags the ad-hoc global V_max and the lack of per-layer/per-accumulator constraints as the weakest assumption. That assumption is load-bearing for the strongest claim: without a guarantee that energy control implies elementwise range safety, the dramatic overflow drop (11–14% → <0.012%) and the recovery of 86.55% accuracy cannot be attributed to the Lyapunov construction itself. The concrete test above isolates exactly this gap; a positive result would strengthen the paper, a negative one would relegate the method to an empirical regulariser whose success is unexplained. Because the evaluation remains confined to a single tiny MNIST transformer and no saturation/A2Q baselines are present, the verdict stays CONDITIONAL; the concern does not force a harsher REJECT but confirms that the reader’s caution is well-placed.","tokens_in":11576,"tokens_out":613,"duration_ms":7180,"concrete_test":"On the same 12-bit QAT+Monotone checkpoint, record the elementwise max |activation| (or the fraction of values with |q| > 2^{B-1}-1) immediately after each repair projection in Alg. 1 line 7, before the next layer. If any coordinate still exceeds the representable range while V(h) ≤ V_max, the energy ball does not enforce the overflow claim; the headline numbers would then rest on model-specific luck rather than the stated mechanism.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that monotone Lyapunov projection after wrapping write-back (Alg. 1, Eqs. 15–24) suppresses activation overflow to <0.012% and thereby restores QAT. The only safety mechanism is a single scalar energy ball V(h) = (1/TD)||h||_F^{2} ≤ min{V(h_ℓ), V_max}, with V_max fixed once from the 0.99-quantile of initial token energies times γ=1.20 (Eq. 14). Radial scaling of the whole tensor does not bound any individual coordinate: a sparse large entry can still exceed q_max even when the average energy is safe. Because overflow is defined elementwise (Sec. III.G) and the fixed-point format uses only two integer bits for activations, the energy constraint is neither necessary nor sufficient for the reported near-zero overflow. The paper never proves or measures that every post-projection activation lies inside [q_min,q_max]; the empirical rates in Table I could be an artefact of the tiny 4-layer/64-dim MNIST model rather than a consequence of the Lyapunov construction.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper argues that two’s-complement overflow wrapping is a first-order failure mode for low-precision neural networks that is missed by clipping-based quantisation models. It formulates layerwise residual/transformer dynamics as a discrete-time system, monitors the normalised Frobenius energy V(h_ℓ)=∥h_ℓ∥_F²/(TD), and applies a monotone radial projection after fixed-point write-back so that V(h_{ℓ+1})≤min{V(h_ℓ),V_max}. On a 4-layer 64-dim patch transformer for MNIST, Monte Carlo experiments (3 seeds) show unconstrained wrapped QAT collapsing to near-chance accuracy for 6–16 bits with activation overflow >11%, while the projected variant suppresses measured overflow to ≲0.012% and recovers 86.55%±0.65% at 12 bits. PTQ is also improved by the same projection.","tokens_in":11898,"tokens_out":1417,"duration_ms":22738,"significance":"If the stabilisation effect generalises beyond the reported setting, the work would be a useful hardware-aware training ingredient: it treats wrapping overflow as a dynamical instability rather than only a rounding/clipping error, and it supplies a simple, implementable projection (Algorithm 1) with independent observables (accuracy, overflow rate, energy, projection rate). Strengths include explicit modelling of wraparound (not saturation), Monte Carlo means±std, and an honest diagnosis that 4-bit failure is representation collapse rather than overflow. Significance is currently limited by the MNIST-only, tiny-transformer evidence base and by the gap between the average-energy constraint and elementwise fixed-point range safety.","major_comments":[{"comment":"§III.C–D, Eqs. (12)–(17), Algorithm 1, and §III.G: the safety claim is load-bearing but not guaranteed by the stated construction. Overflow is defined elementwise (q outside [q_min,q_max] before wrapping), while the only constraint is a scalar energy ball V(h)=∥h∥_F²/(TD) with radial rescaling of the whole tensor. A sparse large coordinate can still exceed the representable range when the average energy is safe, especially with only two integer bits for activations. The paper never proves or reports a post-projection max-|activation| check; Table I’s near-zero overflow rates are therefore empirical on this model, not a consequence of the Lyapunov construction. Either add an elementwise (or per-token) bound, prove that the energy ball implies range safety under the chosen format, or reframe the claim as empirical stabilisation rather than overflow avoidance.","section":null},{"comment":"§IV and Table I: the central claim is evaluated only on MNIST with a 4-layer, 64-dim, 16-token patch transformer, 10k train images, and three seeds. The title and abstract present a general hardware-safe training method for fixed-point inference/training. That scope is not supported by the evidence. At minimum the revision needs (i) a second dataset/architecture (e.g., CIFAR-10 CNN or a deeper transformer) or (ii) a clear limitation statement that results are confined to this compact MNIST model, with the broader claims dialled back.","section":null},{"comment":"§II.B, §IV: missing load-bearing baselines. The paper cites A2Q (accumulator-aware ℓ1 weight constraints) and discusses saturation vs wrapping, but Table I compares only wrap vs wrap+monotone (plus FP32). Without (a) saturating/clipping QAT under the same bit formats, (b) simple per-tensor activation clipping or max-norm projection without the monotone Lyapunov schedule, and (c) an A2Q-style or accumulator-width baseline, it is unclear whether the gain comes from Lyapunov monotonicity, from any energy bound, or from merely preventing large activations. These comparisons are necessary to support the claim that Lyapunov-guided projection is the operative stabilisation mechanism.","section":null},{"comment":"§III.E, Eqs. (20)–(24) and Algorithm 1: projection is applied after wrapping write-back. If wraparound has already flipped signs, radial scaling bounds the energy of a corrupted state; it does not restore the pre-overflow semantics. The paper should analyse (or measure) how often projection is applied to already-wrapped tensors versus merely large but unwrapped ones, and discuss whether post-wrap projection can mask rather than prevent dynamical corruption—especially given the high projection rates (~100%) in Table I for the successful runs.","section":null}],"minor_comments":[{"comment":"Abstract vs Table I: abstract says “86.55% accuracy at 12 bits” without the ±0.65% reported in the table and conclusion; keep the uncertainty consistent.","section":null},{"comment":"§I and §III.B: fixed-point format (I,F allocation) is stated late and only as “unless otherwise stated, weights I=1, activations I=2.” Put the exact (B,I,F) table in the experimental section so overflow rates are interpretable.","section":null},{"comment":"Duplicate bibliography entries for Baranowski et al. ([4] and [5]); clean the reference list.","section":null},{"comment":"Figs. 1–5 use log axes and overlapping markers; add explicit markers/legend for 10- and 16-bit points and state whether error bars are std or min–max over the three seeds.","section":null},{"comment":"Eq. (14): p=0.99 and γ=1.20 are fixed without sensitivity analysis; a short ablation (or appendix) on V_max would strengthen the free-parameter discussion.","section":null},{"comment":"Notation: both eh_ℓ / ez_ℓ and bh_ℓ appear for quantised states; a single consistent diacritic scheme would help.","section":null}],"recommendation":"major_revision","confidential_remarks":"Empirically the MNIST Monte Carlo contrast is real and cleanly reported; the skeptic’s energy-vs-elementwise point is the main technical gap and should be fixed or honestly reframed before acceptance. Scope (MNIST-only tiny transformer) is the other reason this is major_revision rather than minor. Fit is borderline for a top general ML venue without broader experiments; more natural as a hardware-aware / embedded ML journal contribution if the baselines and elementwise-safety discussion are strengthened."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is simple: under two’s-complement wrapping, unconstrained QAT on their small patch transformer collapses to chance across 6–16 bits with double-digit activation overflow, while a radial projection that keeps layer energy non-increasing and below a fixed V_max drives overflow near zero and restores ~86% at 12 bits. That contrast is large, reproducible from the written algorithm, and not something the usual clipping/QAT literature shows.\n\nWhat is actually new is the combination, not the ingredients. Lyapunov views of residual nets and accumulator-aware quantisation already exist. Here they put a monotone energy projection after hardware-style wrap write-back (and a repair step after fixed-point store) inside both PTQ and QAT. The algorithm is explicit, the failure mode is real, and Table I with three-seed means/stds is honest about the collapse-versus-recovery story.\n\nSoft spots, in proportion. Evaluation is MNIST only, 4 layers, 64-d, 16 tokens—so we do not know if this survives ImageNet-scale transformers or real FPGA/ASIC paths. Missing baselines matter: saturation arithmetic, A2Q-style weight constraints, and a soft energy penalty alone would tell us how much of the gain is the hard projection versus any energy control. V_max is a one-shot 0.99-quantile of initial token energy times γ=1.20; free and un-ablated.\n\nThe stress-test point is fair but not fatal to the paper as written. Radial scaling of Frobenius energy does not bound individual coordinates, so the construction does not prove elementwise overflow cannot occur—especially with only two integer bits on activations. The paper’s claim is empirical (measured overflow rates), not a certificate. On this toy model the rates go to ~0; that is still useful engineering evidence, not a general safety proof.\n\nMath is elementary and consistent with the algorithm; citations are appropriate (Jacob, A2Q, Nar & Sastry, quant surveys). No code or hardware loop.\n\nWho it is for: people who ship fixed-point edge models and have seen silent wrap failures. Not a theory paper. I would send it to peer review; a serious referee should demand broader tasks, saturation/A2Q baselines, and either elementwise bounds or a clear statement that overflow control is empirical. Worth engaging if you work on quantised hardware; otherwise skim the table and move on.","headline":"Clear MNIST demo that wrapping overflow can kill QAT, and a simple post-wrap energy projection recovers it—but the safety story is empirical, not guaranteed, and the evaluation is tiny.","tokens_in":12476,"tokens_out":611,"would_cite":false,"duration_ms":14541,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A layerwise energy projection stops fixed-point wraparound from destroying low-precision neural nets.","keywords":["low-precision neural networks","quantisation-aware training","fixed-point arithmetic","overflow wrapping","Lyapunov stability","neural network safety","hardware-aware machine learning","transformer networks"],"falsifier":"Train the same patch transformer (or a deeper one) under identical wrapping arithmetic but with a deliberately larger or layer-dependent V_max; if accuracy collapses again while measured overflow remains near zero, or if overflow reappears despite the projection, the claimed stabilisation fails.","tokens_in":12454,"feed_emoji":"⚖️","tokens_out":668,"duration_ms":6491,"temperature":0.7,"pith_summary":"Low-precision neural networks are attractive for cheap hardware, but real fixed-point chips often wrap overflowed values around the integer range, flipping both magnitude and sign. That wraparound can corrupt hidden activations and make training collapse even at bit-widths that look large enough. This paper treats the network as a depth-indexed dynamical system, monitors the average squared magnitude of each layer’s hidden state with a Lyapunov energy, and after every fixed-point write-back projects the state so that energy never grows and never exceeds a safe threshold. On a compact MNIST transformer the unconstrained wrapping training path falls to chance-level accuracy with more than 11 percent activation overflow; the projected path drives overflow below 0.012 percent and recovers roughly 86.5 percent accuracy at 12 bits. The claim is that keeping hidden-state energy inside a safe set is a practical hardware-aware control that restores reliable fixed-point inference and training.","feed_headline":"Energy projection stops wraparound from wrecking low-bit nets","feed_subtitle":"On MNIST, a Lyapunov bound cuts overflow below 0.012% and lifts 12-bit accuracy to 86.5%","key_machinery":"Monotone Lyapunov projection: after each wrapping quantisation write-back, radially scale the hidden state so its normalised energy satisfies V(h_{ℓ+1}) ≤ min{V(h_ℓ), V_max}, thereby keeping every intermediate activation inside a safe fixed-point set.","core_discovery":"Under two’s-complement wrapping arithmetic, unconstrained quantisation-aware training collapses to near-chance accuracy across 6–16 bits because activation overflow rates exceed 11 percent; applying a monotone Lyapunov projection that forces the layerwise hidden-state energy to be non-increasing and bounded by a fixed safe threshold suppresses overflow to below 0.012 percent and restores stable learning, reaching 86.55 percent accuracy at 12 bits on MNIST.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Lyapunov projection tames wraparound, restores low-bit accuracy","Hidden-state energy bound stops fixed-point overflow collapse","Monotone projection suppresses wrap overflow near zero on MNIST","Lyapunov guidance stabilizes quantisation under two's-complement wrap","Layerwise Lyapunov keeps low-precision nets from wraparound failure"],"cache_read_input_tokens":128,"weakest_assumption_plain":"A single global energy ceiling taken once from the 99th percentile of the initial token energies, plus simple radial scaling after every wrap, is enough to keep all intermediate fixed-point activations safe without destroying the network’s ability to learn.","fun_headline_variants_meta":{"raw":{"variants":["Lyapunov projection tames wraparound, restores low-bit accuracy","Hidden-state energy bound stops fixed-point overflow collapse","Monotone projection suppresses wrap overflow near zero on MNIST","Lyapunov guidance stabilizes quantisation under two's-complement wrap","Layerwise Lyapunov keeps low-precision nets from wraparound failure"]},"model":"grok-4.5","effort":"low","cost_usd":0.004156,"raw_usage":{"total_tokens":1298,"prompt_tokens":812,"num_sources_used":0,"completion_tokens":69,"cost_in_usd_ticks":41560000,"prompt_tokens_details":{"text_tokens":812,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":417,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":812,"tokens_out":69,"duration_ms":4138,"temperature":1.0,"reasoning_tokens":417,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T17:51:21.945794+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Train the same patch transformer (or a deeper one) under identical wrapping arithmetic but with a deliberately larger or layer-dependent V_max; if accuracy collapses again while measured overflow remains near zero, or if overflow reappears despite the projection, the claimed stabilisation fails.","supporting_citations":[],"review_version":1}