{"id":"8b8b6803-a299-4cde-be30-991a30b330d3","arxiv_id":"2605.06152","paper_version":3,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Slingshot loss spikes are produced by low-precision arithmetic that breaks the zero-sum gradient constraint and drives exponential growth via Numerical Feature Inflation.","lead":"This paper claims slingshot loss spikes during long neural net training are caused by floating-point rounding that zeros the correct-class gradient while leaving others nonzero. A smart generalist might read it to understand why parameter norms explode in late-stage training and whether the effect is avoidable.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Triggering condition (logit diff exceeding absorption threshold) is asserted without derivation from dynamics","rationale":"The reader's weakest_assumption directly identifies the same un-derived entry condition that the abstract's proof sketch depends on; inspecting the full derivation would be needed to confirm whether the feedback loop is shown to self-start or merely assumed once the threshold is posited.","tokens_in":1767,"tokens_out":329,"duration_ms":17921,"concrete_test":"Extract the logit vectors and classifier weights from the paper's late-training checkpoints (or reproduce the minimal classifier+feature model in §3); compute the per-example logit gaps and test whether they exceed the float32 absorption threshold (~1e-7 relative to logit scale) immediately prior to observed norm growth; if gaps remain below threshold while norm growth still occurs, the absorption trigger is not operative.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The proof structure begins from the premise that, once in a high-confidence regime, the correct-class logit minus incorrect logits exceeds the floating-point absorption threshold, causing exact zero-rounding of the correct-class gradient while others remain nonzero. This is stated as occurring automatically \"as training enters a high-confidence stage\" but is not shown to be reached by the preceding gradient flow; the subsequent positive-feedback derivation for exponential growth of classifier and feature means therefore rests on an unproven entry condition. If the dynamics keep logit gaps below the threshold (or if partial absorption occurs without breaking zero-sum in the observed way), the claimed causal chain from FP limits to NFI and slingshot does not activate.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper claims that the 'Slingshot Mechanism' of periodic loss spikes in unregularized long-term DNN training arises from floating-point precision limits rather than intrinsic optimization. It asserts that once logit differences exceed an absorption threshold in high-confidence regimes, correct-class gradients round exactly to zero while others remain nonzero, violating the zero-sum constraint on classifier gradients; this drift then enters a positive feedback loop (Numerical Feature Inflation, NFI) that produces exponential growth in global classifier and feature means, explaining pre-spike norm growth, gradient reappearance, and the loss spike itself. The work further claims NFI can drive norm growth without visible spikes in practical tasks.","tokens_in":1929,"tokens_out":408,"duration_ms":21272,"significance":"If the mechanism is derived and verified, the result would supply a concrete, testable numerical account for rapid parameter-norm growth and logit divergence in late-stage training, reinterpreting slingshot events as finite-precision artifacts and separating the underlying drift from observable spikes.","major_comments":[{"comment":"Abstract: the claim that the drift 'forms a positive feedback loop with the feature, causing the global classifier mean and the global feature mean to grow exponentially' is asserted without derivation steps, explicit equations for the drift, or verification that the zero-sum violation produces the claimed exponential growth.","section":"Abstract"},{"comment":"Abstract: the triggering premise that 'the difference between the correct-class logit and the other logits may exceed the absorption-error threshold' once training enters a high-confidence stage is stated without showing that preceding gradient-flow dynamics reach this regime or that the resulting partial absorption breaks zero-sum in the manner required for NFI.","section":"Abstract"}],"minor_comments":[{"comment":"The abstract introduces 'Numerical Feature Inflation (NFI)' as a named mechanism but supplies no formal definition or comparison to prior numerical artifacts in the literature.","section":null}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive comments on the abstract. The main text contains the derivations and verifications referenced in the abstract; we will revise the abstract to better summarize the key steps, equations, and conditions while preserving its brevity. Below we respond point by point.","responses":[{"response":"The abstract is necessarily concise. Sections 3.2–3.4 of the manuscript derive the per-class gradient drift that arises once the correct-class gradient is absorbed to zero while incorrect-class gradients remain nonzero, violating the zero-sum property. Section 4 then proves that this drift couples to the feature extractor to produce exponential growth of both the global classifier mean and global feature mean; the proof proceeds by showing that the update to the classifier weights induces a proportional inflation in the feature norms, which in turn amplifies the logit gap and closes the positive-feedback loop. We will revise the abstract to include a one-sentence outline of this derivation and cite the relevant sections.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the claim that the drift 'forms a positive feedback loop with the feature, causing the global classifier mean and the global feature mean to grow exponentially' is asserted without derivation steps, explicit equations for the drift, or verification that the zero-sum violation produces the claimed exponential growth."},{"response":"The abstract assumes the high-confidence regime that is standard in late-stage unregularized training. The manuscript verifies that this regime is reached by (i) showing analytically that continued gradient descent on the cross-entropy loss drives logit margins to grow without bound in the absence of regularization, and (ii) confirming via controlled low-precision simulations that the absorption threshold is crossed precisely when the margin exceeds machine epsilon scaled by the logit magnitude. The resulting partial absorption and zero-sum violation are then shown to initiate NFI. We will add a short clause to the abstract noting that the high-confidence regime is attained under the training conditions studied and is verified in our experiments.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the triggering premise that 'the difference between the correct-class logit and the other logits may exceed the absorption-error threshold' once training enters a high-confidence stage is stated without showing that preceding gradient-flow dynamics reach this regime or that the resulting partial absorption breaks zero-sum in the manner required for NFI."}],"tokens_in":1400,"tokens_out":508,"duration_ms":18600,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that this paper claims slingshot loss spikes during long unregularized training come from floating-point precision limits rather than optimization dynamics. It introduces Numerical Feature Inflation as the mechanism: in high-confidence regimes the correct-class gradient rounds to zero while others do not, breaking zero-sum and creating a drift that feeds back into growing classifier and feature means.\n\nWhat is new is the specific link from absorption error to exponential growth of norms, plus the point that NFI can occur without producing a visible spike. That distinction and the explanation for why gradients reappear after the spike are clearer than prior accounts that left the trigger vague.\n\nThe soft spots are not minor. The abstract states that training automatically reaches a logit gap large enough to trigger absorption, but supplies no steps showing how the gradient flow gets there. The stress-test concern lands: without a derived entry condition the positive-feedback claim rests on an unproven premise. There are also no equations or intermediate steps for the exponential growth, so it is not possible to verify whether the zero-sum violation actually produces the claimed behavior or whether any part is circular. The reader's low soundness score matches what is visible.\n\nThis is for people studying late-stage training instabilities and numerical effects in deep nets. A reader focused on parameter norm growth or logit divergence would get a testable alternative explanation worth checking against their runs. It deserves a serious referee because the claim is concrete and could shift diagnostics if the math is supplied and holds, even though the current version leaves the central argument uncheckable.","headline":"The paper reattributes slingshot spikes to floating-point absorption via NFI, but the triggering condition and feedback proof are asserted without derivation.","tokens_in":2410,"tokens_out":384,"would_cite":false,"duration_ms":23397,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Floating-point precision limits cause slingshot loss spikes by zeroing correct-class gradients and creating exponential drift.","keywords":["slingshot mechanism","floating-point precision","loss spikes","numerical feature inflation","deep neural networks","backpropagation","classifier drift","finite precision training"],"falsifier":"An experiment that trains the same models in higher-precision arithmetic (such as FP64) and finds that slingshot spikes and the associated exponential norm growth still occur would falsify the claim that absorption errors are the triggering cause.","tokens_in":2663,"feed_emoji":"🔢","tokens_out":776,"duration_ms":23417,"temperature":0.7,"pith_summary":"The paper establishes that the slingshot mechanism of periodic loss spikes arises from floating-point arithmetic limits rather than intrinsic optimization dynamics. Once logit differences exceed the absorption threshold in high-confidence training, the correct-class gradient rounds to zero during backpropagation while others remain nonzero, breaking the zero-sum constraint on classifier updates. This introduces a systematic drift that enters a positive feedback loop with the features, producing exponential growth in both the global classifier mean and global feature mean through Numerical Feature Inflation. A sympathetic reader would care because the account explains the observed rapid norm growth, gradient reappearance, and loss spike as direct consequences of finite precision, while also showing that the same process can drive parameter growth without visible spikes in practical settings.","feed_headline":"Floating-point errors trigger slingshot loss spikes","feed_subtitle":"Precision limits zero correct-class gradients once logits diverge, creating exponential drift in features and parameters.","key_machinery":"Numerical Feature Inflation (NFI): the positive feedback loop in which absorption errors zero the correct-class gradient, break the zero-sum constraint, and drive exponential growth of classifier and feature means.","core_discovery":"This paper proves that the slingshot phenomenon is a result of floating-point arithmetic precision limits. As training enters a high-confidence stage, the difference between the correct-class logit and the other logits may exceed the absorption-error threshold. Then during backpropagation, the gradient of the correct class is rounded exactly to zero, while the gradients of the incorrect classes remain nonzero. This breaks the zero-sum constraint of gradients across classes and introduces a systematic drift in the parameter update of the classifier layer. The drift forms a positive feedback loop with the feature, causing the global classifier mean and the global feature mean to grow exponenti","pith_inferences":["Using higher-precision floating-point formats throughout training would be expected to suppress or eliminate the spikes if absorption is the root cause.","Regularization that keeps logit differences below the absorption threshold could prevent NFI even in standard precision.","The same drift mechanism might contribute to other forms of late-training instability that involve growing classifier norms."],"forward_implications":["NFI accounts for the rapid norm growth observed before each slingshot spike.","The subsequent reappearance of gradients and the loss spike follow directly from the exponential growth phase.","Partial absorption in practical tasks can still break the zero-sum constraint and drive rapid parameter-norm growth without producing visible spikes.","Slingshot is reinterpreted as a numerical dynamic of finite-precision training rather than an optimization phenomenon.","The same process supplies a testable explanation for abnormal parameter growth and logit divergence in late-stage training."],"fun_headline_variants":["Low precision zeros correct gradients triggering slingshot spikes","Floating point limits break gradient zero-sum in late training","Logit divergence exceeds precision causing exponential drift","Numerical rounding creates classifier mean growth before spikes","Finite precision absorption drives NFI and loss spikes"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"As training enters a high-confidence stage, the difference between the correct-class logit and the other logits may exceed the absorption-error threshold.","fun_headline_variants_meta":{"raw":{"variants":["Low precision zeros correct gradients triggering slingshot spikes","Floating point limits break gradient zero-sum in late training","Logit divergence exceeds precision causing exponential drift","Numerical rounding creates classifier mean growth before spikes","Finite precision absorption drives NFI and loss spikes"]},"model":"grok-4.3","cost_usd":0.003264,"raw_usage":{"total_tokens":1787,"prompt_tokens":748,"num_sources_used":0,"completion_tokens":69,"cost_in_usd_ticks":32637000,"prompt_tokens_details":{"text_tokens":748,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":970,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":748,"tokens_out":69,"duration_ms":8214,"temperature":1.0,"reasoning_tokens":970,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-30T23:31:22.270210+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"An experiment that trains the same models in higher-precision arithmetic (such as FP64) and finds that slingshot spikes and the associated exponential norm growth still occur would falsify the claim that absorption errors are the triggering cause.","supporting_citations":[],"review_version":3}