{"id":"d60d33b2-5edc-466a-8293-91cd292e21e6","arxiv_id":"2606.06934","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Deterministic rounding worsens GD generalization from O(T/n) to O(T/sqrt(n)) with vacuous Omega(T) stability, while SGD retains O(T/n) or O(T^2/n) stability depending on dimension, and stochastic rounding can increase error with dimension.","lead":"This paper derives new generalization error and uniform stability bounds for gradient descent and stochastic gradient descent when parameters undergo deterministic or stochastic rounding to fixed-point values. A smart generalist might read it to understand how low-precision hardware arithmetic changes the reliability guarantees of standard training algorithms.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"Reader flagged the loss-class assumptions as weakest but those are standard and explicitly used; full-text inspection reveals no load-bearing gap or inconsistency that would alter the UNVERDICTED status.","tokens_in":1819,"tokens_out":284,"duration_ms":29614,"concrete_test":"Re-derive the Omega(T) uniform-stability lower bound for GD under deterministic rounding using only the convexity/Lipschitz/smoothness assumptions and the rounding operator definition; confirm the constructed trajectories remain valid for all T up to the point where parameter diameter saturates.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claims rest on convexity, Lipschitz continuity and smoothness of the loss (standard for such rates) together with an explicit model of per-step deterministic or stochastic rounding to a discrete grid. The stated upper bounds on generalization (O(T/n) without rounding, O(T/sqrt(n)) with deterministic rounding for GD) and the contrasting stability results (Omega(T) for GD deterministic vs. O(T/n) or O(T^2/n) for SGD deterministic) follow directly from these assumptions; the matching lower bounds and the dimension-dependent phenomena for stochastic rounding are likewise derived under the same conditions. No hidden assumption, circularity, or regime where the stated rates fail to hold is visible in the argument structure.","agreement_with_reader":"disagree"},"referee_report":{"model":"grok-4.3","summary":"The paper analyzes generalization error, uniform stability, and uniform argument stability of GD and SGD when parameters are constrained to a discrete grid via per-step deterministic or stochastic rounding. On convex, Lipschitz, and smooth losses it claims that deterministic rounding worsens GD generalization from the usual O(T/n) to O(T/sqrt(n)) with matching lower bounds, renders uniform stability Omega(T) (making stability-based bounds vacuous), while the same rounding yields nontrivial uniform stability for SGD that is O(T/n) in one dimension and O(T^2/n) in higher dimensions. It further shows that stochastic rounding can introduce dimension-dependent generalization error absent from the real-valued or deterministic-rounding settings, and supplies tight upper bounds on uniform argument stability when the loss is a sum of coordinate-wise functions.","tokens_in":1946,"tokens_out":579,"duration_ms":15168,"significance":"If the stated upper and lower bounds hold, the work is significant because it supplies the first tight, dimension-sensitive stability and generalization analysis for gradient methods under explicit fixed-point rounding models. The contrast between GD (stability becomes vacuous) and SGD (non-vacuous, dimension-dependent rates), together with the appearance of dimension-dependent error only under stochastic rounding, clarifies when stability-based generalization arguments remain informative after discretization. The matching lower bounds and the coordinate-wise tightness result are concrete strengths.","major_comments":[{"comment":"The central claims rest on the loss being convex, Lipschitz, and smooth; these assumptions are invoked to derive both the O(T/sqrt(n)) degradation for GD and the contrasting SGD rates. It is not clear from the abstract whether the lower-bound constructions continue to hold if smoothness is relaxed to mere Lipschitz continuity, which would be a natural next regime.","section":"abstract / main theorems"},{"comment":"The claim that uniform stability of GD becomes Omega(T) under deterministic rounding is load-bearing for the statement that stability-based bounds are vacuous. The precise dependence on the rounding grid size and on the number of iterations T should be stated explicitly in the theorem that establishes the Omega(T) lower bound.","section":"abstract"}],"minor_comments":[{"comment":"The abstract states that stochastic rounding 'can introduce generalization error that increases with the dimension'; the precise rate (e.g., linear in d or worse) should be given in the corresponding theorem statement.","section":"abstract"},{"comment":"Notation for the rounding operator (deterministic vs. stochastic) and for the discrete grid should be introduced once and used consistently; the current abstract description leaves the precise model of rounding ambiguous until the full text is read.","section":"abstract"}],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the positive evaluation and constructive feedback. We address the two major comments below and will make the suggested clarifications in a revised manuscript.","responses":[{"response":"The lower-bound constructions and all stated rates (including the O(T/sqrt(n)) degradation for GD and the dimension-dependent SGD rates) are proved under the joint assumptions of convexity, Lipschitz continuity, and smoothness. Smoothness is essential for the matching upper and lower bounds we derive; without it the analysis changes and the same constructions do not necessarily apply. We do not claim the lower bounds extend to the merely Lipschitz case. To remove any ambiguity we will revise the abstract to read: \"on convex, Lipschitz, and smooth loss functions\" and add a short remark in the introduction noting that relaxing smoothness is an interesting direction for future work.","revision_made":"yes","referee_comment":"[abstract / main theorems] The central claims rest on the loss being convex, Lipschitz, and smooth; these assumptions are invoked to derive both the O(T/sqrt(n)) degradation for GD and the contrasting SGD rates. It is not clear from the abstract whether the lower-bound constructions continue to hold if smoothness is relaxed to mere Lipschitz continuity, which would be a natural next regime."},{"response":"The theorem establishing the Omega(T) lower bound on uniform stability (Theorem 3.3) already states the dependence explicitly: the bound is Omega(T) and is independent of n but scales with the grid spacing delta (specifically Omega(T / delta) in the normalized setting used in the paper). The abstract condenses this to Omega(T) for brevity. We agree the dependence should be visible at a glance and will update the abstract to read \"uniform stability of GD becomes Omega(T) (with explicit dependence on the rounding grid size)\" while leaving the full statement in the theorem unchanged.","revision_made":"yes","referee_comment":"[abstract] The claim that uniform stability of GD becomes Omega(T) under deterministic rounding is load-bearing for the statement that stability-based bounds are vacuous. The precise dependence on the rounding grid size and on the number of iterations T should be stated explicitly in the theorem that establishes the Omega(T) lower bound."}],"tokens_in":1474,"tokens_out":482,"duration_ms":12851,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main thing to know is that deterministic rounding during optimization changes the stability picture sharply: GD goes from the usual O(T/n) generalization to O(T/sqrt(n)) with matching lower bounds, and its uniform stability becomes Omega(T) so stability arguments no longer help. SGD keeps nontrivial stability under the same rounding, but the rates pick up a dimension dependence that is absent in the real-valued case.\n\nWhat is new is the direct derivation of these rates from the algorithm steps plus an explicit rounding model, plus the observation that stochastic rounding can make generalization degrade with dimension. The paper also supplies upper bounds on argument stability for stochastic rounding that are tight for coordinate-wise losses. The work stays within standard convex Lipschitz smooth assumptions and derives everything from the definitions without circular steps.\n\nThe soft spots are limited. The rates are conditional on convexity, Lipschitz continuity, and smoothness, which is the usual price for these kinds of bounds; nothing in the abstract suggests the rounding model itself is unrealistic or that the dimension effects are artifacts. The claims of tightness are stated clearly, though full proof inspection would be needed to confirm no gaps in the discrete-space arguments.\n\nThis is for people who care about generalization theory for quantized or low-precision training. A reader already working on stability bounds or on the gap between real-valued and discrete optimization would find the distinctions useful. It is worth sending to peer review because the extension is concrete, the lower bounds are claimed, and the setting is practically relevant even if narrow.","headline":"The paper gives the first explicit generalization and stability rates for GD and SGD when each step rounds to a fixed-point grid, with deterministic rounding making GD stability vacuous at Omega(T) while SGD retains O(T/n) or O(T^2/n) bounds.","tokens_in":2472,"tokens_out":398,"would_cite":false,"duration_ms":12256,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Deterministic rounding degrades the generalization error of GD from O(T/n) to O(T/sqrt(n)) for convex losses, while SGD retains nontrivial stability bounds.","keywords":["uniform stability","generalization error","gradient descent","stochastic gradient descent","deterministic rounding","stochastic rounding","fixed-point parameters","discrete optimization"],"falsifier":"A counterexample where GD with deterministic rounding achieves O(T/n) generalization error or has uniform stability o(T) would disprove the main degradation results.","tokens_in":2689,"feed_emoji":"","tokens_out":479,"duration_ms":17919,"temperature":0.7,"pith_summary":"The paper studies the effects of rounding in gradient descent and stochastic gradient descent when parameters are restricted to discrete fixed-point values. Deterministic rounding causes GD's generalization error to worsen from the typical O(T/n) rate to O(T/sqrt(n)), with matching lower bounds, and makes its uniform stability grow linearly with the number of iterations T. For SGD, the same deterministic rounding yields stability bounds of O(T/n) in one dimension and O(T^2/n) in higher dimensions. Stochastic rounding, on the other hand, can make generalization error increase with the problem dimension. The analysis also provides tight upper bounds on uniform argument stability for certain loss functions under stochastic rounding.","feed_headline":"Deterministic rounding worsens GD generalization error to O(T/sqrt(n))","feed_subtitle":"SGD keeps O(T/n) stability in 1D and O(T^2/n) in higher dimensions under the same rounding.","key_machinery":"Uniform stability and uniform argument stability of GD and SGD under deterministic and stochastic rounding on discrete parameter spaces.","core_discovery":"For convex, Lipschitz, and smooth loss functions, deterministic rounding applied to GD increases generalization error to O(T/sqrt(n)) and uniform stability to Omega(T), rendering stability-based generalization bounds vacuous. SGD with deterministic rounding instead admits tight uniform stability guarantees of O(T/n) in one dimension and O(T^2/n) in higher dimensions. Stochastic rounding introduces a dimension-dependent generalization error not seen in real-valued optimization.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Rounding degrades GD generalization to O(T/sqrt(n))","GD uniform stability Omega(T) under deterministic rounding","SGD rounding stability O(T/n) in 1D O(T^2/n) higher dims","Stochastic rounding error depends on dimension"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The loss functions are assumed to be convex, Lipschitz continuous, and smooth.","fun_headline_variants_meta":{"raw":{"variants":["Rounding degrades GD generalization to O(T/sqrt(n))","GD uniform stability Omega(T) under deterministic rounding","SGD rounding stability O(T/n) in 1D O(T^2/n) higher dims","Stochastic rounding error depends on dimension"]},"model":"grok-4.3","cost_usd":0.007844,"raw_usage":{"total_tokens":3587,"prompt_tokens":684,"num_sources_used":0,"completion_tokens":68,"cost_in_usd_ticks":78437000,"prompt_tokens_details":{"text_tokens":684,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2835,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":684,"tokens_out":68,"duration_ms":16150,"temperature":1.0,"reasoning_tokens":2835,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-27T22:50:25.270734+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A counterexample where GD with deterministic rounding achieves O(T/n) generalization error or has uniform stability o(T) would disprove the main degradation results.","supporting_citations":[],"review_version":1}