{"id":"f5ae4181-c42f-409e-af19-d8e152e3d35f","arxiv_id":"2606.22249","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Weight-quantized LLMs retain universal approximation up to 1.58 bits with expressive collapse below it and polynomial degradation in capacity as bit count falls.","lead":"The paper claims that weight quantization in large language models reaches a hard limit at 1.58 bits, where universal approximation holds but expressive power collapses below that threshold, with polynomial degradation as bits decrease. A smart generalist might read it to understand theoretical constraints on how far model compression can go before capability is lost.","discovery_kind":"first_principles","skeptic_critique":{"model":"grok-4.3","headline":"Expressive power definition treats quantization cardinality as directly setting a sharp log2(3) threshold","rationale":"The reader's weakest_assumption correctly isolates the load-bearing modeling choice. Because the full text was not supplied in the query, no additional internal inconsistency can be checked, so the verdict remains UNVERDICTED.","tokens_in":1772,"tokens_out":314,"duration_ms":15725,"concrete_test":"Extract the formal definition of 'expressive power' and the statement of the universal-approximation theorem (likely in §3 or §4); re-derive the collapse statement without assuming |Q| directly controls the span; check whether the polynomial rate still holds under an alternative measure such as the number of distinct linear regions.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The claim that 1.58 bits is the limiting precision rests on an implicit definition of expressive power and universal approximation in which a weight-quantized network can approximate target functions if and only if its weight set has cardinality at least 3. This produces both the collapse below log2(3) and the polynomial degradation rate. The modeling choice equates bit precision with |Q| = 2^b and assumes the network's functional span is governed exactly by that cardinality (likely via a result on finite-weight ReLU or similar networks). Changing the definition—e.g., to allow non-uniform quantization, learned codebooks, or a different measure such as Rademacher complexity—would move or eliminate the threshold.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript claims that 1.58 bits is the limiting precision for weight quantization in large language models. It establishes universal approximation and expressive collapse properties with respect to the number of quantization bits, and shows that expressive capacity degrades polynomially as the number of bits decreases. These results are positioned as providing a theoretical foundation for quantization in scaling laws.","tokens_in":1880,"tokens_out":390,"duration_ms":17962,"significance":"If the central claims are rigorously derived, the identification of a sharp threshold at log2(3) bits and the polynomial degradation rate would supply a concrete theoretical limit that could inform practical quantization choices and scaling analyses for model compression.","major_comments":[{"comment":"Abstract: the claim of a sharp 1.58-bit threshold for universal approximation and expressive collapse is load-bearing for the entire contribution, yet no definition of expressive power, no statement of the network class (e.g., ReLU networks with finite weights), and no proof sketch or cardinality argument are supplied; without these the threshold cannot be verified and may be an artifact of an implicit modeling choice that equates |Q| = 2^b directly with functional span.","section":"Abstract"},{"comment":"Abstract: the polynomial degradation statement is presented as a derived result, but no functional form, no dependence on network depth or width, and no supporting derivation or theorem statement appear; this prevents assessment of whether the rate is independent of other modeling assumptions.","section":"Abstract"}],"minor_comments":[{"comment":"Abstract: the term 'large language models' is used throughout, but the stated properties appear to apply to general feed-forward networks; clarify whether the results rely on transformer-specific structure or hold more broadly.","section":"Abstract"}],"recommendation":"uncertain","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the comments on the abstract. We address each point below, noting that the abstract summarizes results whose details appear in the full manuscript.","responses":[{"response":"The abstract is a concise summary. The manuscript defines expressive power via the universal approximation property for continuous functions on compact sets and specifies the network class as ReLU networks whose weights are drawn from a finite quantization set Q with cardinality 2^b. The threshold at log2(3) follows from a cardinality argument establishing that the representable function class becomes strictly smaller than the target class for b < log2(3), producing collapse; this is shown in the main theorems and is independent of equating |Q| directly with functional span. We will expand the abstract with a one-sentence definition of expressive power and the network class.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the claim of a sharp 1.58-bit threshold for universal approximation and expressive collapse is load-bearing for the entire contribution, yet no definition of expressive power, no statement of the network class (e.g., ReLU networks with finite weights), and no proof sketch or cardinality argument are supplied; without these the threshold cannot be verified and may be an artifact of an implicit modeling choice that equates |Q| = 2^b directly with functional span."},{"response":"The polynomial degradation result is derived in the manuscript, with the functional form depending polynomially on 2^b and with explicit dependence on depth and width appearing in the exponent. The supporting theorem and proof are given in the body. We will add a parenthetical reference to the relevant theorem in the abstract to improve traceability.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the polynomial degradation statement is presented as a derived result, but no functional form, no dependence on network depth or width, and no supporting derivation or theorem statement appear; this prevents assessment of whether the rate is independent of other modeling assumptions."}],"tokens_in":1274,"tokens_out":437,"duration_ms":25944,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The one thing to know is that the abstract states 1.58 bits (ternary quantization) as the hard lower limit for weight-quantized LLMs to retain universal approximation, with expressive capacity degrading polynomially as bits drop further. That is the central result being offered.\n\nThe paper applies standard approximation-theory ideas to the quantized case and tries to tie the cardinality of the weight set directly to bit precision. It does connect this to practical concerns like scaling laws and inference hardware, which is a reasonable framing even if the execution details are not shown here.\n\nThe main soft spot is exactly the one the stress test flags: the threshold depends on defining expressive power so that the functional span is governed by |Q| = 2^b with a sharp cutoff at three levels. If the paper uses a different measure (Rademacher complexity, non-uniform codebooks, or something else), the collapse point moves. Without the lemmas or the precise statement of what counts as approximation for these networks, it is impossible to tell whether the argument holds or just restates a modeling choice. Soundness cannot be assessed from the abstract alone.\n\nThe work is aimed at researchers who care about theoretical bounds on model compression rather than immediate engineering tricks. A reader who already follows quantization theory might extract a useful bound if the full derivations are clean and the definitions are stated explicitly.\n\nIt deserves a serious referee because the claim is specific enough to be checked and the topic matters for efficient LLMs. I would send it out for review once the full text is in hand, with the expectation that the definitions and proof steps will need close scrutiny.","headline":"Abstract claims 1.58-bit limit for LLM weight quantization with universal approximation above and polynomial collapse below, but no proofs or definitions are visible so the argument cannot be checked.","tokens_in":2399,"tokens_out":411,"would_cite":false,"duration_ms":13489,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Weight quantization in large language models loses universal approximation ability below 1.58 bits per weight.","keywords":["weight quantization","expressive power","universal approximation","large language models","model compression","bit precision","expressive degradation","quantized networks"],"falsifier":"A construction of a 1-bit quantized network family that can still approximate any continuous function on a compact domain to arbitrary accuracy would disprove the claimed collapse threshold.","tokens_in":2635,"feed_emoji":"","tokens_out":606,"duration_ms":17646,"temperature":0.7,"pith_summary":"The paper examines how encoding large language model weights in fewer bits affects their ability to represent complex functions. It shows that quantized models retain universal approximation when using more than 1.58 bits but collapse in expressive power below that threshold. The loss of capacity occurs polynomially as bit count drops. These results frame quantization limits in terms of scaling and compression tradeoffs. Readers interested in model efficiency would see a concrete bound on how far compression can go before core capabilities erode.","feed_headline":"1.58 bits is the limit for quantized LLM approximation","feed_subtitle":"Below this precision, models lose universal approximation and capacity drops polynomially with each further bit reduction.","key_machinery":"Restriction of network weights to finite discrete sets whose cardinality is governed by the bit precision, combined with analysis of the resulting function class's approximation properties.","core_discovery":"The paper establishes that 1.58-bit is the limiting precision for weight quantization by proving universal approximation holds for weight-quantized models above this level and expressive collapse occurs below it, while also showing that expressive capacity degrades polynomially with decreasing bit count.","pith_inferences":["The 1.58-bit threshold aligns with ternary weight sets such as {-1, 0, 1}, suggesting those representations sit at the boundary between full and collapsed expressivity.","Empirical tests measuring approximation error on simple function classes could directly observe the predicted polynomial rate.","Similar discrete-set arguments might apply to activation quantization or mixed-precision schemes.","The framework could be extended to quantify how quantization interacts with specific architectural choices like attention heads."],"forward_implications":["Models using fewer than 1.58 bits per weight cannot serve as universal approximators regardless of width or depth.","Expressive capacity scales polynomially downward with each reduction in bit precision.","Quantization-aware scaling laws must incorporate this precision-dependent degradation term.","Compression and acceleration techniques gain a theoretical floor below which further bit reduction yields qualitatively different models."],"fun_headline_variants":["1.58-bit weight quantization is the limit","Approximation fails below 1.58 bits in LLMs","Expressive collapse below 1.58-bit weights","Quantized LLM capacity drops polynomially","1.58 bits marks LLM quantization boundary"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Expressive power is defined such that the number of distinct quantization levels directly determines whether universal approximation is possible, with a sharp change at exactly three levels.","fun_headline_variants_meta":{"raw":{"variants":["1.58-bit weight quantization is the limit","Approximation fails below 1.58 bits in LLMs","Expressive collapse below 1.58-bit weights","Quantized LLM capacity drops polynomially","1.58 bits marks LLM quantization boundary"]},"model":"grok-4.3","cost_usd":0.009087,"raw_usage":{"total_tokens":3960,"prompt_tokens":597,"num_sources_used":0,"completion_tokens":71,"cost_in_usd_ticks":90865500,"prompt_tokens_details":{"text_tokens":597,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3292,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":597,"tokens_out":71,"duration_ms":24238,"temperature":1.0,"reasoning_tokens":3292,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-26T11:51:44.725659+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A construction of a 1-bit quantized network family that can still approximate any continuous function on a compact domain to arbitrary accuracy would disprove the claimed collapse threshold.","supporting_citations":[],"review_version":1}