{"id":"954c6d18-6bd5-409b-9669-42a54aa9f45a","arxiv_id":"2607.13511","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Expanding the inner rank of a ternary LLM weight factorization makes the quantization error monotonically decrease and lets effective bit-width approach bf16 arbitrarily closely.","lead":"A new method cuts LLM weight matrices into very cheap -1/0/+1 pieces, then keeps adding pieces until the remaining error is as small as desired. It is a post-training quantization approach whose accuracy can, in principle, be dialed continuously from ternary-level cost toward full-precision quality.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Energy-to-accuracy transfer is the load-bearing gap: Table 3 shows ExTernD trails Q4_K in PPL despite matched reconstruction energy and higher bpw, so the 'any quantization level' claim rests on an unvalidated proxy.","rationale":"The paper's central value proposition is a ternary format that can be tuned to any accuracy target. The theoretical theorem only concerns ||A−Â||_F; the extension to 'bf16 accuracy' is an empirical inference. The paper's own Table 3 is the only direct test of that inference and it goes against the optimistic reading: at ~5.7 bpw, the decomposed model (10.099 PPL) is worse than both Q5_K (9.948 at 5.51 bpw) and Q4_K (10.015 at 4.51 bpw). If energy were a faithful proxy, a model with mean tensor energy 99.45% should not lose to a 4.51 bpw quant that has lower per-tensor energy on the same metric. The discrepancy is the load-bearing soft spot.\n\nI also considered the proof-algorithm mismatch in Appendix A: Prop 2 is proven for an augmented safeguard, while Sec 2.3 runs plain greedy ALS. This is a real secondary gap, but it is fixable by adding the safeguard or proving a lower bound on ALS's per-step decrease; the existence result itself is sound. The energy-to-accuracy transfer is not fixable by a proof tweak and is the condition on which the headline claim actually rests.\n\nThe reader's weakest_assumption identifies the same issue, so agreement is 'agree.' The verdict should remain CONDITIONAL rather than escalate to REJECT because the theoretical contribution (expanded rank with monotone residual, and a valid convergence proof for an augmented variant) is real, and the empirical failure mode is an overclaim that could be repaired with the proposed measurement. I therefore recommend 'UNCHANGED'—the reader's conditional verdict already captures the right level of confidence.","tokens_in":8391,"tokens_out":9437,"duration_ms":91178,"concrete_test":"Re-run the Table 3 conversion on Qwen3.5-4B across μ ∈ {2,3,4,5} with τ=1.0, λ=0, no sweeps, using the same 580-chunk wikitext-2 harness, and record both PPL and mean per-tensor energy. If PPL does not monotonically approach bf16 (9.782) or at least become ≤ Q4_K pure (10.015) by μ=5 while energy keeps rising, the energy proxy is falsified. If it does, additionally build the iso-Q4_K per-matrix μ allocation from Table 2 and confirm end-to-end PPL ≤ 10.015.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that ExTernD 'matches Q4_K's per-matrix accuracy' and thereby 'approaches any quantization level.' The word accuracy is equated with reconstruction energy E = 1 − ||A−Â||²/||A||² (§2.1). But the paper's only end-to-end result, Table 3, shows the μ=3 full conversion at ~5.7 effective bpw has wikitext PPL 10.099, versus 10.015 for Q4_K pure at 4.51 bpw and 9.948 for Q5_K pure at 5.51 bpw. So at 26–30% more bits than Q4_K, ExTernD is still ~0.8–1.5% worse in relative PPL. Per-tensor energy does not transfer to task-level quality. The proof in Appendix A bounds only Frobenius reconstruction error; it says nothing about perplexity, downstream accuracy, or calibration sensitivity. The paper itself flags this ('end-to-end task accuracy is not yet measured,' §5). Thus the headline 'accuracy approaching any quantization level' is unsupported; the defensible claim is weight-reconstruction convergence, not model accuracy parity. A secondary gap: Proposition 2's geometric convergence is proven for the augmented best-single-entry safeguard, not the greedy ALS of §2.3; the assertion that the safeguard never activates is empirical and unproved.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes ExTernD, a post-training factorization of a weight matrix A into A ≈ B diag(D) C, where B and C are ternary matrices and D is a real scale vector. The inner rank is set to k = μ min(m,n), deliberately larger than full rank, so that additional components correct the residual left by previous ones. The method is fitted with a greedy sequential ALS (Sec. 2.3), a batched block-ALS variant (Sec. 2.4), and an importance-weighted objective (Sec. 2.5). The paper claims monotonic residual decrease and convergence to any ε > 0 (Sec. 2.1, Appendix A), and reports per-matrix reconstruction energy matching Q4_K at 5.2–5.5 effective bpw (Table 2). One full Qwen3.5-4B conversion at μ=3 gives wikitext-2 PPL 10.099 versus 9.782 for bf16 and 10.015 for Q4_K pure (Table 3).","tokens_in":8743,"tokens_out":4855,"duration_ms":52979,"significance":"If the central claims held, ExTernD would remove the fixed-capacity ceiling of ternary post-training quantization and provide a continuous accuracy/cost trade-off for multiplication-free LLM inference. The least-squares monotonicity argument in Prop. 1 is correct and clean, and Prop. 2 is a valid construction showing that an augmented single-entry safeguard yields geometric convergence. The empirical observations about the decorrelated block-ALS target (Sec. 2.4) and the corrected V-step for imatrix weighting (Sec. 2.5) are also valuable. However, the convergence theorem does not apply to the implemented greedy ALS without the safeguard, and the empirical 'matching Q4_K accuracy' claim is based on reconstruction energy rather than task-level accuracy; the only end-to-end result trails Q4_K at a higher bitrate. These gaps are load-bearing for the paper's headline claims.","major_comments":[{"comment":"Prop. 2 proves geometric convergence for an augmented algorithm that, at each step, chooses whichever is larger between the ALS update and the best single-entry removal. The algorithm actually implemented and described in Sec. 2.3 (and used in all experiments) does not perform this safeguard. The sentence 'In practice the safeguard never activates' is an unproved empirical assertion, not a proof. Consequently, the abstract's claim that ExTernD 'can be driven below any ε>0' and 'approaches bf16 accuracy arbitrarily closely' is not established for the method that is evaluated. To repair this, either add the safeguard to the implemented algorithm (e.g., as a fallback per iteration, checking whether a single-entry step gives a larger decrease), report its activation frequency, and re-run experiments with that inclusion; or explicitly state that the theorem applies to an augmented variant and","section":"Appendix A, Prop. 2; Sec. 2.3"},{"comment":"The abstract labels the matched Q4_K reconstruction energy as 'per-matrix accuracy,' and Sec. 3.3 chooses μ to match Q4_K's 99.4–99.5% energy values. But Table 3, the paper's only end-to-end measurement, shows the μ=3 conversion at ~5.7 effective bpw reaching 10.099 wikitext-2 PPL, versus 10.015 for Q4_K pure at 4.51 bpw and 9.948 for Q5_K pure at 5.51 bpw. Thus at roughly 26% more bits than Q4_K, the decomposition is still about 0.8–1.5% worse in relative PPL. Reconstruction energy is therefore not a demonstrated proxy for task-level accuracy, and the proof in Appendix A bounds only Frobenius error, not PPL or any downstream metric. The paper itself concedes in Sec. 5 that 'end-to-end task accuracy is not yet measured.' The headline 'accuracy approaching any quantization level' is unsupported on the empirical side. Please either provide end-to-end results with the identified levers appl","section":"Abstract; Sec. 3.3; Table 3"}],"minor_comments":[{"comment":"The transpose placement in V ← Tτ((U^T U + εI)^{-1} U^T R)^T is easy to misread. Clarify the intended dimensions, or rewrite the equation with explicit matrix shapes.","section":"Sec. 2.4, Eq. (3)"},{"comment":"The entries 'Q4 K M' and 'Q4 K pure' should be defined: the reader cannot tell what 'M' means from the table alone, nor whether 'bpw' is theoretical or measured after packing.","section":"Table 3"},{"comment":"The sentence 'our structural and representational claims against them are proven (Sec. 2.1)' is misleading: Sec. 2.1 contains no proof of a comparison against PTQTP or PT2-LLM, and Appendix A's proof concerns residual convergence, not relative comparison. Repoint the citation or temper the wording.","section":"Sec. 5"},{"comment":"The phrase 'This decisively proves' in the iso-BPW discussion is too strong for an energy-equality comparison on a handful of matrices; please soften to 'supports' or 'indicates'.","section":"Sec. 3.6"},{"comment":"Minor typo: 'TQ2 0' should likely be 'TQ2_0' or similar; also ensure the format name matches llama.cpp's actual tensor type notation.","section":"Sec. 4"}],"recommendation":"major_revision","confidential_remarks":"The paper has a potentially interesting core idea and some correct small proofs, but the gap between the proven statement and the implemented algorithm, and the absence of task-level validation for the central empirical claim, are serious. The authors should be encouraged to run the safeguard explicitly (or prove something for the greedy ALS), and to obtain at least one end-to-end comparison where per-matrix μ allocation and sweeps are actually applied. If those experiments continue to show trailing PPL at higher bitrate, the 'approaches any quantization level' wording will need to be removed. The single-author, single-GPU nature of the evaluation is an additional limitation that should be stated more prominently in the abstract or introduction."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: ExTernD is a legitimate new trick — expand the inner rank of a ternary factorization past full rank and fit each new rank-one term to the residual. That converts a fixed-capacity format into a continuous cost/accuracy dial, which is a real improvement over PTQTP's fixed plane count and PT2-LLM's single plane. The monotone residual proof (Prop 1) is correct, and Prop 2's geometric convergence is correct for the augmented single-entry safeguard. The baseline comparisons in Sec 3.6 are honestly drawn, and the algorithm details (batched ALS, importance weighting) show real engagement.\n\nThe soft spots are exactly where the reader's report and the stress-test land. First, the headline claim \"accuracy approaching any quantization level\" means reconstruction energy, not task accuracy. Table 3, the only end-to-end run, shows the μ=3 decomposed model at ~5.7 bpw gets 10.10 PPL vs 10.015 for Q4_K at 4.51 bpw and 9.948 for Q5_K at 5.51 bpw. So at roughly 26% more bits than Q4_K, the decomposed model is still slightly worse in PPL. The energy-to-PPL transfer is unvalidated. The paper's own Limitations section says end-to-end task accuracy is not yet measured, which is honest, but the abstract doesn't carry that caveat.\n\nSecond, the arbitrary-accuracy theorem is proven for an augmented step that the implemented greedy ALS does not run. The authors assert the safeguard never activates, but that is empirical, not shown. It's a minor gap because in practice the ALS decreases are larger, but it means the theorem doesn't exactly cover the algorithm used.\n\nThird, the practical claims need artifacts: no code, no error bars, single GPU, small models. Per-matrix μ selection can look like cherry-picking if not released with the iso-accuracy protocol.\n\nNone of this kills the contribution. The core representational claim — that expanded-rank ternary factors can monotonically reduce reconstruction error and break the fixed-plane ceiling — is real and appears new relative to cited work. The paper deserves a serious referee. It should go out with a request for code, error bars, and a full end-to-end comparison against PTQTP and PT2-LLM on task-level metrics, not just energy.","headline":"Expanded-rank ternary decomposition is a genuine new dial for ternary PTQ, but 'accuracy' currently means reconstruction energy, and the sole end-to-end run trails Q4_K despite more bits.","tokens_in":9194,"tokens_out":2456,"would_cite":true,"duration_ms":25673,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Expanding inner rank lets ternary LLM quantization approach bf16 accuracy arbitrarily closely.","keywords":["ternary quantization","post-training quantization","expanded rank","matrix factorization","sequential deflation","LLM compression","energy preservation","low-bit LLM"],"falsifier":"Run ExTernD with a large rank multiplier (μ≈10) on a 4B model weight matrix and measure energy; if it plateaus below the bf16 reference (e.g., below 99.99%) rather than continuing to increase, the practical 'arbitrarily close to bf16' claim fails for the implemented algorithm. Conversely, if a full conversion with all levers (sweeps, per-matrix μ, PPL-tuned τ/λ) does not close the perplexity gap to Q4_K at matched bits, then matrix-level energy parity is not transferring to task accuracy.","tokens_in":8261,"feed_emoji":"🧮","tokens_out":5256,"duration_ms":50904,"temperature":0.7,"pith_summary":"This paper introduces ExTernD, a post-training factorization that writes each LLM weight matrix as the product of two ternary matrices (-1, 0, +1) with a real scale vector in between, and lets the inner dimension k exceed the matrix's full rank. The central claim is that adding components beyond full rank fits the residual left by previous components, so reconstruction error decreases monotonically and can be made as small as desired: in principle the decomposition can approach full-precision (bf16) accuracy, something no fixed-count ternary scheme can do. Practically, at matched reconstruction energy ExTernD uses about 5.2-5.5 effective bits per weight versus 4.5 bpw for the Q4_K baseline, and a full 4B model conversion at rank multiplier 3 reaches 10.10 wikitext-2 perplexity versus 9.78 for bf16, placing it near the Q4_K/Q5_K accuracy band. The significance is that ternary quantization stops being a lossy format with a hard ceiling and becomes a continuous accuracy/cost dial, tunable per matrix, which matters for multiplication-free inference hardware.","feed_headline":"Ternary LLM weights hit any accuracy by expanding inner rank","feed_subtitle":"Rank multiplier μ drives residual error toward zero, matching Q4_K at ~5.5 effective bits per weight.","key_machinery":"The central object is the factorization A ≈ B diag(D) C with ternary factors B ∈ {-1,0,+1}^{m×k}, C ∈ {-1,0,+1}^{k×n} and a real scale vector D; its key parameter is the inner-rank multiplier μ = k/min(m,n), which acts as an accuracy dial. The argument is carried by sequential deflation: component i is fit to the residual R after previous components, and the optimal scale d_i = uᵀRv/(||u||²||v||²) makes the residual norm decrease monotonically. A batched block-ALS variant approximates this sequential deflation using decorrelated least-squares targets to avoid collapsing onto the dominant residual direction. The convergence proof adds a single-entry removal step (best residual entry) that gua","core_discovery":"ExTernD's core discovery is that a ternary factorization with an expanded inner rank — k = μ·min(m,n) with μ>1 — removes the fixed capacity ceiling of ternary quantization. Each added component is fit to the residual of its predecessors (sequential deflation), and with a least-squares scale the residual norm is monotone non-increasing; the paper proves it can be driven below any ε>0 by growing k, approaching bf16 accuracy. No fixed number of ternary planes has this property. Empirically, matching Q4_K's ~99.4–99.5% per-matrix energy requires only 5.2–5.5 effective bits per weight, and a full 4B conversion at μ=3 reaches 10.10 wikitext-2 perplexity vs 9.78 for bf16, near the Q4_K/Q5_K band.","pith_inferences":["The proof of arbitrary accuracy relies on a best-single-entry augmentation that the implemented ALS does not run; whether the practical algorithm alone converges to arbitrarily small error remains to be demonstrated empirically.","If energy parity transfers to task performance across scales and tasks, the ~17–20% bpw gap to Q4_K is likely a multiplier (algorithmic) problem rather than a coding limit, since the packing is near the entropy floor.","The same sequential-deflation idea could in principle be applied to other structured formats, such as binary or low-bit factors, as a general way to remove fixed-capacity ceilings.","A short straight-through QAT pass on an ExTernD initialization is an obvious next test; if gradients through two chained ternary factors behave, it could recover the remaining perplexity gap faster than training a conventionally quantized model."],"forward_implications":["If the central claim holds, ternary LLM quantization no longer has a hard accuracy ceiling; any target error can be met by raising μ, with memory and compute growing linearly in μ.","Per-matrix μ and τ give a continuous accuracy/cost trade-off, so deployments can hit an exact bit/accuracy target instead of rounding to the next k-quant bit-width.","At matched effective bits, ExTernD matches the reconstruction energy of a 4-plane dense ternary stack while being ~56% sparse, which is friendlier to multiplication-free hardware.","A full 4B model can be converted post-training in ~20 minutes on a single GPU, producing coherent text near the Q4_K/Q5_K perplexity band, with identified levers (sweeps, per-matrix rank allocation, importance weighting) expected to close the remaining gap."],"fun_headline_variants":["Expanding rank lets ternary LLM weights beat fixed-bit limits","Ternary LLM quantization: expand rank to match Q4_K accuracy","ExTernD: ternary weights approach bf16 by expanding rank","Rank expansion turns ternary LLM weights into an accuracy dial","Want Q4_K accuracy? Use ExTernD ternary with expanded rank"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"Per-matrix reconstruction energy (E = 1 − ‖A−Â‖²/‖A‖²) is treated as a proxy for model accuracy; the end-to-end perplexity test still shows a gap (10.10 vs 9.78 bf16), and the convergence proof assumes a best-entry step the implemented algorithm does not run.","fun_headline_variants_meta":{"raw":{"variants":["Expanding rank lets ternary LLM weights beat fixed-bit limits","Ternary LLM quantization: expand rank to match Q4_K accuracy","ExTernD: ternary weights approach bf16 by expanding rank","Rank expansion turns ternary LLM weights into an accuracy dial","Want Q4_K accuracy? Use ExTernD ternary with expanded rank"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000267,"raw_usage":{"total_tokens":1531,"prompt_tokens":905,"completion_tokens":626,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":649,"completion_tokens_details":{"reasoning_tokens":535}},"tokens_in":649,"tokens_out":626,"duration_ms":6830,"temperature":1.0,"reasoning_tokens":535,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T04:57:50.670807+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run ExTernD with a large rank multiplier (μ≈10) on a 4B model weight matrix and measure energy; if it plateaus below the bf16 reference (e.g., below 99.99%) rather than continuing to increase, the practical 'arbitrarily close to bf16' claim fails for the implemented algorithm. Conversely, if a full conversion with all levers (sweeps, per-matrix μ, PPL-tuned τ/λ) does not close the perplexity gap to Q4_K at matched bits, then matrix-level energy parity is not transferring to task accuracy.","supporting_citations":[],"review_version":1}