{"id":"7e43a39e-0b5b-4073-97f4-772726ac8acf","arxiv_id":"2504.19746","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"FineQ quantizes LLM weights to 2.33 bits by protecting outliers inside 3-weight clusters and adds a temporal-coding accelerator that replaces multipliers with adders.","lead":"The paper introduces FineQ, a method that compresses large language models by storing most weights in 2 bits and protecting rare outliers with 3 bits inside tiny clusters of three numbers. It also designs a chip that uses addition instead of multiplication on these low-bit weights, cutting reported chip area by 61.2% and energy use by up to 1.79x.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1's signed min/max outlier test fires for nearly every signed-weight cluster; Table I likely depends on an unstated abs()/magnitude variant.","rationale":"The paper's central numerical claim is the perplexity table, and the mechanism that is supposed to produce it is the intra-cluster outlier test in Algorithm 1. That test is the load-bearing link between the described method and the reported results. As printed, it is not a test for outliers on signed data: whenever any of the three weights is negative, min(C) < 0 and therefore max(C) > 4*min(C) is guaranteed, so the outlier branch is selected. Since LLM weights are approximately zero-centered, this happens for the large majority of clusters; the 2-bit normal coding that the method is named for would rarely be selected, and zeroing the smallest element would discard information even in clusters with no outlier. Eq. 1's scale formula has a related signed-weight ambiguity. The reader's weakest_assumption identified exactly this issue, and I agree: it is concrete, central, and directly testable. The hardware claims are less problematic: the temporal-coding PE array is described with an RTL implementation and 45nm synthesis, and the baseline is stated as a MAC-based systolic array. The absence of released code/RTL does not by itself refute anything, but it turns this pseudocode ambiguity into a reproducibility blocker. If a literal reimplementation matches Table I, the concern disappears; otherwise the paper needs a corrected algorithm statement or a caveat. The existing CONDITIONAL verdict remains appropriate, so no verdict adjustment is needed.","tokens_in":11332,"tokens_out":12083,"duration_ms":116707,"concrete_test":"Reimplement Algorithm 1 exactly as printed on a real LLaMA-2-7B checkpoint (or on a large random matrix with the same signed, centered weight statistics), including Eq. 1 as written; log the fraction of 3-weight clusters that take the outlier branch and compute WikiText-2 and C4 perplexity. If the branch fraction is close to 1 or the perplexity is far from Table I's 10.94/14.95, the reported accuracy depends on an unstated abs()/magnitude variant. Then re-run with the branch condition changed to \"max(abs(C)) > 4 * min(abs(C))\" and magnitude-based top-two selection; Table I should match only the corrected variant, confirming that the printed Algorithm 1 is not what produced the results.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Algorithm 1 lines 6–8 define the outlier branch as \"max val > 4 × min val\" with min and max taken directly from signed weight clusters. For any cluster whose minimum is negative, 4*min < min <= max, so the inequality is trivially true; the outlier branch (zero the smallest value, 3-bit code the other two) would be selected for the majority of clusters in centered LLM weights, contradicting the paper's claim that 2-bit coding is the common case and that fine-tuning is rarely needed. Eq. 1 is also ambiguous for signed weights: s = abs(xmax)/(2^{b-1}-1) is not max(abs(C)), and \"top two values\" is not defined as top two magnitudes. If the actual implementation instead uses min(abs(C)) and magnitude-sorted top-two selection, that variant is not what Algorithm 1 prints. No code or RTL is released to resolve the ambiguity, so Table I (e.g., LLaMA-2-7B 10.94/14.95) cannot currently be reproduced from the pseudocode. A secondary but related inconsistency is Algorithm 1 lines 15–24, where the neighbor-encoding rule as written propagates the first cluster's encoding across each channel, which would defeat fine-grained allocation; the intended four-encoding optimization needs restating.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"FineQ proposes a software-hardware co-design method for low-bit, fine-grained mixed-precision quantization of LLM weights. On the algorithm side, weights are partitioned into per-channel clusters of three values; each cluster is either quantized uniformly to 2 bits or, if an outlier is detected, the two largest values are quantized to 3 bits while the smallest is set to zero. A 2-bit format code and a memory-alignment scheme are used to encode cluster types, and an accelerator based on temporal coding is proposed to replace conventional multipliers in a systolic array. The paper reports perplexity results on WikiText-2 and C4 for LLaMA-2 3B/7B/13B at an average bit-width of 2.33 bits, claiming better accuracy than OWQ and PB-LLM, and reports a 61.2% systolic-array area reduction and up to 1.79x energy efficiency from RTL synthesis and cycle-level simulation.","tokens_in":11555,"tokens_out":13481,"duration_ms":139223,"significance":"If the results are reproducible, FineQ is a useful contribution: it offers a no-retraining, 2.33-bit weight-only quantization scheme with a plausible hardware mapping, and the temporal-coding PE array is a concrete attempt to exploit low-bit computation beyond conventional MAC replacement. The paper is honest about its engineering choices, does not claim a parameter-free derivation, and provides synthesis-based hardware numbers, which is a strength. However, the central algorithmic claims currently depend on ambiguous pseudocode and bit-accounting details, so the significance is conditional on a corrected and released implementation. The gap between FineQ and the baselines in Table I is large enough that the approach is worth pursuing, but the reported numbers cannot be verified from the manuscript as written.","major_comments":[{"comment":"The outlier test as printed uses signed values: 'if max val > 4 × min val'. For signed, roughly centered LLM weights, most clusters contain a negative value, so 4×min is very negative and the inequality is almost always true. The outlier branch would then zero the smallest (usually most negative) element instead of protecting it, which contradicts the paper's claim that 2-bit encoding is the common case and makes the perplexity results in Table I not derivable from the printed algorithm. Please state explicitly whether the intended test uses absolute values, e.g., max(|C|) > 4 × min(|C|), define 'top two values' as top two magnitudes in the 3-bit branch, and reconcile Eq. (1), where s = abs(xmax)/(2^{b-1}-1), with signed per-cluster bit-widths. Figure 4 only shows positive weights and does not resolve this ambiguity.","section":"Section III.A, Algorithm 1 (lines 6-8), Eq. (1)"},{"comment":"The reported average bit-width of 2.33 bits is inconsistent with the text as written. A per-cluster 2-bit format code plus 6 data bits gives 8 bits per 3 weights, i.e., 2.67 bits/weight; to obtain 2.33 bits/weight, the format code must be shared between a pair of adjacent clusters (6 data bits + 1 index bit per 3 weights). This matches the sentence 'four index values are encoded within a single byte ... subsequent eight clusters', but it contradicts Figure 4 step 5, which draws one 2-bit code per cluster ('00 10 00 11'). Algorithm 1's neighbor loop (lines 15-24) copies the first cluster's encoding to every later cluster, which would force one encoding for the whole channel and does not implement pairwise optimization; the prose is also self-contradictory, saying adjacent clusters are required to use the same encoding and then discussing cases where they use different encodings. Please restate the index-sharing scheme and pair-optimization loop, define the arg-min loss in line 22, and give an explicit bit-accounting formula.","section":"Section III.B, Algorithm 1 (lines 15-24), Table I"},{"comment":"The headline hardware numbers need a precise comparison boundary. Table III labels the baseline 'Systolic Array 64×64 PEs' and the proposed design 'FineQ PE Array 64×64 PEs', while Fig. 8 reports that the ACC unit consumes 71.8% of the FineQ PE-array power. Please state explicitly whether the baseline row includes accumulator units and whether the FineQ PE-array row includes ACC and temporal encoders, so the 61.2% area reduction and the up-to-1.79x energy efficiency are computed on identical component boundaries. In addition, because temporal coding is bit-serial, the energy-efficiency comparison in Fig. 9 should state how bitstream length and extra clock cycles are amortized.","section":"Table III, Fig. 9"},{"comment":"The empirical accuracy claim rests on a single configuration with no sensitivity analysis. The cluster size of 3, the 4× threshold in Algorithm 1, and the 2-bit/3-bit bit-width allocation are free engineering parameters, yet no ablation shows how Table I changes with these parameters, and no error bars or repeated-run statistics are reported. Please add this sensitivity analysis and describe the exact evaluation protocol, including tokenizer, perplexity computation, and whether embedding and final layers are quantized; releasing the quantization code and RTL would resolve the ambiguity in Algorithm 1 and allow Table I to be checked.","section":"Section V"}],"minor_comments":[{"comment":"There are typos ('differnet', 'fined-grained') and the x-axis of Fig. 1 shows integer bit-widths 16, 8, 4, 3, 2, so FineQ's 2.33-bit point is not visually placed; consider adding an explicit marker or axis annotation.","section":"Abstract, Section I, Fig. 1"},{"comment":"The entry 'FineQ(Ours)2.3364.47' is missing a separator between the average bit-width and the first perplexity value; this makes the table difficult to read.","section":"Table II"},{"comment":"The table reports no variance or repeated evaluations, and the baselines are described only by average bit-width; adding the exact configuration (e.g., group size for OWQ, retention ratio for PB-LLM) in a column would help readers interpret the comparison.","section":"Table I"},{"comment":"The abstract and conclusion call FineQ 'SOTA' relative to mixed-precision quantization, but the comparison omits several recent low-bit PTQ methods, e.g., SqueezeLLM, APTQ, QuIP#, and AQLM; please qualify the claim or add such comparisons.","section":"Related Work, Section V"},{"comment":"In the decoder block diagram, the zero-padding paths for 2-bit data are shown as '000' wires but are not labeled in the figure; a short caption note would make the padding behavior self-contained.","section":"Fig. 6"}],"recommendation":"major_revision","confidential_remarks":"The main risk is algorithmic reproducibility. The signed-outlier-test ambiguity and the 2.33-bit versus 2.67-bit accounting discrepancy are load-bearing, and both are fixable in a revision. If the authors can provide corrected pseudocode, a bit-accounting formula, and ideally code, the paper could become publishable; if they cannot, the empirical claims in Table I should not be accepted as stated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"FineQ is a legitimate step forward for ultra-low-bit LLM quantization. The core idea — splitting each channel into three-weight clusters and giving each cluster a 2-bit/3-bit choice based on whether it contains an outlier — is new, simple, and effective in the reported numbers. Table I shows clean wins over OWQ and PB-LLM at a comparable average bit width on LLaMA-2 3B/7B/13B, and Table II shows it also holds across sequence lengths. The temporal-coding PE array is a nice fit: it removes hardware multipliers, and the 61.2% area reduction and 1.79x energy efficiency are consistent with what you'd expect from that design choice. The RTL synthesis plus cycle-level simulator is real engineering effort, and the paper is generally clearly written.\n\nThe soft spots are real, and one of them is load-bearing. Algorithm 1's outlier test (lines 6-8) compares signed max val and min val. For any cluster with a negative minimum, max > 4*min is trivially true, so the 3-bit branch would fire for nearly every cluster in typical signed LLM weights. That contradicts the paper's claim that most clusters use 2 bits and implies the perplexity results in Table I can't be reproduced from the pseudocode as printed. The stress-test note I got flags this, and on reading the paper I think it's correct. The fix is likely simple — compare magnitudes, e.g., max(abs(C)) > 4*min(abs(C)) — but it needs to be stated and ideally code or a corrected algorithm released. There's also a secondary issue in Algorithm 1 lines 15-24: the neighbor-encoding rule as written propagates the first cluster's encoding across each channel, which would erase the fine-grained allocation. The accompanying text describes the intended behavior, but the pseudocode doesn't match it. Both are presentation/correctness bugs that an editor should flag.\n\nThe smaller issues are what you'd expect: no error bars on the perplexity tables, no ablation on the 4x threshold or cluster size, and the hardware baseline doesn't specify the MAC precision, which matters for a fair area/power comparison. None of these are fatal; they're standard for a systems paper at this stage.\n\nIf the algorithmic ambiguities are cleared up, this is a solid paper for the efficient-inference community. It deserves a serious peer review — not a desk reject — and I'd want the reviewers to verify the algorithm's actual behavior on signed weights before acceptance. For my own work, I'd cite it as a representative fine-grained mixed-precision scheme.","headline":"FineQ has a genuinely new fine-grained mixed-precision scheme with strong empirical results, but Algorithm 1's signed min/max outlier test is wrong as written and must be fixed before the results are reproducible.","tokens_in":12118,"tokens_out":3079,"would_cite":true,"duration_ms":28028,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FineQ claims that partitioning each LLM weight channel into clusters of three, with 2-bit normal values and 3-bit protected outliers, keeps perplexity competitive at an average of 2.33 bits per weight, and that the matching…","keywords":["large language models","low-bit quantization","mixed-precision quantization","outlier protection","temporal coding","systolic array","hardware accelerator","software-hardware co-design"],"falsifier":"Run Algorithm 1 as printed on the actual LLaMA-2-7B weights and count the clusters that satisfy $\\max > 4\\min$. If the fraction is close to one (as signed arithmetic on mixed-sign weights predicts) rather than the roughly 0.3% outlier rate claimed, then the outlier-protection mechanism as written is not what produces the reported perplexity, and the algorithm is using an unstated absolute-value comparison.","tokens_in":11102,"feed_emoji":"⚙️","tokens_out":11784,"duration_ms":108638,"temperature":0.7,"pith_summary":"FineQ is a software-hardware co-design for pushing LLM weight quantization below 3 bits. The paper's claim is that the reason existing mixed-precision quantizers lose accuracy is coarse granularity: protecting outliers group-wide wastes bits or misses outliers. Its fix is to cut each weight channel into clusters of three, give every member 2 bits normally, and when a cluster contains an outlier give the two largest values 3 bits and zero the third. This reaches an average bit-width of 2.33 bits and, in the paper's measurements on LLaMA-2-3B/7B/13B, achieves lower perplexity on WikiText-2 and C4 than OWQ and PB-LLM. The companion accelerator encodes weights as temporal bitstreams so multiplications become additions, reducing systolic-array area by 61.2% and improving energy efficiency by up to 1.79x.","feed_headline":"Triple-sized weight clusters keep LLMs accurate at 2.33 bits","feed_subtitle":"Outliers get 3 bits, ordinary weights 2, beating coarser mixed-precision baselines; the matching hardware shrinks the systolic array by…","key_machinery":"The load-bearing object is the size-3 weight cluster with its 2-bit format code: one code says all three values are 2-bit, and the three other codes say this position is zeroed while the other two positions are 3-bit. Scale factors are per channel using the symmetric uniform-quantization formula $s = \\mathrm{abs}(x_{\\max})/(2^{b-1}-1)$, where $b$ is the cluster's bit-width. The hardware lever is temporal coding: each low-bit weight is converted into a fixed-length bitstream whose number of ones equals its value, so multiplication by the weight becomes a sequence of conditional additions of the activation, replacing the multiplier with an encoder, selectors, and an adder tree.","core_discovery":"The central discovery is that fine-grained, per-cluster bit allocation makes 2-bit-scale quantization viable without retraining. Within each channel, FineQ computes a per-channel scale factor, forms clusters of three adjacent weights, and makes a binary decision: if the cluster's maximum exceeds four times its minimum, the two largest values are stored in 3 bits and the smallest is zeroed; otherwise all three are stored in 2 bits. A 2-bit format code per cluster, shared with a neighbor to keep memory aligned, tells the decoder which layout to expect. The paper reports this 2.33-bit average scheme beats OWQ at 2.25 bits and PB-LLM at 2.7 bits in perplexity across the LLaMA-2 family, and that the temporal-coding systolic array, which broadcasts unary bitstreams of the weights into a PE array and accumulates selected activations, reduces systolic-array area by 61.2% and power by 62.9% at 45 nm.","pith_inferences":["A reader who implements Algorithm 1 verbatim on signed weights should first count how many clusters satisfy $\\max > 4\\min$; if that fraction is near one, the published accuracy numbers are best explained by an unstated absolute-value variant of the test.","The choice of cluster size 3 is tied to the 2-bit format code packing into one byte; varying the cluster size while keeping the same packing would be a direct ablation of the paper's claim that finer granularity is what protects outliers.","Temporal coding of weights is agnostic to the specific 2/3-bit policy; any scheme that turns weights into small integers can reuse the multiplier-free PE array, so the hardware result applies beyond FineQ's exact bit allocation.","The 61.2% area saving is for the systolic array only; including the decoder, buffers, and control logic would give a fuller end-to-end comparison, although the decoder itself is reported at only 0.008 mm^2."],"forward_implications":["FineQ reaches an average bit-width of 2.33 bits and reports lower perplexity than OWQ at 2.25 bits and PB-LLM at 2.7 bits on LLaMA-2-3B, 7B, and 13B on WikiText-2 and C4, so coarse-grained grouping is not needed to protect outliers.","The 2-bit cluster format code, forced to be shared with a neighboring cluster, packs four allocation codes into one byte and eliminates a separate sparse index for outliers.","The quantization algorithm runs offline without retraining, so an existing FP16 model can be re-encoded and then served without gradient updates.","At 45 nm, replacing MAC multipliers with temporal-coding PEs lowers the 64x64 systolic array area from 0.954 mm^2 to 0.370 mm^2 and its power from 88.793 mW to 32.891 mW.","Energy efficiency improves by up to 1.79x across sequence lengths under input-stationary dataflow, with the accumulation unit, not the PEs, becoming the dominant power consumer."],"supporting_citations":[{"why":"Provides the LLaMA-2-3B/7B/13B models on which all perplexity results are measured.","marker":"[2]"},{"why":"Supplies the temporal/unary coding that the accelerator's bitstream PE array is built on.","marker":"[3]"},{"why":"Defines the RTN baseline that FineQ compares against at 2 bits.","marker":"[4]"},{"why":"Defines the GPTQ baseline, the single-precision method that collapses at ultra-low bits.","marker":"[5]"},{"why":"Defines OWQ, the coarse-grained mixed-precision baseline with FP16 outliers that FineQ claims to beat at close bit-width.","marker":"[9]"},{"why":"Defines PB-LLM, the partially binarized baseline that FineQ claims to beat.","marker":"[11]"},{"why":"Supplies the symmetric uniform-quantization formula and the Uniform baseline.","marker":"[14]"},{"why":"Supplies the logic-synthesis methodology used to report the area and power reductions.","marker":"[20]"},{"why":"Supplies the 45 nm standard-cell library used for the area and power measurements.","marker":"[21]"},{"why":"Defines the input-stationary dataflow used as the common comparison condition for the accelerator results.","marker":"[26]"}],"fun_headline_variants":["FineQ: 2.33-bit LLMs, 3-bit outliers, 61% smaller systolic array","Per-cluster bit allocation yields 2.33-bit LLMs, 61% smaller hardware","Outlier protection in fine clusters: 2.33-bit LLMs beat coarser baselines","2.33-bit LLMs via fine-grained clusters and 3-bit outlier coding","Hardware-aware quantization: 2.33-bit LLMs, 61% less area, 1.79x energy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole accuracy story rests on the outlier test firing only on rare clusters; if the printed comparison is read literally on signed weights, nearly every cluster fires, so the reported gains depend on an interpretation the paper does not state.","fun_headline_variants_meta":{"raw":{"variants":["FineQ: 2.33-bit LLMs, 3-bit outliers, 61% smaller systolic array","Per-cluster bit allocation yields 2.33-bit LLMs, 61% smaller hardware","Outlier protection in fine clusters: 2.33-bit LLMs beat coarser baselines","2.33-bit LLMs via fine-grained clusters and 3-bit outlier coding","Hardware-aware quantization: 2.33-bit LLMs, 61% less area, 1.79x energy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001409,"raw_usage":{"total_tokens":5729,"prompt_tokens":1014,"completion_tokens":4715,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":630,"completion_tokens_details":{"reasoning_tokens":4588}},"tokens_in":630,"tokens_out":4715,"duration_ms":36982,"temperature":1.0,"reasoning_tokens":4588,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:44:42.733996+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Algorithm 1 as printed on the actual LLaMA-2-7B weights and count the clusters that satisfy $\\max > 4\\min$. If the fraction is close to one (as signed arithmetic on mixed-sign weights predicts) rather than the roughly 0.3% outlier rate claimed, then the outlier-protection mechanism as written is not what produces the reported perplexity, and the algorithm is using an unstated absolute-value comparison.","supporting_citations":[{"cited_title":"ubrain: A unary brain computer interface,","cited_arxiv_id":null,"evidence_quote":"Supplies the temporal/unary coding that the accelerator's bitstream PE array is built on."},{"cited_title":"Up or down? adaptive rounding for post-training quantization,","cited_arxiv_id":null,"evidence_quote":"Defines the RTN baseline that FineQ compares against at 2 bits."},{"cited_title":"Owq: Outlier-aware weight quantization for efficient fine-tuning and inference of large language models,","cited_arxiv_id":null,"evidence_quote":"Defines OWQ, the coarse-grained mixed-precision baseline with FP16 outliers that FineQ claims to beat at close bit-width."},{"cited_title":"Kurup and T","cited_arxiv_id":null,"evidence_quote":"Supplies the logic-synthesis methodology used to report the area and power reductions."},{"cited_title":"Ascend-freepdk45: An open source standard cell library for asyn- chronous design,","cited_arxiv_id":null,"evidence_quote":"Supplies the 45 nm standard-cell library used for the area and power measurements."}],"review_version":1}