{"id":"6f8bf8c3-b335-40a0-b488-4708383300f7","arxiv_id":"2412.09687","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"DQA quantizes neural network activations to below 6 bits by using extra precision on important channels, truncating it with bit shifts, and Huffman-coding the truncation error, improving accuracy over direct quantization and NoisyQuant in most, but not all, tested settings.","lead":"This paper offers DQA, a method to compress neural network activations to 3, 4, or 5 bits by giving important channels extra precision and storing the discarded low-order bits as Huffman codes. It beats simple quantization and NoisyQuant in most tested cases, but not all, and the claimed efficiency is not measured.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DQA's central accuracy claim is contradicted by the paper's own Table 1: DQA does not consistently beat NoisyQuant, and the 29.28% figure is only versus direct quantization.","rationale":"The reader's weakest_assumption (distribution shift) is a real limitation but not the most immediate risk. The paper's own results provide a direct counterexample to the strongest version of the central claim, so no external data are needed to see the problem. Table 1 shows DQA below NoisyQuant in U-Net 3-bit and MobileNetV2 5-bit, and the 'up to 29.28%' improvement is only relative to Direct, not NoisyQuant. That makes the abstract's universal statement internally inconsistent. I agree with the CONDITIONAL verdict because the method has value—it beats Direct everywhere and NoisyQuant in 7 of 9 configurations—but the claims need to be re-scoped and ideally supported by error bars and released artifacts. The concrete check is mostly arithmetic plus a verification rerun of the two losing cells; this directly settles whether the strongest form of the claim holds.","tokens_in":7219,"tokens_out":4216,"duration_ms":41786,"concrete_test":"Take Table 1 and compute all pairwise accuracy differences. Specifically: (1) confirm that 29.28 = 82.13−52.85 (DQA−Direct) and not 82.13−71.22 (DQA−NoisyQuant); (2) confirm the two negative DQA−NoisyQuant cells: U-Net 3-bit (90.38−92.16=−1.78) and MobileNetV2 5-bit (91.28−91.33=−0.05). If those arithmetic checks pass, rerun the two losing configurations with multiple seeds and report mean±std over at least 5 runs; if DQA still loses or ties, rewrite the headline to 'DQA improves over direct quantization, and is competitive with NoisyQuant in most configurations' rather than claiming universal superiority over NoisyQuant.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that DQA achieves 'significantly better accuracy (up to 29.28%) compared to the direct quantization method and the state-of-the-art NoisyQuant'—is not supported by Table 1. The 29.28% figure is the absolute percentage-point gap between DQA and Direct for ResNet-32 at 3 bits (82.13−52.85=29.28); the corresponding gap to NoisyQuant is 82.13−71.22=10.91 pp. More seriously, DQA is worse than NoisyQuant in two of the nine reported configurations: U-Net 3-bit (90.38 vs 92.16, −1.78 pp) and MobileNetV2 5-bit (91.28 vs 91.33, −0.05 pp). Thus the method is not uniformly superior to the stated state of the art, and 'significantly better' is not established: runs are averaged over 5 trials but no variance or significance test is reported. The method still consistently beats Direct and often beats NoisyQuant, but the abstract and conclusion overstate the results. This is an internal inconsistency, not a speculation about distribution shift: the evidence needed to refute the strongest form of the claim is already in the paper.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DQA, a post-training quantization method for DNN activations at 3–5 bits. DQA identifies 'important' activation channels offline via a greedy search on calibration/training data, quantizes those channels with m extra bits and then right-shifts them to the target bit width, Huffman-encodes the shifting errors, and adds the residuals back during dequantization; the remaining channels are quantized directly. The method is evaluated against direct uniform symmetric quantization and NoisyQuant on ResNet-32 and MobileNetV2 (CIFAR-10) and U-Net (Cityscapes). The paper claims significantly better accuracy (up to 29.28%) and suitability for resource-constrained devices, with simplicity coming from shifts, table lookups, and Huffman coding.","tokens_in":7528,"tokens_out":12816,"duration_ms":121662,"significance":"DQA is appealing in its simplicity: the online operations are shifts and adds, no matrix multiplications or online searches are required, and the error analysis in Eq. (4) gives a clean explanation of why the extra m bits reduce quantization error exponentially. The offline channel-ranking idea is also practical and clearly described. If the accuracy gains held uniformly, the method would be a useful contribution to sub-6-bit activation quantization. However, the evidence as presented is mixed: Table 1 shows DQA losing to NoisyQuant in two of the nine configurations, no variance is reported despite five runs, and the efficiency claims are not measured. The central idea is worth pursuing, but the manuscript's claims need to be brought in line with the evidence, and several load-bearing details need clarification.","major_comments":[{"comment":"The abstract, introduction, and conclusion state that DQA shows 'significantly better accuracy (up to 29.28%)' compared with both direct quantization and NoisyQuant. Table 1 does not support this claim. At 3-bit U-Net, NoisyQuant achieves 92.16% versus DQA's 90.38%, and at 5-bit MobileNetV2, NoisyQuant achieves 91.33% versus DQA's 91.28%. The 29.28 percentage-point figure is the ResNet-32 3-bit gap to Direct (82.13−52.85); the corresponding gap to NoisyQuant is 10.91 percentage points. Please revise the central claims to state the per-baseline and per-configuration results and avoid implying uniform superiority over NoisyQuant.","section":"Abstract, §1, §5, Table 1"},{"comment":"Every experiment is reported as an average over five runs, but no standard deviation, confidence interval, or significance test is reported anywhere. Without this, 'significantly better' is not established, particularly for the 0.05 percentage-point difference at MobileNetV2 5-bit and the small differences at U-Net 4-bit and 5-bit. Please report variances or run a pairwise significance test between DQA and each baseline.","section":"§4.1"},{"comment":"The title and abstract claim efficiency, but the paper contains no measurement of inference time, throughput, memory footprint, or energy on any device; all experiments run on an RTX 3090 and Section 4.1 explicitly says the goal is accuracy. The only efficiency-related number is the Huffman compression ratio of 1.12 for CIFAR-10, which does not account for the offline rank table, the Huffman code table, or the precomputed shift-error mapping. Please either add efficiency measurements or explicitly limit the contribution to accuracy and soften the efficiency claims.","section":"§1, §3.2.2, §4.1"},{"comment":"The important-channel ranking is a key component of the reported accuracy gains, but its construction has underspecified choices. First, Section 3.1 says the greedy search runs 'inference on evaluation data,' while Section 4.1 says a random subset of training data is used; if the ranking procedure uses test data, there is data leakage and the results are not a valid post-training quantization evaluation. Second, the important-channel ratios (40% for classification, 50% for segmentation) and m=3 are fixed with no sensitivity analysis, so the reported gains may be partly tuned to these choices. Third, the greedy per-channel ranking is not compared with cheaper alternatives such as magnitude-based channel importance. Please clarify the exact data used for ranking and add ablations or a sensitivity discussion.","section":"§3.1, §4.1, Algorithm 3"}],"minor_comments":[{"comment":"The denominator notation in Eq. (1) and Algorithm 1 is ambiguous: '2N−1' and '2n+m−1' could be read as 2^N−1 and 2^{n+m}−1 or as 2^{N-1} and 2^{n+m-1}. This matters because Eq. (4) relies on the relation Δ_N = 2^m Δ_{N+m}. Please state the denominator convention explicitly and ensure Algorithm 2 uses the same convention.","section":"Eq. (1), Algorithm 1"},{"comment":"Please define the 'compression ratio' precisely. If it is the ratio of the raw size of shifting errors to the size of the Huffman-coded errors, clarify whether the Huffman table size is included, and report the ratio for the Cityscapes/U-Net experiments as well as for CIFAR-10.","section":"§3.2.2"},{"comment":"The figure lacks axis labels and a legend. The y-axis presumably shows frequency, but this should be stated, and it is unclear whether the distributions are averaged over batches, channels, or both.","section":"Figure 2"},{"comment":"The statement 'we set m = 3 in all cases to simplify the evaluation' means that for 3-bit quantization m equals n, while for 4- and 5-bit it does not. Reporting at least one configuration with a different m would strengthen the claim that the method is not tied to this particular choice.","section":"§4.1"},{"comment":"The '29.28%' and '0.9%' figures are percentage-point differences, not relative improvements. Please use 'percentage points' or report relative improvements to avoid misleading readers.","section":"Abstract, §4.2"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern is valid: the paper's own Table 1 contradicts the abstract's universal accuracy claim, and no significance or efficiency evidence supports the other central claims. These issues are fixable within the manuscript's scope by rewriting the claims and adding the missing analyses, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a decent, modest paper that overstates its results in the abstract. The core idea — take AWQ's importance-aware treatment, move it from weights to activations, quantize important channels with a few extra bits, right-shift to the target precision, and Huffman-code the shifted-out bits — is a sensible combination of known pieces. The algebraic error analysis (Eqs. 1-4) is correct and shows exponential error reduction in m; that part is solid. The evaluation covers three models, two tasks, two datasets at 3/4/5 bits, and DQA beats direct quantization everywhere and beats NoisyQuant in most configurations, sometimes by a lot (ResNet-32 3-bit: 82.13 vs 71.22). That is real evidence the method has value.\n\nBut the abstract's 'up to 29.28% better than direct and NoisyQuant' is wrong. The 29.28% figure is only vs direct; the gap to NoisyQuant at that point is 10.91 pp. More importantly, Table 1 shows two cells where DQA loses to NoisyQuant: U-Net at 3 bits (90.38 vs 92.16) and MobileNetV2 at 5 bits (91.28 vs 91.33). The paper's own Section 4.2 concedes the U-Net case, but the abstract and conclusion repeat the universal claim. Also, no variance is reported despite five runs, and there is no efficiency measurement anywhere, so the word 'efficient' in the title is aspirational.\n\nOther soft spots: the channel ranking is a greedy search on calibration data, and the ratios (40/50%) and m=3 are hand-picked; the distribution-shift assumption in Sec. 3.1 is stated but never tested. These are normal limitations for this kind of paper, not fatal flaws. The method itself is simple and plausible, and the Huffman error-distribution plot at least supports the compression choice.\n\nBottom line: this is a workshop-quality paper with a useful idea and honest-enough experiments, but the claims need to be scaled back to match the data. A serious referee would catch the mismatch in five minutes. If the authors fix the abstract, add error bars, and ideally measure actual latency/memory, it would be a fine contribution.","headline":"A useful but overclaimed activation quantization paper: the method mostly works, but the abstract's superiority over NoisyQuant is contradicted by its own Table 1.","tokens_in":7990,"tokens_out":1956,"would_cite":false,"duration_ms":18544,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Sub-6-bit activation quantization can stay accurate if important channels get extra bits, right-shifted back, with the discarded errors stored as Huffman codes.","keywords":["activation quantization","deep quantization","Huffman coding","channel importance","mixed-precision quantization","efficient inference","sub-6-bit quantization","CNN"],"falsifier":"Run DQA with ranks computed on clean CIFAR-10 but evaluate on a corrupted or out-of-distribution variant, and compare accuracy against direct quantization; if DQA no longer beats direct quantization once channel importance shifts, the offline-ranking assumption is falsified. Alternatively, implement DQA on a microcontroller and measure end-to-end latency and energy; if Huffman decoding and side-channel overhead outweigh the savings, the resource-constrained motivation collapses.","tokens_in":7031,"feed_emoji":"⚙️","tokens_out":4994,"duration_ms":45364,"temperature":0.7,"pith_summary":"This paper proposes DQA, a method for quantizing DNN activations to fewer than 6 bits while keeping accuracy high enough for real use. Instead of expensive per-image searches or noise injection, DQA ranks activation channels once offline, gives important channels extra bits during quantization, right-shifts them back to the target width, and stores the discarded low bits as Huffman-coded side information that is added back at de-quantization. On ResNet-32, MobileNetV2, and U-Net, DQA reports up to 29.28% higher accuracy than direct uniform symmetric quantization, and generally higher accuracy than NoisyQuant at 3, 4, and 5 bits. The online computation is reduced to shifts and table lookups, making deep activation quantization more plausible on resource-constrained devices.","feed_headline":"DQA keeps 3-bit activations accurate with shifts and Huffman coding","feed_subtitle":"By re-adding right-shifted bits as Huffman codes, DQA beats direct quantization and NoisyQuant below 6 bits.","key_machinery":"The mechanism carrying the argument is a right-shift compensation loop. An important channel is first quantized with $m$ extra bits; a right shift by $m$ bits drops those extra bits, and the dropped bits (the shifting error) are saved as a Huffman code. Because the quantization step size for the $n+m$-bit representation is $2^m$ times finer, re-adding the shifting error yields a residual error $2^m$ times smaller than direct quantization of the same channel. An offline greedy channel ranking decides which channels receive this treatment, and that combination of rank, shift, and Huffman side channel produces the accuracy gains without online search or heavy mathematical operations.","core_discovery":"DQA's central claim is that sub-6-bit activation quantization does not have to sacrifice accuracy when important channels are quantized with extra bits and the discarded shifting errors are encoded with Huffman coding and restored during de-quantization. DQA selects important channels by a greedy offline search over calibration data, then at inference quantizes those channels with $n+m$ bits, right-shifts by $m$ bits so every stored value uses the same $n$ bits, and saves the shifted-off bits as Huffman codes. De-quantization decodes the errors and adds them back, reducing the quantization error by a factor of $2^m$. The paper reports that this beats direct quantization and NoisyQuant on the tested models, with the largest gains at 3 bits and for image-classification CNNs.","pith_inferences":["The paper fixes $m=3$ for all target bit widths; a natural extension would be to search $m$ per layer or per channel, trading side-information size against accuracy, which the paper does not explore.","The offline rank is computed on a subset of the training set, so the first thing likely to break is out-of-distribution input; one could stress-test DQA by computing ranks on one dataset and evaluating on a shifted version.","DQA's measured Huffman compression ratio is modest (up to 1.12x on CIFAR-10), suggesting the real benefit may be accuracy rather than memory; a hardware implementation would need to count the cost of Huffman tables and per-channel rank storage.","The shift-error argument is not architecture-specific, so the method could plausibly extend to transformers or large language models, where NoisyQuant originally operated, though the paper does not test that regime."],"forward_implications":["If DQA's results hold, activation quantization at 3 to 5 bits can be accurate without per-input optimization, so memory-bound inference on small devices could use deeper quantization than before.","The reported accuracy improvements are largest at 3 bits, the regime where direct quantization collapses and NoisyQuant's noise search only partially recovers accuracy.","Because all quantized values keep the same bit width, DQA avoids the storage waste of channel-wise mixed precision while still giving important channels effectively more precision.","The method moves the accuracy-motivated work offline into channel ranking, leaving only shifts, lookups, and Huffman coding for online inference, which is a compute profile friendly to simple hardware.","On U-Net at 3 bits the paper finds NoisyQuant still wins, so DQA's advantage is so far demonstrated mainly for image-classification CNNs rather than all architectures."],"supporting_citations":[{"why":"Defines uniform symmetric quantization and de-quantization, which DQA uses as the Direct baseline and as the treatment of unimportant channels.","marker":"[5]"},{"why":"Introduces channel-importance scaling for weights; DQA adapts this idea to activation channels and borrows the rounding-error analysis.","marker":"[11]"},{"why":"NoisyQuant is the state-of-the-art activation-quantization baseline that DQA compares against and aims to beat at sub-6-bit widths.","marker":"[13]"},{"why":"Huffman coding is the compression scheme DQA applies to shifting errors, justified by the non-uniform error distributions the paper measures.","marker":"[17]"},{"why":"Motivates keeping a uniform bit width after right-shifting, since mixed-precision storage can waste space when values have different bit lengths.","marker":"[12]"}],"fun_headline_variants":["DQA: Huffman-coded shifts for accurate sub-6-bit activations","Shift and Huffman: DQA lifts 3-bit activation accuracy","Sub-6-bit quantization: DQA beats direct and NoisyQuant","DQA adds extra bits to key channels, Huffman for the rest"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Offline channel rankings computed from calibration data remain valid for the data seen during inference, because the paper assumes calibration and inference inputs come from sufficiently similar distributions.","fun_headline_variants_meta":{"raw":{"variants":["DQA: Huffman-coded shifts for accurate sub-6-bit activations","Shift and Huffman: DQA lifts 3-bit activation accuracy","Sub-6-bit quantization: DQA beats direct and NoisyQuant","DQA adds extra bits to key channels, Huffman for the rest"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000263,"raw_usage":{"total_tokens":1588,"prompt_tokens":924,"completion_tokens":664,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":540,"completion_tokens_details":{"reasoning_tokens":584}},"tokens_in":540,"tokens_out":664,"duration_ms":6214,"temperature":1.0,"reasoning_tokens":584,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:50:29.394514+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run DQA with ranks computed on clean CIFAR-10 but evaluate on a corrupted or out-of-distribution variant, and compare accuracy against direct quantization; if DQA no longer beats direct quantization once channel importance shifts, the offline-ranking assumption is falsified. Alternatively, implement DQA on a microcontroller and measure end-to-end latency and energy; if Huffman decoding and side-channel overhead outweigh the savings, the resource-constrained motivation collapses.","supporting_citations":[{"cited_title":"In: Low-Power Computer Vision, pp","cited_arxiv_id":null,"evidence_quote":"Defines uniform symmetric quantization and de-quantization, which DQA uses as the Direct baseline and as the treatment of unimportant channels."},{"cited_title":"In: Gibbons, P., Pekhimenko, G., Sa, C.D","cited_arxiv_id":null,"evidence_quote":"Introduces channel-importance scaling for weights; DQA adapts this idea to activation channels and borrows the rounding-error analysis."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"NoisyQuant is the state-of-the-art activation-quantization baseline that DQA compares against and aims to beat at sub-6-bit widths."},{"cited_title":"In: ICALP","cited_arxiv_id":null,"evidence_quote":"Huffman coding is the compression scheme DQA applies to shifting errors, justified by the non-uniform error distributions the paper measures."},{"cited_title":"IEEE Journal of Solid-State Circuits 59(9), 3082–3093 (2024)","cited_arxiv_id":null,"evidence_quote":"Motivates keeping a uniform bit width after right-shifting, since mixed-precision storage can waste space when values have different bit lengths."}],"review_version":1}