{"id":"19b40f98-ba78-41e0-a806-5739cc328c6f","arxiv_id":"2506.11431","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"TruncQuant uses a floor-based quantizer with 2^n scaling instead of rounding with 2^n-1, so truncating high-precision weights via bit-shift exactly matches direct low-precision quantization, recovering accuracy lost by naive truncation.","lead":"This paper introduces TruncQuant, a quantization training scheme that makes DNN weights robust to bit-shifting, so one model can run at several bit precisions by simply discarding least significant bits. For edge deployment this could cut memory and energy costs, but the benefit is currently demonstrated for weights only and degrades on some existing QAT frameworks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"TruncQuant's floor quantizer matches truncation exactly, but Table III shows 1.2–2.6 pt losses at 5–8 bits on EQ-Net/RobustQuant, so framework-independent accuracy parity is unsupported.","rationale":"The core mathematical observation of the paper is sound: for normalized weights in [0,1), floor(2^b W) >> (b-n) = floor(2^n W), so truncation of TruncQuant-trained weights exactly reproduces direct TruncQuant quantization. The main ImageNet and CIFAR-10 results on Any-Precision are strong and the released code is a plus. The load-bearing weakness is not the identity but the step from identity to 'accuracy on par with the baseline.' Table III is internal evidence that the floor quantizer can degrade accuracy at high precision in some frameworks. This is not a consensus dispute; it is an unexplained internal inconsistency. A controlled 8-bit ablation would determine whether the drop stems from the quantizer or from the EQ-Net/RobustQuant integration procedure. Until that is resolved, the paper's more general claim about being easily implementable in existing QAT frameworks is not established, but the primary TruncQuant result on Any-Precision remains plausible. I also considered the W=1.0 overflow in Eq. (9) as a candidate concern, but it is secondary because it is readily fixed by clipping and does not explain the Table III degradation pattern. Hence the reader's CONDITIONAL verdict stands.","tokens_in":12181,"tokens_out":14975,"duration_ms":147193,"concrete_test":"Re-run the EQ-Net CIFAR-10 experiment at 8-bit with the same training schedule and data, comparing three quantizers in the forward pass: (a) EQ-Net's original LSQ round quantizer, (b) TruncQuant's floor((M_8+1)W), and (c) floor((M_8+1)W) with the same learned scale as (a). If (b) or (c) is more than 1 point below (a), the Table III degradation is caused by the floor quantizer's training dynamics, not by truncation mismatch; the claim of framework-independent parity would then need to be retracted or qualified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section V-C reports TruncQuant integrated with RobustQuant and EQ-Net (Table III). At precisions where truncation is trivial (5–8 bits), TruncQuant is worse than simply truncating the baseline QAT weights: RobustQuant 8-bit 88.5% vs 90.7%; EQ-Net 8-bit 86.3% vs 88.4%; EQ-Net 4-bit 86.5% vs 88.9%. Since at 8 bits truncation is identity, this gap cannot be attributed to quantization-truncation misalignment; it shows the floor((M_n+1)W) quantizer itself changes training dynamics and can reduce accuracy relative to the round-based baseline. The paper's Contribution (3) claims 'accuracy on par with the baseline' and the abstract promises easy integration into existing QAT frameworks, but Table III contradicts this for two of the three frameworks tested, and no analysis of the degradation is given. The central identity in Eq. (9) only proves bin alignment; it does not prove that replacing the STE-modified round quantizer with a floor quantizer preserves optimization quality. This is the load-bearing gap between the mathematical observation and the empirical claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TruncQuant, a truncation-ready weight quantization scheme for quantization-aware training. The key idea is to replace the usual round-based quantizer with qt(W; n) = floor((M_n + 1)W), where M_n = 2^n - 1. Because M_n + 1 is a power of two, quantizing to a high bit-width and then truncating by bit-shifting is exactly equivalent to directly applying the same floor quantizer at the lower bit-width, so the quantization-truncation gap is eliminated by construction. The authors validate the method on CIFAR-10, SVHN, and ImageNet within an Any-Precision-style QAT framework, report integrations with RobustQuant and EQ-Net, and provide Timeloop-based energy estimates showing storage and energy benefits of bit-shifting from a single stored integer model.","tokens_in":12376,"tokens_out":15159,"duration_ms":158722,"significance":"The central mathematical observation in Eqs. (6)-(10) is clean, self-contained, and appears machine-checkable: replacing round-based quantization with floor((M_n+1)W) makes quantized-then-truncated weights exactly equal to directly quantized weights at the lower precision. The low-bit empirical results are also strong, especially the ImageNet ResNet-50 2-bit recovery from 3.9% to 71.4%. The release of code is a practical strength, and the CIFAR-10/SVHN tables include error bars. However, the significance is tempered by Table III, where the same floor quantizer degrades accuracy at 5-8 bits in RobustQuant and EQ-Net relative to simply truncating the baseline QAT weights. Since truncation is the identity at 8 bits, this degradation is not attributable to quantization-truncation misalignment and indicates that the benefit is framework-dependent rather than universal.","major_comments":[{"comment":"The integrated experiments contradict the general claims in Contribution (3) and in Section V-C that TruncQuant achieves accuracy on par with the baseline and integrates well with existing QAT frameworks. At precisions where truncation is identity or nearly harmless, TruncQuant is consistently below the baseline QAT result: RobustQuant at 8 bits gives 88.5% versus 90.7% for the Quant row, and EQ-Net at 8 bits gives 86.3% versus 88.4%. At 4 bits, the EQ-Net TruncQuant result (86.5%) is also below the Trunc baseline (88.9%). Because no truncation occurs at 8 bits, these gaps cannot be caused by quantization-truncation misalignment; they show that replacing the round-based quantizer with floor((M_n+1)W) changes training dynamics and can degrade accuracy in these frameworks. The paper gives no analysis of this degradation. Please either provide such an analysis and a remedy, or explicitly scope the claims to the Any-Precision setting and to low bit-widths.","section":"Section V-C, Table III"},{"comment":"The proposed STE correction is not derived from the forward mapping. For qt(W; n) = floor((M_n+1)W), if the integer output is dequantized by 1/M_n as in the DoReFa-style dequantization used elsewhere, the effective slope of the forward map under the STE is (M_n+1)/M_n, not M_n/(M_n+1); if the output is dequantized by 1/(M_n+1), the slope is 1 and no scaling is needed. The factor M_n/(M_n+1) is the inverse of the dequantized slope. The manuscript also states that the original STE performs well, leaving ambiguous which variant produced the reported results. Please correct or remove Eq. (11), and state explicitly which STE variant was used for Tables I-III.","section":"Section IV, Eq. (11)"}],"minor_comments":[{"comment":"The row labels in Table I and the surrounding text are corrupted by unicode replacement tokens, so it is not possible to tell which block corresponds to which architecture and dataset (e.g., AlexNet, SVHN, ResNet-8, CNN-8, ResNet-20, MobileNetV2). Please fix the typesetting.","section":"Table I"},{"comment":"The ImageNet results in Table II have no error bars or number of runs; given that the 1-bit entries differ by about one point, these single-run results should be labeled as preliminary or supplemented with multiple runs.","section":"Table II"},{"comment":"The binwidth formulas write the first bin as starting at -0.5/M_n and the last bin as ending at M_n + 0.5 over M_n; the edge bins should be clipped to the normalized range [0,1].","section":"Eqs. (6)-(7)"},{"comment":"Equation (5) is stated as an equality, but it is only exact under the assumption that every weight in the QT Gap is assigned to an adjacent bin and therefore contributes exactly one level size; this assumption should be stated explicitly.","section":"Eq. (5)"},{"comment":"The claim that QT Error is inversely proportional to the accuracy drop is supported by only two model curves; please provide a quantitative correlation or additional data points to substantiate this trend.","section":"Fig. 6"}],"recommendation":"major_revision","confidential_remarks":"The core identity in Eq. (9) is correct and the low-bit recovery results are genuinely useful. The main issue is scope: the paper's broad claims of 'on par with the baseline' and easy integration are contradicted by the high-bit degradation in Table III for two of the three frameworks tested. This is fixable within the manuscript's scope by either improving the training recipe or narrowing the claims, so I recommend major revision rather than rejection. The corruption in Table I should also be corrected before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"TruncQuant is a small, honest paper with one clean idea: if you train with floor((Mn+1)W) instead of round(Mn W), your quantized bins line up exactly with truncation bins, so bit-shifting at runtime produces the same integers as direct quantization. The derivation of the QT Gap is straightforward and correct, and the empirical payoff on their own framework is real—ResNet-50 on ImageNet goes from 3.9% to 71.4% at 2-bit truncation, and the CIFAR/SVHN tables show truncation accuracy matching the QAT baseline at all precisions. Code is released, which makes the work reproducible.\n\nWhere the paper oversells is the integration claim. Table III shows that on RobustQuant and EQ-Net at 5-8 bits, TruncQuant is 2-4 points worse than simply truncating the baseline QAT weights. At 8 bits truncation is the identity, so no quantization-truncation misalignment can be blamed; the floor-based quantizer itself changes training dynamics, and in those frameworks it hurts. The abstract and Contribution (3) promise easy integration and accuracy on par with baseline; two of the three frameworks tested contradict that, and the authors do not discuss the degradation. This is the gap between the mathematical identity in Eq. (9) and the general claim: alignment is necessary, but it does not guarantee that the STE plus floor preserves optimization quality.\n\nTwo smaller technical points. The quantizer maps W=1 to 2^n, which overflows the n-bit range [0, 2^n-1]; the paper normalizes weights to [0,1] but never states how the maximum is handled. Also, there is no direct comparison with Any-Precision's published multi-precision accuracies; Table II compares only against the naive truncation baseline. Both are easy to fix.\n\nBottom line: the core observation is correct, the low-bit truncation recovery on their own framework is demonstrated, and the limitations are specific and addressable. It is an incremental but useful contribution to flexible-precision inference. A serious referee should see it; the Table III issue and the boundary case should be raised in revision.","headline":"A simple, correct quantizer design that rescues truncation accuracy on its own framework, but the integration results show the benefit is framework-dependent and the paper overclaims.","tokens_in":12979,"tokens_out":1855,"would_cite":true,"duration_ms":19347,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A floor-based training quantizer, $q_t(W;n)=\\lfloor (M_n+1)W\\rfloor$, makes quantized-then-truncated DNN weights equal to directly quantized weights, so one model can serve many bit precisions by bit-shifting.","keywords":["quantization-aware training","weight truncation","bit-shifting inference","QT gap","flexible bit precision","low-bit DNN inference","once-for-all networks","straight-through estimator"],"falsifier":"Train a model with TruncQuant on a quantization-aware framework whose step sizes are learned, then truncate it to every bit width and compare against directly training at each width. If any width shows an accuracy gap even though the quantized-then-truncated integer values are identical to the directly quantized ones, the claim that bin alignment is sufficient for the recovery is falsified. The paper's own integration results at 5-8 bits already show small such gaps, so the decisive version of this test would use a framework where the QT Error is exactly zero and measure whether accuracy still drifts.","tokens_in":11946,"feed_emoji":"⚙️","tokens_out":8766,"duration_ms":93845,"temperature":0.7,"pith_summary":"The paper claims that the accuracy collapse seen when quantization-aware-trained DNN weights are truncated to lower bit widths comes from a specific, removable mismatch: uniform rounding and bit-shifting divide the weight range into bins that do not line up, so some values map to different integers under the two operations. TruncQuant removes this mismatch by training with the quantizer $q_t(W;n)=\\lfloor (M_n+1)W\\rfloor$, whose bin boundaries coincide with truncation boundaries for any starting and target precision. With this change, truncating a trained model to a lower bit precision at runtime gives the same weight values as training at that precision directly, and the reported accuracy drop disappears (for example, ImageNet ResNet-50 at 2 bits goes from 3.9% when naively truncating a standard QAT model to 71.4% with TruncQuant). If the claim holds, a single stored 8-bit model can serve any lower bit precision by a simple bit-shift, avoiding the extra floating-point model and memory traffic that flexible-precision deployment usually requires.","feed_headline":"Truncated 2-bit ResNet-50 jumps from 3.9% to 71.4% accuracy","feed_subtitle":"One floor-based change to the training quantizer lets a single DNN switch bit precision by bit-shifting at runtime","key_machinery":"The central mechanism is the truncation-ready binwidth: intervals $[k/(M_n+1), (k+1)/(M_n+1))$ for $k=0,\\dots,M_n$, produced by the floor quantizer $q_t(W;n)=\\lfloor (M_n+1)W\\rfloor$. Because truncation from a higher bit precision can be written as dividing the high-precision integer by a power of two and flooring, these evenly spaced bins are invariant under bit-shifting: the most significant bits of a low-precision weight agree with those of its higher-precision parent regardless of the starting precision. The paper also modifies the straight-through estimator to scale gradients by $M_n/(M_n+1)$, matching the slightly narrower binwidth.","core_discovery":"TruncQuant is built on the observation that uniform quantization and truncation are not the same map even when both start from the same floating-point weight. For weights normalized to [0,1], the n-bit rounding quantizer uses bins of width $1/(2^n-1)$ centered at integer multiples, while truncation from b bits uses bins of width $2^{b-n}/(2^b-1)$; the two sets of boundaries differ, producing the QT Gap, the set of values that land in different bins under the two operations. The paper defines the QT Error as the distance between truncated and directly quantized weights and shows it is the product of the number of weights in the gap and the level size. TruncQuant's quantizer, $q_t(W;n)=\\lfloor (M_n+1)W\\rfloor$, makes both quantized and truncated binwidths equal to $1/(M_n+1)$, so the QT Gap vanishes and quantize-then-truncate equals direct quantization. The accompanying straight-through-estimator scaling by $M_n/(M_n+1)$ keeps the backward pass consistent with the new binwidth.","pith_inferences":["Inference: the same bin-alignment principle should apply to truncating activations or to any power-of-two-aligned non-uniform quantizer, but the paper only evaluates weights, so an activation-truncation test would be a natural extension.","Inference: the residual 5-8 bit accuracy gaps reported when TruncQuant is embedded in two other training frameworks suggest that bin alignment is necessary but not always sufficient; the remaining differences likely live in how each framework's step sizes and gradient estimates interact with the floor function.","Inference: the storage argument implies only the highest-precision tensor needs to persist, which opens the door to combining TruncQuant with lossless compression of the MSBs or with error-correction codes for the discarded LSBs; neither is explored in the paper."],"forward_implications":["One model trained at 8 bits can be switched to 1-7 bits at runtime by bit-shifting, with per-precision accuracy close to direct quantization at that precision.","Precision switching requires no floating-point parent model and no re-quantization pass, so runtime energy and DRAM traffic for changing precision drop.","The fix is a one-line change to an existing quantization-aware training loop: replace round-to-nearest with $\\lfloor (M_n+1)W\\rfloor$ for weights; activations and the loss are untouched.","The quantization-truncation error analysis gives a measurable quantity (QT Error) that tracks the accuracy drop, so it can be used to audit other flexible-precision training schemes."],"supporting_citations":[{"why":"Supplies the once-for-all flexible-precision training recipe, the bit-shifting inference setup, and the pretrained model used for ImageNet fine-tuning; it is the main baseline whose truncation failure motivates TruncQuant.","marker":"[22]"},{"why":"Defines the uniform quantization function and the standard quantization-aware training forward/backward loop that TruncQuant plugs into.","marker":"[25]"},{"why":"Introduces the straight-through estimator that lets gradients flow through the nondifferentiable floor quantizer; TruncQuant rescales it.","marker":"[28]"},{"why":"Provides the weight normalization and per-layer scaling factor used in the experiments, and the convention of keeping first and last layers at higher precision.","marker":"[29]"},{"why":"Serves as one alternative flexible-precision framework in which the authors replace the rounding quantizer with TruncQuant, showing the method transfers.","marker":"[17]"},{"why":"Serves as the second alternative framework for integration tests, and its results reveal where bin alignment alone does not fully preserve accuracy.","marker":"[18]"},{"why":"Supplies the learned-step-size quantizer that is modified in the integration experiments to derive lower-precision scales by dividing by powers of two.","marker":"[26]"},{"why":"Provides the simulator used to estimate runtime energy for initial model loading and bit-precision switching.","marker":"[38]"},{"why":"Defines the row-stationary accelerator architecture inside the simulator, the basis for the energy breakdown.","marker":"[39]"}],"fun_headline_variants":["TruncQuant matches truncation and quantization bins, enabling flexible bit precision","Truncation-ready training: one DNN, many precisions via runtime bit-shifting","QT Gap erased: TruncQuant's quantizer aligns bin edges for flexible precision","From 3.9% to 71.4%: TruncQuant powers 2-bit ResNet-50 with flexible precision","TruncQuant: truncation-aware quantization for agile weight bit precision"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that swapping the rounding-based quantizer for the floor-based one, and scaling the straight-through estimator accordingly, does not change how well the network learns; if that premise fails for some training framework, eliminating the bin mismatch would not by itself recover the lost accuracy.","fun_headline_variants_meta":{"raw":{"variants":["TruncQuant matches truncation and quantization bins, enabling flexible bit precision","Truncation-ready training: one DNN, many precisions via runtime bit-shifting","QT Gap erased: TruncQuant's quantizer aligns bin edges for flexible precision","From 3.9% to 71.4%: TruncQuant powers 2-bit ResNet-50 with flexible precision","TruncQuant: truncation-aware quantization for agile weight bit precision"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000404,"raw_usage":{"total_tokens":2118,"prompt_tokens":976,"completion_tokens":1142,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":592,"completion_tokens_details":{"reasoning_tokens":1028}},"tokens_in":592,"tokens_out":1142,"duration_ms":11345,"temperature":1.0,"reasoning_tokens":1028,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:09:45.838137+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a model with TruncQuant on a quantization-aware framework whose step sizes are learned, then truncate it to every bit width and compare against directly training at each width. If any width shows an accuracy gap even though the quantized-then-truncated integer values are identical to the directly quantized ones, the claim that bin alignment is sufficient for the recovery is falsified. The paper's own integration results at 5-8 bits already show small such gaps, so the decisive version of this test would use a framework where the QT Error is exactly zero and measure whether accuracy still drifts.","supporting_citations":[{"cited_title":"Any- precision deep neural networks,","cited_arxiv_id":null,"evidence_quote":"Supplies the once-for-all flexible-precision training recipe, the bit-shifting inference setup, and the pretrained model used for ImageNet fine-tuning; it is the main baseline whose truncation failure motivates TruncQuant."},{"cited_title":"Eq-net: Elastic quanti- zation neural networks,","cited_arxiv_id":null,"evidence_quote":"Serves as one alternative flexible-precision framework in which the authors replace the rounding quantizer with TruncQuant, showing the method transfers."},{"cited_title":"Robust quantization: One model to rule them all,","cited_arxiv_id":null,"evidence_quote":"Serves as the second alternative framework for integration tests, and its results reveal where bin alignment alone does not fully preserve accuracy."},{"cited_title":"Eyeriss: An energy- efficient reconfigurable accelerator for deep convolutional neural net- works,","cited_arxiv_id":null,"evidence_quote":"Defines the row-stationary accelerator architecture inside the simulator, the basis for the energy breakdown."}],"review_version":1}