{"id":"469392ff-a7a8-46da-b0df-3de944f04124","arxiv_id":"2504.17263","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"ASQ uses an input-adaptive step size for activation quantization and POST uses square-root-of-two levels for weights, reporting 4-bit accuracy near or above full precision on ImageNet.","lead":"This paper presents ASQ, a quantization-aware training method that adjusts the activation step size on the fly with a small neural module, plus POST, a square-root-of-two weight quantizer. The authors report 4-bit ImageNet accuracy that matches or slightly beats full-precision baselines, but the headline number in the abstract does not match the experiments.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Dequantization scale mismatch in Eq. (6) makes the ASQ forward pass ill-defined, so the reported accuracy gains cannot be reproduced from the equations as written.","rationale":"The reader's conditional verdict already flags the Eq. (6) mismatch, though the named weakest assumption is the underspecified adapter architecture. My read makes the dequantization mismatch the primary load-bearing issue: it is more fundamental than the adapter ambiguity because it determines whether the training objective and forward pass are well-defined at all. The missing code amplifies the problem, since no implementation can be inspected. I agree with the reader that the issues are addressable in revision, so the appropriate outcome remains CONDITIONAL rather than a hard reject. I do not see a reason to move the verdict; the concern reinforces the reader's rationale. A concrete reimplementation of Eq. (6) versus the corrected dequantization would settle whether this is a typo or a substantive design flaw, and would also provide the reproducibility evidence the paper currently lacks.","tokens_in":14098,"tokens_out":5129,"duration_ms":49690,"concrete_test":"Implement Eqs. (4)-(6) literally in PyTorch for 3-bit ResNet20 on CIFAR-10, and also implement the corrected variant xhat = xint*sa; train both with identical seeds and compare the Table 4 rows. If the literal version does not reproduce the reported 91.7 (or diverges), while the corrected version does, then the published results were produced by a different dequantization rule than the paper states.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Equations (4)-(6) define an internally inconsistent quantizer. Quantization and clipping use the adaptive step sa = s*beta (Eq. 4 and Eq. 5), while dequantization multiplies the integer code by the non-adaptive s (Eq. 6). Unless beta = 1, xhat is not a reconstruction on the sa grid, and the operator is not a standard fake-quantization layer. The STE gradient in Eq. (7) and the chain rules in Eqs. (8)-(9) do not correspond to this forward map: if xhat = xint*s, the derivative of xhat with respect to sa is -s*x/sa^2 (with STE), not the expression written; if xhat = xint*sa was intended, Eq. (6) is wrong. Algorithm 1 step 4 calls Quantize(A, sa, ba) and implicitly relies on Eq. (6) for dequantization, but the paper never states which scale is actually used in training and inference. Because beta is input-dependent and initialized near 1, the inconsistency can be masked in aggregate results yet still change training dynamics and the scale of every activation. The central claim that ASQ matches or beats full precision therefore rests on an ambiguous or erroneous operator, and the text 'complete code will be released after acceptance' provides no way to disambiguate.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a quantization-aware training (QAT) method, Adaptive Step Size Quantization (ASQ), which uses a small learned adapter module to produce an input-dependent multiplicative factor for the activation quantization step size, and a non-uniform weight quantizer, POST, based on powers of the square root of two. The authors report experiments on ImageNet and CIFAR-10 with ResNet and MobileNet-V2, claiming state-of-the-art accuracy at low bit widths, including better-than-full-precision results at 4-bit and 8-bit on ResNet models, as well as negligible computational overhead for the adapter.","tokens_in":14247,"tokens_out":5853,"duration_ms":49915,"significance":"If the method is sound and reproducible, the core idea is interesting: input-adaptive activation step sizes are a plausible way to handle the variability of activation distributions across inputs, and the POST quantizer with a LUT-based implementation could offer a useful efficiency-accuracy trade-off. The empirical results, assuming they hold, would be competitive with the QAT literature. However, the manuscript as written does not specify the adapter architecture or input reduction, contains an internally inconsistent forward/backward definition, and reports a headline number that contradicts the table in the paper. These issues currently prevent verification of the central empirical claims, so the significance is conditional on a careful revision that resolves them.","major_comments":[{"comment":"The forward fake-quantization operator is internally inconsistent. Quantization and clipping in Eq. (5) use the adaptive step size sa = s·β, while dequantization in Eq. (6) multiplies the integer code by the non-adaptive step size s. Unless β = 1, the reconstructed value x̂ is not on the sa grid used to produce xint, so the operator does not implement a standard quantization/dequantization round trip. The gradient in Eq. (7) is also not a correct STE derivative under either interpretation: if x̂ = xint·s, the in-range derivative of x̂ with respect to sa is −s·x/sa², not the expression given; if x̂ = xint·sa was intended, Eq. (6) is wrong and the in-range derivative should be −x/sa + round(x/sa). Additionally, the symbol v in Eq. (7) is undefined. Since Algorithm 1 step 4 calls Quantize(A, sa, ba) and implicitly relies on Eq. (6), the training and inference forward/backward passes are ambiguous. The authors must state explicitly whether dequantization uses sa or s, in both training and inference, and correct the gradient formulas and the undefined variable accordingly.","section":"Section 3.2, Eqs. (4)-(6) and Algorithm 1"},{"comment":"The adapter is underspecified to the point of non-reproducibility. The call Adapter(A, W_adapter, B_adapter) takes the activation tensor A, but the paper never states how A is reduced to the scalar (or vector) β: global average pooling, flattening, channel-wise aggregation, or per-tensor versus per-channel prediction are all plausible but mutually different designs. The architecture is also unspecified (one or two linear layers, hidden size, nonlinearity). These choices directly determine the accuracy and the computational overhead: if the adapter ingests raw feature maps, the flattening and matrix multiplication cost can be substantial for large activations, contradicting the 'negligible overhead' claim in Section 3.2.2 and Table 1; if it pools first, the pooling choice is an untested design decision on which the reported accuracies likely depend. The authors should specify the exact adapter architecture and input reduction, and verify the overhead calculation against that concrete design.","section":"Section 3.2, Algorithm 1 step 2"},{"comment":"The abstract's headline claim is contradicted by the paper's own results. The abstract states that 'its 4-bit quantized ResNet34 model improving accuracy by 1.2% on ImageNet.' Table 2 reports ResNet34 ASQ+POST at 4-bit as +0.8% (74.1 vs. 73.3) and ASQ at 4-bit as +0.6% (73.9 vs. 73.3). The +1.2% figure corresponds to ResNet18 ASQ+POST at 4-bit (71.0 vs. 69.8) or ResNet18 ASQ at 8-bit (71.0 vs. 69.8). The abstract should be corrected to match the table, or the discrepancy should be explained.","section":"Abstract vs. Section 4.2, Table 2"},{"comment":"There is a bit-width inconsistency in the ablation study. The text states 'we conducted a thorough ablation study on the 4-bit quantized ResNet20 network,' but Table 5's caption and header say 'the final performance of a 3-bit quantized ResNet-20 network.' The reported accuracy values (Baseline+POST+ASQ 93.0 vs. real-valued 92.6) are consistent with the text's +0.4% claim, but the bit-width must be clarified. This matters because the ablation is used to attribute gains to ASQ and POST, and the reader cannot tell which quantization setting produced those numbers.","section":"Section 4.4 and Table 5"}],"minor_comments":[{"comment":"The heading 'Implimention Details' contains a typo; it should read 'Implementation Details.'","section":"Section 4.1 heading"},{"comment":"The conclusion contains the typo 'poposed' and the introduction contains the grammatical error 'The researches on QAT'; these should be corrected.","section":"Section 5"},{"comment":"The heading appears as 'Evaluation on CIF AR-10' with an unwanted space; it should be 'CIFAR-10.'","section":"Section 4.3 heading"},{"comment":"The exponent notation in the quantization level sets is ambiguous in the rendering, e.g., '±2−2b−1+1' is hard to parse as a superscript. Please use clear superscript formatting (e.g., ±2^{-2^{b-1}+1}) and verify the exponents, especially in Eq. (17) where the exponent appears to simplify to -2b.","section":"Eqs. (3) and (17)"},{"comment":"The variable v in Eq. (7) is undefined; if it is intended to be x/s or x/s_a, this should be stated explicitly.","section":"Section 3.2.1, Eq. (7)"},{"comment":"The overhead analysis in Table 1 only covers the ASQ adapter; the POST quantizer's LUT storage cost is only described qualitatively ('a slight increase in space utilization'). Please quantify the LUT size and memory overhead for the bit widths used in the experiments.","section":"Table 1 and Section 3.3.1"},{"comment":"Several baseline numbers are marked as 'our re-implementation,' but the paper does not describe how these re-implementations were configured or verified against the original papers. For a fair comparison, please provide details on the re-implementation protocol and, where available, cite the original reported numbers.","section":"Table 2 and Section 4.2"},{"comment":"Algorithm 1 lists Wadapter and Badapter as trainable parameters and updates them, but the backpropagation path through β to these adapter parameters is not shown. Eqs. (8)-(9) only give ∂x̂/∂s and ∂x̂/∂β; the update of Wadapter requires ∂β/∂W_adapter, which is standard but should be stated for completeness.","section":"Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is submitted to Applied Soft Computing; the topic is within scope. The central concern is that the method's forward/backward definition is ambiguous and the experiments cannot be reproduced from the text, which is a blocking issue that requires more than a superficial revision. The abstract/table mismatch and the unspecified adapter architecture are also serious. The paper may become acceptable after a thorough revision that resolves these points and releases code, but as it stands I cannot recommend acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. The core novelty is real: an input-adaptive activation step size produced by a tiny trained adapter is not in the cited QAT literature, and POST is a clean small tweak to POT. But the paper as written has a load-bearing inconsistency in the quantizer equations and a headline/table mismatch. I would not accept it now; I would send it back for a careful revision.\n\nWhat is new and good: The adapter is a genuine extension of LSQ/LSQ+, and the overhead table suggests it is cheap. Initializing the adapter near 1 is sensible. POST replaces base 2 with sqrt(2), giving coarser resolution near zero than POT but cheaper than APOT, with a reasonable LUT trick. The empirical tables show consistent improvements over baselines on ResNet and MobileNet at 4-bit, and the ablation in Table 5 is a clean decomposition. If the numbers hold, this is a modest practical gain.\n\nThe big problem is Eq. (6). You define sa = s*beta, quantize with sa, but dequantize with s. So xhat is not a reconstruction on the sa grid whenever beta != 1. The STE gradient in Eq. (7) does not correspond to either interpretation: with xhat=xint*s the derivative is -s*x/sa^2, not the expression given; with xhat=xint*sa it should be -x/sa + floor(x/sa). This makes the forward pass and training dynamics ill-defined. It may be a typo, but as written the method cannot be reproduced. The adapter itself is underspecified: Adapter(A, ...) never says how A is reduced to a scalar, what architecture is used, or what the hidden size is; the overhead claim depends on that. There are smaller but real inconsistencies: the abstract's 'ResNet34 +1.2%' does not appear in Table 2 (the +1.2 belongs to ResNet18), and the CIFAR text says ASQ outperforms across 2-4 bits while Table 4 shows ASQ 3-bit at 91.7 vs APOT 92.2. No code or error bars are provided, so the central empirical claim is not independently checkable. The citation pattern is fine: relevant prior work is cited, with a few self-cites but nothing egregious.\n\nThis is for QAT researchers who care about step-size learning and low-bit deployment. The idea deserves a serious look and a revise-and-resubmit. Fix the dequantization equation, specify the adapter, correct the abstract and text, release code and seeds. As it stands, I would not cite it in my own work, but I would read a revised version.","headline":"A genuinely novel per-input activation step-size idea, undermined by an ill-defined quantizer equation and inconsistent headline numbers; worth a careful revision.","tokens_in":14971,"tokens_out":5328,"would_cite":false,"duration_ms":47821,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A quantization-aware training scheme with adaptive activation step sizes and square-root-of-two weight levels can make 4-bit ImageNet models match or beat full-precision baselines.","keywords":["quantization aware training","learnable step size","adaptive activation quantization","non-uniform weight quantization","power of square root of two","neural network compression","image classification","ImageNet"],"falsifier":"Re-run the 4-bit ResNet34 training recipe described in the paper on ImageNet and check the Top-1 accuracy against the reported 74.1% (with the 73.3% full-precision baseline and the abstract's claimed +1.2% improvement); separately, implement Algorithm 1's Adapter on the full activation tensor and measure its FLOPs to test the claimed 0.07% added compute.","tokens_in":13759,"feed_emoji":"🧮","tokens_out":9105,"duration_ms":80720,"temperature":0.7,"pith_summary":"The paper tackles a specific failure of low-bit quantization: activations differ from input to input, so a fixed learned step size wastes part of the integer range and hurts accuracy. Its answer is a small adapter that reads the activations of a layer and scales the step size accordingly, plus a non-uniform weight quantizer based on powers of the square root of two that avoids the overly fine resolution near zero of power-of-two levels. The authors report that on ImageNet and CIFAR-10 these changes beat existing quantization-aware training methods and let 4-bit ResNet models match or exceed full-precision accuracy.","feed_headline":"4-bit quantized ResNet34 beats full precision via adaptive steps","feed_subtitle":"A learnable adapter rescales activation step sizes per input while sqrt-2 weight levels cut quantization loss.","key_machinery":"The load-bearing pieces are the Adapter and the POST quantizer. The Adapter is a one- or two-layer linear module that takes a layer's activation tensor and emits a scalar $\\beta$, which rescales the trainable step size $s$ via $s_a = s\\cdot\\beta$; the paper does not specify how the tensor is reduced to that scalar. POST is a non-uniform quantizer whose levels are $\\alpha\\times\\{0, \\pm\\sqrt{2}^{\\,-2^{b-1}+1}, \\ldots, \\pm 1\\}$, a power-of-square-root-of-two ladder that keeps only coarse levels near zero at higher bit widths; a look-up table handles the odd-exponent multiplications so inference stays close to power-of-two cost.","core_discovery":"The central claim is that quantization-aware training can be made input-adaptive at almost no inference cost. A trainable adapter produces a scalar $\\beta$ that multiplies the learned step size $s$, so each layer's quantizer adjusts to the activation distribution of the current input; the same straight-through-estimator gradient machinery used by LSQ still applies. For weights, the paper replaces power-of-two levels with power-of-$\\sqrt{2}$ levels, giving coarser resolution near zero and a better fit to the bell-shaped weight distribution, and uses a look-up table for odd exponents to keep multiplications cheap. The headline result is a 4-bit quantized ResNet34 that improves on its full-precision baseline by 1.2% on ImageNet as stated in the abstract; the detailed results table reports a +0.8% gain for that configuration.","pith_inferences":["The per-layer scalar adapter could be generalized to per-channel or per-token scales; the paper only tests the per-layer version, so there is likely room to extend the same idea.","Because the paper finds ASQ can increase per-layer quantization error while improving end-to-end accuracy, layerwise error appears to be the wrong optimization proxy and input-dependent rescaling likely shifts error into directions the task loss tolerates.","The same adapter mechanism could transfer to transformer-like models, where activation distributions vary strongly across tokens; the paper reports experiments only for CNNs."],"forward_implications":["4-bit and 3-bit ImageNet classifiers can be deployed at roughly one-eighth and one-tenth of full-precision memory while staying within a few tenths of a point of full precision, and sometimes above it.","Because the adapter rescales per input, the method can adapt to shifts in activation distribution between training and deployment without retraining the quantizer.","POST's coarse near-zero levels mean its benefits increase with bit width, consistent with the paper's larger gains at 4 and 8 bits.","On MobileNet-V2, ASQ+POST raises 4-bit Top-1 accuracy to 70.1% from LSQ's 69.5%, indicating the approach helps compact architectures too."],"supporting_citations":[{"why":"PACT, a uniform-activation QAT baseline whose ImageNet results ASQ is compared against.","marker":"[5]"},{"why":"LSQ, the learned-step-size QAT method whose gradient formulation and training recipe ASQ extends and whose accuracy it must beat.","marker":"[6]"},{"why":"LQ-Net, a non-uniform quantization baseline used in the ImageNet and CIFAR-10 comparisons.","marker":"[21]"},{"why":"POT, the power-of-two quantizer whose rigid resolution POST is designed to fix.","marker":"[22]"},{"why":"APOT, the additive power-of-two method that also targets POT's rigid resolution and serves as a comparison.","marker":"[23]"},{"why":"Dynamic convolution, the dynamic-weight technique that motivates the adapter producing the adaptive factor beta.","marker":"[24]"},{"why":"ResNet, the backbone architecture used for the ImageNet experiments and the source of the full-precision baselines.","marker":"[28]"},{"why":"ImageNet-ILSVRC2012, the dataset on which the headline 4-bit ResNet results are measured.","marker":"[30]"}],"fun_headline_variants":["4-bit quantized ResNet34 beats full precision via adaptive steps","Adaptive quantizer lifts 4-bit ResNet34 above full precision","Sqrt-2 weight quantization helps 4-bit net beat full precision","Input-adaptive scaling makes 4-bit ResNet34 more accurate"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that a small module can reduce an entire layer's activations to a single rescaling number without an expensive operation, but the paper never specifies how the tensor is reduced or the module's exact architecture.","fun_headline_variants_meta":{"raw":{"variants":["4-bit quantized ResNet34 beats full precision via adaptive steps","Adaptive quantizer lifts 4-bit ResNet34 above full precision","Sqrt-2 weight quantization helps 4-bit net beat full precision","Input-adaptive scaling makes 4-bit ResNet34 more accurate"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000829,"raw_usage":{"total_tokens":3635,"prompt_tokens":972,"completion_tokens":2663,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":588,"completion_tokens_details":{"reasoning_tokens":2586}},"tokens_in":588,"tokens_out":2663,"duration_ms":18771,"temperature":1.0,"reasoning_tokens":2586,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:45:37.417226+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the 4-bit ResNet34 training recipe described in the paper on ImageNet and check the Top-1 accuracy against the reported 74.1% (with the 73.3% full-precision baseline and the abstract's claimed +1.2% improvement); separately, implement Algorithm 1's Adapter on the full activation tensor and measure its FLOPs to test the claimed 0.07% added compute.","supporting_citations":[{"cited_title":"Zhang, J","cited_arxiv_id":null,"evidence_quote":"LQ-Net, a non-uniform quantization baseline used in the ImageNet and CIFAR-10 comparisons."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Dynamic convolution, the dynamic-weight technique that motivates the adapter producing the adaptive factor beta."}],"review_version":1}